Boosting Persistent Store Performance
Since: 12.5.0
When should I change the Page Size of the Persistent Store?
You should change the page size of the persistent store if you need more persistent message throughput.
A proper Page Size can increase Producer Performance by 500%
SwiftMQ includes a file-based persistent store managed by the Store Swiftlet (have a look here on how it works). The database is stored in a file called page.db. This file stores messages in pages of 2 KB by default.
A message with a size of 2 KB will occupy 1 page whereas a message of 16 KB will need 8 pages. The larger the message, the more pages it will require and the more disk access takes place. Matching the page size with the size of the messages will therefore have a tremendous effect on the performance, especially if you need to inject many messages fast without delaying producers (producer burst).
Example: Sending and receiving 2 million 16 KB persistent Messages
The following is a result of a small benchmark taken for illustration only. We sent 400’000 persistent messages with 5 senders to 5 queues in parallel (1 sender = 1 queue) and then consume them all (1 consumer = 1 queue) after all messages have been sent, so there are 2 million messages in the store. Each message has a payload of 15’000 bytes. With some message overhead, this results in about 16 KB message size.
With SwiftMQ CE and UR
We used the default configuration with 2 GB heap memory and no disk sync.
While the consumption rate remains the same, the production rate increases by a factor of 5:
With SwiftMQ HA and Replicated File Store
The throughput of SwiftMQ HA vs SwiftMQ CE/UR is lower due to synchronous replication of all transactions to the STANDBY instance.
We used the default configuration with 4 GB heap memory and no disk sync.
The consumption rate is slightly higher while the production rate increases by a factor 3:
Changing the Page Size
Scanning the page.db
The first step is to determine the optimal page size by performing a scan of the page.db. The command is located under Store Swiftlet / Database:
On that page, click on Scan page.db:
The scan will be initiated and then performed during the next checkpoint of the database when all transactions pause. Depending on the size of the page.db, the scan can take a moment. Thereafter you'll find the new recommended page size here:
This attribute is read-only and is exclusively set from the scanner to ensure a proper and optimal size. The size is always a multiple of 2048.
A detailed result of the last scan can be found under Store Swiftlet / Usage / Scan Result:
Here you can see the number of messages for each message size:
You can also schedule a scan as a job via the Scheduler Swiftlet:
A scan does not change anything. It only provides a recommended page size according to the current content of the page.db.
Changing the current Page Size to the recommended Page Size
You need to enable Page Resize on Startup in the Store Swiftlet / Database page first:
Changing the page size is a deep operation comparable to changing the block size of a disk. It can only be performed during the startup of a router. To change the page size, perform a reboot (soft reboot or stop/start) or a failover of a SwiftMQ HA Router.
The page size is now changed during the startup. It can take a while depending on the size of the current page.db. After completion, the new page.db has the new page size and operation continues. The old database is saved under page.db_old.
Keep in mind that you need plenty of disk space to perform this operation. This is checked from the Store Swiftlet before a resize is started.
Compacting page.db
If the page.db contains many free pages, you may want to shrink it via the shrink command or a Shrink job. However, there might be just one higher page occupied. This lies in the nature of the free page management as those pages are randomly freed and immediately occupied by some other queue. So a shrink may actually not provide a smaller size of the page.db as expected.
You can compact the page.db by enabling Perform Compact on Startup on the Store Swiftlet / Database page:
The page.db is compacted during startup with all occupied pages being relocated to the beginning of the file. Compacting will only be performed if the current page size is equal to the recommended page size, otherwise, it will be resized while a resize to a smaller page size also includes compacting it.
When to change the maximum Page Size
If the scan results in more than 32 KB recommended page size, you may consider limiting it because the larger the page size, the more startup time is required.
Converting to a higher page size results in all existing pages having a new size. The number of pages remains the same. A page.db with 2 KB pages and a size of 1 GB total converted to 32 KB pages will result in 32 / 2 * 1 = 16 GB size of the new page.db. However, this can be changed by compacting page.db (see above).
You can set the Maximum Page Size here:
Then run the scan again.
Reverting a Resize
To revert a resized page.db follow these steps:
Set the maximum page size to the former page size.
Run a scan.
Enable to perform a resize on startup.
Reboot the router.
Reset the maximum page size to the previous value.
Backups
If you perform backups of the store, you need to know the page size of the page.db that is stored in the save-set. The page size is appended in the save-set name (in this example 16384 bytes):
To use this backup, you need to manually set page-size-current and page-size-recommended in the routerconfig.xml before you start the router:
<database page-size-current="16384" page-size-recommended="16384" path="../data/store/db" perform-compact-on-startup="true" perform-resize-on-startup="true"/>If you want to use a backup created from an older SwiftMQ release, use 2048 as the page size.