HornetQ transparently supports huge queues containing millions of messages while the server is running with limited memory.
In such a situation it's not possible to store all of the queues in memory at any one time, so HornetQ transparently pages messages into and out of memory as they are needed, thus allowing massive queues with a low memory footprint.
HornetQ will start paging messages to disk, when the size of all messages in memory for an address exceeds a configured maximum size.
By default, HornetQ does not page messages - this must be explicitly configured to activate it.
Messages are stored per address on the file system. Each address has an individual folder where messages are stored in multiple files (page files). Each file will contain messages up to a max configured size (page-size-bytes). When reading page-files all messages on the page-file are read, routed and the file is deleted as soon as the messages are recovered.
You can configure the location of the paging folder
Global paging parameters are specified on the main configuration file (hornetq-configuration.xml).
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
...
<paging-directory>/somewhere/paging-directory</paging-directory>
...
Table 24.1. Paging Configuration Parameters
| Property Name | Description | Default |
|---|---|---|
| paging-directory | Where page files are stored. HornetQ will create one folder for each address being paged under this configured location. | data/paging |
As soon as messages delivered to an address exceed the configured size, that address alone goes into page mode.
Paging is done individually per address. If you configure a max-size-bytes for an address, that means each matching address will have a maximum size that you specified. It DOES NOT mean that the total overall size of all matching addresses is limited to max-size-bytes.
Configuration is done at the address settings, done at the main configuration file (hornetq-configuration.xml).
<address-settings>
<address-setting match="jms.someaddress">
<max-size-bytes>104857600</max-size-bytes>
<page-size-bytes>10485760</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
</address-settings>
This is the list of available parameters on the address settings.
Table 24.2. Paging Address Settings
| Property Name | Description | Default |
|---|---|---|
| max-size-bytes | What's the max memory the address could have before entering on page mode. | -1 (disabled) |
| page-size-bytes | The size of each page file used on the paging system | 10MiB (10 * 1024 * 1024 bytes) |
| address-full-policy | This must be set to PAGE for paging to enable. If the value is PAGE then further messages will be paged to disk. If the value is DROP then further messages will be silently dropped. If the value is BLOCK then client message producers will block when they try and send further messages. | PAGE |
Instead of paging messages when the max size is reached, an address can also be configured to just drop messages when the address is full.
To do this just set the address-full-policy to DROP in the address settings
Instead of paging messages when the max size is reached, an address can also be configured to block producers from sending further messages when the address is full, thus preventing the memory being exhausted on the server.
When memory is freed up on the server, producers will automatically unblock and be able to continue sending.
To do this just set the address-full-policy to BLOCK in the address settings
In the default configuration, all addresses are configured to block producers after 10 MiB of data are in the address.
When a message is routed to an address that has multiple queues bound to it, e.g. a JMS subscription, there is only 1 copy of the message in memory. Each queue only deals with a reference to this. Because of this the memory is only freed up once all queues referencing the message have delivered it. This means that if not all queues deliver the message we can end up in a state where messages are not delivered.
For example:
An address has 10 queues
One of the queues does not deliver its messages (maybe because of a slow consumer).
Messages continually arrive at the address and paging is started.
The other 9 queues are empty even though messages have been sent.
In this example we have to wait until the last queue has delivered some of its messages before we depage and the other queues finally receive some more messages.
Please note that message selectors will only operate on messages in memory. If you have a large amount of messages paged to disk and a selector that only matches some of the paged messages, then those messages won't be consumed until the messages in memory have been consumed. HornetQ does not scan through page files on disk to locate matching messages. To do this efficiently would mean implementing and managing indexes amongst other things. Effectively we would be writing a relational database! This is not the primary role of a messaging system. If you find yourself using selectors which only select small subsets of messages in very large queues which are too large to fit in memory at any one time, then you probably want a relational database not a messaging system - you're effectively executing queries over tables.
Please note that message browsers only operate over messages in memory. They do not operate over messages paged to disk. Messages are paged to disk before they are routed to any queues, so when they are paged, they are not in any queues, so will not appear when browsing any queues.
Please note that until messages are acknowledged they are still in memory on the server, so they contribute to the size of messages on a particular address. If messages are paged to disk for an address, and are being consumed, they will be depaged from disk when enough memory has been freed up in that address after messages have been consumed and acknowledged. However if messages are not acknowledged then more messages will not be depaged since there is no free space in memory. In this case message consumption can appear to hang. If not acknowledging explictly messages are acknowledged according to the ack-batch-size setting. Be careful not to set your paging max size to a figure lower than ack-batch-size or your system may appear to hang!
See Section 11.1.34, “Paging” for an example which shows how to use paging with HornetQ.