After increasing the configured database memory size, the DB2
memory manager may not always allocate the underlying OS memory
permitted as part of the increase, resulting in intermittent out
of memory errors.
When dynamically increasing the database memory configuration
size, either through an update to the DATABASE_MEMORY
configuration parameter or an underlying area (eg.
SHEAPTHRES_SHR), DB2 does not immediately allocate a
corresponding amount of OS memory. Rather, the memory is
allocated as required and needed for operations (even operations
such as a bufferpool increase will use any free existing
allocated memory before allocating any additional OS memory
needed).
This "deferred allocation" behaviour is intentional. However,
there are some allocation paths, typically for small allocation
requirements, where the DB2 memory manager may only try to reuse
existing allocated memory, ignoring the possibility of
allocating new OS memory areas as permitted by the configuration
increase. This can result in a wide variety of out of memory
messages (SQL0973, SQL0955, etc.).
The out of memory messages typically occur under an increased
fixed database memory limit or under a fixed instance memory
limit, where the automatic database memory size has
unnecessarily increased and exhausted instance memory. The
unnecessary increase is due to allocating new OS memory under a
further database memory increase (rather than allocate new OS
memory under the increase that already occurred).
The easiest case to confirm is where a small allocation request
fails even though there is an abundance of "unreserved" memory.
For example, the db2diag.log entry for sqloMemLogPoolConditions
shows the following failure to allocate 16000 bytes, yet the
shared sort heap has only used 192GB out of 614GB configured,
and there is 109GB of unreserved left in the set. There is no
obvious reason for the failure.
Out of memory failure for Shared Sort Heap (SHEAPTHRES_SHR) on
node 0.
Requested block size : 16000 bytes.
Physical heap size : 192185696256 bytes.
Configured heap size : 614400000000 bytes.
Unreserved memory used by heap : 0 bytes.
Unreserved memory left in set : 109930479616 bytes.
The immediate fix is to recycle the database. During
activation, DB2 preallocates all OS memory based on the database
memory configured size, and the problem paths are avoided. |
