When running a query which requires a sort on a BLU-MPP
environment, a memory leak could be hit which could later result
in an out-of-memory condition.
The out-of-memory condition is hit with the CDE-MERGE evaluator
when merging sorted results from different members of the MPP
cluster.
Following entries will be logged into the db2diag.log
2016-12-21-11.44.12.840285+540 I98635417E908 LEVEL:
Severe
PID : 9345 TID : 140724683138816 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
DBNAME
APPHDL : 0-3992 APPID: *N0.db2inst1.000000000000
AUTHID : DB2 HOSTNAME:
hostname
EDUID : 292 EDUNAME: db2agntcol (DBNAME) 0
FUNCTION: DB2 UDB, CDE sort, MergeInfo::throwOutOfMemory,
probe:967
DATA #1 : String, 237 bytes
SORTHEAP needs to be larger.
MergeEvaluator.cpp 401
void ibm_cde::query::MergeEvaluator::processWorkunits()
Required Surplus: 640 bytes
Current surplus: -992 bytes
InUse: 2147484640 bytes
Reserved: 2147483648 bytes
DATA #2 : Function name in Library, 0 bytes
Object not dumped: Address: 0x0000000000000000 Size: 0 Reason:
Address
is NULL
DATA #3 : File name, 13 bytes
MergeInfo.cpp
2016-12-21-11.44.13.485354+540 I98636326E892 LEVEL:
Severe
PID : 9345 TID : 140724683138816 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
DBNAME
APPHDL : 0-3992 APPID: *N0.db2inst1.000000000000
AUTHID : DB2 HOSTNAME:
hostname
EDUID : 292 EDUNAME: db2agntcol (DBNAME) 0
FUNCTION: DB2 UDB, CDE Services,
ibm_cde::services::MemConsumer::throwOutOfHeapMemoryError,
probe:551
DATA #1 : String, 189 bytes
Unable to reserve more heap memory.
Consumer: 7
Memory limit: 2147483648 bytes
Reserved memory: 2147483648 bytes
Memory in use: 2147484640 bytes
Surplus: -992 bytes
DATA #2 : Function name in Library, 0 bytes
Object not dumped: Address: 0x0000000000000000 Size: 0 Reason:
Address
is NULL
DATA #3 : File name, 11 bytes
Subpool.cpp
2016-12-21-11.44.14.272407+540 I98637219E3031 LEVEL:
Error
PID : 9345 TID : 140724683138816 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
DBNAME
APPHDL : 0-3992 APPID: *N0.db2inst1.000000000000
AUTHID : DB2 HOSTNAME:
hostname
EDUID : 292 EDUNAME: db2agntcol (DBNAME) 0
FUNCTION: DB2 UDB, CDE Trace, cdeDiagnosticsFacility, probe:560
DATA #1 : String, 332 bytes
DB2ErrorException: DB2 Error Exception zRC=0x0, SQLCODE=-955,
diagnostics:
SQLS_MEMCONSUMER_CDE_MERGE stats (4K pages)
current state:
shsort:
sortheap: 524288
sheapthres: 5452592
reserved, HWM: 2011523, 2011523
physical usage: 2332816
consumer:
reserved: 524288
logical usage: 524288
Thrown at Subpool.cpp : 560
DATA #2 : Function name in Library, 68 bytes
void
ibm_cde::services::MemConsumer::throwOutOfHeapMemoryError(bool)
DATA #3 : File name, 11 bytes
Subpool.cpp
CALLSTCK: (Static functions may not be resolved correctly, as
they are
resolved to the nearest symbol)
[0] 0x00007FFD1B3D18A9
_ZN7ibm_cde8services6Tracer9traceDataENS0_11ComponentIDENS0_16Co
mmonTrac
eLevelENS1_10TraceEventEPKcmS6_PKvmRKSbIcSt11char_trait + 0x369
[1] 0x00007FFD1B372A32
_ZNK7ibm_cde8services13BaseException14traceExceptionENS0_11Compo
nentIDEP
Kc + 0x322
[2] 0x00007FFD1B39CDD1
_ZN7ibm_cde8services11MemConsumer25throwOutOfHeapMemoryErrorEb +
0x291
[3] 0x00007FFD1DD0510D
_ZN7ibm_cde5query9MergeInfo16throwOutOfMemoryEPKcmS3_l + 0x10D
[4] 0x00007FFD1DD0AB95
_ZN7ibm_cde5query9MergeInfo16reserveToSurplusEPKcmS3_l + 0x225
[5] 0x00007FFD1DD86718
_ZN7ibm_cde5query14MergeEvaluator16processWorkunitsEv + 0x3F8
[6] 0x00007FFD1DD850FA
_ZN7ibm_cde5query14MergeEvaluator26processInputsSynchronouslyEv
+ 0xAA
[7] 0x00007FFD1C36B729
_ZN7ibm_cde5query9Evaluator8evaluateEbbRNS1_21EvaluatorRestartSt
ateEPNS0
_19OptPredicateTrackerE + 0x3A9
[8] 0x00007FFD1C28948C
_ZN7ibm_cde5query17EvaluationRoutine8evaluateEjP15sql_static_dat
a +
0x30C
[9] 0x00007FFD1CCB8B35
_ZN7ibm_cde5query9Scheduler13evaluateChainEPNS0_17EvaluationRout
ineERm +
0x285
[10] 0x00007FFD1CCBC013
_ZN7ibm_cde5query9Scheduler15runWorkerThreadEPvPi + 0x363
[11] 0x00007FFD1CCCA99E
_ZN7ibm_cde5query17cdeEntryPointImplEP8sqeAgentPvS3_ + 0x6E
[12] 0x00007FFD1D8CC02E
_ZN12cdeInterface16startCdeSubagentEP8sqeAgent
+ 0x27E
[13] 0x00007FFD21033FCF _Z14sqlriInvokeCdeP8sqlrr_cb + 0x3F
[14] 0x00007FFD20E96BF8
_Z15sqlriSectInvokeP8sqlrr_cbP12sqlri_opparm +
0x4F8
[15] 0x00007FFD208FC067
_Z21sqlrr_subagent_routerP8sqeAgentP12SQLE_DB2RA_T + 0xD87
[16] 0x00007FFD1E4907DD
/db2/db2inst1/sqllib/lib64/libdb2e.so.1 +
0x532B7DD
[17] 0x00007FFD1E48F0CA _Z21sqleProcessSubRequestP8sqeAgent +
0x3DA
[18] 0x00007FFD1E4C28CB _ZN8sqeAgent6RunEDUEv + 0x5AB
[19] 0x00007FFD21C7F796 _ZN9sqzEDUObj9EDUDriverEv + 0x116
[20] 0x00007FFD202D2688 sqloEDUEntry + 0x578
[21] 0x00007FFD2741BDF5 /lib64/libpthread.so.0 + 0x7DF5
[22] 0x00007FFD17C3C1AD clone + 0x6D |
