Db2 may enter FODC processing and dump stack like following:
2018-09-06-17.33.40.130732-240 I59970003E2842 LEVEL: Event
PID : 124501 TID : 140126759937792 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB : SAMPLE
APPHDL : 0-170 APPID:
10.10.10.10.42920.180906213433
AUTHID : 9E2581 HOSTNAME: node01
EDUID : 830 EDUNAME: db2agent (SAMPLE) 0
FUNCTION: DB2 UDB, oper system services,
sqloPGRPRegisterOneCrash, probe:2161
MESSAGE : lastCrashCount
DATA #1 : unsigned integer, 8 bytes
0
DATA #2 : String, 10 bytes
inRecovery
DATA #3 : Boolean, 1 bytes
false
CALLSTCK: (Static functions may not be resolved correctly, as
they are resolved to the nearest symbol)
[0] 0x00007F74BDC7F07C sqloPGRPRegisterOneCrash + 0x20C
[1] 0x00007F74BDDD55EF sqloEDUCodeTrapHandler + 0xB2F
[2] 0x00007F74C5891370 /lib64/libpthread.so.0 + 0xF370
[3] 0x00007F7327BE8E02
/home/db2inst1/sqllib/bin/routine/db2dbroutext + 0x34CE02
[4] 0x00007F732791F9F0
_Z14cfgFormatValuePcmP11db2CfgParamPK15sqlf_param_infoP19sqlf_ad
m_param_info + 0x110
[5] 0x00007F732791F2A4
_Z12fetchDataRowcP16cfg_scratch_areaPcS1_S1_S1_S1_S1_ss + 0x364
[6] 0x00007F7327920B74 db_get_cfg + 0x1A4
[7] 0x00007F74BDCC494B sqloInvokeFnArgs + 0x712B
[8] 0x00007F74BEBA51C0
_Z19sqlriInvokerTrustedP10sqlri_ufobP21sqlriRoutineErrorIntfb +
0x720
[9] 0x00007F74BEB993C5 _Z18sqlriInvokeInvokerP10sqlri_ufobb +
0x1BB5
[10] 0x00007F74BEBD6582 _Z11sqlri_tfftcP8sqlrr_cbP9sqlri_tao +
0x2A2
[11] 0x00007F74BE984415 _Z10sqlriFetchP8sqlrr_cbP9sqlri_taol +
0x155
[12] 0x00007F74BEB00AEC _Z7sqlritaP8sqlrr_cb + 0x64C
[13] 0x00007F74BEB663A1 _Z9sqlrihsjnP8sqlrr_cb + 0x6C1
[14] 0x00007F74BEAAE14A _Z14sqlriNljnPipedP8sqlrr_cb + 0x1FA
[15] 0x00007F74BEAAE14A _Z14sqlriNljnPipedP8sqlrr_cb + 0x1FA
[16] 0x00007F74BEA1D44D
_Z15sqlriSectInvokeP8sqlrr_cbP12sqlri_opparm + 0x4BD
[17] 0x00007F74BE414F46
_Z27sqlrr_process_fetch_requestP14db2UCinterface + 0x676
[18] 0x00007F74BE4169F9
_Z10sqlrr_openP14db2UCinterfaceP15db2UCCursorInfo + 0xEB9
[19] 0x00007F74BC595461
_Z16sqljs_ddm_opnqryP14db2UCinterfaceP13sqljDDMObject + 0x431
[20] 0x00007F74BC57F99B
_Z21sqljsParseRdbAccessedP13sqljsDrdaAsCbP13sqljDDMObjectP14db2U
Cinterface + 0xFB
[21] 0x00007F74BC580A6E
_Z10sqljsParseP13sqljsDrdaAsCbP14db2UCinterfaceP8sqeAgentb +
0x54E
[22] 0x00007F74BC57394D
/home/db2inst1/sqllib/lib64/libdb2e.so.1 + 0x5A6D94D
[23] 0x00007F74BC579F33
/home/db2inst1/sqllib/lib64/libdb2e.so.1 + 0x5A73F33
[24] 0x00007F74BC57ABEF
_Z17sqljsDrdaAsDriverP18SQLCC_INITSTRUCT_T + 0x11F
[25] 0x00007F74BBFAE953 _ZN8sqeAgent6RunEDUEv + 0xDE3
[26] 0x00007F74BF87084E _ZN9sqzEDUObj9EDUDriverEv + 0x1BE
[27] 0x00007F74BDDD1DB8 sqloEDUEntry + 0x578
[28] 0x00007F74C5889DC5 /lib64/libpthread.so.0 + 0x7DC5
[29] 0x00007F74B55BF76D clone + 0x6D
Note that stack above is for db2dbroutext hitting this problem,
however there may be other places in the code calling the same.
The cause of the issue is related to rolling updates scenario in
the PureScale clusters. Routine db_get_cfg had been updated
between v11.1 fix packs (i.e. Mod 2 Fix Pack 2 iFix002 and Mod 3
Fix Pack 3), so the different number of parameters are processed
by db_get_cfg function. Db2 does not compare with actual number
of elements in rolling updates scenario (in db_get_cfg), hence
stack as above. |
