DB2 FOR LUW / DB2FORLUW / 970 - DB2

(This issue affects DB2 9.7 fixpack 6 only.  It should not 
affect AIX and Windows platforms.  Any other "load hanging" 
issues that do not match the diagnostics below, or are not with 
DB2 9.7 fixpack 6, are not related to this APAR.) 
 
A load hangs at startup time with the following symptoms: 
 
(1) in db2diag.log there are error messages indicating a 
SQLO_QUE_NOT_SENT error, followed by messages pointing to the 
sqluAllocTBufs() function, like these: 
 
2012-06-04-16.26.29.483991-240 E473381E3213          LEVEL: 
Error (OS) 
PID     : 25108                TID : 46931657091392  KTID : 
29706 
PROC    : db2sysc 0 
INSTANCE: eunicec              NODE : 000            DB   : 
CORAL 
APPHDL  : 0-82                 APPID: *N0.eunicec.120604202625 
AUTHID  : EUNICEC              HOSTNAME: coralm221 
EDUID   : 143                  EDUNAME: db2agent (CORAL) 0 
FUNCTION: DB2 UDB, oper system services, sqlowqueInternal, 
probe:40 
MESSAGE : ZRC=0x870F0041=-2029060031=SQLO_QUE_NOT_SENT "Message 
Not Sent" 
          DIA8557C No message was sent using the message queue. 
CALLED  : OS, -, msgsnd                           OSERR: EAGAIN 
(11) 
DATA #1 : system V message queue identifier., 
PD_TYPE_SYSV_QUEUE_ID, 4 bytes 
0x35270044 
DATA #2 : Pointer, 8 bytes 
0x00002aaf20beb0a8 
DATA #3 : unsigned integer, 8 bytes 
48 
CALLSTCK: (Static functions may not be resolved correctly, as 
they are resolved to the nearest symbol) 
  [0] 0x00002AAAACDF8EF2 pdLogSysRC + 0x22C 
  [1] 0x00002AAAB074EE77 
_Z16sqlowqueInternalP13SQLO_QUE_DESCP12SQLO_MSG_HDRib + 0x549 
  [2] 0x00002AAAB074FDA7 
_Z15sqlowqueWrapperP13SQLO_QUE_DESCP12SQLO_MSG_HDRib + 0x59 
  [3] 0x00002AAAB075030A sqlowqueNoBlock + 0x10E 
  [4] 0x00002AAAB255BA0A 
_Z14sqluWriteQueueP13SQLO_QUE_DESCP12SQLO_MSG_HDRii + 0xE6 
  [5] 0x00002AAAB26C2330 _Z14sqluAllocTBufsP13SQLUCACB_TYPEPii + 
0x1E8A 
  [6] 0x00002AAAB26226D7 /home/eunicec/sqllib/lib64/libdb2e.so.1 
+ 0x79426D7 
  [7] 0x00002AAAB26099A8 
_Z9sqluvloadPhS_P9SQLP_LSN8P7sqlubmhPP15sqlu_media_listS6_hP26sq 
luCLoadRequestDescriptorP8sqluvlcbPi + 0x3A82 
  [8] 0x00002AAAB25208F2 /home/eunicec/sqllib/lib64/libdb2e.so.1 
+ 0x78408F2 
  [9] 0x00002AAAB25284C4 
_Z17sqluLoadPartitionP8sqeAgentP26sqluCLoadRequestDescriptoriiiP 
iS3_ + 0x11EE 
  [10] 0x00002AAAB251EA23 
_Z24sqlu_register_table_loadP26sqluCLoadRequestDescriptorP24sqlu 
sCLoadMPPCoordinatorP15sql_static_dataPb + 0x1BC9 
  [11] 0x00002AAAB24F2777 _Z17sqluvtld_route_inP12SQLE_DB2RA_T + 
0x2255 
  [12] 0x00002AAAADD958AA _Z8sqlesrvrP14db2UCinterface + 0x1336 
  [13] 0x00002AAAADDFE9BC _Z19sqleMappingFnServerP5sqldaP5sqlca 
+ 0x15EC 
  [14] 0x00002AAAAE3F3EC9 
_Z19sqlerKnownProcedureiPcPiP5sqldaS2_P13sqlerFmpTableP8sqeAgent 
P5sqlca + 0x18C9 
  [15] 0x00002AAAAE3FB397 
_Z11sqlerCallDLP14db2UCinterfaceP9UCstpInfo + 0x248F 
  [16] 0x00002AAAAE6D5D27 
_Z19sqljs_ddm_excsqlsttP14db2UCinterfaceP13sqljDDMObject + 0x4EB 
  [17] 0x00002AAAAE6A4FC2 
_Z21sqljsParseRdbAccessedP13sqljsDrdaAsCbP13sqljDDMObjectP14db2U 
Cinterface + 0x25C 
  [18] 0x00002AAAAE6A6249 
_Z10sqljsParseP13sqljsDrdaAsCbP14db2UCinterfaceP8sqeAgentb + 
0x239 
  [19] 0x00002AAAAE698899 
/home/eunicec/sqllib/lib64/libdb2e.so.1 + 0x39B8899 
  [20] 0x00002AAAAE69AADB 
/home/eunicec/sqllib/lib64/libdb2e.so.1 + 0x39BAADB 
  [21] 0x00002AAAAE695FDF 
/home/eunicec/sqllib/lib64/libdb2e.so.1 + 0x39B5FDF 
  [22] 0x00002AAAAE69584C 
_Z17sqljsDrdaAsDriverP18SQLCC_INITSTRUCT_T + 0x29A 
  [23] 0x00002AAAADE086E6 _ZN8sqeAgent6RunEDUEv + 0x4F8 
  [24] 0x00002AAAB3094260 _ZN9sqzEDUObj9EDUDriverEv + 0x17C 
  [25] 0x00002AAAB3093E88 _Z10sqlzRunEDUPcj + 0x26 
  [26] 0x00002AAAB08E8BB7 sqloEDUEntry + 0x971 
 
.................. 
 
2012-06-04-16.26.40.830447-240 I510014E1061          LEVEL: 
Error 
PID     : 25108                TID : 46931657091392  KTID : 
29706 
PROC    : db2sysc 0 
INSTANCE: eunicec              NODE : 000            DB   : 
CORAL 
APPHDL  : 0-82                 APPID: *N0.eunicec.120604202625 
AUTHID  : EUNICEC              HOSTNAME: coralm221 
EDUID   : 143                  EDUNAME: db2agent (CORAL) 0 
FUNCTION: DB2 UDB, database utilities, sqluAllocTBufs, 
probe:2599 
MESSAGE : ZRC=0x8015006D=-2146107283=SQLU_CA_BUILT 
          "SQLCA has already been built" 
DATA #1 : LOADID, PD_TYPE_LOADID, 47 bytes 
LOADID: 143.2012-06-04-16.26.25.889421.0 (22;4) 
DATA #2 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes 
 sqlcaid : SQLCA     sqlcabc: 136   sqlcode: -2044   sqlerrml: 1 
 sqlerrmc: 3 
 sqlerrp : SQLUBUF2 
 sqlerrd : (1) 0x8015006D      (2) 0x00000000      (3) 
0x00000000 
           (4) 0x00000000      (5) 0x00000000      (6) 
0x00000000 
 sqlwarn : (1)      (2)      (3)      (4)        (5)       (6) 
           (7)      (8)      (9)      (10)        (11) 
 sqlstate: 
 
2012-06-04-16.26.40.834263-240 I511076E555           LEVEL: 
Severe 
PID     : 25108                TID : 46931657091392  KTID : 
29706 
PROC    : db2sysc 0 
INSTANCE: eunicec              NODE : 000            DB   : 
CORAL 
APPHDL  : 0-82                 APPID: *N0.eunicec.120604202625 
AUTHID  : EUNICEC              HOSTNAME: coralm221 
EDUID   : 143                  EDUNAME: db2agent (CORAL) 0 
FUNCTION: DB2 UDB, database utilities, sqluAllocTBufs, 
probe:2599 
MESSAGE : DIA0001E An internal error occurred. Report the 
following error code 
          : "". 
 
 
(2) the hanging DB2 agent's stack trace shows load terminating 
functions (sqlulTermEDU() / sqlulTerminate()) trying to write to 
a message queue (sqlowque()) and is stuck, like this: 
 
0x00002AAAB8B3BD8F 
_Z25ossDumpStackTraceInternalmR11OSSTrapFileiP7siginfoPvmm + 
0x046f 
                (/home/eunicec/sqllib/lib64/libdb2osse.so.1) 
0x00002AAAB8B3C6DF ossDumpStackTraceV98 + 0x0081 
                (/home/eunicec/sqllib/lib64/libdb2osse.so.1) 
0x00002AAAB8B34BE7 _ZN11OSSTrapFile6dumpExEmiP7siginfoPvm + 
0x0377 
                (/home/eunicec/sqllib/lib64/libdb2osse.so.1) 
0x00002AAAB074A8B6 sqlo_trce + 0x0678 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB08EC534 sqloDumpDiagInfoHandler + 0x0212 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAABD4D60 address: 0x00002AAAAABD4D60 ; dladdress: 
0x00002AAAAABC7000 ; offset in lib: 0x000000000000DD60 ; 
                (/lib64/libpthread.so.0) 
0x00002AAAB954A368 msgsnd + 0x0058 
                (/lib64/libc.so.6) 
0x00002AAAB0752747 _Z21uninterruptableMsgSndiPKvmi + 0x0037 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB074EC87 
_Z16sqlowqueInternalP13SQLO_QUE_DESCP12SQLO_MSG_HDRib + 0x0359 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB074FDA7 
_Z15sqlowqueWrapperP13SQLO_QUE_DESCP12SQLO_MSG_HDRib + 0x0059 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB07501F0 sqlowque + 0x0036 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB255B9D8 
_Z14sqluWriteQueueP13SQLO_QUE_DESCP12SQLO_MSG_HDRii + 0x00b4 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB261970E 
_Z15sqlulSndTermMsgP13SqluQueueInfoP13SQLUCACB_TYPEN22SqluTermin 
ationMessage21LoadTerminationReasonE + 0x0230 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB2618AC4 
_Z14sqlulTerminateP13SQLUCACB_TYPEPsN22SqluTerminationMessage21L 
oadTerminationReasonEPi + 0x0550 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB260CB34 _Z12sqlulTermEDUP13SQLUCACB_TYPEiPi + 0x0144 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB260B99B 
_Z9sqluvloadPhS_P9SQLP_LSN8P7sqlubmhPP15sqlu_media_listS6_hP26sq 
luCLoadRequestDescriptorP8sqluvlcbPi + 0x5a75 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB25208F2 address: 0x00002AAAB25208F2 ; dladdress: 
0x00002AAAAACE0000 ; offset in lib: 0x00000000078408F2 ; 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB25284C4 
_Z17sqluLoadPartitionP8sqeAgentP26sqluCLoadRequestDescriptoriiiP 
iS3_ + 0x11ee 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB251EA23 
_Z24sqlu_register_table_loadP26sqluCLoadRequestDescriptorP24sqlu 
sCLoadMPPCoordinatorP15sql_static_dataPb + 0x1bc9 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB24F2777 _Z17sqluvtld_route_inP12SQLE_DB2RA_T + 0x2255 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAADD958AA _Z8sqlesrvrP14db2UCinterface + 0x1336 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAADDFE9BC _Z19sqleMappingFnServerP5sqldaP5sqlca + 
0x15ec 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE3F3EC9 
_Z19sqlerKnownProcedureiPcPiP5sqldaS2_P13sqlerFmpTableP8sqeAgent 
P5sqlca + 0x18c9 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE3FB397 _Z11sqlerCallDLP14db2UCinterfaceP9UCstpInfo + 
0x248f 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE6D5D27 
_Z19sqljs_ddm_excsqlsttP14db2UCinterfaceP13sqljDDMObject + 
0x04eb 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE6A4FC2 
_Z21sqljsParseRdbAccessedP13sqljsDrdaAsCbP13sqljDDMObjectP14db2U 
Cinterface + 0x025c 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE6A6249 
_Z10sqljsParseP13sqljsDrdaAsCbP14db2UCinterfaceP8sqeAgentb + 
0x0239 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE698899 address: 0x00002AAAAE698899 ; dladdress: 
0x00002AAAAACE0000 ; offset in lib: 0x00000000039B8899 ; 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE69AADB address: 0x00002AAAAE69AADB ; dladdress: 
0x00002AAAAACE0000 ; offset in lib: 0x00000000039BAADB ; 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE695FDF address: 0x00002AAAAE695FDF ; dladdress: 
0x00002AAAAACE0000 ; offset in lib: 0x00000000039B5FDF ; 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAE69584C _Z17sqljsDrdaAsDriverP18SQLCC_INITSTRUCT_T + 
0x029a 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAADE086E6 _ZN8sqeAgent6RunEDUEv + 0x04f8 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB3094260 _ZN9sqzEDUObj9EDUDriverEv + 0x017c 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB3093E88 _Z10sqlzRunEDUPcj + 0x0026 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAB08E8BB7 sqloEDUEntry + 0x0971 
                (/home/eunicec/sqllib/lib64/libdb2e.so.1) 
0x00002AAAAABCD193 address: 0x00002AAAAABCD193 ; dladdress: 
0x00002AAAAABC7000 ; offset in lib: 0x0000000000006193 ; 
                (/lib64/libpthread.so.0) 
0x00002AAAB95490DD __clone + 0x006d 
                (/lib64/libc.so.6)

**************************************************************** 
* USERS AFFECTED:                                              * 
* All DB2 LOAD users on Linux/Solaris/HP-UX platforms          * 
**************************************************************** 
* PROBLEM DESCRIPTION:                                         * 
* See Error Description                                        * 
**************************************************************** 
* RECOMMENDATION:                                              * 
* Upgrade to DB2 LUW Version 9.7 Fix Pack 7.                   * 
****************************************************************

Workarounds include 
1) Don't use CPU_PARALLELISM 1 (implicitly or explicitly), or 
explicitly use a higher value 
2) If you cannot change from CPU_PARALLELISM 1 (e.g. if table 
has LOBs), lower the DATA BUFFER value (explicitly, or 
implicitly via lowering the UTIL_HEAP_SZ database configuration 
parameter)

DB2 Version 9.7 Fix Pack 7 for Linux, UNIX, and Windows
 DB2 Version 9.7 Fix Pack 8 for Linux, UNIX, and Windows
 DB2 Version 9.7 Fix Pack 9 for Linux, UNIX, and Windows
 DB2 Version 9.7 Fix Pack 9a for Linux, UNIX, and Windows
 DB2 Version 9.7 Fix Pack 10 for Linux, UNIX, and Windows
 

Problem was first fixed in DB2 LUW Version 9.7 Fix Pack 7.

not known / see Local fix

Date  - problem reported    :
Date  - problem closed      :
Date  - last modified       :

14.06.2012
14.11.2012
14.11.2012

9.7.FP7