DB2 - Problem description
| Problem IT05856 | Status: Closed |
CLP MEMORY CORRUPTION RUNNING STORED PROCEDURE FROM 9.7 CLP CLIENT TO Z/OS | |
| product: | |
DB2 CONNECT / DB2CONNCT / 970 - DB2 | |
| Problem description: | |
When repro like below is executed from a v9.7 CLP client
connected to z/os db, core dump or trap is hit with the stacks
pasted below the repro:
db2 "create procedure test_char1(in p1 char(3), in i1 int, in i2
int, in i3 int, in i4 int, in i5 int, in i6 int, in i7 int,in i8
int, in i9 int,in i10 int, in i11 int, in i12 int, in i13 int,
in
i14 int, in i15 int, in i16 int, in i17 int, in i18 int, in i19
int, in i20 int, in i21 int, in i22 int, in i23 int, in i24 int,
in i25 int, in i26 int, in i27 int, in i28 int, in i29 int, in
i30
int, in i31 int, in i32 int, in i33 int, in i34 int, in i35 int,
in i36 int, in i37 int,in i38 int, in i39 int, in i40 int, in
i41 int, in i42 int, in i43 int, in i44 int, in i45 int, in i46
int,in
i47 int, in i48 int, in i49 int, in i50 int, in i51 int, in i52
int, in i53 int, in i54 int, in i55 int, in i56 int, in i57 int,
in i58 int, in i59 int, in i60 int,out o1 cha
r(3)) LANGUAGE SQL BEGIN select 'XXX' into o1 from
sysibm.sysdummy1; END"
db2 "call test_char1('SAB',
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,2
5,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46
,47,48,49,50,51,52,53,54,55,56,57,58,59,60,?)"
another repro is :
db2 "create procedure test_char(in p1 char(3), in i1 int, in i2
int, in i3 int, in i4 int, in i5 int, in i6 int, in i7 int, in
i8 int, in i9 int,in i10 int, in i11 int, in i12 int, in i13
int, in
i14 int, in i15 int, in i16 int, in i17 int, in i18 int, in i19
int, in i20 int, in i21 int, in i22 int, in i23 int, in i24 int,
in i25 int, in i26 int, in i27 int, in i28 int, in i29 int, in
i30
int, in i31 int, in i32 int, in i33 int, in i34 int, in i35 int,
in i36 int, in i37 int,in i38 int, in i39 int, in i40 int, in
i41 int, in i42 int, out o1 char(3))
LANGUAGE SQL
BEGIN
select 'XXX' into o1 from sysibm.sysdummy1;
END "
db2 "call test_char('SAB',
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,2
5,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46
,47,48,49,50,51,52,53,54,55,56,57,58,59,60,?)"
Stack seen in db2diag.log:
<StackTrace>
---FUNC-ADDR---- ------FUNCTION + OFFSET------
00002AAAAD181A51 ossDumpStackTrace + 0x024b
(/home/sakauser/sqllib/lib/libdb2osse.so.1)
00002AAAAD17C200 _ZN11OSSTrapFile4dumpEmiP7siginfoPv + 0x024e
(/home/sakauser/sqllib/lib/libdb2osse.so.1)
00002AAAAB9DDF67 sqlo_trce + 0x038b
(/home/sakauser/sqllib/lib/libdb2.so.1)
00000000004B2269 clp_bp_sigterm + 0x00f5
(/home/sakauser/sqllib/bin/db2bp)
00002AAAADC44FC0 address: 0x00002AAAADC44FC0 ; dladdress:
0x00002AAAADC15000 ; offset in lib: 0x000000000002FFC0 ;
(/lib64/libc.so.6)
00002AAAADC87DC0 strlen + 0x0040
(/lib64/libc.so.6)
0000000000414AA3 _Z18clpbp_exe_sqlecallv + 0x0919
(/home/sakauser/sqllib/bin/db2bp)
000000000045BA96 _Z18clpbp_exe_sqleprocv + 0x065e
(/home/sakauser/sqllib/bin/db2bp)
00000000004B1446 _Z10clp_bp_conv + 0x126a
(/home/sakauser/sqllib/bin/db2bp)
00000000004AFA86 main + 0x010a
(/home/sakauser/sqllib/bin/db2bp)
00002AAAADC32304 __libc_start_main + 0x00f4
(/lib64/libc.so.6)
0000000000411849 __gxx_personality_v0 + 0x00f9
(/home/sakauser/sqllib/bin/db2bp)
</StackTrace>
or
CALLSTCK:
[0] 0x000007FEF68D0A6A pdLog + 0x3F0
[1] 0x000007FEF65F6AF0 sqloDiagnoseFreeBlockFailure +
0x21C
[2] 0x000007FEF65FA0BD sqlofmblkEx + 0x2653
[3] 0x000000013F9424EA clpbp_exe_sqlecall + 0x29704
[4] 0x000000013F9BD504 clpbp_exe_sqleproc + 0x1216
[5] 0x000000013FA8FEF9 clp_bp_con + 0x2C1F
[6] 0x000000013FA8C30A main + 0x270
[7] 0x000000013FD0670D __tmainCRTStartup + 0x19D
[8] 0x000000013FD0683E mainCRTStartup + 0xE
[9] 0x00000000773DF33D BaseThreadInitThunk + 0xD
[10] 0x00000000779D2CA1 RtlUserThreadStart + 0x21
Entries in db2diag.log:
Example 1:
2014-10-29-15.00.57.806000-120 E853050H499 LEVEL: Severe
PID : 7288 TID : 8980 PROC :
db2bp.exe
INSTANCE: DB2 NODE : 000
APPID : 10.105.1.62.34269.141029170044
EDUID : 8980
FUNCTION: DB2 UDB, SQO Memory Management,
sqloDiagnoseFreeBlockFailure, probe:999
MESSAGE : Memory validation failure, diagnostic file dumped.
DATA #1 : String, 28 bytes
Corrupt pool free tree node.
DATA #2 : File name, 29 bytes
7288.8980.mem_diagnostics.txt
2014-10-29-15.00.57.807000-120 I853551H1192 LEVEL: Warning
PID : 7288 TID : 8980 PROC :
db2bp.exe
INSTANCE: DB2 NODE : 000
APPID : 10.105.1.62.34269.141029170044
EDUID : 8980
FUNCTION: DB2 UDB, SQO Memory Management, sqlogmblkEx,
probe:1000
MESSAGE : ZRC=0x820F0002=-2112946174=SQLO_INV_MEM "Invalid
memory addr"
DIA8561C A invalid memory block was encountered.
DATA #1 : String, 43 bytes
Memory management block allocation failure.
DATA #2 : Codepath, 8 bytes
4:20:28
DATA #3 : Memory pool handle pointer,
PD_TYPE_MEM_POOL_HANDLE_PTR, 4 bytes
0x00000000
DATA #4 : Requested size, PD_TYPE_MEM_REQUESTED_SIZE, 4 bytes
7
DATA #5 : Adjusted block size, PD_TYPE_MEM_ADJUSTED_SIZE, 4
bytes
32
DATA #6 : Options for requested block, PD_TYPE_GET_MEM_OPTIONS,
4 bytes
0x00000000
DATA #7 : Pointer to address that will be set by new allocation,
PD_TYPE_PTR_TO_ADDRESS_OUT, 4 bytes
0x0331a8f0
DATA #8 : File name, PD_TYPE_OSS_MEM_FILE_NAME, 10 bytes
clpbcmd1.C
DATA #9 : Line of code, PD_TYPE_OSS_MEM_LINE_NUM, 4 bytes
5269
DATA #10: Resource binding pointer,
PD_TYPE_RESOURCE_BINDING_PTR, 4 bytes
0x00000000
2014-10-29-15.00.57.808000-120 E854745H815 LEVEL:
Critical
PID : 7288 TID : 8980 PROC :
db2bp.exe
INSTANCE: DB2 NODE : 000
APPID : 10.105.1.62.34269.141029170044
EDUID : 8980
FUNCTION: DB2 UDB, SQO Memory Management,
sqloDiagnoseFreeBlockFailure, probe:10
MESSAGE : ADM14001C An unexpected and critical error has
occurred: "Panic".
The instance may have been shutdown as a result.
"Automatic" FODC
(First Occurrence Data Capture) has been invoked and
diagnostic
information has been recorded in directory
Example 2 :
20653 data DB2 UDB SQO Memory Management
sqloDiagnoseFreeBlockFailure cei (3.3.129.89.2.10)
pid 10340 tid 11504 cpid -1 node 0 probe 10
bytes 1754
Data1 (PD_TYPE_DIAG_LOG_REC,1746) Diagnostic log record:
2014-11-07-01.27.03.880000-300 I3963825F1744 LEVEL:
Severe
PID : 10340 TID : 11504 PROC :
db2bp.exe
INSTANCE: DB2 NODE : 000 DB :
STLEC1
APPID : 9.26.98.10.53632.141107062642
HOSTNAME: HOST1
EDUID : 11504
FUNCTION: DB2 UDB, SQO Memory Management,
sqloDiagnoseFreeBlockFailure, probe:10
MESSAGE : Possible memory corruption detected.
DATA #1 : ZRC, PD_TYPE_ZRC, 4 bytes
0x820F0002
DATA #2 : Corrupt block address, PD_TYPE_CORRUPT_BLK_PTR, 8
bytes
0x00000000005e70c0
DATA #3 : Block header, PD_TYPE_BLK_HEADER, 24 bytes
0x00000000005E70A8 : E8AF 2CDB 9D00 0000 9096 6C2C 410B 5800
..,.......l,A.X.
0x00000000005E70B8 : E005 0000 0000 B0FA
........
code: 0xdb2cafe8
curSize: 157
SMemBlkFileId: 745313936
SMemBlkLOC: 2881
SMemBlkNum: 88
CodedCSG: fab00000000005e0
DATA #4 : Data header, PD_TYPE_BLK_DATA_HEAD, 48 bytes
0x00000000005E70C0 : 4341 4C4C 204E 4557 544F 4E2E 5445 5354
CALL NEWTON.TEST
0x00000000005E70D0 : 5F43 4841 5220 2820 3F2C 203F 2C20 3F2C
_CHAR ( ?, ?, ?,
0x00000000005E70E0 : 203F 2C20 3F2C 203F 2C20 3F2C 203F 2C20
?, ?, ?, ?, ?,
CALLSTCK:
[0] 0x000007FEF68D0A6A pdLog + 0x3F0
[1] 0x000007FEF65F6AF0 sqloDiagnoseFreeBlockFailure +
0x21C
[2] 0x000007FEF65FA0BD sqlofmblkEx + 0x2653
[3] 0x000000013F9424EA clpbp_exe_sqlecall + 0x29704
[4] 0x000000013F9BD504 clpbp_exe_sqleproc + 0x1216
[5] 0x000000013FA8FEF9 clp_bp_con + 0x2C1F
[6] 0x000000013FA8C30A main + 0x270
[7] 0x000000013FD0670D __tmainCRTStartup + 0x19D
[8] 0x000000013FD0683E mainCRTStartup + 0xE
[9] 0x00000000773DF33D BaseThreadInitThunk + 0xD
[10] 0x00000000779D2CA1 RtlUserThreadStart + 0x21 | |
| Problem Summary: | |
**************************************************************** * USERS AFFECTED: * * All DB2 CLP users * **************************************************************** * PROBLEM DESCRIPTION: * * See Error Description * **************************************************************** * RECOMMENDATION: * * upgrade to v97fp11 and later * **************************************************************** | |
| Local Fix: | |
| Solution | |
upgrade to v97fp11 | |
| Workaround | |
not known / see Local fix | |
| Timestamps | |
Date - problem reported : Date - problem closed : Date - last modified : | 02.12.2014 08.10.2015 08.10.2015 |
| Problem solved at the following versions (IBM BugInfos) | |
9.7.FP11 | |
| Problem solved according to the fixlist(s) of the following version(s) | |
| 9.7.0.11 |
|