DB2 - Problem description
Problem IT29736 | Status: Closed |
REPLAYING DROP TABLESPACE WOULD SET POOL ID TO -1 IN FILTERED TABLESPACE LIST ON STANDBY, CAUSING SQLPRSETPOOLENDLFS() TO PANIC | |
product: | |
DB2 FOR LUW / DB2FORLUW / B10 - DB2 | |
Problem description: | |
Takeover from Standby may fail due to SQLB_INV_POOLID. This can happen when any tablespace is in abnormal state on Standby. Diaglog Messages - The tablespace 8 was in abnormal state on standby and is added to filter list: 2019-07-05-17.08.58.370467-300 I56314965A570 LEVEL: Warning PID : 49808374 TID : 69664 PROC : db2sysc 0 INSTANCE: db2inst2 NODE : 000 DB : SAMPLE APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858 HOSTNAME: Standby EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0 FUNCTION: DB2 UDB, recovery manager, sqlpGetTablespacesForFilter, probe:1570 DATA #1 : Table space MONITOR (8) is in rollforward pending state. Another rollforward will be needed to bring this tablespace online. 2019-07-05-17.08.58.371027-300 I56315536A544 LEVEL: Info PID : 49808374 TID : 69664 PROC : db2sysc 0 INSTANCE: db2inst2 NODE : 000 DB : SAMPLE APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858 HOSTNAME: Standby EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0 FUNCTION: DB2 UDB, data protection services, sqlpMasterDbcb::sqlpAddTbspToAbnormalList, probe:20 MESSAGE : Added tablespace to abnormal tablespace list. DATA #1 : Pool ID, PD_TYPE_SQLB_POOL_ID, 2 bytes 8 2019-07-08-13.19.30.816522-300 I46756A434 LEVEL: Info PID : 49808374 TID : 69400 PROC : db2sysc 0 INSTANCE: db2inst2 NODE : 000 DB : SAMPLE HOSTNAME: Standby EDUID : 69400 EDUNAME: db2hadrs.0.0 (SAMPLE) 0 FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrStbyTkHandleInitialRequest, probe:46000 MESSAGE : Standby has initiated a takeover by force. 2019-07-08-13.19.31.861422-300 I61567A7659 LEVEL: Severe PID : 49808374 TID : 69664 PROC : db2sysc 0 INSTANCE: db2inst2 NODE : 000 DB : SAMPLE APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858 HOSTNAME: Standby EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0 FUNCTION: , , , probe:1382 MESSAGE : ZRC=0x87020076=-2029911946=SQLB_INV_POOLID "Invalid pool ID" DATA #1 : String, 112 bytes sqlbGetExistingPoolDef: Error getting pool definition after retrying. The caller may need to use sqlbGetPoolDef. DATA #2 : Pool ID, PD_TYPE_SQLB_POOL_ID, 2 bytes 65535 DATA #3 : SQLB_PTBL, PD_TYPE_SQLB_PTBL, 1160 bytes poolCount: 35 maxPoolID: 34 ptbl_flags: 1 pool_table_latch: { state = 0x0000000000000000 = { held X: 0 reserved for X: 0 shared holders: 0 firstSharIndex: 0x0 firstExclIndex: 0x0 } starve X mode = true xWaitCount = 0 requestCount = 0 identity = SQLB_PTBL::pool_table_latch (142) } DisablePoolCount: 1 numSysTempPools: 0x0A00030001233724 : 0000 0000 0000 0000 0000 0000 0000 0002 ................ defaultSysTempPool: 0x0A00030001233734 : FFFF FFFF FFFF FFFF FFFF FFFF 0000 0001 ................ dropPendingList: 0x0000000000000000 suspendingList: 0x0000000000000000 suspendAction: 0 containersToDelete: 0x0a0003000120eba0 poolCBs: 0x0A00030001233768 : 0A00 0300 0121 0820 0000 0000 0000 0000 .....!. ........ maxIdInGrowableArrays: 511 CALLSTCK: (Static functions may not be resolved correctly, as they are resolved to the nearest symbol) [0] 0x0900000008872598 sqlbPutEndLfs__FP12SQLB_GLOBALSP18SQLB_POOL_STATE_IN + 0x3458 [1] 0x09000000092A50DC sqlprSetPoolEndLfs__FP8sqeAgentCUsCUl + 0xDC [2] 0x090000000C5E42FC sqlpForwardRecovery__FP8sqeAgentUsPCcPiPUiT2 + 0x377C [3] 0x090000000CA995A8 sqlpReplayMaster__FP8sqeAgent + 0x1DC8 [4] 0x090000000A431C8C sqleIndCoordProcessRequest__FP8sqeAgent + 0x10CC [5] 0x090000000794A8E8 RunEDU__8sqeAgentFv + 0xAA8 [6] 0x0900000007924BC0 EDUDriver__9sqzEDUObjFv + 0x300 [7] 0x0900000007806200 sqloEDUEntry + 0x360 [8] 0x090000000055DE10 _pthread_body + 0xF0 [9] 0xFFFFFFFFFFFFFFFC ?unknown + 0xFFFFFFFF 2019-07-08-13.19.31.863054-300 E69227A945 LEVEL: Critical PID : 49808374 TID : 69664 PROC : db2sysc 0 INSTANCE: db2inst2 NODE : 000 DB : SAMPLE APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858 HOSTNAME: Standby EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0 FUNCTION: DB2 UDB, RAS/PD component, pdStartFODC, probe:10 MESSAGE : ADM14001C An unexpected and critical error has occurred: "Panic". The instance may have been shutdown as a result. "Automatic" FODC (First Occurrence Data Capture) has been invoked and diagnostic information has been recorded in directory "/db2inst2/SAMPLE/backup/DB2DUMP/FODC_Panic_2019-07-08-13.19.31. 862 298_0000/". Please look in this directory for detailed evidence about what happened and contact IBM support if necessary to diagnose the problem. 0x0900000000575EF4 : EB81FFE0 ld r28,-32(r1) 0x0900000000575EF8 : EBA1FFE8 ld r29,-24(r1) 0x0900000000575EFC : EBC1FFF0 ld r30,-16(r1) 0x0900000000575F00 : EBE1FFF8 ld r31,-8(r1) 0x0900000000575F04 : 4E800020 br # 20,bit0 0x0900000000575F08 : 3860FFFF li r3,0xFFFF 0x0900000000575F0C : 63840000 ori r4,r28,0x0 0x0900000000575F10 : 48000921 bl 0x900000000576830 >>>> 0x0900000000575F14 : E8410028 ld r2,40(r1) 0x0900000000575F18 : 38600000 li r3,0x0 0x0900000000575F1C : EAA10098 ld r21,152(r1) 0x0900000000575F20 : E9810100 ld r12,256(r1) 0x0900000000575F24 : 382100F0 addi r1,r1,240 0x0900000000575F28 : 7D8803A6 mtlr r12 -------Frame------ ------Function + Offset------ 0x0900000000575F14 pthread_kill + 0xD4 0x0900000000575764 _p_raise + 0x44 0x0900000000039768 raise + 0x48 0x0900000000056164 abort + 0xC4 0x090000000780C9F8 sqloExitEDU + 0x298 0x0900000007878F44 sqle_panic__Fi + 0x7A4 0x09000000086DA228 sqlb_panic + 0xA8 0x09000000088725A0 sqlbPutEndLfs__FP12SQLB_GLOBALSP18SQLB_POOL_STATE_IN + 0x3460 0x09000000092A50DC sqlprSetPoolEndLfs__FP8sqeAgentCUsCUl + 0xDC 0x090000000C5E42FC sqlpForwardRecovery__FP8sqeAgentUsPCcPiPUiT2 + 0x377C 0x090000000CA995A8 sqlpReplayMaster__FP8sqeAgent + 0x1DC8 0x090000000A431C8C sqleIndCoordProcessRequest__FP8sqeAgent + 0x10CC 0x090000000794A8E8 RunEDU__8sqeAgentFv + 0xAA8 0x0900000007924BC0 EDUDriver__9sqzEDUObjFv + 0x300 0x0900000007806200 sqloEDUEntry + 0x360 | |
Problem Summary: | |
**************************************************************** * USERS AFFECTED: * * ALL * **************************************************************** * PROBLEM DESCRIPTION: * * See Error Description * **************************************************************** * RECOMMENDATION: * * Upgrade to Db2 11.1 Mod 4 Fixpack 5 or higher * **************************************************************** | |
Local Fix: | |
The workaround is to deactivate and activate on standby after drop tablespace is replayed on standby. This way the dropped tablespace will not appear in filtered tablespace list and takeover will not have any problem. If the above is not then then only way is to reinitialize HADR on standby. | |
Solution | |
Workaround | |
not known / see Local fix | |
Timestamps | |
Date - problem reported : Date - problem closed : Date - last modified : | 15.07.2019 16.01.2020 16.01.2020 |
Problem solved at the following versions (IBM BugInfos) | |
Problem solved according to the fixlist(s) of the following version(s) |