Takeover from Standby may fail due to SQLB_INV_POOLID. This can
happen when any tablespace is in abnormal state on Standby.
Diaglog Messages -
The tablespace 8 was in abnormal state on standby and is added
to filter list:
2019-07-05-17.08.58.370467-300 I56314965A570 LEVEL:
Warning
PID : 49808374 TID : 69664 PROC :
db2sysc 0
INSTANCE: db2inst2 NODE : 000 DB :
SAMPLE
APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858
HOSTNAME: Standby
EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpGetTablespacesForFilter, probe:1570
DATA #1 :
Table space MONITOR (8) is in rollforward pending state.
Another rollforward will be needed to bring this tablespace
online.
2019-07-05-17.08.58.371027-300 I56315536A544 LEVEL: Info
PID : 49808374 TID : 69664 PROC :
db2sysc 0
INSTANCE: db2inst2 NODE : 000 DB :
SAMPLE
APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858
HOSTNAME: Standby
EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0
FUNCTION: DB2 UDB, data protection services,
sqlpMasterDbcb::sqlpAddTbspToAbnormalList, probe:20
MESSAGE : Added tablespace to abnormal tablespace list.
DATA #1 : Pool ID, PD_TYPE_SQLB_POOL_ID, 2 bytes
8
2019-07-08-13.19.30.816522-300 I46756A434 LEVEL:
Info
PID : 49808374 TID : 69400 PROC :
db2sysc 0
INSTANCE: db2inst2 NODE : 000 DB :
SAMPLE
HOSTNAME: Standby
EDUID : 69400 EDUNAME: db2hadrs.0.0 (SAMPLE)
0
FUNCTION: DB2 UDB, High Availability Disaster Recovery,
hdrStbyTkHandleInitialRequest, probe:46000
MESSAGE : Standby has initiated a takeover by force.
2019-07-08-13.19.31.861422-300 I61567A7659 LEVEL:
Severe
PID : 49808374 TID : 69664 PROC :
db2sysc 0
INSTANCE: db2inst2 NODE : 000 DB :
SAMPLE
APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858
HOSTNAME: Standby
EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0
FUNCTION: , , , probe:1382
MESSAGE : ZRC=0x87020076=-2029911946=SQLB_INV_POOLID
"Invalid pool ID"
DATA #1 : String, 112 bytes
sqlbGetExistingPoolDef: Error getting pool definition after
retrying. The caller may need to use sqlbGetPoolDef.
DATA #2 : Pool ID, PD_TYPE_SQLB_POOL_ID, 2 bytes
65535
DATA #3 : SQLB_PTBL, PD_TYPE_SQLB_PTBL, 1160 bytes
poolCount: 35
maxPoolID: 34
ptbl_flags: 1
pool_table_latch:
{
state = 0x0000000000000000
= {
held X: 0
reserved for X: 0
shared holders: 0
firstSharIndex: 0x0
firstExclIndex: 0x0
}
starve X mode = true
xWaitCount = 0
requestCount = 0
identity = SQLB_PTBL::pool_table_latch (142)
}
DisablePoolCount: 1
numSysTempPools:
0x0A00030001233724 : 0000 0000 0000 0000 0000 0000 0000 0002
................
defaultSysTempPool:
0x0A00030001233734 : FFFF FFFF FFFF FFFF FFFF FFFF 0000 0001
................
dropPendingList: 0x0000000000000000
suspendingList: 0x0000000000000000
suspendAction: 0
containersToDelete: 0x0a0003000120eba0
poolCBs:
0x0A00030001233768 : 0A00 0300 0121 0820 0000 0000 0000 0000
.....!. ........
maxIdInGrowableArrays: 511
CALLSTCK: (Static functions may not be resolved correctly,
as they are resolved to the nearest symbol)
[0] 0x0900000008872598
sqlbPutEndLfs__FP12SQLB_GLOBALSP18SQLB_POOL_STATE_IN + 0x3458
[1] 0x09000000092A50DC
sqlprSetPoolEndLfs__FP8sqeAgentCUsCUl + 0xDC
[2] 0x090000000C5E42FC
sqlpForwardRecovery__FP8sqeAgentUsPCcPiPUiT2 + 0x377C
[3] 0x090000000CA995A8 sqlpReplayMaster__FP8sqeAgent +
0x1DC8
[4] 0x090000000A431C8C
sqleIndCoordProcessRequest__FP8sqeAgent + 0x10CC
[5] 0x090000000794A8E8 RunEDU__8sqeAgentFv + 0xAA8
[6] 0x0900000007924BC0 EDUDriver__9sqzEDUObjFv + 0x300
[7] 0x0900000007806200 sqloEDUEntry + 0x360
[8] 0x090000000055DE10 _pthread_body + 0xF0
[9] 0xFFFFFFFFFFFFFFFC ?unknown + 0xFFFFFFFF
2019-07-08-13.19.31.863054-300 E69227A945 LEVEL:
Critical
PID : 49808374 TID : 69664 PROC :
db2sysc 0
INSTANCE: db2inst2 NODE : 000 DB :
SAMPLE
APPHDL : 0-652 APPID: *LOCAL.DB2.190705220858
HOSTNAME: Standby
EDUID : 69664 EDUNAME: db2agent (SAMPLE) 0
FUNCTION: DB2 UDB, RAS/PD component, pdStartFODC, probe:10
MESSAGE : ADM14001C An unexpected and critical error has
occurred: "Panic".
The instance may have been shutdown as a result.
"Automatic" FODC
(First Occurrence Data Capture) has been invoked
and diagnostic
information has been recorded in directory
"/db2inst2/SAMPLE/backup/DB2DUMP/FODC_Panic_2019-07-08-13.19.31.
862
298_0000/". Please look in this directory for
detailed evidence about
what happened and contact IBM support if necessary
to diagnose the
problem.
0x0900000000575EF4 : EB81FFE0 ld r28,-32(r1)
0x0900000000575EF8 : EBA1FFE8 ld r29,-24(r1)
0x0900000000575EFC : EBC1FFF0 ld r30,-16(r1)
0x0900000000575F00 : EBE1FFF8 ld r31,-8(r1)
0x0900000000575F04 : 4E800020 br # 20,bit0
0x0900000000575F08 : 3860FFFF li r3,0xFFFF
0x0900000000575F0C : 63840000 ori r4,r28,0x0
0x0900000000575F10 : 48000921 bl 0x900000000576830
>>>> 0x0900000000575F14 : E8410028 ld r2,40(r1)
0x0900000000575F18 : 38600000 li r3,0x0
0x0900000000575F1C : EAA10098 ld r21,152(r1)
0x0900000000575F20 : E9810100 ld r12,256(r1)
0x0900000000575F24 : 382100F0 addi r1,r1,240
0x0900000000575F28 : 7D8803A6 mtlr r12
-------Frame------ ------Function + Offset------
0x0900000000575F14 pthread_kill + 0xD4
0x0900000000575764 _p_raise + 0x44
0x0900000000039768 raise + 0x48
0x0900000000056164 abort + 0xC4
0x090000000780C9F8 sqloExitEDU + 0x298
0x0900000007878F44 sqle_panic__Fi + 0x7A4
0x09000000086DA228 sqlb_panic + 0xA8
0x09000000088725A0
sqlbPutEndLfs__FP12SQLB_GLOBALSP18SQLB_POOL_STATE_IN + 0x3460
0x09000000092A50DC sqlprSetPoolEndLfs__FP8sqeAgentCUsCUl + 0xDC
0x090000000C5E42FC sqlpForwardRecovery__FP8sqeAgentUsPCcPiPUiT2
+ 0x377C
0x090000000CA995A8 sqlpReplayMaster__FP8sqeAgent + 0x1DC8
0x090000000A431C8C sqleIndCoordProcessRequest__FP8sqeAgent +
0x10CC
0x090000000794A8E8 RunEDU__8sqeAgentFv + 0xAA8
0x0900000007924BC0 EDUDriver__9sqzEDUObjFv + 0x300
0x0900000007806200 sqloEDUEntry + 0x360 |
