Database System crash causes the production database damaged.
User can not make new connection on database.
Multiple FODC observed before DATABASE CRASH
Db2diag.log error messages:
2019-08-13-13.16.13.506658+000 I28539404E630 LEVEL:
Severe
PID : 4079 TID : 139808458401536 PROC :
db2sysc 0
INSTANCE: db2edwp1 NODE : 000 DB :
DBEDWP1
APPHDL : 0-16862 APPID:
10.5.38.27.47106.190813131543
UOWID : 1 ACTID: 111068
AUTHID : EDAVDM1P HOSTNAME: sl55edwdbp01
EDUID : 231370 EDUNAME: db2agent (DBEDWP1) 0
FUNCTION: DB2 UDB, index manager, sqliUndoAddKey, probe:519
RETCODE : ZRC=0x8510000A=-2062548982=SQLP_LFUL
"Lock list full - SQL0912 Reason code 1"
DIA8310C Lock list was full.
Root Cause:
When undoing a change of a page which another transaction is
currently performing a structure modification operation (SMO)
such as a page split on, the undo process will see the SMO bit
and try to grab the tree lock again, which might hit the lock
manager boundary and crash the database.
The issue happened during undo, typically during undo we do not
acquire any additional resources, ie. we should have acquired
whatever lock during statement execution, not during undo .. or
at least reserved the LRB via IN mode.
But for index undo a special case could happen if another
transaction is doing a structure modification operation(SMO) and
holding a tree lock, the undo process would wait for that SMO
finished by acquiring the tree lock to see if has been released
by the SMO, which might hit the lock manager boundary and crash
the database. |
