DB2 - Problem description
| Problem IC76438 | Status: Closed |
Instance abend and possible crash recovery failure on V9.7 Fix Pack 2 using circular logging. | |
| product: | |
DB2 FOR LUW / DB2FORLUW / 980 - DB2 | |
| Problem description: | |
This problem may manifest itself in numerous ways.
As a first symptom , the instance will abend in a variety of
possible locations.
Some known function in the stack traces are :
1.
sqlp_get_log_filepath_for_lsn
db2diag.log entry :
2010-07-01-01.13.49.945856+540 I14975442A499 LEVEL: Warning
PID : 1781890 TID : 13624 PROC : db2sysc
4
INSTANCE: db2inst1 NODE : 004 DB : DBNAME
APPHDL : 0-157 APPID: *N0.db2inst1.100729011146
AUTHID : DB2INST1
EDUID : 13624 EDUNAME: db2lload 4
FUNCTION: DB2 UDB, RAS/PD component,
pdEDUIsInDB2KernelOperation, probe:600
DATA #1 : String, 33 bytes
sqlp_get_log_filepath_for_lsn__F
DATA #2 : String, 4 bytes
sqlp
2.
sqlpgwlp
sqlpgasn2
2010-07-01-09.56.58.161004-240 I59812464E384 LEVEL:
Warning
PID : 31052 TID : 479683 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000
EDUID : 75 EDUNAME: db2loggw (SAMPLE) 0
FUNCTION: DB2 UDB, RAS/PD component,
pdEDUIsInDB2KernelOperation, probe:600
DATA #1 : String, 12 bytes
_Z8sqlpgwlp
DATA #2 : String, 4 bytes
sqlp
3.
sqlpshrBuildRecord
sqlpshrEdu
2010-07-01-08.20.01.832248+540 I15631649A503 LEVEL: Warning
PID : 835836 TID : 13880 PROC : db2sysc
INSTANCE: db2inst1 NODE : 000 DB : SAMPLE
APPHDL : 2-51 APPID: *N0.db2inst1.100802232000
AUTHID : DB2INST1
EDUID : 13880 EDUNAME: db2shred (SAMPLE)
FUNCTION: DB2 UDB, RAS/PD component,
pdEDUIsInDB2KernelOperation, probe:600
DATA #1 : String, 27 bytes
@109@sqlpshrBuildRecord__F
DATA #2 : String, 4 bytes
sqlp
-
sqlpshrBuildRecord
sqlpshrEdu
other traps are possible as well.
A db2diag.log message which also occurs when the problem happens
is the following :
2010-07-01-09.46.35.712009+540 I13910007A467 LEVEL: Error
PID : 1417464 TID : 3600 PROC : db2sysc
4
INSTANCE: db2inst1 NODE : 004 DB : SAMPLE
EDUID : DB2INST1 EDUNAME: db2loggr (SAMPLE) 4
FUNCTION: DB2 UDB, data protection services, sqlpgspr, probe:550
DATA #1 : String, 117 bytes
Reclaimed too much log space. Recalculating LogBytesInUse.
Old LogBytesInUse: 74753276 New LogBytesInUse: 8124749448
and following this :
2010-07-01-07.06.17.705235+000 E34053A598 LEVEL: Severe
PID : 417804 TID : 5142 PROC : db2sysc
0
INSTANCE: db2cbp NODE : 000
EDUID : 5142 EDUNAME: db2loggw (SAMPLE) 0
FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
probe:1010
MESSAGE : ZRC=0x85100009=-2062548983=SQLP_NOSPACE
"Log File has reached its saturation point"
DIA8309C Log file was full.
DATA #1 : <preformatted>
Error getting next log file to write to. Filecount 20, active
20, inactive 40, tailindex 18446744073709551596 currentRecord 16
Note that this exhaust the available transaction log space (
SQLP_NOSPACE error ) and will bring the database down.
An eyecatcher is the very high tailindex value which , when
converted to hex, is close to 0xFFFF FFFF FFFF FFFF.
When subsequently crash recovery is attempted, this may fail
with the following message during the undo phase :
2010-12-20-09.34.11.751803+000 I5359A415 LEVEL: Info
PID : 12093 TID : 13 PROC : db2sysc
0
INSTANCE: db2inst3 NODE : 000 DB : SAMPLE
EDUID : 13 EDUNAME: db2loggr (SAMPLE) 0
FUNCTION: DB2 UDB, recovery manager, sqlpgSwitchFromRedoToUndo,
probe:806
DATA #1 : <preformatted>
TailIndex 1 firstLso 39333401 nextLso 39349705 logGapSize 0
2010-07-01-09.34.11.962973+000 I7301A532 LEVEL: Severe
PID : 12093 TID : 14 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000
EDUID : 14 EDUNAME: db2loggw (SAMPLE) 0
FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
probe:909
MESSAGE : ZRC=0x8610000D=-2045771763=SQLP_BADLOG "Log File
cannot be used"
DIA8414C Logging can not continue due to an error.
DATA #1 : <preformatted>
TailPage 0 does not match pagePso 39349778 and firstLso 39007321 | |
| Problem Summary: | |
**************************************************************** * USERS AFFECTED: * * All * **************************************************************** * PROBLEM DESCRIPTION: * * This APAR only applies to V9.7 FP2 database that use * * circular logging. * * * * When the database is circular logging(sqlpgicl), * * thefirstLso in FCB * * array are all initialized to * * baselso.2010-07-29-10.13.49.945856+540 I14975442A499 * * LEVEL:WarningPID : 1781890 TID : 13624 * * PROC :db2sysc 4INSTANCE: db2inst1 NODE : 004 * * DB : XXXXXXAPPHDL : 0-157 * * APPID:*N0.db2inst1.100729011146AUTHID : XXXXXEDUID : 13624 * * EDUNAME: db2lload 4FUNCTION: DB2 UDB, RAS/PD * * component,pdEDUIsInDB2KernelOperation, probe:600DATA #1 : * * String, 33 bytessqlp_get_log_filepath_for_lsn__FDATA #2 : * * String, 4 bytessqlpStacks<StackTrace>------Function + * * Offset------sqlp_get_log_filepath_for_lsnsqluRegisterLoadEndca * + 0x260</StackTrace> * **************************************************************** * RECOMMENDATION: * * (1) Upgrade to V9.7 Fix Pack 3, or, * * (2) Use archiving log * **************************************************************** | |
| Local Fix: | |
One of the following methods may be used to attempt to work around the issue: 1. Install Fixpack 3a: https://www-304.ibm.com/support/docview.wss?uid=swg24028306 2. Configure the database to use archive logging, then restart the database. For example: db2 terminate db2 deactivate db dbname db2 update db cfg for dbname using logarchmeth1 <...> db2 activate db dbname 3. Restore the database from a backup image. 4. Contact DB2 Support for assistance. | |
| Solution | |
Problem was first fixed in Version 9.7 Fix Pack 3. | |
| Workaround | |
not known / see Local fix | |
| Timestamps | |
Date - problem reported : Date - problem closed : Date - last modified : | 17.05.2011 17.05.2011 17.05.2011 |
| Problem solved at the following versions (IBM BugInfos) | |
9.7.FP3, 9.7.FP3 | |
| Problem solved according to the fixlist(s) of the following version(s) | |