DB2 - Problem description
| Problem IC70482 | Status: Closed |
Instance abend and possible crash recovery failure on V9.7 Fix Pack 2 using circular logging. | |
| product: | |
DB2 FOR LUW / DB2FORLUW / 970 - DB2 | |
| Problem description: | |
This problem may manifest itself in numerous ways.
As a first symptom , the instance will abend in a variety of
possible locations.
Some known function in the stack traces are :
1.
sqlp_get_log_filepath_for_lsn
db2diag.log entry :
2010-07-01-01.13.49.945856+540 I14975442A499 LEVEL: Warning
PID : 1781890 TID : 13624 PROC : db2sysc
4
INSTANCE: db2inst1 NODE : 004 DB : DBNAME
APPHDL : 0-157 APPID: *N0.db2inst1.100729011146
AUTHID : DB2INST1
EDUID : 13624 EDUNAME: db2lload 4
FUNCTION: DB2 UDB, RAS/PD component,
pdEDUIsInDB2KernelOperation, probe:600
DATA #1 : String, 33 bytes
sqlp_get_log_filepath_for_lsn__F
DATA #2 : String, 4 bytes
sqlp
2.
sqlpgwlp
sqlpgasn2
2010-07-01-09.56.58.161004-240 I59812464E384 LEVEL:
Warning
PID : 31052 TID : 479683 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000
EDUID : 75 EDUNAME: db2loggw (SAMPLE) 0
FUNCTION: DB2 UDB, RAS/PD component,
pdEDUIsInDB2KernelOperation, probe:600
DATA #1 : String, 12 bytes
_Z8sqlpgwlp
DATA #2 : String, 4 bytes
sqlp
3.
sqlpshrBuildRecord
sqlpshrEdu
2010-07-01-08.20.01.832248+540 I15631649A503 LEVEL: Warning
PID : 835836 TID : 13880 PROC : db2sysc
INSTANCE: db2inst1 NODE : 000 DB : SAMPLE
APPHDL : 2-51 APPID: *N0.db2inst1.100802232000
AUTHID : DB2INST1
EDUID : 13880 EDUNAME: db2shred (SAMPLE)
FUNCTION: DB2 UDB, RAS/PD component,
pdEDUIsInDB2KernelOperation, probe:600
DATA #1 : String, 27 bytes
@109@sqlpshrBuildRecord__F
DATA #2 : String, 4 bytes
sqlp
-
sqlpshrBuildRecord
sqlpshrEdu
other traps are possible as well.
A db2diag.log message which also occurs when the problem happens
is the following :
2010-07-01-09.46.35.712009+540 I13910007A467 LEVEL: Error
PID : 1417464 TID : 3600 PROC : db2sysc
4
INSTANCE: db2inst1 NODE : 004 DB : SAMPLE
EDUID : DB2INST1 EDUNAME: db2loggr (SAMPLE) 4
FUNCTION: DB2 UDB, data protection services, sqlpgspr, probe:550
DATA #1 : String, 117 bytes
Reclaimed too much log space. Recalculating LogBytesInUse.
Old LogBytesInUse: 74753276 New LogBytesInUse: 8124749448
and following this :
2010-07-01-07.06.17.705235+000 E34053A598 LEVEL: Severe
PID : 417804 TID : 5142 PROC : db2sysc
0
INSTANCE: db2cbp NODE : 000
EDUID : 5142 EDUNAME: db2loggw (SAMPLE) 0
FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
probe:1010
MESSAGE : ZRC=0x85100009=-2062548983=SQLP_NOSPACE
"Log File has reached its saturation point"
DIA8309C Log file was full.
DATA #1 : <preformatted>
Error getting next log file to write to. Filecount 20, active
20, inactive 40, tailindex 18446744073709551596 currentRecord 16
Note that this exhaust the available transaction log space (
SQLP_NOSPACE error ) and will bring the database down.
An eyecatcher is the very high tailindex value which , when
converted to hex, is close to 0xFFFF FFFF FFFF FFFF.
When subsequently crash recovery is attempted, this may fail
with the following message during the undo phase :
2010-12-20-09.34.11.751803+000 I5359A415 LEVEL: Info
PID : 12093 TID : 13 PROC : db2sysc
0
INSTANCE: db2inst3 NODE : 000 DB : SAMPLE
EDUID : 13 EDUNAME: db2loggr (SAMPLE) 0
FUNCTION: DB2 UDB, recovery manager, sqlpgSwitchFromRedoToUndo,
probe:806
DATA #1 : <preformatted>
TailIndex 1 firstLso 39333401 nextLso 39349705 logGapSize 0
2010-07-01-09.34.11.962973+000 I7301A532 LEVEL: Severe
PID : 12093 TID : 14 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000
EDUID : 14 EDUNAME: db2loggw (SAMPLE) 0
FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
probe:909
MESSAGE : ZRC=0x8610000D=-2045771763=SQLP_BADLOG "Log File
cannot be used"
DIA8414C Logging can not continue due to an error.
DATA #1 : <preformatted>
TailPage 0 does not match pagePso 39349778 and firstLso 39007321 | |
| Problem Summary: | |
**************************************************************** * USERS AFFECTED: * * All * **************************************************************** * PROBLEM DESCRIPTION: * * This APAR only applies to V9.7 FP2 database that use * * circular logging. * * * * When the database is circular logging(sqlpgicl), * * thefirstLso in FCB * * array are all initialized to * * baselso.2010-07-29-10.13.49.945856+540 I14975442A499 * * LEVEL:WarningPID : 1781890 TID : 13624 * * PROC :db2sysc 4INSTANCE: db2inst1 NODE : 004 * * DB : XXXXXXAPPHDL : 0-157 * * APPID:*N0.db2inst1.100729011146AUTHID : XXXXXEDUID : 13624 * * EDUNAME: db2lload 4FUNCTION: DB2 UDB, RAS/PD * * component,pdEDUIsInDB2KernelOperation, probe:600DATA #1 : * * String, 33 bytessqlp_get_log_filepath_for_lsn__FDATA #2 : * * String, 4 bytessqlpStacks<StackTrace>------Function + * * Offset------sqlp_get_log_filepath_for_lsnsqluRegisterLoadEndca * + 0x260</StackTrace> * **************************************************************** * RECOMMENDATION: * * (1) Upgrade to V9.7 Fix Pack 3, or, * * (2) Use archiving log * **************************************************************** | |
| Local Fix: | |
One of the following methods may be used to attempt to work around the issue: 1) Restore database from a backup image 2) Reset log control file(SQLOGCTL.LFH) and bypass Crash recovery. However, db2 should be rebuild after resetting log control file. 2) Contact DB2 Support for assistance. Transaction log may be patched with internal tool. | |
| available fix packs: | |
DB2 Version 9.7 Fix Pack 3 for Linux, UNIX, and Windows | |
| Solution | |
Problem was first fixed in Version 9.7 Fix Pack 3. | |
| Workaround | |
not known / see Local fix | |
| BUG-Tracking | |
forerunner : APAR is sysrouted TO one or more of the following: IC76438 IC77551 follow-up : | |
| Timestamps | |
Date - problem reported : Date - problem closed : Date - last modified : | 10.08.2010 23.09.2010 05.10.2011 |
| Problem solved at the following versions (IBM BugInfos) | |
9.7.FP3, 9.7.FP3, 9.7.FP3.2 | |
| Problem solved according to the fixlist(s) of the following version(s) | |
| 9.7.0.3 |
|
| 9.7.0.3 |
|