DB2 - Problem description
Problem IT27943 | Status: Closed |
Deadlatch between an online backup, prefetchers and an agent doing a write to the same table space | |
product: | |
DB2 FOR LUW / DB2FORLUW / B10 - DB2 | |
Problem description: | |
A deadlatch might happen when an online backup is holding the latch SQLO_LT_SQLB_POOL_CB__readLotch in S mode and waiting on free prefetcher requests in the following stack: -----FUNC-ADDR---- ------FUNCTION + OFFSET------ 0x00007FCD0390CCA6 _Z25ossDumpStackTraceInternalmR11OSSTrapFileiP7siginfoPvmm + 0x0356 (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD0390C8EB ossDumpStackTraceV98 + 0x002b (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD03908AD7 _ZN11OSSTrapFile6dumpExEmiP7siginfoPvm + 0x00d7 (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD0B170187 sqlo_trce + 0x03c7 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B2DBC0B sqloDumpDiagInfoHandler + 0x020b (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD12E71680 address: 0x00007FCD12E71680 ; dladdress: 0x00007FCD12E62000 ; offset in lib: 0x000000000000F680 ; (/lib64/libpthread.so.0) 0x00007FCD029172A3 msgrcv + 0x0033 (/lib64/libc.so.6) 0x00007FCD0B27C259 sqloCSemP + 0x0279 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05DB6D14 _Z26sqlbPFFindAndLatchFreelistiPP15SQLB_pfFreelistP12SQLB_GLOBAL S + 0x04b4 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05DBA0F8 _Z29sqlbPFGetMultipleFreeRequestsjiPP14SQLB_pfRequestPmP12SQLB_G LOBALS14SQLB_pfReqTypeP16SQLB_OBJECT_DESCt + 0x0068 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05DB87DA _Z25sqlbpfParallelDirectReadVP14SQLB_pfParIoCbP17SQLB_DIRECT_IO_ CB + 0x18aa (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05BE2FDD _Z19sqlbDirectReadBlockP19SQLB_DIRECT_READ_CBP14SQLB_pfParIoCbjP jS3_ + 0x05dd (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD09C5B295 _Z12sqlubReadDMSP14SQLU_BUFMAN_CB + 0x0485 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD09C59C14 _Z18sqlubBMProcessDataP14SQLU_BUFMAN_CB + 0x0304 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD09C5915F _Z8sqlubbufP8sqeAgent + 0x051f (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD09476320 _Z26sqleSubCoordProcessRequestP8sqeAgent + 0x0150 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD094931AE _ZN8sqeAgent6RunEDUEv + 0x05ae (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0CDB321E _ZN9sqzEDUObj9EDUDriverEv + 0x01be (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B2D96BA sqloEDUEntry + 0x057a (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD12E69DD5 address: 0x00007FCD12E69DD5 ; dladdress: 0x00007FCD12E62000 ; offset in lib: 0x0000000000007DD5 ; (/lib64/libpthread.so.0) 0x00007FCD02915B3D clone + 0x006d (/lib64/libc.so.6) An agent trying to make some change to the same table space is waiting behind the online backup on the latch SQLO_LT_SQLB_POOL_CB__readLotch in X mode, a sample stack is as follow. Note it is not always the same stack because it depends on what writing operation this agent is doing to the table space. -----FUNC-ADDR---- ------FUNCTION + OFFSET------ 0x00007FCD0390CCA6 _Z25ossDumpStackTraceInternalmR11OSSTrapFileiP7siginfoPvmm + 0x0356 (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD0390C8EB ossDumpStackTraceV98 + 0x002b (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD03908AD7 _ZN11OSSTrapFile6dumpExEmiP7siginfoPvm + 0x00d7 (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD0B170187 sqlo_trce + 0x03c7 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B2DBC0B sqloDumpDiagInfoHandler + 0x020b (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD12E71680 address: 0x00007FCD12E71680 ; dladdress: 0x00007FCD12E62000 ; offset in lib: 0x000000000000F680 ; (/lib64/libpthread.so.0) 0x00007FCD02917337 semop + 0x0007 (/lib64/libc.so.6) 0x00007FCD0B16E3A2 _ZN17SQLO_SLATCH_CAS6418getConflictComplexEm + 0x07e2 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B16DBBC _ZN17SQLO_SLATCH_CAS6411getConflictEm + 0x007c (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05363F6F _ZN12sqlpValLotch12getLatchOnlyEmPKcm + 0x016f (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05454E18 _Z21sqlbLatchPoolR_inlineP12SQLB_POOL_CBibiPKc + 0x00b8 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05BC1799 _Z25sqlbLatchPoolForReadWriteP12SQLB_POOL_CB + 0x0029 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05DEC71E _Z13sqlbUpdateMRTtP9SQLP_LSN8PmP12SQLB_GLOBALS + 0x00ee (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0605E6D3 _Z8sqldmpndP8sqeAgentiPcP9SQLP_LSN8PmP15SQLD_RECOV_INFO + 0x0c23 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B3D8518 _Z8sqlptpplP8sqeAgentiPb + 0x0668 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B3BBEBF _Z8sqlpxcm1P8sqeAgentP15SQLXA_CALL_INFOi + 0x042f (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B96447F _Z12sqlrrcom_dpsP8sqlrr_cbiiP15SQLXA_CALL_INFO + 0x165f (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B961FE0 _Z8sqlrrcomP8sqlrr_cbii + 0x0550 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B93A176 _Z12sqlrr_commitP14db2UCinterface + 0x0136 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD09A8E1E8 _Z16sqljs_ddm_rdbcmmP14db2UCinterfaceP13sqljDDMObject + 0x0228 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) ... At the same time, prefetchers are waiting behind the writer agent on the latch SQLO_LT_SQLB_POOL_CB__readLotch in S mode in the following stack: -----FUNC-ADDR---- ------FUNCTION + OFFSET------ 0x00007FCD0390CCA6 _Z25ossDumpStackTraceInternalmR11OSSTrapFileiP7siginfoPvmm + 0x0356 (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD0390C8EB ossDumpStackTraceV98 + 0x002b (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD03908AD7 _ZN11OSSTrapFile6dumpExEmiP7siginfoPvm + 0x00d7 (/home/db2mcys/sqllib/lib64/libdb2osse.so.1) 0x00007FCD0B170187 sqlo_trce + 0x03c7 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B2DBC0B sqloDumpDiagInfoHandler + 0x020b (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD12E71680 address: 0x00007FCD12E71680 ; dladdress: 0x00007FCD12E62000 ; offset in lib: 0x000000000000F680 ; (/lib64/libpthread.so.0) 0x00007FCD02917337 semop + 0x0007 (/lib64/libc.so.6) 0x00007FCD0B16E3A2 _ZN17SQLO_SLATCH_CAS6418getConflictComplexEm + 0x07e2 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B16DBBC _ZN17SQLO_SLATCH_CAS6411getConflictEm + 0x007c (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05363F6F _ZN12sqlpValLotch12getLatchOnlyEmPKcm + 0x016f (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05454E18 _Z21sqlbLatchPoolR_inlineP12SQLB_POOL_CBibiPKc + 0x00b8 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05B9824B _Z25sqlbReadAndReleaseBuffersP16SQLB_OBJECT_DESCjmmmP11SQLB_pfMi sc + 0x02bb (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05B9B9F7 _Z22sqlbServiceListRequestP11SQLB_pfMiscP16SQLB_OBJECT_DESCPjj + 0x04b7 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05B95C17 _Z26sqlbPFPrefetcherEntryPointP16sqbPrefetcherEdu + 0x1227 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD05B94964 _ZN16sqbPrefetcherEdu6RunEDUEv + 0x0024 (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0CDB321E _ZN9sqzEDUObj9EDUDriverEv + 0x01be (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD0B2D96BA sqloEDUEntry + 0x057a (/home/db2mcys/sqllib/lib64/libdb2e.so.1) 0x00007FCD12E69DD5 address: 0x00007FCD12E69DD5 ; dladdress: 0x00007FCD12E62000 ; offset in lib: 0x0000000000007DD5 ; (/lib64/libpthread.so.0) 0x00007FCD02915B3D clone + 0x006d (/lib64/libc.so.6) | |
Problem Summary: | |
**************************************************************** * USERS AFFECTED: * * ALL * **************************************************************** * PROBLEM DESCRIPTION: * * See Error Description * **************************************************************** * RECOMMENDATION: * * Upgrade to DB2 Version 11.1 Module 4 Fix Pack 5 * **************************************************************** | |
Local Fix: | |
db2set DB2BPVARS= in the config file, have this line: NOREADVBCKUP=1 Note that this might have performance impact on the backup | |
Solution | |
Workaround | |
not known / see Local fix | |
BUG-Tracking | |
forerunner : follow-up : IT30784 | |
Timestamps | |
Date - problem reported : Date - problem closed : Date - last modified : | 31.01.2019 31.10.2019 31.10.2019 |
Problem solved at the following versions (IBM BugInfos) | |
Problem solved according to the fixlist(s) of the following version(s) |