|
Problem IT23357
|
Status: Closed |
On lager systems with many CPUs (typically NUMA) with heavy workload one might notice a spikes in commit response time
|
| product: |
DB2 FOR LUW / DB2FORLUW / A50 - DB2 |
| Problem description: |
1. Problem description
On lager systems with many CPUs (typically NUMA) with heavy
workload one might notice a spikes in commit response time due
to LOGTS Full
DB2 10.5 fp8
- migrating from older hardware (40-core) to newer
hardware (56-core typically NUMA) and started seeing
performance/scalability problems when heavy parallel insert
activity was performed on the database.
- Performance degraded due to db2logts processing -- when the
ring buffer gets full agents are forced to wait for a full
second and try again
- workaround of using DB2PRIORITIES=db2logts,59 was tried and
was unsuccessful
- workaround of using DB2_COLLECT_TS_REC_INFO=OFF was tried and
was successful BUT is not acceptable to the customer due to the
need of tablespace restore and rollforward
- this is a known defect, fixed in v95fp8 by changing
sqlosleep(1) to sqlorest(1)
- that fix was backed out in v101fp2 and fixed in an alternate
way by increasing the size of the ring buffer from 128 to 512
Example Stack:
-----FUNC-ADDR---- ------FUNCTION + OFFSET------
0x00007FFFEBDDBD05
_Z25ossDumpStackTraceInternalmR11OSSTrapFileiP7siginfoPvmm +
0x0385
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2osse.so.1)
0x00007FFFEBDDB90C ossDumpStackTraceV98 + 0x002c
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2osse.so.1)
0x00007FFFEBDD79AD _ZN11OSSTrapFile6dumpExEmiP7siginfoPvm +
0x00fd
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2osse.so.1)
0x00007FFFF2444004 sqlo_trce + 0x0404
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF24CE7FE sqloDumpDiagInfoHandler + 0x010e
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF7BD0370 address: 0x00007FFFF7BD0370 ; dladdress:
0x00007FFFF7BC1000 ; offset in lib: 0x000000000000F370 ;
(/lib64/libpthread.so.0)
0x00007FFFEB06466D nanosleep + 0x002d
(/lib64/libc.so.6)
0x00007FFFEB064504 sleep + 0x00d4
(/lib64/libc.so.6)
0x00007FFFF0400AC3 sqlosleep + 0x00c3
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF24FD099
_Z24sqlpUploadDirtyPoolEntryP11SQLP_TENTRYP9SQLP_DBCBb + 0x0339
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF04B7285 _Z17sqlpEndUowRuntimeP8sqeAgent + 0x0a35
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF04A72CF _Z8sqlpxcm1P8sqeAgentP15SQLXA_CALL_INFOi +
0x062f
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF0669801
_Z12sqlrrcom_dpsP8sqlrr_cbiiP15SQLXA_CALL_INFO + 0x0251
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF0666D64 _Z8sqlrrcomP8sqlrr_cbii + 0x0374
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF065610F _Z12sqlrr_commitP14db2UCinterface + 0x052f
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFEF7AA4B7
_Z16sqljs_ddm_rdbcmmP14db2UCinterfaceP13sqljDDMObject + 0x01c7
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF3412AB8
_Z21sqljsParseRdbAccessedP13sqljsDrdaAsCbP13sqljDDMObjectP14db2U
Cinterface + 0x0128
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFEF78E37B
_Z10sqljsParseP13sqljsDrdaAsCbP14db2UCinterfaceP8sqeAgentb +
0x036b
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFEF78897F address: 0x00007FFFEF78897F ; dladdress:
0x00007FFFEC302000 ; offset in lib: 0x000000000348697F ;
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFEF786DFC address: 0x00007FFFEF786DFC ; dladdress:
0x00007FFFEC302000 ; offset in lib: 0x0000000003484DFC ;
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFEF783E49 address: 0x00007FFFEF783E49 ; dladdress:
0x00007FFFEC302000 ; offset in lib: 0x0000000003481E49 ;
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFEF783A3B _Z17sqljsDrdaAsDriverP18SQLCC_INITSTRUCT_T +
0x00eb
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFEF48147F _ZN8sqeAgent6RunEDUEv + 0x0acf
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF0CB9897 _ZN9sqzEDUObj9EDUDriverEv + 0x00f7
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF0469A83 sqloEDUEntry + 0x0303
(/bb/db2/home/db2inst1/sqllib/lib64/libdb2e.so.1)
0x00007FFFF7BC8DC5 address: 0x00007FFFF7BC8DC5 ; dladdress:
0x00007FFFF7BC1000 ; offset in lib: 0x0000000000007DC5 ;
(/lib64/libpthread.so.0)
0x00007FFFEB09D76D clone + 0x006d
(/lib64/libc.so.6)
Holding Latch type: (SQLO_LT_SQLP_TENTRY__tranEntryLatch) -
Address: (0x2aaaabc8d948), Line: 1365, File:
/view/db2_v105fp8_linuxamd64_s160901/vbs/engn/include/sqlpt_inli
nes.h HoldCount: 1 |
| Problem Summary: |
****************************************************************
* USERS AFFECTED: *
* Look Error Description. *
****************************************************************
* PROBLEM DESCRIPTION: *
* See Error Description *
****************************************************************
* RECOMMENDATION: *
* Upgrade to 10.5Fp10 *
**************************************************************** |
| Local Fix: |
db2set DB2_COLLECT_TS_REC_INFO=OFF
recycle the instance.
Or upgrade to 10.5 FP10 and above.
db2set DB2_NUM_LOGTS_ENTRY=X , Default is 1024. Ideally default
value should work on most systems. Reactivation of database is
needed if this is set. |
| Solution |
|
| Workaround |
not known / see Local fix |
| Timestamps |
Date - problem reported :
Date - problem closed :
Date - last modified :
| 30.11.2017
11.07.2018
28.03.2019 |
| Problem solved at the following versions (IBM BugInfos) |
|
| Problem solved according to the fixlist(s) of the following version(s) |
