db2set DB2_AGENT_CACHING=OFF, dbm cfg KEEPFENCED = YES,
FENCED_POOL = x (where x is a very small number than the
db2agent pool).
admin fmp worker is used by 2 DB2 shipped system routines, SAP
workload uses one of them SYSPROC.ADMIN_CMD. SAP workload
generally runs normal non-threadsafe C routines, which will
fill the fenced pool, with DB2_AGENT_CACHING=OFF they are
returned to the pool as soon as the routine execution is
completed. Following that SYSPROC.ADMIN_CMD is called with
current clients, and given that the pool is full, they are
being terminated as soon as their work is done. A more
efficient routine infrastructure would try to keep a percentage
of this type of works in the pool, so that future workload can
use them instead of creating new ones.
As a result of this shortcoming in design, the db2diag.log is
filled with admin fmp worker being terminated messages.
Here is an example of the db2diag.log messages :
2009-05-20-12.54.45.235332-240 E1660A535 LEVEL: Warning
PID : 2257034 TID : 14138 PROC : db2sysc
0
INSTANCE: xxxxxx NODE : 000 DB : xxx
APPHDL : 0-39 APPID:
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
AUTHID : xxxxxx
EDUID : 14138 EDUNAME: db2agent (xxx) 0
FUNCTION: DB2 UDB, routine_infrastructure, sqlerReturnFmpToPool,
probe:2000
DATA #1 : String, 55 bytes
Sending a signal to clean up NOT THREADED FMP process:
DATA #2 : Process ID, 4 bytes
1966198
2009-05-20-12.54.45.235788-240 I2196A1216 LEVEL: Error
PID : 2257034 TID : 14138 PROC : db2sysc
0
INSTANCE: xxxxxx NODE : 000 DB : xxx
APPHDL : 0-39 APPID:
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
AUTHID : xxxxxx
EDUID : 14138 EDUNAME: db2agent (xxx) 0
FUNCTION: DB2 UDB, oper system services, sqlossig, probe:10
MESSAGE : Sending SIGKILL to the following process id
DATA #1 : signed integer, 4 bytes
1966198
CALLSTCK:
[0] 0x0900000020314580 pdLog + 0xD0
[1] 0x090000001FED0C44 pdLog@glue3E1 + 0xD0
[2] 0x09000000213DA524 sqlossig + 0xEC
[3] 0x09000000238A1C08 sqlossig@glue5E8 + 0x7C
[4] 0x090000001FD74968
sqlerReturnFmpToPool__FcT1P14sqlerFmpHandleP8sqeAgent + 0x620
[5] 0x090000002014DBF0
sqlerReturnFmpToPool__FcT1P14sqlerFmpHandleP8sqeAgent@glue5B3 +
0x7C
[6] 0x09000000204B7E98
sqlri_close_udf_ufob_cleanup__FP10sqlri_ufobP5sqlcaP8sqlrr_cb +
0x3C
[7] 0x090000002035D4D0
sqlri_close_udf_eos__FP25sqlra_sql_context_siblingiP8sqlrr_cb +
0x160
[8] 0x0900000022D2589C
sqlri_close_udf_eos__FP25sqlra_sql_context_siblingiP8sqlrr_cb@gl
ueF80 + 0x78
[9] 0x09000000204FDED8 sqlricls_complex__FP8sqlrr_cbilN23 +
0xA4
2009-05-20-12.54.45.240918-240 E3413A470 LEVEL: Warning
PID : 2523314 TID : 258 PROC : db2wdog
0
INSTANCE: xxxxxx NODE : 000
EDUID : 258 EDUNAME: db2wdog 0
FUNCTION: DB2 UDB, routine_infrastructure, sqlerReturnFmpToPool,
probe:999
DATA #1 : String, 34 bytes
Removing FMP from pool FMP handle:
DATA #2 : sqlerFmpHandle, PD_SQLER_TYPE_FMP_HANDLE, 16 bytes
fmpPid: 1966198 pFmpEntry: 0x0000000000000000
2009-05-20-12.54.45.241285-240 E3884A490 LEVEL: Warning
PID : 2523314 TID : 258 PROC : db2wdog
0
INSTANCE: xxxxxx NODE : 000
EDUID : 258 EDUNAME: db2wdog 0
FUNCTION: DB2 UDB, routine_infrastructure, sqlerReturnFmpToPool,
probe:1000
DATA #1 : String, 10 bytes
Fmp Entry:
DATA #2 : sqlerFmpThreadList, PD_SQLER_TYPE_FMP_THREAD_LIST, 408
bytes
Object not dumped: Address: 0x0000000000000000 Size: 408 Reason:
Address is NULL
2009-05-20-12.54.45.241590-240 E4375A396 LEVEL: Error
PID : 2523314 TID : 258 PROC : db2wdog
0
INSTANCE: xxxxxx NODE : 000
EDUID : 258 EDUNAME: db2wdog 0
FUNCTION: DB2 UDB, base sys utilities, sqleChildCrashHandler,
probe:15
DATA #1 : <preformatted>
A non-EDU child crashed. Process id: 1966198, OSS term code:
0x102, signal: 9
This APAR is to make improvement to this design and allows the
routine infrastructure to be more effective in handling a mixed
type of non-threadsafe routine workload.
More noticeablely, user should see much less routine termination
messages with SAP workload due to admin fmp worker termination
in db2diag.log with this fix in place. |
