** THIS APAR APPLIES ONLY TO Db2 Multi Partition scenario **
Legitimate application / no connections can sometimes be forced
off
inadvertently when trying to force off system application
connections holding low priority locks e.g. ADC ( db2adcbld ) or
AutoGroom( db2abpgroom ) or Automatic runstats.
The messages in the diag log will look like ( only if db2set
DB2_FORCE_DIAGS=TRUE is set ):
Application gets this error:
------------------------------
SQL30081N A communication error has been detected.
Communication protocol
being used: "TCP/IP". Communication API being used: "SOCKETS".
Location
where the error was detected: "10.153.74.143". Communication
function
detecting the error: "send". Protocol specific error code(s):
"32", "*", "0".
SQLSTATE=08001
The application connection which initiates the force:
----------------------------------------------------------
2019-09-14-17.31.25.392889+120 I10679429E2907 LEVEL: Info
PID : 48633 TID : 47364194690816 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 25-45466 APPID:
10.151.187.181.36840.190914153109
AUTHID : BLUADMIN HOSTNAME: hlaawme2ia03
EDUID : 41563 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, lock manager, sqlplWaitOnWP, probe:1370
DATA #1 :
Forcing application following information
Workstation Location: elaawme2ba05
Application Name : db2abpgroom
Application ID : *N40.DB2.190914153122
Tablespace ID : 0
Object ID : 5
CALLSTCK: (Static functions may not be resolved correctly, as
they are resolved to the nearest symbol)
[0] 0x00002AEFE16C3F4D pdLogPrintf + 0x8D
[1] 0x00002AEFE8BB1E9C
_Z32sqlpDumpForceConflictingAppDiagsP12SQLZ_APP_HDLiiP14SQLP_LOC
K_INFOb + 0x26C
[2] 0x00002AEFE8BA0244
_Z13sqlplWaitOnWPP9sqeBsuEduP14SQLP_LOCK_INFOP8SQLP_LRBP15SQLP_L
TRN_CHAINbbb + 0x3014
[3] 0x00002AEFE8B95B19
_Z24sqlplMakeNewRequestNonSDP9sqeBsuEduP14SQLP_LOCK_INFOP11SQLP_
TENTRYP8SQLP_LRBS6_P15SQLP_LTRN_CHAINbbb + 0x899
[4] 0x00002AEFE89DA0FB _Z7sqlplrqP9sqeBsuEduP14SQLP_LOCK_INFO
+ 0xEDB
The force received on the node where the system application is
running (when everything is forced off):
----------------------------------------------------------------
----------------
2019-09-14-17.31.25.604271+120 I16088052E1427 LEVEL:
Warning
PID : 110145 TID : 47390635583232 PROC :
db2sysc 40
INSTANCE: db2inst1 NODE : 040
HOSTNAME: hlaawme2ia18
EDUID : 64 EDUNAME: db2pdbc 40
FUNCTION: DB2 UDB, base sys utilities,
sqeAppServices::ForceAllApplications, probe:100
DATA #1 :
Forcing application with the following information
Workstation Location: elaawme2ba06
Application Name : pmdtm
Application ID : 10.151.187.181.52105.190914150518
Application Handle : 40-44651
CALLSTCK: (Static functions may not be resolved correctly, as
they are resolved to the nearest symbol)
[0] 0x00002B19D5EE7F4D pdLogPrintf + 0x8D
[1] 0x00002B19DB10A885
_ZN14sqeAppServices20ForceAllApplicationsEP14sqeApplicationiiP16
sqeLocalDatabaseiiP12sqzDataChainI19SQLE_COORDINATOR_CB16sqzCha
+ 0x285
[2] 0x00002B19DB10A29E
_ZN14sqeAppServices17ForceApplicationsEP14sqeApplicationiP12SQLZ
_APP_HDLiiii + 0x15E
[3] 0x00002B19DAFB2BB9
_Z15sqleForceAgentsiiP12SQLZ_APP_HDLP5sqlcaiii + 0x159
[4] 0x00002B19DAFD3E21
_Z23sqlePdbSyscProcessForceP23SQLKD_RQST_REPLY_FORMAT + 0x121
[5] 0x00002B19DB0588D2 _ZN22sqePdbSystemController6RunEDUEv +
0x302
[6] 0x00002B19DEB09BBE _ZN9sqzEDUObj9EDUDriverEv + 0x1BE
[7] 0x00002B19DCF62F8A sqloEDUEntry + 0x57A
[8] 0x00002B19D36D6DD5 /lib64/libpthread.so.0 + 0x7DD5
[9] 0x00002B19E69DB02D clone + 0x6D
If nothing is forced off the following is observed
----------------------------------------------------------------
----------------
2019-09-23-15.33.01.970310-240 I1501029E471 LEVEL:
Severe
PID : 7247 TID : 139864200701696 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000
HOSTNAME: hotellnx120
EDUID : 36 EDUNAME: db2pdbc 0
FUNCTION: DB2 UDB, base sys utilities,
sqeAppServices::ForceSpecificApplications, probe:5
MESSAGE : Invalid force agent ID:
DATA #1 : Hexdump, 4 bytes
0x00007F34ABFFD23C : 1600 0000
....
Pls note the intention was to remove only the system application
( in this case the groom connection ) but it issues a force
application all on the node where the groom connection had
connected to which in turn causes the TCP/IP error for the
legitimate application connection. In our second example the
wrong application handle was sent, so nothing gets forced off. |
