It's possible for the database to hang in a check point request
when the current logical log fills, such that a thread trying to
write to that log is in a critical section, but the server can't
switch into the next logical log, because a previous long
transaction that was rolled back originated in that log. The
main_loop thread is stuck waiting for all threads to exit
critical sections, and the thread in the critical section is
waiting to switch into the next log, but we can't switch into
the next log until a new checkpoint gets written into the
logical logs. So the server is hung.
onstat -u output would have at least 1 thread in a critical
section:
IBM Informix Dynamic Server Version 12.10.FC4W1 -- On-Line (CKPT
REQ) -- Up 00:02:03 -- 148076 Kbytes
Blocked:CKPT
Userthreads
address flags sessid user tty wait
tout locks nreads nwrites
...
44c121e8 --BPX-- 31 jrenaut 21 0
0 1 0 998
...
The stack trace for main_loop would look like this:
mt_yield
wait4critex
checkpoint
main_loop
th_init_initgls
startup
The stack trace for the session trying to switch the server into
the next log would look like this:
mt_yield
logchange **** this function may not show up as on some
platforms it will be in-lined and not visible in the stack trace
***
logwrite
log_put
logm_write
logputx
rscommit
iscommit
sqiscommit
committx
commitcmd
excommand
sq_execute
sqmain
listen_verify
spawn_thread
th_init_initgls
startup
If the logchange() function isn't in the stack, 1 way to infer
that stack is present is if the sqlexec thread for user in the
critical section shows up in onstat -g ath with a status of
"sleeping secs: 5" (where 5 could be anything in the range of
1-5) so something like this:
IBM Informix Dynamic Server Version 12.10.FC4W1 -- On-Line (CKPT
REQ) -- Up 00:06:32 -- 148076 Kbytes
Blocked:CKPT
Threads:
tid tcb rstcb prty status
vp-class name
...
58 45e407f0 44c121e8 1 sleeping secs:
5 1cpu sqlexec
...
Also you would see this message in MSGPATH file:
10:25:47 Logical Log 91 Complete, timestamp: 0x60e91.
10:25:54 Logical Log 92 Complete, timestamp: 0x60e95.
10:25:59 Waiting for Next Logical Log File to be Freed |
