It is expected behavior that when an instance of IDS is started
with the oninit -s command (or the onmonitor->Mode->Startup
equivalent), it will not start the btree scanner thread(s).
When converting an IDS instance from 11.70.xC1/2 to 11.70.xC3 or
higher, it goes through special conversion code. When this
special conversion code is complete, it marks the instance as
OnLine and starts the dbScheduler threads but it does not start
the btree scanner thread(s).
The above scenario in and of itself is not fatal. But because
it does not start the btree scanner threads, it also does not
initialize the necessary btree scanner structures in shared
memory. And because it does not initialize the btree scanner
shared memory structures, there are some side effects:
1) Because the btree scanner thread is responsible for removing
deleted index keys and keeping indices from becoming severely
bloated and therefore less efficient, this can lead to
performance problems.
2) A query of the sysmaster:sysshmhdr pseudo table directly
references the uninitialized btree cleaner memory structure and
will segv and crash the IDS instance (example stack is included
later in the description)
3) The onstat -C command will not work, reporting only "Changing
data structure forced command termination".
After the fact, any of these problems can avoided by starting
the instance before conversion with "oninit" or by taking the
instance offline after conversion (assuming started with oninit
-s) and bringing it back up with "oninit". Simply starting the
btree scanner thread(s) with onmode -C start will also crash the
IDS instance.
dbScheduler has a task called mon_profile which is enabled by
default to run every 4 hours. This task does a query of the
sysmaster:sysshmhdr pseudo table which will induce the crash.
The crash looks something like this:
20:55:10 Assert Failed: Exception Caught. Type: MT_EX_OS,
Context: mem
20:55:10 Who: Session(127, informix@, 0, 0x4ddf5088)
Thread(141, dbWorker2, 4ddb6e00, 20)
File: mtex.c Line: 416
20:55:10 Action: Please notify IBM Informix Technical Support.
20:55:10 Raw hex dump of stack located in
/home/informix/tmp/af.4756d8e.rawstk
20:55:10 Stack for thread: 141 dbWorker2
base: 0x000000005063f000
len: 135168
pc: 0x0000000001317a03
tos: 0x0000000001c2a030
state: running
vp: 20
afstack
afhandler
affail_interface
mt_ex_throw_sig
afsig_handler
<signal frame>
prshmhdr
pstread
pst_rsread
rsread
fmread
readseq_single
gettupl
scan_next
insert_next
doinsert
excommand
sq_execute
mi_exec
ph_execute
do_sensor
db_sch_worker
udrlm_clang_execute_internal
udrlm_clang_execute
udrlm_exec_routine
udr_execute
dbsched_start_udr
th_init_initgls
startup |
