[OpenSIPS-Users] Command "opensipsctl dialplan reload" randomly hangs

Sun Aug 1 19:30:10 CEST 2010

Hi Dan,

Interesting....the BT shows that the reloader process is blocked waiting 
for ongoing reader processes to finish....

Could you print in gdb the values:  *reload_flag and *data_refcnt  ?

Also, please check the logs to be sure that you do not have any child 
proc died .....

Regards,
Bogdan

DanB wrote:
> Hi Bogdan,
>
> After so long time I finally got a debug for you (running in debug 5).
> As higher the debug is as smaller the chances are to happen. Running
> in debug 6 we could never reproduce it (even in debug 5 it took some
> time). If we run it in debug 2 we get the hang every few days.
> Let me know if you need additional traces.
>
>
> Ta,
> DanB
>
>
>
> ############################################### Start BT
> #######################################
>
>   
>> sip1:~# opensipsctl ps|grep FIFO
>> Process::  ID=13 PID=3838 Type=MI FIFO
>> sip1:~# opensipsctl ps|grep FIFO
>> Process::  ID=13 PID=3838 Type=MI FIFO
>> sip1:~# opensipsctl dialplan reload
>> Terminated
>> sip1:~# /etc/init.d/
>> sip1:~# gdb /usr/sbin/opensips 3838
>> GNU gdb 6.8-debian
>> Copyright (C) 2008 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-linux-gnu"...
>> (no debugging symbols found)
>> Attaching to program: /usr/sbin/opensips, process 3838
>> Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
>> Loaded symbols for /lib/libdl.so.2
>> Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
>> Loaded symbols for /lib/libresolv.so.2
>> Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
>> Loaded symbols for /lib/libc.so.6
>> Reading symbols from /lib/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
>> Loaded symbols for /lib64/ld-linux-x86-64.so.2
>> Reading symbols from /usr/lib/opensips/modules/options.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/options.so
>> Reading symbols from /usr/lib/opensips/modules/sl.so...
>> (no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/sl.so
>> Reading symbols from /usr/lib/opensips/modules/tm.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/tm.so
>> Reading symbols from /usr/lib/opensips/modules/signaling.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/signaling.so
>> Reading symbols from /usr/lib/opensips/modules/db_mysql.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/db_mysql.so
>> Reading symbols from /usr/lib/libmysqlclient.so.15...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libmysqlclient.so.15
>> Reading symbols from /lib/libpthread.so.0...
>> (no debugging symbols found)...done.
>> [Thread debugging using libthread_db enabled]
>> [New Thread 0x7ffac71226e0 (LWP 3838)]
>> Loaded symbols for /lib/libpthread.so.0
>> Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done.
>> Loaded symbols for /lib/libcrypt.so.1
>> Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
>> Loaded symbols for /lib/libnsl.so.1
>> Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
>> Loaded symbols for /lib/libm.so.6
>> Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libz.so.1
>> Reading symbols from /usr/lib/opensips/modules/rr.so...
>> (no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/rr.so
>> Reading symbols from /usr/lib/opensips/modules/maxfwd.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/maxfwd.so
>> Reading symbols from /usr/lib/opensips/modules/usrloc.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/usrloc.so
>> Reading symbols from /usr/lib/opensips/modules/textops.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/textops.so
>> Reading symbols from /usr/lib/opensips/modules/mi_fifo.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/mi_fifo.so
>> Reading symbols from /usr/lib/opensips/modules/mi_datagram.so...
>> (no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/mi_datagram.so
>> Reading symbols from /usr/lib/opensips/modules/uri.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/uri.so
>> Reading symbols from /usr/lib/opensips/modules/xlog.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/xlog.so
>> Reading symbols from /usr/lib/opensips/modules/acc.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/acc.so
>> Reading symbols from /usr/lib/opensips/modules/dialog.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/dialog.so
>> Reading symbols from /usr/lib/opensips/modules/mediaproxy.so...
>> (no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/mediaproxy.so
>> Reading symbols from /usr/lib/opensips/modules/nathelper.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/nathelper.so
>> Reading symbols from /usr/lib/opensips/modules/dialplan.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/dialplan.so
>> Reading symbols from /usr/lib/opensips/modules/drouting.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/drouting.so
>> Reading symbols from /usr/lib/opensips/modules/permissions.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/permissions.so
>> Reading symbols from /usr/lib/opensips/modules/aaa_radius.so...
>> (no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/aaa_radius.so
>> Reading symbols from /usr/lib/libradiusclient-ng.so.2...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libradiusclient-ng.so.2
>> Reading symbols from /usr/lib/opensips/modules/auth.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/auth.so
>> Reading symbols from /usr/lib/opensips/modules/avpops.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/avpops.so
>> Reading symbols from /usr/lib/opensips/modules/uac.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/uac.so
>> Reading symbols from /usr/lib/opensips/modules/mi_xmlrpc.so...
>> (no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/opensips/modules/mi_xmlrpc.so
>> Reading symbols from /usr/lib/libxmlrpc_server.so.3...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libxmlrpc_server.so.3
>> Reading symbols from /usr/lib/libxmlrpc.so.3...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libxmlrpc.so.3
>> Reading symbols from /usr/lib/libxmlrpc_util.so.3...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libxmlrpc_util.so.3
>> Reading symbols from /usr/lib/libxmlrpc_xmlparse.so.3...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libxmlrpc_xmlparse.so.3
>> Reading symbols from /usr/lib/libxmlrpc_xmltok.so.3...
>> (no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libxmlrpc_xmltok.so.3
>> Reading symbols from /usr/lib/libnss_db.so.2...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libnss_db.so.2
>> Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
>> Loaded symbols for /lib/libnss_files.so.2
>> Reading symbols from /usr/lib/libdb-4.6.so...(no debugging symbols found)...done.
>> Loaded symbols for /usr/lib/libdb-4.6.so
>> (no debugging symbols found)
>> 0x00007ffac6843fc0 in nanosleep () from /lib/libc.so.6
>> (gdb) bt
>> #0  0x00007ffac6843fc0 in nanosleep () from /lib/libc.so.6
>> #1  0x00007ffac686f4c4 in usleep () from /lib/libc.so.6
>> #2  0x00007ffac334528a in dp_load_db () from /usr/lib/opensips/modules/dialplan.so
>> #3  0x00007ffac3340c0a in ?? () from /usr/lib/opensips/modules/dialplan.so
>> #4  0x00007ffac43e6e19 in mi_fifo_server () from /usr/lib/opensips/modules/mi_fifo.so
>> #5  0x00007ffac43e7d70 in ?? () from /usr/lib/opensips/modules/mi_fifo.so
>> #6  0x000000000045de29 in start_module_procs ()
>> #7  0x000000000042432e in main ()
>> (gdb) quit
>> The program is running.  Quit anyway (and detach it)? (y or n) /etc/init.d      op/y
>> Please answer y or n.
>> The program is running.  Quit anyway (and detach it)? (y or n) y
>> Quitting: Quit
>>     
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>   

-- 
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro