[OpenSIPS-Users] Command "opensipsctl dialplan reload" randomly hangs

[DanB]

Hey Bogdan,

Do I need to wait for the crash to print the values or I can do it any time?
In the logs I could not spot anything.

Ta,
DanB

On Sun, Aug 1, 2010 at 7:30 PM, Bogdan-Andrei Iancu
<bogdan at voice-system.ro> wrote:
> Hi Dan,
>
> Interesting....the BT shows that the reloader process is blocked waiting
> for ongoing reader processes to finish....
>
> Could you print in gdb the values:  *reload_flag and *data_refcnt  ?
>
> Also, please check the logs to be sure that you do not have any child
> proc died .....
>
> Regards,
> Bogdan
>
> DanB wrote:
>> Hi Bogdan,
>>
>> After so long time I finally got a debug for you (running in debug 5).
>> As higher the debug is as smaller the chances are to happen. Running
>> in debug 6 we could never reproduce it (even in debug 5 it took some
>> time). If we run it in debug 2 we get the hang every few days.
>> Let me know if you need additional traces.
>>
>>
>> Ta,
>> DanB
>>
>>
>>
>> ############################################### Start BT
>> #######################################
>>
>>
>>> sip1:~# opensipsctl ps|grep FIFO
>>> Process::  ID=13 PID=3838 Type=MI FIFO
>>> sip1:~# opensipsctl ps|grep FIFO
>>> Process::  ID=13 PID=3838 Type=MI FIFO
>>> sip1:~# opensipsctl dialplan reload
>>> Terminated
>>> sip1:~# /etc/init.d/
>>> sip1:~# gdb /usr/sbin/opensips 3838
>>> GNU gdb 6.8-debian
>>> Copyright (C) 2008 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-linux-gnu"...
>>> (no debugging symbols found)
>>> Attaching to program: /usr/sbin/opensips, process 3838
>>> Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
>>> Loaded symbols for /lib/libdl.so.2
>>> Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
>>> Loaded symbols for /lib/libresolv.so.2
>>> Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
>>> Loaded symbols for /lib/libc.so.6
>>> Reading symbols from /lib/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
>>> Loaded symbols for /lib64/ld-linux-x86-64.so.2
>>> Reading symbols from /usr/lib/opensips/modules/options.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/options.so
>>> Reading symbols from /usr/lib/opensips/modules/sl.so...
>>> (no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/sl.so
>>> Reading symbols from /usr/lib/opensips/modules/tm.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/tm.so
>>> Reading symbols from /usr/lib/opensips/modules/signaling.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/signaling.so
>>> Reading symbols from /usr/lib/opensips/modules/db_mysql.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/db_mysql.so
>>> Reading symbols from /usr/lib/libmysqlclient.so.15...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libmysqlclient.so.15
>>> Reading symbols from /lib/libpthread.so.0...
>>> (no debugging symbols found)...done.
>>> [Thread debugging using libthread_db enabled]
>>> [New Thread 0x7ffac71226e0 (LWP 3838)]
>>> Loaded symbols for /lib/libpthread.so.0
>>> Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done.
>>> Loaded symbols for /lib/libcrypt.so.1
>>> Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
>>> Loaded symbols for /lib/libnsl.so.1
>>> Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
>>> Loaded symbols for /lib/libm.so.6
>>> Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libz.so.1
>>> Reading symbols from /usr/lib/opensips/modules/rr.so...
>>> (no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/rr.so
>>> Reading symbols from /usr/lib/opensips/modules/maxfwd.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/maxfwd.so
>>> Reading symbols from /usr/lib/opensips/modules/usrloc.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/usrloc.so
>>> Reading symbols from /usr/lib/opensips/modules/textops.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/textops.so
>>> Reading symbols from /usr/lib/opensips/modules/mi_fifo.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/mi_fifo.so
>>> Reading symbols from /usr/lib/opensips/modules/mi_datagram.so...
>>> (no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/mi_datagram.so
>>> Reading symbols from /usr/lib/opensips/modules/uri.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/uri.so
>>> Reading symbols from /usr/lib/opensips/modules/xlog.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/xlog.so
>>> Reading symbols from /usr/lib/opensips/modules/acc.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/acc.so
>>> Reading symbols from /usr/lib/opensips/modules/dialog.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/dialog.so
>>> Reading symbols from /usr/lib/opensips/modules/mediaproxy.so...
>>> (no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/mediaproxy.so
>>> Reading symbols from /usr/lib/opensips/modules/nathelper.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/nathelper.so
>>> Reading symbols from /usr/lib/opensips/modules/dialplan.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/dialplan.so
>>> Reading symbols from /usr/lib/opensips/modules/drouting.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/drouting.so
>>> Reading symbols from /usr/lib/opensips/modules/permissions.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/permissions.so
>>> Reading symbols from /usr/lib/opensips/modules/aaa_radius.so...
>>> (no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/aaa_radius.so
>>> Reading symbols from /usr/lib/libradiusclient-ng.so.2...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libradiusclient-ng.so.2
>>> Reading symbols from /usr/lib/opensips/modules/auth.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/auth.so
>>> Reading symbols from /usr/lib/opensips/modules/avpops.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/avpops.so
>>> Reading symbols from /usr/lib/opensips/modules/uac.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/uac.so
>>> Reading symbols from /usr/lib/opensips/modules/mi_xmlrpc.so...
>>> (no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/opensips/modules/mi_xmlrpc.so
>>> Reading symbols from /usr/lib/libxmlrpc_server.so.3...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libxmlrpc_server.so.3
>>> Reading symbols from /usr/lib/libxmlrpc.so.3...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libxmlrpc.so.3
>>> Reading symbols from /usr/lib/libxmlrpc_util.so.3...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libxmlrpc_util.so.3
>>> Reading symbols from /usr/lib/libxmlrpc_xmlparse.so.3...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libxmlrpc_xmlparse.so.3
>>> Reading symbols from /usr/lib/libxmlrpc_xmltok.so.3...
>>> (no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libxmlrpc_xmltok.so.3
>>> Reading symbols from /usr/lib/libnss_db.so.2...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libnss_db.so.2
>>> Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
>>> Loaded symbols for /lib/libnss_files.so.2
>>> Reading symbols from /usr/lib/libdb-4.6.so...(no debugging symbols found)...done.
>>> Loaded symbols for /usr/lib/libdb-4.6.so
>>> (no debugging symbols found)
>>> 0x00007ffac6843fc0 in nanosleep () from /lib/libc.so.6
>>> (gdb) bt
>>> #0  0x00007ffac6843fc0 in nanosleep () from /lib/libc.so.6
>>> #1  0x00007ffac686f4c4 in usleep () from /lib/libc.so.6
>>> #2  0x00007ffac334528a in dp_load_db () from /usr/lib/opensips/modules/dialplan.so
>>> #3  0x00007ffac3340c0a in ?? () from /usr/lib/opensips/modules/dialplan.so
>>> #4  0x00007ffac43e6e19 in mi_fifo_server () from /usr/lib/opensips/modules/mi_fifo.so
>>> #5  0x00007ffac43e7d70 in ?? () from /usr/lib/opensips/modules/mi_fifo.so
>>> #6  0x000000000045de29 in start_module_procs ()
>>> #7  0x000000000042432e in main ()
>>> (gdb) quit
>>> The program is running.  Quit anyway (and detach it)? (y or n) /etc/init.d      op/y
>>> Please answer y or n.
>>> The program is running.  Quit anyway (and detach it)? (y or n) y
>>> Quitting: Quit
>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>>
>
>
> --
> Bogdan-Andrei Iancu
> OpenSIPS Bootcamp
> 20 - 24 September 2010, Frankfurt, Germany
> www.voice-system.ro
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>