[OpenSIPS-Users] Command "opensipsctl dialplan reload" randomly hangs

Bogdan-Andrei Iancu bogdan at voice-system.ro
Mon Aug 2 13:37:09 CEST 2010 

Hi Dan,

you need to print those values from GDB when attached to a blocked FIFO 
process. So, wait again to block, attach with GDB, go to frame 2 an d 
print the values.

BTW, are you using mi_xmlrpc in the same time also ?

Regards,
Bogdan

DanB wrote:
> Hey Bogdan,
>
> Do I need to wait for the crash to print the values or I can do it any time?
> In the logs I could not spot anything.
>
> Ta,
> DanB
>
> On Sun, Aug 1, 2010 at 7:30 PM, Bogdan-Andrei Iancu
> <bogdan at voice-system.ro> wrote:
>   
>> Hi Dan,
>>
>> Interesting....the BT shows that the reloader process is blocked waiting
>> for ongoing reader processes to finish....
>>
>> Could you print in gdb the values:  *reload_flag and *data_refcnt  ?
>>
>> Also, please check the logs to be sure that you do not have any child
>> proc died .....
>>
>> Regards,
>> Bogdan
>>
>> DanB wrote:
>>     
>>> Hi Bogdan,
>>>
>>> After so long time I finally got a debug for you (running in debug 5).
>>> As higher the debug is as smaller the chances are to happen. Running
>>> in debug 6 we could never reproduce it (even in debug 5 it took some
>>> time). If we run it in debug 2 we get the hang every few days.
>>> Let me know if you need additional traces.
>>>
>>>
>>> Ta,
>>> DanB
>>>
>>>
>>>
>>> ############################################### Start BT
>>> #######################################
>>>
>>>
>>>       
>>>> sip1:~# opensipsctl ps|grep FIFO
>>>> Process::  ID=13 PID=3838 Type=MI FIFO
>>>> sip1:~# opensipsctl ps|grep FIFO
>>>> Process::  ID=13 PID=3838 Type=MI FIFO
>>>> sip1:~# opensipsctl dialplan reload
>>>> Terminated
>>>> sip1:~# /etc/init.d/
>>>> sip1:~# gdb /usr/sbin/opensips 3838
>>>> GNU gdb 6.8-debian
>>>> Copyright (C) 2008 Free Software Foundation, Inc.
>>>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>>>> This is free software: you are free to change and redistribute it.
>>>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>>>> and "show warranty" for details.
>>>> This GDB was configured as "x86_64-linux-gnu"...
>>>> (no debugging symbols found)
>>>> Attaching to program: /usr/sbin/opensips, process 3838
>>>> Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib/libdl.so.2
>>>> Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib/libresolv.so.2
>>>> Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib/libc.so.6
>>>> Reading symbols from /lib/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib64/ld-linux-x86-64.so.2
>>>> Reading symbols from /usr/lib/opensips/modules/options.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/options.so
>>>> Reading symbols from /usr/lib/opensips/modules/sl.so...
>>>> (no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/sl.so
>>>> Reading symbols from /usr/lib/opensips/modules/tm.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/tm.so
>>>> Reading symbols from /usr/lib/opensips/modules/signaling.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/signaling.so
>>>> Reading symbols from /usr/lib/opensips/modules/db_mysql.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/db_mysql.so
>>>> Reading symbols from /usr/lib/libmysqlclient.so.15...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libmysqlclient.so.15
>>>> Reading symbols from /lib/libpthread.so.0...
>>>> (no debugging symbols found)...done.
>>>> [Thread debugging using libthread_db enabled]
>>>> [New Thread 0x7ffac71226e0 (LWP 3838)]
>>>> Loaded symbols for /lib/libpthread.so.0
>>>> Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib/libcrypt.so.1
>>>> Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib/libnsl.so.1
>>>> Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib/libm.so.6
>>>> Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libz.so.1
>>>> Reading symbols from /usr/lib/opensips/modules/rr.so...
>>>> (no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/rr.so
>>>> Reading symbols from /usr/lib/opensips/modules/maxfwd.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/maxfwd.so
>>>> Reading symbols from /usr/lib/opensips/modules/usrloc.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/usrloc.so
>>>> Reading symbols from /usr/lib/opensips/modules/textops.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/textops.so
>>>> Reading symbols from /usr/lib/opensips/modules/mi_fifo.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/mi_fifo.so
>>>> Reading symbols from /usr/lib/opensips/modules/mi_datagram.so...
>>>> (no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/mi_datagram.so
>>>> Reading symbols from /usr/lib/opensips/modules/uri.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/uri.so
>>>> Reading symbols from /usr/lib/opensips/modules/xlog.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/xlog.so
>>>> Reading symbols from /usr/lib/opensips/modules/acc.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/acc.so
>>>> Reading symbols from /usr/lib/opensips/modules/dialog.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/dialog.so
>>>> Reading symbols from /usr/lib/opensips/modules/mediaproxy.so...
>>>> (no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/mediaproxy.so
>>>> Reading symbols from /usr/lib/opensips/modules/nathelper.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/nathelper.so
>>>> Reading symbols from /usr/lib/opensips/modules/dialplan.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/dialplan.so
>>>> Reading symbols from /usr/lib/opensips/modules/drouting.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/drouting.so
>>>> Reading symbols from /usr/lib/opensips/modules/permissions.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/permissions.so
>>>> Reading symbols from /usr/lib/opensips/modules/aaa_radius.so...
>>>> (no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/aaa_radius.so
>>>> Reading symbols from /usr/lib/libradiusclient-ng.so.2...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libradiusclient-ng.so.2
>>>> Reading symbols from /usr/lib/opensips/modules/auth.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/auth.so
>>>> Reading symbols from /usr/lib/opensips/modules/avpops.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/avpops.so
>>>> Reading symbols from /usr/lib/opensips/modules/uac.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/uac.so
>>>> Reading symbols from /usr/lib/opensips/modules/mi_xmlrpc.so...
>>>> (no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/opensips/modules/mi_xmlrpc.so
>>>> Reading symbols from /usr/lib/libxmlrpc_server.so.3...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libxmlrpc_server.so.3
>>>> Reading symbols from /usr/lib/libxmlrpc.so.3...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libxmlrpc.so.3
>>>> Reading symbols from /usr/lib/libxmlrpc_util.so.3...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libxmlrpc_util.so.3
>>>> Reading symbols from /usr/lib/libxmlrpc_xmlparse.so.3...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libxmlrpc_xmlparse.so.3
>>>> Reading symbols from /usr/lib/libxmlrpc_xmltok.so.3...
>>>> (no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libxmlrpc_xmltok.so.3
>>>> Reading symbols from /usr/lib/libnss_db.so.2...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libnss_db.so.2
>>>> Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
>>>> Loaded symbols for /lib/libnss_files.so.2
>>>> Reading symbols from /usr/lib/libdb-4.6.so...(no debugging symbols found)...done.
>>>> Loaded symbols for /usr/lib/libdb-4.6.so
>>>> (no debugging symbols found)
>>>> 0x00007ffac6843fc0 in nanosleep () from /lib/libc.so.6
>>>> (gdb) bt
>>>> #0  0x00007ffac6843fc0 in nanosleep () from /lib/libc.so.6
>>>> #1  0x00007ffac686f4c4 in usleep () from /lib/libc.so.6
>>>> #2  0x00007ffac334528a in dp_load_db () from /usr/lib/opensips/modules/dialplan.so
>>>> #3  0x00007ffac3340c0a in ?? () from /usr/lib/opensips/modules/dialplan.so
>>>> #4  0x00007ffac43e6e19 in mi_fifo_server () from /usr/lib/opensips/modules/mi_fifo.so
>>>> #5  0x00007ffac43e7d70 in ?? () from /usr/lib/opensips/modules/mi_fifo.so
>>>> #6  0x000000000045de29 in start_module_procs ()
>>>> #7  0x000000000042432e in main ()
>>>> (gdb) quit
>>>> The program is running.  Quit anyway (and detach it)? (y or n) /etc/init.d      op/y
>>>> Please answer y or n.
>>>> The program is running.  Quit anyway (and detach it)? (y or n) y
>>>> Quitting: Quit
>>>>
>>>>         
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opensips.org
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>>>
>>>       
>> --
>> Bogdan-Andrei Iancu
>> OpenSIPS Bootcamp
>> 20 - 24 September 2010, Frankfurt, Germany
>> www.voice-system.ro
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>>     
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>   


-- 
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro

More information about the Users mailing list