[OpenSIPS-Users] opensips crash when doing load test

Bogdan-Andrei Iancu bogdan at opensips.org
Tue Oct 4 17:52:13 CEST 2016


Could you also print r_buf and*r_buf  ?

Regards,

Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com

On 04.10.2016 16:08, Ramachandran, Agalya (Contractor) wrote:
>
> Hi Bogdan,
>
> I have printed the “t” variable value. It says “$1 = (struct cell *) 
> 0x0”.
>
> Is there any link available to upload/share the core files? Because 
> size of the core file 45KB and couldn’t attach in e-mail.
>
> (gdb) frame 0
>
> #0 final_response_handler (fr_tl=0x7fded2484110) at timer.c:383
>
> 383             if (is_invite(t) && should_cancel_branch(t, 
> r_buf->branch) ) {
>
> (gdb) list +
>
> 378 current_processing_ctx = my_ctx;
>
> 379             /* set the T context too */
>
> 380 set_t( t );
>
> 381
>
> 382             /* out-of-lock do the cancel I/O */
>
> 383             if (is_invite(t) && should_cancel_branch(t, 
> r_buf->branch) ) {
>
> 384 set_cancel_extra_hdrs( CANCEL_REASON_SIP_480, 
> sizeof(CANCEL_REASON_SIP_480)-1);
>
> 385 cancel_branch(t, r_buf->branch );
>
> 386 set_cancel_extra_hdrs( NULL, 0);
>
> 387             }
>
> (gdb) p t
>
> $1 = (struct cell *) 0x0
>
> Regards,
>
> Agalya
>
> *From:*Bogdan-Andrei Iancu [mailto:bogdan at opensips.org]
> *Sent:* Tuesday, October 04, 2016 6:44 AM
> *To:* Ramachandran, Agalya (Contractor) 
> <Agalya_Ramachandran at comcast.com>; OpenSIPS users mailling list 
> <users at lists.opensips.org>
> *Subject:* Re: opensips crash when doing load test
>
> Hello Agalya,
>
> In frame 0, could you print the "t" variable ? BTW, is there any way 
> to get access to the core file ?
>
> Best regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developer
> http://www.opensips-solutions.com
>
> On 30.09.2016 16:48, Ramachandran, Agalya (Contractor) wrote:
>
>     Hi Bogdan,
>
>     One more dump
>
>     #0 final_response_handler (fr_tl=0x7fded2484110) at timer.c:383
>
>     383 if (is_invite(t) && should_cancel_branch(t, r_buf->branch) ) {
>
>     Missing separate debuginfos, use: debuginfo-install
>     cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
>     glibc-2.17-106.el7_2.4.x86_64 keyutils-libs-1.5.8-3.el7.x86_64
>     krb5-libs-1.13.2-12.el7_2.x86_64 libcom_err-1.42.9-7.el7.x86_64
>     libcurl-7.29.0-25.el7.centos.x86_64 libidn-1.28-4.el7.x86_64
>     libselinux-2.2.2-6.el7.x86_64 libssh2-1.4.3-10.el7_2.1.x86_64
>     nspr-4.10.8-2.el7_1.x86_64 nss-3.19.1-19.el7_2.x86_64
>     nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
>     nss-util-3.19.1-9.el7_2.x86_64 openldap-2.4.40-9.el7_2.x86_64
>     openssl-libs-1.0.1e-51.el7_2.4.x86_64 pcre-8.32-15.el7.x86_64
>     xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64
>
>     (gdb) bt
>
>     #0 final_response_handler (fr_tl=0x7fded2484110) at timer.c:383
>
>     #1 timer_routine (ticks=<optimized out>, set=<optimized out>) at
>     timer.c:1066
>
>     #2 0x000000000049d3cd in handle_timer_job () at timer.c:597
>
>     #3 0x000000000059223e in handle_io (fm=<optimized out>,
>     fm=<optimized out>, fm=<optimized out>, idx=<optimized out>,
>     event_type=2)
>
>         at net/net_udp.c:265
>
>     #4 io_wait_loop_epoll (h=<optimized out>, t=<optimized out>,
>     repeat=<optimized out>) at net/../io_wait_loop.h:225
>
>     #5  udp_rcv_loop (si=si at entry=0x7fded841dcb0) at net/net_udp.c:308
>
>     #6 0x0000000000593e05 in udp_start_processes
>     (chd_rank=chd_rank at entry=0x84c570 <chd_rank.10940>,
>     startup_done=startup_done at entry=0x0)
>
>         at net/net_udp.c:448
>
>     #7 0x0000000000419b73 in main_loop () at main.c:736
>
>     #8  main (argc=<optimized out>, argv=<optimized out>) at main.c:1282
>
>     *From:* Ramachandran, Agalya (Contractor)
>     *Sent:* Friday, September 30, 2016 9:44 AM
>     *To:* 'Bogdan-Andrei Iancu' <bogdan at opensips.org>
>     <mailto:bogdan at opensips.org>; OpenSIPS users mailling list
>     <users at lists.opensips.org> <mailto:users at lists.opensips.org>
>     *Subject:* opensips crash when doing load test
>
>     Hi Bogdan,
>
>     I have taken 2.1.4 code base, added the below fixes on top of it
>
>     1.Fix potential starvation for the timer tasks.
>     66c337cc89a2b5c3e1bbd78c289371efcda21886
>
>     2.rest_client: Improve polling logic during TCP connect -
>     c40f2b057209b8078520abc95e5d1bf5e5a28db6
>
>     3.rest_client: Add "connect_poll_interval" module parameter -
>     f5abfb3dee649a14ae39ad74d7f04454f815748f
>
>     4.rest_client: Fix bogus condition
>     -3ba7be6692c6987faecb9c4c904035d5e7c0f916
>
>     When I try to perform load test on that code base, randomly am
>     seeing OpenSIPS is crashing.
>
>     Here is the full trace of the core file. Let me know if you need
>     any additional information or logs.
>
>     Am seeing crash is occurring at start_timer_extra_processes().
>
>     [root at poc-siplb-cmc-e-004 corefiles]# gdb /usr/local/sbin/opensips
>     core.opensips.sig11.3977
>
>     GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7
>
>     Copyright (C) 2013 Free Software Foundation, Inc.
>
>     License GPLv3+: GNU GPL version 3 or later
>     <http://gnu.org/licenses/gpl.html>
>
>     This is free software: you are free to change and redistribute it.
>
>     There is NO WARRANTY, to the extent permitted by law.  Type "show
>     copying"
>
>     and "show warranty" for details.
>
>     This GDB was configured as "x86_64-redhat-linux-gnu".
>
>     For bug reporting instructions, please see:
>
>     <http://www.gnu.org/software/gdb/bugs/>...
>
>     Reading symbols from /usr/local/sbin/opensips...done.
>
>     [New LWP 3977]
>
>     [Thread debugging using libthread_db enabled]
>
>     Using host libthread_db library "/lib64/libthread_db.so.1".
>
>     Core was generated by `/usr/local/sbin/opensips -P
>     /var/run/opensips.pid'.
>
>     Program terminated with signal 11, Segmentation fault.
>
>     #0  final_response_handler (fr_tl=0x7f55fcd82948) at timer.c:383
>
>     383 if (is_invite(t) && should_cancel_branch(t, r_buf->branch) ) {
>
>     Missing separate debuginfos, use: debuginfo-install
>     cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
>     glibc-2.17-106.el7_2.4.x86_64 keyutils-libs-1.5.8-3.el7.x86_64
>     krb5-libs-1.13.2-12.el7_2.x86_64 libcom_err-1.42.9-7.el7.x86_64
>     libcurl-7.29.0-25.el7.centos.x86_64 libidn-1.28-4.el7.x86_64
>     libselinux-2.2.2-6.el7.x86_64 libssh2-1.4.3-10.el7_2.1.x86_64
>     nspr-4.10.8-2.el7_1.x86_64 nss-3.19.1-19.el7_2.x86_64
>     nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
>     nss-util-3.19.1-9.el7_2.x86_64 openldap-2.4.40-9.el7_2.x86_64
>     openssl-libs-1.0.1e-51.el7_2.4.x86_64 pcre-8.32-15.el7.x86_64
>     xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64
>
>     (gdb) bt
>
>     #0 final_response_handler (fr_tl=0x7f55fcd82948) at timer.c:383
>
>     #1  timer_routine (ticks=<optimized out>, set=<optimized out>) at
>     timer.c:1066
>
>     #2 0x000000000049d03a in handle_timer_job () at timer.c:597
>
>     #3 start_timer_extra_processes (chd_rank=chd_rank at entry=0x84c570
>     <chd_rank.10940>) at timer.c:557
>
>     #4 0x0000000000419b9a in main_loop () at main.c:748
>
>     #5  main (argc=<optimized out>, argv=<optimized out>) at main.c:1282
>
>     Regards,
>
>     Agalya
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20161004/0f094db1/attachment-0001.htm>


More information about the Users mailing list