[OpenSIPS-Users] core dump in dialog
Bogdan-Andrei Iancu
bogdan at opensips.org
Tue Oct 30 17:21:22 EDT 2018
Hi Jennifer,
OK, I'm glad you found a way to keep things under control - it seems the
crash may be more related to the profile replication. Nevertheless,
please post the full backtrace after installing the dbg symbols.
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
OpenSIPS Bootcamp 2018
http://opensips.org/training/OpenSIPS_Bootcamp_2018/
On 10/30/2018 09:38 PM, Jennifer Hashimoto wrote:
> Thanks Bogdan,
>
> I will check out the debug symbols. Per your advice that I don’t need
> to share both profiles and dialogs, I turned off profile sharing since
> I do want dialog replication for the master/master aspect of stuff.
> Turning off profile replication fixed the problem and now it is
> running smoothly.
>
> Jennifer
>
> ---------------------------------------------------
> Jennifer Akemi Hashimoto
> Caztel Communications
> jennifer.hashimoto at caztel.com <mailto:jennifer.hashimoto at caztel.com>
> 905-836-5445
>
>> On Oct 26, 2018, at 2:29 AM, Bogdan-Andrei Iancu <bogdan at opensips.org
>> <mailto:bogdan at opensips.org>> wrote:
>>
>> Hi Jennifer,
>>
>> The crash seems to happen when checking the sharing tag used for
>> dialog replication. Maybe you could install the debug symbols (it is
>> a separate package) to get more information in the backtrace.
>>
>> As a note, if you want to only do sharing profiles, you do not have
>> to do dialog sharing also - the two (dialog sharing and profile
>> sharing) are completely independent.
>>
>> Best regards,
>> Bogdan-Andrei Iancu
>>
>> OpenSIPS Founder and Developer
>> http://www.opensips-solutions.com
>> OpenSIPS Bootcamp 2018
>> http://opensips.org/training/OpenSIPS_Bootcamp_2018/
>> On 10/24/2018 10:33 PM, Jennifer Hashimoto wrote:
>>> Hi guys,
>>> I’m getting ready to switch to using opensips 2.4 but as soon as i
>>> put traffic on it (like 3 calls, not a lot) it crashed.
>>> it happened twice - once when I didn’t have the dump cores on, so I
>>> turned it on, then started and it crashed almost immediately after that
>>>
>>> I thought maybe it was because it did a insert on my postgres DB
>>> which doesn’t happen every call,
>>>
>>> It’s really strange because when i try test calls it seems fine, but
>>> real traffic it doesn’t’ like :(
>>>
>>> I think it’s part of where i do lookups for limit like this
>>> tutorial:
>>> https://www.opensips.org/Documentation/Tutorials-ConcurrentCallsLimitation
>>>
>>>
>>> *version: opensips 2.4.2 (x86_64/linux)*
>>> flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC,
>>> F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
>>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN
>>> 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
>>> poll method support: poll, epoll, sigio_rt, select.
>>> main.c compiled on with gcc 4.9.2
>>>
>>> Here are some relevant bits from my opensips.cfg
>>>
>>>
>>> loadmodule "db_virtual.so"
>>> modparam("db_virtual", "db_urls", "define debdb FAILOVER")
>>> modparam("db_virtual", "db_urls", "postgres://opensips:
>>> <postgres://opensips>passwd at 192.168.5.65/opensips_cazcore")
>>> modparam("db_virtual", "db_urls", "postgres://opensips:
>>> <postgres://opensips>passwd at 192.168.6.65/opensips_cazcore")
>>>
>>> #### DIALOG module
>>> loadmodule "dialog.so"
>>> modparam("dialog", "dlg_match_mode", 1)
>>> modparam("dialog", "default_timeout", 21600) # 6 hours timeout
>>> modparam("dialog", "db_mode", 1)
>>> modparam("dialog", "db_update_period", 5)
>>>
>>> #modparam("dialog",
>>> "db_url","postgres://opensips:passwd@192.168.5.65/opensips_cazcore")
>>>
>>> modparam("dialog", "db_url", "virtual://debdb")
>>> modparam("dialog", "dialog_replication_cluster", 1)
>>> modparam("dialog", "profile_replication_cluster", 1)
>>> #tags for VIP handling in cluster
>>> modparam("dialog", "dlg_sharing_tag", "vipA=MY_VIPA")
>>> modparam("dialog", "dlg_sharing_tag", "vipB=MY_VIPB")
>>> # ani dnis to limit calls from ani or to dnis. /b means replicated
>>> modparam("dialog", "profiles_with_value", "dnis/b;ani/b")
>>>
>>> *#I called the lookup like this*
>>>
>>> route("limit_dnis",$var(limit_dnis),2);
>>> route("limit_ani",$var(limit_ani),2);
>>>
>>>
>>> *#The lookup route looks like this*
>>>
>>> route[limit_dnis]
>>> {
>>> # first add to the profile, just to avoid "test and set" false
>>> results
>>> set_dlg_profile("dnis/b","$param(1)");
>>>
>>>
>>> # do the actual test - see how many calls the user has so far
>>> get_profile_size("dnis/b","$param(1)","$var(calls)");
>>> xlog("$ft $Ri There are $var(calls) ongoing calls to $param(1),
>>> limit is $param(2)\n");
>>>
>>> # check within limit
>>> if( $var(calls)>$param(2) )
>>> {
>>> xlog("$ft $Ri LIMIT_DNIS [$var(calls)/$param(2)]: From:$fU
>>> To:$param(1) Callid:$ci\n");
>>> avp_db_query("insert into
>>> caztel.opensips_log(event,fulltext,linenr,digits,hostname)
>>> values ('LIMIT DNIS','$ft $Ri LIMIT_DNIS
>>> [$var(calls)/$param(2)]: From:$fU To:$param(1)
>>> Callid:$ci','$fU','$tU','MY_HOSTNAME')");
>>> send_reply("486","DNIS LIMIT");
>>> exit;
>>> # terminating this call will automatically remove the call from
>>> the profile
>>> }
>>>
>>> # call was added to the profile without exceeding the limit,
>>> simply continue
>>> }
>>>
>>>
>>> route[limit_ani]
>>> {
>>> # first add to the profile, just to avoid "test and set" false
>>> results
>>> set_dlg_profile("ani/b","$param(1)");
>>>
>>>
>>> # do the actual test - see how many calls the user has so far
>>> get_profile_size("ani/b","$param(1)","$var(calls_ani)");
>>> xlog("$ft $Ri User $param(1) has $var(calls_ani) ongoing calls
>>> so far, limit is $param(2)\n");
>>>
>>>
>>> # check within limit
>>> if( $var(calls_ani)>$param(2) )
>>> {
>>> xlog("$ft $Ri LIMIT_ANI [$var(calls_ani)/$param(2)]:
>>> From:$param(1) To:$tU Callid:$ci\n");
>>> avp_db_query("insert into
>>> caztel.opensips_log(event,fulltext,linenr,digits,hostname)
>>> values ('LIMIT ANI','$ft $Ri LIMIT_ANI
>>> [$var(calls_ani)/$param(2)]: From:$param(1) To:$tU
>>> Callid:$ci','$fU','$tU','MY_HOSTNAME')");
>>> send_reply("486", "ANI LIMIT\n");
>>> exit;
>>> # terminating this call will automatically remove the call from
>>> the profile
>>> }
>>>
>>> # call was added to the profile without exceeding the limit,
>>> simply continue
>>> }
>>>
>>>
>>> *from the log: *
>>>
>>> Oct 24 12:21:29 CazcoreA /usr/sbin/opensips[5690]: 20138040-1546
>>> 10.10.10.109 There are 1 ongoing calls to 491232412323, limit is 2
>>> Oct 24 12:21:29 CazcoreA /usr/sbin/opensips[5690]: 20138040-1546
>>> 10.10.10.109 User 4186880000 has 1 ongoing calls so far, limit is 2
>>> Oct 24 12:21:29 CazcoreA /usr/sbin/opensips[5690]:
>>> CRITICAL:core:sig_usr: segfault in process pid: 5690, id: 11
>>> Oct 24 12:21:29 CazcoreA /usr/sbin/opensips[5688]: 20138040-1546
>>> 10.10.10.109 Call from: caz5400. Outcarrierid = 24.
>>> sip:4186880000 at 10.10.10.118 -> sip:24491232412323 at 10.10.10.109:5060
>>> Oct 24 12:21:29 CazcoreA /usr/sbin/opensips[5688]: 20138040-1546
>>> 10.10.10.109 There are 2 ongoing calls to 491232412323, limit is 2
>>> Oct 24 12:21:29 CazcoreA /usr/sbin/opensips[5688]: 20138040-1546
>>> 10.10.10.109 User 4186880000 has 2 ongoing calls so far, limit is 2
>>> Oct 24 12:21:30 CazcoreA /usr/sbin/opensips[5686]: 20138040-1546
>>> 10.10.10.109 Call from: caz5400. Outcarrierid = 24.
>>> sip:4186880000 at 10.10.10.118 -> sip:24491232412323 at 10.10.10.109:5060
>>> Oct 24 12:21:30 CazcoreA /usr/sbin/opensips[5686]: 20138040-1546
>>> 10.10.10.109 There are 3 ongoing calls to 491232412323, limit is 2
>>> Oct 24 12:21:30 CazcoreA /usr/sbin/opensips[5686]: 20138040-1546
>>> 10.10.10.109 LIMIT_DNIS [3/2]: From:4186880000 To:491232412323
>>> Callid:146FDAEB-D6E811E8-A098C2A6-4A2230D0 at 10.10.10.118
>>> <mailto:146FDAEB-D6E811E8-A098C2A6-4A2230D0 at 10.10.10.118>
>>> Oct 24 12:21:30 CazcoreA /usr/sbin/opensips[5749]:
>>> CRITICAL:core:handle_worker: dead child 11 (EOF received), pid 5690
>>> Oct 24 12:21:32 CazcoreA /usr/sbin/opensips[5678]:
>>> INFO:core:handle_sigs: child process 5690 exited by a signal 11
>>> Oct 24 12:21:32 CazcoreA /usr/sbin/opensips[5678]:
>>> INFO:core:handle_sigs: core was generated
>>> Oct 24 12:21:32 CazcoreA /usr/sbin/opensips[5678]:
>>> INFO:core:handle_sigs: terminating due to SIGCHLD
>>>
>>> *backtrace*
>>>
>>> GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
>>> Copyright (C) 2014 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later
>>> <http://gnu.org/licenses/gpl.html>
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show
>>> copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-linux-gnu".
>>> Type "show configuration" for configuration details.
>>> For bug reporting instructions, please see:
>>> <http://www.gnu.org/software/gdb/bugs/>.
>>> Find the GDB manual and other documentation resources online at:
>>> <http://www.gnu.org/software/gdb/documentation/>.
>>> For help, type "help".
>>> Type "apropos word" to search for commands related to "word"...
>>> Reading symbols from /usr/sbin/opensips...(no debugging symbols
>>> found)...done.
>>> [New LWP 5690]
>>> [Thread debugging using libthread_db enabled]
>>> Using host libthread_db library
>>> "/lib/x86_64-linux-gnu/libthread_db.so.1".
>>> Core was generated by `/usr/sbin/opensips -P
>>> /run/opensips/opensips.pid -f /etc/opensips/opensips.cfg'.
>>> Program terminated with signal SIGSEGV, Segmentation fault.
>>> #0 0x00007f5afe73b35c in fetch_dlg_value () from
>>> /usr/lib/x86_64-linux-gnu/opensips/modules/dialog.so
>>> (gdb) bt full
>>> #0 0x00007f5afe73b35c in fetch_dlg_value () from
>>> /usr/lib/x86_64-linux-gnu/opensips/modules/dialog.so
>>> No symbol table info available.
>>> #1 0x00007f5afe78022b in get_shtag_state () from
>>> /usr/lib/x86_64-linux-gnu/opensips/modules/dialog.so
>>> No symbol table info available.
>>> #2 0x00007f5afe742a05 in get_profile_size () from
>>> /usr/lib/x86_64-linux-gnu/opensips/modules/dialog.so
>>> No symbol table info available.
>>> #3 0x00007f5afda28022 in lb_route () from
>>> /usr/lib/x86_64-linux-gnu/opensips/modules/load_balancer.so
>>> No symbol table info available.
>>> #4 0x00007f5afda1ef35 in ?? () from
>>> /usr/lib/x86_64-linux-gnu/opensips/modules/load_balancer.so
>>> No symbol table info available.
>>> #5 0x0000000000433585 in do_action ()
>>> No symbol table info available.
>>> #6 0x0000000000438390 in run_action_list ()
>>> No symbol table info available.
>>> #7 0x0000000000462f3c in eval_expr ()
>>> No symbol table info available.
>>> #8 0x0000000000462b89 in eval_expr ()
>>> No symbol table info available.
>>> #9 0x0000000000462b33 in eval_expr ()
>>> No symbol table info available.
>>> #10 0x0000000000433526 in do_action ()
>>> ---Type <return> to continue, or q <return> to quit---
>>> No symbol table info available.
>>> #11 0x0000000000438390 in run_action_list ()
>>> No symbol table info available.
>>> #12 0x00000000004384ba in ?? ()
>>> No symbol table info available.
>>> #13 0x000000000043594b in do_action ()
>>> No symbol table info available.
>>> #14 0x0000000000438390 in run_action_list ()
>>> No symbol table info available.
>>> #15 0x0000000000436b3c in do_action ()
>>> No symbol table info available.
>>> #16 0x0000000000438390 in run_action_list ()
>>> No symbol table info available.
>>> #17 0x0000000000438648 in run_top_route ()
>>> No symbol table info available.
>>> #18 0x000000000043dd28 in receive_msg ()
>>> No symbol table info available.
>>> #19 0x000000000052e7d8 in ?? ()
>>> No symbol table info available.
>>> #20 0x0000000000519ad9 in udp_start_processes ()
>>> No symbol table info available.
>>> ---Type <return> to continue, or q <return> to quit---
>>> #21 0x000000000041c5a6 in main ()
>>> No symbol table info available.
>>> (gdb)
>>> ---------------------------------------------------
>>> Jennifer Akemi Hashimoto
>>> Caztel Communications
>>> jennifer.hashimoto at caztel.com <mailto:jennifer.hashimoto at caztel.com>
>>> 905-836-5445
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opensips.org
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20181030/922b3f40/attachment-0001.html>
More information about the Users
mailing list