[OpenSIPS-Users] opensips 2.3 latest crashing
Tito Cumpen
tito at xsvoce.com
Wed Aug 9 04:24:00 EDT 2017
Here is the
bt after installing all recommended debug infos
I am not sure why the debug info ssl version is asking for a previous one.
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/opensips...done.
[New LWP 984]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/sbin/opensips -P /var/run/opensips.pid -f
/etc/opensips/opensips.cfg -w /etc/o'.
Program terminated with signal 6, Aborted.
#0 0x00007f4c2e4b41d7 in __GI_raise (sig=sig at entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) bt full
#0 0x00007f4c2e4b41d7 in __GI_raise (sig=sig at entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
resultvar = 0
pid = 984
selftid = 984
#1 0x00007f4c2e4b58c8 in __GI_abort () at abort.c:90
save_stage = 2
act = {__sigaction_handler = {sa_handler = 0x7f4c24d49d81
<__FUNCTION__.8131>, sa_sigaction = 0x7f4c24d49d81 <__FUNCTION__.8131>},
sa_mask = {__val = {3472120737606682963, 139965138952436, 104,
140720825224720, 139965030355629, 140720825224768, 5,
9038976, 10292044412622288640, 140720825224896, 5234904, 5472328, 5444843,
139965169110616, 0, 140720825224752}},
sa_flags = 746622792, sa_restorer = 0x89ec80 <mem_lock>}
sigs = {__val = {32, 0 <repeats 15 times>}}
#2 0x00007f4c24d3a63c in shm_free (_p=<optimized out>) at
../tm/../../mem/shm_mem.h:570
No locals.
#3 free_extra_array (tags=<optimized out>, array=<optimized out>,
tags_len=<optimized out>) at acc_logic.c:198
i = <optimized out>
#4 free_acc_ctx (ctx=0x7f4c2c7344b0) at acc_logic.c:220
i = <optimized out>
#5 tm_free_acc_ctx (param=0x7f4c2c7344b0) at acc_logic.c:260
param = 0x7f4c2c7344b0
ctx = 0x7f4c2c7344b0
#6 0x00007f4c2b33f693 in empty_tmcb_list (head=head at entry=0x7f4c2c7239e0)
at t_hooks.c:53
cbp = 0x0
cbp_tmp = 0x7f4c2c80a290
#7 0x00007f4c2b30d9bf in free_cell (dead_cell=0x7f4c2c723970) at
h_table.c:127
b = <optimized out>
i = <optimized out>
rpl = <optimized out>
tt = <optimized out>
foo = <optimized out>
p = <optimized out>
#8 0x00007f4c2b310087 in free_hash_table () at h_table.c:383
p_cell = <optimized out>
tmp_cell = 0x0
i = 13944
#9 0x00007f4c2b30cf75 in tm_shutdown () at t_funcs.c:89
__FUNCTION__ = "tm_shutdown"
#10 0x0000000000501b0d in destroy_modules () at sr_module.c:527
t = 0x7f4c2e289900
foo = 0x7f4c2e289820
__FUNCTION__ = "destroy_modules"
#11 0x00000000004fc203 in cleanup (show_status=show_status at entry=1) at
main.c:319
__FUNCTION__ = "cleanup"
#12 0x00000000004fcb03 in handle_sigs () at main.c:456
chld = <optimized out>
chld_status = 1
overall_status = 0
i = <optimized out>
do_exit = <optimized out>
__FUNCTION__ = "handle_sigs"
#13 0x000000000041c990 in main_loop () at main.c:722
startup_done = <optimized out>
chd_rank = 25
#14 main (argc=<optimized out>, argv=<optimized out>) at main.c:1283
cfg_stream = <optimized out>
c = <optimized out>
r = <optimized out>
tmp = 0x1 <Address 0x1 out of bounds>
tmp_len = <optimized out>
port = <optimized out>
proto = <optimized out>
protos_no = <optimized out>
options = 0x6317c8 "f:cCm:M:b:l:n:N:rRvdDFETSVhw:t:u:g:P:G:W:o:"
---Type <return> to continue, or q <return> to quit---
ret = -1
seed = 1199514634
rfd = <optimized out>
__FUNCTION__ = "main"
(gdb)
On Wed, Aug 9, 2017 at 1:16 AM, Tito Cumpen <tito at xsvoce.com> wrote:
> here is the backtrace
>
>
> *gdb /usr/sbin/opensips core.984 *
> *GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-94.el7*
> *Copyright (C) 2013 Free Software Foundation, Inc.*
> *License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html <http://gnu.org/licenses/gpl.html>>*
> *This is free software: you are free to change and redistribute it.*
> *There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"*
> *and "show warranty" for details.*
> *This GDB was configured as "x86_64-redhat-linux-gnu".*
> *For bug reporting instructions, please see:*
> *<http://www.gnu.org/software/gdb/bugs/
> <http://www.gnu.org/software/gdb/bugs/>>...*
> *Reading symbols from /usr/sbin/opensips...done.*
> *[New LWP 984]*
> *[Thread debugging using libthread_db enabled]*
> *Using host libthread_db library "/lib64/libthread_db.so.1".*
> *Core was generated by `/sbin/opensips -P /var/run/opensips.pid -f
> /etc/opensips/opensips.cfg -w /etc/o'.*
> *Program terminated with signal 6, Aborted.*
> *#0 0x00007f4c2e4b41d7 in __GI_raise (sig=sig at entry=6) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:56*
> *56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);*
> *Missing separate debuginfos, use: debuginfo-install
> cyrus-sasl-lib-2.1.26-20.el7_2.x86_64 keyutils-libs-1.5.8-3.el7.x86_64
> krb5-libs-1.14.1-27.el7_3.x86_64 libcom_err-1.42.9-9.el7.x86_64
> libcurl-7.29.0-35.el7.centos.x86_64 libgcc-4.8.5-11.el7.x86_64
> libidn-1.28-4.el7.x86_64 libselinux-2.5-6.el7.x86_64
> libssh2-1.4.3-10.el7_2.1.x86_64 libstdc++-4.8.5-11.el7.x86_64
> libxml2-2.9.1-6.el7_2.3.x86_64 nspr-4.13.1-1.0.el7_3.x86_64
> nss-3.28.4-1.2.el7_3.x86_64 nss-util-3.28.4-1.0.el7_3.x86_64
> openldap-2.4.40-13.el7.x86_64 openssl-libs-1.0.1e-60.el7_3.1.x86_64
> pcre-8.32-15.el7_2.1.x86_64 xz-libs-5.2.2-1.el7.x86_64
> zlib-1.2.7-17.el7.x86_64*
> *(gdb) bt full*
> *#0 0x00007f4c2e4b41d7 in __GI_raise (sig=sig at entry=6) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:56*
> * resultvar = 0*
> * pid = 984*
> * selftid = 984*
> *#1 0x00007f4c2e4b58c8 in __GI_abort () at abort.c:90*
> * save_stage = 2*
> * act = {__sigaction_handler = {sa_handler = 0x7f4c24d49d81
> <__FUNCTION__.8131>, sa_sigaction = 0x7f4c24d49d81 <__FUNCTION__.8131>},
> sa_mask = {__val = {3472120737606682963, 139965138952436, 104, *
> * 140720825224720, 139965030355629, 140720825224768, 5,
> 9038976, 10292044412622288640, 140720825224896, 5234904, 5472328, 5444843,
> 139965169110616, 0, 140720825224752}}, *
> * sa_flags = 746622792, sa_restorer = 0x89ec80 <mem_lock>}*
> * sigs = {__val = {32, 0 <repeats 15 times>}}*
> *#2 0x00007f4c24d3a63c in shm_free (_p=<optimized out>) at
> ../tm/../../mem/shm_mem.h:570*
> *No locals.*
> *#3 free_extra_array (tags=<optimized out>, array=<optimized out>,
> tags_len=<optimized out>) at acc_logic.c:198*
> * i = <optimized out>*
> *#4 free_acc_ctx (ctx=0x7f4c2c7344b0) at acc_logic.c:220*
> * i = <optimized out>*
> *#5 tm_free_acc_ctx (param=0x7f4c2c7344b0) at acc_logic.c:260*
> * param = 0x7f4c2c7344b0*
> * ctx = 0x7f4c2c7344b0*
> *#6 0x00007f4c2b33f693 in empty_tmcb_list
> (head=head at entry=0x7f4c2c7239e0) at t_hooks.c:53*
> * cbp = 0x0*
> * cbp_tmp = 0x7f4c2c80a290*
> *#7 0x00007f4c2b30d9bf in free_cell (dead_cell=0x7f4c2c723970) at
> h_table.c:127*
> * b = <optimized out>*
> * i = <optimized out>*
> * rpl = <optimized out>*
> * tt = <optimized out>*
> * foo = <optimized out>*
> * p = <optimized out>*
> *#8 0x00007f4c2b310087 in free_hash_table () at h_table.c:383*
> * p_cell = <optimized out>*
> * tmp_cell = 0x0*
> * i = 13944*
> *#9 0x00007f4c2b30cf75 in tm_shutdown () at t_funcs.c:89*
> * __FUNCTION__ = "tm_shutdown"*
> *#10 0x0000000000501b0d in destroy_modules () at sr_module.c:527*
> * t = 0x7f4c2e289900*
> * foo = 0x7f4c2e289820*
> * __FUNCTION__ = "destroy_modules"*
> *#11 0x00000000004fc203 in cleanup (show_status=show_status at entry=1) at
> main.c:319*
> * __FUNCTION__ = "cleanup"*
> *#12 0x00000000004fcb03 in handle_sigs () at main.c:456*
> * chld = <optimized out>*
> * chld_status = 1*
> * overall_status = 0*
> * i = <optimized out>*
> * do_exit = <optimized out>*
> * __FUNCTION__ = "handle_sigs"*
> *#13 0x000000000041c990 in main_loop () at main.c:722*
> * startup_done = <optimized out>*
> * chd_rank = 25*
> *#14 main (argc=<optimized out>, argv=<optimized out>) at main.c:1283*
> * cfg_stream = <optimized out>*
> * c = <optimized out>*
> * r = <optimized out>*
> * tmp = 0x1 <Address 0x1 out of bounds>*
> * tmp_len = <optimized out>*
> * port = <optimized out>*
> * proto = <optimized out>*
> * protos_no = <optimized out>*
> * options = 0x6317c8 "f:cCm:M:b:l:n:N:rRvdDFETSVhw:t:u:g:P:G:W:o:"*
> *---Type <return> to continue, or q <return> to quit---*
> * ret = -1*
> * seed = 1199514634*
> * rfd = <optimized out>*
> * __FUNCTION__ = "main"*
>
>
> On Wed, Aug 9, 2017 at 12:30 AM, Tito Cumpen <tito at xsvoce.com> wrote:
>
>> Liviu,
>>
>>
>> I am also seeing this
>>
>> : ERROR:proto_wss:tls_print_errstack: TLS errstack:
>> error:0606E06F:digital envelope routines:EVP_MD_CTX_copy_ex:input not
>> initialized
>>
>> eventually it crashes. I think there may be an issue with tls. I am using
>> OpenSSL 1.0.2l 25 May 2017
>>
>>
>> Any idea what this could be ?
>>
>>
>>
>>
>> On Fri, Aug 4, 2017 at 1:53 AM, Tito Cumpen <tito at xsvoce.com> wrote:
>>
>>> Liviu,
>>>
>>>
>>> I am not sure if it's the same issue. I disabled all mongo related
>>> modules from loading and I am still getting some crashes with the following
>>> error:
>>>
>>> Aug 4 08:00:56 cloud-server-09 /sbin/opensips[6630]:
>>> ERROR:proto_wss:tls_blocking_write: TLS failed to send data
>>> Aug 4 08:00:56 cloud-server-09 /sbin/opensips[6630]:
>>> ERROR:proto_wss:tls_write: TLS connection to someip:50803 write failed
>>> Aug 4 08:00:56 cloud-server-09 /sbin/opensips[6630]:
>>> ERROR:proto_wss:tls_write: TLS write error:
>>> Aug 4 08:00:56 cloud-server-09 /sbin/opensips[6630]:
>>> ERROR:proto_wss:tls_blocking_write: TLS failed to send data
>>> Aug 4 08:01:56 cloud-server-09 /sbin/opensips[6630]:
>>> CRITICAL:core:sig_alarm_abort: BUG - shutdown timeout triggered, dying...
>>>
>>>
>>>
>>>
>>> On Thu, Aug 3, 2017 at 1:09 AM, Liviu Chircu <liviu at opensips.org> wrote:
>>>
>>>> Hi, Tito!
>>>>
>>>> It seems there are quite a few issues with the new cachedb_mongodb,
>>>> reported here [1]. On a happy note, most of them have also been fixed in
>>>> that pull request, and a merge is pending soon.
>>>>
>>>> Provided that you are not using tls_mgm, case in which you'd likely
>>>> need to recompile your libmongoc with "--disable-automatic-init-and-cleanup",
>>>> the upcoming merge should fix your crashes.
>>>>
>>>> Best regards,
>>>>
>>>> [1]: https://github.com/OpenSIPS/opensips/pull/1163
>>>>
>>>> Liviu Chircu
>>>> OpenSIPS Developerhttp://www.opensips-solutions.com
>>>>
>>>> On 03.08.2017 11:03, Tito Cumpen wrote:
>>>>
>>>> I am getting crashes on a daily basis with the following message
>>>>
>>>> Aug 3 07:42:52 /sbin/opensips[7681]: CRITICAL:core:receive_fd: EOF on
>>>> 106
>>>> Aug 3 07:42:52 /sbin/opensips[7641]: NOTICE:presence:destroy: destroy
>>>> module ...
>>>> Aug 3 07:42:52 /sbin/opensips[7641]: NOTICE:cachedb_mongodb:destroy:
>>>> destroy module cachedb_mongodb ...
>>>>
>>>>
>>>> I am running opensips with the following memory flags -M 1024 -m 256
>>>>
>>>> I am not seeing any core dumps being generated either. Anyone have any
>>>> ideas?
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opensips.org
>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20170809/88c7a94e/attachment-0001.html>
More information about the Users
mailing list