[OpenSIPS-Users] OpenSIPS crashed after "out of pkg memory"
Bogdan-Andrei Iancu
bogdan at opensips.org
Thu Sep 4 20:10:55 CEST 2014
Hi Kevin,
Any chance to get the mem dump after "waiting" ?
Thanks & Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
On 04.09.2014 13:09, Kevin Mathy wrote:
> Hi Bogdan,
>
> So, I've faced the problem once again, and followed the "Determining
> the cause" part of the link you gave me.
> I've waited more than 20 minutes (near an hour), and re-sent some
> voice trafic to the problematic opensips server : memory errors just
> came after that.
>
> So, I think it's a memory leak, no ? Now, what can I do to solve this ?
>
> Thanks for your help,
>
> Kevin
>
>
>
> *
> Bien cordialement,
> Best Regards,
>
> **Kevin MATHY* |**Ingénieur VoIP
> *
> *
>
>
> 2014-08-20 17:22 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org
> <mailto:bogdan at opensips.org>>:
>
> Hi Kevin,
>
> So it is more a mem leak than a crash. Any chances to get a
> memdump after memory is exhausted ? See:
> http://www.opensips.org/Documentation/TroubleShooting-OutOfMem
>
> Regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developer
> http://www.opensips-solutions.com
>
> On 20.08.2014 15:35, Kevin Mathy wrote:
>> Hi Bogdan,
>>
>> In fact, opensips didn't crashed : we got the previous error
>> messages in the logs, but then we switched our voice traffic to
>> another server, and opensips on the 1st one never crashed... We
>> restarted the service in the following night, to be sure to make
>> it rework well.
>>
>> So, is there a way in this case to get some backtrace, even if it
>> didn't crashed ? Maybe something generated at startup ?
>>
>> Thanks for your help,
>>
>> Kevin
>>
>> *
>> Bien cordialement,
>> Best Regards,
>>
>> **Kevin MATHY* |**Ingénieur VoIP
>> *
>> *
>>
>>
>> 2014-08-19 20:34 GMT+02:00 Bogdan-Andrei Iancu
>> <bogdan at opensips.org <mailto:bogdan at opensips.org>>:
>>
>> Hi Kevin,
>>
>> Try to extract at least a backtrace, to see what kind of
>> problem we are facing. Then we can decide if a mem dump is
>> required or not.
>>
>> Thanks,
>>
>> Bogdan-Andrei Iancu
>> OpenSIPS Founder and Developer
>> http://www.opensips-solutions.com
>>
>> On 19.08.2014 12:03, Kevin Mathy wrote:
>>> Hi Bogdan,
>>>
>>> Coming back to you after some holidays, and unfortunately,
>>> new Opensips crashes...
>>> Some weeks ago, we upgraded our servers to OpenSIPS 1.11,
>>> but we faced a new crash, always with the "out of pkg
>>> memory" error, last week.
>>>
>>> Here is the exact version :
>>>
>>> root at asbc2:/home/kemathy# opensips -V
>>> version: opensips 1.11.2-notls (x86_64/linux)
>>>
>>> flags: STATS: On, USE_IPV6, USE_TCP, DISABLE_NAGLE,
>>> USE_MCAST, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC,
>>> FAST_LOCK-ADAPTIVE_WAIT
>>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144,
>>> MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
>>> poll method support: poll, epoll_lt, epoll_et, sigio_rt,
>>> select.
>>> git revision: d41eb41
>>> main.c compiled on 19:23:35 Jul 23 2014 with gcc 4.7
>>>
>>>
>>>
>>> Here are an extract of logs we get at the crash-time (I
>>> masked phone numbers and IPs for security purposes) :
>>>
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:parse_contacts: failed to parse contact
>>> parameters
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:contact_parser: failed to parse contacts
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:parse_contact: failed to parse contact
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:pv_get_contact_body: failed to parse contact hdr
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> 90tzz16g at pbx -- New REGISTER received from
>>> 188.94.XXX.XXX with Contact : -- IP 188.94.XXX.XXX --
>>> PORT 5060
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> 90tzz16g at pbx -- REGISTER Failed because of : No
>>> Credentials -- From 188.94.XXX.XXX
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:build_res_buf_from_sip_req: out of pkg
>>> memory; needs 526
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:sl:sl_send_reply_helper: response building failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:signaling:sig_send_reply_mod: failed to send reply
>>> with sl module
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:auth:challenge: failed to send the response
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25606]:
>>> iy4l7y1r at pbx -- New REGISTER received from
>>> 188.94.XXX.XXX with Contact :
>>> <sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0>
>>> <mailto:sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0>
>>> -- IP 188.94.XXX.XXX -- PORT 5060
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25606]:
>>> iy4l7y1r at pbx -- REGISTER Failed because of : No
>>> Credentials -- From 188.94.XXX.XXX
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> iy4l7y1r at pbx -- New REGISTER received from
>>> 188.94.XXX.XXX with Contact :
>>> <sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0>
>>> <mailto:sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0>
>>> -- IP 188.94.XXX.XXX -- PORT 5060
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:core:db_allocate_rows: no memory left
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:db_mysql:db_mysql_convert_result: error while
>>> converting rows
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:db_mysql:db_mysql_store_result: error while
>>> converting result
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:core:db_do_query: error while storing result for
>>> query [select
>>> contact,expires,q,callid,cseq,flags,cflags,user_agent,received,path,socket,methods,last_modified,sip_instance,attr
>>> from location where username='0326xxxxxx' order by q]
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:usrloc:db_load_urecord: db_query failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> CRITICAL:db_mysql:wrapper_single_mysql_stmt_execute:
>>> driver error (1062): Duplicate entry
>>> '0326xxxxxx--sip:0326xxxxxx at 188.94.XX.XXX:5060;transport=udp;line
>>> <mailto:0326xxxxxx--sip:0326xxxxxx at 188.94.XX.XXX:5060;transport=udp;line>'
>>> for key 'account_contact_idx'
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:usrloc:db_insert_ucontact: inserting contact in db
>>> failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:usrloc:insert_ucontact: failed to insert in database
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:core:build_res_buf_from_sip_req: out of pkg
>>> memory; needs 373
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:sl:sl_send_reply_helper: response building failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:signaling:sig_send_reply_mod: failed to send reply
>>> with sl module
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:options:opt_reply: failed to send 200 via send_reply
>>>
>>>
>>> Unfortunately, we don't have the memory dump :-s ...
>>>
>>> So, I don't know if there's something we can try to avoid
>>> those errors to happen again; maybe we made something wrong
>>> in our configuration file ?
>>> We could provide you anything you want to help us find
>>> where's the problem.
>>>
>>> Thanks a lot for your help,
>>>
>>> Kevin
>>>
>>> *
>>> Bien cordialement,
>>> Best Regards,
>>>
>>> **Kevin MATHY* |**Ingénieur VoIP
>>> *
>>> *
>>>
>>>
>>> 2014-07-08 8:43 GMT+02:00 Kevin Mathy <k.mathy at hexanet.fr
>>> <mailto:k.mathy at hexanet.fr>>:
>>>
>>> Hi Bogdan,
>>>
>>> Sounds really good :-) ! I'm going to check if there's
>>> no configuration change between 1.9's and 1.11's modules
>>> version, and then will upgrade our version !
>>>
>>> I hope this will correct our problem ;-)
>>>
>>> Thanks for your help,
>>>
>>> Kevin
>>>
>>>
>>>
>>> *
>>> Bien cordialement,
>>> Best Regards,
>>>
>>> **Kevin MATHY* |**Ingénieur VoIP
>>> *
>>> *
>>>
>>>
>>> 2014-07-07 18:40 GMT+02:00 Bogdan-Andrei Iancu
>>> <bogdan at opensips.org <mailto:bogdan at opensips.org>>:
>>>
>>> Hi Kevin,
>>>
>>> OK, found the problem. As expected, this was fixed
>>> in 1.11, but not ported to 1.9 as this version is
>>> not maintained any more. Here is the link to the fix
>>> for your crash:
>>>
>>> https://github.com/OpenSIPS/opensips/commit/eef59c1d7b47ecc35438c01e7dcbdfeea903115a#diff-30710a8823b167e0aae5a14bb4878b75
>>>
>>>
>>>
>>> Regards,
>>>
>>> Bogdan-Andrei Iancu
>>> OpenSIPS Founder and Developer
>>> http://www.opensips-solutions.com
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20140904/c0ce4f68/attachment-0001.htm>
More information about the Users
mailing list