[OpenSIPS-Users] OpenSIPS crashed after "out of pkg memory"

Thu Sep 4 12:09:41 CEST 2014

Hi Bogdan,

So, I've faced the problem once again, and followed the "Determining the
cause" part of the link you gave me.
I've waited more than 20 minutes (near an hour), and re-sent some voice
trafic to the problematic opensips server : memory errors just came after
that.

So, I think it's a memory leak, no ? Now, what can I do to solve this ?

Thanks for your help,

Kevin

*Bien cordialement, Best Regards, **Kevin MATHY* | Ingénieur VoIP

2014-08-20 17:22 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:

>  Hi Kevin,
>
> So it is more a mem leak than a crash. Any chances to get a memdump after
> memory is exhausted ? See:
>     http://www.opensips.org/Documentation/TroubleShooting-OutOfMem
>
> Regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>
> On 20.08.2014 15:35, Kevin Mathy wrote:
>
> Hi Bogdan,
>
>  In fact, opensips didn't crashed : we got the previous error messages in
> the logs, but then we switched our voice traffic to another server, and
> opensips on the 1st one never crashed... We restarted the service in the
> following night, to be sure to make it rework well.
>
>  So, is there a way in this case to get some backtrace, even if it didn't
> crashed ? Maybe something generated at startup ?
>
>  Thanks for your help,
>
>  Kevin
>
>
> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>
>
>
> 2014-08-19 20:34 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:
>
>>  Hi Kevin,
>>
>> Try to extract at least a backtrace, to see what kind of problem we are
>> facing. Then we can decide if a mem dump is required or not.
>>
>> Thanks,
>>
>> Bogdan-Andrei Iancu
>> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>>
>>   On 19.08.2014 12:03, Kevin Mathy wrote:
>>
>>  Hi Bogdan,
>>
>>  Coming back to you after some holidays, and unfortunately, new Opensips
>> crashes...
>> Some weeks ago, we upgraded our servers to OpenSIPS 1.11, but we faced a
>> new crash, always with the "out of pkg memory" error, last week.
>>
>>  Here is the exact version :
>>
>>> root at asbc2:/home/kemathy# opensips -V
>>> version: opensips 1.11.2-notls (x86_64/linux)
>>>
>>> flags: STATS: On, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM,
>>> SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
>>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
>>> MAX_URI_SIZE 1024, BUF_SIZE 65535
>>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>>>  git revision: d41eb41
>>> main.c compiled on 19:23:35 Jul 23 2014 with gcc 4.7
>>
>>
>>
>>  Here are an extract of logs we get at the crash-time (I masked phone
>> numbers and IPs for security purposes) :
>>
>>  Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:parse_contacts: failed to parse contact parameters
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:contact_parser: failed to parse contacts
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:parse_contact: failed to parse contact
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:pv_get_contact_body: failed to parse contact hdr
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]: 90tzz16g at pbx --
>>> New REGISTER received from 188.94.XXX.XXX with Contact :  -- IP
>>> 188.94.XXX.XXX -- PORT 5060
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]: 90tzz16g at pbx --
>>> REGISTER Failed because of : No Credentials -- From 188.94.XXX.XXX
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 526
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:sl:sl_send_reply_helper: response building failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:signaling:sig_send_reply_mod: failed to send reply with sl module
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25622]:
>>> ERROR:auth:challenge: failed to send the response
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25606]: iy4l7y1r at pbx --
>>> New REGISTER received from 188.94.XXX.XXX with Contact :
>>> <sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0>
>>> <sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0> -- IP
>>> 188.94.XXX.XXX -- PORT 5060
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25606]: iy4l7y1r at pbx --
>>> REGISTER Failed because of : No Credentials -- From 188.94.XXX.XXX
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]: iy4l7y1r at pbx --
>>> New REGISTER received from 188.94.XXX.XXX with Contact :
>>> <sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0>
>>> <sip:0326xxxxxx at 188.94.XXX.XXX:5060;transport=udp;line=6364d3f0> -- IP
>>> 188.94.XXX.XXX -- PORT 5060
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:core:db_allocate_rows: no memory left
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:db_mysql:db_mysql_convert_result: error while converting rows
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:db_mysql:db_mysql_store_result: error while converting result
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:core:db_do_query: error while storing result for query [select
>>> contact,expires,q,callid,cseq,flags,cflags,user_agent,received,path,socket,methods,last_modified,sip_instance,attr
>>> from location where username='0326xxxxxx' order by q]
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:usrloc:db_load_urecord: db_query failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> CRITICAL:db_mysql:wrapper_single_mysql_stmt_execute: driver error (1062):
>>> Duplicate entry '
>>> 0326xxxxxx--sip:0326xxxxxx at 188.94.XX.XXX:5060;transport=udp;line' for
>>> key 'account_contact_idx'
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:usrloc:db_insert_ucontact: inserting contact in db failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25614]:
>>> ERROR:usrloc:insert_ucontact: failed to insert in database
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 373
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:sl:sl_send_reply_helper: response building failed
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:signaling:sig_send_reply_mod: failed to send reply with sl module
>>> Aug 12 11:48:34 asbc2 /usr/local/sbin/opensips[25620]:
>>> ERROR:options:opt_reply: failed to send 200 via send_reply
>>
>>
>>  Unfortunately, we don't have the memory dump :-s ...
>>
>>  So, I don't know if there's something we can try to avoid those errors
>> to happen again; maybe we made something wrong in our configuration file ?
>> We could provide you anything you want to help us find where's the
>> problem.
>>
>>  Thanks a lot for your help,
>>
>>  Kevin
>>
>>
>> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>>
>>
>>
>> 2014-07-08 8:43 GMT+02:00 Kevin Mathy <k.mathy at hexanet.fr>:
>>
>>> Hi Bogdan,
>>>
>>>  Sounds really good :-) ! I'm going to check if there's no
>>> configuration change between 1.9's and 1.11's modules version, and then
>>> will upgrade our version !
>>>
>>>  I hope this will correct our problem ;-)
>>>
>>>  Thanks for your help,
>>>
>>>  Kevin
>>>
>>>
>>>
>>>
>>> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>>>
>>>
>>>
>>>  2014-07-07 18:40 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:
>>>
>>>  Hi Kevin,
>>>>
>>>> OK, found the problem. As expected, this was fixed in 1.11, but not
>>>> ported to 1.9 as this version is not maintained any more. Here is the
>>>> link to the fix for your crash:
>>>>
>>>>
>>>> https://github.com/OpenSIPS/opensips/commit/eef59c1d7b47ecc35438c01e7dcbdfeea903115a#diff-30710a8823b167e0aae5a14bb4878b75
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Bogdan-Andrei Iancu
>>>> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
>
>

-- 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20140904/c43670fa/attachment-0001.htm>