[OpenSIPS-Users] OpenSIPS crashed after "out of pkg memory"

Tue Sep 16 15:12:52 CEST 2014

Hi all,

after upgrade Opensips, I've faced same memory error (see below).

Have you a help for me please ?

Thanks in advance

Sep 16 15:05:23 asbc2 /usr/local/sbin/opensips[15783]:
ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 395
Sep 16 15:05:23 asbc2 /usr/local/sbin/opensips[15776]:
ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 388
Sep 16 15:05:24 asbc2 /usr/local/sbin/opensips[15777]:
ERROR:core:build_req_buf_from_sip_req: out of pkg memory
Sep 16 15:05:24 asbc2 /usr/local/sbin/opensips[15783]:
ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 433
Sep 16 15:05:24 asbc2 /usr/local/sbin/opensips[15776]:
ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 427
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15783]:
ERROR:core:parse_from_header: out of pkg_memory
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15783]:
ERROR:core:parse_from_header: out of pkg_memory
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15783]:
ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 355
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15783]:
ERROR:core:parse_from_header: out of pkg_memory
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15783]:
ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 531
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15773]:
ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 355
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15773]:
ERROR:core:new_credentials: no pkg memory left
Sep 16 15:05:25 asbc2 /usr/local/sbin/opensips[15773]:
ERROR:core:new_credentials: no pkg memory left

version: opensips 1.11.2-notls (x86_64/linux)
flags: STATS: On, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM,
SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: c529bb1
main.c compiled on 16:58:09 Sep 11 2014 with gcc 4.7

2014-09-10 16:16 GMT+02:00 Mickael Hubert <m.hubert at hexanet.fr>:

> Ok thanks for your response,
> I'll have upgrade my opensips immediately.
>
> but for me, my memory conf is wrong.
> -m 64 -M 4 with 20 childrens: 4Mo * 20 childrens = 80 Mo shared memory,
> but with -m option with 64Mo it's impossible no ?
>
> thanks
>
>
> 2014-09-10 13:27 GMT+02:00 Vlad Paiu <vladpaiu at opensips.org>:
>
>>  Hello,
>>
>> Can you please upgrade to the latest 1.11 from GIT ?
>> There has been a fix related to this on GIT starting from
>> https://github.com/OpenSIPS/opensips/commit/ea9191b6993a36b6190eb218f318d0a876421311
>>
>> Best Regards,
>>
>> Vlad Paiu
>> OpenSIPS Developerhttp://www.opensips-solutions.com
>>
>> On 10.09.2014 12:58, Mickael Hubert wrote:
>>
>>  Hi Bogdan,
>>  in addition, we use these options: "-m 64 -M 4" and in opensips.cfg
>> "children=20"
>>
>> private memory: 4 Mo
>>
>> Share memory: 64Mo
>> We use a server with 16Go of ram
>>
>> it's good conf ?
>>
>>  Thanks a lot for your help.
>>
>>
>> 2014-09-10 11:44 GMT+02:00 Kevin Mathy <k.mathy at hexanet.fr>:
>>
>>> Hi Bogdan,
>>>
>>>  Here's the version of opensips we use :
>>>
>>>> [root at asbc2b kemathy]# /usr/local/sbin/opensips -V
>>>> version: opensips 1.11.2-notls (x86_64/linux)
>>>> flags: STATS: On, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM,
>>>> SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
>>>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
>>>> MAX_URI_SIZE 1024, BUF_SIZE 65535
>>>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>>>> git revision: ff23ed0
>>>> main.c compiled on 19:24:42 Jul 23 2014 with gcc 4.4.7
>>>
>>>
>>>  About those points :
>>>   "- the "out of mem" was reported for pkg memory (and not for shm)"
>>> ==> The out of mem was reported for pkg memory
>>>
>>> " - you sent the SIGUSR1 signal to the process complaining about lack of
>>> mem "
>>>  ==> I don't know if I made it well... So, here are two links pointing
>>> to other memdumps, taken on processes which seem to lack of mem
>>>
>>>  Sep 10 05:34:05 asbc2b /usr/local/sbin/opensips[920]:
>>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory; needs 522
>>>> Sep 10 05:34:04 asbc2b /usr/local/sbin/opensips[923]:
>>>> ERROR:core:received_builder: out of pkg memory
>>>> Sep 10 05:33:57 asbc2b /usr/local/sbin/opensips[937]:
>>>> ERROR:core:parse_headers: pkg memory allocation failed
>>>> Sep 10 05:33:57 asbc2b /usr/local/sbin/opensips[937]:
>>>> ERROR:core:add_lump_rpl: no free pkg memory !
>>>
>>>
>>>  https://www.dropbox.com/s/p62hb16wpk0i86e/memdump_process_920.txt?dl=0
>>>  https://www.dropbox.com/s/uim49o6zvj6zvu3/memdump_process_923.txt?dl=0
>>>  https://www.dropbox.com/s/c6tksw4j4lgk5d3/memdump_process_937.txt?dl=0
>>>
>>>  So now, I'm sure that the memdumps I've taken concern processes
>>> impacted by the lack of pkg mem...
>>>
>>>  Thanks for your help,
>>>
>>>  Kevin
>>>
>>>
>>>
>>>
>>> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>>>
>>>
>>>   2014-09-10 10:04 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:
>>>
>>>>  Kevin,
>>>> This is stranger as nothing in the dump points to a leak/issue - do you
>>>> get the out of mem error during a SB query ?
>>>>
>>>> Regards,
>>>>
>>>> Bogdan-Andrei Iancu
>>>> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>>>>
>>>>   On 10.09.2014 09:53, Kevin Mathy wrote:
>>>>
>>>>  Hi Bogdan,
>>>>
>>>>  Got the same errors on my second opensips server (second node of the
>>>> cluster). Here is the memdump :
>>>> https://www.dropbox.com/s/d2gacgrxik3xs1t/memdump_server2_1.txt?dl=0
>>>>
>>>>  Thanks for your help,
>>>>
>>>>  Kevin
>>>>
>>>>
>>>>
>>>>
>>>> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>>>>
>>>>
>>>> 2014-09-09 14:56 GMT+02:00 Kevin Mathy <k.mathy at hexanet.fr>:
>>>>
>>>>> Hi Bogdan,
>>>>>
>>>>>  I've got now a memdump !
>>>>> https://www.dropbox.com/s/5hb678r0ym9vtt1/memdump1.txt?dl=0
>>>>>
>>>>>  And if you need the list of the opensips processes running at the
>>>>> moment :
>>>>>
>>>>> https://www.dropbox.com/s/hqurr3ilamc0a0g/opensips_processes_list.txt?dl=0
>>>>>
>>>>>  I hope it'll be ok... If you need more things, I didn't have
>>>>> restarted the service yet, so I'm able to get other memdumps.
>>>>>
>>>>>  Thanks for your help,
>>>>>
>>>>>  Kevin
>>>>>
>>>>>
>>>>>
>>>>> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>>>>>
>>>>>
>>>>>  2014-09-09 13:06 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:
>>>>>
>>>>>>  Hi Kevin,
>>>>>>
>>>>>>  yes, that's the right way of getting a mem dump during runtime. Also
>>>>>> you will automatically get one during the shutdown procedure.
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Bogdan-Andrei Iancu
>>>>>> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> Cordialement
>>
>> HUBERT Mickaël
>> Ingénieur VOIP - Hexanet
>>
>>
>>
>> _______________________________________________
>> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>>
>>
>
>
> --
> Cordialement
>
> HUBERT Mickaël
> Ingénieur VOIP - Hexanet
>

-- 
Cordialement

HUBERT Mickaël
Ingénieur VOIP - Hexanet

-- 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20140916/b3561926/attachment-0001.htm>