[OpenSIPS-Users] OpenSIPS crashed after "out of pkg memory"

Kevin Mathy k.mathy at hexanet.fr
Fri Jun 27 09:36:08 CEST 2014


Hi Bogdan,

I've set given options, and now I'm waiting for a new crash of the
service... Where the memdump will be located ? In another logfile than
opensips.log, or in the same ?

Thanks


*Bien cordialement, Best Regards,  **Kevin MATHY* | Ingénieur VoIP



2014-06-26 18:32 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:

>  Kevin,
>
> Restarting should not make you loose ongoing calls (even if you use the
> dialog module), do do not worry on that.
>
> Regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>
> On 26.06.2014 18:31, Kevin Mathy wrote:
>
> Yes, I'll set those options, and wait 'till the trafic on the server
> reduces a lot (tonight I think) to restart opensips.
> Unless if you have another method to reload the config without losing
> calls ?
>
>  Thanks a lot,
>
>
>
> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>
>
>
> 2014-06-26 17:20 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:
>
>>  Kevin,
>>
>> The trick is to set (in this order):
>>     memlog = 4
>>     memdump = 1
>>
>> This will do no logging at runtime, but it will log the mem dump (at
>> shutdown or at signal).
>>
>> Could you try that ?
>>
>> Regards,
>>
>> Bogdan-Andrei Iancu
>> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>>
>>   On 26.06.2014 18:09, Kevin Mathy wrote:
>>
>>  Hi Bogdan,
>>
>>  We already have compiled opensips as described, and set memlog=1 in the
>> configuration file.
>> But as we had a lot of debug messages about memory usage, we unset
>> memlog=1 in the config, so we don't have any report now.
>> Is there a way to get those messages in a different log file, instead of
>> inside the "classical" opensips.log ? I've found an option like "memdump"
>> that can be set in opensips.cfg, but what about it ?
>>
>>  And as cacti doesn't seems to be relevant to check opensips's mem
>> usage, do you have any other solution to check it ?
>>
>>  Thanks for your help,
>>
>>
>> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>>
>>
>>
>> 2014-06-26 16:46 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org>:
>>
>>>  Hi Kevin,
>>>
>>> For debugging potential mem leaks, please look into:
>>>     http://www.opensips.org/Documentation/TroubleShooting-OutOfMem
>>> If you need help with that, let me know.
>>>
>>> Also, cacti is monitoring the system memory which is irrelevant for
>>> OpenSIPS  (as OpenSIPS is pre-allocating - from OS - all the needed memory
>>> at startup).
>>>
>>> Regards,
>>>
>>> Bogdan-Andrei Iancu
>>> OpenSIPS Founder and Developerhttp://www.opensips-solutions.com
>>>
>>>  On 26.06.2014 15:04, Kevin Mathy wrote:
>>>
>>>   Hi List,
>>>
>>>  I'm facing a memory usage issue with OpenSIPS 1.9.2 on a Debian 7
>>> server
>>>
>>>   root at asbc2:/home/kemathy# uname -a
>>>> Linux asbc2 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux
>>>>
>>>
>>>
>>>> root at asbc2:/home/kemathy# opensips -V
>>>
>>> version: opensips 1.9.2-notls (x86_64/linux)
>>>> flags: STATS: On, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM,
>>>> SHM_MMAP, PKG_MALLOC, DBG_F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
>>>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
>>>> MAX_URI_SIZE 1024, BUF_SIZE 65535
>>>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>>>> svnrevision: unknown
>>>> @(#) $Id$
>>>> main.c compiled on 11:15:37 Jun 20 2014 with gcc 4.7
>>>
>>>
>>>   The server handles something like 70000 to 90000 calls per day, with
>>> around 20 to 40 Calls Per Second
>>>
>>>
>>>  As you can see on the following Cacti graphs, system's free memory
>>> decreases constantly to finally reach the lower value of 160M :
>>>
>>>
>>> https://www.dropbox.com/s/np3fnc6ni4vuoet/ASBC2_Memory_Usage_Evolution.jpg
>>>
>>>   The system can stay 2-3 days working in this state, but for example,
>>> yesterday evening (2014-06-25 18:50:00), opensips crashed, after showing
>>> those log messages :
>>>
>>>  Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11390]:
>>>> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11390]:
>>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 412
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11380]:
>>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11380]:
>>>> ERROR:core:received_builder: out of pkg memory
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11364]:
>>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11364]:
>>>> ERROR:core:add_lump_rpl: no free pkg memory !
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>>>> ERROR:core:db_allocate_rows: no memory left
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>>>> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 412
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11356]:
>>>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
>>>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11356]:
>>>> ERROR:core:add_lump_rpl: no free pkg memory !
>>>>
>>>
>>>  And after the service's restart, everything was OK :
>>>
>>>> Jun 25 19:12:39 asbc2 /usr/local/sbin/opensips[29351]: INFO:core:main:
>>>> using 64 Mb shared memory
>>>> Jun 25 19:12:39 asbc2 /usr/local/sbin/opensips[29351]: INFO:core:main:
>>>> using 4 Mb private memory per process
>>>
>>>
>>>
>>>  For information, here are the memory values at this moment; opensips
>>> has handled 5000 calls since his bootup :
>>>
>>>> root at asbc2:/home/kemathy# free -m
>>>>              total       used       free     shared    buffers
>>>> cached
>>>> Mem:         16082      12887       3195          0        339
>>>>  11393
>>>> -/+ buffers/cache:       1154      14928
>>>> Swap:        32755          0      32755
>>>
>>>
>>>
>>>  For now, OpenSIPS is launched with S_MEMORY = 64 and P_MEMORY = 4
>>> My first idea would be to increase the P_MEMORY value to 8 for example,
>>> as it seems to be a lack of PKG Memory, but in case it's a memory-freeing
>>> problem, it'll not solve our issue (I think).
>>>
>>>  My questions are : How can we calculate the appropriate value for
>>> S_MEMORY and P_MEMORY ? And how can we solve our "out of pkg memory"
>>> problem ?
>>>
>>>  If you need further informations, or anything else, feel free to ask
>>> me !
>>>
>>>  Thanks a lot for your help,
>>>
>>>
>>> * Bien cordialement,  Best Regards,  **Kevin MATHY* | Ingénieur VoIP
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>>>
>>>
>>
>>
>>
>
>
>

-- 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20140627/8a08b4df/attachment-0001.htm>


More information about the Users mailing list