[OpenSIPS-Users] OpenSIPS crashed after "out of pkg memory"
Bogdan-Andrei Iancu
bogdan at opensips.org
Thu Jun 26 17:20:10 CEST 2014
Kevin,
The trick is to set (in this order):
memlog = 4
memdump = 1
This will do no logging at runtime, but it will log the mem dump (at
shutdown or at signal).
Could you try that ?
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
On 26.06.2014 18:09, Kevin Mathy wrote:
> Hi Bogdan,
>
> We already have compiled opensips as described, and set memlog=1 in
> the configuration file.
> But as we had a lot of debug messages about memory usage, we unset
> memlog=1 in the config, so we don't have any report now.
> Is there a way to get those messages in a different log file, instead
> of inside the "classical" opensips.log ? I've found an option like
> "memdump" that can be set in opensips.cfg, but what about it ?
>
> And as cacti doesn't seems to be relevant to check opensips's mem
> usage, do you have any other solution to check it ?
>
> Thanks for your help,
>
> *
> Bien cordialement,
> Best Regards,
>
> **Kevin MATHY* |**Ingénieur VoIP
> *
> *
>
>
> 2014-06-26 16:46 GMT+02:00 Bogdan-Andrei Iancu <bogdan at opensips.org
> <mailto:bogdan at opensips.org>>:
>
> Hi Kevin,
>
> For debugging potential mem leaks, please look into:
> http://www.opensips.org/Documentation/TroubleShooting-OutOfMem
> If you need help with that, let me know.
>
> Also, cacti is monitoring the system memory which is irrelevant
> for OpenSIPS (as OpenSIPS is pre-allocating - from OS - all the
> needed memory at startup).
>
> Regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developer
> http://www.opensips-solutions.com
>
> On 26.06.2014 15 <tel:26.06.2014%2015>:04, Kevin Mathy wrote:
>> Hi List,
>>
>> I'm facing a memory usage issue with OpenSIPS 1.9.2 on a Debian 7
>> server
>>
>> root at asbc2:/home/kemathy# uname -a
>> Linux asbc2 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2
>> x86_64 GNU/Linux
>>
>> root at asbc2:/home/kemathy# opensips -V
>>
>> version: opensips 1.9.2-notls (x86_64/linux)
>> flags: STATS: On, USE_IPV6, USE_TCP, DISABLE_NAGLE,
>> USE_MCAST, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_F_MALLOC,
>> FAST_LOCK-ADAPTIVE_WAIT
>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144,
>> MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>> svnrevision: unknown
>> @(#) $Id$
>> main.c compiled on 11:15:37 Jun 20 2014 with gcc 4.7
>>
>>
>> The server handles something like 70000 to 90000 calls per day,
>> with around 20 to 40 Calls Per Second
>>
>>
>> As you can see on the following Cacti graphs, system's free
>> memory decreases constantly to finally reach the lower value of
>> 160M :
>>
>> https://www.dropbox.com/s/np3fnc6ni4vuoet/ASBC2_Memory_Usage_Evolution.jpg
>>
>>
>> The system can stay 2-3 days working in this state, but for
>> example, yesterday evening (2014-06-25 18:50:00), opensips
>> crashed, after showing those log messages :
>>
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11390]:
>> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11390]:
>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory ;
>> needs 412
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11380]:
>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory ;
>> needs 355
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11380]:
>> ERROR:core:received_builder: out of pkg memory
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11364]:
>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory ;
>> needs 355
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11364]:
>> ERROR:core:add_lump_rpl: no free pkg memory !
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory ;
>> needs 355
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>> ERROR:core:db_allocate_rows: no memory left
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory ;
>> needs 412
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11356]:
>> ERROR:core:build_res_buf_from_sip_req: out of pkg memory ;
>> needs 355
>> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11356]:
>> ERROR:core:add_lump_rpl: no free pkg memory !
>>
>>
>> And after the service's restart, everything was OK :
>>
>> Jun 25 19:12:39 asbc2 /usr/local/sbin/opensips[29351]:
>> INFO:core:main: using 64 Mb shared memory
>> Jun 25 19:12:39 asbc2 /usr/local/sbin/opensips[29351]:
>> INFO:core:main: using 4 Mb private memory per process
>>
>>
>>
>> For information, here are the memory values at this moment;
>> opensips has handled 5000 calls since his bootup :
>>
>> root at asbc2:/home/kemathy# free -m
>> total used free shared buffers
>> cached
>> Mem: 16082 12887 3195 0 339
>> 11393
>> -/+ buffers/cache: 1154 14928
>> Swap: 32755 0 32755
>>
>>
>> For now, OpenSIPS is launched with S_MEMORY = 64 and P_MEMORY = 4
>> My first idea would be to increase the P_MEMORY value to 8 for
>> example, as it seems to be a lack of PKG Memory, but in case it's
>> a memory-freeing problem, it'll not solve our issue (I think).
>>
>> My questions are : How can we calculate the appropriate value for
>> S_MEMORY and P_MEMORY ? And how can we solve our "out of pkg
>> memory" problem ?
>>
>> If you need further informations, or anything else, feel free to
>> ask me !
>>
>> Thanks a lot for your help,
>>
>> *
>> Bien cordialement,
>> Best Regards,
>>
>> **Kevin MATHY* |**Ingénieur VoIP
>> *
>> *
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org <mailto:Users at lists.opensips.org>
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20140626/cb7eacb1/attachment-0001.htm>
More information about the Users
mailing list