[OpenSIPS-Users] memory errors, too many transactions

[Jeff Pyle]

Hello,

Everything is running along just fine for weeks on 1.6.4, and then bam, this shows up in the logs:

/usr/sbin/opensips[12290]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation
/usr/sbin/opensips[12290]: ERROR:tm:insert_tmcb: no more shared memory
/usr/sbin/opensips[12290]: ERROR:uac:replace_uri: failed to install TM callback
/usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation
/usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem
/usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure
/usr/sbin/opensips[12306]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation
/usr/sbin/opensips[12306]: ERROR:tm:new_t: out of mem
/usr/sbin/opensips[12302]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation
/usr/sbin/opensips[12302]: ERROR:tm:relay_reply: no more share memory
/usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation
/usr/sbin/opensips[12294]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation
/usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem
/usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure
/usr/sbin/opensips[12306]: ERROR:tm:t_newtran: new_t failed
/usr/sbin/opensips[12294]: ERROR:tm:relay_reply: no more share memory

Hundreds of lines like it.  This is just a sample.  But, it did not crash.  I have shared memory set to 32M.  Normally I run less than 8M used.  After this event the max_used_size was at 32M, but the real_used_size was around 24M.  And it wasn't changing.  Strange.

My first thought was a DoS attack but a pcap I took right after looked normal.

Normally this proxy runs anywhere from 70-150 transactions in progress at any given moment.  After this event, it was stuck at right around 1300.  Things seemed to be processing; no symptoms on outbound calls.  Very strange.

I restarted Opensips and everything seems normal.

I'm not sure where to begin on this one.  I'm not sure if the shared memory problem was a symptom or a cause.  In the few years I've been working with Opensips I've never seen anything like this.  Any direction from the list would be great.


- Jeff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20110429/1803de59/attachment.htm>