[OpenSIPS-Users] memory errors, too many transactions

[Kamen Petrov]

Hi Anca,

I had the same issue last night.

My script is as follow:
*
if (subst_uri('/^sip:([0-9]+)@(.*)$/$json(cs/fwd_domainport)/i')){
    if ($var(debug_level) > 3) { xlog("[leg2]LOG: We got:
BillingID=$avp(s:billing_id) , Provider RateID=$avp(s:prov_raterec_id) ,
SrcipID=$avp(s:srcip_id) , Termination GW ID=$avp(s:prov_termgw_id)\n"); }
}

b2b_init_request("top hiding");
xlog("Saving to $avp(i:99): $json(cs)\n");*


and my log says:
*Jul  6 01:00:03 db01 opensips-leg2: [leg2]LOG: We got: BillingID=<null> ,
Provider RateID=<null> , SrcipID=<null> , Termination GW ID=<null>
Jul  6 01:00:03 db01 opensips-leg2: WARNING:core:fm_malloc: Not enough free
memory, will atempt defragmenation
Jul  6 01:00:03 db01 opensips-leg2: ERROR:core:parse_headers: pkg memory
allocation failed
Jul  6 01:00:03 db01 opensips-leg2: ERROR:b2b_logic:init_request: failed to
parse message
Jul  6 01:00:03 db01 opensips-leg2: Saving to <null>: { "orig_callid": "
2524605-3518920775-311907 at domain.net", "fwd_domainport":
"sip:16414656000 at nnn.nnn.nnn.nnn:5060", "billing_id": 1, "prov_raterec_id":
2, "srcip_id": 1, "prov_termgw_id": 11, "ani": "111111111111",
"display_name": "111111111111" }*


I had those errors for every single call attempt. It seems this happens on:
*b2b_init_request("top hiding");

*I had to restart the B2B to get rid of the issue.
After the restart, here is how the same log looks like:
*Jul  6 07:51:27 db01 opensips-leg2: [leg2]LOG: We got: BillingID=<null> ,
Provider RateID=<null> , SrcipID=<null> , Termination GW ID=<null>
Jul  6 07:51:27 db01 opensips-leg2:
[leg2][xxx.xxx.xxx.xxx]=========LOCAL_ROUTE (INVITE - <null>) -
CallID=B2B.131.4459132 , Term=nnn.nnn.nnn.nnn =========
Jul  6 07:51:27 db01 opensips-leg2: Saving to 131.0: { "orig_callid": "
2529570-3518945466-704362 at domain.net", "fwd_domainport":
"sip:16414656050 at nnn.nnn.nnn.nnn:5060", "billing_id": 1, "prov_raterec_id":
2, "srcip_id": 1, "prov_termgw_id": 11, "ani": "111111111111",
"display_name": "111111111111" }*


Here is my local route:
*local_route {
        $var(debug_level) = 10;
        if (is_present_hf("LEG1CID")) xlog("LEG1CID AFTER TOP HIDING\n");
        if ($var(debug_level) > 2) { xlog("[leg2][$si]=========LOCAL_ROUTE
($rm - $rr) - CallID=$ci , Term=$rd =========\n"); }
        if (is_method("ACK") && $ci=~"B2B\.[0-9]+\.[0-9]+") {
                route(19);
        }
        else if (is_method("BYE") && $ci=~"B2B\.[0-9]+\.[0-9]+" && src_ip=="
**xxx.xxx.xxx.xxx**") {
                route(91);
        }
}*


Any idea?*


Thanks
-- Kamen*


On 4 May 2011 11:45, Anca Vamanu <anca at opensips.org> wrote:

> **
> Hi Jeff,
>
> Is this with B2BUA?
> I have a report with a complex b2b scenario and transactions hanged in
> memory.
>
> Regards,
> Anca
>
>
> On 29/04/11 16:28, Jeff Pyle wrote:
>
>  Hello,
>
>  Everything is running along just fine for weeks on 1.6.4, and then *bam*,
> this shows up in the logs:
>
>   /usr/sbin/opensips[12290]: WARNING:core:fm_malloc: Not enough free
> memory, will atempt defragmenation
> /usr/sbin/opensips[12290]: ERROR:tm:insert_tmcb: no more shared memory
> /usr/sbin/opensips[12290]: ERROR:uac:replace_uri: failed to install TM
> callback
> /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory,
> will atempt defragmenation
> /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem
> /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp
> structure
> /usr/sbin/opensips[12306]: WARNING:core:fm_malloc: Not enough free memory,
> will atempt defragmenation
> /usr/sbin/opensips[12306]: ERROR:tm:new_t: out of mem
> /usr/sbin/opensips[12302]: WARNING:core:fm_malloc: Not enough free memory,
> will atempt defragmenation
> /usr/sbin/opensips[12302]: ERROR:tm:relay_reply: no more share memory
> /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory,
> will atempt defragmenation
> /usr/sbin/opensips[12294]: WARNING:core:fm_malloc: Not enough free memory,
> will atempt defragmenation
> /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem
> /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp
> structure
> /usr/sbin/opensips[12306]: ERROR:tm:t_newtran: new_t failed
> /usr/sbin/opensips[12294]: ERROR:tm:relay_reply: no more share memory
>
>
>  Hundreds of lines like it.  This is just a sample.  But, it did not
> crash.  I have shared memory set to 32M.  Normally I run less than 8M used.
>  After this event the max_used_size was at 32M, but the real_used_size was
> around 24M.  And it wasn't changing.  Strange.
>
>  My first thought was a DoS attack but a pcap I took right after looked
> normal.
>
>  Normally this proxy runs anywhere from 70-150 transactions in progress at
> any given moment.  After this event, it was stuck at right around 1300.
>  Things seemed to be processing; no symptoms on outbound calls.  Very
> strange.
>
>  I restarted Opensips and everything seems normal.
>
>  I'm not sure where to begin on this one.  I'm not sure if the shared
> memory problem was a symptom or a cause.  In the few years I've been working
> with Opensips I've never seen anything like this.  Any direction from the
> list would be great.
>
>
>  - Jeff
>
>
> _______________________________________________
> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
>
> --
> Anca Vamanu
> OpenSIPS Developer
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20110706/dfd0629a/attachment.htm>