[OpenSIPS-Users] 1.6 rev 6147 and dp_translate memory error?
Bogdan-Andrei Iancu
bogdan at voice-system.ro
Thu Sep 24 14:06:54 CEST 2009
Hi Ron,
Are you sure the DB content is what you used during the crash? The log says:
"string 4085557777 matched the match_exp ^([2-9][0-9]{2}[2-9][0-9]{6})
but not the subst_exp ^([2-9][0-9]{2}[2-9][0-9]{6})(.+)!" ,
But I see no record with match_exp="([2-9][0-9]{2}[2-9][0-9]{6})" and
subst_exp = "^([2-9][0-9]{2}[2-9][0-9]{6})(.+)" in the DB dump you posted !
Also, could you also print (from dgb) the match_nb and subst_comp and
subst_comp->_matches[match_nb] vars?
To speed up this, will it be possible to grant me access to the machine
where the core is ? only the core file is useless for me as I need the
binaries, the modules, and some libraries from your server :(...
Regards,
Bogdan
Ron McCarthy wrote:
> I ran one more test and this is the final results:
>
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]:
> ERROR:dialplan:rule_translate: the string 4085557777 matched the
> match_exp ^([2-9][0-9]{2}[2-9][0-9]{6}) but not the subst_exp
> ^([2-9][0-9]{2}[2-9][0-9]{6})(.+)!
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]:
> ERROR:dialplan:translate: could not build the output
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]:
> ERROR:core:do_assign: no value in right expression
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]:
> ERROR:core:do_assign: error at line: 432
>
> Line 432 is: $rU = $avp(s:carto_did);
>
> Then we get:
>
> Sep 22 08:17:48 sips /usr/local/sbin/opensips[52643]:
> ERROR:core:do_assign: error at line: 299
> Sep 22 08:17:48 sips /usr/local/sbin/opensips[52643]:
> ERROR:dialplan:dp_get_svalue: no PV or NULL or non-STR val found
> (error in scripts)
> Sep 22 08:17:48 sips /usr/local/sbin/opensips[52643]:
> ERROR:dialplan:dp_translate_f: invalid param 2
>
> Line 299: $avp(s:aninpa)=$(avp(s:newfrom_did){s.substr,0,3});
>
> At this point the switch is still taking on new calls, we get those
> errors some more times, then we core dump
>
> Here is the gdb output:
> (gdb) print result
> $1 = (str *) 0x7fffffffe170
> (gdb) print result->len
> $2 = 0
> (gdb) print result->s
> $3 = 0x801fe3f00 "5679990000"
> (gdb) print match
> $4 = {begin = 0x802f7e0a8 "5679990000", len = -43123475}
>
> Core file is 2GB but you can download via HTTP if that will help.
>
> Thanks and let me know what ever else I can do.
>
>
>
>
> Dialplan rules are:
>
> dpid pr match_op match_exp match_len subst_exp
> repl_exp
> 200 40 1 ^([2-9][0-9]{2}[2-9][0-9]{6}) 0
> ^([2-9][0-9]{2}[2-9][0-9]{6}) \1
> 200 50 1 ^1([2-9][0-9]{2}[2-9][0-9]{6}) 0
> ^1([2-9][0-9]{2}[2-9][0-9]{6})(.+) \1
>
> On Tue, Sep 22, 2009 at 7:51 AM, Bogdan-Andrei Iancu
> <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>> wrote:
>
> Hi Ron,
>
> there couple of options:
> 1) we try remove debugging
> 2) give me access to the core file so that I can investigate it
> 3) send the the DB content and the input for the dp_translate() to
> try to reproduce it
>
> so, let's try 1)
> please print in gdb the following values: result , result->len,
> result->s, match.
>
> Thanks and regards,
> Bogdan
>
> Ron McCarthy wrote:
> > Hi,
> >
> > Just updated and ran a test and now it's core dumping with no errors
> > in syslog, a backtrace shows:
> >
> > #0 0x0000000801ed9e41 in rule_translate (msg=0x775c88, string={s =
> > 0x8029e4e99 "6025504004", len = 10}, rule=Variable "rule" is not
> > available.
> > ) at dp_repl.c:192
> > #1 0x0000000801edb124 in translate (msg=0x775c88, input={s =
> > 0x8029e4e99 "6025504004", len = 10}, output=0x7fffffffd570,
> > idp=Variable "idp" is not available.
> > ) at dp_repl.c:346
> > #2 0x0000000801ed2472 in dp_translate_f (msg=0x775c88,
> str1=Variable
> > "str1" is not available.
> > ) at dialplan.c:368
> > #3 0x000000000040dbb1 in do_action (a=0x6dfb90, msg=0x775c88) at
> > action.c:962
> > #4 0x000000000040c6a7 in run_action_list (a=Variable "a" is not
> > available.
> > ) at action.c:139
> > #5 0x000000000040fb8e in do_action (a=0x6e3d10, msg=0x775c88) at
> > action.c:706
> > #6 0x000000000040c6a7 in run_action_list (a=Variable "a" is not
> > available.
> > ) at action.c:139
> > #7 0x000000000040eebe in do_action (a=0x6d4d08, msg=0x775c88) at
> > action.c:119
> > #8 0x000000000040c6a7 in run_action_list (a=Variable "a" is not
> > available.
> > ) at action.c:139
> > #9 0x000000000040fb8e in do_action (a=0x6d4f48, msg=0x775c88) at
> > action.c:706
> > #10 0x000000000040c6a7 in run_action_list (a=Variable "a" is not
> > available.
> > ) at action.c:139
> > #11 0x000000000040fb8e in do_action (a=0x6d6f90, msg=0x775c88) at
> > action.c:706
> > #12 0x000000000040c6a7 in run_action_list (a=Variable "a" is not
> > available.
> > ) at action.c:139
> > #13 0x0000000000410d29 in run_top_route (a=0x6bd588,
> msg=0x775c88) at
> > action.c:119
> > #14 0x0000000000455d6b in receive_msg (
> > buf=0x65dd80 "INVITE sip:1234568888 at 192.168.1.100:5060
> <http://sip:1234568888@192.168.1.100:5060>
> > <http://sip:1234568888@192.168.1.100:5060> SIP/2.0\r\nVia:
> SIP/2.0/UDP
> > 192.168.1.100:5061;branch=z9hG4bK-33820-611-0\r\nFrom: sipp
> > <sip:1234569999 at 192.168.1.100:5060
> <http://sip:1234569999@192.168.1.100:5060>
> >
> <http://sip:1234569999@192.168.1.100:5060>>;tag=33820SIPpTag00611\r\nTo:
> > sut <sip:12345"..., len=555, rcv_info=0x7fffffffea70) at
> receive.c:162
> > #15 0x0000000000499f3e in udp_rcv_loop () at udp_server.c:490
> > #16 0x0000000000428a2a in main (argc=7, argv=Variable "argv" is not
> > available.
> > ) at main.c:818
> >
> > That was about 800 calls into it at 10 CPS.
> >
> > What else can I try?
> >
> > Thanks
> >
> >
> > On Tue, Sep 22, 2009 at 5:26 AM, Bogdan-Andrei Iancu
> > <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>
> <mailto:bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>>>
> wrote:
> >
> > Hi Ron,
> >
> > If you update from SVN trunk, this should be fixed. Please
> let me know
> > if it works for you or not.
> >
> > Regards,
> > Bogdan
> >
> >
> > Ron McCarthy wrote:
> > > Hi,
> > >
> > > dp_translate("200", "$avp(s:from_did)/$avp(s:newfrom_did)");
> > >
> > > We call that after the INVITE and allow_trusted, etc. It gets
> > called 4
> > > times total, twice to normalize the $fU and $fU vars then
> two more
> > > times to change the values if needed, (adds a 1, +1, etc).
> > >
> > > Thanks
> > >
> > >
> > > On Fri, Sep 18, 2009 at 1:24 AM, Bogdan-Andrei Iancu
> > > <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>
> <mailto:bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>>
> > <mailto:bogdan at voice-system.ro
> <mailto:bogdan at voice-system.ro> <mailto:bogdan at voice-system.ro
> <mailto:bogdan at voice-system.ro>>>>
> > wrote:
> > >
> > > Hi Ron,
> > >
> > > Hoe do you call the dp_translate function from the script?
> > >
> > > Regards,
> > > Bogdan
> > >
> > > Ron McCarthy wrote:
> > > > Hi list,
> > > >
> > > > Ive done quite a bit of troubleshooting and ive
> found the
> > switch
> > > runs
> > > > clean with not using dp_translate, but when we do the
> > errors appear.
> > > >
> > > > After a few thousand calls we start getting: (no errors
> > before this)
> > > >
> > > > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
> > > > ERROR:dialplan:dp_get_svalue: no AVP or SCRIPTVAR
> found (error
> > > in scripts)
> > > > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
> > > > ERROR:dialplan:dp_translate_f: invalid param 2
> > > > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
> > > > ERROR:core:do_assign: no value in right expression
> > > > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
> > > > ERROR:core:do_assign: error at line: 298
> > > >
> > > > Backtrace shows:
> > > > #0 0x0000000801ff0211 in rule_translate (msg=0x6fe600,
> > string={s =
> > > > 0x80282a9c3 "1234569999", len = 10}, rule=Variable
> "rule"
> > is not
> > > > available.
> > > > ) at dp_repl.c:192
> > > > 192 memcpy(result->s + result->len, match.begin,
> match.len);
> > > > (gdb)
> > > >
> > > > Were using sipP to test this, im setting the source and
> > dest number
> > > > manually with a AVP var then having dp_translate run on
> > it, its
> > > taking
> > > > a 10 digit number and turning it into 11 digits, we have
> > about 45
> > > > rules loaded into the database for the dialplan,
> with this
> > > particular
> > > > dialplan ID their is 2 rules total, we call
> dp_translate a
> > total
> > > of 4
> > > > times for each new call.
> > > >
> > > > vmstat is basically all 0's when dp_translate disabled,
> > when enabled
> > > > it looks like:
> > > >
> > > > 0 9 0 2891M 2574M 1484 0 0 0 3737 0 0 0
> > 2744 29807
> > > > 11711 13 15 72
> > > > 1 7 0 2899M 2569M 1493 0 0 0 1983 0
> 0 0
> > 2678 39221
> > > > 11355 13 11 76
> > > > 0 8 0 2891M 2568M 1119 0 0 0 2821 0
> 0 0
> > 2360 28331
> > > > 10401 13 15 72
> > > > 0 8 0 2901M 2565M 1477 0 0 0 2086 0
> 0 0
> > 2226 39722
> > > > 9430 11 15 74
> > > > 1 8 0 2893M 2560M 1250 0 0 0 1993 0
> 0 0
> > 2912 23983
> > > > 12123 11 15 74
> > > > 4 6 0 2901M 2551M 1557 0 0 0 2035 0
> 0 0
> > 3075 38446
> > > > 13035 12 18 70
> > > > 0 9 0 2893M 2548M 1103 0 0 0 1877 0
> 0 0
> > 2772 26050
> > > > 11474 12 12 76
> > > > 0 8 0 2901M 2539M 1434 0 0 0 743 0
> 0 0
> > 3289 34833
> > > > 13759 8 17 75
> > > > 0 9 0 2893M 2534M 943 0 0 0 1533 0
> 0 0
> > 3372 23843
> > > > 14379 8 24 68
> > > > 2 7 0 2901M 2528M 1252 0 0 0 1207 0
> 0 0
> > 2762 39615
> > > > 11275 12 13 75
> > > > 0 8 0 2902M 2521M 1134 0 0 0 703 0
> 0 0
> > 3364 18464
> > > > 14069 6 18 76
> > > > 0 8 0 2901M 2514M 1670 0 0 0 1737 0
> 0 0
> > 3771 17832
> > > > 17211 1 16 82
> > > > 0 8 0 2902M 2508M 1212 0 0 0 803 0
> 0 0
> > 3141 5263
> > > > 13990 1 14 85
> > > > 0 8 0 2901M 2499M 1542 0 0 0 1241 0
> 0 0
> > 3720 17120
> > > > 16641 1 17 82
> > > > 0 7 0 2902M 2497M 1260 0 0 0 2027 0
> 0 0
> > 2561 6328
> > > > 11863 1 14 85
> > > > 0 7 0 2901M 2499M 1979 0 0 0 3653 0
> 0 0
> > 2442 19121
> > > > 11724 3 13 85
> > > > 1 8 0 2902M 2498M 1387 0 0 0 3062 0
> 0 0
> > 2183 6172
> > > > 10662 0 13 87
> > > >
> > > >
> > > > We have ran this at 5CPS and the switch will run
> fine for
> > several
> > > > thousand calls, then at 60+ CPS and runs for several
> thousand
> > > calls as
> > > > well, so it appears to be a memory issue to me as
> when the
> > total
> > > > number of processed calls goes up is when it dies on us.
> > > >
> > > > Let me know what else I can do to test/debug on my
> side to
> > help
> > > with this.
> > > >
> > > > Thanks
> > > >
> > >
>
More information about the Users
mailing list