[OpenSIPS-Users] sched_yield()

Alex Massover alex at jajah.com
Thu Jan 21 13:04:54 CET 2010


Hi Andrei,
Hopefully this is it (with FASTLOCK)

#0  0xb77d5424 in __kernel_vsyscall ()
#1  0xb772babb in poll () from /lib/i686/cmov/libc.so.6
#2  0xb77ba83a in ?? () from /lib/i686/cmov/libresolv.so.2
#3  0xb77b8946 in __libc_res_nquery () from /lib/i686/cmov/libresolv.so.2
#4  0xb77b8fdb in ?? () from /lib/i686/cmov/libresolv.so.2
#5  0xb77b92ae in __libc_res_nsearch () from /lib/i686/cmov/libresolv.so.2
#6  0xb77b96d4 in __res_nsearch () from /lib/i686/cmov/libresolv.so.2
#7  0xb77b808a in res_search () from /lib/i686/cmov/libresolv.so.2
#8  0x0808c613 in get_record ()
#9  0x0808cf05 in ?? ()
#10 0x0808e385 in sip_resolvehost ()
#11 0x0807a26c in mk_proxy ()
#12 0xb7627d39 in t_relay_to () from /usr/lib/opensips/modules/tm.so
#13 0xb7634501 in ?? () from /usr/lib/opensips/modules/tm.so
#14 0x08055030 in do_action ()
#15 0x08053ebf in run_action_list ()
#16 0x08095cf2 in eval_expr ()
#17 0x080958d9 in eval_expr ()
#18 0x08095919 in eval_expr ()
#19 0x080554e2 in do_action ()
#20 0x08053ebf in run_action_list ()
#21 0x08056e7a in do_action ()
#22 0x08053ebf in run_action_list ()
---Type <return> to continue, or q <return> to quit---
#23 0x080569d8 in do_action ()
#24 0x08053ebf in run_action_list ()
#25 0x08056e7a in do_action ()
#26 0x08053ebf in run_action_list ()
#27 0x08057d99 in run_top_route ()
#28 0x0808ad6c in receive_msg ()
#29 0x080bd2f2 in udp_rcv_loop ()
#30 0x08069339 in main ()
(gdb)

--
Best Regards,
Alex Massover
VoIP R&D TL
Jajah Inc.
> -----Original Message-----
> From: users-bounces at lists.opensips.org [mailto:users-
> bounces at lists.opensips.org] On Behalf Of Andrei Dragus
> Sent: Wednesday, January 20, 2010 2:58 PM
> To: OpenSIPS users mailling list
> Subject: Re: [OpenSIPS-Users] sched_yield()
> 
> Hi,
> 
> I think that there is a lock that is being held more than it should be
> and that's what causes starvation. It would help us if you could attach
> to a process using gdb and give us a full backtrace.
> 
> Temporary solutions which should work would be to reduce the number of
> processes to 4-6 or to recompile replacing -DFAST_LOCK with one of the
> other options (-DUSE_POSIX_SEM or -DUSE_PTHREAD_MUTEX) but we should
> see
> where this is from to fix it.
> 
> Alex Massover wrote:
> > Hi!
> >
> > Yes, from the source on debian, I build deb package. (I did some
> minor changes to the source, but the problem happens also without my
> changes)
> > 16 children on 4 cores.
> >
> > What do you suggest to reduce it to 4? It runs on 2.6.32 on VMware
> ESX.
> >
> > I'm also trying now sleep(0) instead of sched_yield().
> >
> > --
> > Best Regards,
> > Alex Massover
> > VoIP R&D TL
> > Jajah Inc.
> >
> >> -----Original Message-----
> >> From: users-bounces at lists.opensips.org [mailto:users-
> >> bounces at lists.opensips.org] On Behalf Of Andrei Dragus
> >> Sent: Wednesday, January 20, 2010 1:05 PM
> >> To: OpenSIPS users mailling list
> >> Subject: Re: [OpenSIPS-Users] sched_yield()
> >>
> >> Hi Alex,
> >>
> >> Are you building OpenSIPS from source?
> >> How many processes do you have and on how many cores?
> >>
> >>
> >> Alex Massover wrote:
> >>
> >>> Hello!
> >>>
> >>> I'm facing a strange problem, sometimes under a stress OpenSIPS
> >>> "locks" - load average jumps, SIP processing delays, opensips msg
> >>> queue fills with a lot of sip messages, opensips processes start to
> >>> comsume a lot of CPU.
> >>>
> >>> And strace shows:
> >>>
> >>> sched_yield()
> >>>
> >>> sched_yield()
> >>>
> >>> sched_yield()
> >>>
> >>> sched_yield()
> >>>
> >>> ....
> >>>
> >>> for all processes.
> >>>
> >>> If I stop the stress - after a while (not immediately) - it
> unlocks,
> >>> also suddenly, I can see in top that all opensips processes stop to
> >>> consume CPU.
> >>>
> >>> What can it be? Some kind of starvation?
> >>>
> >>> --
> >>>
> >>> Best Regards,
> >>>
> >>> Alex Massover
> >>>
> >>> VoIP R&D TL
> >>>
> >>> Jajah Inc.
> >>>
> >>>
> >>>
> >>> This mail was sent via Mail-SeCure System.
> >>> -------------------------------------------------------------------
> --
> >>>
> >> ---
> >>
> >>> _______________________________________________
> >>> Users mailing list
> >>> Users at lists.opensips.org
> >>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >>>
> >>>
> >> --
> >> Andrei Dragus
> >> www.voice-system.ro
> >>
> >>
> >> _______________________________________________
> >> Users mailing list
> >> Users at lists.opensips.org
> >> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >>
> >> This mail was received via Mail-SeCure System.
> >>
> >>
> >
> >
> > This mail was sent via Mail-SeCure System.
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users at lists.opensips.org
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
> 
> --
> Andrei Dragus
> www.voice-system.ro
> 
> 
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> 
> This mail was received via Mail-SeCure System.
> 


This mail was sent via Mail-SeCure System.





More information about the Users mailing list