[OpenSIPS-Users] Setting opensips for maximum performance
Bogdan-Andrei Iancu
bogdan at voice-system.ro
Wed Feb 24 12:06:06 CET 2010
Hi Jan,
I see you find the fix for the mem error :)
Regards,
Bogdan
Jan Rozhon wrote:
> Hi Bogdan,
>
> I am using the latest 1.6.1 version of opensips, so I hope it should not
> be the problem.
>
> My syslog got messy, so I ran the test again, here are the reports:
>
> first I got about 10 of these errors:
> Feb 23 03:18:53 chieftec /usr/sbin/opensips[32454]:
> ERROR:tm:new_t: out of mem
> Feb 23 03:18:53 chieftec /usr/sbin/opensips[32454]:
> ERROR:tm:t_newtran: new_t failed
>
> Then these errors:
>
> Feb 23 03:18:53 chieftec /usr/sbin/opensips[32462]:
> ERROR:usrloc:shm_str_dup: no shared memory left
> Feb 23 03:18:53 chieftec /usr/sbin/opensips[32462]:
> ERROR:usrloc:new_ucontact: no more shm memory
> Feb 23 03:18:53 chieftec /usr/sbin/opensips[32462]:
> ERROR:usrloc:mem_insert_ucontact: failed to create new contact
> Feb 23 03:18:53 chieftec /usr/sbin/opensips[32462]:
> ERROR:usrloc:insert_ucontact: failed to insert contact
> Feb 23 03:18:53 chieftec /usr/sbin/opensips[32462]:
> ERROR:registrar:update_contacts: failed to insert contact
>
> Then these 2 groups of errors occure again and again.
>
> Somehow one of my SIPp instances managed to quit properly, so I was able
> to read trace_screen log - all messages have quite low number of
> retransmissions (hundreds), however 2nd REGISTER and 2nd INVITE (both
> with authentication) have huge number of retransmissions (hundreds of
> thousands) and timeouts (tens of thousands) - so I guess the problem has
> something to do with the speed of mysql.
>
> Am I right? Is there some way to improve the performance?
>
> Thanks in advance, Jan.
>
>
> Dne 22.2.2010 14:49, Bogdan-Andrei Iancu napsal(a):
>
>> No other errors before (from same process) ? I mean errors from the DB
>> backend itself ?
>>
>> Regards,
>> Bogdan
>>
>> Jan Rozhon wrote:
>>
>>
>>> Now I found block of these errors -
>>>
>>> Feb 21 03:30:57 chieftec /usr/sbin/opensips[4795]:
>>> ERROR:auth_db:get_ha1: failed to query database
>>> Feb 21 03:30:57 chieftec /usr/sbin/opensips[4782]:
>>> ERROR:auth_db:get_ha1: failed to query database
>>> Feb 21 03:30:57 chieftec /usr/sbin/opensips[4791]:
>>> ERROR:auth_db:get_ha1: failed to query database
>>> Feb 21 03:30:57 chieftec /usr/sbin/opensips[4792]:
>>> ERROR:auth_db:get_ha1: failed to query database
>>> Feb 21 03:30:57 chieftec /usr/sbin/opensips[4783]:
>>> ERROR:auth_db:get_ha1: failed to query database
>>>
>>> So there is a problem with database as well. Does it mean, that even if
>>> system utilization tools (SAR) shows, that only about 1200 MB is used,
>>> opensips has run out of memory?
>>>
>>>
>>> Dne 22.2.2010 10:05, Bogdan-Andrei Iancu napsal(a):
>>>
>>>
>>>
>>>> If you received a 500 reply from opensips, you may check opensips logs
>>>> for errors - maybe opensips is running out of memory or some other
>>>> internal error.
>>>>
>>>> Regards,
>>>> Bogdan
>>>>
>>>> Jan Rozhon wrote:
>>>>
>>>>
>>>>
>>>>
>>>>> Hi Bogdan,
>>>>>
>>>>> I also thought, that the problem may be in SIPp, however most error I
>>>>> get in SIPp are 500 Server Error Occured, so It directed me to
>>>>> Opensips, moreover I tried to increase the load on a single VM and the
>>>>> problems didnt occure until I reached 100% CPU utilization on SIPp VM.
>>>>>
>>>>> Disabling the authentication is not an option for me, because I need
>>>>> it in my thesis, but I will try it to see, if it is not the
>>>>> bottleneck.
>>>>>
>>>>> Thanks
>>>>> Jan
>>>>>
>>>>> 2010/2/22 Bogdan-Andrei Iancu<bogdan at voice-system.ro>:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi Jan,
>>>>>>
>>>>>> Jan Rozhon wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi Bogdan,
>>>>>>>
>>>>>>> thank you for your advice, but no help so far. I ran a test again using
>>>>>>> a load of 280 calls per second (2800 simultaneous calls) and watched
>>>>>>> receive buffer errors using netstat -su, but only 2930 errors occured
>>>>>>> out of almost 3 000 000 packets received. So if I understand it well,
>>>>>>> the problem is not caused by udp buffer size, because despite the low
>>>>>>> number of udp datagrams discarded more than 60 000 calls were not
>>>>>>> succesful (aproximately 25% of total calls).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> In sipp, what are the the cps and max parallel call you are using ?
>>>>>> Also, what kind of failures does the sipp report (missing 200ok ?
>>>>>> missing BYE? etc).
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Then I checked the database setting - I am using MySQL with "0"
>>>>>>> parameter (no database persistency), so I dont know how else I can make
>>>>>>> it run faster (I cannot use separate computer for the database).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> so no DB persistence for usrloc, but you mentioned using auth - is this
>>>>>> correct? as auth is doing real time DB queries maybe you should try
>>>>>> disabling the auth also to see how this affects the performance.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> As an opensips newbie, I dont have a clue, where the real problem could
>>>>>>> be, so any further help would be appreciated.
>>>>>>>
>>>>>>> Thanks, Jan
>>>>>>>
>>>>>>> Dne 19.2.2010 14:56, Bogdan-Andrei Iancu napsal(a):
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hi Jan,
>>>>>>>>
>>>>>>>> You need to see where the bottleneck is. If it is not the CPU, it must
>>>>>>>> be some I/O blocking opensips. For example the DB may be too slow
>>>>>>>> blocking opesips processes -> no process to read traffic from network ->
>>>>>>>> traffic discarded, package lost.
>>>>>>>>
>>>>>>>> So, you should first look with netstat at the UDP sock, to see if the
>>>>>>>> in-buffer is full or not (if full, the kernel will discard packages).
>>>>>>>>
>>>>>>>> Regarding the timeouts (for calls, for retransmissions) see the TM module:
>>>>>>>> http://www.opensips.org/html/docs/modules/1.6.x/tm.html#id228443
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Bogdan
>>>>>>>>
>>>>>>>> Jan Rozhon wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> as a part of my diploma thesis I'm trying to do some performance testing
>>>>>>>>> of opensips running on low cost machine with 4GB of RAM and dual-core
>>>>>>>>> AMD Athlon processor. As a generator of SIP traffic I use SIPp v3.1
>>>>>>>>> running on 4 virtual computers as UAC and two computers as UAS all
>>>>>>>>> connected just by gigabit ports of a single switch. My proublem is, that
>>>>>>>>> as soon as I reach the load of 200 calls per second (2000 simultaneous
>>>>>>>>> calls) many of those calls (aroud 50%) don't finish succesfully. CPU
>>>>>>>>> utilization of opensips machine is around 40% and memory around 25%, so
>>>>>>>>> the opensips has still a lot of free resources, but it doesn't use them.
>>>>>>>>> Could you please advise me, how to change default configuration script
>>>>>>>>> and opensips settings to achieve better results?
>>>>>>>>>
>>>>>>>>> Right now, the only changes I made is :
>>>>>>>>> -enabling registrar/proxy authentication;
>>>>>>>>> -disabled tcp, since all the traffic is carried by udp;
>>>>>>>>> -set shared memory to 2048 MB;
>>>>>>>>> -set number of children to 16
>>>>>>>>> -limit debug level to 1
>>>>>>>>>
>>>>>>>>> PS. How can I increase the time, after which opensips retransmits SIP
>>>>>>>>> messages?
>>>>>>>>>
>>>>>>>>> Thanks in advance,
>>>>>>>>> Jan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Users mailing list
>>>>>>>>> Users at lists.opensips.org
>>>>>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opensips.org
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>>>
>>>
>>>
>>
>>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
--
Bogdan-Andrei Iancu
www.voice-system.ro
More information about the Users
mailing list