[Users] memory leak in presence module?

Klaus Darilion klaus.mailinglists at pernau.at
Thu Apr 26 15:59:58 CEST 2007


Hi!

I also tried now with QM_JOIN_FREE but the same result:
The shared mem (64M) is full after ~8000 PUBLISH (=8000 NOTIFY) and does 
not get freed:

darilion at ds3000:~/software/openser-svn-1.2/openser$ sudo openserctl fifo 
get_statistics all
core:rcv_requests = 8743
core:rcv_replies = 8198
core:fwd_requests = 0
core:fwd_replies = 0
core:drop_requests = 0
core:drop_replies = 0
core:err_requests = 0
core:err_replies = 0
core:bad_URIs_rcvd = 0
core:unsupported_methods = 0
core:bad_msg_hdr = 0
shmem:total_size = 67108864
shmem:used_size = 65865712
shmem:real_used_size = 67004532
shmem:max_used_size = 67108376
shmem:free_size = 104332
shmem:fragments = 239
sl:1xx_replies = 0
sl:2xx_replies = 8221
sl:3xx_replies = 0
sl:4xx_replies = 196
sl:5xx_replies = 186
sl:6xx_replies = 0
sl:sent_replies = 8603
sl:sent_err_replies = 186
sl:received_ACKs = 0
tm:received_replies = 8198
tm:relayed_replies = 8198
tm:local_replies = 0
tm:UAS_transactions = 8218
tm:UAC_transactions = 8200
tm:2xx_transactions = 8198
tm:3xx_transactions = 0
tm:4xx_transactions = 0
tm:5xx_transactions = 0
tm:6xx_transactions = 0
tm:inuse_transactions = 8217
usrloc:registered_users = 0
usrloc:location-users = 0
usrloc:location-contacts = 0
usrloc:location-expires = 0
registrar:max_expires = 0
registrar:max_contacts = 0
registrar:default_expire = 3600
registrar:accepted_regs = 1
registrar:rejected_regs = 0
darilion at ds3000:~/software/openser-svn-1.2/openser$


regards
klaus



Klaus Darilion wrote:
> memory logs:
> 
> kill -SIGUSR1:
> http://pernau.at/kd/openser/debug-kill-usr1.txt
> 
> openser stop:
> http://pernau.at/kd/openser/debug-stop3.txt
> 
> regards
> klaus
> 
> Klaus Darilion wrote:
>>
>> Openser runs out of mem:
>> $ sudo openserctl fifo get_statistics all
>> core:rcv_requests = 13893
>> core:rcv_replies = 18007
>> core:fwd_requests = 0
>> core:fwd_replies = 0
>> core:drop_requests = 0
>> core:drop_replies = 0
>> core:err_requests = 0
>> core:err_replies = 0
>> core:bad_URIs_rcvd = 0
>> core:unsupported_methods = 0
>> core:bad_msg_hdr = 0
>> shmem:total_size = 67108864
>> shmem:used_size = 65351684
>> shmem:real_used_size = 66591880
>> shmem:max_used_size = 66953980
>> shmem:free_size = 516984
>> shmem:fragments = 1052
>> sl:1xx_replies = 0
>> sl:2xx_replies = 8878
>> sl:3xx_replies = 0
>> sl:4xx_replies = 2322
>> sl:5xx_replies = 2308
>> sl:6xx_replies = 0
>> sl:sent_replies = 13508
>> sl:sent_err_replies = 2308
>> sl:received_ACKs = 0
>> tm:received_replies = 18007
>> tm:relayed_replies = 17652
>> tm:local_replies = 1
>> tm:UAS_transactions = 8861
>> tm:UAC_transactions = 17653
>> tm:2xx_transactions = 17652
>> tm:3xx_transactions = 0
>> tm:4xx_transactions = 1
>> tm:5xx_transactions = 0
>> tm:6xx_transactions = 0
>> tm:inuse_transactions = 8861
>> usrloc:registered_users = 2
>> usrloc:location-users = 2
>> usrloc:location-contacts = 2
>> usrloc:location-expires = 0
>> registrar:max_expires = 0
>> registrar:max_contacts = 0
>> registrar:default_expire = 3600
>> registrar:accepted_regs = 12
>> registrar:rejected_regs = 0
>>
>>
>> after 1h of inactivity:
>> $ sudo openserctl fifo get_statistics all
>> Password:
>> core:rcv_requests = 14285
>> core:rcv_replies = 18007
>> core:fwd_requests = 0
>> core:fwd_replies = 0
>> core:drop_requests = 0
>> core:drop_replies = 0
>> core:err_requests = 0
>> core:err_replies = 0
>> core:bad_URIs_rcvd = 0
>> core:unsupported_methods = 0
>> core:bad_msg_hdr = 0
>> shmem:total_size = 67108864
>> shmem:used_size = 65351684
>> shmem:real_used_size = 66591880
>> shmem:max_used_size = 66953980
>> shmem:free_size = 516984
>> shmem:fragments = 1052
>> sl:1xx_replies = 0
>> sl:2xx_replies = 8922
>> sl:3xx_replies = 0
>> sl:4xx_replies = 2481
>> sl:5xx_replies = 2497
>> sl:6xx_replies = 0
>> sl:sent_replies = 13900
>> sl:sent_err_replies = 2497
>> sl:received_ACKs = 0
>> tm:received_replies = 18007
>> tm:relayed_replies = 17652
>> tm:local_replies = 1
>> tm:UAS_transactions = 8861
>> tm:UAC_transactions = 17653
>> tm:2xx_transactions = 17652
>> tm:3xx_transactions = 0
>> tm:4xx_transactions = 1
>> tm:5xx_transactions = 0
>> tm:6xx_transactions = 0
>> tm:inuse_transactions = 8861
>> usrloc:registered_users = 2
>> usrloc:location-users = 2
>> usrloc:location-contacts = 2
>> usrloc:location-expires = 0
>> registrar:max_expires = 0
>> registrar:max_contacts = 0
>> registrar:default_expire = 3600
>> registrar:accepted_regs = 51
>> registrar:rejected_regs = 0
>>
>>
>>
>> regards
>> klaus
>>
>> Daniel-Constantin Mierla wrote:
>>> my mistake:
>>>
>>> # openserctl fifo get_statistics all
>>>
>>> Cheers,
>>> Daniel
>>>
>>>
>>> On 04/26/07 12:36, Klaus Darilion wrote:
>>>> # openserctl fifo get_statistics
>>>> 400 Too few or too many arguments
>>>>
>>>> :-(
>>>>
>>>> I'm using 1.2
>>>>
>>>>
>>>> regards
>>>> klaus
>>>>
>>>>
>>>>
>>>> Klaus Darilion wrote:
>>>>>
>>>>>
>>>>> Daniel-Constantin Mierla wrote:
>>>>>> Hello Klaus,
>>>>>>
>>>>>> if that is the full log, seems no mem leak in shm. What is the shm 
>>>>>> memory size you use?
>>>>>
>>>>> I don't know - I use the default values. But the size should be 
>>>>> irrelevant as the "no memory left" is still there after 20 minutes 
>>>>> of inactivity.
>>>>>
>>>>>> Once you get out of memory can you issue" openserctl fifo 
>>>>>> get_statistics" and send output?
>>>>> ok
>>>>>>
>>>>>> I don't think is fragmentation, but you can try to compile with 
>>>>>> -DQM_JOIN_FREE
>>>>>>
>>>>> ok
>>>>>> Do you do the tests over TLS?
>>>>> ok
>>>>>>
>>>>>> Cheers,
>>>>>> Daniel
>>>>>>
>>>>>>
>>>>>> On 04/26/07 10:17, Klaus Darilion wrote:
>>>>>>> btw: there is no
>>>>>>> "PRESENCE:notify: ERROR in function tmb.t_request_within"
>>>>>>> in my log file, thus I guess the problem is somewhere else.
>>>>>>>
>>>>>>> regards
>>>>>>> klaus
>>>>>>>
>>>>>>> Klaus Darilion wrote:
>>>>>>>> Hi Anca!
>>>>>>>>
>>>>>>>> There is still a leak:
>>>>>>>> http://pernau.at/kd/openser/debug-stop2.txt
>>>>>>>>
>>>>>>>> regards
>>>>>>>> klaus
>>>>>>>>
>>>>>>>> Anca-Maria Vamanu wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I have made a fix. Please test again with sipp and say if it 
>>>>>>>>> works now.
>>>>>>>>>
>>>>>>>>> regards,
>>>>>>>>>
>>>>>>>>> Anca
>>>>>>>>>
>>>>>>>>> Klaus Darilion wrote:
>>>>>>>>>
>>>>>>>>>> Hi!
>>>>>>>>>>
>>>>>>>>>> Openser 1.2 runs out of memory during presence load tests with 
>>>>>>>>>> sipp.
>>>>>>>>>>
>>>>>>>>>> Config snippet:
>>>>>>>>>>
>>>>>>>>>> if (is_method("PUBLISH")) {
>>>>>>>>>>   xlog("L_INFO","$ci PUBLISH detected ... outside t_newtran\n");
>>>>>>>>>>   if (t_newtran()) {
>>>>>>>>>>     xlog("L_INFO","$ci PUBLISH detected ... inside t_newtran\n");
>>>>>>>>>>     handle_publish();
>>>>>>>>>>     xlog("L_INFO","$ci PUBLISH detected, handle_publish ... 
>>>>>>>>>> done\n");
>>>>>>>>>>     exit;
>>>>>>>>>>   } else {
>>>>>>>>>>     sl_reply_error();
>>>>>>>>>>     exit;
>>>>>>>>>>   }
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Memory log after openser stop (after 20 minutes of inactivity):
>>>>>>>>>> http://pernau.at/kd/openser/debug-stop.txt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> regards
>>>>>>>>>> klaus Klaus Darilion wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Cesc!
>>>>>>>>>>>
>>>>>>>>>>> Where do you have t_newtran() - in ser or openser?
>>>>>>>>>>>
>>>>>>>>>>> I have it here in openser just before handle_publish(). It 
>>>>>>>>>>> works but now openser has run out of mem during my sipp 
>>>>>>>>>>> performance tests :-(
>>>>>>>>>>>
>>>>>>>>>>> ...more to come....
>>>>>>>>>>>
>>>>>>>>>>> Cesc wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think I can answer that one ... stateless?
>>>>>>>>>>>>
>>>>>>>>>>>> I have a presence openser proxied by a SER, both on same 
>>>>>>>>>>>> machine ...
>>>>>>>>>>>> For easier configuration purposes, openser and ser 
>>>>>>>>>>>> communicate via
>>>>>>>>>>>> 127.0.0.1 ...
>>>>>>>>>>>>
>>>>>>>>>>>> The result is that OK from the PUAs do not get to openser (i 
>>>>>>>>>>>> guess SER
>>>>>>>>>>>> gets confussed by the vias of openser being 0.0.0.0:5060 and 
>>>>>>>>>>>> the
>>>>>>>>>>>> 127.0.0.1 stuff) ... So, openser gets no OK, but it does not 
>>>>>>>>>>>> matter
>>>>>>>>>>>> ...
>>>>>>>>>>>> BTW, i have a t_newtran() in there ...
>>>>>>>>>>>>
>>>>>>>>>>>> Oh, for sure my configuration is messy and improvable ... 
>>>>>>>>>>>> but for now
>>>>>>>>>>>> it works steady :) But the OKs not going through is worrying 
>>>>>>>>>>>> me a bit
>>>>>>>>>>>> ... and also openser not "timing out" or giving an error.
>>>>>>>>>>>>
>>>>>>>>>>>> Cesc
>>>>>>>>>>>>
>>>>>>>>>>>> On 4/25/07, Klaus Darilion <klaus.mailinglists at pernau.at> 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Does presence module handles PUBLISH/SUBSCRIBE stateful or 
>>>>>>>>>>>>> stateless?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thus, how should retransmission be handled. Will they be 
>>>>>>>>>>>>> detected by
>>>>>>>>>>>>> presence module or is it possible to wrap the 
>>>>>>>>>>>>> handle_publish inside
>>>>>>>>>>>>> t_newtran()?
>>>>>>>>>>>>>
>>>>>>>>>>>>> regards
>>>>>>>>>>>>> klaus
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>> Users at openser.org
>>>>>>>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Users mailing list
>>>>>>>>>>> Users at openser.org
>>>>>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Users mailing list
>>>>>>>>>> Users at openser.org
>>>>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users at openser.org
>>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users at openser.org
>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users
>>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at openser.org
>>>>> http://openser.org/cgi-bin/mailman/listinfo/users
>>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at openser.org
>> http://openser.org/cgi-bin/mailman/listinfo/users




More information about the Users mailing list