[OpenSIPS-Users] How to avoid increasingly memory comsuption with AVPs?

Liviu Chircu liviu at opensips.org
Mon Apr 18 10:39:24 CEST 2016


The generic DB backend from OpenSIPS is built to re-use any connection. 
Just to make it clear, from a network point of view, SQLite is 
connection-less, since, after all, it's just a bunch of files with some 
libraries that work on top of them.

Regarding the leak, sqlite3_open() is not an OpenSIPS function, and it 
looks to be called only in the init phase, which is a good thing at 
least. If we want to get it fixed, we should first confirm it, then 
escalate it to the SQLite project.

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

On 15.04.2016 19:32, Rodrigo Pimenta Carvalho wrote:
>
> Hi Liviu.
>
>
> Thank you very much for answering my question and for these hints.
>
> I will talk to my team about to consider using OpenSIPS 2.2+. I will 
> also read more about "-M" flag and see how should I use it.
>
>
> I have one more question:
>
> Is there one more opened connection to the Sqlite database every time 
> when my script executes a qvp_db_query?  If yes, these connections 
> remain opened or it is closed immediately after the query terminates?
>
> What I need to know is if the OpenSIPS is creating too many 
> connections to the database.
>
>
> Today, with valgrind I saw this log:
>
>
> ==501== 167,528 (848 direct, 166,680 indirect) bytes in 1 blocks are 
> definitely lost in loss record 425 of 428
> ==501==    at 0x4C2AB80: malloc (in 
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==501==    by 0x738EDC6: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7369569: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7371C47: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7371C6C: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x73EF012: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7132D9A: db_sqlite_connect (my_con.c:62)
> ==501==    by 0x7133314: db_sqlite_new_connection (my_con.c:134)
> ==501==    by 0x5693A3: db_do_init (db.c:309)
> ==501==    by 0x7A5F606: register_udomain (dlist.c:655)
> ==501==    by 0x7C9014D: domain_fixup (reg_mod.c:386)
> ==501==    by 0x7C9014D: registrar_fixup (reg_mod.c:421)
> ==501==    by 0x4A731B: fix_actions (route.c:459)
> ==501==
> ==501== 169,216 (848 direct, 168,368 indirect) bytes in 1 blocks are 
> definitely lost in loss record 426 of 428
> ==501==    at 0x4C2AB80: malloc (in 
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==501==    by 0x738EDC6: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7369569: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7371C47: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7371C6C: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x73EF012: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7132D9A: db_sqlite_connect (my_con.c:62)
> ==501==    by 0x7133314: db_sqlite_new_connection (my_con.c:134)
> ==501==    by 0x5693A3: db_do_init (db.c:309)
> ==501==    by 0x643CFF4: domain_db_init (domain.c:65)
> ==501==    by 0x643E8C8: mod_init (domain_mod.c:232)
> ==501==    by 0x4C8EF8: init_mod (sr_module.c:632)
> ==501==
> ==501== 328,352 (1,696 direct, 326,656 indirect) bytes in 2 blocks are 
> definitely lost in loss record 427 of 428
> ==501==    at 0x4C2AB80: malloc (in 
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==501==    by 0x738EDC6: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7369569: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7371C47: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7371C6C: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x73EF012: ??? (in 
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==501==    by 0x7132D9A: db_sqlite_connect (my_con.c:62)
> ==501==    by 0x7133314: db_sqlite_new_connection (my_con.c:134)
> ==501==    by 0x5693A3: db_do_init (db.c:309)
> ==501==    by 0x8577C0E: auth_fixup (authdb_mod.c:257)
> ==501==    by 0x4A731B: fix_actions (route.c:459)
> ==501==    by 0x4AA7EF: fix_expr (route.c:214)
> ==501==
> ==501== LEAK SUMMARY:
> ==501==    definitely lost: 4,708 bytes in 8 blocks
> ==501==    indirectly lost: 825,032 bytes in 6,077 blocks
> ==501==      possibly lost: 328,824 bytes in 2,431 blocks
> ==501==    still reachable: 4,259,419 bytes in 234 blocks
> ==501==         suppressed: 0 bytes in 0 blocks
> ==501== Reachable blocks (those to which a pointer was found) are not 
> shown.
> ==501== To see them, rerun with: --leak-check=full --show-leak-kinds=all
>
> Any hint will be very helpful!
>
> Best regards.
>
>
>
> RODRIGO PIMENTA CARVALHO
> Inatel Competence Center
> Software
> Ph: +55 35 3471 9200 RAMAL 979
>
>
> ------------------------------------------------------------------------
> *De:* users-bounces at lists.opensips.org 
> <users-bounces at lists.opensips.org> em nome de Liviu Chircu 
> <liviu at opensips.org>
> *Enviado:* sexta-feira, 15 de abril de 2016 11:20
> *Para:* users at lists.opensips.org
> *Assunto:* Re: [OpenSIPS-Users] How to avoid increasingly memory 
> comsuption with AVPs?
> Hi Rodrigo,
>
> That's actually PKG memory, so you should fine tune the "-M" parameter 
> according to your data and any instructions found in the SQLite module.
>
> As a side note, newer versions of OpenSIPS (2.2+) will properly 
> indicate which type of memory you ran out of, so troubleshooting 
> memory issues will become easier if you do the upgrade.
> Liviu Chircu
> OpenSIPS Developer
> http://www.opensips-solutions.com
> <http://www.opensips-solutions.com/>
> 	
> Home — OpenSIPS Solutions <http://www.opensips-solutions.com/>
> www.opensips-solutions.com
> OpenSIPS is a mature Open Source implementation of a SIP server. 
> OpenSIPS is more than a SIP proxy/router as it includes 
> application-level functionalities.
>
> On 15.04.2016 16:50, Rodrigo Pimenta Carvalho wrote:
>>
>>
>> Hi.
>>
>>
>> After a stress test with my OpenSIPS and few user agent clients 
>> on-line, just sending SIP REGISTER messages (1 at each minute), not 
>> making calls, I saw this log:
>>
>>
>>
>> Apr 15 10:09:20 [20441] WARNING:core:fm_malloc: Not enough free 
>> memory, will attempt defragmentation
>> Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_allocate_rows: no 
>> memory left
>> Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_convert_rows: no 
>> private memory left
>> Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_convert_result: 
>> error while converting rows
>> Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_store_result: error 
>> while converting result
>> Apr 15 10:09:20 [20441] ERROR:avpops:db_query_avp: raw_query failed: 
>> db0(usr_preferences) select Value from GeneralConfigurations ...
>>
>>
>> So, I suspect that I have codified something wrong in my opensips.cfg 
>> file.
>>
>> More precisely, I have used some AVPs and I'm note sure if  such use 
>> is correct or if it is the cause of increasingly memory consumption.
>>
>> For example, I have codified somethings like this:
>>
>>
>>
>> 1        if($avp(I_A) == NULL){
>>                  avp_db_query("select Value from 
>> GeneralConfigurations where Attribute = 
>> 'CONFIGURATION_INTERCOM_A_NAME'", "$avp(I_A)");
>>         }
>>
>>
>> .
>>
>> .
>>
>> .
>>
>>
>> 2        if ($rU==$avp(M_TU)){
>>            $rU = $avp(M_T);
>>         }
>>
>> .
>>
>> .
>>
>> .
>>
>>
>> 3    if (nat_uac_test("34")) {
>>         if (is_method("REGISTER")) {
>>             fix_nated_register();
>>             setbflag(NAT);
>>             $avp(attr) = "in_another_network";
>>         } else {
>>             fix_nated_contact();
>>             setflag(NAT);
>>         }
>>     }
>>
>>
>> .
>>
>> .
>>
>> .
>>
>> 4                $avp(ContatoRegister) = $(ct.fields(uri){s.select,0,;});
>>                   if($fn != NULL){
>>                      avp_db_query("UPDATE location SET 
>> callerName='$fn'  where contact like '$avp(ContatoRegister)%'");
>>                 }
>>                 $avp(ContatoRegister) = NULL;
>>
>>
>>
>>
>> Could some of the 4 piece of codes above causes some increasingly 
>> memory consumption?
>>
>>
>> Any hint will be very helpful!
>>
>>
>> Best regards.
>>
>>
>>
>> RODRIGO PIMENTA CARVALHO
>> Inatel Competence Center
>> Software
>> Ph: +55 35 3471 9200 RAMAL 979
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20160418/b8e6a44b/attachment-0001.htm>


More information about the Users mailing list