[OpenSIPS-Users] How to avoid increasingly memory comsuption with AVPs?

Rodrigo Pimenta Carvalho pimenta at inatel.br
Mon Apr 18 14:10:09 CEST 2016


Hi Liviu.


Thank you very much!

So, I'm comfortable with OpenSIPS.


In my OpenSIPS config file I have:


loadmodule "usrloc.so"
modparam("usrloc", "db_mode",   2)


Can I change db_mode to zero and still have every avp_db_query working well?

That is,  with db_mode=0 I will avoid using Sqlite and every SQL operation over user location will apply just over data in memory cache. Am I correct?


Any hint will be very helpful!


Thanks a lot.


RODRIGO PIMENTA CARVALHO
Inatel Competence Center
Software
Ph: +55 35 3471 9200 RAMAL 979


________________________________
De: users-bounces at lists.opensips.org <users-bounces at lists.opensips.org> em nome de Liviu Chircu <liviu at opensips.org>
Enviado: segunda-feira, 18 de abril de 2016 05:39
Para: users at lists.opensips.org
Assunto: Re: [OpenSIPS-Users] How to avoid increasingly memory comsuption with AVPs?

The generic DB backend from OpenSIPS is built to re-use any connection. Just to make it clear, from a network point of view, SQLite is connection-less, since, after all, it's just a bunch of files with some libraries that work on top of them.

Regarding the leak, sqlite3_open() is not an OpenSIPS function, and it looks to be called only in the init phase, which is a good thing at least. If we want to get it fixed, we should first confirm it, then escalate it to the SQLite project.

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

[http://www.opensips-solutions.com/imgs/slideshow/slide3.jpg]<http://www.opensips-solutions.com/>

Home — OpenSIPS Solutions<http://www.opensips-solutions.com/>
www.opensips-solutions.com
OpenSIPS is a mature Open Source implementation of a SIP server. OpenSIPS is more than a SIP proxy/router as it includes application-level functionalities.

On 15.04.2016 19:32, Rodrigo Pimenta Carvalho wrote:

Hi Liviu.


Thank you very much for answering my question and for these hints.

I will talk to my team about to consider using OpenSIPS 2.2+. I will also read more about "-M" flag and see how should I use it.


I have one more question:

Is there one more opened connection to the Sqlite database every time when my script executes a qvp_db_query?  If yes, these connections remain opened or it is closed immediately after the query terminates?

What I need to know is if the OpenSIPS is creating too many connections to the database.


Today, with valgrind I saw this log:


==501== 167,528 (848 direct, 166,680 indirect) bytes in 1 blocks are definitely lost in loss record 425 of 428
==501==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==501==    by 0x738EDC6: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7369569: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7371C47: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7371C6C: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x73EF012: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7132D9A: db_sqlite_connect (my_con.c:62)
==501==    by 0x7133314: db_sqlite_new_connection (my_con.c:134)
==501==    by 0x5693A3: db_do_init (db.c:309)
==501==    by 0x7A5F606: register_udomain (dlist.c:655)
==501==    by 0x7C9014D: domain_fixup (reg_mod.c:386)
==501==    by 0x7C9014D: registrar_fixup (reg_mod.c:421)
==501==    by 0x4A731B: fix_actions (route.c:459)
==501==
==501== 169,216 (848 direct, 168,368 indirect) bytes in 1 blocks are definitely lost in loss record 426 of 428
==501==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==501==    by 0x738EDC6: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7369569: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7371C47: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7371C6C: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x73EF012: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7132D9A: db_sqlite_connect (my_con.c:62)
==501==    by 0x7133314: db_sqlite_new_connection (my_con.c:134)
==501==    by 0x5693A3: db_do_init (db.c:309)
==501==    by 0x643CFF4: domain_db_init (domain.c:65)
==501==    by 0x643E8C8: mod_init (domain_mod.c:232)
==501==    by 0x4C8EF8: init_mod (sr_module.c:632)
==501==
==501== 328,352 (1,696 direct, 326,656 indirect) bytes in 2 blocks are definitely lost in loss record 427 of 428
==501==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==501==    by 0x738EDC6: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7369569: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7371C47: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7371C6C: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x73EF012: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==501==    by 0x7132D9A: db_sqlite_connect (my_con.c:62)
==501==    by 0x7133314: db_sqlite_new_connection (my_con.c:134)
==501==    by 0x5693A3: db_do_init (db.c:309)
==501==    by 0x8577C0E: auth_fixup (authdb_mod.c:257)
==501==    by 0x4A731B: fix_actions (route.c:459)
==501==    by 0x4AA7EF: fix_expr (route.c:214)
==501==
==501== LEAK SUMMARY:
==501==    definitely lost: 4,708 bytes in 8 blocks
==501==    indirectly lost: 825,032 bytes in 6,077 blocks
==501==      possibly lost: 328,824 bytes in 2,431 blocks
==501==    still reachable: 4,259,419 bytes in 234 blocks
==501==         suppressed: 0 bytes in 0 blocks
==501== Reachable blocks (those to which a pointer was found) are not shown.
==501== To see them, rerun with: --leak-check=full --show-leak-kinds=all


Any hint will be very helpful!

Best regards.



RODRIGO PIMENTA CARVALHO
Inatel Competence Center
Software
Ph: +55 35 3471 9200 RAMAL 979


________________________________
De: users-bounces at lists.opensips.org<mailto:users-bounces at lists.opensips.org> <users-bounces at lists.opensips.org><mailto:users-bounces at lists.opensips.org> em nome de Liviu Chircu <liviu at opensips.org><mailto:liviu at opensips.org>
Enviado: sexta-feira, 15 de abril de 2016 11:20
Para: users at lists.opensips.org<mailto:users at lists.opensips.org>
Assunto: Re: [OpenSIPS-Users] How to avoid increasingly memory comsuption with AVPs?

Hi Rodrigo,

That's actually PKG memory, so you should fine tune the "-M" parameter according to your data and any instructions found in the SQLite module.

As a side note, newer versions of OpenSIPS (2.2+) will properly indicate which type of memory you ran out of, so troubleshooting memory issues will become easier if you do the upgrade.

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

[http://www.opensips-solutions.com/imgs/opensips-solutions-logo.gif]<http://www.opensips-solutions.com/>

Home — OpenSIPS Solutions<http://www.opensips-solutions.com/>
www.opensips-solutions.com<http://www.opensips-solutions.com>
OpenSIPS is a mature Open Source implementation of a SIP server. OpenSIPS is more than a SIP proxy/router as it includes application-level functionalities.

On 15.04.2016 16:50, Rodrigo Pimenta Carvalho wrote:


Hi.


After a stress test with my OpenSIPS and few user agent clients on-line, just sending SIP REGISTER messages (1 at each minute), not making calls, I saw this log:



Apr 15 10:09:20 [20441] WARNING:core:fm_malloc: Not enough free memory, will attempt defragmentation
Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_allocate_rows: no memory left
Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_convert_rows: no private memory left
Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_convert_result: error while converting rows
Apr 15 10:09:20 [20441] ERROR:db_sqlite:db_sqlite_store_result: error while converting result
Apr 15 10:09:20 [20441] ERROR:avpops:db_query_avp: raw_query failed: db0(usr_preferences) select Value from GeneralConfigurations ...


So, I suspect that I have codified something wrong in my opensips.cfg file.

More precisely, I have used some AVPs and I'm note sure if  such use is correct or if it is the cause of increasingly memory consumption.

For example, I have codified somethings like this:



1        if($avp(I_A) == NULL){
                 avp_db_query("select Value from GeneralConfigurations where Attribute = 'CONFIGURATION_INTERCOM_A_NAME'", "$avp(I_A)");
        }


.

.

.


2        if ($rU==$avp(M_TU)){
           $rU = $avp(M_T);
        }

.

.

.


3    if (nat_uac_test("34")) {
        if (is_method("REGISTER")) {
            fix_nated_register();
            setbflag(NAT);
            $avp(attr) = "in_another_network";
        } else {
            fix_nated_contact();
            setflag(NAT);
        }
    }


.

.

.

4                $avp(ContatoRegister) = $(ct.fields(uri){s.select,0,;});
                  if($fn != NULL){
                     avp_db_query("UPDATE location SET callerName='$fn'  where contact like '$avp(ContatoRegister)%'");
                }
                $avp(ContatoRegister) = NULL;




Could some of the 4 piece of codes above causes some increasingly memory consumption?


Any hint will be very helpful!


Best regards.



RODRIGO PIMENTA CARVALHO
Inatel Competence Center
Software
Ph: +55 35 3471 9200 RAMAL 979



_______________________________________________
Users mailing list
Users at lists.opensips.org<mailto:Users at lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/users





_______________________________________________
Users mailing list
Users at lists.opensips.org<mailto:Users at lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20160418/926ae7ca/attachment-0001.htm>


More information about the Users mailing list