[OpenSIPS-Users] Memory Issues because of AVPs?
Anca Vamanu
anca at opensips.org
Fri Apr 8 10:56:13 CEST 2011
Hi Duane,
For the pua, pua_dialoginfo and the presence part, I advise you to
update your code from svn. Recently there has been a change in them for
exactly this - cleaning up the records faster to avoid memory filling up.
Regards,
--
Anca Vamanu
OpenSIPS Developer
On 04/06/2011 06:21 PM, duane.larson at gmail.com wrote:
> Sorry for the confusion. Orginally when I was testing my opensips
> config with the dialoginfo_set() function used in the script I was
> running into opensips memory (pk and sh) running out. This is most
> likely due to the fact that a ton of records were building up in the
> PUA table because opensips was generating pua message because of the
> dialoginfo_set() even though I had commented out everything else in my
> opensips script that had to do with pua and presence. I am hoping that
> when I am done with stress testing and enable all my presence and pua
> stuff the dialoginfo_set() functions will work as normal and not place
> a ton of records in the pua table (something I will need to verify
> later on).
>
> I think where I was getting confused was that when I did the top
> command I would see that my opensips processes were building up memory
> under the %MEM column and that number under %MEM was never decreasing
> after I waited 20 minutes. Yet last night after doing more research
> via the mailing list I ran across the command
>
> opensipsctl fifo get_statistics pkmem: shmem:
>
> When I look at the info from that commands output I see that SHMEM
> does get memory back instead of just steadily running out. I also see
> that with PKMEM all my processes are using about 1M of memory with 3M
> left over and that extra free memory doesn't get touched. So that
> leads me to believe that my opensips processes have also stabilized
> and aren't just gobbling up memory until it runs out. So even though
> the TOP command appears to show that the opensips processes aren't
> releasing that %MEM I can see with the opensipsctl fifo get_statistics
> command that pkmem and shmem isn't growing and being overloaded.
>
> Does that make more since? Everything for now appears to be good. I
> can run my SIPP test for a very long time and OpenSIPS doesn't bomb
> out on me with memory issues.
>
> On Apr 6, 2011 9:10am, Bogdan-Andrei Iancu <bogdan at opensips.org> wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Duane,
> >
> >
> >
> >
> >
> > trying to follow what you are saying :) - the mem leak you observe
> >
> > is for system memory (when you look with top) or is opensips memory
> >
> > (private or shared) ?
> >
> >
> >
> >
> >
> > Regards,
> >
> >
> > Bogdan
> >
> >
> >
> >
> >
> > On 04/06/2011 06:18 AM, Duane Larson wrote:
> >
> >
> >
> > OK after more testing it looks like my current test config
> >
> > with dialoginfo_set(); commented out is stable. My opensips
> >
> > processes grow to around 8.1 and 7.9 via the top command and
> >
> > stay there without growing anymore. Also I just found the
> >
> > following command
> >
> >
> >
> >
> >
> > opensipsctl fifo get_statistics pkmem: shmem:
> >
> >
> >
> >
> >
> > Didn't know about it. After looking at that I see that I
> >
> > have a good amount of free memory with all the pkmem's and the
> >
> > shmem.
> >
> >
> >
> >
> >
> > As for all the records in the PUA table that is probably
> >
> > because I have edited my script so much just to test calls per
> >
> > second. I had disabled all my presence processing but forgot
> >
> > to comment out the dialoginfo_set() function. Hopefully when I
> >
> > am done stress testing the PUA stuff will not be an issue.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Tue, Apr 5, 2011 at 6:02 PM, Duane
> >
> > Larson duane.larson at gmail.com>
> >
> > wrote:
> >
> >
> >
> >
> >
> > Thanks for the reply. I actually have been following the
> >
> > directions on
> >
> >
> >
> >
> > http://www.opensips.org/Resources/DocsTsMem
> >
> >
> >
> >
> >
> >
> > Perhaps my memory is building up because I am receiving a
> >
> > ton of the following in my PUA database table
> >
> >
> > mysql> select * from pua;
> >
> >
> >
> +--------+------------------+----------------+-------+---------+-----------------+------+------+----------+-------------+--------+---------+--------+----------+------+--------------+---------+----------------+---------+---------------+
> >
> >
> > | id | pres_uri | pres_id | event |
> >
> > expires | desired_expires | flag | etag | tuple_id |
> >
> > watcher_uri | to_uri | call_id | to_tag | from_tag | cseq |
> >
> > record_route | contact | remote_contact | version |
> >
> > extra_headers |
> >
> >
> >
> +--------+------------------+----------------+-------+---------+-----------------+------+------+----------+-------------+--------+---------+--------+----------+------+--------------+---------+----------------+---------+---------------+
> >
> >
> > | 641966 | sip:102 at test.com
> >
> > | DIALOG_PUBLISH | 64 | 0 | 1302067600 | 1024
> >
> > | | | | | |
> >
> > | | 0 | | |
> >
> > | 0 | |
> >
> >
> > | 642014 | sip:102 at test.com
> >
> > | DIALOG_PUBLISH | 64 | 0 | 1302067599 | 1024
> >
> > | | | | | |
> >
> > | | 0 | | |
> >
> > | 0 | |
> >
> >
> > This is because of the following function in my
> >
> > opensips.cfg script
> >
> >
> > dialoginfo_set(); (using this so I can have the BLF
> >
> > feature)
> >
> >
> > When I comment this out my script appears to run fine when
> >
> > I stress test it with SIPP. I have the following
> >
> >
> > children = 10
> >
> >
> > Server has 1 Gig of RAM
> >
> >
> > I configured my config.h to be "define PKG_MEM_POOL_SIZE
> >
> > 4*1024*1024"
> >
> >
> > I execute the Opensips service with 128M of memory
> >
> >
> > I run my SIPP test for about 5 minutes and 30 seconds and
> >
> > get the following outcome
> >
> >
> > Counter Name | Periodic value |
> >
> > Cumulative value
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Elapsed Time | 00:00:00:000 |
> >
> > 00:05:32:211
> >
> >
> > Call Rate | 0.000 cps |
> >
> > 18.455 cps
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Incoming call created | 0
> >
> > | 0
> >
> >
> > OutGoing call created | 0 |
> >
> > 6131
> >
> >
> > Total Call created | |
> >
> > 6131
> >
> >
> > Current Call | 0
> >
> > |
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Counter 5 | 0 |
> >
> > 6130
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Successful call | 0 |
> >
> > 6130
> >
> >
> > Failed call | 0
> >
> > | 1
> >
> >
> >
> >
> >
> > With this test I have nothing in my PUA table. Also at the
> >
> > end of this test I see that I have the folllowing when I
> >
> > execute the "top" command
> >
> >
> > Mem: 1022536k total, 785916k used, 236620k free,
> >
> > 29708k buffers
> >
> >
> > Swap: 2097148k total, 0k used, 2097148k free,
> >
> > 457508k cached
> >
> >
> > PID USER PR NI VIRT RES SHR S %CPU %MEM
> >
> > TIME+ COMMAND
> >
> >
> > 15327 mysql 20 0 333m 183m 10m S 1 18.4 9:24.24
> >
> > mysqld
> >
> >
> > 1084 root 20 0 63012 15m 2788 S 0 1.5 0:00.28
> >
> > opensips-mi-pro
> >
> >
> > 1096 root 20 0 73296 14m 3128 S 0 1.5 0:00.89
> >
> > media-dispatche
> >
> >
> > 18654 opensips 20 0 283m 11m 9404 S 0 1.2 0:00.74
> >
> > opensips
> >
> >
> > 18643 opensips 20 0 281m 11m 9464 S 0 1.2 0:01.07
> >
> > opensips
> >
> >
> > 18647 opensips 20 0 281m 11m 9420 S 0 1.2 0:01.10
> >
> > opensips
> >
> >
> > 18646 opensips 20 0 281m 11m 9376 S 0 1.2 0:01.09
> >
> > opensips
> >
> >
> > 18649 opensips 20 0 281m 11m 9320 S 0 1.2 0:01.10
> >
> > opensips
> >
> >
> > 18650 opensips 20 0 281m 11m 9284 S 0 1.1 0:01.12
> >
> > opensips
> >
> >
> > 18651 opensips 20 0 281m 11m 9276 S 0 1.1 0:01.10
> >
> > opensips
> >
> >
> > 18645 opensips 20 0 281m 11m 9272 S 0 1.1 0:01.10
> >
> > opensips
> >
> >
> > 18644 opensips 20 0 281m 11m 9140 S 0 1.1 0:01.10
> >
> > opensips
> >
> >
> > 18648 opensips 20 0 281m 11m 9076 S 0 1.1 0:01.08
> >
> > opensips
> >
> >
> > 18652 opensips 20 0 281m 11m 8972 S 0 1.1 0:01.11
> >
> > opensips
> >
> >
> > 18639 opensips 20 0 281m 8360 6096 S 0 0.8 0:00.09
> >
> > opensips
> >
> >
> > 18640 opensips 20 0 283m 4120 1500 S 0 0.4 0:00.08
> >
> > opensips
> >
> >
> > 18656 opensips 20 0 283m 4044 1760 S 0 0.4 0:00.12
> >
> > opensips
> >
> >
> > 18641 opensips 20 0 281m 3448 1144 S 0 0.3 0:00.00
> >
> > opensips
> >
> >
> > 18642 opensips 20 0 281m 3224 960 S 0 0.3 0:00.00
> >
> > opensips
> >
> >
> > 18655 opensips 20 0 281m 3208 940 S 0 0.3 0:00.07
> >
> > opensips
> >
> >
> > 18653 opensips 20 0 281m 2320 88 S 0 0.2 0:00.51
> >
> > opensips
> >
> >
> >
> >
> >
> >
> >
> >
> > Then I run a second test with SIPP and get the following
> >
> > Counter Name | Periodic value |
> >
> > Cumulative value
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Elapsed Time | 00:00:00:000 |
> >
> > 00:13:35:590
> >
> >
> > Call Rate | 0.000 cps |
> >
> > 77.586 cps
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Incoming call created | 0
> >
> > | 0
> >
> >
> > OutGoing call created | 0 |
> >
> > 63278
> >
> >
> > Total Call created | |
> >
> > 63278
> >
> >
> > Current Call | 0
> >
> > |
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Counter 5 | 0 |
> >
> > 63201
> >
> >
> >
> -------------------------+---------------------------+--------------------------
> >
> >
> > Successful call | 0 |
> >
> > 63201
> >
> >
> > Failed call | 0 |
> >
> > 77
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > And then when I do the "top" command again I see the
> >
> > following
> >
> >
> >
> >
> >
> >
> >
> >
> > Mem: 1022536k total, 675176k used, 347360k
> >
> > free, 30956k buffers
> >
> >
> > Swap: 2097148k total, 0k used, 2097148k free,
> >
> > 345944k cached
> >
> >
> > PID USER PR NI VIRT RES SHR S %CPU %MEM
> >
> > TIME+ COMMAND
> >
> >
> > 15327 mysql 20 0 333m 183m 10m S 1 18.4
> >
> > 12:29.06 mysqld
> >
> >
> > 18643 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.51 opensips
> >
> >
> > 18644 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.26 opensips
> >
> >
> > 18647 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.42 opensips
> >
> >
> > 18646 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.34 opensips
> >
> >
> > 18648 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.41 opensips
> >
> >
> > 18649 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.27 opensips
> >
> >
> > 18650 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.32 opensips
> >
> >
> > 18651 opensips 20 0 281m 28m 26m S 0 2.9
> >
> > 0:12.28 opensips
> >
> >
> > 18645 opensips 20 0 281m 28m 26m S 0 2.8
> >
> > 0:12.35 opensips
> >
> >
> > 18652 opensips 20 0 281m 28m 26m S 0 2.8
> >
> > 0:12.32 opensips
> >
> >
> > 18654 opensips 20 0 283m 26m 24m S 0 2.7
> >
> > 0:02.51 opensips
> >
> >
> > 1084 root 20 0 63012 15m 2788 S 0 1.5
> >
> > 0:00.28 opensips-mi-pro
> >
> >
> > 1096 root 20 0 73296 14m 3128 S 0 1.5
> >
> > 0:00.89 media-dispatche
> >
> >
> > 18642 opensips 20 0 281m 9860 7404 S 0 1.0
> >
> > 0:00.15 opensips
> >
> >
> > 18639 opensips 20 0 281m 8360 6096 S 0 0.8
> >
> > 0:00.09 opensips
> >
> >
> > 18640 opensips 20 0 283m 4192 1572 S 0 0.4
> >
> > 0:00.15 opensips
> >
> >
> > 18656 opensips 20 0 283m 4084 1800 S 0 0.4
> >
> > 0:00.25 opensips
> >
> >
> > 18641 opensips 20 0 281m 3448 1144 S 0 0.3
> >
> > 0:00.00 opensips
> >
> >
> > 18655 opensips 20 0 281m 3208 940 S 0 0.3
> >
> > 0:00.15 opensips
> >
> >
> > 18653 opensips 20 0 281m 2320 88 S 0 0.2
> >
> > 0:01.06 opensips
> >
> >
> >
> >
> >
> > So a lot of the opensips processes have about 2.9%
> >
> > memory. I wait more than 20 minutes to see if the
> >
> > memory would be released but it didn't.
> >
> >
> > Not sure if this is a bad thing or not. I could let my
> >
> > SIPP test just keep running to see if OpenSIPS crashes
> >
> > from no memory if the memory keeps creeping up or I
> >
> > could do a "kill -SIGUSR1 18643" to see what is
> >
> > currently using the memory? Any thoughts or is this not
> >
> > an issue at all?
> >
More information about the Users
mailing list