[OpenSIPS-Users] OpenSIPS core dumps

thrillerbee thrillerbee at gmail.com
Fri Nov 19 15:47:30 CET 2010


Bogdan,

Are these memlogs useful?  Please let me know if I can provide any other
info.

Thanks again.

On Sat, Nov 13, 2010 at 4:20 PM, thrillerbee <thrillerbee at gmail.com> wrote:

> Bogdan,
>
> Since I made those changes, it doesn't core dump anymore - it just runs out
> of memory and stops processing packets.  It happened on both of my proxies -
> the mem dump can be downloaded here:
> http://www.starviewconnect.com/tmp/core1_mem_dump_20101112.gz
> http://www.starviewconnect.com/tmp/core2_mem_dump_20101112.gz
>
> Thanks.
>
> On Thu, Nov 11, 2010 at 12:43 PM, thrillerbee <thrillerbee at gmail.com>wrote:
>
>> Bogdan,
>>
>> I had already increased the PKG_MEM_POOL_SIZE by *10.  I'm making the
>> changes suggested in the link you provided to try to narrow down the memory
>> issue.  I'll send over as soon as I have another crash with mem info.
>>
>> Thanks.
>>
>>
>> On Thu, Nov 11, 2010 at 5:10 AM, Bogdan-Andrei Iancu <
>> bogdan at voice-system.ro> wrote:
>>
>>> This last crash had the same bt as the one from previous email ? the prev
>>> fix I made reports this:
>>>
>>>         ERROR:db_flatstore:flat_db_insert: uninitialized connection
>>> and does not crash, so this time the crash may be in a different place.
>>>
>>> Now, it seams after all that the root of your problem is the mem
>>> exhaustion. To see what  causes this  (if a mem leak or simply not enough
>>> mem), there is a doc - http://www.opensips.org/Resources/DocsTsMem (it
>>> is for shm, but the same for pkg).
>>>
>>> Maybe, first you should simply try to increase the pkg mem ( in config.h
>>> you have PKG_MEM_POOL_SIZE that you can increase) - you NEED to recompile
>>> and reinstall after that.
>>>
>>> If more pkg mem does not solve the problem, I can help you with using the
>>> memory debugger to see what is going on there.
>>>
>>> Regards,
>>> Bogdan
>>>
>>>
>>> thrillerbee wrote:
>>>
>>>> My other proxy crashed as well with these ERRORs in the syslog:
>>>>
>>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]:
>>>> ERROR:db_flatstore:get_name: pkg memory allocation failure
>>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]:
>>>> ERROR:db_flatstore:flat_reopen_connection: failed to get_name
>>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]:
>>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection
>>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]:
>>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection
>>>> ...
>>>> Nov 10 22:01:21 core2 /usr/local/sbin/opensips[22959]:
>>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection
>>>> Nov 10 22:01:22 core2 /usr/local/sbin/opensips[22959]:
>>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection
>>>> Nov 10 22:01:22 core2 /usr/local/sbin/opensips[22959]:
>>>> ERROR:db_flatstore:new_flat_id: no pkg memory left
>>>> Nov 10 22:01:22 core2 kernel: [4297088.404734] opensips[22959]: segfault
>>>> at 10 ip 7f3db577e21f sp 7fffa260d640 error 4 in
>>>> db_flatstore.so[7f3db577b000+5000]
>>>>
>>>> On Wed, Nov 10, 2010 at 10:19 AM, thrillerbee <thrillerbee at gmail.com<mailto:
>>>> thrillerbee at gmail.com>> wrote:
>>>>
>>>>    Bogdan,
>>>>
>>>>    Well, I spoke too soon - it's not just an issue with the
>>>>    opensipsctl fifo calls - looks more like a memory leak.  It
>>>>    crashed again today, but I did get some errors in the syslog this
>>>>    time right before the crash:
>>>>    Nov 10 15:42:32 core1 /usr/local/sbin/opensips[27044]:
>>>>    ERROR:db_flatstore:new_flat_id: no pkg memory left
>>>>    Nov 10 15:42:32 core1 kernel: [5508366.582447] opensips[27044]:
>>>>    segfault at 10 ip 7fa7ff74c21f sp 7fffdc101700 error 4 in
>>>>    db_flatstore.so[7fa7ff749000+5000]
>>>>    To be thorough, I've attached the backtrace & output from print
>>>>    commands (although they're the same as before).
>>>>
>>>>    To answer your question, yes - I do use the flat_rotate MI command.
>>>>
>>>>    Thanks again.
>>>>
>>>>    On Wed, Nov 10, 2010 at 4:04 AM, Bogdan-Andrei Iancu
>>>>    <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>> wrote:
>>>>
>>>>        Hi,
>>>>
>>>>        opensipsctl takes care that each command takes a separate fifo
>>>>        reply, so here it should be no problem. But the problem may be
>>>>        when comes with sending multiple commands (via FIFO) in the
>>>>        same time - this translates into parallel writes to the same
>>>>        file and depends on the atomicity of the write op.
>>>>
>>>>        But in the worst case, a mixture at the FIFO level may lead to
>>>>        bogus command and not in any kind of crash....Do you use the
>>>>        "flat_rotate" MI command ?
>>>>
>>>>        Regards,
>>>>        Bogdan
>>>>
>>>>        thrillerbee wrote:
>>>>
>>>>            Bogdan,
>>>>
>>>>            It seems the issue is with 'opensipsctl fifo' - it's very
>>>>            sensitive to simultaneous calls.  Basically, I've combined
>>>>            all my scripts to prevent 'opensipsctl fifo' from being
>>>>            called too frequently and that seems (so far) to have
>>>>            mitigated the issue.  Is there anything one should know
>>>>            about how (not) to use /opensipsctl/?
>>>>
>>>>            Thanks.
>>>>
>>>>            On Mon, Nov 8, 2010 at 6:07 AM, Bogdan-Andrei Iancu
>>>>            <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>
>>>>            <mailto:bogdan at voice-system.ro
>>>>            <mailto:bogdan at voice-system.ro>>> wrote:
>>>>
>>>>               Hi,
>>>>
>>>>               strange if you do not have any errors :(....
>>>>
>>>>               I just made a fix on both trunk and 1.6 to extend some
>>>>            checks in
>>>>               flatstore and prevent crashing (even if the DB op will
>>>>            not be
>>>>               executed).
>>>>
>>>>               Could you update from SVN and see if stops crashing ?
>>>>
>>>>               Regards,
>>>>               Bogdan
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opensips.org
>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>>>>
>>>
>>>
>>> --
>>> Bogdan-Andrei Iancu
>>> OpenSIPS Bootcamp
>>> 15 - 19 November 2010, Edison, New Jersey, USA
>>> www.voice-system.ro
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opensips.org
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20101119/e642d1bf/attachment-0001.htm>


More information about the Users mailing list