[OpenSIPS-Users] Opensips 1.6 crash

Bogdan-Andrei Iancu bogdan at voice-system.ro
Wed Jan 20 19:26:12 CET 2010


yeah, sometimes a double coredump happens: a worker process crashes due 
a bug -> opensips starts shutdown in main process -> this one crashes 
too mainly because inconsistency in data/mem/locks from the first 
crashed process.

I got your email -> I will take a look.

Thanks and regards,
Bogdan

Jeff Pyle wrote:
> Sorry, you asked for logs.  Here they are:
>
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: child process 27595 exited by a signal 11 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: terminating due to SIGCHLD 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27588]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27597]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27596]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27598]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27604]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27605]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27572]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27580]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27570]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27573]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27581]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27577]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27587]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27578]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27579]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27589]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27590]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27591]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27606]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27571]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27594]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27607]: INFO:core:sig_usr: signal 15 received 
> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:get_new_stmt_ctx: disconect event for 0x8248390 
> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:reset_all_statements: reseting all statements on connection: (0x8248798) 0x8248390 
> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:get_new_stmt_ctx: re-connected successful for 0x8248390 
>
> I wrote it very poorly before... there were two cores dumped.  I posted the backtrace for the one mentioned in the syslog.  The backtrace for other I can send you privately.
>
>
> Thanks,
> Jeff
>
>
> On Jan 20, 2010, at 12:56 PM, Jeff Pyle wrote:
>
>   
>> Hi Bogdan,
>>
>> The only mention of a core in the syslog was the 27560 one:
>>
>> /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated
>>
>> The backtrace I sent was for this one.  The only other core generated.  Its backtrace is significantly longer.  Because of the semi-private nature of some of the headers it contains I can send it to you privately if you think it would be helpful.
>>
>>
>> - Jeff
>>
>>
>> On Jan 20, 2010, at 12:36 PM, Bogdan-Andrei Iancu wrote:
>>
>>     
>>> Hi Jeff,
>>>
>>> For the records, if no otherwise configured (via the -w cli param), 
>>> opensips (when forking) will change the working dir to /   (root 
>>> filesystem).
>>>
>>> Now, about the core- the back trace you see here is showing a crash 
>>> during shutdown (main->handle_sigs->cleanup...). So, you need to 
>>> identify if indeed the crash was during a normal shutdown or if the 
>>> crash you have the core for is just a side effect of a previous crash of 
>>> another process (that triggered shutdown).
>>>
>>> Could you post the relevant (to crash) logs ?
>>>
>>> Regards,
>>> Bogdan
>>>
>>> Jeff Pyle wrote:
>>>       
>>>> Never mind, I'm a little slow today.  I found /core.27560.  A backtrace shows the following:
>>>>
>>>> #0  0x00a16827 in free () from /lib/i686/nosegneg/libc.so.6
>>>> No symbol table info available.
>>>> #1  0x0017e021 in my_no_flags_free (ptr=0x9b00000) at my_malloc.c:59
>>>> No locals.
>>>> #2  0x0017f863 in free_root (root=0x9b1dc30, MyFlags=0) at my_alloc.c:355
>>>> 	next = <value optimized out>
>>>> #3  0x0017b218 in mysql_stmt_close (stmt=0x9bffd00) at libmysql.c:5058
>>>> 	mysql = (MYSQL *) 0x82483c4
>>>> #4  0x0016cc2c in db_mysql_free_stmt_list (head=0x0) at dbase.c:160
>>>> No locals.
>>>> #5  0x00173186 in db_mysql_free_connection (con=0x8248390) at my_con.c:138
>>>> No locals.
>>>> #6  0x08126449 in db_do_close (_h=0x82482d4, free_connection=0x17315c <db_mysql_free_connection>) at db/db.c:310
>>>> 	con = (struct pool_con *) 0x8248390
>>>> 	__FUNCTION__ = "db_do_close"
>>>> #7  0x0016cbb3 in db_mysql_close (_h=0x82482d4) at dbase.c:583
>>>> No locals.
>>>> #8  0x00b537ae in destroy () at ul_mod.c:369
>>>> 	__FUNCTION__ = "destroy"
>>>> #9  0x080be211 in destroy_modules () at sr_module.c:370
>>>> 	t = (struct sr_module *) 0x81b7cc8
>>>> 	foo = (struct sr_module *) 0x81b7c20
>>>> #10 0x0806b0a0 in cleanup (show_status=1) at main.c:336
>>>> No locals.
>>>> #11 0x0806bfd4 in handle_sigs () at main.c:533
>>>> 	chld = 0
>>>> 	chld_status = 139
>>>> 	i = 6
>>>> 	do_exit = 1
>>>> ---Type <return> to continue, or q <return> to quit---
>>>> 	__FUNCTION__ = "handle_sigs"
>>>> #12 0x08070564 in main (argc=3, argv=0xbfdc6654) at main.c:913
>>>> 	cfg_log_stderr = 0
>>>> 	cfg_stream = (FILE *) 0x9b0a008
>>>> 	c = <value optimized out>
>>>> 	r = 0
>>>> 	tmp = 0xbfdc7f74 ""
>>>> 	tmp_len = <value optimized out>
>>>> 	port = 10309893
>>>> 	proto = <value optimized out>
>>>> 	ret = <value optimized out>
>>>> 	seed = 266001097
>>>> 	rfd = 4
>>>> 	__FUNCTION__ = "main"
>>>>
>>>>
>>>> This is on 1.6 rev 6511.  I the mysql libraries come from RPMs installed, version 6.3.20 (cluster) from Mysql's site.
>>>>
>>>> Any thoughts?
>>>>
>>>>
>>>> Thanks,
>>>> Jeff
>>>>
>>>>
>>>>
>>>> On Jan 20, 2010, at 12:00 PM, Jeff Pyle wrote:
>>>>
>>>>
>>>>         
>>>>> Hello,
>>>>>
>>>>> Opensips 1.6 dumped on me.  I see the following in the syslog:
>>>>>
>>>>> /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated
>>>>>
>>>>> Where might this file be, and what might it be called?  I'm having trouble locating it.  I start Opensips with the Fedora init.d file, calling the daemon like this:
>>>>>
>>>>> daemon $oser -m 128 $OPTIONS 2>/dev/null | tail -1
>>>>>
>>>>> There is nothing in $OPTIONS.  $oser = /usr/local/sbin/opensips
>>>>>
>>>>> I've seen cores dump into /tmp before but that's not the case this time.  Any suggestions would be appreciated it.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Jeff  
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at lists.opensips.org
>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>>
>>>>>           
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opensips.org
>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>>>>
>>>>         
>>> -- 
>>> Bogdan-Andrei Iancu
>>> www.voice-system.ro
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opensips.org
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>       
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>   


-- 
Bogdan-Andrei Iancu
www.voice-system.ro




More information about the Users mailing list