[opensips] OpenSIPS 2.1 Crash (#732)

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2

Hi,

I recently installed opensips version 21 with git revision a173499 on a 32bit debian wheezy and it is keep getting crashed while acquiring lock in TM module Here is BT given by GDB,

Core was generated by '/usr/local/adx-ipc/sbin/opensips -P /var/run/opensips/opensipspid -m 256 -M 16'
Program terminated with signal 11, Segmentation fault
#0  0xb660a188 in tsl (lock=0x0) at //mem//fastlockh:86

warning: Source file is more recent than executable
▒▒/usr/src/svn-src/opensips/fastlockh:86:2617:beg:0xb660a188
(gdb) bt
#0  0xb660a188 in tsl (lock=0x0) at //mem//fastlockh:86
#1  get_lock (lock=) at //mem//fastlockh:176
#2  _lock (s=) at lockh:100
#3  set_timer (ext_timeout=0x0, list_id=RT_T1_TO_1, new_tl=0xa61072c0) at timerc:899
#4  set_timer (new_tl=new_tl@entry=0xa61072c0, list_id=list_id@entry=RT_T1_TO_1, ext_timeout=0x0, ext_timeout@entry=0x1) at timerc:876
#5  0xb66078db in _set_fr_retr (retr=, rb=0xa610728c) at t_funcsh:137
#6  start_retr (rb=0xa610728c) at t_funcsh:151
#7  t_forward_nonack (t=t@entry=0xa6107174, p_msg=p_msg@entry=0xb66aa29c, proxy=proxy@entry=0x0) at t_fwdc:728
#8  0xb6602f6d in t_relay_to (p_msg=p_msg@entry=0xb66aa29c, proxy=proxy@entry=0x0, flags=flags@entry=0) at t_funcsc:247
#9  0xb6619814 in w_t_relay (p_msg=0xb66aa29c, proxy=0x0, flags=0x0) at tmc:1213
#10 0x0806441c in do_action (a=a@entry=0xb66752d0, msg=msg@entry=0xb66aa29c) at actionc:1862
#11 0x0806bd34 in run_action_list (a=0xb66752d0, msg=msg@entry=0xb66aa29c) at actionc:169
#12 0x080dbb48 in eval_elem (e=0xb6675344, msg=msg@entry=0xb66aa29c, val=val@entry=0x0) at routec:1643
#13 0x080db439 in eval_expr (e=0xb6675344, msg=msg@entry=0xb66aa29c, val=val@entry=0x0) at routec:1988
#14 0x080db567 in eval_expr (e=0xb6675370, msg=msg@entry=0xb66aa29c, val=val@entry=0x0) at routec:2004
#15 0x080db421 in eval_expr (e=0xb667539c, msg=msg@entry=0xb66aa29c, val=val@entry=0x0) at routec:2009
#16 0x080646fa in do_action (a=a@entry=0xb66754c8, msg=msg@entry=0xb66aa29c) at actionc:1111
#17 0x0806b722 in run_action_list (msg=0xb66aa29c, a=) at actionc:169
#18 run_actions (a=a@entry=0xb6660dac, msg=msg@entry=0xb66aa29c) at actionc:134
#19 0x0806c106 in run_actions (msg=0xb66aa29c, a=0xb6660dac) at actionc:127
#20 run_top_route (a=a@entry=0xb6660dac, msg=msg@entry=0xb66aa29c) at actionc:209
#21 0x080c5182 in receive_msg (
    buf=0x827c5e0 "INVITE sip:997101701111@1721617121 SIP/20\r\nVia: SIP/20/UDP 1721625335:5060;branch=z9hG4bKgQwBAxWsi;rport\r\nFrom: ;tag=1~GfF8zam\r\nTo: sip:997101701111@17216171", len=, rcv_info=rcv_info@entry=0xbfa16260) at receivec:196
#22 0x081c3137 in udp_read_req (si=0xb665aaf8, bytes_read=0xbfa16328) at net/proto_udp/proto_udpc:190
#23 0x081b42d1 in handle_io (fm=, idx=, event_type=) at net/net_udpc:260
#24 handle_io (fm=, idx=, event_type=) at net/net_udpc:287
#25 io_wait_loop_epoll (h=, t=, repeat=) at net//io_wait_looph:215
#26 udp_rcv_loop (si=si@entry=0xb665aaf8) at net/net_udpc:308
#27 0x081b6b01 in udp_start_processes (chd_rank=chd_rank@entry=0x823ff7c, startup_done=startup_done@entry=0x0) at net/net_udpc:448
#28 0x080603f6 in main_loop () at mainc:723
#29 main (argc=11, argv=0xbfa165d4) at mainc:1260

I remain available for any further info you may need regarding it

Thank you


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2

Guys, any update on this? I can register on opensips just fine but as soon as it receives an invite, it just crashes.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Does this happen every time you receive an INVITE? Is this something reproducible?


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

shaheryarkh
Yes. It happens on every INVITE. Not a single call is possible.

Thank you.



On Wed, Dec 30, 2015 at 2:00 PM, Răzvan Crainea <[hidden email]> wrote:

Does this happen every time you receive an INVITE? Is this something reproducible?


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel



_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

I noticed a warning in your backtrace: warning: Source file is more recent than executable.

Please provide a backtrace with the correct sources, otherwise we'll not get accurate information.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

shaheryarkh
I am not sure what this warning means. I will recompile opensips on same machine where it will be deployed and test again.

Thank you.



On Wed, Dec 30, 2015 at 4:57 PM, Răzvan Crainea <[hidden email]> wrote:

I noticed a warning in your backtrace: warning: Source file is more recent than executable.

Please provide a backtrace with the correct sources, otherwise we'll not get accurate information.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel



_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

OK, I have fixed the warning but crash is still there.

Here is latest dump core,

GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/local/adx-ipc/sbin/opensips...done.
[New LWP 31541]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
Core was generated by `/usr/local/adx-ipc/sbin/opensips -P /var/run/opensips/opensips.pid -m 64 -M 4 -'.
Program terminated with signal 11, Segmentation fault.
#0  0xb7165188 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
86      asm volatile(
(gdb) bt
#0  0xb7165188 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
#1  get_lock (lock=) at ../../mem/../fastlock.h:176
#2  _lock (s=) at lock.h:100
#3  set_timer (ext_timeout=0xbfbb8ef8, list_id=FR_INV_TIMER_LIST, new_tl=0xb2c610f8) at timer.c:899
#4  set_timer (new_tl=new_tl@entry=0xb2c610f8, list_id=list_id@entry=FR_INV_TIMER_LIST, ext_timeout=ext_timeout@entry=0xbfbb8ef8) at timer.c:876
#5  0xb7183dd0 in reply_received (p_msg=0xb721a020) at t_reply.c:1535
#6  0x080823e2 in forward_reply (msg=msg@entry=0xb721a020) at forward.c:517
#7  0x080c4977 in receive_msg (
    buf=0x827c5e0 "SIP/2.0 100 Trying\r\nVia: SIP/2.0/UDP 172.16.171.21:5060;branch=z9hG4bKbb5e.d572daa2.0;received=172.16.171.21\r\nVia: SIP/2.0/UDP 192.168.1.16:6000;received=172.16.253.35;rport=6000;branch=z9hG4bK5910230"..., len=, rcv_info=rcv_info@entry=0xbfbb9060) at receive.c:243
#8  0x081c3137 in udp_read_req (si=0xb71b5af8, bytes_read=0xbfbb9128) at net/proto_udp/proto_udp.c:190
#9  0x081b42d1 in handle_io (fm=, idx=, event_type=) at net/net_udp.c:260
#10 handle_io (fm=, idx=, event_type=) at net/net_udp.c:287
#11 io_wait_loop_epoll (h=, t=, repeat=) at net/../io_wait_loop.h:215
#12 udp_rcv_loop (si=si@entry=0xb71b5af8) at net/net_udp.c:308
#13 0x081b6b01 in udp_start_processes (chd_rank=chd_rank@entry=0x823ff7c, startup_done=startup_done@entry=0x0) at net/net_udp.c:448
#14 0x080603f6 in main_loop () at main.c:723
#15 main (argc=11, argv=0xbfbb93d4) at main.c:1260

Thank you.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

It may be related to #723 , The fix will be shortly uploaded on GIT


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

The fix is available on GIT - please update and let me know if it solves your problem


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Thanks. I will test and update you guys later today.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

I have updated opensips to v2.1.1 Git Rev. 4c775f2. I still have crash through frequency is a bit reduced now, previously it was crashing on every call, now it crashes every 2nd or 3rd call. I see two core dumps, here are their back traces.

Contents of core.opensips.8051.loadbalancer-dev.1452761363,


GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /usr/local/adx-ipc/sbin/opensips...done.
[New LWP 8051]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
Core was generated by `/usr/local/adx-ipc/sbin/opensips -P /var/run/opensips/opensips.pid -m 64 -M 4 -'.
Program terminated with signal 11, Segmentation fault.
#0 0xb712c9e2 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
86 asm volatile(
(gdb) bt
#0 0xb712c9e2 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
#1 get_lock (lock=) at ../../mem/../fastlock.h:176
#2 _lock (s=) at lock.h:100
#3 set_timer (ext_timeout=0xbfa9b7c8, list_id=FR_INV_TIMER_LIST, new_tl=0xb2c211d0) at timer.c:907
#4 set_timer (new_tl=new_tl@entry=0xb2c211d0, list_id=list_id@entry=FR_INV_TIMER_LIST, ext_timeout=ext_timeout@entry=0xbfa9b7c8) at timer.c:884
#5 0xb714b800 in reply_received (p_msg=0xb71c7300) at t_reply.c:1535
#6 0x080823e2 in forward_reply (msg=msg@entry=0xb71c7300) at forward.c:517
#7 0x080c4977 in receive_msg (
buf=0x827c720 "SIP/2.0 100 Trying\r\nVia: SIP/2.0/UDP 172.16.171.21:5060;branch=z9hG4bKc4b9.a142e77.0;received=172.16.171.21\r\nVia: SIP/2.0/UDP 192.168.1.21:6000;received=172.16.253.35;rport=6000;branch=z9hG4bK17684605"..., len=, rcv_info=rcv_info@entry=0xbfa9b930) at receive.c:243
#8 0x081c3227 in udp_read_req (si=0xb7177af8, bytes_read=0xbfa9b9f8) at net/proto_udp/proto_udp.c:190
#9 0x081b43c1 in handle_io (fm=, idx=, event_type=) at net/net_udp.c:260
#10 handle_io (fm=, idx=, event_type=) at net/net_udp.c:287
#11 io_wait_loop_epoll (h=, t=, repeat=) at net/../io_wait_loop.h:215
#12 udp_rcv_loop (si=si@entry=0xb7177af8) at net/net_udp.c:308
#13 0x081b6bf1 in udp_start_processes (chd_rank=chd_rank@entry=0x82400bc, startup_done=startup_done@entry=0x0) at net/net_udp.c:448
#14 0x080603f6 in main_loop () at main.c:723
#15 main (argc=11, argv=0xbfa9bca4) at main.c:1260

Contents of core.opensips.8052.loadbalancer-dev.1452761363,


GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /usr/local/adx-ipc/sbin/opensips...done.
[New LWP 8052]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
Core was generated by `/usr/local/adx-ipc/sbin/opensips -P /var/run/opensips/opensips.pid -m 64 -M 4 -'.
Program terminated with signal 11, Segmentation fault.
#0 0xb712c9e2 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
86 asm volatile(
(gdb) bt
#0 0xb712c9e2 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
#1 get_lock (lock=) at ../../mem/../fastlock.h:176
#2 _lock (s=) at lock.h:100
#3 set_timer (ext_timeout=0x0, list_id=RT_T1_TO_1, new_tl=0xb2c211b4) at timer.c:907
#4 set_timer (new_tl=new_tl@entry=0xb2c211b4, list_id=list_id@entry=RT_T1_TO_1, ext_timeout=0x0, ext_timeout@entry=0x1) at timer.c:884
#5 0xb71298b0 in _set_fr_retr (retr=, rb=0xb2c21180) at t_funcs.h:137
#6 start_retr (rb=0xb2c21180) at t_funcs.h:151
#7 t_forward_nonack (t=t@entry=0xb2c21068, p_msg=p_msg@entry=0xb71c6e44, proxy=proxy@entry=0x0) at t_fwd.c:736
#8 0xb712503d in t_relay_to (p_msg=p_msg@entry=0xb71c6e44, proxy=proxy@entry=0x0, flags=flags@entry=0) at t_funcs.c:247
#9 0xb713c244 in w_t_relay (p_msg=0xb71c6e44, proxy=0x0, flags=0x0) at tm.c:1213
#10 0x0806441c in do_action (a=a@entry=0xb7192344, msg=msg@entry=0xb71c6e44) at action.c:1862
#11 0x0806bd34 in run_action_list (a=0xb7192344, msg=msg@entry=0xb71c6e44) at action.c:169
#12 0x080dbb48 in eval_elem (e=0xb71923b8, msg=msg@entry=0xb71c6e44, val=val@entry=0x0) at route.c:1643
#13 0x080db439 in eval_expr (e=0xb71923b8, msg=msg@entry=0xb71c6e44, val=val@entry=0x0) at route.c:1988
#14 0x080db567 in eval_expr (e=0xb71923e4, msg=msg@entry=0xb71c6e44, val=val@entry=0x0) at route.c:2004
#15 0x080db421 in eval_expr (e=0xb7192410, msg=msg@entry=0xb71c6e44, val=val@entry=0x0) at route.c:2009
#16 0x080646fa in do_action (a=a@entry=0xb719253c, msg=msg@entry=0xb71c6e44) at action.c:1111
#17 0x0806b722 in run_action_list (msg=0xb71c6e44, a=) at action.c:169
#18 run_actions (a=a@entry=0xb717ddac, msg=msg@entry=0xb71c6e44) at action.c:134
#19 0x0806c106 in run_actions (msg=0xb71c6e44, a=0xb717ddac) at action.c:127
#20 run_top_route (a=a@entry=0xb717ddac, msg=msg@entry=0xb71c6e44) at action.c:209
#21 0x080c5182 in receive_msg (
buf=0x827c720 "INVITE sip:997101701101@172.16.171.21 SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.1.21:6000;rport;branch=z9hG4bK1768460587\r\nFrom: sip:997101701108@172.16.171.21;tag=1885834922\r\nTo: , rcv_info=rcv_info@entry=0xbfa9b930) at receive.c:196
#22 0x081c3227 in udp_read_req (si=0xb7177af8, bytes_read=0xbfa9b9f8) at net/proto_udp/proto_udp.c:190
#23 0x081b43c1 in handle_io (fm=, idx=, event_type=) at net/net_udp.c:260
#24 handle_io (fm=, idx=, event_type=) at net/net_udp.c:287
#25 io_wait_loop_epoll (h=, t=, repeat=) at net/../io_wait_loop.h:215
#26 udp_rcv_loop (si=si@entry=0xb7177af8) at net/net_udp.c:308
#27 0x081b6bf1 in udp_start_processes (chd_rank=chd_rank@entry=0x82400bc, startup_done=startup_done@entry=0x0) at net/net_udp.c:448
#28 0x080603f6 in main_loop () at main.c:723
#29 main (argc=11, argv=0xbfa9bca4) at main.c:1260

Let me know if you need more info.

Thank you.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Hi @mshary , are you sure you are running the latest 2.1 git ? if so, do you use timer partitioning in TM ?

In frame 4, could you print *new_tl .


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Actually i downloaded and installed opensips v2.1 from scratch using,

git clone https://github.com/OpenSIPS/opensips.git -b 2.1 opensips

I believe this would get updated git repo for version 2.1 which would include the patch for this crash problem. Let me know if i need to do something else.

I will do a fresh / latest opensips v2.1 download and install today and provide you with requested info regarding frame 4.

Thank you.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

I now have opensips v2.1.2 rev. b602a7f. I still observe sigfault.

Here is the info you requested.

In first dump core we have,

GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/local/adx-ipc/sbin/opensips...done.
[New LWP 7002]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
Core was generated by `/usr/local/adx-ipc/sbin/opensips -P /var/run/opensips/opensips.pid -m 64 -M 4 -'.
Program terminated with signal 11, Segmentation fault.
#0  0xb72038b6 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
86      asm volatile(
(gdb) frame 4
#4  set_timer (new_tl=new_tl@entry=0xb2cf71b4, list_id=list_id@entry=RT_T1_TO_1, ext_timeout=0x0, ext_timeout@entry=0x1) at timer.c:884
884 void set_timer( struct timer_link *new_tl, enum lists list_id,
(gdb) print *new_tl
$1 = {next_tl = 0x0, prev_tl = 0x0, ld_tl = 0x0, time_out = 0, timer_list = 0x0, deleted = 1, set = 65533}

In second dump core we have,

GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/local/adx-ipc/sbin/opensips...done.
[New LWP 7003]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
Core was generated by `/usr/local/adx-ipc/sbin/opensips -P /var/run/opensips/opensips.pid -m 64 -M 4 -'.
Program terminated with signal 11, Segmentation fault.
#0  0xb72038b6 in tsl (lock=0x0) at ../../mem/../fastlock.h:86
86      asm volatile(
(gdb) frame 4
#4  set_timer (new_tl=new_tl@entry=0xb2cf71d0, list_id=list_id@entry=FR_INV_TIMER_LIST, ext_timeout=ext_timeout@entry=0xbfe01b68) at timer.c:884
884 void set_timer( struct timer_link *new_tl, enum lists list_id,
(gdb) print *new_tl
$1 = {next_tl = 0x0, prev_tl = 0x0, ld_tl = 0x0, time_out = 0, timer_list = 0x0, deleted = 0, set = 65533}

Thank you.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Do you use timer partitions in TM module ?


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Yes, I guess. I have,

modparam("tm", "timer_partitions", 4)

Should I disable it and test again?

Thank you.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

After commenting out this parameter line in opensips.cfg, it seems to work fine and does not crash any more.

Let me know if you would like to debug this further.

Thank you.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Hi @mshary - this is a progress - that we found that the timer partitioning is crashing it...Let me try to reproduce it on my machine.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

Hi @mshary , I have a hard time reproducing your crash - I tried enabling timer partitioning and pumped 100cps for 30 mins - nothing happened .

Is there a way to get access to the core file on your machine in order to investigate ?


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: [opensips] OpenSIPS 2.1 Crash (#732)

Liviu Chircu-2
In reply to this post by Liviu Chircu-2

The core files are on private lan of a production system. However i can upload core dump along with binary executable files to another server where you may access them.

A few things about the machine where the crash is happening, not sure if these are of any help to you,

  1. Crash happens on almost every call, even with zero load.
  2. The machine is debian wheezy 32bit Xen VM with single cpu and 2 GB ram.
  3. I reduced opensips children from 8 to 2 but no use. Timer partition was set to 4 (and never changed)
  4. Binary listeners were enabled for userloc and dialog modules.
  5. The opensips configuration is significantly complex with due to very complex network architecture with class 5 service features etc.

Thank you.


Reply to this email directly or view it on GitHub.


_______________________________________________
Devel mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
12