Quantcast

Opensips crash

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Opensips crash

OpenSIPS - Users mailing list
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

Opensips_crash_04_04_2017.txt (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" <[hidden email]>:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
OK,please enter gdb and print:
    p shm_block->first_frag
   
p shm_block->last_frag_end

    p mem_block->first_frag
   
p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" <[hidden email]>:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" <[hidden email]>:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2

    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" <[hidden email]>:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first

p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" <[hidden email]>:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" <[hidden email]>:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
Hi Denis,

Please get some more info from the corefile :

    f 10
    p p_cell->method.s
    p p_cell->uas.request
    p p_cell->uas.status

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:56 AM, Denis wrote:
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Hello, Bogdan.
 
Probably i made a big mistake.
I have already updated Opensips from latest git (to solve another problem)
 
So, when i enter in gdb i get such warning message
 
[New LWP 14909]
warning: .dynamic section for "/usr/local/opensips2.2/lib64/opensips/modules/dialog.so" is not at the expected address (wrong library or version mismatch?)
 
Anyway,
 
f 10
#10 0x00007fdf6c4ac158 in ?? ()
 
p p_cell->method.s
No symbol "p_cell" in current context.
 
p p_cell->uas.request
No symbol "p_cell" in current context.
 
p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 11:08, "Bogdan-Andrei Iancu" <[hidden email]>:
Hi Denis,

Please get some more info from the corefile :

    f 10
    p p_cell->method.s
    p p_cell->uas.request
    p p_cell->uas.status

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:56 AM, Denis wrote:
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
yeah, the opensips binary and module libs do not match the corefile anymore :(.....Do you have the old build ?  Also, was this crash on time only ? or can you reproduce it ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:17 PM, Denis wrote:
Hello, Bogdan.
 
Probably i made a big mistake.
I have already updated Opensips from latest git (to solve another problem)
 
So, when i enter in gdb i get such warning message
 
[New LWP 14909]
warning: .dynamic section for "/usr/local/opensips2.2/lib64/opensips/modules/dialog.so" is not at the expected address (wrong library or version mismatch?)
 
Anyway,
 
f 10
#10 0x00007fdf6c4ac158 in ?? ()
 
p p_cell->method.s
No symbol "p_cell" in current context.
 
p p_cell->uas.request
No symbol "p_cell" in current context.
 
p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 11:08, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Please get some more info from the corefile :

    f 10
    p p_cell->method.s
    p p_cell->uas.request
    p p_cell->uas.status

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:56 AM, Denis wrote:
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Before making upgrade i made tar archive of the directory where opensips has been installed. Can this help us?
It was crash on time only. I do not think, that o can reproduce it.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 12:27, "Bogdan-Andrei Iancu" <[hidden email]>:
yeah, the opensips binary and module libs do not match the corefile anymore :(.....Do you have the old build ?  Also, was this crash on time only ? or can you reproduce it ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:17 PM, Denis wrote:
Hello, Bogdan.
 
Probably i made a big mistake.
I have already updated Opensips from latest git (to solve another problem)
 
So, when i enter in gdb i get such warning message
 
[New LWP 14909]
warning: .dynamic section for "/usr/local/opensips2.2/lib64/opensips/modules/dialog.so" is not at the expected address (wrong library or version mismatch?)
 
Anyway,
 
f 10
#10 0x00007fdf6c4ac158 in ?? ()
 
p p_cell->method.s
No symbol "p_cell" in current context.
 
p p_cell->uas.request
No symbol "p_cell" in current context.
 
p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 11:08, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Please get some more info from the corefile :

    f 10
    p p_cell->method.s
    p p_cell->uas.request
    p p_cell->uas.status

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:56 AM, Denis wrote:
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

Bogdan-Andrei Iancu-2
Oh, that is good !

When starting gdb, point to the old opensips binary. To solve the problem with the modules (which are libraries), see:
    http://stackoverflow.com/questions/84341/how-do-i-prepend-a-directory-the-library-path-when-loading-a-core-file-in-gdb-on
(the first answer)

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:31 PM, Denis wrote:
Before making upgrade i made tar archive of the directory where opensips has been installed. Can this help us?
It was crash on time only. I do not think, that o can reproduce it.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 12:27, "Bogdan-Andrei Iancu" [hidden email]:
yeah, the opensips binary and module libs do not match the corefile anymore :(.....Do you have the old build ?  Also, was this crash on time only ? or can you reproduce it ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:17 PM, Denis wrote:
Hello, Bogdan.
 
Probably i made a big mistake.
I have already updated Opensips from latest git (to solve another problem)
 
So, when i enter in gdb i get such warning message
 
[New LWP 14909]
warning: .dynamic section for "/usr/local/opensips2.2/lib64/opensips/modules/dialog.so" is not at the expected address (wrong library or version mismatch?)
 
Anyway,
 
f 10
#10 0x00007fdf6c4ac158 in ?? ()
 
p p_cell->method.s
No symbol "p_cell" in current context.
 
p p_cell->uas.request
No symbol "p_cell" in current context.
 
p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 11:08, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Please get some more info from the corefile :

    f 10
    p p_cell->method.s
    p p_cell->uas.request
    p p_cell->uas.status

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:56 AM, Denis wrote:
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Bogdan, I copied tar to the backup server and restore opensips from tar there (on the same path as in main was).
Now, when entering to the gdb on backup, it does not show any warning messages, but the result is the same
 
f 10
#0  0x0000000000000000 in ?? ()
(gdb) p p_cell->method.s
No symbol "p_cell" in current context.
(gdb) p p_cell->uas.request
No symbol "p_cell" in current context.
(gdb) p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 14:38, "Bogdan-Andrei Iancu" <[hidden email]>:
Oh, that is good !

When starting gdb, point to the old opensips binary. To solve the problem with the modules (which are libraries), see:
    http://stackoverflow.com/questions/84341/how-do-i-prepend-a-directory-the-library-path-when-loading-a-core-file-in-gdb-on
(the first answer)

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:31 PM, Denis wrote:
Before making upgrade i made tar archive of the directory where opensips has been installed. Can this help us?
It was crash on time only. I do not think, that o can reproduce it.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 12:27, "Bogdan-Andrei Iancu" [hidden email]:
yeah, the opensips binary and module libs do not match the corefile anymore :(.....Do you have the old build ?  Also, was this crash on time only ? or can you reproduce it ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:17 PM, Denis wrote:
Hello, Bogdan.
 
Probably i made a big mistake.
I have already updated Opensips from latest git (to solve another problem)
 
So, when i enter in gdb i get such warning message
 
[New LWP 14909]
warning: .dynamic section for "/usr/local/opensips2.2/lib64/opensips/modules/dialog.so" is not at the expected address (wrong library or version mismatch?)
 
Anyway,
 
f 10
#10 0x00007fdf6c4ac158 in ?? ()
 
p p_cell->method.s
No symbol "p_cell" in current context.
 
p p_cell->uas.request
No symbol "p_cell" in current context.
 
p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 11:08, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Please get some more info from the corefile :

    f 10
    p p_cell->method.s
    p p_cell->uas.request
    p p_cell->uas.status

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:56 AM, Denis wrote:
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips crash

OpenSIPS - Users mailing list
Hello, Bogdan!
 
Is there any additional information that i should gather about problem?
Or everything is clear?
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
 
06.04.2017, 15:18, "Denis via Users" <[hidden email]>:
Bogdan, I copied tar to the backup server and restore opensips from tar there (on the same path as in main was).
Now, when entering to the gdb on backup, it does not show any warning messages, but the result is the same
 
f 10
#0  0x0000000000000000 in ?? ()
(gdb) p p_cell->method.s
No symbol "p_cell" in current context.
(gdb) p p_cell->uas.request
No symbol "p_cell" in current context.
(gdb) p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 14:38, "Bogdan-Andrei Iancu" <[hidden email]>:
Oh, that is good !

When starting gdb, point to the old opensips binary. To solve the problem with the modules (which are libraries), see:
    http://stackoverflow.com/questions/84341/how-do-i-prepend-a-directory-the-library-path-when-loading-a-core-file-in-gdb-on
(the first answer)

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:31 PM, Denis wrote:
Before making upgrade i made tar archive of the directory where opensips has been installed. Can this help us?
It was crash on time only. I do not think, that o can reproduce it.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 12:27, "Bogdan-Andrei Iancu" [hidden email]:
yeah, the opensips binary and module libs do not match the corefile anymore :(.....Do you have the old build ?  Also, was this crash on time only ? or can you reproduce it ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/06/2017 12:17 PM, Denis wrote:
Hello, Bogdan.
 
Probably i made a big mistake.
I have already updated Opensips from latest git (to solve another problem)
 
So, when i enter in gdb i get such warning message
 
[New LWP 14909]
warning: .dynamic section for "/usr/local/opensips2.2/lib64/opensips/modules/dialog.so" is not at the expected address (wrong library or version mismatch?)
 
Anyway,
 
f 10
#10 0x00007fdf6c4ac158 in ?? ()
 
p p_cell->method.s
No symbol "p_cell" in current context.
 
p p_cell->uas.request
No symbol "p_cell" in current context.
 
p p_cell->uas.status
No symbol "p_cell" in current context.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
06.04.2017, 11:08, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Please get some more info from the corefile :

    f 10
    p p_cell->method.s
    p p_cell->uas.request
    p p_cell->uas.status

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:56 AM, Denis wrote:
Ok, Bogdan, I'll be waiting any result.
 
Thank you.
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
05.04.2017, 11:53, "Bogdan-Andrei Iancu" [hidden email]:
Hi Denis,

Many thanks for the output - this confirm a suspicious of mine in regards to the cause of the crash - let me try to see if we can reproduce it on our side or if we need more info from your side.
Please do not delete the corefile for the moment :)

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/05/2017 11:03 AM, Denis wrote:
Hello, Bogdan!
 
f 5
#5  0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158) at dlg_hash.c:183
183                     destroy_dlg_callbacks_list(dlg->cbs.first);
 
p *dlg->cbs.first
$1 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x0, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x7fdf84191080}
 
p *dlg->cbs.first->next
$2 = {types = 96, callback = 0x7fdf67531300 <acc_dlg_callback>, param = 0x7fdf83c9f340, callback_param_free = 0x7fdf675318fe <dlg_free_acc_mask>, next = 0x0}
 
p *dlg->cbs.first->next->next
Cannot access memory at address 0x0
 
p *dlg->cbs.first->next->next->next
Cannot access memory at address 0x20
 
and so on with ....at address 0x20
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 18:43, "Bogdan-Andrei Iancu" [hidden email]:
Super :)....we got to something useful.

Now, again in GDB please do:

f 5
p *dlg->cbs.first
p *dlg->cbs.first->next
p *dlg->cbs.first->next->next

and keep adding "->next" until you get error from gdb.

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 06:07 PM, Денис Путято wrote:
Sorry, Bogdan, i missed that
 
I found such CRITICAL message
"CRITICAL:core:qm_free: freeing already freed pointer, first free: acc_logic.c: dlg_free_acc_mask(211) - aborting"
 
And after that and till crash only "WARNING:core:utimer_ticker:...." reports in syslog.
 
(gdb)
f 2
#2  0x0000000000515592 in qm_free (qm=<optimized out>, p=p@entry=0x7fdf83c9f340, file=file@entry=0x7fdf6753ad62 "acc_logic.c",
    func=func@entry=0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line=line@entry=211) at mem/q_malloc.c:483
483                     abort();
 
 
p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0, is_free = 140597661320880}, file = 0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90 <__FUNCTION__.12529> "dlg_free_acc_mask", line = 211,
  check = 17361641481138401520}
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 17:58, "Bogdan-Andrei Iancu" [hidden email]:
The RADIUS part is not relevant.

Still you didn;t answer to my question on the CRITICAL error - according to the backtrace, you should have a CRITICAL message in the opensips log, just before the crash.

In GDB, do:
    f 2
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))  

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 05:55 PM, Denis wrote:
During crash there was another problem with LAN network.
Because of this problem Opensips could not connect to radius server.
rc_ip_hostname: couldn't look up host by addr: xxxxxx
rc_send_server: no reply from RADIUS server unknown:1812
 
No more critical messages.
 
Bogdan, i am sorry, but
"in frame 2". What does it mean?
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 16:35, "Bogdan-Andrei Iancu" [hidden email]:
Any CRITICAL message into the log before the crash ?

also, in gdb in frame 2, please print:
    p *((struct qm_frag*) ((char*)p-sizeof(struct qm_frag)))
   

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 03:19 PM, Denis wrote:
p shm_block->first_frag
$1 = (struct qm_frag *) 0x7fdf6b9d44f8
 
p shm_block->last_frag_end
$2 = (struct qm_frag_end *) 0x7fe02899efd0
 
 p mem_block->first_frag
$3 = (struct qm_frag *) 0x7fe029214508
 
p mem_block->last_frag_end
$4 = (struct qm_frag_end *) 0x7fe0691defe0
 
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 14:51, "Bogdan-Andrei Iancu" [hidden email]:
OK,please enter gdb and print:
    p shm_block->first_frag
    p shm_block->last_frag_end

    p mem_block->first_frag
    p mem_block->last_frag_end

Thanks,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 01:42 PM, Denis wrote:
Hello, Bogdan!
 
No, i cannot see in the log any messages you mentioned it your letter.
The situation dealing with crash had such stage.
 
1. I prepared to migrate to 2.2.3 from 2.2.2. But do not have a time to restart Opensips
2. As a result "i waited" when Opensips have crashed. During crash of 2.2.2 no core file has been generated, but Opensips has been restarted using cron.
3. After 2) Opensips became 2.2.3. It was working about 1 minute and crashed again, but in that time, core file has been generated and i attached information from it.
So, between 2) and 3) , as i mentioned above, there were no messages about memory, but i see many messages like
'WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 71500 ms
WARNING:core:timer_ticker: timer task <pike-clean> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <nh-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <dlg-timer> already scheduled for 72400 ms
WARNING:core:timer_ticker: timer task <blcore-expire> already scheduled for 72400 ms'
and so on
 
As for about version
 
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 7922417
main.c compiled on 06:29:28 Mar 23 2017 with gcc 5.4.0
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
04.04.2017, 12:43, "Bogdan-Andrei Iancu" [hidden email]:
Hello Denis,

Thank you for the report. Do you see any CRITICAL log just before the crash, something like:
    "bad pointer 0xxxxxxxx (out of memory block!) - aborting"
?
If yes, please post here the exact line.

Also, do you use the latest OpenSIPS 2.2 from GIT repo ? (run "opensips -v")

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/04/2017 10:58 AM, Denis via Users wrote:
Hello!
 
Server:: OpenSIPS (2.2.3 (x86_64/linux))
 
Thank you for any help.
 
P.S. Opensips has been compiled with QM_MALLOC and DBG_MALLOC flags
 
-- 
С уважением, Денис.
Best regards, Denis
 
 
 
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
,

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
12
Loading...