What is protocol/port mismatch?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

What is protocol/port mismatch?

opensipslist

Hello list,

I'm using:

  Solaris 11 x86 (nv-b91)
  OpenSIPS 1.6.0 with TLS

...and I see this in the log:

  <warning> opensips[2717]: WARNING:core:get_send_socket: protocol/port mismatch

some hundreds of times per day (about once every 20 minutes
per registered UAC.)

I have this in the route script:

  listen = tls:name.host.tld:5061
  [...]
  t_relay("name.host.tld:5080");

There is a voicemail server (not OpenSIPS) listening on TCP (not
TLS) port 5080 on the same host. OpenSIPS and the other server
exchange traffic such as SUBSCRIBE and NOTIFY for MWI and INVITEs
to voicemail. The voicemail server sends SIP messages to OpenSIPS
port 5061 over TLS (I think.)

Question:
What exactly is the meaning of these warnings in the log, and does
it seem that I'm doing something wrong in the route script?

Thanks,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

Bogdan-Andrei Iancu
Hi Brian,

Do you use "force_send_socket()" functions in your script, before the
t_relay ?

Regards,
Bogdan

[hidden email] wrote:

> Hello list,
>
> I'm using:
>
>   Solaris 11 x86 (nv-b91)
>   OpenSIPS 1.6.0 with TLS
>
> ...and I see this in the log:
>
>   <warning> opensips[2717]: WARNING:core:get_send_socket: protocol/port mismatch
>
> some hundreds of times per day (about once every 20 minutes
> per registered UAC.)
>
> I have this in the route script:
>
>   listen = tls:name.host.tld:5061
>   [...]
>   t_relay("name.host.tld:5080");
>
> There is a voicemail server (not OpenSIPS) listening on TCP (not
> TLS) port 5080 on the same host. OpenSIPS and the other server
> exchange traffic such as SUBSCRIBE and NOTIFY for MWI and INVITEs
> to voicemail. The voicemail server sends SIP messages to OpenSIPS
> port 5061 over TLS (I think.)
>
> Question:
> What exactly is the meaning of these warnings in the log, and does
> it seem that I'm doing something wrong in the route script?
>
> Thanks,
> Brian
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>  


--
Bogdan-Andrei Iancu
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello Bogdan,

An ven., janv 29, 2010, Bogdan-Andrei Iancu schrieb:

>[hidden email] wrote:
>> ...and I see this in the log:
>>
>>   <warning> opensips[2717]: WARNING:core:get_send_socket: protocol/port mismatch
>>
>> some hundreds of times per day (about once every 20 minutes
>> per registered UAC.)
>>
>> I have this in the route script:
>>
>>   listen = tls:name.host.tld:5061
>>   [...]
>>   t_relay("name.host.tld:5080");
>>
>Do you use "force_send_socket()" functions in your script,
>before the t_relay ?
>
No, I have 'force_rport()' in the script, but neither
'force_send_socket()' nor 'force_tcp_alias()' is there at
all. Are you suggesting that I use it before t_relay to the
server with a different transport?

Is the basic idea of this warning message that OpenSIPS
exchanges a SIP message with a UA over a certain transport
(TLS in this case) and port number (5061 in this case), but
a t_relay in the route script forwards the message over a
different transport or to a different port number?

If so, what is being compared and how is it compared?

  Port1 == Port2
  Transport1 == Transport2
  Transport1 & Port1 == Transport2 & Port2
  ...?

Thanks,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello list,

The problem was that the voicemail server was respecting the host
and port of the routeset which OpenSIPS was rewriting to acommodate
the transport change from TLS to TCP. The voicemail server was
configured to send over TLS to the outbound proxy however, so
traffic was arriving at OpenSIPS on the port 5060 using TLS.

It's clear now why the warnings were appearing in the logs, but
even more it seems that OpenSIPS sometimes crashes when sending
TLS traffic to a TCP port which it is expecting clear text from.

To know about the solution to this problem, read below.

An ven., janv 29, 2010, [hidden email] schrieb:

>An ven., janv 29, 2010, Bogdan-Andrei Iancu schrieb:
>>[hidden email] wrote:
>>> ...and I see this in the log:
>>>
>>>   <warning> opensips[2717]: WARNING:core:get_send_socket: protocol/port mismatch
>>>
>>> some hundreds of times per day (about once every 20 minutes
>>> per registered UAC.)
>>>
>>> I have this in the route script:
>>>
>>>   listen = tls:name.host.tld:5061
>>>   [...]
>>>   t_relay("name.host.tld:5080");
>>>
First, although all UAs were exchanging SIP traffic with OpenSIPS
over TLS, t_relay was relaying the messages over UDP. I assume that
t_relay is either hardcoded to use UDP or OpenSIPS was routing the
messages to itself internally over UDP (to recursively resolve
aliases.) In this case maybe t_relay implicitly uses the transport
over which the message was last received (even internally.)

In any case, the call to t_relay should look like this if relaying
over the TCP transport is desired:

  t_relay("tcp:name.host.tld:5080");

The problem then is that the rr module will add two Record-Route
headers, expecting a symetrical fashion of SIP traffic. I turned
this off with modparam("rr", "enable_double_rr", 0) because the
voicemail server (Asterisk) is very tricky (maybe even buggy) to
configure to use TLS and outbound proxies.

At this point I saw that traffic was arriving at OpenSIPS port 5061
over TLS as expected, and as indicated by the routeset. The problem
is solved.

>Is the basic idea of this warning message that OpenSIPS
>exchanges a SIP message with a UA over a certain transport
>(TLS in this case) and port number (5061 in this case), but
>a t_relay in the route script forwards the message over a
>different transport or to a different port number?
>
The basic idea of this warning message is that OpenSIPS expects
a certain type (TLS or not TLS) of traffic at a certain TCP port
as configured by the 'listen' directives. If a UAC sends TLS
traffic to a port only configured to listen to plain text traffic
then this warning will appear. What happens to the mismatched
traffic is not clear, but I assume that it is ignored by OpenSIPS.

>If so, what is being compared and how is it compared?
>
The TCP port on which OpenSIPS listens to and the type of traffic,
either TLS encrypted or plain text.

Regards,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

Bogdan-Andrei Iancu
Hi Brian,

Regarding the crash you mentioned - do you have any backtraces ?

About some of your doubts:
1) t_relay() is not forcing any proto by itself: it preserves the
inbound proto if the RURI (or socket) is not saying otherwise.
2) turning off the double RR may brake some things as opensips will not
be able to properly route between the original inbound and outbound
interfaces....only if you do it manually from the script.

To me, the solution looks like a dirty hack, but if makes Asterisk
happy....what can I say :)

Regards,
Bogdan

[hidden email] wrote:

> Hello list,
>
> The problem was that the voicemail server was respecting the host
> and port of the routeset which OpenSIPS was rewriting to acommodate
> the transport change from TLS to TCP. The voicemail server was
> configured to send over TLS to the outbound proxy however, so
> traffic was arriving at OpenSIPS on the port 5060 using TLS.
>
> It's clear now why the warnings were appearing in the logs, but
> even more it seems that OpenSIPS sometimes crashes when sending
> TLS traffic to a TCP port which it is expecting clear text from.
>
> To know about the solution to this problem, read below.
>
> An ven., janv 29, 2010, [hidden email] schrieb:
>  
>> An ven., janv 29, 2010, Bogdan-Andrei Iancu schrieb:
>>    
>>> [hidden email] wrote:
>>>      
>>>> ...and I see this in the log:
>>>>
>>>>   <warning> opensips[2717]: WARNING:core:get_send_socket: protocol/port mismatch
>>>>
>>>> some hundreds of times per day (about once every 20 minutes
>>>> per registered UAC.)
>>>>
>>>> I have this in the route script:
>>>>
>>>>   listen = tls:name.host.tld:5061
>>>>   [...]
>>>>   t_relay("name.host.tld:5080");
>>>>
>>>>        
> First, although all UAs were exchanging SIP traffic with OpenSIPS
> over TLS, t_relay was relaying the messages over UDP. I assume that
> t_relay is either hardcoded to use UDP or OpenSIPS was routing the
> messages to itself internally over UDP (to recursively resolve
> aliases.) In this case maybe t_relay implicitly uses the transport
> over which the message was last received (even internally.)
>
> In any case, the call to t_relay should look like this if relaying
> over the TCP transport is desired:
>
>   t_relay("tcp:name.host.tld:5080");
>
> The problem then is that the rr module will add two Record-Route
> headers, expecting a symetrical fashion of SIP traffic. I turned
> this off with modparam("rr", "enable_double_rr", 0) because the
> voicemail server (Asterisk) is very tricky (maybe even buggy) to
> configure to use TLS and outbound proxies.
>
> At this point I saw that traffic was arriving at OpenSIPS port 5061
> over TLS as expected, and as indicated by the routeset. The problem
> is solved.
>
>  
>> Is the basic idea of this warning message that OpenSIPS
>> exchanges a SIP message with a UA over a certain transport
>> (TLS in this case) and port number (5061 in this case), but
>> a t_relay in the route script forwards the message over a
>> different transport or to a different port number?
>>
>>    
> The basic idea of this warning message is that OpenSIPS expects
> a certain type (TLS or not TLS) of traffic at a certain TCP port
> as configured by the 'listen' directives. If a UAC sends TLS
> traffic to a port only configured to listen to plain text traffic
> then this warning will appear. What happens to the mismatched
> traffic is not clear, but I assume that it is ignored by OpenSIPS.
>
>  
>> If so, what is being compared and how is it compared?
>>
>>    
> The TCP port on which OpenSIPS listens to and the type of traffic,
> either TLS encrypted or plain text.
>
> Regards,
> Brian
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>  


--
Bogdan-Andrei Iancu
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello Bogdan,

An mer., févr 03, 2010, Bogdan-Andrei Iancu schrieb:
>Regarding the crash you mentioned - do you have any backtraces ?
>
There's quite a lot of situations in which OpenSIPS crashes, so
I'm not sure this one is related to TLS traffic arriving on a non
TLS port. In any case, here's the last backtrace:

$ gdb /pfx/sbin/opensips opensips.17272.name.host.tld.core
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-pc-solaris2.11"...
(no debugging symbols found)
Cannot access memory at address 0x70795464
(gdb) bt
#0  0x0810eefc in fm_realloc ()
#1  0xd0d8cc8c in ?? ()
#2  0x080465a8 in ?? ()
#3  0xd0d859d4 in ?? ()
#4  0x08353c2c in mem_pool ()
#5  0x00000191 in ?? ()
#6  0x08046640 in ?? ()
#7  0xffffffff in ?? ()
#8  0xffffffff in ?? ()
#9  0x080467d8 in ?? ()
#10 0x08046638 in ?? ()
#11 0x0812437e in parse_min_se ()
#12 0x083132e0 in mem_pool ()
#13 0xffffffff in ?? ()
#14 0x81af694b in ?? ()
#15 0xd0d8c6fc in ?? ()
#16 0x00000001 in ?? ()
#17 0x08353c2c in mem_pool ()
#18 0xce43365d in ?? ()
#19 0x080467c4 in ?? ()
#20 0x0804678c in ?? ()
#21 0x08353f9c in mem_pool ()
#22 0x08046640 in ?? ()
#23 0x08353f9c in mem_pool ()
#24 0x00000073 in ?? ()
#25 0x082ff8e8 in ?? ()
#26 0xd13dbbe9 in ?? ()
#27 0x00000002 in ?? ()
#28 0x082ff8f0 in ?? ()
#29 0x00003332 in ?? ()
#30 0x00000000 in ?? ()
(gdb)

$ pstack opensips.17272.name.host.tld.core core 'opensips.17272.name.host.tld.core' of 17272:        /pfx/sbin/opensips -P /pfx/var/opensips/opensips.pid
 0810eefc fm_realloc (0, 80467cc, 0, 80467d8, d10f0000, 8354104) + 4bc
 d1011537 dlg_validate_dialog (8353c2c, ce4334b0, 8046a38, d101459b, ce4334b0, 8323390) + 290
 d10035ba w_validate_dialog (8353c2c, 0, 0, 0, 0, 0) + 2a
 08076240 do_action (8323390, 8353c2c, 7275652e, 8046c00, 8046bf0, 0) + 15a5
 08079877 run_action_list (8323390, 8353c2c, 844f7c8, 844f735, 83132ba, 13) + 281
 080d03a7 eval_expr (83233fc, 8353c2c, 0, 0, 313c5, 1) + 46a
 080d014a eval_expr (8323428, 8353c2c, 0, 0, 0, 0) + 20d
 080d0019 eval_expr (8323454, 8353c2c, 0, d0ff18b3, 0, 8354890) + dc
 080d0172 eval_expr (8323480, 8353c2c, 0, 8354700, 8354a2c, 27) + 235
 0807643d do_action (83238cc, 8353c2c, 40, 82b1c50, 80472d4, 80472d4) + 17a2
 08079877 run_action_list (83238cc, 8353c2c, 0, 81aaf92, 82b1c50, ce4350a8) + 281
 08078381 do_action (832536c, 8353c2c, ce415b0d, d0dadd84, ce433690, ce415b08) + 36e6
 08079877 run_action_list (832536c, 8353c2c, 0, 0, 0, 0) + 281
 08078381 do_action (8325654, 8353c2c, ce415790, 0, 8353c30, 1) + 36e6
 08079877 run_action_list (832108c, 8353c2c, d106ea15, 8344074, 8353c2c, 0) + 281
 08079c50 get_bl_head_by_name (832108c, 8353c2c, 8353c2c, 80478f0, 308, 8353c2c) + f3
 080be5aa receive_msg (ce415808, 308, ce4157a0, 80478d4, ce375d9c, 2) + 5ac
 080f4cf4 tcp_read_req (ce415790, 8047978, d11d10c5, d11bfa64, 17, ce375d4c) + 18c
 080f687d ???????? (b, d001, 8047ac4, d06e7e60, d06e9ae0, 831a84c)
 080f77a4 tcp_receive_loop (c, 2, 0, 8047b10, 9, 0) + 94b
 080ed6f8 tcp_send (82cb1f4, 138a, 83178e8, 805f130, d1169cc8, 8047c5c) + 4b
 08091d47 main     (80749c0, 3, 8047bdc) + 3bab
 080749c0 do_assign (3, 8047cc4, 8047cd8, 8047cdb, 0, 8047cfb) + d0

$ pflags opensips.17272.name.host.tld.core
core 'opensips.17272.name.host.tld.core' of 17272:        /pfx/sbin/opensips -P /pfx/var/opensips/opensips.pid
        data model = _ILP32  flags = ORPHAN|MSACCT|MSFORK
 /1:    flags = 0
        sigmask = 0xffffbefc,0x0000ffff  cursig = SIGSEGV

>1) t_relay() is not forcing any proto by itself: it preserves the
>inbound proto if the RURI (or socket) is not saying otherwise.
>
Okay, then I assume the t_relay() tried sending TLS traffic to
the voicemail server, could not negotiate a TLS connection, and
so resent the message over UDP instead. Is that right? Does
t_relay() choose UDP instead of TCP when its first choice (the
preserved inbound proto) cannot be used?

>2) turning off the double RR may brake some things as opensips will
>not be able to properly route between the original inbound and
>outbound interfaces... only if you do it manually from the script.
>
I've configured OpenSIPS to listen on only one interface, so I guess
that I'm not affected by this breakage. Next I'll try to solve the
problem another way without disabling double Record-Route headers.

>To me, the solution looks like a dirty hack, but if makes Asterisk
>happy... what can I say :)
>
Now that I know more I'll try to solve it another way.

Regards,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

Bogdan-Andrei Iancu
Hi Brian,

The backtrace from gdb seams really corrupted, but the pstack output is
a bit consistent.
What is strange is that dlg_validate_dialog () seams to call
fm_realloc() which is not in the script.

So, to eliminate any possibility of a mem corruption (the crash was in
the mem manager), I suggest to compile the mem debugger to see if there
are no mem issues (mem overwritten, double free, etc).

Regards,
Bogdan


[hidden email] wrote:

> Hello Bogdan,
>
> An mer., févr 03, 2010, Bogdan-Andrei Iancu schrieb:
>  
>> Regarding the crash you mentioned - do you have any backtraces ?
>>
>>    
> There's quite a lot of situations in which OpenSIPS crashes, so
> I'm not sure this one is related to TLS traffic arriving on a non
> TLS port. In any case, here's the last backtrace:
>
> $ gdb /pfx/sbin/opensips opensips.17272.name.host.tld.core
> GNU gdb 6.8
> Copyright (C) 2008 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "i386-pc-solaris2.11"...
> (no debugging symbols found)
> Cannot access memory at address 0x70795464
> (gdb) bt
> #0  0x0810eefc in fm_realloc ()
> #1  0xd0d8cc8c in ?? ()
> #2  0x080465a8 in ?? ()
> #3  0xd0d859d4 in ?? ()
> #4  0x08353c2c in mem_pool ()
> #5  0x00000191 in ?? ()
> #6  0x08046640 in ?? ()
> #7  0xffffffff in ?? ()
> #8  0xffffffff in ?? ()
> #9  0x080467d8 in ?? ()
> #10 0x08046638 in ?? ()
> #11 0x0812437e in parse_min_se ()
> #12 0x083132e0 in mem_pool ()
> #13 0xffffffff in ?? ()
> #14 0x81af694b in ?? ()
> #15 0xd0d8c6fc in ?? ()
> #16 0x00000001 in ?? ()
> #17 0x08353c2c in mem_pool ()
> #18 0xce43365d in ?? ()
> #19 0x080467c4 in ?? ()
> #20 0x0804678c in ?? ()
> #21 0x08353f9c in mem_pool ()
> #22 0x08046640 in ?? ()
> #23 0x08353f9c in mem_pool ()
> #24 0x00000073 in ?? ()
> #25 0x082ff8e8 in ?? ()
> #26 0xd13dbbe9 in ?? ()
> #27 0x00000002 in ?? ()
> #28 0x082ff8f0 in ?? ()
> #29 0x00003332 in ?? ()
> #30 0x00000000 in ?? ()
> (gdb)
>
> $ pstack opensips.17272.name.host.tld.core core 'opensips.17272.name.host.tld.core' of 17272:        /pfx/sbin/opensips -P /pfx/var/opensips/opensips.pid
>  0810eefc fm_realloc (0, 80467cc, 0, 80467d8, d10f0000, 8354104) + 4bc
>  d1011537 dlg_validate_dialog (8353c2c, ce4334b0, 8046a38, d101459b, ce4334b0, 8323390) + 290
>  d10035ba w_validate_dialog (8353c2c, 0, 0, 0, 0, 0) + 2a
>  08076240 do_action (8323390, 8353c2c, 7275652e, 8046c00, 8046bf0, 0) + 15a5
>  08079877 run_action_list (8323390, 8353c2c, 844f7c8, 844f735, 83132ba, 13) + 281
>  080d03a7 eval_expr (83233fc, 8353c2c, 0, 0, 313c5, 1) + 46a
>  080d014a eval_expr (8323428, 8353c2c, 0, 0, 0, 0) + 20d
>  080d0019 eval_expr (8323454, 8353c2c, 0, d0ff18b3, 0, 8354890) + dc
>  080d0172 eval_expr (8323480, 8353c2c, 0, 8354700, 8354a2c, 27) + 235
>  0807643d do_action (83238cc, 8353c2c, 40, 82b1c50, 80472d4, 80472d4) + 17a2
>  08079877 run_action_list (83238cc, 8353c2c, 0, 81aaf92, 82b1c50, ce4350a8) + 281
>  08078381 do_action (832536c, 8353c2c, ce415b0d, d0dadd84, ce433690, ce415b08) + 36e6
>  08079877 run_action_list (832536c, 8353c2c, 0, 0, 0, 0) + 281
>  08078381 do_action (8325654, 8353c2c, ce415790, 0, 8353c30, 1) + 36e6
>  08079877 run_action_list (832108c, 8353c2c, d106ea15, 8344074, 8353c2c, 0) + 281
>  08079c50 get_bl_head_by_name (832108c, 8353c2c, 8353c2c, 80478f0, 308, 8353c2c) + f3
>  080be5aa receive_msg (ce415808, 308, ce4157a0, 80478d4, ce375d9c, 2) + 5ac
>  080f4cf4 tcp_read_req (ce415790, 8047978, d11d10c5, d11bfa64, 17, ce375d4c) + 18c
>  080f687d ???????? (b, d001, 8047ac4, d06e7e60, d06e9ae0, 831a84c)
>  080f77a4 tcp_receive_loop (c, 2, 0, 8047b10, 9, 0) + 94b
>  080ed6f8 tcp_send (82cb1f4, 138a, 83178e8, 805f130, d1169cc8, 8047c5c) + 4b
>  08091d47 main     (80749c0, 3, 8047bdc) + 3bab
>  080749c0 do_assign (3, 8047cc4, 8047cd8, 8047cdb, 0, 8047cfb) + d0
>
> $ pflags opensips.17272.name.host.tld.core
> core 'opensips.17272.name.host.tld.core' of 17272:        /pfx/sbin/opensips -P /pfx/var/opensips/opensips.pid
>         data model = _ILP32  flags = ORPHAN|MSACCT|MSFORK
>  /1:    flags = 0
>         sigmask = 0xffffbefc,0x0000ffff  cursig = SIGSEGV
>
>  
>> 1) t_relay() is not forcing any proto by itself: it preserves the
>> inbound proto if the RURI (or socket) is not saying otherwise.
>>
>>    
> Okay, then I assume the t_relay() tried sending TLS traffic to
> the voicemail server, could not negotiate a TLS connection, and
> so resent the message over UDP instead. Is that right? Does
> t_relay() choose UDP instead of TCP when its first choice (the
> preserved inbound proto) cannot be used?
>
>  
>> 2) turning off the double RR may brake some things as opensips will
>> not be able to properly route between the original inbound and
>> outbound interfaces... only if you do it manually from the script.
>>
>>    
> I've configured OpenSIPS to listen on only one interface, so I guess
> that I'm not affected by this breakage. Next I'll try to solve the
> problem another way without disabling double Record-Route headers.
>
>  
>> To me, the solution looks like a dirty hack, but if makes Asterisk
>> happy... what can I say :)
>>
>>    
> Now that I know more I'll try to solve it another way.
>
> Regards,
> Brian
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>  


--
Bogdan-Andrei Iancu
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello Bogdan,

An jeu., févr 04, 2010, Bogdan-Andrei Iancu schrieb:
>The backtrace from gdb seams really corrupted, but the pstack
>output is a bit consistent.
>
Sorry about that.

>What is strange is that dlg_validate_dialog () seams to call
>fm_realloc() which is not in the script.
>
Maybe a linked library is calling it?

>So, to eliminate any possibility of a mem corruption (the crash was
>in the mem manager), I suggest to compile the mem debugger to see
>if there are no mem issues (mem overwritten, double free, etc).
>
Good idea but... What do you mean by 'mem debugger'?

  $ w3m http://www.opensips.org/
  Results of search for  memory debugger:
  0 pages found out of 318 pages searched.

  opensips-1.6.1-tls $ grep -i 'mem.*debugger' *
  opensips-1.6.1-tls $

I see a variety of things on the page 'Resources/Documentation/
Out Of Memory' http://www.opensips.org/Resources/DocsTsMem, but
I'm not sure if you are referring to it which one exactly then?

Regards,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

Bogdan-Andrei Iancu
Hi Brian,

[hidden email] wrote:
> I see a variety of things on the page 'Resources/Documentation/
> Out Of Memory' http://www.opensips.org/Resources/DocsTsMem, but
> I'm not sure if you are referring to it which one exactly then?
>  
That is the right one - use the compile instructions from there and run
use memlog=10 (to avoid extra debugs from memory) and memdump=1 (to get
mem dump at shutdown)

If a mem issue is found, your opensips will automatically coredump and
dump mem stats.

Regards,
Bogdan

--
Bogdan-Andrei Iancu
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello Bogdan,

An ven., févr 05, 2010, Bogdan-Andrei Iancu schrieb:

>[hidden email] wrote:
>> I see a variety of things on the page 'Resources/Documentation/
>> Out Of Memory' http://www.opensips.org/Resources/DocsTsMem, but
>> I'm not sure if you are referring to it which one exactly then?
>>
>That is the right one - use the compile instructions from there and
>run use memlog=10 (to avoid extra debugs from memory) and memdump=1
>(to get mem dump at shutdown)
>
>If a mem issue is found, your opensips will automatically coredump
>and dump mem stats.
>
Okay I finally got everything compiled, configured, running, and...

  ...dumping. Here's the backtrace:

Core was generated by `/pfx/sbin/opensips -P /pfx/var/opensips/opensips.pid -m 64'.
Program terminated with signal 11, Segmentation fault.
[New process 12345    ]
#0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50, hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
271 replace.c: No such file or directory.
    in replace.c
(gdb) bt
#0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50,
                hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
#1  0xd0e1c701 in w_replace_to (msg=0x8355f48, p1=0x8348a88
                "\030?3\b\004", p2=0x0)
    at uac.c:408
#2  0x08076000 in do_action ()
#3  0x0807961b in run_action_list ()
#4  0x08077574 in do_action ()
#5  0x0807961b in run_action_list ()
#6  0x08078177 in do_action ()
#7  0x0807961b in run_action_list ()
#8  0x080799f0 in run_top_route ()
#9  0x080be52a in receive_msg ()
#10 0x080f4c74 in tcp_read_req ()
#11 0x080f67fd in ?? ()
#12 0xcc78ebe8 in ?? ()
(gdb)

It seems to me that the crash came just after trying memcpy(3),
but looking at modules/uac/replace.c:271 I still don't see the
reason. I am using uac_replace_from and uac_replace_to in the
route script but there is no mention of 'no more pkg mem' in
the logs at all (from replace.c:268.)

There is a lot in the log (I used debug=4) as well as sip_trace
so I don't know what to post. It will be a nightmare sanitizing
the post to hide private information. What do you recommend?

Regards,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

Bogdan-Andrei Iancu
Hi Brian,

In frame 0, could you print "*display" and "buf.s" ?

Regards.
Bogdan

[hidden email] wrote:

> Hello Bogdan,
>
> An ven., févr 05, 2010, Bogdan-Andrei Iancu schrieb:
>  
>> [hidden email] wrote:
>>    
>>> I see a variety of things on the page 'Resources/Documentation/
>>> Out Of Memory' http://www.opensips.org/Resources/DocsTsMem, but
>>> I'm not sure if you are referring to it which one exactly then?
>>>
>>>      
>> That is the right one - use the compile instructions from there and
>> run use memlog=10 (to avoid extra debugs from memory) and memdump=1
>> (to get mem dump at shutdown)
>>
>> If a mem issue is found, your opensips will automatically coredump
>> and dump mem stats.
>>
>>    
> Okay I finally got everything compiled, configured, running, and...
>
>   ...dumping. Here's the backtrace:
>
> Core was generated by `/pfx/sbin/opensips -P /pfx/var/opensips/opensips.pid -m 64'.
> Program terminated with signal 11, Segmentation fault.
> [New process 12345    ]
> #0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50, hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
> 271 replace.c: No such file or directory.
>     in replace.c
> (gdb) bt
> #0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50,
>                 hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
> #1  0xd0e1c701 in w_replace_to (msg=0x8355f48, p1=0x8348a88
>                 "\030?3\b\004", p2=0x0)
>     at uac.c:408
> #2  0x08076000 in do_action ()
> #3  0x0807961b in run_action_list ()
> #4  0x08077574 in do_action ()
> #5  0x0807961b in run_action_list ()
> #6  0x08078177 in do_action ()
> #7  0x0807961b in run_action_list ()
> #8  0x080799f0 in run_top_route ()
> #9  0x080be52a in receive_msg ()
> #10 0x080f4c74 in tcp_read_req ()
> #11 0x080f67fd in ?? ()
> #12 0xcc78ebe8 in ?? ()
> (gdb)
>
> It seems to me that the crash came just after trying memcpy(3),
> but looking at modules/uac/replace.c:271 I still don't see the
> reason. I am using uac_replace_from and uac_replace_to in the
> route script but there is no mention of 'no more pkg mem' in
> the logs at all (from replace.c:268.)
>
> There is a lot in the log (I used debug=4) as well as sip_trace
> so I don't know what to post. It will be a nightmare sanitizing
> the post to hide private information. What do you recommend?
>
> Regards,
> Brian
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>  


--
Bogdan-Andrei Iancu
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello Bogdan,

I should tell you that I've recompiled OpenSIPS since the crash.
The changes are unrelated to the core or uac module so there's no
problem, however it might be that the core you've asked to study
reports memory locations that don't match the new running OpenSIPS.

Also, I've changed the route script to use subst() from textops
instead of uac_replace_(from|to) to avoid getting this crash. I'm
still not convinced 100% that it's in replace.c, because it seemed
to me that uac_replace_(from|to) sometimes worked fine.

An lun., févr 08, 2010, Bogdan-Andrei Iancu schrieb:
>In frame 0, could you print "*display" and "buf.s" ?
>
Program terminated with signal 11, Segmentation fault.
[New process 95015    ]
#0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50, hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
271  memcpy( buf.s, display->s, display->len);
(gdb) bt
#0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50, hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
#1  0xd0e1c701 in w_replace_to (msg=0x8355f48, p1=0x8348a88
    "\030?3\b\004", p2=0x0)
    at uac.c:408
#2  0x08076000 in do_action ()
#3  0x0807961b in run_action_list ()
#4  0x08077574 in do_action ()
#5  0x0807961b in run_action_list ()
#6  0x08078177 in do_action ()
#7  0x0807961b in run_action_list ()
#8  0x080799f0 in run_top_route ()
#9  0x080be52a in receive_msg ()
#10 0x080f4c74 in tcp_read_req ()
(gdb) print *display
$1 = {s = 0x1 <Address 0x1 out of bounds>, len = 18}
(gdb) print buf.s
Attempt to extract a component of a value that is not a structure.
(gdb) print *buf.s
Attempt to extract a component of a value that is not a structure.
(gdb)

Regards,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

Bogdan-Andrei Iancu
Hi Brian,

How do you invoke the uac_replace_xxx() which crashes?

Regards,
Bogdan

[hidden email] wrote:

> Hello Bogdan,
>
> I should tell you that I've recompiled OpenSIPS since the crash.
> The changes are unrelated to the core or uac module so there's no
> problem, however it might be that the core you've asked to study
> reports memory locations that don't match the new running OpenSIPS.
>
> Also, I've changed the route script to use subst() from textops
> instead of uac_replace_(from|to) to avoid getting this crash. I'm
> still not convinced 100% that it's in replace.c, because it seemed
> to me that uac_replace_(from|to) sometimes worked fine.
>
> An lun., févr 08, 2010, Bogdan-Andrei Iancu schrieb:
>  
>> In frame 0, could you print "*display" and "buf.s" ?
>>
>>    
> Program terminated with signal 11, Segmentation fault.
> [New process 95015    ]
> #0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50, hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
> 271  memcpy( buf.s, display->s, display->len);
> (gdb) bt
> #0  0xd0e19f23 in replace_uri (msg=0x8355f48, display=0x8046f48, uri=0x8046f50, hdr=0x8356658, rr_param=0xd0e1fa68) at replace.c:271
> #1  0xd0e1c701 in w_replace_to (msg=0x8355f48, p1=0x8348a88
>     "\030?3\b\004", p2=0x0)
>     at uac.c:408
> #2  0x08076000 in do_action ()
> #3  0x0807961b in run_action_list ()
> #4  0x08077574 in do_action ()
> #5  0x0807961b in run_action_list ()
> #6  0x08078177 in do_action ()
> #7  0x0807961b in run_action_list ()
> #8  0x080799f0 in run_top_route ()
> #9  0x080be52a in receive_msg ()
> #10 0x080f4c74 in tcp_read_req ()
> (gdb) print *display
> $1 = {s = 0x1 <Address 0x1 out of bounds>, len = 18}
> (gdb) print buf.s
> Attempt to extract a component of a value that is not a structure.
> (gdb) print *buf.s
> Attempt to extract a component of a value that is not a structure.
> (gdb)
>
> Regards,
> Brian
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>  


--
Bogdan-Andrei Iancu
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello Bogdan,

An jeu., févr 11, 2010, Bogdan-Andrei Iancu schrieb:

>[hidden email] wrote:
>> I should tell you that I've recompiled OpenSIPS since the crash.
>> The changes are unrelated to the core or uac module so there's no
>> problem, however it might be that the core you've asked to study
>> reports memory locations that don't match the new running OpenSIPS.
>>
>> Also, I've changed the route script to use subst() from textops
>> instead of uac_replace_(from|to) to avoid getting this crash. I'm
>> still not convinced 100% that it's in replace.c, because it seemed
>> to me that uac_replace_(from|to) sometimes worked fine.
>>
>How do you invoke the uac_replace_xxx() which crashes?
>
I had:

  uac_replace_to("sip:$tU@$avp(s:realm)");
  uac_replace_from("sip:$avp(s:user)@$fd");

Now I'm using:

  subst('/^(To:[^:]+:)[^@]+@[^>;]+/\1$rU@$avp(s:realm)/');
  subst('/^(From:[^:]+):[^@]+@[^>;]+/\1:$avp(s:user)@$avp(s:realm)/');

...but I can't remember anymore the reason that made me think the
uac_replace_*** functions were at fault for crashes. If you're
interested then I can diagnose this further. In that case please
suggest a course of testing, how you recommend I figure this out.

Regards,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

Bogdan-Andrei Iancu
Hi Brian,

Asking as the trace you previously sent me was showing a crash in the
uac_replace_xxx() function, because of a bogus "display" pointer. I
wanted to see how you call the functions to see if they try to change
the display part or not. And it looks it doesn't, so the "display" param
should be NULL.

Is the crash reproducible ?

Regards,
Bogdan

[hidden email] wrote:

> Hello Bogdan,
>
> An jeu., févr 11, 2010, Bogdan-Andrei Iancu schrieb:
>  
>> [hidden email] wrote:
>>    
>>> I should tell you that I've recompiled OpenSIPS since the crash.
>>> The changes are unrelated to the core or uac module so there's no
>>> problem, however it might be that the core you've asked to study
>>> reports memory locations that don't match the new running OpenSIPS.
>>>
>>> Also, I've changed the route script to use subst() from textops
>>> instead of uac_replace_(from|to) to avoid getting this crash. I'm
>>> still not convinced 100% that it's in replace.c, because it seemed
>>> to me that uac_replace_(from|to) sometimes worked fine.
>>>
>>>      
>> How do you invoke the uac_replace_xxx() which crashes?
>>
>>    
> I had:
>
>   uac_replace_to("sip:$tU@$avp(s:realm)");
>   uac_replace_from("sip:$avp(s:user)@$fd");
>
> Now I'm using:
>
>   subst('/^(To:[^:]+:)[^@]+@[^>;]+/\1$rU@$avp(s:realm)/');
>   subst('/^(From:[^:]+):[^@]+@[^>;]+/\1:$avp(s:user)@$avp(s:realm)/');
>
> ...but I can't remember anymore the reason that made me think the
> uac_replace_*** functions were at fault for crashes. If you're
> interested then I can diagnose this further. In that case please
> suggest a course of testing, how you recommend I figure this out.
>
> Regards,
> Brian
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>  


--
Bogdan-Andrei Iancu
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: What is protocol/port mismatch?

opensipslist

Hello Bogdan,

An lun., févr 22, 2010, Bogdan-Andrei Iancu schrieb:
>Asking as the trace you previously sent me was showing a crash in the
>uac_replace_xxx() function, because of a bogus "display" pointer. I
>wanted to see how you call the functions to see if they try to change
>the display part or not. And it looks it doesn't, so the "display" param
>should be NULL.
>
>Is the crash reproducible ?
>
I'll see if I can reproduce it, but it will take some time. Thanks
for the interest.

Regards,
Brian

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users