dialog bye_on_timeout and other issues

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

dialog bye_on_timeout and other issues

Alex Massover

Hi,

 

I have a strange behavior of OpenSIPS 1.6.2. First dialog module _sometimes_ sends a wrong bye (generated by dialog module on timeout):

 

Here’s a correct one:

 

BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0

Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0

To: <sip:+496925420990@212.179.159.18:5060>;tag=8548

From: <sip:+972542384166@212.179.159.9:5061>;tag=8547

CSeq: 2 BYE

Call-ID: 8547-15512@212.179.159.9

Content-Length: 0

Max-Forwards: 70

 

And here’s a wrong one:

 

BYE sip:212.179.159.9:7640 SIP/2.0

Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0

To: <sip:+496925420989@212.179.159.18:5060>;tag=8547

From: <sip:+972542384166@212.179.159.9:5061>;tag=8546

CSeq: 2 BYE

Call-ID: 8546-15512@212.179.159.9

Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>

Content-Length: 0

Max-Forwards

 

In a wrong one there’s Route header inserted (by mistake?) and the message is cut at Max-Forwards line. It’s missing “:70\r\n”.

 

Both of the BYEs above I got just by running test with SIPP. This can happen even with single call, not related to stress. I.e. one call it might send a correct BYE and another call a corrupted BYE, without any reason, because calls are exactly the same.

 

 

 

Another issue is, looks like t_newtran() is unable to handle retransmissions. In this test UAC and UAS are in the same machine (.9), and you can’t see  INVITE from OpenSIPS (.18) to UAS because it’s fragmented.

 

|Time     | x.x.x.9     | x.x.x.18    |

|13.501   |         INVITE SDP ( MP4V-ES)          |SIP From: sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060

|         |(5061)   ------------------>  (5060)   |

|14.003   |         INVITE SDP ( MP4V-ES)          |SIP From: sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060

|         |(5061)   ------------------>  (5060)   |

|15.005   |         INVITE SDP ( MP4V-ES)          |SIP From: sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060

|         |(5061)   ------------------>  (5060)   |

|15.743   |         100 Trying|                   |SIP Status

|         |(5061)   <------------------  (5060)   |

|15.800   |         181 Call is being forwarded          |SIP Status

|         |(5061)   <------------------  (5060)   |

|15.801   |         100 Trying|                   |SIP Status

|         |(7640)   ------------------>  (5060)   |

|15.801   |         180 Ringing                   |SIP Status

|         |(7640)   ------------------>  (5060)   |

|15.801   |         200 OK SDP ( G723)            |SIP Status

|         |(7640)   ------------------>  (5060)   |

|15.840   |         181 Call is being forwarded          |SIP Status

|         |(5061)   <------------------  (5060)   |

|16.041   |         181 Call is being forwarded          |SIP Status

|         |(5061)   <------------------  (5060)   |

|16.188   |         180 Ringing                   |SIP Status

|         |(5061)   <------------------  (5060)   |

|16.188   |         200 OK SDP ( G723)            |SIP Status

|         |(5061)   <------------------  (5060)   |

|16.189   |         ACK       |                   |SIP Request

|         |(5061)   ------------------>  (5060)   |

|16.302   |         200 OK SDP ( G723)            |SIP Status

|         |(7640)   ------------------>  (5060)   |

|16.357   |         ACK       |                   |SIP Request

|         |(7640)   <------------------  (5060)   |

|16.651   |         200 OK SDP ( G723)            |SIP Status

|         |(5061)   <------------------  (5060)   |

|16.652   |         ACK       |                   |SIP Request

|         |(5061)   ------------------>  (5060)   |

|17.075   |         ACK       |                   |SIP Request

|         |(7640)   <------------------  (5060)   |

|36.730   |         BYE       |                   |SIP Request

|         |(5061)   <------------------  (5060)   |

|36.731   |         BYE       |                   |SIP Request

|         |(7640)   <------------------  (5060)   |

|36.731   |         200 OK    |                   |SIP Status

|         |(5061)   ------------------>  (5060)   |

|36.731   |         200 OK    |                   |SIP Status

|         |(7640)   ------------------>  (5060)   |

 

 

This issue happens during stress test.

 

Any ideas, please? The OpenSIPS 1.6.2 is compiled with system malloc and runs over VMware.

 

--

Best Regards,

Alex Massover

 



This mail was sent via Mail-SeCure System.

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Bogdan-Andrei Iancu
Hi Alex,

First, about the Route hdr - opensips adds a Route hdr in BYE only if
the dialog (on that specific leg) received a 200 OK INVITE with RR
header - can you confirm this at (1) signalling level and (2) at dialog
level (do a dlg_list via MI).

Now, about the bogus BYE - indeed, it is strange - do you use a local
route for accessing the BYEs requests? Attached is a small debugging
patch - please apply it a nd see if you get any WARNINGs at runtime.

Regards,
Bogdan

--
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro



Alex Massover wrote:

>
> Hi,
>
> I have a strange behavior of OpenSIPS 1.6.2. First dialog module
> _/sometimes/_ sends a wrong bye (generated by dialog module on timeout):
>
> Here’s a correct one:
>
> BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
>
> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
>
> To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
>
> From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
>
> CSeq: 2 BYE
>
> Call-ID: 8547-15512@212.179.159.9
>
> Content-Length: 0
>
> Max-Forwards: 70
>
> And here’s a wrong one:
>
> BYE sip:212.179.159.9:7640 SIP/2.0
>
> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
>
> To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
>
> From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
>
> CSeq: 2 BYE
>
> Call-ID: 8546-15512@212.179.159.9
>
> Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
>
> Content-Length: 0
>
> Max-Forwards
>
> In a wrong one there’s Route header inserted (by mistake?) and the
> message is cut at Max-Forwards line. It’s missing “:70\r\n”.
>
> Both of the BYEs above I got just by running test with SIPP. This can
> happen even with single call, not related to stress. I.e. one call it
> might send a correct BYE and another call a corrupted BYE, without any
> reason, because calls are exactly the same.
>
> Another issue is, looks like t_newtran() is unable to handle
> retransmissions. In this test UAC and UAS are in the same machine
> (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS because it’s
> fragmented.
>
> |Time | x.x.x.9 | x.x.x.18 |
>
> |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
>
> | |(5061) ------------------> (5060) |
>
> |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
>
> | |(5061) ------------------> (5060) |
>
> |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
>
> | |(5061) ------------------> (5060) |
>
> |15.743 | 100 Trying| |SIP Status
>
> | |(5061) <------------------ (5060) |
>
> |15.800 | 181 Call is being forwarded |SIP Status
>
> | |(5061) <------------------ (5060) |
>
> |15.801 | 100 Trying| |SIP Status
>
> | |(7640) ------------------> (5060) |
>
> |15.801 | 180 Ringing |SIP Status
>
> | |(7640) ------------------> (5060) |
>
> |15.801 | 200 OK SDP ( G723) |SIP Status
>
> | |(7640) ------------------> (5060) |
>
> |15.840 | 181 Call is being forwarded |SIP Status
>
> | |(5061) <------------------ (5060) |
>
> |16.041 | 181 Call is being forwarded |SIP Status
>
> | |(5061) <------------------ (5060) |
>
> |16.188 | 180 Ringing |SIP Status
>
> | |(5061) <------------------ (5060) |
>
> |16.188 | 200 OK SDP ( G723) |SIP Status
>
> | |(5061) <------------------ (5060) |
>
> |16.189 | ACK | |SIP Request
>
> | |(5061) ------------------> (5060) |
>
> |16.302 | 200 OK SDP ( G723) |SIP Status
>
> | |(7640) ------------------> (5060) |
>
> |16.357 | ACK | |SIP Request
>
> | |(7640) <------------------ (5060) |
>
> |16.651 | 200 OK SDP ( G723) |SIP Status
>
> | |(5061) <------------------ (5060) |
>
> |16.652 | ACK | |SIP Request
>
> | |(5061) ------------------> (5060) |
>
> |17.075 | ACK | |SIP Request
>
> | |(7640) <------------------ (5060) |
>
> |36.730 | BYE | |SIP Request
>
> | |(5061) <------------------ (5060) |
>
> |36.731 | BYE | |SIP Request
>
> | |(7640) <------------------ (5060) |
>
> |36.731 | 200 OK | |SIP Status
>
> | |(5061) ------------------> (5060) |
>
> |36.731 | 200 OK | |SIP Status
>
> | |(7640) ------------------> (5060) |
>
> This issue happens during stress test.
>
> Any ideas, please? The OpenSIPS 1.6.2 is compiled with system malloc
> and runs over VMware.
>
> --
>
> Best Regards,
>
> Alex Massover
>
>
>
> This mail was sent via Mail-SeCure System.
> ------------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>  

Index: modules/dialog/dlg_req_within.c
===================================================================
--- modules/dialog/dlg_req_within.c (revision 6917)
+++ modules/dialog/dlg_req_within.c (working copy)
@@ -226,6 +226,10 @@
  p += extra_hdrs->len;
  }
 
+ if (str_hdr->len != p-str_hdr->s )
+ LM_WARN("BUG in computing extra hdrs: computed len = %d ;"
+ " build len = %d",str_hdr->len,(int)(long)(p-str_hdr->s) );
+
  return 0;
 
 error:

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Alex Massover
Hi Bogdan,

At signaling level 200 OK to INVITE contains RR header (always):

Record-Route: <sip:212.179.159.9:7640>;lr

But at dialog level I have only rare appearance of route set:

        callee_route_set:: <sip:212.179.159.9:7640>;lr

absolutely most of the dialogs do not have it:

opensipsctl fifo dlg_list | grep route_set gives:

        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set:: <sip:212.179.159.9:7640>;lr
        caller_route_set::
        callee_route_set::
        caller_route_set::
        callee_route_set::

And looks that it corresponds with the corrupted BYEs. Most of the BYEs do not have Route headers and they are not corrupted, but some of them have it and they are corrupted.

And there's no warning after applying the patch.


> -----Original Message-----
> From: [hidden email] [mailto:users-
> [hidden email]] On Behalf Of Bogdan-Andrei Iancu
> Sent: יום ב 21 יוני 2010 15:12
> To: OpenSIPS users mailling list
> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
>
> Hi Alex,
>
> First, about the Route hdr - opensips adds a Route hdr in BYE only if
> the dialog (on that specific leg) received a 200 OK INVITE with RR
> header - can you confirm this at (1) signalling level and (2) at dialog
> level (do a dlg_list via MI).
>
> Now, about the bogus BYE - indeed, it is strange - do you use a local
> route for accessing the BYEs requests? Attached is a small debugging
> patch - please apply it a nd see if you get any WARNINGs at runtime.
>
> Regards,
> Bogdan
>
> --
> Bogdan-Andrei Iancu
> OpenSIPS Bootcamp
> 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
>
>
>
> Alex Massover wrote:
> >
> > Hi,
> >
> > I have a strange behavior of OpenSIPS 1.6.2. First dialog module
> > _/sometimes/_ sends a wrong bye (generated by dialog module on
> timeout):
> >
> > Here’s a correct one:
> >
> > BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
> >
> > Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
> >
> > To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
> >
> > From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
> >
> > CSeq: 2 BYE
> >
> > Call-ID: 8547-15512@212.179.159.9
> >
> > Content-Length: 0
> >
> > Max-Forwards: 70
> >
> > And here’s a wrong one:
> >
> > BYE sip:212.179.159.9:7640 SIP/2.0
> >
> > Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
> >
> > To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
> >
> > From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
> >
> > CSeq: 2 BYE
> >
> > Call-ID: 8546-15512@212.179.159.9
> >
> > Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
> >
> > Content-Length: 0
> >
> > Max-Forwards
> >
> > In a wrong one there’s Route header inserted (by mistake?) and the
> > message is cut at Max-Forwards line. It’s missing “:70\r\n”.
> >
> > Both of the BYEs above I got just by running test with SIPP. This can
> > happen even with single call, not related to stress. I.e. one call it
> > might send a correct BYE and another call a corrupted BYE, without
> any
> > reason, because calls are exactly the same.
> >
> > Another issue is, looks like t_newtran() is unable to handle
> > retransmissions. In this test UAC and UAS are in the same machine
> > (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS because
> it’s
> > fragmented.
> >
> > |Time | x.x.x.9 | x.x.x.18 |
> >
> > |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> >
> > | |(5061) ------------------> (5060) |
> >
> > |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> >
> > | |(5061) ------------------> (5060) |
> >
> > |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> >
> > | |(5061) ------------------> (5060) |
> >
> > |15.743 | 100 Trying| |SIP Status
> >
> > | |(5061) <------------------ (5060) |
> >
> > |15.800 | 181 Call is being forwarded |SIP Status
> >
> > | |(5061) <------------------ (5060) |
> >
> > |15.801 | 100 Trying| |SIP Status
> >
> > | |(7640) ------------------> (5060) |
> >
> > |15.801 | 180 Ringing |SIP Status
> >
> > | |(7640) ------------------> (5060) |
> >
> > |15.801 | 200 OK SDP ( G723) |SIP Status
> >
> > | |(7640) ------------------> (5060) |
> >
> > |15.840 | 181 Call is being forwarded |SIP Status
> >
> > | |(5061) <------------------ (5060) |
> >
> > |16.041 | 181 Call is being forwarded |SIP Status
> >
> > | |(5061) <------------------ (5060) |
> >
> > |16.188 | 180 Ringing |SIP Status
> >
> > | |(5061) <------------------ (5060) |
> >
> > |16.188 | 200 OK SDP ( G723) |SIP Status
> >
> > | |(5061) <------------------ (5060) |
> >
> > |16.189 | ACK | |SIP Request
> >
> > | |(5061) ------------------> (5060) |
> >
> > |16.302 | 200 OK SDP ( G723) |SIP Status
> >
> > | |(7640) ------------------> (5060) |
> >
> > |16.357 | ACK | |SIP Request
> >
> > | |(7640) <------------------ (5060) |
> >
> > |16.651 | 200 OK SDP ( G723) |SIP Status
> >
> > | |(5061) <------------------ (5060) |
> >
> > |16.652 | ACK | |SIP Request
> >
> > | |(5061) ------------------> (5060) |
> >
> > |17.075 | ACK | |SIP Request
> >
> > | |(7640) <------------------ (5060) |
> >
> > |36.730 | BYE | |SIP Request
> >
> > | |(5061) <------------------ (5060) |
> >
> > |36.731 | BYE | |SIP Request
> >
> > | |(7640) <------------------ (5060) |
> >
> > |36.731 | 200 OK | |SIP Status
> >
> > | |(5061) ------------------> (5060) |
> >
> > |36.731 | 200 OK | |SIP Status
> >
> > | |(7640) ------------------> (5060) |
> >
> > This issue happens during stress test.
> >
> > Any ideas, please? The OpenSIPS 1.6.2 is compiled with system malloc
> > and runs over VMware.
> >
> > --
> >
> > Best Regards,
> >
> > Alex Massover
> >
> >
> >
> > This mail was sent via Mail-SeCure System.
> > ---------------------------------------------------------------------
> ---
> >
> > _______________________________________________
> > Users mailing list
> > [hidden email]
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
>
> This mail was received via Mail-SeCure System.
>


This mail was sent via Mail-SeCure System.



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Alex Massover
One more observation. In my test UAS answers with 180 and 200OK to INVITE without delay, i.e. the delay between 180 and 200 is only about some ms.

BAD scenario (rare):
180 is not processed by OpenSIPS - received from UAS but not forwarded to UAC
route_set appears at dialog level
BYE contains Route header and is corrupted

GOOD scenario (common):
180 is processed - received and forwarded
Route_set does not appears at dialog level
BYE does not contain Route header and is not corrupted


After putting 2s delay between 180 and 200 in UAS the BAD scenario is not reproduced anymore.

BTW, yes, I intercept BYE with local route to account it.


> -----Original Message-----
> From: [hidden email] [mailto:users-
> [hidden email]] On Behalf Of Alex Massover
> Sent: יום ב 21 יוני 2010 15:52
> To: OpenSIPS users mailling list
> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
>
> Hi Bogdan,
>
> At signaling level 200 OK to INVITE contains RR header (always):
>
> Record-Route: <sip:212.179.159.9:7640>;lr
>
> But at dialog level I have only rare appearance of route set:
>
> callee_route_set:: <sip:212.179.159.9:7640>;lr
>
> absolutely most of the dialogs do not have it:
>
> opensipsctl fifo dlg_list | grep route_set gives:
>
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set:: <sip:212.179.159.9:7640>;lr
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
>
> And looks that it corresponds with the corrupted BYEs. Most of the BYEs
> do not have Route headers and they are not corrupted, but some of them
> have it and they are corrupted.
>
> And there's no warning after applying the patch.
>
>
> > -----Original Message-----
> > From: [hidden email] [mailto:users-
> > [hidden email]] On Behalf Of Bogdan-Andrei Iancu
> > Sent: יום ב 21 יוני 2010 15:12
> > To: OpenSIPS users mailling list
> > Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
> >
> > Hi Alex,
> >
> > First, about the Route hdr - opensips adds a Route hdr in BYE only if
> > the dialog (on that specific leg) received a 200 OK INVITE with RR
> > header - can you confirm this at (1) signalling level and (2) at
> dialog
> > level (do a dlg_list via MI).
> >
> > Now, about the bogus BYE - indeed, it is strange - do you use a local
> > route for accessing the BYEs requests? Attached is a small debugging
> > patch - please apply it a nd see if you get any WARNINGs at runtime.
> >
> > Regards,
> > Bogdan
> >
> > --
> > Bogdan-Andrei Iancu
> > OpenSIPS Bootcamp
> > 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
> >
> >
> >
> > Alex Massover wrote:
> > >
> > > Hi,
> > >
> > > I have a strange behavior of OpenSIPS 1.6.2. First dialog module
> > > _/sometimes/_ sends a wrong bye (generated by dialog module on
> > timeout):
> > >
> > > Here’s a correct one:
> > >
> > > BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
> > >
> > > Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
> > >
> > > To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
> > >
> > > From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
> > >
> > > CSeq: 2 BYE
> > >
> > > Call-ID: 8547-15512@212.179.159.9
> > >
> > > Content-Length: 0
> > >
> > > Max-Forwards: 70
> > >
> > > And here’s a wrong one:
> > >
> > > BYE sip:212.179.159.9:7640 SIP/2.0
> > >
> > > Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
> > >
> > > To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
> > >
> > > From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
> > >
> > > CSeq: 2 BYE
> > >
> > > Call-ID: 8546-15512@212.179.159.9
> > >
> > > Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
> > >
> > > Content-Length: 0
> > >
> > > Max-Forwards
> > >
> > > In a wrong one there’s Route header inserted (by mistake?) and the
> > > message is cut at Max-Forwards line. It’s missing “:70\r\n”.
> > >
> > > Both of the BYEs above I got just by running test with SIPP. This
> can
> > > happen even with single call, not related to stress. I.e. one call
> it
> > > might send a correct BYE and another call a corrupted BYE, without
> > any
> > > reason, because calls are exactly the same.
> > >
> > > Another issue is, looks like t_newtran() is unable to handle
> > > retransmissions. In this test UAC and UAS are in the same machine
> > > (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS because
> > it’s
> > > fragmented.
> > >
> > > |Time | x.x.x.9 | x.x.x.18 |
> > >
> > > |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> > > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > >
> > > | |(5061) ------------------> (5060) |
> > >
> > > |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> > > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > >
> > > | |(5061) ------------------> (5060) |
> > >
> > > |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> > > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > >
> > > | |(5061) ------------------> (5060) |
> > >
> > > |15.743 | 100 Trying| |SIP Status
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |15.800 | 181 Call is being forwarded |SIP Status
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |15.801 | 100 Trying| |SIP Status
> > >
> > > | |(7640) ------------------> (5060) |
> > >
> > > |15.801 | 180 Ringing |SIP Status
> > >
> > > | |(7640) ------------------> (5060) |
> > >
> > > |15.801 | 200 OK SDP ( G723) |SIP Status
> > >
> > > | |(7640) ------------------> (5060) |
> > >
> > > |15.840 | 181 Call is being forwarded |SIP Status
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |16.041 | 181 Call is being forwarded |SIP Status
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |16.188 | 180 Ringing |SIP Status
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |16.188 | 200 OK SDP ( G723) |SIP Status
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |16.189 | ACK | |SIP Request
> > >
> > > | |(5061) ------------------> (5060) |
> > >
> > > |16.302 | 200 OK SDP ( G723) |SIP Status
> > >
> > > | |(7640) ------------------> (5060) |
> > >
> > > |16.357 | ACK | |SIP Request
> > >
> > > | |(7640) <------------------ (5060) |
> > >
> > > |16.651 | 200 OK SDP ( G723) |SIP Status
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |16.652 | ACK | |SIP Request
> > >
> > > | |(5061) ------------------> (5060) |
> > >
> > > |17.075 | ACK | |SIP Request
> > >
> > > | |(7640) <------------------ (5060) |
> > >
> > > |36.730 | BYE | |SIP Request
> > >
> > > | |(5061) <------------------ (5060) |
> > >
> > > |36.731 | BYE | |SIP Request
> > >
> > > | |(7640) <------------------ (5060) |
> > >
> > > |36.731 | 200 OK | |SIP Status
> > >
> > > | |(5061) ------------------> (5060) |
> > >
> > > |36.731 | 200 OK | |SIP Status
> > >
> > > | |(7640) ------------------> (5060) |
> > >
> > > This issue happens during stress test.
> > >
> > > Any ideas, please? The OpenSIPS 1.6.2 is compiled with system
> malloc
> > > and runs over VMware.
> > >
> > > --
> > >
> > > Best Regards,
> > >
> > > Alex Massover
> > >
> > >
> > >
> > > This mail was sent via Mail-SeCure System.
> > > -------------------------------------------------------------------
> --
> > ---
> > >
> > > _______________________________________________
> > > Users mailing list
> > > [hidden email]
> > > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > >
> >
> > This mail was received via Mail-SeCure System.
> >
>
>
> This mail was sent via Mail-SeCure System.
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
> This mail was received via Mail-SeCure System.

This mail was sent via Mail-SeCure System.



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Alex Massover
One more observation. In my test UAS answers with 180 and 200OK to INVITE without delay, i.e. the delay between 180 and 200 is only about some ms.

BAD scenario (rare):
        180 is not processed by OpenSIPS - received from UAS but not forwarded to UAC.
        Route_set appears at dialog level.
        BYE contains Route header and is corrupted.

GOOD scenario (common):
        180 is processed - received and forwarded.
        Route_set does not appears at dialog level.
        BYE does not contain Route header and is not corrupted.


After putting 2s delay between 180 and 200 in UAS the BAD scenario is not reproduced anymore.

BTW, yes, I intercept BYE with local route to account it.

> > -----Original Message-----
> > From: [hidden email] [mailto:users-
> > [hidden email]] On Behalf Of Alex Massover
> > Sent: יום ב 21 יוני 2010 15:52
> > To: OpenSIPS users mailling list
> > Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
> >
> > Hi Bogdan,
> >
> > At signaling level 200 OK to INVITE contains RR header (always):
> >
> > Record-Route: <sip:212.179.159.9:7640>;lr
> >
> > But at dialog level I have only rare appearance of route set:
> >
> > callee_route_set:: <sip:212.179.159.9:7640>;lr
> >
> > absolutely most of the dialogs do not have it:
> >
> > opensipsctl fifo dlg_list | grep route_set gives:
> >
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set:: <sip:212.179.159.9:7640>;lr
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> >
> > And looks that it corresponds with the corrupted BYEs. Most of the
> BYEs
> > do not have Route headers and they are not corrupted, but some of
> them
> > have it and they are corrupted.
> >
> > And there's no warning after applying the patch.
> >
> >
> > > -----Original Message-----
> > > From: [hidden email] [mailto:users-
> > > [hidden email]] On Behalf Of Bogdan-Andrei Iancu
> > > Sent: יום ב 21 יוני 2010 15:12
> > > To: OpenSIPS users mailling list
> > > Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other
> issues
> > >
> > > Hi Alex,
> > >
> > > First, about the Route hdr - opensips adds a Route hdr in BYE only
> if
> > > the dialog (on that specific leg) received a 200 OK INVITE with RR
> > > header - can you confirm this at (1) signalling level and (2) at
> > dialog
> > > level (do a dlg_list via MI).
> > >
> > > Now, about the bogus BYE - indeed, it is strange - do you use a
> local
> > > route for accessing the BYEs requests? Attached is a small
> debugging
> > > patch - please apply it a nd see if you get any WARNINGs at
> runtime.
> > >
> > > Regards,
> > > Bogdan
> > >
> > > --
> > > Bogdan-Andrei Iancu
> > > OpenSIPS Bootcamp
> > > 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
> > >
> > >
> > >
> > > Alex Massover wrote:
> > > >
> > > > Hi,
> > > >
> > > > I have a strange behavior of OpenSIPS 1.6.2. First dialog module
> > > > _/sometimes/_ sends a wrong bye (generated by dialog module on
> > > timeout):
> > > >
> > > > Here’s a correct one:
> > > >
> > > > BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
> > > >
> > > > Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
> > > >
> > > > To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
> > > >
> > > > From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
> > > >
> > > > CSeq: 2 BYE
> > > >
> > > > Call-ID: 8547-15512@212.179.159.9
> > > >
> > > > Content-Length: 0
> > > >
> > > > Max-Forwards: 70
> > > >
> > > > And here’s a wrong one:
> > > >
> > > > BYE sip:212.179.159.9:7640 SIP/2.0
> > > >
> > > > Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
> > > >
> > > > To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
> > > >
> > > > From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
> > > >
> > > > CSeq: 2 BYE
> > > >
> > > > Call-ID: 8546-15512@212.179.159.9
> > > >
> > > > Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
> > > >
> > > > Content-Length: 0
> > > >
> > > > Max-Forwards
> > > >
> > > > In a wrong one there’s Route header inserted (by mistake?) and
> the
> > > > message is cut at Max-Forwards line. It’s missing “:70\r\n”.
> > > >
> > > > Both of the BYEs above I got just by running test with SIPP. This
> > can
> > > > happen even with single call, not related to stress. I.e. one
> call
> > it
> > > > might send a correct BYE and another call a corrupted BYE,
> without
> > > any
> > > > reason, because calls are exactly the same.
> > > >
> > > > Another issue is, looks like t_newtran() is unable to handle
> > > > retransmissions. In this test UAC and UAS are in the same machine
> > > > (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS because
> > > it’s
> > > > fragmented.
> > > >
> > > > |Time | x.x.x.9 | x.x.x.18 |
> > > >
> > > > |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> > > > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > > >
> > > > | |(5061) ------------------> (5060) |
> > > >
> > > > |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> > > > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > > >
> > > > | |(5061) ------------------> (5060) |
> > > >
> > > > |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> > > > sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > > >
> > > > | |(5061) ------------------> (5060) |
> > > >
> > > > |15.743 | 100 Trying| |SIP Status
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |15.800 | 181 Call is being forwarded |SIP Status
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |15.801 | 100 Trying| |SIP Status
> > > >
> > > > | |(7640) ------------------> (5060) |
> > > >
> > > > |15.801 | 180 Ringing |SIP Status
> > > >
> > > > | |(7640) ------------------> (5060) |
> > > >
> > > > |15.801 | 200 OK SDP ( G723) |SIP Status
> > > >
> > > > | |(7640) ------------------> (5060) |
> > > >
> > > > |15.840 | 181 Call is being forwarded |SIP Status
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |16.041 | 181 Call is being forwarded |SIP Status
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |16.188 | 180 Ringing |SIP Status
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |16.188 | 200 OK SDP ( G723) |SIP Status
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |16.189 | ACK | |SIP Request
> > > >
> > > > | |(5061) ------------------> (5060) |
> > > >
> > > > |16.302 | 200 OK SDP ( G723) |SIP Status
> > > >
> > > > | |(7640) ------------------> (5060) |
> > > >
> > > > |16.357 | ACK | |SIP Request
> > > >
> > > > | |(7640) <------------------ (5060) |
> > > >
> > > > |16.651 | 200 OK SDP ( G723) |SIP Status
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |16.652 | ACK | |SIP Request
> > > >
> > > > | |(5061) ------------------> (5060) |
> > > >
> > > > |17.075 | ACK | |SIP Request
> > > >
> > > > | |(7640) <------------------ (5060) |
> > > >
> > > > |36.730 | BYE | |SIP Request
> > > >
> > > > | |(5061) <------------------ (5060) |
> > > >
> > > > |36.731 | BYE | |SIP Request
> > > >
> > > > | |(7640) <------------------ (5060) |
> > > >
> > > > |36.731 | 200 OK | |SIP Status
> > > >
> > > > | |(5061) ------------------> (5060) |
> > > >
> > > > |36.731 | 200 OK | |SIP Status
> > > >
> > > > | |(7640) ------------------> (5060) |
> > > >
> > > > This issue happens during stress test.
> > > >
> > > > Any ideas, please? The OpenSIPS 1.6.2 is compiled with system
> > malloc
> > > > and runs over VMware.
> > > >
> > > > --
> > > >
> > > > Best Regards,
> > > >
> > > > Alex Massover
> > > >
> > > >
> > > >
> > > > This mail was sent via Mail-SeCure System.
> > > > -----------------------------------------------------------------
> --
> > --
> > > ---
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > [hidden email]
> > > > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > > >
> > >
> > > This mail was received via Mail-SeCure System.
> > >
> >
> >
> > This mail was sent via Mail-SeCure System.
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > [hidden email]
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
> > This mail was received via Mail-SeCure System.
>
> This mail was sent via Mail-SeCure System.
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
> This mail was received via Mail-SeCure System.

This mail was sent via Mail-SeCure System.



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Bogdan-Andrei Iancu
In reply to this post by Alex Massover
Hi Alex,

You mentioned you generate these calls with SIPP - right ? are all of
them the same ?

looking at signalling, we should see why that calls have the
callee_route_set (some RR done after your proxy)..

also, have you checked the patch I sent you?

Regards,
Bogdan

Alex Massover wrote:

> Hi Bogdan,
>
> At signaling level 200 OK to INVITE contains RR header (always):
>
> Record-Route: <sip:212.179.159.9:7640>;lr
>
> But at dialog level I have only rare appearance of route set:
>
> callee_route_set:: <sip:212.179.159.9:7640>;lr
>
> absolutely most of the dialogs do not have it:
>
> opensipsctl fifo dlg_list | grep route_set gives:
>
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set:: <sip:212.179.159.9:7640>;lr
> caller_route_set::
> callee_route_set::
> caller_route_set::
> callee_route_set::
>
> And looks that it corresponds with the corrupted BYEs. Most of the BYEs do not have Route headers and they are not corrupted, but some of them have it and they are corrupted.
>
> And there's no warning after applying the patch.
>
>
>  
>> -----Original Message-----
>> From: [hidden email] [mailto:users-
>> [hidden email]] On Behalf Of Bogdan-Andrei Iancu
>> Sent: יום ב 21 יוני 2010 15:12
>> To: OpenSIPS users mailling list
>> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
>>
>> Hi Alex,
>>
>> First, about the Route hdr - opensips adds a Route hdr in BYE only if
>> the dialog (on that specific leg) received a 200 OK INVITE with RR
>> header - can you confirm this at (1) signalling level and (2) at dialog
>> level (do a dlg_list via MI).
>>
>> Now, about the bogus BYE - indeed, it is strange - do you use a local
>> route for accessing the BYEs requests? Attached is a small debugging
>> patch - please apply it a nd see if you get any WARNINGs at runtime.
>>
>> Regards,
>> Bogdan
>>
>> --
>> Bogdan-Andrei Iancu
>> OpenSIPS Bootcamp
>> 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
>>
>>
>>
>> Alex Massover wrote:
>>    
>>> Hi,
>>>
>>> I have a strange behavior of OpenSIPS 1.6.2. First dialog module
>>> _/sometimes/_ sends a wrong bye (generated by dialog module on
>>>      
>> timeout):
>>    
>>> Here’s a correct one:
>>>
>>> BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
>>>
>>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
>>>
>>> To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
>>>
>>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
>>>
>>> CSeq: 2 BYE
>>>
>>> Call-ID: 8547-15512@212.179.159.9
>>>
>>> Content-Length: 0
>>>
>>> Max-Forwards: 70
>>>
>>> And here’s a wrong one:
>>>
>>> BYE sip:212.179.159.9:7640 SIP/2.0
>>>
>>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
>>>
>>> To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
>>>
>>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
>>>
>>> CSeq: 2 BYE
>>>
>>> Call-ID: 8546-15512@212.179.159.9
>>>
>>> Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
>>>
>>> Content-Length: 0
>>>
>>> Max-Forwards
>>>
>>> In a wrong one there’s Route header inserted (by mistake?) and the
>>> message is cut at Max-Forwards line. It’s missing “:70\r\n”.
>>>
>>> Both of the BYEs above I got just by running test with SIPP. This can
>>> happen even with single call, not related to stress. I.e. one call it
>>> might send a correct BYE and another call a corrupted BYE, without
>>>      
>> any
>>    
>>> reason, because calls are exactly the same.
>>>
>>> Another issue is, looks like t_newtran() is unable to handle
>>> retransmissions. In this test UAC and UAS are in the same machine
>>> (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS because
>>>      
>> it’s
>>    
>>> fragmented.
>>>
>>> |Time | x.x.x.9 | x.x.x.18 |
>>>
>>> |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
>>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
>>>
>>> | |(5061) ------------------> (5060) |
>>>
>>> |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
>>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
>>>
>>> | |(5061) ------------------> (5060) |
>>>
>>> |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
>>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
>>>
>>> | |(5061) ------------------> (5060) |
>>>
>>> |15.743 | 100 Trying| |SIP Status
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |15.800 | 181 Call is being forwarded |SIP Status
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |15.801 | 100 Trying| |SIP Status
>>>
>>> | |(7640) ------------------> (5060) |
>>>
>>> |15.801 | 180 Ringing |SIP Status
>>>
>>> | |(7640) ------------------> (5060) |
>>>
>>> |15.801 | 200 OK SDP ( G723) |SIP Status
>>>
>>> | |(7640) ------------------> (5060) |
>>>
>>> |15.840 | 181 Call is being forwarded |SIP Status
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |16.041 | 181 Call is being forwarded |SIP Status
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |16.188 | 180 Ringing |SIP Status
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |16.188 | 200 OK SDP ( G723) |SIP Status
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |16.189 | ACK | |SIP Request
>>>
>>> | |(5061) ------------------> (5060) |
>>>
>>> |16.302 | 200 OK SDP ( G723) |SIP Status
>>>
>>> | |(7640) ------------------> (5060) |
>>>
>>> |16.357 | ACK | |SIP Request
>>>
>>> | |(7640) <------------------ (5060) |
>>>
>>> |16.651 | 200 OK SDP ( G723) |SIP Status
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |16.652 | ACK | |SIP Request
>>>
>>> | |(5061) ------------------> (5060) |
>>>
>>> |17.075 | ACK | |SIP Request
>>>
>>> | |(7640) <------------------ (5060) |
>>>
>>> |36.730 | BYE | |SIP Request
>>>
>>> | |(5061) <------------------ (5060) |
>>>
>>> |36.731 | BYE | |SIP Request
>>>
>>> | |(7640) <------------------ (5060) |
>>>
>>> |36.731 | 200 OK | |SIP Status
>>>
>>> | |(5061) ------------------> (5060) |
>>>
>>> |36.731 | 200 OK | |SIP Status
>>>
>>> | |(7640) ------------------> (5060) |
>>>
>>> This issue happens during stress test.
>>>
>>> Any ideas, please? The OpenSIPS 1.6.2 is compiled with system malloc
>>> and runs over VMware.
>>>
>>> --
>>>
>>> Best Regards,
>>>
>>> Alex Massover
>>>
>>>
>>>
>>> This mail was sent via Mail-SeCure System.
>>> ---------------------------------------------------------------------
>>>      
>> ---
>>    
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>>>      
>> This mail was received via Mail-SeCure System.
>>
>>    
>
>
> This mail was sent via Mail-SeCure System.
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>  


--
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Bogdan-Andrei Iancu
In reply to this post by Alex Massover
Alex Massover wrote:
> One more observation. In my test UAS answers with 180 and 200OK to INVITE without delay, i.e. the delay between 180 and 200 is only about some ms.
>
> BAD scenario (rare):
> 180 is not processed by OpenSIPS - received from UAS but not forwarded to UAC
> route_set appears at dialog level
> BYE contains Route header and is corrupted
>  
Hi  Alex,

could you get the trace and logs (debug=4) for one of these BAD scenario ?

Regards,
Bogdan

--
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Alex Massover
In reply to this post by Bogdan-Andrei Iancu
Hi,

> You mentioned you generate these calls with SIPP - right ? are all of
> them the same ?
>
[Alex] Yes, they all are the same.

> looking at signalling, we should see why that calls have the
> callee_route_set (some RR done after your proxy)..
>
[Alex] UAS is also a SIPP script. But it's not the reason for OpenSIPS to send corrupted BYE anyway ;)

> also, have you checked the patch I sent you?

[Alex]
Yes, I get no warnings.

> Regards,
> Bogdan
>
> Alex Massover wrote:
> > Hi Bogdan,
> >
> > At signaling level 200 OK to INVITE contains RR header (always):
> >
> > Record-Route: <sip:212.179.159.9:7640>;lr
> >
> > But at dialog level I have only rare appearance of route set:
> >
> > callee_route_set:: <sip:212.179.159.9:7640>;lr
> >
> > absolutely most of the dialogs do not have it:
> >
> > opensipsctl fifo dlg_list | grep route_set gives:
> >
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set:: <sip:212.179.159.9:7640>;lr
> > caller_route_set::
> > callee_route_set::
> > caller_route_set::
> > callee_route_set::
> >
> > And looks that it corresponds with the corrupted BYEs. Most of the
> BYEs do not have Route headers and they are not corrupted, but some of
> them have it and they are corrupted.
> >
> > And there's no warning after applying the patch.
> >
> >
> >
> >> -----Original Message-----
> >> From: [hidden email] [mailto:users-
> >> [hidden email]] On Behalf Of Bogdan-Andrei Iancu
> >> Sent: יום ב 21 יוני 2010 15:12
> >> To: OpenSIPS users mailling list
> >> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
> >>
> >> Hi Alex,
> >>
> >> First, about the Route hdr - opensips adds a Route hdr in BYE only
> if
> >> the dialog (on that specific leg) received a 200 OK INVITE with RR
> >> header - can you confirm this at (1) signalling level and (2) at
> dialog
> >> level (do a dlg_list via MI).
> >>
> >> Now, about the bogus BYE - indeed, it is strange - do you use a
> local
> >> route for accessing the BYEs requests? Attached is a small debugging
> >> patch - please apply it a nd see if you get any WARNINGs at runtime.
> >>
> >> Regards,
> >> Bogdan
> >>
> >> --
> >> Bogdan-Andrei Iancu
> >> OpenSIPS Bootcamp
> >> 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
> >>
> >>
> >>
> >> Alex Massover wrote:
> >>
> >>> Hi,
> >>>
> >>> I have a strange behavior of OpenSIPS 1.6.2. First dialog module
> >>> _/sometimes/_ sends a wrong bye (generated by dialog module on
> >>>
> >> timeout):
> >>
> >>> Here’s a correct one:
> >>>
> >>> BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
> >>>
> >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
> >>>
> >>> To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
> >>>
> >>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
> >>>
> >>> CSeq: 2 BYE
> >>>
> >>> Call-ID: 8547-15512@212.179.159.9
> >>>
> >>> Content-Length: 0
> >>>
> >>> Max-Forwards: 70
> >>>
> >>> And here’s a wrong one:
> >>>
> >>> BYE sip:212.179.159.9:7640 SIP/2.0
> >>>
> >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
> >>>
> >>> To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
> >>>
> >>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
> >>>
> >>> CSeq: 2 BYE
> >>>
> >>> Call-ID: 8546-15512@212.179.159.9
> >>>
> >>> Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
> >>>
> >>> Content-Length: 0
> >>>
> >>> Max-Forwards
> >>>
> >>> In a wrong one there’s Route header inserted (by mistake?) and the
> >>> message is cut at Max-Forwards line. It’s missing “:70\r\n”.
> >>>
> >>> Both of the BYEs above I got just by running test with SIPP. This
> can
> >>> happen even with single call, not related to stress. I.e. one call
> it
> >>> might send a correct BYE and another call a corrupted BYE, without
> >>>
> >> any
> >>
> >>> reason, because calls are exactly the same.
> >>>
> >>> Another issue is, looks like t_newtran() is unable to handle
> >>> retransmissions. In this test UAC and UAS are in the same machine
> >>> (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS because
> >>>
> >> it’s
> >>
> >>> fragmented.
> >>>
> >>> |Time | x.x.x.9 | x.x.x.18 |
> >>>
> >>> |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> >>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> >>>
> >>> | |(5061) ------------------> (5060) |
> >>>
> >>> |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> >>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> >>>
> >>> | |(5061) ------------------> (5060) |
> >>>
> >>> |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> >>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> >>>
> >>> | |(5061) ------------------> (5060) |
> >>>
> >>> |15.743 | 100 Trying| |SIP Status
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |15.800 | 181 Call is being forwarded |SIP Status
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |15.801 | 100 Trying| |SIP Status
> >>>
> >>> | |(7640) ------------------> (5060) |
> >>>
> >>> |15.801 | 180 Ringing |SIP Status
> >>>
> >>> | |(7640) ------------------> (5060) |
> >>>
> >>> |15.801 | 200 OK SDP ( G723) |SIP Status
> >>>
> >>> | |(7640) ------------------> (5060) |
> >>>
> >>> |15.840 | 181 Call is being forwarded |SIP Status
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |16.041 | 181 Call is being forwarded |SIP Status
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |16.188 | 180 Ringing |SIP Status
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |16.188 | 200 OK SDP ( G723) |SIP Status
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |16.189 | ACK | |SIP Request
> >>>
> >>> | |(5061) ------------------> (5060) |
> >>>
> >>> |16.302 | 200 OK SDP ( G723) |SIP Status
> >>>
> >>> | |(7640) ------------------> (5060) |
> >>>
> >>> |16.357 | ACK | |SIP Request
> >>>
> >>> | |(7640) <------------------ (5060) |
> >>>
> >>> |16.651 | 200 OK SDP ( G723) |SIP Status
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |16.652 | ACK | |SIP Request
> >>>
> >>> | |(5061) ------------------> (5060) |
> >>>
> >>> |17.075 | ACK | |SIP Request
> >>>
> >>> | |(7640) <------------------ (5060) |
> >>>
> >>> |36.730 | BYE | |SIP Request
> >>>
> >>> | |(5061) <------------------ (5060) |
> >>>
> >>> |36.731 | BYE | |SIP Request
> >>>
> >>> | |(7640) <------------------ (5060) |
> >>>
> >>> |36.731 | 200 OK | |SIP Status
> >>>
> >>> | |(5061) ------------------> (5060) |
> >>>
> >>> |36.731 | 200 OK | |SIP Status
> >>>
> >>> | |(7640) ------------------> (5060) |
> >>>
> >>> This issue happens during stress test.
> >>>
> >>> Any ideas, please? The OpenSIPS 1.6.2 is compiled with system
> malloc
> >>> and runs over VMware.
> >>>
> >>> --
> >>>
> >>> Best Regards,
> >>>
> >>> Alex Massover
> >>>
> >>>
> >>>
> >>> This mail was sent via Mail-SeCure System.
> >>> -------------------------------------------------------------------
> --
> >>>
> >> ---
> >>
> >>> _______________________________________________
> >>> Users mailing list
> >>> [hidden email]
> >>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >>>
> >>>
> >> This mail was received via Mail-SeCure System.
> >>
> >>
> >
> >
> > This mail was sent via Mail-SeCure System.
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > [hidden email]
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
>
>
> --
> Bogdan-Andrei Iancu
> OpenSIPS Bootcamp
> 20 - 24 September 2010, Frankfurt, Germany
> www.voice-system.ro
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
> This mail was received via Mail-SeCure System.

This mail was sent via Mail-SeCure System.
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Alex Massover
Hi Bogdan,

Have you received the trace and the log I sent?

Also one more observation. The problem is not reproducible with children=1 but is reproducible with children>=1.

I noticed long time ago that children>=1 misbehaves:

a) performance is worse instead of getting better
b) it can hang
c) with big number of children it hangs/deadlocks very easily

I might be mistaken but it looks for me that something is wrong either with interprocessing or with locks or with both of them.

A corrupted BYE is just a symptom of bigger problem.

For example in this scenario 180 is processed within one child and 200 within another child, there's race condition/locks misbehave/something wrong with memory, and as a result something gets corrupted and wrong BYE is sent.

> -----Original Message-----
> From: [hidden email] [mailto:users-
> [hidden email]] On Behalf Of Alex Massover
> Sent: יום ג 22 יוני 2010 12:04
> To: OpenSIPS users mailling list
> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
>
> Hi,
>
> > You mentioned you generate these calls with SIPP - right ? are all of
> > them the same ?
> >
> [Alex] Yes, they all are the same.
>
> > looking at signalling, we should see why that calls have the
> > callee_route_set (some RR done after your proxy)..
> >
> [Alex] UAS is also a SIPP script. But it's not the reason for OpenSIPS
> to send corrupted BYE anyway ;)
>
> > also, have you checked the patch I sent you?
>
> [Alex]
> Yes, I get no warnings.
>
> > Regards,
> > Bogdan
> >
> > Alex Massover wrote:
> > > Hi Bogdan,
> > >
> > > At signaling level 200 OK to INVITE contains RR header (always):
> > >
> > > Record-Route: <sip:212.179.159.9:7640>;lr
> > >
> > > But at dialog level I have only rare appearance of route set:
> > >
> > > callee_route_set:: <sip:212.179.159.9:7640>;lr
> > >
> > > absolutely most of the dialogs do not have it:
> > >
> > > opensipsctl fifo dlg_list | grep route_set gives:
> > >
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set:: <sip:212.179.159.9:7640>;lr
> > > caller_route_set::
> > > callee_route_set::
> > > caller_route_set::
> > > callee_route_set::
> > >
> > > And looks that it corresponds with the corrupted BYEs. Most of the
> > BYEs do not have Route headers and they are not corrupted, but some
> of
> > them have it and they are corrupted.
> > >
> > > And there's no warning after applying the patch.
> > >
> > >
> > >
> > >> -----Original Message-----
> > >> From: [hidden email] [mailto:users-
> > >> [hidden email]] On Behalf Of Bogdan-Andrei Iancu
> > >> Sent: יום ב 21 יוני 2010 15:12
> > >> To: OpenSIPS users mailling list
> > >> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other
> issues
> > >>
> > >> Hi Alex,
> > >>
> > >> First, about the Route hdr - opensips adds a Route hdr in BYE only
> > if
> > >> the dialog (on that specific leg) received a 200 OK INVITE with RR
> > >> header - can you confirm this at (1) signalling level and (2) at
> > dialog
> > >> level (do a dlg_list via MI).
> > >>
> > >> Now, about the bogus BYE - indeed, it is strange - do you use a
> > local
> > >> route for accessing the BYEs requests? Attached is a small
> debugging
> > >> patch - please apply it a nd see if you get any WARNINGs at
> runtime.
> > >>
> > >> Regards,
> > >> Bogdan
> > >>
> > >> --
> > >> Bogdan-Andrei Iancu
> > >> OpenSIPS Bootcamp
> > >> 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
> > >>
> > >>
> > >>
> > >> Alex Massover wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> I have a strange behavior of OpenSIPS 1.6.2. First dialog module
> > >>> _/sometimes/_ sends a wrong bye (generated by dialog module on
> > >>>
> > >> timeout):
> > >>
> > >>> Here’s a correct one:
> > >>>
> > >>> BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
> > >>>
> > >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
> > >>>
> > >>> To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
> > >>>
> > >>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
> > >>>
> > >>> CSeq: 2 BYE
> > >>>
> > >>> Call-ID: 8547-15512@212.179.159.9
> > >>>
> > >>> Content-Length: 0
> > >>>
> > >>> Max-Forwards: 70
> > >>>
> > >>> And here’s a wrong one:
> > >>>
> > >>> BYE sip:212.179.159.9:7640 SIP/2.0
> > >>>
> > >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
> > >>>
> > >>> To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
> > >>>
> > >>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
> > >>>
> > >>> CSeq: 2 BYE
> > >>>
> > >>> Call-ID: 8546-15512@212.179.159.9
> > >>>
> > >>> Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
> > >>>
> > >>> Content-Length: 0
> > >>>
> > >>> Max-Forwards
> > >>>
> > >>> In a wrong one there’s Route header inserted (by mistake?) and
> the
> > >>> message is cut at Max-Forwards line. It’s missing “:70\r\n”.
> > >>>
> > >>> Both of the BYEs above I got just by running test with SIPP. This
> > can
> > >>> happen even with single call, not related to stress. I.e. one
> call
> > it
> > >>> might send a correct BYE and another call a corrupted BYE,
> without
> > >>>
> > >> any
> > >>
> > >>> reason, because calls are exactly the same.
> > >>>
> > >>> Another issue is, looks like t_newtran() is unable to handle
> > >>> retransmissions. In this test UAC and UAS are in the same machine
> > >>> (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS because
> > >>>
> > >> it’s
> > >>
> > >>> fragmented.
> > >>>
> > >>> |Time | x.x.x.9 | x.x.x.18 |
> > >>>
> > >>> |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> > >>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > >>>
> > >>> | |(5061) ------------------> (5060) |
> > >>>
> > >>> |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> > >>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > >>>
> > >>> | |(5061) ------------------> (5060) |
> > >>>
> > >>> |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> > >>> sip:+[hidden email].9:5061 To:sip:+[hidden email].18:5060
> > >>>
> > >>> | |(5061) ------------------> (5060) |
> > >>>
> > >>> |15.743 | 100 Trying| |SIP Status
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |15.800 | 181 Call is being forwarded |SIP Status
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |15.801 | 100 Trying| |SIP Status
> > >>>
> > >>> | |(7640) ------------------> (5060) |
> > >>>
> > >>> |15.801 | 180 Ringing |SIP Status
> > >>>
> > >>> | |(7640) ------------------> (5060) |
> > >>>
> > >>> |15.801 | 200 OK SDP ( G723) |SIP Status
> > >>>
> > >>> | |(7640) ------------------> (5060) |
> > >>>
> > >>> |15.840 | 181 Call is being forwarded |SIP Status
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |16.041 | 181 Call is being forwarded |SIP Status
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |16.188 | 180 Ringing |SIP Status
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |16.188 | 200 OK SDP ( G723) |SIP Status
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |16.189 | ACK | |SIP Request
> > >>>
> > >>> | |(5061) ------------------> (5060) |
> > >>>
> > >>> |16.302 | 200 OK SDP ( G723) |SIP Status
> > >>>
> > >>> | |(7640) ------------------> (5060) |
> > >>>
> > >>> |16.357 | ACK | |SIP Request
> > >>>
> > >>> | |(7640) <------------------ (5060) |
> > >>>
> > >>> |16.651 | 200 OK SDP ( G723) |SIP Status
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |16.652 | ACK | |SIP Request
> > >>>
> > >>> | |(5061) ------------------> (5060) |
> > >>>
> > >>> |17.075 | ACK | |SIP Request
> > >>>
> > >>> | |(7640) <------------------ (5060) |
> > >>>
> > >>> |36.730 | BYE | |SIP Request
> > >>>
> > >>> | |(5061) <------------------ (5060) |
> > >>>
> > >>> |36.731 | BYE | |SIP Request
> > >>>
> > >>> | |(7640) <------------------ (5060) |
> > >>>
> > >>> |36.731 | 200 OK | |SIP Status
> > >>>
> > >>> | |(5061) ------------------> (5060) |
> > >>>
> > >>> |36.731 | 200 OK | |SIP Status
> > >>>
> > >>> | |(7640) ------------------> (5060) |
> > >>>
> > >>> This issue happens during stress test.
> > >>>
> > >>> Any ideas, please? The OpenSIPS 1.6.2 is compiled with system
> > malloc
> > >>> and runs over VMware.
> > >>>
> > >>> --
> > >>>
> > >>> Best Regards,
> > >>>
> > >>> Alex Massover
> > >>>
> > >>>
> > >>>
> > >>> This mail was sent via Mail-SeCure System.
> > >>> -----------------------------------------------------------------
> --
> > --
> > >>>
> > >> ---
> > >>
> > >>> _______________________________________________
> > >>> Users mailing list
> > >>> [hidden email]
> > >>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > >>>
> > >>>
> > >> This mail was received via Mail-SeCure System.
> > >>
> > >>
> > >
> > >
> > > This mail was sent via Mail-SeCure System.
> > >
> > >
> > >
> > > _______________________________________________
> > > Users mailing list
> > > [hidden email]
> > > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > >
> >
> >
> > --
> > Bogdan-Andrei Iancu
> > OpenSIPS Bootcamp
> > 20 - 24 September 2010, Frankfurt, Germany
> > www.voice-system.ro
> >
> >
> > _______________________________________________
> > Users mailing list
> > [hidden email]
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
> > This mail was received via Mail-SeCure System.
>
> This mail was sent via Mail-SeCure System.
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
> This mail was received via Mail-SeCure System.

This mail was sent via Mail-SeCure System.
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Alex Massover
Hi Bogdan,

I was able to reproduce the corrupted BYE problem even with children=1. With single working child the problem happens only when OpenSIPS is under massive stress/at maximum performance level (with blocking auth 20 cps is enough to push OpenSIPS into a corner).

With children>=1 the problem is sometimes reproducible even with single call.

> -----Original Message-----
> From: [hidden email] [mailto:users-
> [hidden email]] On Behalf Of Alex Massover
> Sent: יום ד 23 יוני 2010 11:09
> To: OpenSIPS users mailling list
> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
>
> Hi Bogdan,
>
> Have you received the trace and the log I sent?
>
> Also one more observation. The problem is not reproducible with
> children=1 but is reproducible with children>=1.
>
> I noticed long time ago that children>=1 misbehaves:
>
> a) performance is worse instead of getting better
> b) it can hang
> c) with big number of children it hangs/deadlocks very easily
>
> I might be mistaken but it looks for me that something is wrong either
> with interprocessing or with locks or with both of them.
>
> A corrupted BYE is just a symptom of bigger problem.
>
> For example in this scenario 180 is processed within one child and 200
> within another child, there's race condition/locks misbehave/something
> wrong with memory, and as a result something gets corrupted and wrong
> BYE is sent.
>
> > -----Original Message-----
> > From: [hidden email] [mailto:users-
> > [hidden email]] On Behalf Of Alex Massover
> > Sent: יום ג 22 יוני 2010 12:04
> > To: OpenSIPS users mailling list
> > Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
> >
> > Hi,
> >
> > > You mentioned you generate these calls with SIPP - right ? are all
> of
> > > them the same ?
> > >
> > [Alex] Yes, they all are the same.
> >
> > > looking at signalling, we should see why that calls have the
> > > callee_route_set (some RR done after your proxy)..
> > >
> > [Alex] UAS is also a SIPP script. But it's not the reason for
> OpenSIPS
> > to send corrupted BYE anyway ;)
> >
> > > also, have you checked the patch I sent you?
> >
> > [Alex]
> > Yes, I get no warnings.
> >
> > > Regards,
> > > Bogdan
> > >
> > > Alex Massover wrote:
> > > > Hi Bogdan,
> > > >
> > > > At signaling level 200 OK to INVITE contains RR header (always):
> > > >
> > > > Record-Route: <sip:212.179.159.9:7640>;lr
> > > >
> > > > But at dialog level I have only rare appearance of route set:
> > > >
> > > > callee_route_set:: <sip:212.179.159.9:7640>;lr
> > > >
> > > > absolutely most of the dialogs do not have it:
> > > >
> > > > opensipsctl fifo dlg_list | grep route_set gives:
> > > >
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set:: <sip:212.179.159.9:7640>;lr
> > > > caller_route_set::
> > > > callee_route_set::
> > > > caller_route_set::
> > > > callee_route_set::
> > > >
> > > > And looks that it corresponds with the corrupted BYEs. Most of
> the
> > > BYEs do not have Route headers and they are not corrupted, but some
> > of
> > > them have it and they are corrupted.
> > > >
> > > > And there's no warning after applying the patch.
> > > >
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: [hidden email] [mailto:users-
> > > >> [hidden email]] On Behalf Of Bogdan-Andrei Iancu
> > > >> Sent: יום ב 21 יוני 2010 15:12
> > > >> To: OpenSIPS users mailling list
> > > >> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other
> > issues
> > > >>
> > > >> Hi Alex,
> > > >>
> > > >> First, about the Route hdr - opensips adds a Route hdr in BYE
> only
> > > if
> > > >> the dialog (on that specific leg) received a 200 OK INVITE with
> RR
> > > >> header - can you confirm this at (1) signalling level and (2) at
> > > dialog
> > > >> level (do a dlg_list via MI).
> > > >>
> > > >> Now, about the bogus BYE - indeed, it is strange - do you use a
> > > local
> > > >> route for accessing the BYEs requests? Attached is a small
> > debugging
> > > >> patch - please apply it a nd see if you get any WARNINGs at
> > runtime.
> > > >>
> > > >> Regards,
> > > >> Bogdan
> > > >>
> > > >> --
> > > >> Bogdan-Andrei Iancu
> > > >> OpenSIPS Bootcamp
> > > >> 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
> > > >>
> > > >>
> > > >>
> > > >> Alex Massover wrote:
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> I have a strange behavior of OpenSIPS 1.6.2. First dialog
> module
> > > >>> _/sometimes/_ sends a wrong bye (generated by dialog module on
> > > >>>
> > > >> timeout):
> > > >>
> > > >>> Here’s a correct one:
> > > >>>
> > > >>> BYE sip:972123456789@212.179.159.9:7640;transport=UDP SIP/2.0
> > > >>>
> > > >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
> > > >>>
> > > >>> To: <sip:+496925420990@212.179.159.18:5060>;tag=8548
> > > >>>
> > > >>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8547
> > > >>>
> > > >>> CSeq: 2 BYE
> > > >>>
> > > >>> Call-ID: 8547-15512@212.179.159.9
> > > >>>
> > > >>> Content-Length: 0
> > > >>>
> > > >>> Max-Forwards: 70
> > > >>>
> > > >>> And here’s a wrong one:
> > > >>>
> > > >>> BYE sip:212.179.159.9:7640 SIP/2.0
> > > >>>
> > > >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
> > > >>>
> > > >>> To: <sip:+496925420989@212.179.159.18:5060>;tag=8547
> > > >>>
> > > >>> From: <sip:+972542384166@212.179.159.9:5061>;tag=8546
> > > >>>
> > > >>> CSeq: 2 BYE
> > > >>>
> > > >>> Call-ID: 8546-15512@212.179.159.9
> > > >>>
> > > >>> Route: <sip:972123456789@212.179.159.9:7640;transport=UDP>
> > > >>>
> > > >>> Content-Length: 0
> > > >>>
> > > >>> Max-Forwards
> > > >>>
> > > >>> In a wrong one there’s Route header inserted (by mistake?) and
> > the
> > > >>> message is cut at Max-Forwards line. It’s missing “:70\r\n”.
> > > >>>
> > > >>> Both of the BYEs above I got just by running test with SIPP.
> This
> > > can
> > > >>> happen even with single call, not related to stress. I.e. one
> > call
> > > it
> > > >>> might send a correct BYE and another call a corrupted BYE,
> > without
> > > >>>
> > > >> any
> > > >>
> > > >>> reason, because calls are exactly the same.
> > > >>>
> > > >>> Another issue is, looks like t_newtran() is unable to handle
> > > >>> retransmissions. In this test UAC and UAS are in the same
> machine
> > > >>> (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS
> because
> > > >>>
> > > >> it’s
> > > >>
> > > >>> fragmented.
> > > >>>
> > > >>> |Time | x.x.x.9 | x.x.x.18 |
> > > >>>
> > > >>> |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> > > >>> sip:+[hidden email].9:5061
> To:sip:+[hidden email].18:5060
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> > > >>> sip:+[hidden email].9:5061
> To:sip:+[hidden email].18:5060
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> > > >>> sip:+[hidden email].9:5061
> To:sip:+[hidden email].18:5060
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |15.743 | 100 Trying| |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |15.800 | 181 Call is being forwarded |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |15.801 | 100 Trying| |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |15.801 | 180 Ringing |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |15.801 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |15.840 | 181 Call is being forwarded |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.041 | 181 Call is being forwarded |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.188 | 180 Ringing |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.188 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.189 | ACK | |SIP Request
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |16.302 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |16.357 | ACK | |SIP Request
> > > >>>
> > > >>> | |(7640) <------------------ (5060) |
> > > >>>
> > > >>> |16.651 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.652 | ACK | |SIP Request
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |17.075 | ACK | |SIP Request
> > > >>>
> > > >>> | |(7640) <------------------ (5060) |
> > > >>>
> > > >>> |36.730 | BYE | |SIP Request
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |36.731 | BYE | |SIP Request
> > > >>>
> > > >>> | |(7640) <------------------ (5060) |
> > > >>>
> > > >>> |36.731 | 200 OK | |SIP Status
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |36.731 | 200 OK | |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> This issue happens during stress test.
> > > >>>
> > > >>> Any ideas, please? The OpenSIPS 1.6.2 is compiled with system
> > > malloc
> > > >>> and runs over VMware.
> > > >>>
> > > >>> --
> > > >>>
> > > >>> Best Regards,
> > > >>>
> > > >>> Alex Massover
> > > >>>
> > > >>>
> > > >>>
> > > >>> This mail was sent via Mail-SeCure System.
> > > >>> ---------------------------------------------------------------
> --
> > --
> > > --
> > > >>>
> > > >> ---
> > > >>
> > > >>> _______________________________________________
> > > >>> Users mailing list
> > > >>> [hidden email]
> > > >>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > > >>>
> > > >>>
> > > >> This mail was received via Mail-SeCure System.
> > > >>
> > > >>
> > > >
> > > >
> > > > This mail was sent via Mail-SeCure System.
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > [hidden email]
> > > > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > > >
> > >
> > >
> > > --
> > > Bogdan-Andrei Iancu
> > > OpenSIPS Bootcamp
> > > 20 - 24 September 2010, Frankfurt, Germany
> > > www.voice-system.ro
> > >
> > >
> > > _______________________________________________
> > > Users mailing list
> > > [hidden email]
> > > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > >
> > > This mail was received via Mail-SeCure System.
> >
> > This mail was sent via Mail-SeCure System.
> > _______________________________________________
> > Users mailing list
> > [hidden email]
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
> > This mail was received via Mail-SeCure System.
>
> This mail was sent via Mail-SeCure System.
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
> This mail was received via Mail-SeCure System.

This mail was sent via Mail-SeCure System.
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Bogdan-Andrei Iancu
In reply to this post by Alex Massover
Alex Massover wrote:

> Hi Bogdan,
>
> Have you received the trace and the log I sent?
>
> Also one more observation. The problem is not reproducible with children=1 but is reproducible with children>=1.
>
> I noticed long time ago that children>=1 misbehaves:
>
> a) performance is worse instead of getting better
> b) it can hang
> c) with big number of children it hangs/deadlocks very easily
>
> I might be mistaken but it looks for me that something is wrong either with interprocessing or with locks or with both of them.
>
> A corrupted BYE is just a symptom of bigger problem.
>
> For example in this scenario 180 is processed within one child and 200 within another child, there's race condition/locks misbehave/something wrong with memory, and as a result something gets corrupted and wrong BYE is sent.
>  
Hi Alex,

could you sent me (off list) the sipp scripts and the the opensips cfg
you are using to reproduce this problem? I would like to try to
reproduce it.

Regards,
Bogdan

PS: haven't received the log you mentioned - may you can send it again
off-list.

--
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Bogdan-Andrei Iancu
In reply to this post by Alex Massover
Hi Alex,


Alex Massover wrote:
>> looking at signalling, we should see why that calls have the
>> callee_route_set (some RR done after your proxy)..
>>
>>    
> [Alex] UAS is also a SIPP script. But it's not the reason for OpenSIPS to send corrupted BYE anyway ;)
>  

I agree, just trying to narrow down the faulty cases.

Just send me the logs (only if in debug 4) to check some
debugs....Otherwise we need a different approach.

Thanks and regards,
Bogdan




--
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Bogdan-Andrei Iancu
In reply to this post by Alex Massover
Alex Massover wrote:
> Hi Bogdan,
>
> I was able to reproduce the corrupted BYE problem even with children=1. With single working child the problem happens only when OpenSIPS is under massive stress/at maximum performance level (with blocking auth 20 cps is enough to push OpenSIPS into a corner).
>
> With children>=1 the problem is sometimes reproducible even with single call.
>  
Hi Alex,

do you see any 200 OK reply retransmissions ?

Regards,
Bogdan

--
Bogdan-Andrei Iancu
OpenSIPS Bootcamp
20 - 24 September 2010, Frankfurt, Germany
www.voice-system.ro



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: dialog bye_on_timeout and other issues

Alex Massover
Hi Bogdan,

There're retransmission, somewhere near 200 OK/ACK. Hard to say what exactly happens, because when OpenSIPS reach it maximum performance level (because of blocking auth) everything goes wrong.

Also I noticed that OpenSIPS retransmits BYE (if BYE is generated by dialog).
There's a BYE to both directions and there's an 200 OK from both sides. But OpenSIPS can keep sending BYE. Looks like 200OK goes to another process and gets blocked while the original process keeps sending BYEs. Something like this.


I'll send off-line the trace, log and opensips.cfg and sipp scripts from my gmail. I noticed twice that you don't receive direct emails from my corporate account.

But opensips.cfg works with radius, it might need some adaptations/configuration. Probably blocking radius auth is one of the causes to the issue (not because it's radius, but because it's blocking).

> -----Original Message-----
> From: [hidden email] [mailto:users-
> [hidden email]] On Behalf Of Bogdan-Andrei Iancu
> Sent: יום ה 24 יוני 2010 00:13
> To: OpenSIPS users mailling list
> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
>
> Alex Massover wrote:
> > Hi Bogdan,
> >
> > I was able to reproduce the corrupted BYE problem even with
> children=1. With single working child the problem happens only when
> OpenSIPS is under massive stress/at maximum performance level (with
> blocking auth 20 cps is enough to push OpenSIPS into a corner).
> >
> > With children>=1 the problem is sometimes reproducible even with
> single call.
> >
> Hi Alex,
>
> do you see any 200 OK reply retransmissions ?
>
> Regards,
> Bogdan
>
> --
> Bogdan-Andrei Iancu
> OpenSIPS Bootcamp
> 20 - 24 September 2010, Frankfurt, Germany
> www.voice-system.ro
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
> This mail was received via Mail-SeCure System.
>


This mail was sent via Mail-SeCure System.



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users