Quantcast

Opensips 2.2.3 crash on async.

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Opensips 2.2.3 crash on async.

Qasim Ayyaz
Hi,

I have upgraded my script from 1.11 to 2.2.3 which works fine until i put async function on a single rest_get query. When the async line is executed i get following errors:

2017-04-13T14:03:45.320300+05:00 sip01 kernel: [31710929.448312] opensips[11475]: segfault at 10 ip 0000000000427260 sp 00007ffe8c29f6e8 error 6 in opensips[400000+228000]
2017-04-13T14:03:45.908172+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19570 ms), it may overlap..
2017-04-13T14:03:46.008338+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19670 ms), it may overlap..

I also have a core dump available, after opening in gdb i get the following info:

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `./sbin/opensips -P /var/run/opensips.pid'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000427260 in context_put_int (type=CONTEXT_GLOBAL, ctx=0x0, pos=4, data=1) at context.c:173
173             ((int *)ctx)[pos] = data;

Please note i will keep core dumps if you guys need some more info on it.

Regards,
Qasim

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips 2.2.3 crash on async.

Bogdan-Andrei Iancu-2
Hi Qasim,

Thank you for your report. Could you please run a "bt full" in gdb a post the output ?

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:09 PM, [hidden email] wrote:
Hi,

I have upgraded my script from 1.11 to 2.2.3 which works fine until i put async function on a single rest_get query. When the async line is executed i get following errors:

2017-04-13T14:03:45.320300+05:00 sip01 kernel: [31710929.448312] opensips[11475]: segfault at 10 ip 0000000000427260 sp 00007ffe8c29f6e8 error 6 in opensips[400000+228000]
2017-04-13T14:03:45.908172+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19570 ms), it may overlap..
2017-04-13T14:03:46.008338+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19670 ms), it may overlap..

I also have a core dump available, after opening in gdb i get the following info:

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `./sbin/opensips -P /var/run/opensips.pid'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000427260 in context_put_int (type=CONTEXT_GLOBAL, ctx=0x0, pos=4, data=1) at context.c:173
173             ((int *)ctx)[pos] = data;

Please note i will keep core dumps if you guys need some more info on it.

Regards,
Qasim


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips 2.2.3 crash on async.

Qasim Ayyaz
Hi Bogdan,

PFA required logs. Also fund below some more info on the setup i am using.

OS:
Distributor ID: SUSE LINUX
Description:    openSUSE Leap 42.1 (x86_64)
Release:        42.1

Opensips:
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 815a81abb
main.c compiled on 13:56:49 Apr 12 2017 with gcc 4.8

Please also note that i have tried compiling it from git but the problem persists.

Regards,
Qasim



On Thu, Apr 13, 2017 at 2:26 PM, Bogdan-Andrei Iancu <[hidden email]> wrote:
Hi Qasim,

Thank you for your report. Could you please run a "bt full" in gdb a post the output ?

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:09 PM, [hidden email] wrote:
Hi,

I have upgraded my script from 1.11 to 2.2.3 which works fine until i put async function on a single rest_get query. When the async line is executed i get following errors:

2017-04-13T14:03:45.320300+05:00 sip01 kernel: [31710929.448312] opensips[11475]: segfault at 10 ip 0000000000427260 sp 00007ffe8c29f6e8 error 6 in opensips[400000+228000]
2017-04-13T14:03:45.908172+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19570 ms), it may overlap..
2017-04-13T14:03:46.008338+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19670 ms), it may overlap..

I also have a core dump available, after opening in gdb i get the following info:

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `./sbin/opensips -P /var/run/opensips.pid'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000427260 in context_put_int (type=CONTEXT_GLOBAL, ctx=0x0, pos=4, data=1) at context.c:173
173             ((int *)ctx)[pos] = data;

Please note i will keep core dumps if you guys need some more info on it.

Regards,
Qasim


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

OpensipsBTFULL.txt (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips 2.2.3 crash on async.

Bogdan-Andrei Iancu-2
Hi Qasim,

This looks very very similar to an old report:
    http://lists.opensips.org/pipermail/users/2017-January/036231.html

And this was fixed in 2.2:
    https://github.com/OpenSIPS/opensips/commit/4023a5d797960d245b52eb73dc9fc26b8cdf2914

I guess you try to do some async stuff in the reply route, right ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:56 PM, [hidden email] wrote:
Hi Bogdan,

PFA required logs. Also fund below some more info on the setup i am using.

OS:
Distributor ID: SUSE LINUX
Description:    openSUSE Leap 42.1 (x86_64)
Release:        42.1

Opensips:
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 815a81abb
main.c compiled on 13:56:49 Apr 12 2017 with gcc 4.8

Please also note that i have tried compiling it from git but the problem persists.

Regards,
Qasim



On Thu, Apr 13, 2017 at 2:26 PM, Bogdan-Andrei Iancu <[hidden email]> wrote:
Hi Qasim,

Thank you for your report. Could you please run a "bt full" in gdb a post the output ?

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:09 PM, [hidden email] wrote:
Hi,
I have upgraded my script from 1.11 to 2.2.3 which works fine until i put async function on a single rest_get query. When the async line is executed i get following errors:
2017-04-13T14:03:45.320300+05:00 sip01 kernel: [31710929.448312] opensips[11475]: segfault at 10 ip 0000000000427260 sp 00007ffe8c29f6e8 error 6 in opensips[400000+228000] 2017-04-13T14:03:45.908172+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19570 ms), it may overlap.. 2017-04-13T14:03:46.008338+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19670 ms), it may overlap..
I also have a core dump available, after opening in gdb i get the following info:
[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./sbin/opensips -P /var/run/opensips.pid'. Program terminated with signal SIGSEGV, Segmentation fault. #0  0x0000000000427260 in context_put_int (type=CONTEXT_GLOBAL, ctx=0x0, pos=4, data=1) at context.c:173
173             ((int *)ctx)[pos] = data;
Please note i will keep core dumps if you guys need some more info on it.
Regards,
Qasim
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips 2.2.3 crash on async.

Qasim Ayyaz
Hi Bogdan,

Yes i have to do some accounting calls based on reply recieved. So i guess that means that async is not available in reply route? Is it still planned for future release or maybe in 2.3?

Regards,
Qasim

On Thu, Apr 13, 2017 at 3:15 PM, Bogdan-Andrei Iancu <[hidden email]> wrote:
Hi Qasim,

This looks very very similar to an old report:
    http://lists.opensips.org/pipermail/users/2017-January/036231.html

And this was fixed in 2.2:
    https://github.com/OpenSIPS/opensips/commit/4023a5d797960d245b52eb73dc9fc26b8cdf2914

I guess you try to do some async stuff in the reply route, right ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:56 PM, [hidden email] wrote:
Hi Bogdan,

PFA required logs. Also fund below some more info on the setup i am using.

OS:
Distributor ID: SUSE LINUX
Description:    openSUSE Leap 42.1 (x86_64)
Release:        42.1

Opensips:
version: opensips 2.2.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
git revision: 815a81abb
main.c compiled on 13:56:49 Apr 12 2017 with gcc 4.8

Please also note that i have tried compiling it from git but the problem persists.

Regards,
Qasim



On Thu, Apr 13, 2017 at 2:26 PM, Bogdan-Andrei Iancu <[hidden email][hidden email]> wrote:
Hi Qasim,

Thank you for your report. Could you please run a "bt full" in gdb a post the output ?

Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:09 PM, [hidden email] wrote:
Hi,
I have upgraded my script from 1.11 to 2.2.3 which works fine until i put async function on a single rest_get query. When the async line is executed i get following errors:
2017-04-13T14:03:45.320300+05:00 sip01 kernel: [31710929.448312] opensips[11475]: segfault at 10 ip 0000000000427260 sp 00007ffe8c29f6e8 error 6 in opensips[400000+228000] 2017-04-13T14:03:45.908172+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19570 ms), it may overlap.. 2017-04-13T14:03:46.008338+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19670 ms), it may overlap..
I also have a core dump available, after opening in gdb i get the following info:
[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./sbin/opensips -P /var/run/opensips.pid'. Program terminated with signal SIGSEGV, Segmentation fault. #0  0x0000000000427260 in context_put_int (type=CONTEXT_GLOBAL, ctx=0x0, pos=4, data=1) at context.c:173
173             ((int *)ctx)[pos] = data;
Please note i will keep core dumps if you guys need some more info on it.
Regards,
Qasim
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips 2.2.3 crash on async.

Bogdan-Andrei Iancu-2
The async(), as based on transactions, works only for requests, not for replies. This is limitation that will be changed in the next releases (but not in 2.3).

What you could do in 2.3 is to use launch() statement to run your async stuff.
    launch( blocking_function(...) );
The launch will start your function in background and continue your script.
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 02:00 PM, [hidden email] wrote:
Hi Bogdan,

Yes i have to do some accounting calls based on reply recieved. So i guess that means that async is not available in reply route? Is it still planned for future release or maybe in 2.3?

Regards,
Qasim

On Thu, Apr 13, 2017 at 3:15 PM, Bogdan-Andrei Iancu <[hidden email]> wrote:
Hi Qasim,

This looks very very similar to an old report:
    http://lists.opensips.org/pipermail/users/2017-January/036231.html

And this was fixed in 2.2:
    https://github.com/OpenSIPS/opensips/commit/4023a5d797960d245b52eb73dc9fc26b8cdf2914

I guess you try to do some async stuff in the reply route, right ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:56 PM, [hidden email] wrote:
Hi Bogdan,
PFA required logs. Also fund below some more info on the setup i am using. OS: Distributor ID: SUSE LINUX Description:    openSUSE Leap 42.1 (x86_64) Release:        42.1
Opensips:
version: opensips 2.2.3 (x86_64/linux) flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535 poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. git revision: 815a81abb main.c compiled on 13:56:49 Apr 12 2017 with gcc 4.8
Please also note that i have tried compiling it from git but the problem persists.
Regards,
Qasim
On Thu, Apr 13, 2017 at 2:26 PM, Bogdan-Andrei Iancu <[hidden email]> wrote:
Hi Qasim, Thank you for your report. Could you please run a "bt full" in gdb a post the output ? Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:09 PM, [hidden email] wrote:
Hi,
I have upgraded my script from 1.11 to 2.2.3 which works fine until i put async function on a single rest_get query. When the async line is executed i get following errors:
2017-04-13T14:03:45.320300+05:00 sip01 kernel: [31710929.448312] opensips[11475]: segfault at 10 ip 0000000000427260 sp 00007ffe8c29f6e8 error 6 in opensips[400000+228000] 2017-04-13T14:03:45.908172+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19570 ms), it may overlap.. 2017-04-13T14:03:46.008338+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19670 ms), it may overlap..
I also have a core dump available, after opening in gdb i get the following info:
[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./sbin/opensips -P /var/run/opensips.pid'. Program terminated with signal SIGSEGV, Segmentation fault. #0  0x0000000000427260 in context_put_int (type=CONTEXT_GLOBAL, ctx=0x0, pos=4, data=1) at context.c:173
173             ((int *)ctx)[pos] = data;
Please note i will keep core dumps if you guys need some more info on it.
Regards,
Qasim
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Opensips 2.2.3 crash on async.

Qasim Ayyaz
Thanks bogdan, ill see how I can work around it keeping launch() in mind.

-Qasim

On Apr 13, 2017 4:28 PM, "Bogdan-Andrei Iancu" <[hidden email]> wrote:
The async(), as based on transactions, works only for requests, not for replies. This is limitation that will be changed in the next releases (but not in 2.3).

What you could do in 2.3 is to use launch() statement to run your async stuff.
    launch( blocking_function(...) );
The launch will start your function in background and continue your script.
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 02:00 PM, [hidden email] wrote:
Hi Bogdan,

Yes i have to do some accounting calls based on reply recieved. So i guess that means that async is not available in reply route? Is it still planned for future release or maybe in 2.3?

Regards,
Qasim

On Thu, Apr 13, 2017 at 3:15 PM, Bogdan-Andrei Iancu <[hidden email]> wrote:
Hi Qasim,

This looks very very similar to an old report:
    http://lists.opensips.org/pipermail/users/2017-January/036231.html

And this was fixed in 2.2:
    https://github.com/OpenSIPS/opensips/commit/4023a5d797960d245b52eb73dc9fc26b8cdf2914

I guess you try to do some async stuff in the reply route, right ?

Regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:56 PM, [hidden email] wrote:
Hi Bogdan,
PFA required logs. Also fund below some more info on the setup i am using. OS: Distributor ID: SUSE LINUX Description:    openSUSE Leap 42.1 (x86_64) Release:        42.1
Opensips:
version: opensips 2.2.3 (x86_64/linux) flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535 poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. git revision: 815a81abb main.c compiled on 13:56:49 Apr 12 2017 with gcc 4.8
Please also note that i have tried compiling it from git but the problem persists.
Regards,
Qasim
On Thu, Apr 13, 2017 at 2:26 PM, Bogdan-Andrei Iancu <[hidden email]> wrote:
Hi Qasim, Thank you for your report. Could you please run a "bt full" in gdb a post the output ? Best regards,
Bogdan-Andrei Iancu
  OpenSIPS Founder and Developer
  http://www.opensips-solutions.com

OpenSIPS Summit May 2017 Amsterdam
  http://www.opensips.org/events/Summit-2017Amsterdam.html
On 04/13/2017 12:09 PM, [hidden email] wrote:
Hi,
I have upgraded my script from 1.11 to 2.2.3 which works fine until i put async function on a single rest_get query. When the async line is executed i get following errors:
2017-04-13T14:03:45.320300+05:00 sip01 kernel: [31710929.448312] opensips[11475]: segfault at 10 ip 0000000000427260 sp 00007ffe8c29f6e8 error 6 in opensips[400000+228000] 2017-04-13T14:03:45.908172+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19570 ms), it may overlap.. 2017-04-13T14:03:46.008338+05:00 sip01 ./sbin/opensips[10993]: WARNING:core:utimer_ticker: utimer task <tm-utimer> already scheduled for 19470 ms (now 19670 ms), it may overlap..
I also have a core dump available, after opening in gdb i get the following info:
[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./sbin/opensips -P /var/run/opensips.pid'. Program terminated with signal SIGSEGV, Segmentation fault. #0  0x0000000000427260 in context_put_int (type=CONTEXT_GLOBAL, ctx=0x0, pos=4, data=1) at context.c:173
173             ((int *)ctx)[pos] = data;
Please note i will keep core dumps if you guys need some more info on it.
Regards,
Qasim
_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Loading...