Asynchronous processing in OpenSIPS 1.12

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Asynchronous processing in OpenSIPS 1.12

Răzvan Crainea-2
Hi all,

Among the last discussion of the last IRC meeting[1] was related to
Asynchronous processing in OpenSIPS script - we want to add a new
mechanism that allows you to perform asynchronous operations (such as DB
, REST or exec operations) directly from the script. Using this feature
will increase the throughput of OpenSIPS, since the SIP processes will
no longer be stuck waiting for I/O responses.

When reaching an asynchronous operation, the SIP message processing will
be stopped, and resumed in a different script route. In the script, this
should be reflected something like this:

route {
     ...
     # other non I/O operations
     async(my_resume_route) {
         avp_db_query("SELECT setid from subscribers where username =
'$fU'", "$avp(setid)");
     }
     # we never get here.
     exit;
}

route[my_resume_route] {
     xlog("The set used by the callee is $avp(setid)\n");
     # continue message processing
}

Only the functions that are used in an "async" block will be ran
asynchronously. If the function does not support async processing, it
will block waiting for the response and resume in the route indicated by
the "async" block - so the scripting experience will be the same even if
the async OP could not be done.
In order to resume the processing in a different route, we need to store
a context of the message. This will be done using the transaction
module[2]. If this module is not used, the asynchronous mechanism will
not work.

lirakis suggested to have different processes that waits for the
asynchronous I/O responses and continues the processing. However, we
think having the SIP worker processes resume the operations will behave
better, since we'll have fewer processes and we won't loose time with
context switches.

How do you see this feature? How would you like to use it? Feel free to
contribute with any input you find suitable!

[1] http://www.opensips.org/Community/IRCmeeting20140827
[2] http://www.opensips.org/html/docs/modules/1.12.x/tm

Best regards,

--
Răzvan Crainea
OpenSIPS Solutions
www.opensips-solutions.com


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Brett Nemeroff
I'm curious what the failure cases look like here..
What happens when your "long running db query" is excessively long? Is there a timeout? What happens if you have 1000s of backed up processes. Would love to see some sort of fifo command somehow quantify the effectiveness of it being async. Not sure how to say this right. But I'd like to be able to query a value that says either "bg tasks are being handled in a timely fashion" or "bg tasks are so slow it's starting to affect performance"

Of course, these are not "normal" cases... wondering how "graceful" we can allow it to be broken. I assume that the context saving will take X memory and eventually it will fill up? Where is that memory stored? How is it allocated? How much memory do you need for each backgrounded task?

Another thought..  how are dialog/transaction "variables" handled? AVPs? "vars"? How are dialogs handled across such an operation? What about standard (SIP) timer operations? What about dialog fr_timer and fr_inv_timer, do they continue to run?

Thanks, really looking forward to async!
-Brett



On Tue, Sep 2, 2014 at 11:26 AM, Răzvan Crainea <[hidden email]> wrote:
Hi all,

Among the last discussion of the last IRC meeting[1] was related to Asynchronous processing in OpenSIPS script - we want to add a new mechanism that allows you to perform asynchronous operations (such as DB , REST or exec operations) directly from the script. Using this feature will increase the throughput of OpenSIPS, since the SIP processes will no longer be stuck waiting for I/O responses.

When reaching an asynchronous operation, the SIP message processing will be stopped, and resumed in a different script route. In the script, this should be reflected something like this:

route {
    ...
    # other non I/O operations
    async(my_resume_route) {
        avp_db_query("SELECT setid from subscribers where username = '$fU'", "$avp(setid)");
    }
    # we never get here.
    exit;
}

route[my_resume_route] {
    xlog("The set used by the callee is $avp(setid)\n");
    # continue message processing
}

Only the functions that are used in an "async" block will be ran asynchronously. If the function does not support async processing, it will block waiting for the response and resume in the route indicated by the "async" block - so the scripting experience will be the same even if the async OP could not be done.
In order to resume the processing in a different route, we need to store a context of the message. This will be done using the transaction module[2]. If this module is not used, the asynchronous mechanism will not work.

lirakis suggested to have different processes that waits for the asynchronous I/O responses and continues the processing. However, we think having the SIP worker processes resume the operations will behave better, since we'll have fewer processes and we won't loose time with context switches.

How do you see this feature? How would you like to use it? Feel free to contribute with any input you find suitable!

[1] http://www.opensips.org/Community/IRCmeeting20140827
[2] http://www.opensips.org/html/docs/modules/1.12.x/tm

Best regards,

--
Răzvan Crainea
OpenSIPS Solutions
www.opensips-solutions.com


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Bogdan-Andrei Iancu-2
Brett,

The timeout for the async ops will be control by the reactor (like triggering the timouts) and set by the module starting the op - like the rest_client module will set the timeout for the HTTP rest queries.

The processes are not suspended, just the context of an execution. So at the end is just about memory - of course there should be a safety net to limit the number of suspended async I/Os per type of I/O. Each type of I/O op will have a priority in relation to other I/O ops (like timer jobs higher than aysnc resume, async resume higher than reading from network).

An async I/O is done during handling a SIP message, so it does not interfere with TM timers (which are activated only when sending the traffic out). otherwise the dialogs and transactions (with their eco-system) are not affected at all.

Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
On 02.09.2014 19:53, Brett Nemeroff wrote:
I'm curious what the failure cases look like here..
What happens when your "long running db query" is excessively long? Is there a timeout? What happens if you have 1000s of backed up processes. Would love to see some sort of fifo command somehow quantify the effectiveness of it being async. Not sure how to say this right. But I'd like to be able to query a value that says either "bg tasks are being handled in a timely fashion" or "bg tasks are so slow it's starting to affect performance"

Of course, these are not "normal" cases... wondering how "graceful" we can allow it to be broken. I assume that the context saving will take X memory and eventually it will fill up? Where is that memory stored? How is it allocated? How much memory do you need for each backgrounded task?

Another thought..  how are dialog/transaction "variables" handled? AVPs? "vars"? How are dialogs handled across such an operation? What about standard (SIP) timer operations? What about dialog fr_timer and fr_inv_timer, do they continue to run?

Thanks, really looking forward to async!
-Brett



On Tue, Sep 2, 2014 at 11:26 AM, Răzvan Crainea <[hidden email]> wrote:
Hi all,

Among the last discussion of the last IRC meeting[1] was related to Asynchronous processing in OpenSIPS script - we want to add a new mechanism that allows you to perform asynchronous operations (such as DB , REST or exec operations) directly from the script. Using this feature will increase the throughput of OpenSIPS, since the SIP processes will no longer be stuck waiting for I/O responses.

When reaching an asynchronous operation, the SIP message processing will be stopped, and resumed in a different script route. In the script, this should be reflected something like this:

route {
    ...
    # other non I/O operations
    async(my_resume_route) {
        avp_db_query("SELECT setid from subscribers where username = '$fU'", "$avp(setid)");
    }
    # we never get here.
    exit;
}

route[my_resume_route] {
    xlog("The set used by the callee is $avp(setid)\n");
    # continue message processing
}

Only the functions that are used in an "async" block will be ran asynchronously. If the function does not support async processing, it will block waiting for the response and resume in the route indicated by the "async" block - so the scripting experience will be the same even if the async OP could not be done.
In order to resume the processing in a different route, we need to store a context of the message. This will be done using the transaction module[2]. If this module is not used, the asynchronous mechanism will not work.

lirakis suggested to have different processes that waits for the asynchronous I/O responses and continues the processing. However, we think having the SIP worker processes resume the operations will behave better, since we'll have fewer processes and we won't loose time with context switches.

How do you see this feature? How would you like to use it? Feel free to contribute with any input you find suitable!

[1] http://www.opensips.org/Community/IRCmeeting20140827
[2] http://www.opensips.org/html/docs/modules/1.12.x/tm

Best regards,

--
Răzvan Crainea
OpenSIPS Solutions
www.opensips-solutions.com


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Saúl Ibarra Corretgé
In reply to this post by Răzvan Crainea-2

On 02 Sep 2014, at 18:26, Răzvan Crainea <[hidden email]> wrote:

> Hi all,
>
> Among the last discussion of the last IRC meeting[1] was related to Asynchronous processing in OpenSIPS script - we want to add a new mechanism that allows you to perform asynchronous operations (such as DB , REST or exec operations) directly from the script. Using this feature will increase the throughput of OpenSIPS, since the SIP processes will no longer be stuck waiting for I/O responses.
>
> When reaching an asynchronous operation, the SIP message processing will be stopped, and resumed in a different script route. In the script, this should be reflected something like this:
>
> route {
>    ...
>    # other non I/O operations
>    async(my_resume_route) {
>        avp_db_query("SELECT setid from subscribers where username = '$fU'", "$avp(setid)");
>    }
>    # we never get here.
>    exit;
> }
>
> route[my_resume_route] {
>    xlog("The set used by the callee is $avp(setid)\n");
>    # continue message processing
> }
>
> Only the functions that are used in an "async" block will be ran asynchronously. If the function does not support async processing, it will block waiting for the response and resume in the route indicated by the "async" block - so the scripting experience will be the same even if the async OP could not be done.
I find this quite problematic. How does John Doe know a function supports async or not? Maybe the configuration checker can know this and not let OpenSIPS start if you are doing it wrong? Alternatively, blocking functions could be ran in a thread pool thus giving the illusion of them being async.


Regards,

--
Saúl Ibarra Corretgé
AG Projects




_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Bogdan-Andrei Iancu-2
On 03.09.2014 12:06, Saúl Ibarra Corretgé wrote:

> On 02 Sep 2014, at 18:26, Răzvan Crainea <[hidden email]> wrote:
>
>> Hi all,
>>
>> Among the last discussion of the last IRC meeting[1] was related to Asynchronous processing in OpenSIPS script - we want to add a new mechanism that allows you to perform asynchronous operations (such as DB , REST or exec operations) directly from the script. Using this feature will increase the throughput of OpenSIPS, since the SIP processes will no longer be stuck waiting for I/O responses.
>>
>> When reaching an asynchronous operation, the SIP message processing will be stopped, and resumed in a different script route. In the script, this should be reflected something like this:
>>
>> route {
>>     ...
>>     # other non I/O operations
>>     async(my_resume_route) {
>>         avp_db_query("SELECT setid from subscribers where username = '$fU'", "$avp(setid)");
>>     }
>>     # we never get here.
>>     exit;
>> }
>>
>> route[my_resume_route] {
>>     xlog("The set used by the callee is $avp(setid)\n");
>>     # continue message processing
>> }
>>
>> Only the functions that are used in an "async" block will be ran asynchronously. If the function does not support async processing, it will block waiting for the response and resume in the route indicated by the "async" block - so the scripting experience will be the same even if the async OP could not be done.
> I find this quite problematic. How does John Doe know a function supports async or not? Maybe the configuration checker can know this and not let OpenSIPS start if you are doing it wrong? Alternatively, blocking functions could be ran in a thread pool thus giving the illusion of them being async.
First of all, the function will be documented as supporting Async and
the script parser may check it (we will export some extra flags for the
function to say "it can do async").

Secondly, the script will allow you to use any combination of functions
and script macro - I mean the script will allow you to use an async
function in a non-async way (and the function will act as sync) or to
use a non-async function in an async way (the function will act as async
and with a jump to resume route).

Regards,
Bogdan

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Saúl Ibarra Corretgé

On 03 Sep 2014, at 11:59, Bogdan-Andrei Iancu <[hidden email]> wrote:

> On 03.09.2014 12:06, Saúl Ibarra Corretgé wrote:
>> On 02 Sep 2014, at 18:26, Răzvan Crainea <[hidden email]> wrote:
>>
>>> Hi all,
>>>
>>> Among the last discussion of the last IRC meeting[1] was related to Asynchronous processing in OpenSIPS script - we want to add a new mechanism that allows you to perform asynchronous operations (such as DB , REST or exec operations) directly from the script. Using this feature will increase the throughput of OpenSIPS, since the SIP processes will no longer be stuck waiting for I/O responses.
>>>
>>> When reaching an asynchronous operation, the SIP message processing will be stopped, and resumed in a different script route. In the script, this should be reflected something like this:
>>>
>>> route {
>>>    ...
>>>    # other non I/O operations
>>>    async(my_resume_route) {
>>>        avp_db_query("SELECT setid from subscribers where username = '$fU'", "$avp(setid)");
>>>    }
>>>    # we never get here.
>>>    exit;
>>> }
>>>
>>> route[my_resume_route] {
>>>    xlog("The set used by the callee is $avp(setid)\n");
>>>    # continue message processing
>>> }
>>>
>>> Only the functions that are used in an "async" block will be ran asynchronously. If the function does not support async processing, it will block waiting for the response and resume in the route indicated by the "async" block - so the scripting experience will be the same even if the async OP could not be done.
>> I find this quite problematic. How does John Doe know a function supports async or not? Maybe the configuration checker can know this and not let OpenSIPS start if you are doing it wrong? Alternatively, blocking functions could be ran in a thread pool thus giving the illusion of them being async.
> First of all, the function will be documented as supporting Async and the script parser may check it (we will export some extra flags for the function to say "it can do async").
>
> Secondly, the script will allow you to use any combination of functions and script macro - I mean the script will allow you to use an async function in a non-async way (and the function will act as sync) or to use a non-async function in an async way (the function will act as async and with a jump to resume route).
>
My concern is with that last part: if a function which doesn’t support async is called within an async block it will block but then make the jump thus deceiving the user into believing it was async when it isn’t. We need to fail loudly and tell users they are doing it wrong, iMHO.


Regards,

--
Saúl Ibarra Corretgé
AG Projects




_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Bogdan-Andrei Iancu-2
On 03.09.2014 13:02, Saúl Ibarra Corretgé wrote:

> On 03 Sep 2014, at 11:59, Bogdan-Andrei Iancu <[hidden email]> wrote:
>
>> On 03.09.2014 12:06, Saúl Ibarra Corretgé wrote:
>>> On 02 Sep 2014, at 18:26, Răzvan Crainea <[hidden email]> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Among the last discussion of the last IRC meeting[1] was related to Asynchronous processing in OpenSIPS script - we want to add a new mechanism that allows you to perform asynchronous operations (such as DB , REST or exec operations) directly from the script. Using this feature will increase the throughput of OpenSIPS, since the SIP processes will no longer be stuck waiting for I/O responses.
>>>>
>>>> When reaching an asynchronous operation, the SIP message processing will be stopped, and resumed in a different script route. In the script, this should be reflected something like this:
>>>>
>>>> route {
>>>>     ...
>>>>     # other non I/O operations
>>>>     async(my_resume_route) {
>>>>         avp_db_query("SELECT setid from subscribers where username = '$fU'", "$avp(setid)");
>>>>     }
>>>>     # we never get here.
>>>>     exit;
>>>> }
>>>>
>>>> route[my_resume_route] {
>>>>     xlog("The set used by the callee is $avp(setid)\n");
>>>>     # continue message processing
>>>> }
>>>>
>>>> Only the functions that are used in an "async" block will be ran asynchronously. If the function does not support async processing, it will block waiting for the response and resume in the route indicated by the "async" block - so the scripting experience will be the same even if the async OP could not be done.
>>> I find this quite problematic. How does John Doe know a function supports async or not? Maybe the configuration checker can know this and not let OpenSIPS start if you are doing it wrong? Alternatively, blocking functions could be ran in a thread pool thus giving the illusion of them being async.
>> First of all, the function will be documented as supporting Async and the script parser may check it (we will export some extra flags for the function to say "it can do async").
>>
>> Secondly, the script will allow you to use any combination of functions and script macro - I mean the script will allow you to use an async function in a non-async way (and the function will act as sync) or to use a non-async function in an async way (the function will act as async and with a jump to resume route).
>>
> My concern is with that last part: if a function which doesn’t support async is called within an async block it will block but then make the jump thus deceiving the user into believing it was async when it isn’t. We need to fail loudly and tell users they are doing it wrong, iMHO.
Of course, we can detect that at script parsing and decide how to handle it:
     - ignore and do it blocking
     - warning and do it blocking
     - error and do not start.

Regards,
Bogdan

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Saúl Ibarra Corretgé
>>>
>> My concern is with that last part: if a function which doesn’t support async is called within an async block it will block but then make the jump thus deceiving the user into believing it was async when it isn’t. We need to fail loudly and tell users they are doing it wrong, iMHO.
> Of course, we can detect that at script parsing and decide how to handle it:
>    - ignore and do it blocking
>    - warning and do it blocking
>    - error and do not start.
>

I’d say we shouldn’t start :-)


Regards,

--
Saúl Ibarra Corretgé
AG Projects




_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous processing in OpenSIPS 1.12

Bogdan-Andrei Iancu-2
On 03.09.2014 14:43, Saúl Ibarra Corretgé wrote:
>>> My concern is with that last part: if a function which doesn’t support async is called within an async block it will block but then make the jump thus deceiving the user into believing it was async when it isn’t. We need to fail loudly and tell users they are doing it wrong, iMHO.
>> Of course, we can detect that at script parsing and decide how to handle it:
>>     - ignore and do it blocking
>>     - warning and do it blocking
>>     - error and do not start.
>>
> I’d say we shouldn’t start :-)
I will keep that in mind ;)

_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users