Solution to storing a lot of siptrace

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Solution to storing a lot of siptrace

Daniel Zanutti
Hi

I have around 2000 simultaneous calls, 50 CPS and would like to store sip trace for all of them.

Storing on MySQL is not working. If you have some indexes on the table, after 1M register it starts to slow down the whole server. If no indexes, it's not searchable.

Do you guys have a good solution to store a lot of sip trace on Opensips? Like 200 GB.

It needs to be searchable almost real-time (a few minutes is fine).

I was thinking on store on a flat text file and move to a NOSQL solution, what do you guys think?

Thanks




_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Solution to storing a lot of siptrace

Jon Abrams
You might try using TokuDB table storage for MySql/MariaDB without compression. The penalty for indexing is much reduced over InnoDB.

You might also replicate the sip traces off your live proxy to another OpenSIPs capture node and let that node handle the inserts. Take a look at Web Homer as well.

On Fri, Dec 8, 2017 at 1:43 PM, Daniel Zanutti <[hidden email]> wrote:
Hi

I have around 2000 simultaneous calls, 50 CPS and would like to store sip trace for all of them.

Storing on MySQL is not working. If you have some indexes on the table, after 1M register it starts to slow down the whole server. If no indexes, it's not searchable.

Do you guys have a good solution to store a lot of sip trace on Opensips? Like 200 GB.

It needs to be searchable almost real-time (a few minutes is fine).

I was thinking on store on a flat text file and move to a NOSQL solution, what do you guys think?

Thanks




_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Solution to storing a lot of siptrace

Arsen
+1 for Homer. Nosql also seems a good idea. 



Arsen Semionov
cell: +442035198881

On Fri, Dec 8, 2017 at 10:21 PM, Jon Abrams <[hidden email]> wrote:
You might try using TokuDB table storage for MySql/MariaDB without compression. The penalty for indexing is much reduced over InnoDB.

You might also replicate the sip traces off your live proxy to another OpenSIPs capture node and let that node handle the inserts. Take a look at Web Homer as well.

On Fri, Dec 8, 2017 at 1:43 PM, Daniel Zanutti <[hidden email]> wrote:
Hi

I have around 2000 simultaneous calls, 50 CPS and would like to store sip trace for all of them.

Storing on MySQL is not working. If you have some indexes on the table, after 1M register it starts to slow down the whole server. If no indexes, it's not searchable.

Do you guys have a good solution to store a lot of sip trace on Opensips? Like 200 GB.

It needs to be searchable almost real-time (a few minutes is fine).

I was thinking on store on a flat text file and move to a NOSQL solution, what do you guys think?

Thanks




_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Solution to storing a lot of siptrace

Giovanni Maruzzelli
The guys at Homer/Sipcapture have built some impressive big things for carriers.

They sure can help you.


Sent via mobile, please forgive typos and brevity.

cell: +39 347 266 56 18
Giovanni Maruzzelli
OpenTelecom.IT

On Dec 8, 2017 9:47 PM, "Arsen" <[hidden email]> wrote:
+1 for Homer. Nosql also seems a good idea. 



Arsen Semionov
cell: <a href="tel:+44%2020%203519%208881" value="+442035198881" target="_blank">+442035198881

On Fri, Dec 8, 2017 at 10:21 PM, Jon Abrams <[hidden email]> wrote:
You might try using TokuDB table storage for MySql/MariaDB without compression. The penalty for indexing is much reduced over InnoDB.

You might also replicate the sip traces off your live proxy to another OpenSIPs capture node and let that node handle the inserts. Take a look at Web Homer as well.

On Fri, Dec 8, 2017 at 1:43 PM, Daniel Zanutti <[hidden email]> wrote:
Hi

I have around 2000 simultaneous calls, 50 CPS and would like to store sip trace for all of them.

Storing on MySQL is not working. If you have some indexes on the table, after 1M register it starts to slow down the whole server. If no indexes, it's not searchable.

Do you guys have a good solution to store a lot of sip trace on Opensips? Like 200 GB.

It needs to be searchable almost real-time (a few minutes is fine).

I was thinking on store on a flat text file and move to a NOSQL solution, what do you guys think?

Thanks




_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users


_______________________________________________
Users mailing list
[hidden email]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users