VoIP Mailing List Archives
Mailing list archives for the VoIP community |
|
View previous topic :: View next topic |
Author |
Message |
lists at telefaks.de Guest
|
Posted: Wed Mar 10, 2021 11:16 am Post subject: [Freeswitch-users] BLF under high load |
|
|
Hello
we have a Freeswitch installation for a multi tenant environment. We have extensively tested our Freeswitch for all call scenarios and all worked fine. This is the good news.
However, under higher load (>250 Channels and >6000 Subscriptions), presence does not seem to work reliably. I've found some ealier posts on this issue, which seem to lead to the following 2 solutions - use PostgreSQL (we use Mariadb via ODBC, also tested mod_mariadb, which made things worse)
- Move presence handling to Kamailio
As your environment is based on MySQL, we do not want to move to PostgreSQL without knowing, whether this will improve the situation.
So far we are catching events via ESL or even grepping the network and correct the status of presence informatons sent to the phone. However we discover, that even if we send corrected PRESENCE_IN via event socket, our Freeswitch keeps on sending the wrong BLF information afterwards to the phone. Means BLFs are blinking or in state red event if the watched phone is not in a call. So at present we are grepping all presence informations on the network, crosscheck this with the channels table and resend the corrected BLF status via event_socket if needed. But still this is not sufficiant.
So I am now asking the community: What is your approach to overcome this?
- Do newer Freeswitch versions perform better for this case?(Ours is "FreeSWITCH Version 1.10.5-release+git~20200818T185121Z")
- which work have you done to overcome this?
- does anybody have an idea how to query all internal presence status from Freewitch or maybe how to correct it?
Thank you in advance. Any help is appreciated.
Peter |
|
Back to top |
|
|
grcamauer at gmail.com Guest
|
Posted: Wed Mar 10, 2021 12:41 pm Post subject: [Freeswitch-users] BLF under high load |
|
|
From your description, it sounds like the DB is not keeping up with the updates FreeSwitch is making. Have you tried tuning the db? More memory, faster/more disks?
Guillermo
On Wed, Mar 10, 2021 at 1:16 PM Peter Steinbach <lists@telefaks.de (lists@telefaks.de)> wrote:
Quote: |
Hello
we have a Freeswitch installation for a multi tenant environment. We have extensively tested our Freeswitch for all call scenarios and all worked fine. This is the good news.
However, under higher load (>250 Channels and >6000 Subscriptions), presence does not seem to work reliably. I've found some ealier posts on this issue, which seem to lead to the following 2 solutions - use PostgreSQL (we use Mariadb via ODBC, also tested mod_mariadb, which made things worse)
- Move presence handling to Kamailio
As your environment is based on MySQL, we do not want to move to PostgreSQL without knowing, whether this will improve the situation.
So far we are catching events via ESL or even grepping the network and correct the status of presence informatons sent to the phone. However we discover, that even if we send corrected PRESENCE_IN via event socket, our Freeswitch keeps on sending the wrong BLF information afterwards to the phone. Means BLFs are blinking or in state red event if the watched phone is not in a call. So at present we are grepping all presence informations on the network, crosscheck this with the channels table and resend the corrected BLF status via event_socket if needed. But still this is not sufficiant.
So I am now asking the community: What is your approach to overcome this?
- Do newer Freeswitch versions perform better for this case?(Ours is "FreeSWITCH Version 1.10.5-release+git~20200818T185121Z")
- which work have you done to overcome this?
- does anybody have an idea how to query all internal presence status from Freewitch or maybe how to correct it?
Thank you in advance. Any help is appreciated.
Peter
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com (sales@freeswitch.com)
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org (FreeSWITCH-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
--
Guillermo Ruiz Camauer |
|
Back to top |
|
|
lists at telefaks.de Guest
|
Posted: Thu Mar 11, 2021 7:43 am Post subject: [Freeswitch-users] BLF under high load |
|
|
We made some investigations on our Production database. We set mysql slow log to 0.02 sec and we did not discover any long running qeuries concerning presence.
The command
select * from INFORMATION_SCHEMA.PROCESSLIST where command <> 'Sleep';
shows, that presence queries run between 0.1 and 1 milliseconds. So I do not really feel, that the Mysql database is the bottleneck here. MySQL show about 10-15%CPU on an 8 core machine. Freeswitch runs at about 50% CPU on a different 8 core machine.
Nevertheless we have severe problems with presence. I have e.g. a Snom phone and a Yealink phone on my desk with a BLF for a 3rd phone, which is not in a call at present. BLF on Snom is "on" and BLF on Yealink is "blinking". Both should be "off". This is very strange to me. How can this happen? BTW when this system was under no load, before we migrated our customers, everything worked fine.
So any help is very much appreciated here.
Another question is still:
- does anybody have an idea how to query all internal presence status from Freewitch or maybe how to correct it?
- e.g. "sofia_presence_data list number@domain" returns e.g.
status|rpid|user_agent|network_ip|network_port
unknown|unknown|N510 IP PRO/42.250.00.000.000|xx.xxx.xxx.xx|21549
unknown|unknown|Yealink SIP-T41S 66.83.0.35 805EC025FDA9|xx.xx.xxx.xxx|20901
Quote: | - and "sofia_presence_data status number@domain" always returns
|
unknownunknown
Quote: | - Database table "sip_subscriptions" however is fine
|
Best regards
Peter
Am 10.03.21 um 18:07 schrieb Guillermo Ruiz Camauer:
Quote: | From your description, it sounds like the DB is not keeping up with the updates FreeSwitch is making. Have you tried tuning the db? More memory, faster/more disks?
Guillermo
On Wed, Mar 10, 2021 at 1:16 PM Peter Steinbach <lists@telefaks.de (lists@telefaks.de)> wrote:
Quote: |
Hello
we have a Freeswitch installation for a multi tenant environment. We have extensively tested our Freeswitch for all call scenarios and all worked fine. This is the good news.
However, under higher load (>250 Channels and >6000 Subscriptions), presence does not seem to work reliably. I've found some ealier posts on this issue, which seem to lead to the following 2 solutions - use PostgreSQL (we use Mariadb via ODBC, also tested mod_mariadb, which made things worse)
- Move presence handling to Kamailio
As your environment is based on MySQL, we do not want to move to PostgreSQL without knowing, whether this will improve the situation.
So far we are catching events via ESL or even grepping the network and correct the status of presence informatons sent to the phone. However we discover, that even if we send corrected PRESENCE_IN via event socket, our Freeswitch keeps on sending the wrong BLF information afterwards to the phone. Means BLFs are blinking or in state red event if the watched phone is not in a call. So at present we are grepping all presence informations on the network, crosscheck this with the channels table and resend the corrected BLF status via event_socket if needed. But still this is not sufficiant.
So I am now asking the community: What is your approach to overcome this?
- Do newer Freeswitch versions perform better for this case?(Ours is "FreeSWITCH Version 1.10.5-release+git~20200818T185121Z")
- which work have you done to overcome this?
- does anybody have an idea how to query all internal presence status from Freewitch or maybe how to correct it?
Thank you in advance. Any help is appreciated.
Peter
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com (sales@freeswitch.com)
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org (FreeSWITCH-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
--
Guillermo Ruiz Camauer
|
|
|
Back to top |
|
|
lists at telefaks.de Guest
|
Posted: Thu Mar 11, 2021 8:47 am Post subject: [Freeswitch-users] BLF under high load |
|
|
While debugging presence, I found a lot of the following lines on the freeswitch console
2021-03-11 13:26:34.496493 [CRIT] sofia_presence.c:1377 CHECK SQL: 9886@customer.mydomain.net (9886@customer.mydomain.net) [select state,status,rpid,presence_id,uuid from sip_dialogs where call_info_state != 'seized' and hostname='fs02.mydomain.net' and profile_name='internal' and ((sip_from_user='9886' and sip_from_host='customer.mydomain.net') or presence_id='9886@customer.mydomain.net (9886@customer.mydomain.net)') order by rcd desc]
2021-03-11 13:26:34.496493 [ERR] sofia_presence.c:1435 PRES SQL update sip_subscriptions set version=version+1 where hostname='fs02.mydomain.net' and profile_name='internal' and sip_subscriptions.event != 'line-seize' and sip_subscriptions.proto='sip' and (event='presence' or event='dialog') and sub_to_user='9886' and (sub_to_host='customer.mydomain.net' or sub_to_host='xxx.xx.x.xxx' or sub_to_host='N/A' or presence_hosts like '%customer.mydomain.net%') and (sip_subscriptions.profile_name = 'internal' or presence_hosts like '%customer.mydomain.net%')
- The first query returns 0 entries, when I run it manually, line is marked as [CRIT]
- my impression is that this query is always empty because _all_ values of sip_dialogs.call_info_state are empty, allthough we have about 100 entries in sip_dialogs table. Running 100k continuous queries against this table, I could not find any other value than '' for sip_dialogs.call_info_state
- The second query updates 7 entries, when I run it manually, but line is marked as [ERR]
Maybe this info creates some new ideas.
Best regards
Peter
Am 11.03.21 um 13:08 schrieb Peter Steinbach:
Quote: |
We made some investigations on our Production database. We set mysql slow log to 0.02 sec and we did not discover any long running qeuries concerning presence.
The command
select * from INFORMATION_SCHEMA.PROCESSLIST where command <> 'Sleep';
shows, that presence queries run between 0.1 and 1 milliseconds. So I do not really feel, that the Mysql database is the bottleneck here. MySQL show about 10-15%CPU on an 8 core machine. Freeswitch runs at about 50% CPU on a different 8 core machine.
Nevertheless we have severe problems with presence. I have e.g. a Snom phone and a Yealink phone on my desk with a BLF for a 3rd phone, which is not in a call at present. BLF on Snom is "on" and BLF on Yealink is "blinking". Both should be "off". This is very strange to me. How can this happen? BTW when this system was under no load, before we migrated our customers, everything worked fine.
So any help is very much appreciated here.
Another question is still:
- does anybody have an idea how to query all internal presence status from Freewitch or maybe how to correct it?
- e.g. "sofia_presence_data list number@domain" returns e.g.
status|rpid|user_agent|network_ip|network_port
unknown|unknown|N510 IP PRO/42.250.00.000.000|xx.xxx.xxx.xx|21549
unknown|unknown|Yealink SIP-T41S 66.83.0.35 805EC025FDA9|xx.xx.xxx.xxx|20901
Quote: | - and "sofia_presence_data status number@domain" always returns
|
unknownunknown
Quote: | - Database table "sip_subscriptions" however is fine
|
Best regards
Peter
Am 10.03.21 um 18:07 schrieb Guillermo Ruiz Camauer:
Quote: | From your description, it sounds like the DB is not keeping up with the updates FreeSwitch is making. Have you tried tuning the db? More memory, faster/more disks?
Guillermo
On Wed, Mar 10, 2021 at 1:16 PM Peter Steinbach <lists@telefaks.de (lists@telefaks.de)> wrote:
Quote: |
Hello
we have a Freeswitch installation for a multi tenant environment. We have extensively tested our Freeswitch for all call scenarios and all worked fine. This is the good news.
However, under higher load (>250 Channels and >6000 Subscriptions), presence does not seem to work reliably. I've found some ealier posts on this issue, which seem to lead to the following 2 solutions - use PostgreSQL (we use Mariadb via ODBC, also tested mod_mariadb, which made things worse)
- Move presence handling to Kamailio
As your environment is based on MySQL, we do not want to move to PostgreSQL without knowing, whether this will improve the situation.
So far we are catching events via ESL or even grepping the network and correct the status of presence informatons sent to the phone. However we discover, that even if we send corrected PRESENCE_IN via event socket, our Freeswitch keeps on sending the wrong BLF information afterwards to the phone. Means BLFs are blinking or in state red event if the watched phone is not in a call. So at present we are grepping all presence informations on the network, crosscheck this with the channels table and resend the corrected BLF status via event_socket if needed. But still this is not sufficiant.
So I am now asking the community: What is your approach to overcome this?
- Do newer Freeswitch versions perform better for this case?(Ours is "FreeSWITCH Version 1.10.5-release+git~20200818T185121Z")
- which work have you done to overcome this?
- does anybody have an idea how to query all internal presence status from Freewitch or maybe how to correct it?
Thank you in advance. Any help is appreciated.
Peter
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com (sales@freeswitch.com)
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org (FreeSWITCH-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
--
Guillermo Ruiz Camauer
| |
|
|
Back to top |
|
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|