VoIP Mailing List Archives
Mailing list archives for the VoIP community |
|
View previous topic :: View next topic |
Author |
Message |
dragos at freeswitch.org Guest
|
Posted: Wed Oct 06, 2021 6:07 am Post subject: [Freeswitch-users] Jitter and dropping MOS score |
|
|
if it really consumes 100% CPU with few calls then it's abnormal. kill it with SIGABRT (make sure it will dump core before that), get a backtrace with gdb and open a github issue.
before doing this you need to make sure it's really eating 100 % CPU, this screenshot is not very relevant.
Also, you need to make sure you don't have real packet loss. Perhaps it's a matter of dimensioning, eg: how many simultaneous calls do you expect to have on that hardware, what codec, etc.
On Wed, Oct 6, 2021 at 2:26 AM Christoph Russow <russow@emtex.de (russow@emtex.de)> wrote:
Quote: | Hi all,
we are currently experiencing jitter and dropping MOS score on an
increasing amount of calls on our system.
i can't see any increased load on our system (neither CPU nor I/O wise).
Current stats:
~ 15Mbit/s network traffic (on a 1Gbit/s network)
load average: 1,10, 0,82, 0,91 (16 core machine)
memory used: 2,26GB/16GB
freeswitch@lc1> status
UP 0 years, 12 days, 13 hours, 30 minutes, 24 seconds, 649 milliseconds,
831 microseconds
FreeSWITCH (Version 1.10.1 -release-12-f9990221e6 64bit) is ready
137634 session(s) since startup
218 session(s) - peak 245, last 5min 222
1 session(s) per Sec out of max 30, peak 8, last 5min 2
1000 session(s) max
min idle cpu 0.00/95.37
Current Stack Size/Max 244K/244K
System ist not virtualized.
the only "strange" thing i see is the freeswitch "main"process that
consumes (according to htop) >=100% cpu (see attached screenshot).
Can this be the problem? if yes what can i do against it.
if not what are possible bottlenecks i should weed out?
if you need any additional informations to help me sort out this problem
please don't hesitate to ask.
Thanks in advance and best regards
Christoph
--
________________________________________________________________
EMTEX GmbH
Christoph Russow
Software Engineer
Bischof-Otto-Weg 9
D-91086 Aurachtal
Geschäftsführer: Markus Enzinger
Tel. +49 9132 7490 0 Sitz der Gesellschaft: 91086 Aurachtal
Fax. +49 9132 7490 900 Amtsgericht Fürth: HRB6804
_________________________________________________________________________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com (sales@freeswitch.com)
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org (FreeSWITCH-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
|
|
Back to top |
|
|
david.villasmil.work a... Guest
|
Posted: Wed Oct 06, 2021 6:14 am Post subject: [Freeswitch-users] Jitter and dropping MOS score |
|
|
First thing to look at is check your lua scripts if you have any. It is very strange for fs to take 100% of cpu on such low loads. So check your call flows.
On Wed, 6 Oct 2021 at 00:58, Christoph Russow <russow@emtex.de (russow@emtex.de)> wrote:
Quote: | Hi all,
we are currently experiencing jitter and dropping MOS score on an
increasing amount of calls on our system.
i can't see any increased load on our system (neither CPU nor I/O wise).
Current stats:
~ 15Mbit/s network traffic (on a 1Gbit/s network)
load average: 1,10, 0,82, 0,91 (16 core machine)
memory used: 2,26GB/16GB
freeswitch@lc1> status
UP 0 years, 12 days, 13 hours, 30 minutes, 24 seconds, 649 milliseconds,
831 microseconds
FreeSWITCH (Version 1.10.1 -release-12-f9990221e6 64bit) is ready
137634 session(s) since startup
218 session(s) - peak 245, last 5min 222
1 session(s) per Sec out of max 30, peak 8, last 5min 2
1000 session(s) max
min idle cpu 0.00/95.37
Current Stack Size/Max 244K/244K
System ist not virtualized.
the only "strange" thing i see is the freeswitch "main"process that
consumes (according to htop) >=100% cpu (see attached screenshot).
Can this be the problem? if yes what can i do against it.
if not what are possible bottlenecks i should weed out?
if you need any additional informations to help me sort out this problem
please don't hesitate to ask.
Thanks in advance and best regards
Christoph
--
________________________________________________________________
EMTEX GmbH
Christoph Russow
Software Engineer
Bischof-Otto-Weg 9
D-91086 Aurachtal
Geschäftsführer: Markus Enzinger
Tel. +49 9132 7490 0 Sitz der Gesellschaft: 91086 Aurachtal
Fax. +49 9132 7490 900 Amtsgericht Fürth: HRB6804
_________________________________________________________________________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com (sales@freeswitch.com)
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org (FreeSWITCH-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
--
Regards,
David Villasmilemail: david.villasmil.work@gmail.com (david.villasmil.work@gmail.com)
phone: +34669448337 |
|
Back to top |
|
|
john.hardiman at missi... Guest
|
Posted: Wed Oct 06, 2021 12:08 pm Post subject: [Freeswitch-users] Jitter and dropping MOS score |
|
|
Hi Christoph,
For the CPU part: On linux boxes if a process in top goes over 100% or top shows >1.0 in load, this means that more than one core is occupied. Since you have 16 cores this is fine as you are using 1/16 of your CPU for the FreeSWITCH process. So 200% would be 2 cores fully used etc, etc.
For the Jitter part/dropping MoS, we might need a bit more info, like times or patterns that is happens or even what the MoS drops by and how much jitter is seen. I personally use SNGrep and a combination of Homer/VoIPMonitor to get all of the stuff I need.
Accurate monitoring will point you in the right direction if there is an issue (Homer 7 is free and really good as a side note)
Thanks,
John.
-----Original Message-----
From: Christoph Russow <russow@emtex.de>
Sent: 05 October 2021 10:34
To: freeswitch-users@lists.freeswitch.org
Subject: [Freeswitch-users] Jitter and dropping MOS score
Hi all,
we are currently experiencing jitter and dropping MOS score on an increasing amount of calls on our system.
i can't see any increased load on our system (neither CPU nor I/O wise).
Current stats:
~ 15Mbit/s network traffic (on a 1Gbit/s network) load average: 1,10, 0,82, 0,91 (16 core machine) memory used: 2,26GB/16GB
freeswitch@lc1> status
UP 0 years, 12 days, 13 hours, 30 minutes, 24 seconds, 649 milliseconds,
831 microseconds
FreeSWITCH (Version 1.10.1 -release-12-f9990221e6 64bit) is ready
137634 session(s) since startup
218 session(s) - peak 245, last 5min 222
1 session(s) per Sec out of max 30, peak 8, last 5min 2
1000 session(s) max
min idle cpu 0.00/95.37
Current Stack Size/Max 244K/244K
System ist not virtualized.
the only "strange" thing i see is the freeswitch "main"process that consumes (according to htop) >=100% cpu (see attached screenshot).
Can this be the problem? if yes what can i do against it.
if not what are possible bottlenecks i should weed out?
if you need any additional informations to help me sort out this problem please don't hesitate to ask.
Thanks in advance and best regards
Christoph
--
________________________________________________________________
EMTEX GmbH
Christoph Russow
Software Engineer
Bischof-Otto-Weg 9
D-91086 Aurachtal
Geschäftsführer: Markus Enzinger
Tel. +49 9132 7490 0 Sitz der Gesellschaft: 91086 Aurachtal
Fax. +49 9132 7490 900 Amtsgericht Fürth: HRB6804
________________________________________________________________
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
|
Back to top |
|
|
Jerry.Kendall at Bisho... Guest
|
Posted: Wed Oct 06, 2021 12:09 pm Post subject: [Freeswitch-users] Jitter and dropping MOS score |
|
|
Christoph,
If you are running any lua scripts, you might want to look at those...
I had a huge load issue I went through in testing and found out my lua scripts were killing the CPU during "high" usage of 3-5 calls...
Search for my postings on "High CPU usage and Slow BLFs with only 8 channels" or... https://lists.freeswitch.org/pipermail/freeswitch-users/2021-September/135066.html
The issue I had was looking at the EVENTS in the lua script.... Every channel created 20-30 events
I changed the strategy for catching the EVENTS and the load has never been over 0.2. The MOS is now steady at 4+ and ZERO jitter....
While I never commented on the jitter in my postings, there were issues and I knew it was due to the high CPU usage. My issue, why the high load with only a few calls.
Hope it helps,
Jerry
On 10/5/2021 5:33 AM, Christoph Russow wrote:
Quote: | Hi all,
we are currently experiencing jitter and dropping MOS score on an increasing amount of calls on our system.
i can't see any increased load on our system (neither CPU nor I/O wise).
Current stats:
~ 15Mbit/s network traffic (on a 1Gbit/s network)
load average: 1,10, 0,82, 0,91 (16 core machine)
memory used: 2,26GB/16GB
freeswitch@lc1> status
UP 0 years, 12 days, 13 hours, 30 minutes, 24 seconds, 649 milliseconds, 831 microseconds
FreeSWITCH (Version 1.10.1 -release-12-f9990221e6 64bit) is ready
137634 session(s) since startup
218 session(s) - peak 245, last 5min 222
1 session(s) per Sec out of max 30, peak 8, last 5min 2
1000 session(s) max
min idle cpu 0.00/95.37
Current Stack Size/Max 244K/244K
System ist not virtualized.
the only "strange" thing i see is the freeswitch "main"process that consumes (according to htop) >=100% cpu (see attached screenshot).
Can this be the problem? if yes what can i do against it.
if not what are possible bottlenecks i should weed out?
if you need any additional informations to help me sort out this problem please don't hesitate to ask.
Thanks in advance and best regards
Christoph
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com
|
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
|
Back to top |
|
|
jurijs.ivolga at gmail... Guest
|
Posted: Wed Oct 06, 2021 12:18 pm Post subject: [Freeswitch-users] Jitter and dropping MOS score |
|
|
Hi,
I can not tell you about everybody, but usually when I get this problem 99% of the time the issue is on the provider side. I would try to reproduce this issue by myself and capture pcap files and analyze those.
With kind regards,
Jurijs
On Wed, Oct 6, 2021 at 7:43 PM John Hardiman <john.hardiman@missionlabs.co.uk (john.hardiman@missionlabs.co.uk)> wrote:
Quote: | Hi Christoph,
For the CPU part: On linux boxes if a process in top goes over 100% or top shows >1.0 in load, this means that more than one core is occupied. Since you have 16 cores this is fine as you are using 1/16 of your CPU for the FreeSWITCH process. So 200% would be 2 cores fully used etc, etc.
For the Jitter part/dropping MoS, we might need a bit more info, like times or patterns that is happens or even what the MoS drops by and how much jitter is seen. I personally use SNGrep and a combination of Homer/VoIPMonitor to get all of the stuff I need.
Accurate monitoring will point you in the right direction if there is an issue (Homer 7 is free and really good as a side note)
Thanks,
John.
-----Original Message-----
From: Christoph Russow <russow@emtex.de (russow@emtex.de)>
Sent: 05 October 2021 10:34
To: freeswitch-users@lists.freeswitch.org (freeswitch-users@lists.freeswitch.org)
Subject: [Freeswitch-users] Jitter and dropping MOS score
Hi all,
we are currently experiencing jitter and dropping MOS score on an increasing amount of calls on our system.
i can't see any increased load on our system (neither CPU nor I/O wise).
Current stats:
~ 15Mbit/s network traffic (on a 1Gbit/s network) load average: 1,10, 0,82, 0,91 (16 core machine) memory used: 2,26GB/16GB
freeswitch@lc1> status
UP 0 years, 12 days, 13 hours, 30 minutes, 24 seconds, 649 milliseconds,
831 microseconds
FreeSWITCH (Version 1.10.1 -release-12-f9990221e6 64bit) is ready
137634 session(s) since startup
218 session(s) - peak 245, last 5min 222
1 session(s) per Sec out of max 30, peak 8, last 5min 2
1000 session(s) max
min idle cpu 0.00/95.37
Current Stack Size/Max 244K/244K
System ist not virtualized.
the only "strange" thing i see is the freeswitch "main"process that consumes (according to htop) >=100% cpu (see attached screenshot).
Can this be the problem? if yes what can i do against it.
if not what are possible bottlenecks i should weed out?
if you need any additional informations to help me sort out this problem please don't hesitate to ask.
Thanks in advance and best regards
Christoph
--
________________________________________________________________
EMTEX GmbH
Christoph Russow
Software Engineer
Bischof-Otto-Weg 9
D-91086 Aurachtal
Geschäftsführer: Markus Enzinger
Tel. +49 9132 7490 0 Sitz der Gesellschaft: 91086 Aurachtal
Fax. +49 9132 7490 900 Amtsgericht Fürth: HRB6804
________________________________________________________________
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com (sales@freeswitch.com)
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org (FreeSWITCH-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
|
|
Back to top |
|
|
russow at emtex.de Guest
|
Posted: Thu Oct 07, 2021 4:45 am Post subject: [Freeswitch-users] Jitter and dropping MOS score |
|
|
Hi all,
thanks for all the input.
We could sort out the Jitter/MOS problem as a temporary Network problem
with one of our carriers.
about the CPU:
how should i make sure that it is really eating 100% CPU?
"ps aux" shows the freeswitch process at arround 30% so it seems to not
really consume 100%cpu as htop still tells me.
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
freeswi+ 30074 30.6 3.9 8582604 636456 ? SNsl Sep22 6431:58
/usr/bin/freeswitch -ncwait -u freeswitch -g emtex-files -nonat
best regards
Christoph
Am 06.10.21 um 12:33 schrieb Dragos Oancea:
Quote: | if it really consumes 100% CPU with few calls then it's abnormal.
kill it with SIGABRT (make sure it will dump core before that), get a
backtrace with gdb and open a github issue.
before doing this you need to make sure it's really eating 100 % CPU,
this screenshot is not very relevant.
Also, you need to make sure you don't have real packet loss. Perhaps
it's a matter of dimensioning, eg: how many simultaneous calls do you
expect to have on that hardware, what codec, etc.
On Wed, Oct 6, 2021 at 2:26 AM Christoph Russow <russow@emtex.de
<mailto:russow@emtex.de>> wrote:
Hi all,
we are currently experiencing jitter and dropping MOS score on an
increasing amount of calls on our system.
i can't see any increased load on our system (neither CPU nor I/O wise).
Current stats:
~ 15Mbit/s network traffic (on a 1Gbit/s network)
load average: 1,10, 0,82, 0,91 (16 core machine)
memory used: 2,26GB/16GB
freeswitch@lc1> status
UP 0 years, 12 days, 13 hours, 30 minutes, 24 seconds, 649
milliseconds,
831 microseconds
FreeSWITCH (Version 1.10.1 -release-12-f9990221e6 64bit) is ready
137634 session(s) since startup
218 session(s) - peak 245, last 5min 222
1 session(s) per Sec out of max 30, peak 8, last 5min 2
1000 session(s) max
min idle cpu 0.00/95.37
Current Stack Size/Max 244K/244K
System ist not virtualized.
the only "strange" thing i see is the freeswitch "main"process that
consumes (according to htop) >=100% cpu (see attached screenshot).
Can this be the problem? if yes what can i do against it.
if not what are possible bottlenecks i should weed out?
if you need any additional informations to help me sort out this
problem
please don't hesitate to ask.
Thanks in advance and best regards
Christoph
--
________________________________________________________________
EMTEX GmbH
Christoph Russow
Software Engineer
Bischof-Otto-Weg 9
D-91086 Aurachtal
Geschäftsführer: Markus Enzinger
Tel. +49 9132 7490 0 Sitz der Gesellschaft: 91086 Aurachtal
Fax. +49 9132 7490 900 Amtsgericht Fürth: HRB6804
_________________________________________________________________________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire
https://signalwire.com <https://signalwire.com>
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN
services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time
https://signalwire.community <https://signalwire.community>
Professional FreeSWITCH Services
sales@freeswitch.com <mailto:sales@freeswitch.com>
https://freeswitch.com <https://freeswitch.com>
Official FreeSWITCH Sites
https://freeswitch.com/oss <https://freeswitch.com/oss>
https://freeswitch.org/confluence <https://freeswitch.org/confluence>
https://cluecon.com <https://cluecon.com>
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
<mailto:FreeSWITCH-users@lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
<http://lists.freeswitch.org/mailman/listinfo/freeswitch-users>
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
<http://lists.freeswitch.org/mailman/options/freeswitch-users>
https://freeswitch.com <https://freeswitch.com>
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com
|
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services
sales@freeswitch.com
https://freeswitch.com
Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com |
|
Back to top |
|
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|