Sponsor: VoiceMeUp - Corporate & Wholesale VoIP Services

VoIP Mailing List Archives
Mailing list archives for the VoIP community
 SearchSearch 

[Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel


 
Post new topic   Reply to topic    VoIP Mailing List Archives Forum Index -> freeSWITCH Users
View previous topic :: View next topic  
Author Message
jason at jasonjgw.net
Guest





PostPosted: Sun Apr 26, 2009 8:21 pm    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

After upgrading to the 2.6.29 kernel (the Debian packaged version), FreeSWITCH
takes up more CPU time than usual, e.g., 7% as reported by top, and the load
average is high (e.g., 0.87) even when the machine is idle and there are no
calls in progress. When top is run, FreeSWITCH appears at the start of the
list.

Is anybody else seeing this? The proportion of CPU time devoted to system
calls seems higher than it should be.

I would be interested in reports from anyone else who is running FreeSWITCH
under Linux 2.6.29.1 or equivalent from a distribution.


_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Back to top
pawel at voiceworks.pl
Guest





PostPosted: Mon Apr 27, 2009 7:00 am    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

boot Your kernel with "divider=10 nohz=off" options Smile

Recent kernels are tickless which basically causes all freeswitch timers/sleeps to fire at requested microsecond intervals.
With nohz kernels You get hundred times more system calls with freeswitch Sad

On 2009-04-27, at 03:00, Jason White wrote:
Quote:
After upgrading to the 2.6.29 kernel (the Debian packaged version), FreeSWITCH
takes up more CPU time than usual, e.g., 7% as reported by top, and the load
average is high (e.g., 0.87) even when the machine is idle and there are no
calls in progress. When top is run, FreeSWITCH appears at the start of the
list.

Is anybody else seeing this? The proportion of CPU time devoted to system
calls seems higher than it should be.

I would be interested in reports from anyone else who is running FreeSWITCH
under Linux 2.6.29.1 or equivalent from a distribution.


_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org

Back to top
jason at jasonjgw.net
Guest





PostPosted: Mon Apr 27, 2009 7:23 pm    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Paweł Pierścionek <pawel@voiceworks.pl> wrote:
Quote:
boot Your kernel with "divider=10 nohz=off" options :)

Recent kernels are tickless which basically causes all freeswitch
timers/sleeps to fire at requested microsecond intervals.
With nohz kernels You get hundred times more system calls with
freeswitch Sad

Is there anything FreeSWITCH could do about this, i.e., a configuration
parameter to adjust the timers appropriately?

I'm speaking from ignorance here; it just occurs to me that the application
should be able to control this behaviour without kernel options having to be
changed, but perhaps this is not the case.


_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Back to top
jason at jasonjgw.net
Guest





PostPosted: Tue Apr 28, 2009 1:28 am    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Just to add data to this:


PowerTOP 1.11 (C) 2007, 2008 Intel Corporation

Collecting data for 15 seconds


< Detailed C-state information is not available.>
P-states (frequencies)
2.34 Ghz 0.0%
2.00 Ghz 100.0%
Wakeups-from-idle per second : 405.4 interval: 15.0s
no ACPI power usage estimate available
Top causes for wakeups:
82.2% (1067.3) freeswitch : schedule_hrtimeout_range (hrtimer_wakeup)
[snip]


_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Back to top
daniel at rimspace.net
Guest





PostPosted: Thu Apr 30, 2009 8:04 am    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Paweł Pierścionek <pawel@voiceworks.pl>
writes:

G'day Paweł.

Quote:
boot Your kernel with "divider=10 nohz=off" options :)

Recent kernels are tickless which basically causes all freeswitch
timers/sleeps to fire at requested microsecond intervals. With nohz
kernels You get hundred times more system calls with freeswitch Sad

Like Jason, I am also interested to know why the tickless kernel causes
the timers to generate so much more load. I can't find anything
documented anywhere, really, about the issue — and this thread is the
only thing Google turns up on the topic.

I am looking to move my SIP system to FreeSwitch some time soon, if
I can, but I would love to know why nohz is so hostile to FreeSwitch
before I do, if possible.

Regards,
Daniel


_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Back to top
pawel at voiceworks.pl
Guest





PostPosted: Thu Apr 30, 2009 1:31 pm    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Hi,

With really old kernels (100Hz) if You do sleep(1ms) You sleep for
10ms on average.
With enterprise kernels (250Hz) Your sleep resolution increases by a
factor of 4.
With fresh kernels (1000Hz) You get real 1ms timer resolution -
10fold increase compared to old kernels.

With tickless You get whatever resolution You want - eg when You
sleep for 100 microseconds(micro not mili) then You get exactly what
You wish for.

Now for reasons I do no try to understand Smile there are a lot of
really short sleeps and fast timers in FreeSwitch - like 100
micro(1/10th of a ms).
So with CentOS such a 100 microsecond sleep cannot "fire" faster
then 250 times a second.
With tickless kernel same 100 microsecond sleep "fires" 10k times a
second.

Pawel,

_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Back to top
brian at freeswitch.org
Guest





PostPosted: Thu Apr 30, 2009 1:45 pm    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Then it would be recommended to not do tickless clock Razz

/b

On Apr 30, 2009, at 1:28 PM, Paweł Pierścionek wrote:
Quote:
Hi,

With really old kernels (100Hz) if You do sleep(1ms) You sleep for
10ms on average.
With enterprise kernels (250Hz) Your sleep resolution increases by a
factor of 4.
With fresh kernels (1000Hz) You get real 1ms timer resolution -
10fold increase compared to old kernels.

With tickless You get whatever resolution You want - eg when You
sleep for 100 microseconds(micro not mili) then You get exactly what
You wish for.

Now for reasons I do no try to understand Smile there are a lot of
really short sleeps and fast timers in FreeSwitch - like 100
micro(1/10th of a ms).
So with CentOS such a 100 microsecond sleep cannot "fire" faster
then 250 times a second.
With tickless kernel same 100 microsecond sleep "fires" 10k times a
second.

Pawel,

_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org



Brian West
brian@freeswitch.org (brian@freeswitch.org)



-- Meet us at ClueCon! http://www.cluecon.com
Back to top
mike at jerris.com
Guest





PostPosted: Thu Apr 30, 2009 1:48 pm    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Can you point out any place we do sub milli second sleeps? The timer
thread should be doing 1ms, I can't think of any that would be less.

MIke

On Apr 30, 2009, at 2:28 PM, Paweł Pierścionek wrote:

Quote:
Hi,

With really old kernels (100Hz) if You do sleep(1ms) You sleep for
10ms on average.
With enterprise kernels (250Hz) Your sleep resolution increases by a
factor of 4.
With fresh kernels (1000Hz) You get real 1ms timer resolution -
10fold increase compared to old kernels.

With tickless You get whatever resolution You want - eg when You
sleep for 100 microseconds(micro not mili) then You get exactly what
You wish for.

Now for reasons I do no try to understand Smile there are a lot of
really short sleeps and fast timers in FreeSwitch - like 100
micro(1/10th of a ms).
So with CentOS such a 100 microsecond sleep cannot "fire" faster
then 250 times a second.
With tickless kernel same 100 microsecond sleep "fires" 10k times a
second.

Pawel,

_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Back to top
pablosaro at gmail.com
Guest





PostPosted: Sat May 09, 2009 12:57 am    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Jason:

IMHO, it is related with the following kernel options: CONFIG_HZ, CONFIG_NO_HZ and CONFIG_HIGH_RES_TIMERS.
Take a look at those options in your kernel and try modifying them until get the desired result. Google that options and you will find lots of discussions that will clarify your mind. Here you will find a simple explanation: http://www.smk.co.za/2007/07/21/a-tickless-kernel/
My configuration is 100 Hz + tickless + High-Res timers and everything goes perfect. When system is idle and FS has no calls, I see my CPU at 0%.
Hope it helps.

Pablo

On Thu, Apr 30, 2009 at 3:48 PM, Michael Jerris <mike@jerris.com (mike@jerris.com)> wrote:
Quote:
Can you point out any place we do sub milli second sleeps?  The timer
thread should be doing 1ms, I can't think of any that would be less.

MIke

On Apr 30, 2009, at 2:28 PM, Pawe³ Pier¶cionek wrote:



Quote:
Hi,

 With really old kernels (100Hz) if You do sleep(1ms)  You sleep for
10ms on average.
 With enterprise kernels (250Hz) Your sleep resolution increases by a
factor of 4.
 With fresh kernels (1000Hz) You get real 1ms timer resolution -
10fold increase compared to old kernels.

 With tickless You get whatever resolution You want - eg when You
sleep for 100 microseconds(micro not mili) then You get exactly what
You wish for.

 Now for reasons I do no try to understand Smile there are a lot of
really short sleeps and fast timers in FreeSwitch - like 100
micro(1/10th of a ms).
 So with CentOS such a 100 microsecond sleep cannot "fire" faster
then 250 times a second.
 With tickless kernel same 100 microsecond sleep "fires" 10k times a
second.

Pawel,

_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org


Back to top
jason at jasonjgw.net
Guest





PostPosted: Sat May 09, 2009 1:15 am    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Pablo Hernan Saro <pablosaro@gmail.com> wrote:

Quote:
IMHO, it is related with the following kernel options: CONFIG_HZ,
CONFIG_NO_HZ and CONFIG_HIGH_RES_TIMERS.
Take a look at those options in your kernel and try modifying them until get
the desired result. Google that options and you will find lots of
discussions that will clarify your mind. Here you will find a simple
explanation: http://www.smk.co.za/2007/07/21/a-tickless-kernel/

Thank you for the references. I think I'll modify my kernel parameters in the
grub configuration, since I am using Debian kernels at the moment. (I do know
how to compile my own, which I will gladly do if it becomes necessary).



_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Back to top
kjv at ken-ton.com
Guest





PostPosted: Mon May 11, 2009 8:13 am    Post subject: [Freeswitch-users] FreeSWITCH under the Linux 2.6.29 kernel Reply with quote

Folks;

Bear in mind that the frequency is (X)Hz * (num cores), hence saying 100Hz on a dual core winds up being 200Hz.


My setup is 250Hz on a Dual-Core and the quality is perfect.


Oh, btw folks, don't attempt to do anything involving QOS (be it TBF, CBQ, HTB, or whatnot) on anything less than kernel 2.6.28.4


I don't know why exactly that is, but extensive testing here in the lab showed that this was entirely FUBAR until 2.6.25.7 where it got better, but not perfect until 2.6.28.4
(I may not be exact on the revisions, but close enough...)


There's also some options in the kernel that you must disable (not compile in) if you expect packet shaping to work.


If there's interest in this, e-mail me directly and I'll see if I can toss it into the FS Wiki some time this upcoming weekend.



Best Regards,
Karl J. Vesterling
kjv@ken-ton.com (kjv@ken-ton.com)
202-461-3231 x0






On May 9, 2009, at 2:14 AM, Jason White wrote:
Quote:
Pablo Hernan Saro <pablosaro@gmail.com (pablosaro@gmail.com)> wrote:

Quote:
IMHO, it is related with the following kernel options: CONFIG_HZ,
CONFIG_NO_HZ and CONFIG_HIGH_RES_TIMERS.
Take a look at those options in your kernel and try modifying them until get
the desired result. Google that options and you will find lots of
discussions that will clarify your mind. Here you will find a simple
explanation: http://www.smk.co.za/2007/07/21/a-tickless-kernel/

Thank you for the references. I think I'll modify my kernel parameters in the
grub configuration, since I am using Debian kernels at the moment. (I do know
how to compile my own, which I will gladly do if it becomes necessary).



_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org



Back to top
Display posts from previous:   
Post new topic   Reply to topic    VoIP Mailing List Archives Forum Index -> freeSWITCH Users All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group

VoiceMeUp - Corporate & Wholesale VoIP Services