VoIP Mailing List Archives
Mailing list archives for the VoIP community |
|
View previous topic :: View next topic |
Author |
Message |
e at musinghalfwit.org Guest
|
Posted: Mon Nov 03, 2008 5:46 pm Post subject: [Freeswitch-users] Scaling Question |
|
|
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100 for
admin and 2 GiGe connections for private/public peering(SIP signaling
and media). I've been testing with some traffic and am finding I hit a
ceiling around 800-900 active calls. The system CPU ends up being about
40% which seemed really high to me. Is this simply due to the amount of
networking happening with the rtp in and out the different interfaces or am I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the interrupt
throttling but in the end the dynamic setting seemed to work best. Given
that a chassis has done 3k simultaneous calls with about half the
processing power I currently have in this server I must be running sub
optimal.
Any idea's of where I might look to correct this ? kernel tuning ?
network tuning ? Other than seeing a lot of irq's from the network cards
I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases housed in a
tmpfs partiion, and I've also disabled the sql scoreboard (-nosql). Any
other areas I should be exploring to increase the number of calls this
chassis can process ?
Thanks in advance for any ideas
-Eric
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
anthony.minessale at g... Guest
|
Posted: Mon Nov 03, 2008 6:25 pm Post subject: [Freeswitch-users] Scaling Question |
|
|
Did you increase the artificial limits in switch.conf.xml ?
try these uimits
Quote: | ulimit -c unlimitedulimit -d unlimitedulimit -f unlimitedulimit -i unlimitedulimit -n 999999ulimit -q unlimited
ulimit -u unlimitedulimit -v unlimitedulimit -x unlimitedulimit -s 244ulimit -l unlimited |
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Quote: | Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100 for
admin and 2 GiGe connections for private/public peering(SIP signaling
and media). I've been testing with some traffic and am finding I hit a
ceiling around 800-900 active calls. The system CPU ends up being about
40% which seemed really high to me. Is this simply due to the amount of
networking happening with the rtp in and out the different interfaces or am I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the interrupt
throttling but in the end the dynamic setting seemed to work best. Given
that a chassis has done 3k simultaneous calls with about half the
processing power I currently have in this server I must be running sub
optimal.
Any idea's of where I might look to correct this ? kernel tuning ?
network tuning ? Other than seeing a lot of irq's from the network cards
I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases housed in a
tmpfs partiion, and I've also disabled the sql scoreboard (-nosql). Any
other areas I should be exploring to increase the number of calls this
chassis can process ?
Thanks in advance for any ideas
-Eric
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
|
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400 |
|
Back to top |
|
|
e at musinghalfwit.org Guest
|
Posted: Mon Nov 03, 2008 6:54 pm Post subject: [Freeswitch-users] Scaling Question |
|
|
Yessir. I am using the exact same ulimits below. I also increased the
sessions per second to 50 and max-sessions to 5000. Also as we are
moving into a high traffic period based on time of day. I've had it die
repeatedly on me, the following line was the last line printed ...
freeswitch: nua_stack.c:2396: nua_client_restart_request: Assertion
`nua_client_is_queued(cr)' failed.
this is the other last line I get if the system itself seems to be
overloaded
freeswitch: sofia.c:2138: sofia_handle_sip_i_state: Assertion
`tech_pvt->nh != ((void *)0)' failed.
I don't know if that is of any help or not. At the time I was seeing
approx 80-90 calls per sec hitting the server, based on the 'status'
command in the FS cli.
I don't have this built with debug symbols either at the moment. I
wanted to get a baseline of server with x can do y , based on how we use
it. I will likely get it rebuilt and find a way to reduce some of the
traffic load so I can effectively tshoot.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 05:19:42PM -0600 , Anthony Minessale said:
Quote: | Did you increase the artificial limits in switch.conf.xml ?
try these uimits
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 244
ulimit -l unlimited
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org> wrote:
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100 for
admin and 2 GiGe connections for private/public peering(SIP signaling
and media). I've been testing with some traffic and am finding I hit a
ceiling around 800-900 active calls. The system CPU ends up being about
40% which seemed really high to me. Is this simply due to the amount of
networking happening with the rtp in and out the different interfaces or am
I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the interrupt
throttling but in the end the dynamic setting seemed to work best. Given
that a chassis has done 3k simultaneous calls with about half the
processing power I currently have in this server I must be running sub
optimal.
Any idea's of where I might look to correct this ? kernel tuning ?
network tuning ? Other than seeing a lot of irq's from the network cards
I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases housed in a
tmpfs partiion, and I've also disabled the sql scoreboard (-nosql). Any
other areas I should be exploring to increase the number of calls this
chassis can process ?
Thanks in advance for any ideas
-Eric
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
anthony.minessale at g... Guest
|
Posted: Mon Nov 03, 2008 9:57 pm Post subject: [Freeswitch-users] Scaling Question |
|
|
What revision are you on because the line numbers of those errors don't match up even slightly close?
Did you do something to disable building with debug symbols because the current default is to build with them?
What platform are you on?
Could you try a clean build of the latest revision?
On Mon, Nov 3, 2008 at 5:43 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Quote: | Yessir. I am using the exact same ulimits below. I also increased the
sessions per second to 50 and max-sessions to 5000. Also as we are
moving into a high traffic period based on time of day. I've had it die
repeatedly on me, the following line was the last line printed ...
freeswitch: nua_stack.c:2396: nua_client_restart_request: Assertion
`nua_client_is_queued(cr)' failed.
this is the other last line I get if the system itself seems to be
overloaded
freeswitch: sofia.c:2138: sofia_handle_sip_i_state: Assertion
`tech_pvt->nh != ((void *)0)' failed.
I don't know if that is of any help or not. At the time I was seeing
approx 80-90 calls per sec hitting the server, based on the 'status'
command in the FS cli.
I don't have this built with debug symbols either at the moment. I
wanted to get a baseline of server with x can do y , based on how we use
it. I will likely get it rebuilt and find a way to reduce some of the
traffic load so I can effectively tshoot.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 05:19:42PM -0600 , Anthony Minessale said:
Quote: | Did you increase the artificial limits in switch.conf.xml ?
try these uimits
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 244
ulimit -l unlimited
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100 for
admin and 2 GiGe connections for private/public peering(SIP signaling
and media). I've been testing with some traffic and am finding I hit a
ceiling around 800-900 active calls. The system CPU ends up being about
40% which seemed really high to me. Is this simply due to the amount of
networking happening with the rtp in and out the different interfaces or am
I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the interrupt
throttling but in the end the dynamic setting seemed to work best. Given
that a chassis has done 3k simultaneous calls with about half the
processing power I currently have in this server I must be running sub
optimal.
Any idea's of where I might look to correct this ? kernel tuning ?
network tuning ? Other than seeing a lot of irq's from the network cards
I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases housed in a
tmpfs partiion, and I've also disabled the sql scoreboard (-nosql). Any
other areas I should be exploring to increase the number of calls this
chassis can process ?
Thanks in advance for any ideas
-Eric
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
|
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400 |
|
Back to top |
|
|
e at musinghalfwit.org Guest
|
Posted: Mon Nov 03, 2008 10:32 pm Post subject: [Freeswitch-users] Scaling Question |
|
|
I'm running the 1.0.1-phoenix release.
No debug because our admin built the debian packages to install with so
dh_strip got em.
It's running on an ubuntu 8.04 server.
I certainly can. I will let you know how it goes.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 08:57:01PM -0600 , Anthony Minessale said:
Quote: | What revision are you on because the line numbers of those errors don't match
up even slightly close?
Did you do something to disable building with debug symbols because the current
default is to build with them?
What platform are you on?
Could you try a clean build of the latest revision?
On Mon, Nov 3, 2008 at 5:43 PM, Eric Liedtke <e@musinghalfwit.org> wrote:
Yessir. I am using the exact same ulimits below. I also increased the
sessions per second to 50 and max-sessions to 5000. Also as we are
moving into a high traffic period based on time of day. I've had it die
repeatedly on me, the following line was the last line printed ...
freeswitch: nua_stack.c:2396: nua_client_restart_request: Assertion
`nua_client_is_queued(cr)' failed.
this is the other last line I get if the system itself seems to be
overloaded
freeswitch: sofia.c:2138: sofia_handle_sip_i_state: Assertion
`tech_pvt->nh != ((void *)0)' failed.
I don't know if that is of any help or not. At the time I was seeing
approx 80-90 calls per sec hitting the server, based on the 'status'
command in the FS cli.
I don't have this built with debug symbols either at the moment. I
wanted to get a baseline of server with x can do y , based on how we use
it. I will likely get it rebuilt and find a way to reduce some of the
traffic load so I can effectively tshoot.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 05:19:42PM -0600 ,
Anthony Minessale said:
Quote: | Did you increase the artificial limits in switch.conf.xml ?
try these uimits
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 244
ulimit -l unlimited
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org> wrote:
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100 for
admin and 2 GiGe connections for private/public peering(SIP signaling
and media). I've been testing with some traffic and am finding I hit
| a
Quote: | ceiling around 800-900 active calls. The system CPU ends up being
| about
Quote: | 40% which seemed really high to me. Is this simply due to the amount
| of
Quote: | networking happening with the rtp in and out the different interfaces
| or am
Quote: | I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the
| interrupt
Quote: | throttling but in the end the dynamic setting seemed to work best.
| Given
Quote: | that a chassis has done 3k simultaneous calls with about half the
processing power I currently have in this server I must be running
| sub
Quote: | optimal.
Any idea's of where I might look to correct this ? kernel tuning ?
network tuning ? Other than seeing a lot of irq's from the network
| cards
Quote: | I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases housed
| in a
Quote: | tmpfs partiion, and I've also disabled the sql scoreboard (-nosql).
| Any
Quote: | other areas I should be exploring to increase the number of calls
| this
freeswitch-users
Quote: | http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
wasim at convergence.pk Guest
|
Posted: Tue Nov 04, 2008 12:00 am Post subject: [Freeswitch-users] Scaling Question |
|
|
Please check if your net interface is full duplex (ethtool should give you this info).
We had this problem once, and spent weeks trying to identify and turned out to be a half-duplex.
Also, see if you can offload checksum to the NIC if it'll do that.
-wasim
On Tue, Nov 4, 2008 at 8:20 AM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Quote: | I'm running the 1.0.1-phoenix release.
No debug because our admin built the debian packages to install with so
dh_strip got em.
It's running on an ubuntu 8.04 server.
I certainly can. I will let you know how it goes.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 08:57:01PM -0600 , Anthony Minessale said:
Quote: | What revision are you on because the line numbers of those errors don't match
up even slightly close?
Did you do something to disable building with debug symbols because the current
default is to build with them?
What platform are you on?
Could you try a clean build of the latest revision?
On Mon, Nov 3, 2008 at 5:43 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Yessir. I am using the exact same ulimits below. I also increased the
sessions per second to 50 and max-sessions to 5000. Also as we are
moving into a high traffic period based on time of day. I've had it die
repeatedly on me, the following line was the last line printed ...
freeswitch: nua_stack.c:2396: nua_client_restart_request: Assertion
`nua_client_is_queued(cr)' failed.
this is the other last line I get if the system itself seems to be
overloaded
freeswitch: sofia.c:2138: sofia_handle_sip_i_state: Assertion
`tech_pvt->nh != ((void *)0)' failed.
I don't know if that is of any help or not. At the time I was seeing
approx 80-90 calls per sec hitting the server, based on the 'status'
command in the FS cli.
I don't have this built with debug symbols either at the moment. I
wanted to get a baseline of server with x can do y , based on how we use
it. I will likely get it rebuilt and find a way to reduce some of the
traffic load so I can effectively tshoot.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 05:19:42PM -0600 ,
Anthony Minessale said:
Quote: | Did you increase the artificial limits in switch.conf.xml ?
try these uimits
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 244
ulimit -l unlimited
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100 for
admin and 2 GiGe connections for private/public peering(SIP signaling
and media). I've been testing with some traffic and am finding I hit
| a
Quote: | ceiling around 800-900 active calls. The system CPU ends up being
| about
Quote: | 40% which seemed really high to me. Is this simply due to the amount
| of
Quote: | networking happening with the rtp in and out the different interfaces
| or am
Quote: | I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the
| interrupt
Quote: | throttling but in the end the dynamic setting seemed to work best.
| Given
Quote: | that a chassis has done 3k simultaneous calls with about half the
processing power I currently have in this server I must be running
| sub
Quote: | optimal.
Any idea's of where I might look to correct this ? kernel tuning ?
network tuning ? Other than seeing a lot of irq's from the network
| cards
Quote: | I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases housed
| in a
Quote: | tmpfs partiion, and I've also disabled the sql scoreboard (-nosql).
| Any
Quote: | other areas I should be exploring to increase the number of calls
| this
freeswitch-users
Quote: | http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
|
--
wasim h. baig | principal consultant | convergence pk | +92 300 8508070 | as you scope creep, so shall we reap ... |
|
Back to top |
|
|
e at musinghalfwit.org Guest
|
Posted: Tue Nov 04, 2008 1:30 am Post subject: [Freeswitch-users] Scaling Question |
|
|
Yep Full-Duplex on both and these are what the offload parameters look
like
Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
udp fragmentation offload: off
generic segmentation offload: off
Since it's lot of little packets I left udp frag seg off because I
didn't think it would be of much use.
-e
It's seems fuzzy now but I think on Tue, Nov 04, 2008 at 09:59:54AM +0500 , Wasim Baig said:
Quote: | Please check if your net interface is full duplex (ethtool should give you this
info).
We had this problem once, and spent weeks trying to identify and turned out to
be a half-duplex.
Also, see if you can offload checksum to the NIC if it'll do that.
-wasim
On Tue, Nov 4, 2008 at 8:20 AM, Eric Liedtke <e@musinghalfwit.org> wrote:
I'm running the 1.0.1-phoenix release.
No debug because our admin built the debian packages to install with so
dh_strip got em.
It's running on an ubuntu 8.04 server.
I certainly can. I will let you know how it goes.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 08:57:01PM -0600 ,
Anthony Minessale said:
Quote: | What revision are you on because the line numbers of those errors don't
| match
Quote: | up even slightly close?
Did you do something to disable building with debug symbols because the
| current
Quote: | default is to build with them?
What platform are you on?
Could you try a clean build of the latest revision?
On Mon, Nov 3, 2008 at 5:43 PM, Eric Liedtke <e@musinghalfwit.org> wrote:
Yessir. I am using the exact same ulimits below. I also increased the
sessions per second to 50 and max-sessions to 5000. Also as we are
moving into a high traffic period based on time of day. I've had it
| die
Quote: | repeatedly on me, the following line was the last line printed ...
freeswitch: nua_stack.c:2396: nua_client_restart_request: Assertion
`nua_client_is_queued(cr)' failed.
this is the other last line I get if the system itself seems to be
overloaded
freeswitch: sofia.c:2138: sofia_handle_sip_i_state: Assertion
`tech_pvt->nh != ((void *)0)' failed.
I don't know if that is of any help or not. At the time I was seeing
approx 80-90 calls per sec hitting the server, based on the 'status'
command in the FS cli.
I don't have this built with debug symbols either at the moment. I
wanted to get a baseline of server with x can do y , based on how we
| use
Quote: | it. I will likely get it rebuilt and find a way to reduce some of the
traffic load so I can effectively tshoot.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 05:19:42PM
| -0600 ,
Quote: | Anthony Minessale said:
Quote: | Did you increase the artificial limits in switch.conf.xml ?
try these uimits
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 244
ulimit -l unlimited
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org>
|
| wrote:
Quote: | Quote: |
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100
|
| for
Quote: | Quote: | admin and 2 GiGe connections for private/public peering(SIP
|
| signaling
Quote: | Quote: | and media). I've been testing with some traffic and am finding
|
| I hit
Quote: | a
Quote: | ceiling around 800-900 active calls. The system CPU ends up
|
| being
Quote: | about
Quote: | 40% which seemed really high to me. Is this simply due to the
|
| amount
Quote: | of
Quote: | networking happening with the rtp in and out the different
|
| interfaces
Quote: | or am
Quote: | I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the
| interrupt
Quote: | throttling but in the end the dynamic setting seemed to work
|
| best.
Quote: | Given
Quote: | that a chassis has done 3k simultaneous calls with about half
|
| the
Quote: | Quote: | processing power I currently have in this server I must be
|
| running
Quote: | sub
Quote: | optimal.
Any idea's of where I might look to correct this ? kernel
|
| tuning ?
Quote: | Quote: | network tuning ? Other than seeing a lot of irq's from the
|
| network
Quote: | cards
Quote: | I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases
|
| housed
Quote: | in a
Quote: | tmpfs partiion, and I've also disabled the sql scoreboard
|
| (-nosql).
Quote: | Any
Quote: | other areas I should be exploring to increase the number of
|
| calls
Quote: | this
freeswitch-users
Quote: | http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
| freeswitch-users
freeswitch-users
Quote: | http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
wasim h. baig | principal consultant | convergence pk | +92 300 8508070 | as
you scope creep, so shall we reap ...
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
anthony.minessale at g... Guest
|
Posted: Tue Nov 04, 2008 9:51 am Post subject: [Freeswitch-users] Scaling Question |
|
|
Thank you.
Since we are in final beta stage for 1.0.2 I appreciate you testing it for us.
On Mon, Nov 3, 2008 at 9:20 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Quote: | I'm running the 1.0.1-phoenix release.
No debug because our admin built the debian packages to install with so
dh_strip got em.
It's running on an ubuntu 8.04 server.
I certainly can. I will let you know how it goes.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 08:57:01PM -0600 , Anthony Minessale said:
Quote: | What revision are you on because the line numbers of those errors don't match
up even slightly close?
Did you do something to disable building with debug symbols because the current
default is to build with them?
What platform are you on?
Could you try a clean build of the latest revision?
On Mon, Nov 3, 2008 at 5:43 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Yessir. I am using the exact same ulimits below. I also increased the
sessions per second to 50 and max-sessions to 5000. Also as we are
moving into a high traffic period based on time of day. I've had it die
repeatedly on me, the following line was the last line printed ...
freeswitch: nua_stack.c:2396: nua_client_restart_request: Assertion
`nua_client_is_queued(cr)' failed.
this is the other last line I get if the system itself seems to be
overloaded
freeswitch: sofia.c:2138: sofia_handle_sip_i_state: Assertion
`tech_pvt->nh != ((void *)0)' failed.
I don't know if that is of any help or not. At the time I was seeing
approx 80-90 calls per sec hitting the server, based on the 'status'
command in the FS cli.
I don't have this built with debug symbols either at the moment. I
wanted to get a baseline of server with x can do y , based on how we use
it. I will likely get it rebuilt and find a way to reduce some of the
traffic load so I can effectively tshoot.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 05:19:42PM -0600 ,
Anthony Minessale said:
Quote: | Did you increase the artificial limits in switch.conf.xml ?
try these uimits
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 244
ulimit -l unlimited
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org (e@musinghalfwit.org)> wrote:
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100 for
admin and 2 GiGe connections for private/public peering(SIP signaling
and media). I've been testing with some traffic and am finding I hit
| a
Quote: | ceiling around 800-900 active calls. The system CPU ends up being
| about
Quote: | 40% which seemed really high to me. Is this simply due to the amount
| of
Quote: | networking happening with the rtp in and out the different interfaces
| or am
Quote: | I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the
| interrupt
Quote: | throttling but in the end the dynamic setting seemed to work best.
| Given
Quote: | that a chassis has done 3k simultaneous calls with about half the
processing power I currently have in this server I must be running
| sub
Quote: | optimal.
Any idea's of where I might look to correct this ? kernel tuning ?
network tuning ? Other than seeing a lot of irq's from the network
| cards
Quote: | I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases housed
| in a
Quote: | tmpfs partiion, and I've also disabled the sql scoreboard (-nosql).
| Any
Quote: | other areas I should be exploring to increase the number of calls
| this
freeswitch-users
Quote: | http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org (Freeswitch-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
|
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400 |
|
Back to top |
|
|
e at musinghalfwit.org Guest
|
Posted: Tue Nov 04, 2008 11:59 am Post subject: [Freeswitch-users] Scaling Question |
|
|
Not a problem. So far this looks to be a vast improvement. I'm already
at around 800 calls, which yesteday had me down to 30% idle cpu, and
today am running around 75% idle. Also my system cpu is back down under
the user cpu. So some definite marked improvements. I will let you know
where we get with it today.
Thanks!!!
-e
It's seems fuzzy now but I think on Tue, Nov 04, 2008 at 08:50:32AM -0600 , Anthony Minessale said:
Quote: | Thank you.
Since we are in final beta stage for 1.0.2 I appreciate you testing it for us.
On Mon, Nov 3, 2008 at 9:20 PM, Eric Liedtke <e@musinghalfwit.org> wrote:
I'm running the 1.0.1-phoenix release.
No debug because our admin built the debian packages to install with so
dh_strip got em.
It's running on an ubuntu 8.04 server.
I certainly can. I will let you know how it goes.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 08:57:01PM -0600 ,
Anthony Minessale said:
Quote: | What revision are you on because the line numbers of those errors don't
| match
Quote: | up even slightly close?
Did you do something to disable building with debug symbols because the
| current
Quote: | default is to build with them?
What platform are you on?
Could you try a clean build of the latest revision?
On Mon, Nov 3, 2008 at 5:43 PM, Eric Liedtke <e@musinghalfwit.org> wrote:
Yessir. I am using the exact same ulimits below. I also increased the
sessions per second to 50 and max-sessions to 5000. Also as we are
moving into a high traffic period based on time of day. I've had it
| die
Quote: | repeatedly on me, the following line was the last line printed ...
freeswitch: nua_stack.c:2396: nua_client_restart_request: Assertion
`nua_client_is_queued(cr)' failed.
this is the other last line I get if the system itself seems to be
overloaded
freeswitch: sofia.c:2138: sofia_handle_sip_i_state: Assertion
`tech_pvt->nh != ((void *)0)' failed.
I don't know if that is of any help or not. At the time I was seeing
approx 80-90 calls per sec hitting the server, based on the 'status'
command in the FS cli.
I don't have this built with debug symbols either at the moment. I
wanted to get a baseline of server with x can do y , based on how we
| use
Quote: | it. I will likely get it rebuilt and find a way to reduce some of the
traffic load so I can effectively tshoot.
-e
It's seems fuzzy now but I think on Mon, Nov 03, 2008 at 05:19:42PM
| -0600 ,
Quote: | Anthony Minessale said:
Quote: | Did you increase the artificial limits in switch.conf.xml ?
try these uimits
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 244
ulimit -l unlimited
On Mon, Nov 3, 2008 at 4:46 PM, Eric Liedtke <e@musinghalfwit.org>
|
| wrote:
Quote: | Quote: |
Greetings,
I'm working on a server in an SBC setup. It has a copper 10/100
|
| for
Quote: | Quote: | admin and 2 GiGe connections for private/public peering(SIP
|
| signaling
Quote: | Quote: | and media). I've been testing with some traffic and am finding
|
| I hit
Quote: | a
Quote: | ceiling around 800-900 active calls. The system CPU ends up
|
| being
Quote: | about
Quote: | 40% which seemed really high to me. Is this simply due to the
|
| amount
Quote: | of
Quote: | networking happening with the rtp in and out the different
|
| interfaces
Quote: | or am
Quote: | I
operating something in a non optimized fasion ?
It's an e1000 based card, I've tried messing around with the
| interrupt
Quote: | throttling but in the end the dynamic setting seemed to work
|
| best.
Quote: | Given
Quote: | that a chassis has done 3k simultaneous calls with about half
|
| the
Quote: | Quote: | processing power I currently have in this server I must be
|
| running
Quote: | sub
Quote: | optimal.
Any idea's of where I might look to correct this ? kernel
|
| tuning ?
Quote: | Quote: | network tuning ? Other than seeing a lot of irq's from the
|
| network
Quote: | cards
Quote: | I"m not sure what else might be using up the system cpu.
To squeeze out as much as possible I have the core databases
|
| housed
Quote: | in a
Quote: | tmpfs partiion, and I've also disabled the sql scoreboard
|
| (-nosql).
Quote: | Any
Quote: | other areas I should be exploring to increase the number of
|
| calls
Quote: | this
freeswitch-users
Quote: | http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
| freeswitch-users
freeswitch-users
Quote: | http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org
pstn:213-799-1400
|
_______________________________________________
Freeswitch-users mailing list
Freeswitch-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|