VoIP Mailing List Archives
Mailing list archives for the VoIP community |
|
| View previous topic :: View next topic |
| Author |
Message |
jlamanna at gmail.com
Guest
|
 Posted: Fri Nov 21, 2014 10:54 am Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
 |
|
Hi,I have an Asterisk server that's been running now for around 2 days.
I've noticed that the resident memory seems to be very high for its current call load:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18321 asterisk 20 0 8050m 5.2g 6968 S 13 66.2 363:11.80 asterisk
$ asterisk -rx "core show channels"
24 active channels
12 active calls
25216 calls processed
This server has a bunch of IAXModems hooked up to it and is mainly used as a Fax gateway to hylafax. Is this normal? 5.2Gig of memory used after 2 days with only 12 currently active calls?
I am not using any realtime peers.
There are 100 registered SIP peers on this server as well.
Thanks.
-- James |
|
| Back to top |
|
 |
jlamanna at gmail.com
Guest
|
| Posted: Fri Nov 21, 2014 2:02 pm Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
Its up to 5.8G of resident memory with 28321 calls processed.The OOM killer is going to kill this soon at this rate (8GB RAM machine).
This seems like a pretty serious problem.
It looks like I'll need to restart asterisk every night....
On Fri, Nov 21, 2014 at 10:53 AM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: | Hi,I have an Asterisk server that's been running now for around 2 days.
I've noticed that the resident memory seems to be very high for its current call load:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18321 asterisk 20 0 8050m 5.2g 6968 S 13 66.2 363:11.80 asterisk
$ asterisk -rx "core show channels"
24 active channels
12 active calls
25216 calls processed
This server has a bunch of IAXModems hooked up to it and is mainly used as a Fax gateway to hylafax. Is this normal? 5.2Gig of memory used after 2 days with only 12 currently active calls?
I am not using any realtime peers.
There are 100 registered SIP peers on this server as well.
Thanks.
-- James
|
|
|
| Back to top |
|
|
fh at danovation.dk
Guest
|
| Posted: Sat Nov 22, 2014 1:30 pm Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
| Quote: |
Its up to 5.8G of resident memory with 28321 calls processed. The OOM killer is going to kill this soon at this rate (8GB RAM machine).
This seems like a pretty serious problem.
It looks like I'll need to restart asterisk every night....
|
Hi the number of cpu cores that you see with top times 512Mbyte is the level of ram that's needed
e.g. a hp-gen8 with 2 octo core cpu's and hyperthreading enabled will be ( 2 x 8 x 2 x 0,5 gb ) = 16 gb + a bit exstra.
So from start memory usage increases until it reaches 17.3 gb and then stabilizes. at that level.
You can disables hypertreading and cut your ram usage to half of that.
I can't see what hardware you are using but I think you need to check that the rule above fits your hardware.
b.r.
Freddi
| Quote: |
On Fri, Nov 21, 2014 at 10:53 AM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
Hi, I have an Asterisk server that's been running now for around 2 days.
I've noticed that the resident memory seems to be very high for its current call load:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18321 asterisk 20 0 8050m 5.2g 6968 S 13 66.2 363:11.80 asterisk
$ asterisk -rx "core show channels"
24 active channels
12 active calls
25216 calls processed
This server has a bunch of IAXModems hooked up to it and is mainly used as a Fax gateway to hylafax. Is this normal? 5.2Gig of memory used after 2 days with only 12 currently active calls?
I am not using any realtime peers.
There are 100 registered SIP peers on this server as well.
Thanks.
-- James |
|
|
| Back to top |
|
|
jlamanna at gmail.com
Guest
|
| Posted: Mon Nov 24, 2014 9:02 am Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
cat /proc/cpuinfo lists 4 cores.So even if that's not showing hyperthreading, maximum 8.
By your rule, that would be 8 cores * 0.5GB = 4GB memory.
I've seen resident memory be up over 6GB.
On Sat, Nov 22, 2014 at 1:29 PM, Freddi Hansen <fh@danovation.dk (fh@danovation.dk)> wrote:
| Quote: | | Quote: |
Its up to 5.8G of resident memory with 28321 calls processed. The OOM killer is going to kill this soon at this rate (8GB RAM machine).
This seems like a pretty serious problem.
It looks like I'll need to restart asterisk every night....
|
Hi the number of cpu cores that you see with top times 512Mbyte is the level of ram that's needed
e.g. a hp-gen8 with 2 octo core cpu's and hyperthreading enabled will be ( 2 x 8 x 2 x 0,5 gb ) = 16 gb + a bit exstra.
So from start memory usage increases until it reaches 17.3 gb and then stabilizes. at that level.
You can disables hypertreading and cut your ram usage to half of that.
I can't see what hardware you are using but I think you need to check that the rule above fits your hardware.
b.r.
Freddi
| Quote: |
On Fri, Nov 21, 2014 at 10:53 AM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
Hi, I have an Asterisk server that's been running now for around 2 days.
I've noticed that the resident memory seems to be very high for its current call load:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18321 asterisk 20 0 8050m 5.2g 6968 S 13 66.2 363:11.80 asterisk
$ asterisk -rx "core show channels"
24 active channels
12 active calls
25216 calls processed
This server has a bunch of IAXModems hooked up to it and is mainly used as a Fax gateway to hylafax. Is this normal? 5.2Gig of memory used after 2 days with only 12 currently active calls?
I am not using any realtime peers.
There are 100 registered SIP peers on this server as well.
Thanks.
-- James |
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
|
|
|
| Back to top |
|
|
jlamanna at gmail.com
Guest
|
| Posted: Mon Nov 24, 2014 3:13 pm Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
Also, how big does the cache in frame.c grow to?I've recompiled with MALLOC_DEBUG on that server:
asterisk -rx "memory show summary"
....
1780466242 bytes (1780181594 cache) in 2352909 allocations in file frame.c
...
Seems like a ridiculous cache.
On Mon, Nov 24, 2014 at 9:02 AM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: | cat /proc/cpuinfo lists 4 cores.So even if that's not showing hyperthreading, maximum 8.
By your rule, that would be 8 cores * 0.5GB = 4GB memory.
I've seen resident memory be up over 6GB.
On Sat, Nov 22, 2014 at 1:29 PM, Freddi Hansen <fh@danovation.dk (fh@danovation.dk)> wrote:
| Quote: | | Quote: |
Its up to 5.8G of resident memory with 28321 calls processed. The OOM killer is going to kill this soon at this rate (8GB RAM machine).
This seems like a pretty serious problem.
It looks like I'll need to restart asterisk every night....
|
Hi the number of cpu cores that you see with top times 512Mbyte is the level of ram that's needed
e.g. a hp-gen8 with 2 octo core cpu's and hyperthreading enabled will be ( 2 x 8 x 2 x 0,5 gb ) = 16 gb + a bit exstra.
So from start memory usage increases until it reaches 17.3 gb and then stabilizes. at that level.
You can disables hypertreading and cut your ram usage to half of that.
I can't see what hardware you are using but I think you need to check that the rule above fits your hardware.
b.r.
Freddi
| Quote: |
On Fri, Nov 21, 2014 at 10:53 AM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
Hi, I have an Asterisk server that's been running now for around 2 days.
I've noticed that the resident memory seems to be very high for its current call load:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18321 asterisk 20 0 8050m 5.2g 6968 S 13 66.2 363:11.80 asterisk
$ asterisk -rx "core show channels"
24 active channels
12 active calls
25216 calls processed
This server has a bunch of IAXModems hooked up to it and is mainly used as a Fax gateway to hylafax. Is this normal? 5.2Gig of memory used after 2 days with only 12 currently active calls?
I am not using any realtime peers.
There are 100 registered SIP peers on this server as well.
Thanks.
-- James |
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
|
|
|
|
| Back to top |
|
|
mjordan at digium.com
Guest
|
| Posted: Tue Nov 25, 2014 8:15 am Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
On Mon, Nov 24, 2014 at 2:12 PM, James Lamanna <jlamanna@gmail.com> wrote:
| Quote: | Also, how big does the cache in frame.c grow to?
I've recompiled with MALLOC_DEBUG on that server:
asterisk -rx "memory show summary"
....
1780466242 bytes (1780181594 cache) in 2352909 allocations in file
frame.c
...
Seems like a ridiculous cache.
|
I'm not going to respond to your new thread, since it is the same
discussion as this one.
The frame cache is a per-thread local cache of frames that prevents
having to re-allocate frames as they pass through Asterisk. Clearly,
something is abusing it.
I think you'll need to provide some more information on how you're
producing this situation. Specifically:
* Channel technologies involved, and the formats on the channels
* Dialplan that reproduces the problem
Are you using any non-core dialplan applications or channel drivers?
--
Matthew Jordan
Digium, Inc. | Engineering Manager
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users |
|
| Back to top |
|
|
jlamanna at gmail.com
Guest
|
| Posted: Tue Nov 25, 2014 10:22 am Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
On Tue, Nov 25, 2014 at 8:14 AM, Matthew Jordan <mjordan@digium.com (mjordan@digium.com)> wrote:
| Quote: | On Mon, Nov 24, 2014 at 2:12 PM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: | Also, how big does the cache in frame.c grow to?
I've recompiled with MALLOC_DEBUG on that server:
asterisk -rx "memory show summary"
....
1780466242 bytes (1780181594 cache) in 2352909 allocations in file
frame.c
...
Seems like a ridiculous cache.
|
I'm not going to respond to your new thread, since it is the same
discussion as this one.
The frame cache is a per-thread local cache of frames that prevents
having to re-allocate frames as they pass through Asterisk. Clearly,
something is abusing it.
I think you'll need to provide some more information on how you're
producing this situation. Specifically:
* Channel technologies involved, and the formats on the channels
* Dialplan that reproduces the problem
Are you using any non-core dialplan applications or channel drivers?
|
This PBX has about 100 registered SIP clients, along with 23 PRI channels, 2 inbound/outbound SIP trunks and around 100 IAXModems registered to it. It primarily handles faxing.
I am not using any non-standard channel drivers. I am using the T.38 gateway funcionality.
The jist of the dialplan is this: (example of the PRI and a SIP trunk, inbound)
[pri-in]
exten => _X.,1,Set(__FROM_DID=${EXTEN})
exten => _X.,n,Set(FAX_IDX=700)
exten => _X.,n,Set(MAX_IDX=719)
exten => _X.,n,Goto(dial-hylafax,s,1)
[sip-trunk-in]
exten => _X.,1(normal),Set(__FROM_DID=${EXTEN})
exten => _X.,n,Set(FAX_IDX=950)
exten => _X.,n,Set(MAX_IDX=959)
exten => _X.,n,Set(FAXOPT(gateway)=yes)
exten => _X.,n,Goto(dial-hylafax,s,1)
[dial-hylafax]
exten => s,1,GotoIf($["${FROM_DID:0:1}" = "1"]?prune:cont)
exten => s,n(prune),Set(__FROM_DID=${FROM_DID:1})
exten => s,n(cont),GotoIf($[${FAX_IDX} <= ${MAX_IDX}]?tryfax:nofax)
exten => s,n(tryfax),Set(STATE=${DEVICE_STATE(Custom:iaxmodem${FAX_IDX})})
exten => s,n,NoOp(${STATE})
exten => s,n,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=INUSE)
exten => s,n,Dial(IAX2/iaxmodem${FAX_IDX}/${FROM_DID},60,g)
exten => s,n,Goto(s-${DIALSTATUS},1)
exten => s,n(nofax),Playtones(busy)
exten => s,n,NoOp(NO MODEMS AVAILABLE)
exten => s,n,Wait(20)
exten => s,n,Hangup()
exten => s-ANSWER,1,NoOp(IAXMODEM HANGUP)
exten => s-ANSWER,n,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=NOT_INUSE)
exten => s-ANSWER,n,Hangup()
exten => _s-.,1,Set(FAX_IDX=${MATH(1+${FAX_IDX},i)})
exten => _s-.,n,Goto(s,1)
exten => h,1,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=NOT_INUSE)
The current state requires me to restart Asterisk almost every day.
I'm also seeing this on a completely different machine after upgrading from Asterisk10 to 11.
-- James
|
|
| Back to top |
|
|
jlamanna at gmail.com
Guest
|
| Posted: Wed Nov 26, 2014 3:20 pm Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
On Tue, Nov 25, 2014 at 10:21 AM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: |
On Tue, Nov 25, 2014 at 8:14 AM, Matthew Jordan <mjordan@digium.com (mjordan@digium.com)> wrote:
| Quote: | On Mon, Nov 24, 2014 at 2:12 PM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: | Also, how big does the cache in frame.c grow to?
I've recompiled with MALLOC_DEBUG on that server:
asterisk -rx "memory show summary"
....
1780466242 bytes (1780181594 cache) in 2352909 allocations in file
frame.c
...
Seems like a ridiculous cache.
|
I'm not going to respond to your new thread, since it is the same
discussion as this one.
The frame cache is a per-thread local cache of frames that prevents
having to re-allocate frames as they pass through Asterisk. Clearly,
something is abusing it.
I think you'll need to provide some more information on how you're
producing this situation. Specifically:
* Channel technologies involved, and the formats on the channels
* Dialplan that reproduces the problem
Are you using any non-core dialplan applications or channel drivers?
|
This PBX has about 100 registered SIP clients, along with 23 PRI channels, 2 inbound/outbound SIP trunks and around 100 IAXModems registered to it. It primarily handles faxing.
I am not using any non-standard channel drivers. I am using the T.38 gateway funcionality.
The jist of the dialplan is this: (example of the PRI and a SIP trunk, inbound)
[pri-in]
exten => _X.,1,Set(__FROM_DID=${EXTEN})
exten => _X.,n,Set(FAX_IDX=700)
exten => _X.,n,Set(MAX_IDX=719)
exten => _X.,n,Goto(dial-hylafax,s,1)
[sip-trunk-in]
exten => _X.,1(normal),Set(__FROM_DID=${EXTEN})
exten => _X.,n,Set(FAX_IDX=950)
exten => _X.,n,Set(MAX_IDX=959)
exten => _X.,n,Set(FAXOPT(gateway)=yes)
exten => _X.,n,Goto(dial-hylafax,s,1)
[dial-hylafax]
exten => s,1,GotoIf($["${FROM_DID:0:1}" = "1"]?prune:cont)
exten => s,n(prune),Set(__FROM_DID=${FROM_DID:1})
exten => s,n(cont),GotoIf($[${FAX_IDX} <= ${MAX_IDX}]?tryfax:nofax)
exten => s,n(tryfax),Set(STATE=${DEVICE_STATE(Custom:iaxmodem${FAX_IDX})})
exten => s,n,NoOp(${STATE})
exten => s,n,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=INUSE)
exten => s,n,Dial(IAX2/iaxmodem${FAX_IDX}/${FROM_DID},60,g)
exten => s,n,Goto(s-${DIALSTATUS},1)
exten => s,n(nofax),Playtones(busy)
exten => s,n,NoOp(NO MODEMS AVAILABLE)
exten => s,n,Wait(20)
exten => s,n,Hangup()
exten => s-ANSWER,1,NoOp(IAXMODEM HANGUP)
exten => s-ANSWER,n,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=NOT_INUSE)
exten => s-ANSWER,n,Hangup()
exten => _s-.,1,Set(FAX_IDX=${MATH(1+${FAX_IDX},i)})
exten => _s-.,n,Goto(s,1)
exten => h,1,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=NOT_INUSE)
The current state requires me to restart Asterisk almost every day.
I'm also seeing this on a completely different machine after upgrading from Asterisk10 to 11.
|
I'm wondering if this is a problem in the SLIN converter?
I do use SLIN with iaxmodem.
-- James |
|
| Back to top |
|
|
jlamanna at gmail.com
Guest
|
| Posted: Fri Nov 28, 2014 6:09 pm Post subject: [asterisk-users] High resident memory with 11.14.0 ? |
|
|
On Wed, Nov 26, 2014 at 3:20 PM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: |
On Tue, Nov 25, 2014 at 10:21 AM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: |
On Tue, Nov 25, 2014 at 8:14 AM, Matthew Jordan <mjordan@digium.com (mjordan@digium.com)> wrote:
| Quote: | On Mon, Nov 24, 2014 at 2:12 PM, James Lamanna <jlamanna@gmail.com (jlamanna@gmail.com)> wrote:
| Quote: | Also, how big does the cache in frame.c grow to?
I've recompiled with MALLOC_DEBUG on that server:
asterisk -rx "memory show summary"
....
1780466242 bytes (1780181594 cache) in 2352909 allocations in file
frame.c
...
Seems like a ridiculous cache.
|
I'm not going to respond to your new thread, since it is the same
discussion as this one.
The frame cache is a per-thread local cache of frames that prevents
having to re-allocate frames as they pass through Asterisk. Clearly,
something is abusing it.
I think you'll need to provide some more information on how you're
producing this situation. Specifically:
* Channel technologies involved, and the formats on the channels
* Dialplan that reproduces the problem
Are you using any non-core dialplan applications or channel drivers?
|
This PBX has about 100 registered SIP clients, along with 23 PRI channels, 2 inbound/outbound SIP trunks and around 100 IAXModems registered to it. It primarily handles faxing.
I am not using any non-standard channel drivers. I am using the T.38 gateway funcionality.
The jist of the dialplan is this: (example of the PRI and a SIP trunk, inbound)
[pri-in]
exten => _X.,1,Set(__FROM_DID=${EXTEN})
exten => _X.,n,Set(FAX_IDX=700)
exten => _X.,n,Set(MAX_IDX=719)
exten => _X.,n,Goto(dial-hylafax,s,1)
[sip-trunk-in]
exten => _X.,1(normal),Set(__FROM_DID=${EXTEN})
exten => _X.,n,Set(FAX_IDX=950)
exten => _X.,n,Set(MAX_IDX=959)
exten => _X.,n,Set(FAXOPT(gateway)=yes)
exten => _X.,n,Goto(dial-hylafax,s,1)
[dial-hylafax]
exten => s,1,GotoIf($["${FROM_DID:0:1}" = "1"]?prune:cont)
exten => s,n(prune),Set(__FROM_DID=${FROM_DID:1})
exten => s,n(cont),GotoIf($[${FAX_IDX} <= ${MAX_IDX}]?tryfax:nofax)
exten => s,n(tryfax),Set(STATE=${DEVICE_STATE(Custom:iaxmodem${FAX_IDX})})
exten => s,n,NoOp(${STATE})
exten => s,n,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=INUSE)
exten => s,n,Dial(IAX2/iaxmodem${FAX_IDX}/${FROM_DID},60,g)
exten => s,n,Goto(s-${DIALSTATUS},1)
exten => s,n(nofax),Playtones(busy)
exten => s,n,NoOp(NO MODEMS AVAILABLE)
exten => s,n,Wait(20)
exten => s,n,Hangup()
exten => s-ANSWER,1,NoOp(IAXMODEM HANGUP)
exten => s-ANSWER,n,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=NOT_INUSE)
exten => s-ANSWER,n,Hangup()
exten => _s-.,1,Set(FAX_IDX=${MATH(1+${FAX_IDX},i)})
exten => _s-.,n,Goto(s,1)
exten => h,1,Set(DEVICE_STATE(Custom:iaxmodem${FAX_IDX})=NOT_INUSE)
The current state requires me to restart Asterisk almost every day.
I'm also seeing this on a completely different machine after upgrading from Asterisk10 to 11.
|
I'm wondering if this is a problem in the SLIN converter?
I do use SLIN with iaxmodem.
|
Also of note,
A quick valgrind run and attempting to send a few faxes produces a bunch of these in the valgrind output:
==30640== 217,259 bytes in 287 blocks are definitely lost in loss record 1,778 of 1,789
==30640== at 0x4C267CC: calloc (vg_replace_malloc.c:467)
==30640== by 0x4DC50E: ast_frdup (utils.h:523)
==30640== by 0x47125F: __ast_queue_frame (channel.c:1284)
==30640== by 0x1EF75589: __do_deliver (chan_iax2.c:3102)
==30640== by 0x1EF76C5A: schedule_delivery (chan_iax2.c:4374)
==30640== by 0x1EF8F497: socket_process_helper (chan_iax2.c:12010)
==30640== by 0x1EF99C37: iax2_process_thread (chan_iax2.c:12030)
==30640== by 0x56C458: dummy_start (utils.c:1192)
==30640== by 0x5E359C9: start_thread (pthread_create.c:300)
==30640== by 0x270326FF: ???
|
|
| Back to top |
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
Search
