VoIP Mailing List Archives
Mailing list archives for the VoIP community |
|
View previous topic :: View next topic |
Author |
Message |
fraunhofer.lists.frees... Guest
|
Posted: Thu Aug 27, 2009 8:27 am Post subject: [Freeswitch-users] memory leak |
|
|
Hello *,
a memory leak showed up in our loadtests. It's (still) the same setup as in the
http://jira.freeswitch.org/browse/MODSOFIA-22 bugfix.
One thing I'd like to add is that "fsctl shutdown restart" was unable
to shutdown freeswitch.
The last line printed is "switch_core_memory.c:567 Stopping memory pool queue."
attached file is a the collected and graphed output of some "ps waux"
command. "sz" should be
in "physical pages" (2k?).
I rerun the test and this time it coredumped trying to malloc() space
for some playback.
Anything else you need (full backtraces?) to dig into it?
Cheers
Beni.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
anthony.minessale at g... Guest
|
Posted: Thu Aug 27, 2009 9:46 am Post subject: [Freeswitch-users] memory leak |
|
|
600k is not a leak? FS can use as much as a gig of ram or more depending on what you are doing.
you may want to install a fresh copy of FS, removing all your old files etc and make sure they build clean.
We also have not had much luck running on ubuntu which is more of a desktop centric OS.
I recommend you try your application on 64 bit CentOS which is the platform all of our paid customers use.
On Thu, Aug 27, 2009 at 8:15 AM, Benedikt Fraunhofer <fraunhofer.lists.freeswitch-001@traced.net (fraunhofer.lists.freeswitch-001@traced.net)> wrote:
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400 |
|
Back to top |
|
|
fraunhofer.lists.frees... Guest
|
Posted: Thu Aug 27, 2009 11:30 am Post subject: [Freeswitch-users] memory leak |
|
|
Hello Anthony,
2009/8/27 Anthony Minessale <anthony.minessale@gmail.com>:
Quote: | 600k is not a leak? FS can use as much as a gig of ram or more depending on
what you are doing.
|
it's 600*1024 * 4* 1024 /1024/1024 => ~ 2.4 Gig
that's what i tried to express with the "physical pages" in my first post.
Quote: | you may want to install a fresh copy of FS, removing all your old files etc
and make sure they build clean.
|
i "make current" to get the SOFIA-22 bugfix.
Quote: | We also have not had much luck running on ubuntu which is more of a desktop
centric OS.
|
really? even the "server" edition? (not the netbook remix
Is 64bit so grave different if i don't need to address more than 4 Gig
of ram per process?
Quote: | I recommend you try your application on 64 bit CentOS which is the platform
all of our paid customers use.
|
hmm. our operations-people wont be happy with that.
But as i carefully followed the other mailing-list-threads it came to
my mind that i could be the culprit: somehow the original example.xml
came back into the directory tree and included a lot of things, most
notably the hash-inserts for last-dailed-numbers and stuff.
I stripped it down to what it was before and gave it another try.
thx again and (hopefully) sorry
Beni.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
jaybinks at gmail.com Guest
|
Posted: Thu Aug 27, 2009 5:02 pm Post subject: [Freeswitch-users] memory leak |
|
|
Anthony can you ( or anyone else alao ). Please elaborate on what
makes centos 5.3 o much better for Freeswitch.
Is there some specific library vesiion on centos that makes a massive
difference ?
Reason I ask ... I personally only have a preference for debian,
but others may have policy mandated Os's
For their companies, and it would be great to have some info about this.
I'd imagine this actually boils down to a requirement for libfoo
version X ... And if we ran those library versions on other OS's we'd
be fine ?
Jay
Quote: |
Quote: | I recommend you try your application on 64 bit CentOS which is the
platform
all of our paid customers use.
|
hmm. our operations-people wont be happy with that.
|
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
anthony.minessale at g... Guest
|
Posted: Thu Aug 27, 2009 5:56 pm Post subject: [Freeswitch-users] memory leak |
|
|
Mostly based on that is what we developed it with and continue to use and have had the best luck and least problems with.
I am sure you are right about it being certain version of the core toolchain
I will guess it's the combo of libc and kernel and the fact that CentOS has a purist attitude towards patching such code.
Some people use stable debian with no issues, other have reported problems resolved by switching to CentOS
so presumably you could roll your own debian that was similar but we don't have the resources to try to find out what makes what happen etc.
If you can configure debian to work, it's fine, if we get some inexplicable problem on debian we will start to wonder though.
it's happened more than once with people doing nothing other than changing OS =D
On Thu, Aug 27, 2009 at 4:55 PM, Jay Binks <jaybinks@gmail.com (jaybinks@gmail.com)> wrote:
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400 |
|
Back to top |
|
|
jason at jasonjgw.net Guest
|
Posted: Thu Aug 27, 2009 7:17 pm Post subject: [Freeswitch-users] memory leak |
|
|
Jay Binks <jaybinks@gmail.com> wrote:
Quote: | Reason I ask ... I personally only have a preference for debian,
but others may have policy mandated Os's
For their companies, and it would be great to have some info about this.
|
The only problem I've had with FreeSWITCH under Debian Squeeze and Sid
involves TLS-related segmentation faults that appear to be related to
something in the version of libssl supplied with Debian. The same problem
can't be reproduced on Fedora, for example, but it does occur under Debian
Lenny as well as Debian Squeeze and Sid (i.e., "testing" and "unstable",
respectively).
besides this, all of the issues that I have encountered turned out to be
(usually short-lived) bugs in FreeSWITCH or one of the libraries included in
the source tree - they're mostly FreeSWITCH issues.
I should point out that the FreeSWITCH developers are very good at avoiding
the introduction of bugs into their code and that known bugs get fixed. It
appears to be an unchanging fact about programming that with a large and
complex project, even given highly knowledgeable, experienced, committed and
talented developers (as we have with FreeSWITCH), sometimes, bugs do slip
through.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
diego.viola at gmail.com Guest
|
Posted: Fri Aug 28, 2009 6:07 am Post subject: [Freeswitch-users] memory leak |
|
|
Cool, feel free to report these bugs in Jira. If they are indeed FreeSWITCH bugs.
On Fri, Aug 28, 2009 at 12:10 AM, Jason White <jason@jasonjgw.net (jason@jasonjgw.net)> wrote:
Quote: | Jay Binks <jaybinks@gmail.com (jaybinks@gmail.com)> wrote:
Quote: | Reason I ask ... I personally only have a preference for debian,
but others may have policy mandated Os's
For their companies, and it would be great to have some info about this.
|
The only problem I've had with FreeSWITCH under Debian Squeeze and Sid
involves TLS-related segmentation faults that appear to be related to
something in the version of libssl supplied with Debian. The same problem
can't be reproduced on Fedora, for example, but it does occur under Debian
Lenny as well as Debian Squeeze and Sid (i.e., "testing" and "unstable",
respectively).
besides this, all of the issues that I have encountered turned out to be
(usually short-lived) bugs in FreeSWITCH or one of the libraries included in
the source tree - they're mostly FreeSWITCH issues.
I should point out that the FreeSWITCH developers are very good at avoiding
the introduction of bugs into their code and that known bugs get fixed. It
appears to be an unchanging fact about programming that with a large and
complex project, even given highly knowledgeable, experienced, committed and
talented developers (as we have with FreeSWITCH), sometimes, bugs do slip
through.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org (FreeSWITCH-users@lists.freeswitch.org)
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
|
|
|
Back to top |
|
|
fraunhofer.lists.frees... Guest
|
Posted: Mon Aug 31, 2009 7:28 am Post subject: [Freeswitch-users] memory leak |
|
|
Hello *,
still no luck. Removing the default dialplan with all those
hash-inserts made the memory-usage-curve flatter but it oom-coredumped
stating:
freeswitch: src/switch_core_memory.c:443: switch_core_perform_alloc:
Assertion `ptr != ((void *)0)' failed.
after eating up 3.2 gig ram+swap.
Any chance or idea to track that down or any hint where to look at?
The other two freeswitch-machines in this scenario play well, they're
dumb load-sinks. what makes this machine different is on one hand the
lua-scripts that are run on it and on the other hand it's the only
machine who gets it's commands via xml_rpc.
Cheers
Beni.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
brian at freeswitch.org Guest
|
Posted: Mon Aug 31, 2009 8:39 am Post subject: [Freeswitch-users] memory leak |
|
|
Use valgrind.
/b
On Aug 31, 2009, at 7:22 AM, Benedikt Fraunhofer wrote:
Quote: |
The other two freeswitch-machines in this scenario play well, they're
dumb load-sinks. what makes this machine different is on one hand the
lua-scripts that are run on it and on the other hand it's the only
machine who gets it's commands via xml_rpc.
|
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
rupa at rupa.com Guest
|
Posted: Mon Aug 31, 2009 9:16 am Post subject: [Freeswitch-users] memory leak |
|
|
Isn't there a known issue with lua+sql leaking memory on some platforms?
On Mon, Aug 31, 2009 at 8:32 AM, Brian West<brian@freeswitch.org> wrote:
Quote: | Use valgrind.
/b
On Aug 31, 2009, at 7:22 AM, Benedikt Fraunhofer wrote:
Quote: |
The other two freeswitch-machines in this scenario play well, they're
dumb load-sinks. what makes this machine different is on one hand the
lua-scripts that are run on it and on the other hand it's the only
machine who gets it's commands via xml_rpc.
|
|
Quote: | http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
|
--
-Rupa
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
fraunhofer.lists.frees... Guest
|
Posted: Wed Sep 02, 2009 8:07 am Post subject: [Freeswitch-users] memory leak |
|
|
Hello *,
2009/8/31 Rupa Schomaker <rupa@rupa.com>:
Quote: | Isn't there a known issue with lua+sql leaking memory on some platforms?
|
just lua, no sql in use
Quote: | On Mon, Aug 31, 2009 at 8:32 AM, Brian West<brian@freeswitch.org> wrote:
|
i tried that... in the beginning valgrind segfaultet several times
with some error message that it's unable to monitor so many threads.
vg_alloc_ThreadState: no free slots available
Increase VG_N_THREADS, rebuild and try again.
valgrind: the 'impossible' happened:
VG_N_THREADS is too low
it happeneded with 400 concurrent calls pretty soon, with 100 it took
a while, but at least it consumed half a gig of ram.
Can someone interpret the file it generated so far? i put it at
http://ns42.ath.cx/B0GdWh/vg.log.bz2
(5k compressed, 452k uncompressed) or is there just not enough
information contained?
I'll give it another shot with 50 concurrent calls but i guess this
will take ages...
Thx in advance
Beni.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
brian at freeswitch.org Guest
|
Posted: Wed Sep 02, 2009 9:13 am Post subject: [Freeswitch-users] memory leak |
|
|
What are you doing in these lua scripts? Because there are a few things you can do in the lua script itself that will cause you to leak like crazy due to improper use.
/b
On Sep 2, 2009, at 7:59 AM, Benedikt Fraunhofer wrote:
Quote: | Hello *,
2009/8/31 Rupa Schomaker <rupa@rupa.com (rupa@rupa.com)>:
Quote: | Isn't there a known issue with lua+sql leaking memory on some platforms?
|
just lua, no sql in use |
|
|
Back to top |
|
|
fraunhofer.lists.frees... Guest
|
Posted: Wed Sep 02, 2009 9:29 am Post subject: [Freeswitch-users] memory leak |
|
|
Hello Brian,
2009/9/2 Brian West <brian@freeswitch.org>:
Quote: | What are you doing in these lua scripts? Â Because there are a few things you
can do in the lua script itself that will cause you to leak like crazy due
to improper use.
/b
|
the setup is the same as in http://jira.freeswitch.org/browse/MODSOFIA-22
one is
-----
local reason = session:getVariable("originate_disposition");
session:setAutoHangup(false);
if(reason) then
if(reason == "NO_ANSWER") then
-- nothing
end
if(reason == "USER_BUSY") then
-- nothing
end
end
freeswitch.consoleLog(...
------
anotherone is
----
local sess = "nil";
if(argv[1]) then
sess=argv[1];
end
freeswitch.consoleLog(...
api = freeswitch.API();
local res = api:execute("sched_api" ...
freeswitch.consoleLog(...
----
and the scheduled script does
---
function log(msg)
freeswitch.consoleLog("notice", "c2c-hangup-timeout.lua: " .. msg .. "\n");
end
local sess = argv[1];
if(sess)
then
freeswitch.consoleLog("INFO", "hangup-timeout.lua for uuid " ..
sess .. "\n");
api = freeswitch.API();
local stillValid = api:execute("uuid_getvar", sess .. "
Dummy-DoesChannelExists");
if(stillValid:sub(1,4) == "-ERR")
then
log("session uuid " .. sess .. " disappeared (nothing bad)");
else
-- this is important!!! Otherwise the aleg get's just hung up!
api:execute("uuid_media", sess);
api:execute("uuid_transfer", sess .. " -both timeout");
end
else -- /if(sess)
log("called with nil session?");
end -- /if(sess)
---
i don't have to do some kind of "free()" on all local variables, do i?
Thx.
Beni.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org |
|
Back to top |
|
|
anthony.minessale at g... Guest
|
Posted: Wed Sep 02, 2009 10:20 am Post subject: [Freeswitch-users] memory leak |
|
|
You have to reduce the load when running valgrind.
On Wed, Sep 2, 2009 at 7:59 AM, Benedikt Fraunhofer <fraunhofer.lists.freeswitch-001@traced.net (fraunhofer.lists.freeswitch-001@traced.net)> wrote:
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400 |
|
Back to top |
|
|
anthony.minessale at g... Guest
|
Posted: Wed Sep 02, 2009 10:30 am Post subject: [Freeswitch-users] memory leak |
|
|
run it slower and make sure it shuts down clean.
valgrind --tool=memcheck --log-file-exactly=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
On Wed, Sep 2, 2009 at 10:11 AM, Anthony Minessale <anthony.minessale@gmail.com (anthony.minessale@gmail.com)> wrote:
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale@hotmail.com ([email]MSN%3Aanthony_minessale@hotmail.com[/email])
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com ([email]PAYPAL%3Aanthony.minessale@gmail.com[/email])
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org ([email]sip%3A888@conference.freeswitch.org[/email])
iax:guest@conference.freeswitch.org/888
googletalk:conf+888@conference.freeswitch.org ([email]googletalk%3Aconf%2B888@conference.freeswitch.org[/email])
pstn:213-799-1400 |
|
Back to top |
|
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|