<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hello,<br>
<br>
the leak is due to keep defining counters or statistics. That should
happen only at startup, but if you execute from perl script some
function from a kamailio module, it may trigger this issue. Can you
share what functions from kamailio are you execute inside the perl
script?<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<div class="moz-cite-prefix">On 10/2/13 9:26 AM, David Cunningham
wrote:<br>
</div>
<blockquote
cite="mid:CAHGbP--1pvG_HQhEkCZwOnaJrhK9Yn+2W64KqsNV+hpQ6DPkyw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>Hi Daniel,<br>
<br>
</div>
Thank you - I used the restart method and got a whole load of
memory information. 10 megabytes of it! I've compressed that
and attached it to this email.<br>
<br>
</div>
Thanks for your help.<br>
<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On 2 October 2013 17:17,
Daniel-Constantin Mierla <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:miconda@gmail.com"
target="_blank">miconda@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
<br>
ahh, right, forgot that it was disabled because it was not
safe to use at runtime for worker processes. You have to
use:<br>
<br>
kamcmd cfg.set_now_int core mem_dump_pkg <pid><br>
<br>
then wait for a bit so that process is receiving a SIP
message.<br>
<br>
Or, if you restart, the stats are printed in syslog for
all processes.<br>
<br>
Another alternative is to call pkg_status() or
pkg_summary() from cfgutils module in config file, like:<br>
<br>
if(src_ip==127.0.0.1 && is_method("OPTIONS")
&& $rU=="pkgstats") {<br>
send_reply("200", "OK");<br>
pkg_summary();<br>
exit;<br>
}<br>
<br>
You can use sipsak to send an options request to <a
moz-do-not-send="true">"sip:pkgstats@..."</a><br>
<br>
Cheers,<br>
Daniel
<div>
<div class="h5"><br>
<br>
<div>On 10/2/13 2:30 AM, David Cunningham wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Daniel,<br>
<br>
When I send SIGUSR1 to a worker process nothing is
logged. Any ideas there? The following is the
output of "sercmd pkg.stats".<br>
<br>
{<br>
entry: 0<br>
pid: 14570<br>
rank: 0<br>
used: 191100<br>
free: 3731504<br>
real_used: 462800<br>
}<br>
{<br>
entry: 1<br>
pid: 14571<br>
rank: 1<br>
used: 202656<br>
free: 3714236<br>
real_used: 480068<br>
}<br>
{<br>
entry: 2<br>
pid: 14572<br>
rank: 2<br>
used: 202720<br>
free: 3714220<br>
real_used: 480084<br>
}<br>
{<br>
entry: 3<br>
pid: 14573<br>
rank: 3<br>
used: 202664<br>
free: 3714324<br>
real_used: 479980<br>
}<br>
{<br>
entry: 4<br>
pid: 14574<br>
rank: 4<br>
used: 202744<br>
free: 3714100<br>
real_used: 480204<br>
}<br>
{<br>
entry: 5<br>
pid: 14575<br>
rank: 5<br>
used: 201908<br>
free: 3715176<br>
real_used: 479128<br>
}<br>
{<br>
entry: 6<br>
pid: 14576<br>
rank: 6<br>
used: 202572<br>
free: 3714368<br>
real_used: 479936<br>
}<br>
{<br>
entry: 7<br>
pid: 14577<br>
rank: 7<br>
used: 202740<br>
free: 3714488<br>
real_used: 479816<br>
}<br>
{<br>
entry: 8<br>
pid: 14578<br>
rank: 8<br>
used: 199760<br>
free: 3717084<br>
real_used: 477220<br>
}<br>
{<br>
entry: 9<br>
pid: 14579<br>
rank: 9<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 10<br>
pid: 14580<br>
rank: 10<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 11<br>
pid: 14581<br>
rank: 11<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 12<br>
pid: 14582<br>
rank: 12<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 13<br>
pid: 14583<br>
rank: 13<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 14<br>
pid: 14584<br>
rank: 14<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 15<br>
pid: 14585<br>
rank: 15<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 16<br>
pid: 14586<br>
rank: 16<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 17<br>
pid: 14587<br>
rank: -1<br>
used: 199400<br>
free: 3722868<br>
real_used: 471436<br>
}<br>
{<br>
entry: 18<br>
pid: 14588<br>
rank: -1<br>
used: 2105844<br>
free: 3722868<br>
real_used: 2337944<br>
}<br>
{<br>
entry: 19<br>
pid: 14589<br>
rank: -2<br>
used: 198552<br>
free: 3723236<br>
real_used: 471068<br>
}<br>
{<br>
entry: 20<br>
pid: 0<br>
rank: 0<br>
used: 209756<br>
free: 3712368<br>
real_used: 481936<br>
}<br>
{<br>
entry: 21<br>
pid: 14591<br>
rank: -4<br>
used: 253660<br>
free: 3668704<br>
real_used: 525600<br>
}<br>
<br>
<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On 1 October 2013 22:45,
Daniel-Constantin Mierla <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:miconda@gmail.com"
target="_blank">miconda@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
<br>
send SIGUSR1 to one of the worker processes
(e.g., UDP listener). The main process of
kamailio doesn't process SIP messages, so it
is not exposed to leaks.<br>
<br>
You can used 'kamcmd pkg.stats" to see
statistics about usage of pkg per each
process (older versions have sercmd). <br>
<br>
'kamctl ps' gives the list of processes.<br>
<br>
Cheers,<br>
Daniel
<div>
<div><br>
<br>
<div>On 10/1/13 2:35 PM, David
Cunningham wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Hi Daniel,<br>
<br>
</div>
What I copied and pasted is all
that's listed when I do a kill
-SIGUSR1 on the Kamailio master
process. How do I get fuller logs?<br>
<br>
</div>
Thanks for your help.<br>
<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On 1
October 2013 21:57,
Daniel-Constantin Mierla <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:miconda@gmail.com"
target="_blank">miconda@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">Hello,<br>
<br>
you gave only the first part of
memory log output. The chunks
listed there are from startup,
thus not part of a leak, because
they are not allocated again at
runtime.<br>
<br>
Can you give the full logs with
messages from qm_status? The
leaks are typically listed
towards the end of those
messages.<br>
<br>
Yesterday someone else reported
a potential leak when using perl
module, so it might be something
else there. I found one, which
is related to an error case, but
perhaps you don't get that,
otherwise you will notice the
error message.<br>
<br>
Cheers,<br>
Daniel
<div>
<div><br>
<br>
On 10/1/13 12:03 PM, David
Cunningham wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
Hello,<br>
<br>
Following up on this after
a long break. It seems we
are still having a memory
leak. We've added the
memory debugging options
as per the documentation
with the following output.<br>
I'm not sure how to read
it though - can anyone
help? Thank you.<br>
<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: <core>
[main.c:754]: Memory
status (pkg):<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
(0x4019b008):<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: heap
size= 4194304<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: used=
191100,
used+overhead=462800,
free=3731504<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: max
used (+overhead)= 480224<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: dumping
all alloc'ed. fragments:<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 0.
N address=0x401cf414
frag=0x401cf3fc size=512
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 1.
N address=0x401cf644
frag=0x401cf62c size=128
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 2.
N address=0x401cf6f4
frag=0x401cf6dc size=256
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: counters.c:
cnt_hash_add(331)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 3.
N address=0x401cf824
frag=0x401cf80c size=64
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: counters.c:
init_counters(121)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 4.
N address=0x401cf894
frag=0x401cf87c size=8
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: route.c:
init_rlist(180)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 5.
N address=0x401cf8cc
frag=0x401cf8b4 size=64
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 6.
N address=0x401cf93c
frag=0x401cf924 size=24
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: route.c:
route_add(159)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 7.
N address=0x401cf984
frag=0x401cf96c size=8
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: route.c:
init_rlist(180)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 8.
N address=0x401cf9bc
frag=0x401cf9a4 size=64
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 9.
N address=0x401cfa2c
frag=0x401cfa14 size=24
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: route.c:
route_add(159)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 10.
N address=0x401cfa74
frag=0x401cfa5c size=8
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: db_id.c:
dupl_string(50)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 11.
N address=0x401cfaac
frag=0x401cfa94 size=64
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 12.
N address=0x401cfb1c
frag=0x401cfb04 size=24
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: route.c:
route_add(159)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 13.
N address=0x401cfb64
frag=0x401cfb4c size=8
used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
alloc'd from
<core>: route.c:
init_rlist(180)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status:
start check=f0f0f0f0, end
check= c0c0c0c0, abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]:
NOTICE: qm_status: 14.
N address=0x401cfb9c
frag=0x401cfb84 size=64
used=1<br>
<br>
<br>
</blockquote>
<br>
</div>
</div>
<div> -- <br>
Daniel-Constantin Mierla - <a
moz-do-not-send="true"
href="http://www.asipto.com"
target="_blank">http://www.asipto.com</a><br>
<a moz-do-not-send="true"
href="http://twitter.com/#%21/miconda"
target="_blank">http://twitter.com/#!/miconda</a>
- <a moz-do-not-send="true"
href="http://www.linkedin.com/in/miconda"
target="_blank">http://www.linkedin.com/in/miconda</a><br>
</div>
Kamailio Advanced Trainings -
Berlin, Nov 25-28; Miami, Nov
18-20, 2013<br>
- more details about Kamailio
trainings at <a
moz-do-not-send="true"
href="http://www.asipto.com"
target="_blank">http://www.asipto.com</a>
-<br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
David Cunningham, Voisonics<br>
<a moz-do-not-send="true"
href="http://voisonics.com/"
target="_blank">http://voisonics.com/</a><br>
USA: <a moz-do-not-send="true"
href="tel:%2B1%20213%20221%201092"
value="+12132211092"
target="_blank">+1 213 221 1092</a><br>
UK: <a moz-do-not-send="true"
href="tel:%2B44%20%280%29%2020%203298%201642"
value="+442032981642"
target="_blank">+44 (0) 20 3298
1642</a><br>
Australia: <a
moz-do-not-send="true"
href="tel:%2B61%20%280%29%202%208063%209019"
value="+61280639019"
target="_blank">+61 (0) 2 8063
9019</a><br>
</div>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
David Cunningham, Voisonics<br>
<a moz-do-not-send="true"
href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
USA: <a moz-do-not-send="true"
href="tel:%2B1%20213%20221%201092"
value="+12132211092" target="_blank">+1 213 221
1092</a><br>
UK: <a moz-do-not-send="true"
href="tel:%2B44%20%280%29%2020%203298%201642"
value="+442032981642" target="_blank">+44 (0) 20
3298 1642</a><br>
Australia: <a moz-do-not-send="true"
href="tel:%2B61%20%280%29%202%208063%209019"
value="+61280639019" target="_blank">+61 (0) 2
8063 9019</a><br>
</div>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
David Cunningham, Voisonics<br>
<a moz-do-not-send="true" href="http://voisonics.com/"
target="_blank">http://voisonics.com/</a><br>
USA: +1 213 221 1092<br>
UK: +44 (0) 20 3298 1642<br>
Australia: +61 (0) 2 8063 9019<br>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a> -
</pre>
</body>
</html>