<div dir="ltr"><div><div>Hi Daniel,<br><br></div>Thank you - I used the restart method and got a whole load of memory information. 10 megabytes of it! I've compressed that and attached it to this email.<br><br></div>Thanks for your help.<br>
<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 2 October 2013 17:17, Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hello,<br>
<br>
ahh, right, forgot that it was disabled because it was not safe to
use at runtime for worker processes. You have to use:<br>
<br>
kamcmd cfg.set_now_int core mem_dump_pkg <pid><br>
<br>
then wait for a bit so that process is receiving a SIP message.<br>
<br>
Or, if you restart, the stats are printed in syslog for all
processes.<br>
<br>
Another alternative is to call pkg_status() or pkg_summary() from
cfgutils module in config file, like:<br>
<br>
if(src_ip==127.0.0.1 && is_method("OPTIONS") &&
$rU=="pkgstats") {<br>
send_reply("200", "OK");<br>
pkg_summary();<br>
exit;<br>
}<br>
<br>
You can use sipsak to send an options request to <a>"sip:pkgstats@..."</a><br>
<br>
Cheers,<br>
Daniel<div><div class="h5"><br>
<br>
<div>On 10/2/13 2:30 AM, David Cunningham
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Daniel,<br>
<br>
When I send SIGUSR1 to a worker process nothing is logged. Any
ideas there? The following is the output of "sercmd pkg.stats".<br>
<br>
{<br>
entry: 0<br>
pid: 14570<br>
rank: 0<br>
used: 191100<br>
free: 3731504<br>
real_used: 462800<br>
}<br>
{<br>
entry: 1<br>
pid: 14571<br>
rank: 1<br>
used: 202656<br>
free: 3714236<br>
real_used: 480068<br>
}<br>
{<br>
entry: 2<br>
pid: 14572<br>
rank: 2<br>
used: 202720<br>
free: 3714220<br>
real_used: 480084<br>
}<br>
{<br>
entry: 3<br>
pid: 14573<br>
rank: 3<br>
used: 202664<br>
free: 3714324<br>
real_used: 479980<br>
}<br>
{<br>
entry: 4<br>
pid: 14574<br>
rank: 4<br>
used: 202744<br>
free: 3714100<br>
real_used: 480204<br>
}<br>
{<br>
entry: 5<br>
pid: 14575<br>
rank: 5<br>
used: 201908<br>
free: 3715176<br>
real_used: 479128<br>
}<br>
{<br>
entry: 6<br>
pid: 14576<br>
rank: 6<br>
used: 202572<br>
free: 3714368<br>
real_used: 479936<br>
}<br>
{<br>
entry: 7<br>
pid: 14577<br>
rank: 7<br>
used: 202740<br>
free: 3714488<br>
real_used: 479816<br>
}<br>
{<br>
entry: 8<br>
pid: 14578<br>
rank: 8<br>
used: 199760<br>
free: 3717084<br>
real_used: 477220<br>
}<br>
{<br>
entry: 9<br>
pid: 14579<br>
rank: 9<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 10<br>
pid: 14580<br>
rank: 10<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 11<br>
pid: 14581<br>
rank: 11<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 12<br>
pid: 14582<br>
rank: 12<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 13<br>
pid: 14583<br>
rank: 13<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 14<br>
pid: 14584<br>
rank: 14<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 15<br>
pid: 14585<br>
rank: 15<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 16<br>
pid: 14586<br>
rank: 16<br>
used: 199440<br>
free: 3722828<br>
real_used: 471476<br>
}<br>
{<br>
entry: 17<br>
pid: 14587<br>
rank: -1<br>
used: 199400<br>
free: 3722868<br>
real_used: 471436<br>
}<br>
{<br>
entry: 18<br>
pid: 14588<br>
rank: -1<br>
used: 2105844<br>
free: 3722868<br>
real_used: 2337944<br>
}<br>
{<br>
entry: 19<br>
pid: 14589<br>
rank: -2<br>
used: 198552<br>
free: 3723236<br>
real_used: 471068<br>
}<br>
{<br>
entry: 20<br>
pid: 0<br>
rank: 0<br>
used: 209756<br>
free: 3712368<br>
real_used: 481936<br>
}<br>
{<br>
entry: 21<br>
pid: 14591<br>
rank: -4<br>
used: 253660<br>
free: 3668704<br>
real_used: 525600<br>
}<br>
<br>
<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On 1 October 2013 22:45,
Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
<br>
send SIGUSR1 to one of the worker processes (e.g., UDP
listener). The main process of kamailio doesn't process
SIP messages, so it is not exposed to leaks.<br>
<br>
You can used 'kamcmd pkg.stats" to see statistics about
usage of pkg per each process (older versions have
sercmd). <br>
<br>
'kamctl ps' gives the list of processes.<br>
<br>
Cheers,<br>
Daniel
<div>
<div><br>
<br>
<div>On 10/1/13 2:35 PM, David Cunningham wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Hi Daniel,<br>
<br>
</div>
What I copied and pasted is all that's listed
when I do a kill -SIGUSR1 on the Kamailio master
process. How do I get fuller logs?<br>
<br>
</div>
Thanks for your help.<br>
<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On 1 October 2013 21:57,
Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
you gave only the first part of memory log
output. The chunks listed there are from
startup, thus not part of a leak, because they
are not allocated again at runtime.<br>
<br>
Can you give the full logs with messages from
qm_status? The leaks are typically listed
towards the end of those messages.<br>
<br>
Yesterday someone else reported a potential
leak when using perl module, so it might be
something else there. I found one, which is
related to an error case, but perhaps you
don't get that, otherwise you will notice the
error message.<br>
<br>
Cheers,<br>
Daniel
<div>
<div><br>
<br>
On 10/1/13 12:03 PM, David Cunningham
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Hello,<br>
<br>
Following up on this after a long break.
It seems we are still having a memory
leak. We've added the memory debugging
options as per the documentation with
the following output.<br>
I'm not sure how to read it though - can
anyone help? Thank you.<br>
<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
<core> [main.c:754]: Memory status
(pkg):<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: (0x4019b008):<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: heap size= 4194304<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: used= 191100,
used+overhead=462800, free=3731504<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: max used (+overhead)= 480224<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: dumping all alloc'ed.
fragments:<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 0. N address=0x401cf414
frag=0x401cf3fc size=512 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 1. N address=0x401cf644
frag=0x401cf62c size=128 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 2. N address=0x401cf6f4
frag=0x401cf6dc size=256 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: counters.c:
cnt_hash_add(331)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 3. N address=0x401cf824
frag=0x401cf80c size=64 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: counters.c:
init_counters(121)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 4. N address=0x401cf894
frag=0x401cf87c size=8 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: route.c: init_rlist(180)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 5. N address=0x401cf8cc
frag=0x401cf8b4 size=64 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 6. N address=0x401cf93c
frag=0x401cf924 size=24 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: route.c: route_add(159)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 7. N address=0x401cf984
frag=0x401cf96c size=8 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: route.c: init_rlist(180)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 8. N address=0x401cf9bc
frag=0x401cf9a4 size=64 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 9. N address=0x401cfa2c
frag=0x401cfa14 size=24 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: route.c: route_add(159)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 10. N address=0x401cfa74
frag=0x401cfa5c size=8 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: db_id.c: dupl_string(50)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 11. N address=0x401cfaac
frag=0x401cfa94 size=64 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: str_hash.h:
str_hash_alloc(69)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 12. N address=0x401cfb1c
frag=0x401cfb04 size=24 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: route.c: route_add(159)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 13. N address=0x401cfb64
frag=0x401cfb4c size=8 used=1<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: alloc'd from
<core>: route.c: init_rlist(180)<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: start
check=f0f0f0f0, end check= c0c0c0c0,
abcdefed<br>
Oct 1 05:58:48 sip0-test
/sbin/kamailio[14570]: NOTICE:
qm_status: 14. N address=0x401cfb9c
frag=0x401cfb84 size=64 used=1<br>
<br>
<br>
</blockquote>
<br>
</div>
</div>
<div> -- <br>
Daniel-Constantin Mierla - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a><br>
<a href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a>
- <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a><br>
</div>
Kamailio Advanced Trainings - Berlin, Nov
25-28; Miami, Nov 18-20, 2013<br>
- more details about Kamailio trainings at <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
-<br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
David Cunningham, Voisonics<br>
<a href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
USA: <a href="tel:%2B1%20213%20221%201092" value="+12132211092" target="_blank">+1 213 221
1092</a><br>
UK: <a href="tel:%2B44%20%280%29%2020%203298%201642" value="+442032981642" target="_blank">+44 (0) 20
3298 1642</a><br>
Australia: <a href="tel:%2B61%20%280%29%202%208063%209019" value="+61280639019" target="_blank">+61 (0) 2
8063 9019</a><br>
</div>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
David Cunningham, Voisonics<br>
<a href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
USA: <a href="tel:%2B1%20213%20221%201092" value="+12132211092" target="_blank">+1 213 221 1092</a><br>
UK: <a href="tel:%2B44%20%280%29%2020%203298%201642" value="+442032981642" target="_blank">+44 (0) 20 3298 1642</a><br>
Australia: <a href="tel:%2B61%20%280%29%202%208063%209019" value="+61280639019" target="_blank">+61 (0) 2 8063 9019</a><br>
</div>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a href="http://twitter.com/#!/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
</div></div></div>
</blockquote></div><br><br clear="all"><br>-- <br>David Cunningham, Voisonics<br><a href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>USA: +1 213 221 1092<br>UK: +44 (0) 20 3298 1642<br>Australia: +61 (0) 2 8063 9019<br>
</div>