<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hello,<br>
<br>
do you get errors like "pv_sprintf: ERROR: wrong format ..." from
perl module? It looks like a leak in this error case.<br>
<br>
If not, can you compile with MEMDBG=1, then set memlog in config
less that debug and run/test it for a while?<br>
<br>
Restart it and look at the logs at shut down to spot any memory leak.
You can send it to me if you want me to analyze them.<br>
<br>
I recommend you set children=2 or 4 to make the leak faster visible.<br>
<br>
Also, you can decide what types of stats you want via:<br>
<br>
<a class="moz-txt-link-freetext" href="http://www.kamailio.org/wiki/cookbooks/4.0.x/core#mem_summary">http://www.kamailio.org/wiki/cookbooks/4.0.x/core#mem_summary</a><br>
<br>
You can use value 12 to make the output more compact.<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<div class="moz-cite-prefix">On 9/30/13 4:25 PM, Dragos Oancea
wrote:<br>
</div>
<blockquote
cite="mid:1380551108.25456.YahooMailNeo@web124906.mail.ne1.yahoo.com"
type="cite">
<div style="color:#000; background-color:#fff; font-family:arial,
helvetica, sans-serif;font-size:10pt">
<div><span>Hi Daniel,</span></div>
<div style="color: rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; background-color: transparent;
font-style: normal;"><span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; background-color: transparent;
font-style: normal;"><span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; background-color: transparent;
font-style: normal;"><span>We are running Kamailio 4.0.3 . So
it appears that yes, we have this commit.</span></div>
<div><br>
</div>
<div style="color: rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; background-color: transparent;
font-style: normal;">Regards,</div>
<div style="color: rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; background-color: transparent;
font-style: normal;">Dragos</div>
<div style="color: rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; background-color: transparent;
font-style: normal;"><br>
</div>
<div style="font-family: arial, helvetica, sans-serif;
font-size: 10pt;">
<div style="font-family: 'times new roman', 'new york', times,
serif; font-size: 12pt;">
<div dir="ltr">
<hr size="1"> <font face="Arial" size="2"> <b><span
style="font-weight:bold;">From:</span></b>
Daniel-Constantin Mierla <a class="moz-txt-link-rfc2396E" href="mailto:miconda@gmail.com"><miconda@gmail.com></a><br>
<b><span style="font-weight: bold;">To:</span></b>
Dragos Oancea <a class="moz-txt-link-rfc2396E" href="mailto:droancea@yahoo.com"><droancea@yahoo.com></a>; Kamailio (SER)
- Users Mailing List
<a class="moz-txt-link-rfc2396E" href="mailto:sr-users@lists.sip-router.org"><sr-users@lists.sip-router.org></a> <br>
<b><span style="font-weight: bold;">Sent:</span></b>
Monday, September 30, 2013 4:06 PM<br>
<b><span style="font-weight: bold;">Subject:</span></b>
Re: [SR-Users] potential memory leak<br>
</font> </div>
<div class="y_msg_container"><br>
<div id="yiv5168215037">
<div> Hello,<br>
<br>
what version are you running? Is it after the commit:<br>
<br>
<a moz-do-not-send="true" rel="nofollow"
class="yiv5168215037moz-txt-link-freetext"
target="_blank"
href="http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=341f810dca0cc0596e22f2ac1bca86de0b8d142d">http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=341f810dca0cc0596e22f2ac1bca86de0b8d142d</a>
?<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<div class="yiv5168215037moz-cite-prefix">On 9/30/13
3:59 PM, Dragos Oancea wrote:<br>
</div>
<blockquote type="cite">
<div style="color: rgb(0, 0, 0); background-color:
rgb(255, 255, 255); font-family: arial, helvetica,
sans-serif; font-size: 10pt;">
<div style="font-family: arial, helvetica,
sans-serif; font-size: 10pt;">Hello</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 10pt;"><br>
</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 13px; color: rgb(0, 0,
0); background-color: transparent; font-style:
normal;"><br>
</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 13px; color: rgb(0, 0,
0); background-color: transparent; font-style:
normal;">We've put Kamailio on a machine with a
SIP-Proxy role, we replaced another SIP proxy
implementation.</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 13px; color: rgb(0, 0,
0); background-color: transparent; font-style:
normal;">We started with the default 4Mb of PKG
Memory and 1024 Mb of Shared Memory but we had
messages like these in the logs:</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 13px; color: rgb(0, 0,
0); background-color: transparent; font-style:
normal;"><br>
</div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:26,939 ERROR: app_perl
[kamailioxs.xs:371]: pv_sprintf(): pv_sprintf:
Memory exhausted!</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:50,267 ERROR: app_perl
[kamailioxs.xs:371]: pv_sprintf(): pv_sprintf:
Memory exhausted!</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,822 ERROR: <core>
[data_lump.c:197]: insert_subst_lump_after():
ERROR: insert_new_lump_after: out of memory</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,823 ERROR: <core>
[parser/contact/contact.c:194]:
parse_contacts(): parse_contacts(): No memory
left</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,823 ERROR: memcached
[../../parser/../ut.h:736]: pkg_str_dup():
could not allocate private memory from
available pool</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,823 ERROR: <core>
[data_lump.c:373]: anchor_lump(): ERROR:
anchor_lump: out of memory</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,823 ERROR: <core>
[dset.c:525]: rewrite_uri(): No memory left to
rewrite r-uri</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,823 ERROR: <core>
[action.c:810]: do_action(): ERROR: do_action:
memory allocation failure</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,823 ERROR: <core>
[msg_translator.c:2164]:
build_res_buf_from_sip_req(): ERROR:
build_res_buf_from_sip_req: out of memory ;
needs 357</font></div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">2013-09-26
13:48:51,823 ERROR: <core>
[msg_translator.c:2164]:
build_res_buf_from_sip_req(): ERROR:
build_res_buf_from_sip_req: out of memory ;
needs 376</font></div>
<div><br>
</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 13px; color: rgb(0, 0,
0); background-color: transparent; font-style:
normal;"><br>
</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 13px; color: rgb(0, 0,
0); background-color: transparent; font-style:
normal;">Then we started kamailio like below,
with 1024Mb of private memory (per process) :</div>
<div style="font-family: arial, helvetica,
sans-serif; font-size: 13px; color: rgb(0, 0,
0); background-color: transparent; font-style:
normal;"><br>
</div>
<div style="background-color:transparent;"><font
face="arial, helvetica, sans-serif" size="2">/opt/kamailio/sbin/kamailio
-f /opt/kamailio/etc/kamailio//kamailio.cfg -w
/opt/kamailio/core_dir/corefiles -P
/opt/kamailio/var/run//kamailio.pid -m 1024 -M
1024 -u kamailio -g kamailio</font><br>
</div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 16px; font-family:
'Times New Roman'; font-style: normal;"><font
face="arial, helvetica, sans-serif" size="2"><br>
</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2">Then after about 4 days
we realized that we have a steady increase of
used memory ( in the output of free command
the '-/+ buffers/cache:' row ), and we simply
restarted the kamailio process.</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2"><br>
</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2">The machine has 12 Gb of
RAM , we run with children=32 . 1gb of memory
per process x 32 child processes means that we
are over the physical capacity of the machine.</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2">The question is , are we
in any danger of running out of memory on this
box where we only run kamailio ? We don't want
to reach the point where the kernel starts
killing processes.</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2">Shall we suspect some
memory leak in the app_perl module ? (we do
some http queries over SSL from a perl script)</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2"><br>
</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2"><br>
</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2">Thank you.</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2"><br>
</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2">Regards,</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2">Dragos Oancea</font></div>
<div style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family:
arial, helvetica, sans-serif; font-style:
normal;"><font face="arial, helvetica,
sans-serif" size="2"><br>
</font></div>
</div>
<br>
<fieldset class="yiv5168215037mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a moz-do-not-send="true" rel="nofollow" class="yiv5168215037moz-txt-link-abbreviated" ymailto="mailto:sr-users@lists.sip-router.org" target="_blank" href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a>
<a moz-do-not-send="true" rel="nofollow" class="yiv5168215037moz-txt-link-freetext" target="_blank" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<br>
<pre class="yiv5168215037moz-signature">--
Daniel-Constantin Mierla - <a moz-do-not-send="true" rel="nofollow" class="yiv5168215037moz-txt-link-freetext" target="_blank" href="http://www.asipto.com/">http://www.asipto.com</a>
<a moz-do-not-send="true" rel="nofollow" class="yiv5168215037moz-txt-link-freetext" target="_blank" href="http://twitter.com/#%21/miconda">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" rel="nofollow" class="yiv5168215037moz-txt-link-freetext" target="_blank" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at <a moz-do-not-send="true" rel="nofollow" class="yiv5168215037moz-txt-link-freetext" target="_blank" href="http://www.asipto.com/">http://www.asipto.com</a> -
</pre>
</div>
</div>
<br>
<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a> -
</pre>
</body>
</html>