<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
color:black;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle23
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle24
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle25
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle26
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle27
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle28
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle29
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle30
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Thanks for your help Daniel!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">-Tim<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> Daniel-Constantin Mierla [mailto:miconda@gmail.com]
<br>
<b>Sent:</b> Thursday, September 11, 2014 9:59 AM<br>
<b>To:</b> Heenan, Timothy Steven; Kamailio (SER) - Users Mailing List<br>
<b>Subject:</b> Re: [SR-Users] Kamailio 4.0.4 slow memory leak<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Short note to say that last log you sent had more leads, and apparently there is a leak at least in 4.0.<br>
<br>
I need a bit more time to look at it and check if still on master and 4.1 (hopefully event later today) - I will come back with an update about it soon.<br>
<br>
Daniel<o:p></o:p></p>
<div>
<p class="MsoNormal">On 08/09/14 18:11, Daniel-Constantin Mierla wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt">Were you taking the memory logs after the errors, before restarting?<br>
<br>
I didn't get the chance to look over the previous email, I will do it soon.<br>
<br>
Daniel<o:p></o:p></p>
<div>
<p class="MsoNormal">On 08/09/14 17:48, Heenan, Timothy Steven wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="color:#1F497D">After restarting Kamailio on September 3rd, the problem has again gotten to the point of no call processing.
<br>
<br>
Here are the memory statistics; additionally, I can see the virtual memory has ballooned up to 400mb again.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">shmem:fragments = 2378</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">shmem:free_size = 209840</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">shmem:max_used_size = 268393216</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">shmem:real_used_size = 268225616</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">shmem:total_size = 268435456</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">shmem:used_size = 220534792</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">-Tim</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">
<a href="mailto:sr-users-bounces@lists.sip-router.org">sr-users-bounces@lists.sip-router.org</a> [<a href="mailto:sr-users-bounces@lists.sip-router.org">mailto:sr-users-bounces@lists.sip-router.org</a>]
<b>On Behalf Of </b>Heenan, Timothy Steven<br>
<b>Sent:</b> Wednesday, September 03, 2014 3:28 PM<br>
<b>To:</b> '<a href="mailto:miconda@gmail.com">miconda@gmail.com</a>'; Kamailio (SER) - Users Mailing List<br>
<b>Subject:</b> Re: [SR-Users] Kamailio 4.0.4 slow memory leak</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I have about 460 destination sets. In each set I could have from 1 address up to 6.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">When I took that initial memory log, Kamailio had been restarted shortly beforehand. I just took another one after the system had decayed so Kamailio was responding to Invites with “SIP/2.0 500 No error (2/SL).”
(exhibiting the issue after a few days) and the log is about 400Mb! I’ll attach a small snippet.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Seems to be going on and on about the dispatcher module.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">-Tim</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> Daniel-Constantin Mierla [<a href="mailto:miconda@gmail.com">mailto:miconda@gmail.com</a>]
<br>
<b>Sent:</b> Thursday, August 28, 2014 5:03 AM<br>
<b>To:</b> Heenan, Timothy Steven; Kamailio (SER) - Users Mailing List<br>
<b>Subject:</b> Re: [SR-Users] Kamailio 4.0.4 slow memory leak</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Looking at the logs, there are 1336 URIs, which seems ok given your numbers, because the module keeps also the previous set of records (needed because at reload time there can be kamailio worker processes using
the records -- perhaps we can improve a bit here, I will review the current reload code a bit, since it was a contribution from another developer quite long time ago).<br>
<br>
Some more questions:<br>
- how many destination sets (groups) do you have? Can you estimate the minimum and maximum addresses in a set?<br>
- when did you take the memory log? In other words, for how long was kamailio running? Did you wait enough to notice the steady increase of memory usage?<br>
<br>
Cheers,<br>
Daniel<o:p></o:p></p>
<div>
<p class="MsoNormal">On 28/08/14 01:56, Heenan, Timothy Steven wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="color:#1F497D">Thank you for the help. </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">For dispatcher, I’m using a database that contains about 700 records.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">A reload is performed via cronjob every 5 minutes. The command being used is:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">kamctl dispatcher reload</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">-Tim</span><o:p></o:p></p>
</blockquote>
</blockquote>
</blockquote>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif""><br>
<br>
<o:p></o:p></span></p>
<pre>-- <o:p></o:p></pre>
<pre>Daniel-Constantin Mierla<o:p></o:p></pre>
<pre><a href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a><o:p></o:p></pre>
<pre>Next Kamailio Advanced Trainings 2014 - <a href="http://www.asipto.com">http://www.asipto.com</a><o:p></o:p></pre>
<pre>Sep 22-25, Berlin, Germany<o:p></o:p></pre>
</div>
<HR>This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments.<BR>
</body>
</html>