<div dir="ltr"><div>Hi,</div><div> </div><div>I recompiled with MEMDBG=1 and installed.</div><div>here are the results for reloading 5 million rows with MTREE:</div><div> </div><div>mem_join=1 -->takes 56 seconds and the real_used_size of shmem is around 1.2Gb.</div>
<div>mem_join=0 --> takes 10 seconds and the real_used_size of shmem is around 2Gb<br></div><div>does it seems normal?</div><div>56 seconds is a lot of time......</div><div> </div><div>by the way, when the f_malloc was used, the size of the real_used shmem was twice smaller.</div>
<div> </div><div>Thanks.<br></div><div class="gmail_quote">On Tue, Nov 20, 2012 at 9:45 PM, Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span> wrote:<br>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<div bgcolor="#FFFFFF" text="#000000">
Hello,<div class="im"><br>
<br>
<div>On 11/20/12 7:34 PM, Uri Shacked wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hi,</div>
<div> </div>
<div>can you be a litle more specific of the steps of the
install and where do i make the changes?</div>
</div>
</blockquote>
<br></div>
in the source tree, edit the file Makefile.defs and set:<br>
<br>
MEMDBG=1<br>
<br>
then run:<br>
<br>
make all<br>
make install<div class="im"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>some words of what is the diff between f_malloc and
q_malloc will be great :-).</div>
</div>
</blockquote>
<br></div>
q_malloc is more debugging purposes, keeping more information for
each chunk, therefore the overhead is a bit higher than with
f_malloc, but because keeps more details, it is faster to find the
fragments that can be joined.<br>
<br>
Cheers,<br>
Daniel<div><div class="h5"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div> </div>
<div>thanks,</div>
<div>Uri<br>
<br>
</div>
<div class="gmail_quote">On Tue, Nov 20, 2012 at 6:26 PM,
Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span> wrote:<br>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
<br>
ok, I will look over it. At this moment the f_malloc
(which is enabled for 3.3) has a pretty inefficient mem
join implementation, can you try with q_malloc? Edit
Makefile.defs and set:<br>
<br>
MEMDBG=1<br>
<br>
Then compile and install.<br>
<br>
The join operation should be faster, let's see if you get
blocking issues with this one.<br>
<br>
Cheers,<br>
Daniel
<div>
<div><br>
<br>
<div>On 11/20/12 2:57 PM, Uri Shacked wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Daniel hi,</div>
<div> </div>
<div>I attached 2 txt files.</div>
<div>One with mem_join=1, the other with
mem_join=0, and the info you asked for.</div>
<div>Let me know if it is OK.</div>
<div> </div>
<div>Thanks,</div>
<div>Uri<br>
<br>
</div>
<div class="gmail_quote">On Mon, Nov 19, 2012 at
10:50 AM, Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span>
wrote:<br>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
<br>
if you set memjoin to 0, do you see any
difference?<br>
<br>
Can you try again (with memjoin 1 as well as
0) and send the output of:<br>
<br>
kamctl mi get_statistics shmem:<br>
<br>
before executing the reload commands?<br>
<br>
When it gets to 100%, can you see which
process is using the cpu and attach to it
with:<br>
<br>
gdb /path/to/kamailio PID<br>
<br>
then do:<br>
<br>
bt full<br>
<br>
and send output here?<br>
<br>
Cheers,<br>
Daniel
<div>
<div><br>
<br>
<div>On 11/18/12 4:09 PM, Uri Shacked
wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>
<div dir="ltr">
<div>After some testing I notice the
following:</div>
<div>First reload of 5 million
records after kamailio started
took about 9 sec.</div>
<div>Second reload (4 minutes after
the first one) took 60 sec.</div>
<div>The third one (again about 4
minutes after the secind) got
kamailio to use 100% cpu and after
13 minutes! i killed it.....</div>
<div> </div>
<div>I can understand that the
memory manger works harder, still,
any ideas on how to use mem_join
and keep on reloading data.</div>
<div>(in real life our data loads 5
million records once a day when
almost no traffic. still after a
few days it stops...)</div>
<div> </div>
<div>Thanks,</div>
<div>Uri</div>
<div><br>
<br>
</div>
<div class="gmail_quote">On Sun, Nov
18, 2012 at 11:52 AM, Uri Shacked
<span dir="ltr"><<a href="mailto:ushacked@gmail.com" target="_blank">ushacked@gmail.com</a>></span>
wrote:<br>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<div dir="ltr">
<div><font size="3" face="Times New
Roman"> </font>
<p style="margin:0cm 0cm 0pt;line-height:normal" class="MsoNormal"><span>Hi,</span></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0cm 0cm 0pt;line-height:normal" class="MsoNormal"><span>I
am using MTREE and
DIALPLAN modules to load
lots of info to
kamailio. (6 million
rows).</span></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0cm 0cm 0pt;line-height:normal" class="MsoNormal"><span>When
kamailio was running
with 3.2.1 (no
mem_join=1 option), the
used size was increasing
but the process of
loading the data was
fast eanough.</span></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0cm 0cm 0pt;line-height:normal" class="MsoNormal"><span>I
upgraded to 3.3.2 and
set mem_join=1. Now the
loading process take
about 10 time longer and
sometimes stops kamailio
from responding to
traffic.</span></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0cm 0cm 0pt;line-height:normal" class="MsoNormal"><span>Any
ideas?</span></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0cm 0cm 0pt;line-height:normal" class="MsoNormal"><span>Thanks,</span></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0cm 0cm 0pt;line-height:normal" class="MsoNormal"><span>Uri</span></p>
<font size="3" face="Times New Roman"> </font></div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><span><font color="#888888">
</font></span></pre>
<span><font color="#888888"> </font></span></blockquote>
<span><font color="#888888"> <br>
<pre cols="72">--
Daniel-Constantin Mierla - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a></pre>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a></pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a href="http://twitter.com/#!/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a></pre>
</div></div></div>
</blockquote></div><br></div>