<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hello,<br>
<br>
<div class="moz-cite-prefix">On 12/05/14 17:29, Igor Potjevlesch
wrote:<br>
</div>
<blockquote cite="mid:008c01cf6df6$f5bcd790$e13686b0$@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"Préformaté HTML Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.PrformatHTMLCar
{mso-style-name:"Préformaté HTML Car";
mso-style-priority:99;
mso-style-link:"Préformaté HTML";
font-family:Consolas;
color:black;
mso-fareast-language:EN-US;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><a moz-do-not-send="true"
name="_MailEndCompose"><span style="color:#1F497D"
lang="EN-GB">Hello Daniel,<o:p></o:p></span></a></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">Thank
you for this feedback.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">We
are running 4.0.4 without custom module.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">I
do a sqlops request in a very small table (40 rows) but on
each REGISTER and INVITE.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">The
bigger table is “aliases” with 154k rows.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">Can
you highlight me the particular bug concerned and corrected?
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">In
the meantime, for the memory allocation, I have to grow the
shared and/or private memory? What would be the next
increment : -M 128?</span></p>
</div>
</blockquote>
it doesn't really matter the increment. I think the private memory
is already too big, so if you don't load big amount of data, there
could be a leak.<br>
<br>
I backported related patches to branch 4.0 -- you should upgrade to
latest git branch 4.0. In this way you are sure you have all the
fixes, because 4.0.4 is old in the 4.0 branch.<br>
<br>
You can use 'kamcmd pkg.stats' to monitor available pkg memory. When
you see the available memory keep decreasing, you can issue:<br>
<br>
kam
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<span class="st">cmd cfg.set_now_int core mem_dump_pkg </span>_PID_<br>
<br>
replace PID with an appropriate value you take from pkg.stats. You
can send the output from syslog here for investigation, to see what
can be the potential leak.<br>
<br>
If you still have the core file, you can look inside it to see
allocated memory chunks -- see the gdb script at:<br>
<br>
-
<a class="moz-txt-link-freetext" href="http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory#walking_through_pkg_with_gdb">http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory#walking_through_pkg_with_gdb</a><br>
<br>
Cheers,<br>
Daniel<br>
<br>
<br>
<blockquote cite="mid:008c01cf6df6$f5bcd790$e13686b0$@gmail.com"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB">Igor.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-GB"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="color:windowtext;mso-fareast-language:FR"
lang="EN-GB">De :</span></b><span
style="color:windowtext;mso-fareast-language:FR"
lang="EN-GB"> <a class="moz-txt-link-abbreviated" href="mailto:sr-users-bounces@lists.sip-router.org">sr-users-bounces@lists.sip-router.org</a>
[<a class="moz-txt-link-freetext" href="mailto:sr-users-bounces@lists.sip-router.org">mailto:sr-users-bounces@lists.sip-router.org</a>] <b>De la
part de</b> Daniel-Constantin Mierla<br>
<b>Envoyé :</b> lundi 12 mai 2014 16:43<br>
<b>À :</b> Kamailio (SER) - Users Mailing List<br>
<b>Objet :</b> Re: [SR-Users] Memory issue<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
lang="EN-GB">Hello,<br>
<br>
what version are you using? Are you doing sql queries that
return large number of records (e.g., via sqlops)? Are you
using any custom module you developed?<br>
<br>
The crash was fixed in master branch, but somehow forgotten
to be backported -- it is now in branch 4.1.<br>
<br>
Cheers,<br>
Daniel</span><span
style="font-size:12.0pt;mso-fareast-language:FR"
lang="EN-GB"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-GB">On 12/05/14 15:58,
Igor Potjevlesch wrote:<o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span lang="EN-GB">Hello,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">I had a memory issue
this morning.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">First, I could see the
following logs in /var/log/message :<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:21
localhost /usr/local/sbin/kamailio[30268]: ERROR:
<core> [rvalue.c:2370]: rval_expr_eval(): rv eval
int expression: out of memory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:25
localhost /usr/local/sbin/kamailio[30268]: ERROR:
<core> [msg_translator.c:2164]:
build_res_buf_from_sip_req(): ERROR:
build_res_buf_from_sip_req: out of memory ; needs 482<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:26
localhost /usr/local/sbin/kamailio[30268]: ERROR:
<core> [msg_translator.c:2012]:
build_res_buf_from_sip_res(): ERROR:
build_res_buf_from_sip_res: out of mem<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:27
localhost /usr/local/sbin/kamailio[30268]: ERROR:
<core> [msg_translator.c:1910]:
build_req_buf_from_sip_req(): ERROR:
build_req_buf_from_sip_req: out of memory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:28
localhost /usr/local/sbin/kamailio[30268]: ERROR:
<core> [rvalue.c:2370]: rval_expr_eval(): rv eval
int expression: out of memory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:31
localhost /usr/local/sbin/kamailio[30268]: ERROR:
<core> [msg_translator.c:1910]:
build_req_buf_from_sip_req(): ERROR:
build_req_buf_from_sip_req: out of memory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:32
localhost /usr/local/sbin/kamailio[30268]: ERROR:
<core> [db_res.c:181]: db_allocate_rows(): no
private memory left<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:32
localhost /usr/local/sbin/kamailio[30268]: ERROR: db_mysql
[km_res.c:175]: db_mysql_convert_rows(): could not
allocate rows<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:32
localhost /usr/local/sbin/kamailio[30268]: ERROR: db_mysql
[km_res.c:217]: db_mysql_convert_result(): error while
converting rows<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:32
localhost /usr/local/sbin/kamailio[30268]: ERROR: db_mysql
[km_dbase.c:191]: db_mysql_store_result(): error while
converting result<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:33
localhost kernel: kamailio[30268]: segfault at 30 ip
00007ff44dd93f93 sp 00007fffcb00a200 error 4 in
db_mysql.so[7ff44dd8f000+19000]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:37
localhost /usr/local/sbin/kamailio[30235]: ALERT:
<core> [main.c:788]: handle_sigs(): child process
30268 exited by a signal 11<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:37
localhost /usr/local/sbin/kamailio[30235]: ALERT:
<core> [main.c:791]: handle_sigs(): core was
generated<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">May 12 11:08:37
localhost /usr/local/sbin/kamailio[30235]: INFO:
<core> [main.c:803]: handle_sigs(): INFO:
terminating due to SIGCHLD<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">This problem have
already occurred and I have set –M 64 to avoid it. It
looks that it’s not enough anymore.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">What is the next
increment recommended after 64M?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">On the contrary of the
first occurrence, Kamailio crash with coredump just after
these errors. I’m not sure that the memory message errors
are linked to the crash. I have tried a “bt full”:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">(gdb) bt full<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">#0
db_mysql_store_result (_h=0x7ff44e0e70e0,
_r=0x7fffcb00a408) at km_dbase.c:198<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> code =
<value optimized out><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> __FUNCTION__ =
"db_mysql_store_result"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">#1 0x00007ff44d971e72
in db_do_query_internal (_h=0x7ff44e0e70e0,
_k=0x7fffcb00a3e0, _op=0x0, _v=0x7fffcb00a3a0,
_c=<value optimized out>, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> _n=<value
optimized out>, _nc=2, _o=0x0, _r=0x7fffcb00a408,
val2str=0x7ff44dd97fa0 <db_mysql_val2str>, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> submit_query=0x7ff44dd92ed0
<db_mysql_submit_query>, store_result=0x7ff44dd93d80
<db_mysql_store_result>, _l=0) at db_query.c:137<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> tmp =
<value optimized out><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> off =
<value optimized out><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> ret =
<value optimized out><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> __FUNCTION__ =
"db_do_query_internal"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">#2 0x00007ff44d97269a
in db_do_query (_h=<value optimized out>,
_k=<value optimized out>, _op=<value optimized
out>, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> _v=<value
optimized out>, _c=<value optimized out>,
_n=<value optimized out>, _nc=2, _o=0x0,
_r=0x7fffcb00a408, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> val2str=0x7ff44dd97fa0
<db_mysql_val2str>, submit_query=0x7ff44dd92ed0
<db_mysql_submit_query>, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> store_result=0x7ff44dd93d80
<db_mysql_store_result>) at db_query.c:156<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Thanks in advance for
your help.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Igor.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:FR"
lang="EN-GB"><br>
<br>
<br>
<o:p></o:p></span></p>
<pre><span lang="EN-GB">_______________________________________________<o:p></o:p></span></pre>
<pre><span lang="EN-GB">SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list<o:p></o:p></span></pre>
<pre><a moz-do-not-send="true" href="mailto:sr-users@lists.sip-router.org"><span lang="EN-GB">sr-users@lists.sip-router.org</span></a><span lang="EN-GB"><o:p></o:p></span></pre>
<pre><a moz-do-not-send="true" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users"><span lang="EN-GB">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</span></a><span lang="EN-GB"><o:p></o:p></span></pre>
</blockquote>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:FR"
lang="EN-GB"><br>
<br>
<o:p></o:p></span></p>
<pre>-- <o:p></o:p></pre>
<pre>Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com">http://www.asipto.com</a><o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://twitter.com/#%21/miconda">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a><o:p></o:p></pre>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a></pre>
</body>
</html>