<div dir="ltr"><div>Hi Daniel,<br><br></div><div>We do not do any external operations.<br><br></div><div>We are using janson 2.7 everywhere. I will try to update to latest janson version tomorrow.<br></div><div>All json operation is pretty much same, we are using only jansson_get.<br><br></div><div>In attachment you can see memory consumption. On the right 2 servers which are faced internet on the left which don't face internet. As you can see memory consumption is pretty dramatic.<br></div><div><br></div><div>Thank you for your help!<br></div><div><br></div><div>With kind regards,<br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Jurijs<br></div></div></div>
<br><div class="gmail_quote">On Thu, Oct 6, 2016 at 5:17 PM, Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hello,</p>
<p>are you doing different external operations than on the other
instances, like mi/rpc commands.</p>
<p>From the list of the modules you exposed, I think jansson has the
higher probability to work with system memory. Are you doing
different json operations in config that in the other instances of
kamailio? Are you using same version of libjansson everywhere?</p>
<p>Cheers,<br>
Daniel<br>
</p><div><div class="h5">
<div class="m_-4545043238606932376moz-cite-prefix">On 06/10/16 13:46, Jurijs Ivolga wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>Hi Daniel,<br>
<br>
</div>
This modules what we are using:<br>
<br>
loadmodule "mi_fifo.so"<br>
loadmodule "kex.so"<br>
loadmodule "corex.so"<br>
loadmodule "tm.so"<br>
loadmodule "tmx.so"<br>
loadmodule "sl.so"<br>
loadmodule "rr.so"<br>
loadmodule "pv.so"<br>
loadmodule "maxfwd.so"<br>
loadmodule "textops.so"<br>
loadmodule "siputils.so"<br>
loadmodule "xlog.so"<br>
loadmodule "sanity.so"<br>
loadmodule "ctl.so"<br>
loadmodule "cfg_rpc.so"<br>
loadmodule "mi_rpc.so"<br>
loadmodule "dispatcher.so"<br>
loadmodule "utils.so"<br>
loadmodule "path.so"<br>
loadmodule "ipops.so"<br>
loadmodule "jansson.so"<br>
loadmodule "auth.so"<br>
loadmodule "nathelper.so"<br>
loadmodule "tls.so"<br>
loadmodule "htable.so"<br>
loadmodule "pike.so"<br>
<br>
</div>
We have several other Kamailio instances but they are not
faced to internet and they do not have such memory issue. That
other Kamailio instances have same modules, except modules
listed below. So if you think that issue is inside external
library, probably we need to check first modules from list
below.<br>
<br>
<span class="m_-4545043238606932376gmail-diff-chunk m_-4545043238606932376gmail-diff-chunk-equal">loadmodule
"ipops.so"<br>
loadmodule "auth.so"<br>
loadmodule "nathelper.so"<br>
loadmodule "pike.so"</span><span class="m_-4545043238606932376gmail-diff-chunk m_-4545043238606932376gmail-diff-chunk-removed"></span><br>
<br>
</div>
<div>But maybe this other Kamailio instances do not have this
memory issue, just because they did not face to internet and
did not have same load as instances with memory issue.<br>
<br>
kamailio -v<br>
version: kamailio 4.4.3 (x86_64/linux) 5a2195<br>
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS,
USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM,
SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC,
DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT,
USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST,
HAVE_RESOLV_RES<br>
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144,
MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT
PKG_SIZE 8MB<br>
poll method support: poll, epoll_lt, epoll_et, sigio_rt,
select.<br>
id: 5a2195<br>
compiled on 08:30:51 Sep 15 2016 with gcc 4.4.7<br>
<br>
<br>
</div>
<div>With kind regards,<br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div class="m_-4545043238606932376gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">Jurijs<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Thu, Oct 6, 2016 at 12:52 PM,
Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hello,</p>
<p>it looks like a leak from the system memory, not from
kamailio's pkg or shm memory. This can be due to an
improper use of an external library (e.g., libxml2) by a
kamailio module or because of a problem in the library.</p>
<p>Can you list the modules used in your config (the
loadmodule lines)? I will try to guess from the list
which one relying on external libs with higher risk of
leak issues.</p>
<p>Also, provide the version of kamailio you are using
(kamailio -v).</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>
<div class="m_-4545043238606932376h5"> <br>
<div class="m_-4545043238606932376m_2624872147299544990moz-cite-prefix">On
04/10/16 15:42, Jurijs Ivolga wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="m_-4545043238606932376h5">
<div dir="ltr">
<div>Hi,<br>
<br>
</div>
Our Kamailio server is crashing once per week,
with following error:<br>
<br>
Oct 1 06:25:06 kamailio kernel: [26982632.803789]
Out of memory in UB 210: OOM killed process 12261
(kamailio) score 0 vm:1614768kB, rss:280200kB,
swap:131408kB<br>
<br>
<div>
<div>
<div>Core dump was never created, probably it
is because of my environment, but I will try
to get it.<br>
<br>
</div>
<div>Server constantly eats memory, maybe some
kind of memory leak?<br>
<br>
</div>
<div>Any help is highly appreciated!<br>
</div>
<div><br clear="all">
<div>
<div class="m_-4545043238606932376m_2624872147299544990m_-8889579837191443002m_-9076721087679106191gmail_signature">
<div dir="ltr">Jurijs<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="m_-4545043238606932376m_2624872147299544990mimeAttachmentHeader"></fieldset>
<br>
</div>
</div>
<pre>______________________________<wbr>_________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a class="m_-4545043238606932376m_2624872147299544990moz-txt-link-abbreviated" href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a class="m_-4545043238606932376m_2624872147299544990moz-txt-link-freetext" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cg<wbr>i-bin/mailman/listinfo/sr-user<wbr>s</a><span class="m_-4545043238606932376HOEnZb"><font color="#888888">
</font></span></pre><span class="m_-4545043238606932376HOEnZb"><font color="#888888">
</font></span></blockquote><span class="m_-4545043238606932376HOEnZb"><font color="#888888">
<pre class="m_-4545043238606932376m_2624872147299544990moz-signature" cols="72">--
Daniel-Constantin Mierla
<a class="m_-4545043238606932376m_2624872147299544990moz-txt-link-freetext" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a class="m_-4545043238606932376m_2624872147299544990moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/mic<wbr>onda</a>
Kamailio Advanced Training, Berlin, Oct 24-26, 2016 - <a class="m_-4545043238606932376m_2624872147299544990moz-txt-link-freetext" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a></pre>
</font></span></div>
______________________________<wbr>_________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" rel="noreferrer" target="_blank">http://lists.sip-router.org/cg<wbr>i-bin/mailman/listinfo/sr-user<wbr>s</a>
</blockquote></div>
</div>
</blockquote>
<pre class="m_-4545043238606932376moz-signature" cols="72">--
Daniel-Constantin Mierla
<a class="m_-4545043238606932376moz-txt-link-freetext" href="http://twitter.com/#!/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a class="m_-4545043238606932376moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/<wbr>miconda</a>
Kamailio Advanced Training, Berlin, Oct 24-26, 2016 - <a class="m_-4545043238606932376moz-txt-link-freetext" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a></pre></div></div></div></blockquote></div><br></div>