<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi everyone,<br>
<br>
I have a problem with Kamailio consuming memory. I started testing
with Kamailio 3.2.1 upgraded to 3.2.2 and also tested 3.1.5,
everywhere the same problem. Whereas Kamailio 1.5.0 is working fine.<br>
Kamailio -V shows the following:<br>
<pre> kamailio@test:~$ kamailio -V
version: kamailio 3.2.2 (i386/linux) 98ba92
flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 98ba92
compiled on 17:51:30 Feb 29 2012 with gcc 4.2.4
kamailio@test:~$</pre>
My test scenario consists of two SIPp servers (a UAC and a UAS) and
the Kamailio in the middle proxying the SIP traffic.<br>
The UAC sends 300 calls per second (INVITE, 100, 180, 200, ACK) and
releases the calls (BYE, 200) after 20 sec call duration. UAS is
simply sending back the right responds. Protocol is UDP.<br>
<br>
After some minutes the Kamailio floods the syslog file with error
messages indicating that he is running out of memory. The SIPp shows
retransmissions and also timeouts. When I stop the test and do a
single call I get some lines of error messages. I found three
different sets of error messages for a single call so far: <br>
<pre> Mar 2 09:35:44 test P-CSCFtest[21251]: ERROR: <core> [parser/msg_parser.c:172]: get_hdr_field: out of memory
Mar 2 09:35:44 test P-CSCFtest[21251]: INFO: <core> [parser/msg_parser.c:353]: ERROR: bad header field [To: <<a class="moz-txt-link-freetext" href="sip:00498111111">sip:00498111111</a>]
Mar 2 09:35:44 test P-CSCFtest[21251]: ERROR: tm [t_lookup.c:1118]: ERROR: reply cannot be parsed
Mar 2 09:35:44 test P-CSCFtest[21251]: ERROR: <core> [msg_translator.c:1884]: ERROR: build_res_buf_from_sip_res: out of mem
Mar 2 09:35:44 test P-CSCFtest[21251]: ERROR: <core> [forward.c:794]: ERROR: forward_reply: building failed
Mar 2 09:35:44 test P-CSCFtest[21228]: ERROR: <core> [msg_translator.c:1884]: ERROR: build_res_buf_from_sip_res: out of mem
Mar 2 09:35:44 test P-CSCFtest[21228]: ERROR: tm [t_reply.c:1786]: ERROR: relay_reply: no mem for outbound reply buffer
Mar 2 09:35:44 test P-CSCFtest[21228]: ERROR: <core> [msg_translator.c:2036]: ERROR: build_res_buf_from_sip_req: out of memory ; needs 311
Mar 2 10:19:43 test P-CSCFtest[21346]: ERROR: <core> [msg_translator.c:1884]: ERROR: build_res_buf_from_sip_res: out of mem
Mar 2 10:19:43 test P-CSCFtest[21346]: ERROR: tm [t_reply.c:1786]: ERROR: relay_reply: no mem for outbound reply buffer
Mar 2 10:19:43 test P-CSCFtest[21346]: ERROR: <core> [msg_translator.c:2036]: ERROR: build_res_buf_from_sip_req: out of memory ; needs 311
Mar 2 10:20:16 test P-CSCFtest[21328]: ERROR: <core> [action.c:852]: ERROR: do_action: memory allocation failure
Mar 2 10:20:16 test P-CSCFtest[21328]: ERROR: <core> [action.c:1568]: run action error at: /usr/local/etc/kamailio//kamailio.cfg:28
Mar 2 10:20:16 test P-CSCFtest[21328]: ERROR: <core> [action.c:852]: ERROR: do_action: memory allocation failure
Mar 2 10:20:16 test P-CSCFtest[21328]: ERROR: <core> [action.c:1568]: run action error at: /usr/local/etc/kamailio//kamailio.cfg:32
Mar 2 10:20:16 test P-CSCFtest[21328]: ERROR: <core> [parser/parse_from.c:71]: ERROR:parse_from_header: out of pkg_memory
Mar 2 10:20:16 test P-CSCFtest[21328]: ERROR: tm [t_lookup.c:1326]: ERROR: new_t: no valid From in INVITE
Mar 2 10:20:16 test P-CSCFtest[21328]: ERROR: tm [t_lookup.c:1478]: ERROR: t_newtran: new_t failed
Mar 2 10:20:16 test P-CSCFtest[21328]: WARNING: <core> [receive.c:209]: WARNING: receive_msg: error while trying script
</pre>
<title></title>
<meta name="GENERATOR" content="OpenOffice.org 3.1 (Win32)">
<style type="text/css">
<!--
@page { margin: 2cm }
P { margin-bottom: 0.21cm }
--</style>I repeated the test various times, it always generates a
memory problem. The "top" command shows that the free memory is
decreasing constantly during the test. Stopping the test and waiting
for some time (1 hour) doesn't increase the free memory showed by
"top". Only a restart of the Kamailio frees the memory again. <br>
I executed the test with 30 calls per second too and got the same
behaviour (after some more minutes).<br>
<br>
I finally simplified the Kamailio config as far as possible and came
to this minimum:<br>
<meta http-equiv="CONTENT-TYPE" content="text/html;
charset=ISO-8859-1">
<pre> #!KAMAILIO
listen=10.10.10.10:5060
children=30
user="kamailio"
group="kamailio"
debug=2
disable_core_dump=yes
log_facility=LOG_LOCAL0
log_name="P-CSCFtest"
server_signature=no
sip_warning=0
syn_branch=0
disable_tcp=yes
####### Modules Section #######
mpath="/usr/local/lib/kamailio/modules_k/:/usr/local/lib/kamailio/modules/"
loadmodule "tm.so"
####### Main Routing Logic ########
route{
if (src_ip==1.2.3.4) {
rewritehost("5.6.7.8");
} else {
rewritehost("1.2.3.4");
};
rewriteport("5061");
t_relay();
}</pre>
<title></title>
<meta name="GENERATOR" content="OpenOffice.org 3.1 (Win32)">
<style type="text/css">
<!--
@page { margin: 2cm }
P { margin-bottom: 0.21cm }
-->
</style>This config still generates the memory problem. After replacing
the "t_relay()" with the stateless "forward()" the problem is gone.<br>
I played around with the number of child processes too, but no
difference.<br>
<br>
Is this a memory leak of the TM module or is there a different
explanation? <br>
<br>
Regards Fred <br>
<br>
</body>
</html>