[SR-Users] kamailio memory leak in private memory

[david]

hello Daniel

i forgot to specify that in the dump attached, the first lines are for
the process UDP receiver carrying the traffic on an affected IP,
yesterday during the test we started the kamailio only with 1 children
per IP, so all the messages would have been processed by that process

i see that all the information of allocated mem for the root process is
the same in both moments after and before the traffic load, but in the
PID affected we see more hashes used after the load, i dont know how to
deal with that information, anything you need from us please let us know
and we will pleased to provide if possible

best regards
david



El vie, 17-02-2017 a las 14:17 +0100, david escribió:

> hello DAniel
> 
> we have 4.4.1 sorry
> we are only handling SIP calls without REGISTER messages
> 
> best regards
> david
> 
> El vie, 17-02-2017 a las 13:53 +0100, Daniel-Constantin Mierla
> escribió:
> 
> > Hello,
> > 
> > what kamailio version?
> > 
> > Are registrations handled by kamailio, or is just used for calls?
> > 
> > Cheers,
> > Daniel
> > 
> > 
> > 
> > On 17/02/2017 13:31, david wrote:
> > 
> > 
> > > Hello all
> > > 
> > > We are observing an issue in one kamailio proxy we use.
> > > The consecuence of an unknow cause is that the private memory of
> > > some UDP receiver processes in where we listen from a particular
> > > IP, seems to be draining along the time while processing traffic.
> > > Quite "slowly" but maybe aroung 200K in 3-4hours, depending on the
> > > moment of the day.
> > > 
> > > This is quite weird for us, because it seemed like it was caused
> > > by some particular traffic profile, since we have several IP
> > > listening (8 children each) and we only see the issue in the UDP
> > > receiver processes of some of those IPs. Besides we have moved the
> > > kamailio service listening on the same IP to several different
> > > physical servers, and the behaviour is the same.
> > > We look at it ussing pkg.stats kamcmd command and for instance if
> > > we track one of the process affected
> > > pid: 23784
> > > rank: 53
> > > used: 2838616
> > > free: 13005408
> > > real_used: 3771808
> > > 
> > > pid: 23784
> > > rank: 53
> > > used: 2842584
> > > free: 12999280
> > > real_used: 3777936
> > > 
> > > This 2 outputs were taken with 10min diff between each other.
> > > 
> > > But we dont know where or how to look at it.
> > > We have been these last days trying to search for patterns to
> > > allocate some particular traffic source causing something weird,
> > > but no luck.
> > > We have checked all the SIP INVITE messages looking for anything
> > > weird at any field but no luck, besides we sometimes see some bad
> > > headers with blank spaces in the uri or thing like that but those
> > > cases we have an error and we dont think it can cause a memory
> > > leak.
> > > Shared memory seems to be stable.
> > > 
> > > We also have checked the config we are using and we have been
> > > discarding possible issues since we dont think the $var variables
> > > we are using could cause that issue (but you never know....)
> > > 
> > > We tried to debug the kamailio on one affected IP compiling it
> > > with DBG_SYS_MALLOC , and doing the dump of the pkg mem status
> > > when the kamailio was restarted and no traffic was processed, and
> > > also when it had traffic processed. in this last case, we removed
> > > traffic, waited for 20min and made the dump
> > > 
> > > i attach you both files, we have the debug logs, but they are very
> > > big to be passed here, and we dont know what to look for :)
> > > 
> > > could you please give us some guidance or clue to at least start
> > > to searching for something that can cause this issue?
> > > 
> > > thanks  alot and regards
> > > david 
> > > 
> > > 
> > > _______________________________________________
> > > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
> > > sr-users at lists.sip-router.org
> > > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
> > 
> > 
> > 
> > _______________________________________________
> > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
> > sr-users at lists.sip-router.org
> > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20170217/506eac85/attachment.html>