[sr-dev] Per module memmory consumption

[Daniel-Constantin Mierla]

Hello,

On 16/06/15 14:43, smititelu wrote:
> Hello everyone,
>
> We are thinking of a way to debug per module memory consumption (both
> private and shared). Something like a "kamctl fifo
> shmem***module_name*" to print how much memory has been alloc'ed for
> that module and which function had alloc'ed.
>
> One of the solutions would be to make use of the shm_info() and
> pkg_info(), call them at the beginning and at the end of a function
> and do some subtraction at the end. Then correlate this data with a
> per module stats which will be printed with "kamctl stats
> *module_name*". This would be ok if one needs mem stats only for some
> specific modules.

this would eventually work for pkg, but for shm is not a safe approach,
because in parallel, another process can allocate/dealocate shm.

Also, for shm, what is allocated inside a module can be dealocated
somewhere else (e.g., avps can be set by modules, but freed by core or tm).

>
> Another solution would be to make use of the *_malloc stubs used when
> MEM_DEBUG is enabled and keep the function name and size. Buf it would
> be hard to count how much memory is alloc'ed by the higher module
> function in scenarios when the actual allocations are done in other
> modules (e.g. db_allocate_rows()).

Given the above, when some memory is allocated by a module and freed by
another component, perhaps this is the best approach (at least the one
coming in my mind right now). Trying to keep a counter per module is not
sufficient, as allocated size won't be decreased when freeing is in
another place.
>
> Also we have in mind that one would not want to compile kamailio with
> MEM_DEBUG flag but still want to make use of the feature...
>
> If you think of some other solution or have some advices for this,
> please comment.
This has to be done as add on to the memory manager via a module,
something similar to what rpc command pkg.stats does, using some
callbacks, to be sure it is not slowing down the core when this feature
is not wanted/enabled.

Perhaps an array with the shm and pkg counters, updated on callbacks for
*malloc/*realloc/*free, still relying on module name stored in the
memory chunk structure at malloc time, in order to decrease it properly
at free time.

To speed up, instead of module name, it can be stored an index
associated at startup with the module name to avoid string comparisons.

When coding the feature, it would be good to add it in between #ifdef
... #endif to be able to do performance measurments and disable it
completely if has big impact. It has relevance for profiling and would
be useful to have, just to be sure is not slowing down a lot.

Cheers,
Daniel

-- 
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Book: SIP Routing With Kamailio - http://www.asipto.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20150617/020287c8/attachment.html>