[SR-Users] async workers causing unexplained high load average

[Alex Balashov]

I just tested the same thing on a Rackspace VPS (Xen I think) and am 
seeing the same climbing load average with 2 async workers. It seems to 
top out at 1.05.

On 10/23/2014 02:13 PM, Alex Balashov wrote:

> Another thing I have found is that having a certain amount of async
> workers running, even if they are not doing anything, appears to cause
> unexplained CPU load, even if the Kamailio instance is completely idle
> and not processing any calls.
>
> Here is the baseline load with no async workers:
>
> [root at centosity6 ~]# uptime
>   14:06:28 up 2 min,  2 users,  load average: 0.01, 0.00, 0.00
>
> If I start Kamailio with 1 async worker...
>
>     modparam("async", "workers", 1)
>     async_workers=1
>
>   14:07:59 up 3 min,  2 users,  load average: 0.00, 0.00, 0.00
>
> Still nothing.
>
> Now I try 2:
>
>   14:08:54 up 4 min,  2 users,  load average: 0.45, 0.11, 0.04
>   14:08:59 up 4 min,  2 users,  load average: 0.49, 0.12, 0.04
>   14:09:04 up 4 min,  2 users,  load average: 0.53, 0.14, 0.04
>   14:09:09 up 4 min,  2 users,  load average: 0.57, 0.15, 0.05
>   14:09:14 up 5 min,  2 users,  load average: 0.60, 0.17, 0.05
>   14:09:19 up 5 min,  2 users,  load average: 0.63, 0.18, 0.06
>   14:09:31 up 5 min,  2 users,  load average: 0.69, 0.21, 0.07
>   14:09:37 up 5 min,  2 users,  load average: 0.71, 0.22, 0.07
>   14:10:35 up 6 min,  2 users,  load average: 0.89, 0.36, 0.13
>   14:11:07 up 6 min,  2 users,  load average: 0.93, 0.42, 0.16
>   ...
>
> I don't have time right now to see if it will rise to infinity, but it
> just keeps going up.
>
> There are no SIP messages being received at all, and there is absolutely
> nothing going on.
>
> The ascent of the load average is much higher with >= 4 threads. I have
> 8 "CPUs" available from /proc/cpuinfo (quad-core processor with
> HyperThreading).
>
> When I attach strace to the PID of the Async Worker, there's nothing
> going on except the normal recvfrom() blocking wait:
>
> [root at centosity6 kamailio]# kamctl ps | grep Async
> which: no greadlink in
> (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/)
> Process::  ID=11 PID=1559 Type=Async Task Worker
> Process::  ID=12 PID=1560 Type=Async Task Worker
>
> [root at centosity6 kamailio]# strace -p 1559
> Process 1559 attached - interrupt to quit
> recvfrom(9,
>
> [root at centosity6 ~]# strace -p 1560
> Process 1560 attached - interrupt to quit
> recvfrom(9,
>
> Nor do the other Kamailio threads seem to be doing anything unusual. 'ps
> aux' and 'top' show 0.0% CPU load on the Kamailio processes themselves,
> and 'iowait' shows the system as 98-99% idle.
>
> I suppose it's possible that this is a peculiarity of the testing
> environment: I am running this on CentOS 6.5 inside VirtualBox, which I
> think is KVM-based. I have not had a chance to duplicate this experiment
> in a different kind of execution environment yet. It's possible that the
> load average is simply being improperly indicated because of the VM
> environment.
>
>


-- 
Alex Balashov - Principal
Evariste Systems LLC
Tel: +1-678-954-0670
Web: http://www.evaristesys.com/, http://www.alexbalashov.com/

Please be kind to the English language:

http://www.entrepreneur.com/article/232906