[SR-Users] Kamailio not responding after stress test

[Daniel-Constantin Mierla]

Still now showing the debug symbols for libssl/libcrypto:

 1.
    0x00007fdc7d07754d in __lll_lock_wait () from /lib64/libpthread.so.0
 2.
    #0 0x00007fdc7d07754d in __lll_lock_wait () from /lib64/libpthread.so.0
 3.
    No symbol table info available.
 4.
    #1 0x00007fdc7d0743d2 in pthread_rwlock_wrlock () from
    /lib64/libpthread.so.0
 5.
    No symbol table info available.
 6.
    #2 0x00007fdc737f2c59 in CRYPTO_THREAD_write_lock () from
    /usr/local/lib64/kamailio/modules/tlsa.so
 7.
    No symbol table info available.
 8.
    #3 0x00007fdc7379e32e in CRYPTO_free_ex_data () from
    /usr/local/lib64/kamailio/modules/tlsa.so
 9.
    No symbol table info available.
10.
    #4 0x00007fdc73708ae2 in SSL_free () from
    /usr/local/lib64/kamailio/modules/tlsa.so
11.
    No symbol table info available.
12.

How are these libs installed?

Cheers,
Daniel

On 29.11.22 10:38, Ihor Olkhovskyi wrote:
>
> Daniel,
>
> Thanks for the tip, recompiled and got a new trace
>
> https://pastebin.com/NxCXA1gT
>
> Not sure what happened, but sometimes running kamctl trap restores
> Kamailio functionality to accept connections.
>
> Thanks in advance!
>
> Le 29/11/2022 à 08:07, Daniel-Constantin Mierla a écrit :
>>
>> Hello,
>>
>> is libssl/libcrypto compiled/installed with debugging symbols? They
>> are not shown in the backtrace.
>>
>> Cheers,
>> Daniel
>>
>> On 28.11.22 17:38, Igor Olhovskiy wrote:
>>> Hello,
>>>
>>> I'm continuing investigations on Kamailio and stress test. Got it
>>> again in the state where it's not accepting any new TCP/TLS
>>> connections (UDP still works though), but all looks good from
>>> lsof/netnstat part, like system is not reporting any zombie
>>> connections. Restart of Kamailio process helps
>>>
>>> This time I got output of kamctl trap
>>>
>>> Put it here: https://pastebin.com/iYrNZ8U9
>>>
>>> kamailio --version
>>> version: kamailio 5.6.2 (x86_64/linux) 54a9c1
>>> flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS,
>>> DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC,
>>> Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX,
>>> FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR,
>>> USE_DST_BLOCKLIST, HAVE_RESOLV_RES
>>> ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE
>>> 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
>>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>>> id: 54a9c1
>>> compiled on 14:01:01 Oct 18 2022 with gcc 4.8.5
>>>
>>> It's statically linked with tlsa pointing on openssl-1.1.1q
>>>
>>> Settings related to TLS are:
>>>
>>> fork=yes
>>> children=4
>>> tcp_children=12
>>> enable_tls=yes
>>>
>>> tcp_accept_no_cl=yes
>>> tcp_max_connections=63536
>>> tls_max_connections=63536
>>> tcp_accept_aliases=no
>>> tcp_async=yes
>>> tcp_connect_timeout=10
>>> tcp_conn_wq_max=63536
>>> tcp_crlf_ping=yes
>>> tcp_delayed_ack=yes
>>> tcp_fd_cache=yes
>>> tcp_keepalive=yes
>>> tcp_keepcnt=3
>>> tcp_keepidle=30
>>> tcp_keepintvl=10
>>> tcp_linger2=30
>>> tcp_rd_buf_size=80000
>>> tcp_send_timeout=10
>>> tcp_wq_blk_size=2100
>>> tcp_wq_max=10485760
>>> open_files_limit=63536
>>>
>>>
>>> Can you please help to read gdb output and understand where I missed
>>> in config?
>>>
>>> Thanks in advance!
>>>
>>>
>>> -- 
>>> Best regards,
>>> Ihor (Igor)
>>>
>>> __________________________________________________________
>>> Kamailio - Users Mailing List - Non Commercial Discussions
>>> sr-users at lists.kamailio.org
>>> Important: keep the mailing list in the recipients, do not reply only to the sender!
>>> Edit mailing list options or unsubscribe:
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>> -- 
>> Daniel-Constantin Mierla -- www.asipto.com
>> www.twitter.com/miconda -- www.linkedin.com/in/miconda

-- 
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20221202/cf010426/attachment.htm>