<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hello,<br>
<br>
I kind of forgot about this thread ...<br>
<br>
The backtrace is clean and it doesn't show like a block/infinite
loop when you took it. Are you sure you took it when kamailio used
100%CPU?<br>
<br>
How many times you run 'n' after backtrace? The loop in cbp could
have tens of iterations.<br>
<br>
Maybe you can try again and let's see if you get same results on
backtrace/gdb.<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<br>
<div class="moz-cite-prefix">On 25/11/13 21:43, Daniel-Constantin
Mierla wrote:<br>
</div>
<blockquote cite="mid:5293B654.9050606@gmail.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Hello,<br>
<br>
the backtrace is ok for the moment. I will look over it and come
back with results.<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<div class="moz-cite-prefix">On 11/25/13 12:10 PM, Efelin Novak
wrote:<br>
</div>
<blockquote
cite="mid:CABKTgArt2i=Sk38RTeZjUKs===3YPYh_boMb6jjy_LrSdDYhaA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>Hi Daniel,<br>
<br>
</div>
sorry it took me more than I expected. Is this sufficient?
Meanwhile I found out that this happens when fr_inv_timer
triggers and dialog module is turned on.<br>
<br>
</div>
Backtrace:<br>
<div>
<div><br>
bt<br>
#0 futex_release (lock=0xb4b90798) at
../../mem/../futexlock.h:137<br>
#1 next_state_dlg (dlg=dlg@entry=0xb4bc5a38,
event=event@entry=4, old_state=old_state@entry=0xbfef08a0,
new_state=new_state@entry=0xbfef089c,
unref=unref@entry=0xbfef08a4) at dlg_hash.c:950<br>
#2 0xb6b05479 in dlg_onreply (t=0xb4bc5ce8, type=1048576,
param=0xbfef097c) at dlg_handlers.c:469<br>
#3 0xb6cc2336 in run_trans_callbacks_internal
(cb_lst=0xb4bc5d28, type=type@entry=1048576,
trans=0xb4bc5ce8, params=params@entry=0xbfef097c) at
t_hooks.c:290<br>
#4 0xb6cc25da in run_trans_callbacks_with_buf
(type=type@entry=1048576, rbuf=rbuf@entry=0xb4bc5d54,
req=req@entry=0xb4bc6a08, repl=repl@entry=0xffffffff,
flags=flags@entry=505) at t_hooks.c:336<br>
#5 0xb6cee953 in _reply_light
(trans=trans@entry=0xb4bc5ce8, <br>
buf=0xb731f588 "SIP/2.0 505 Error\r\nVia: SIP/2.0/UDP
192.168.21.4;branch=z9hG4bKa6f5.8ca73657.0\r\nVia:
SIP/2.0/UDP
10.0.0.95;rport=5060;branch=z9hG4bKBc143aFe81jDj\r\nFrom:
\"USER\" <<a moz-do-not-send="true"
href="mailto:sip%3AUSER@10.0.0.95">sip:USER@10.0.0.95</a>>;t"...,
len=len@entry=434, code=code@entry=505, lock=0,
bm=0xbfef0e48, to_tag_len=<error reading variable:
Unhandled dwarf expression opcode 0xfa>, <br>
to_tag=<error reading variable: Unhandled dwarf
expression opcode 0xfa>) at t_reply.c:628<br>
#6 0xb6ceedeb in _reply (trans=0xb4bc5ce8,
trans@entry=0x1f9, p_msg=p_msg@entry=0xb731f568, code=505,
text=0xb731f568 "Error", lock=0) at t_reply.c:782<br>
#7 0xb6ceefd7 in t_reply_unsafe (t=0x1f9,
t@entry=0xb4bc5ce8, p_msg=0xb731f568,
p_msg@entry=0xb6d196c0, code=0, text=text@entry=0xb731f568
"Error") at t_reply.c:1531<br>
#8 0xb6cd6904 in w_t_reply (msg=0xb6d196c0, p1=0xb731c988
"8]0\267\001", p2=0xb731c9b8 "8P0\267 ") at tm.c:1283<br>
#9 0x080651cf in do_action (h=h@entry=0xbfef1258,
a=a@entry=0xb7305d98, msg=msg@entry=0xb6d196c0) at
action.c:1080<br>
#10 0x08064137 in run_actions (h=h@entry=0xbfef1258,
a=a@entry=0xb73053c8, msg=msg@entry=0xb6d196c0) at
action.c:1573<br>
#11 0x0806cedd in run_top_route (a=0xb73053c8,
msg=msg@entry=0xb6d196c0, c=c@entry=0x0) at action.c:1658<br>
#12 0xb6ceba98 in run_failure_handlers
(t=t@entry=0xb4bc5ce8, rpl=0xffffffff, code=408,
extra_flags=96) at t_reply.c:1028<br>
#13 0xb6cece22 in t_should_relay_response
(Trans=Trans@entry=0xb4bc5ce8,
new_code=new_code@entry=408, branch=branch@entry=0,
should_store=should_store@entry=0xbfef1464, <br>
should_relay=should_relay@entry=0xbfef1460,
cancel_data=cancel_data@entry=0xbfef1510,
reply=reply@entry=0xffffffff) at t_reply.c:1304<br>
#14 0xb6cef076 in relay_reply (t=t@entry=0xb4bc5ce8,
p_msg=p_msg@entry=0xffffffff, branch=branch@entry=0,
msg_status=msg_status@entry=408,
cancel_data=cancel_data@entry=0xbfef1510, <br>
do_put_on_wait=do_put_on_wait@entry=0) at
t_reply.c:1707<br>
#15 0xb6cc28b4 in fake_reply (t=t@entry=0xb4bc5ce8,
branch=0, code=code@entry=408) at timer.c:354<br>
#16 0xb6cc34b0 in final_response_handler (t=0xb4bc5ce8,
r_buf=0xb4bc5dc8) at timer.c:526<br>
#17 retr_buf_handler (ticks=272914717, tl=0xb4bc5ddc,
p=0x3e8) at timer.c:584<br>
#18 0x08183325 in timer_list_expire (slow_mark=12,
slow_l=0xb49f2438, h=0xb49f2338, t=272914717) at
timer.c:894<br>
#19 timer_handler () at timer.c:959<br>
#20 timer_main () at timer.c:998<br>
#21 0x080b8c65 in main_loop () at main.c:1709<br>
#22 0x08063bfe in main (argc=1, argv=0xbfef1844) at
main.c:2566<br>
<br>
</div>
<div>(I have changed USER and IP address to hide my network
:))<br>
<br>
</div>
<div>After this when I hit 'n' multiple times I got
following output in infinite loop:<br>
<br>
22 0x08063bfe in main (argc=1, argv=0xbfef1844) at
main.c:2566<br>
(gdb) n<br>
next_state_dlg (dlg=dlg@entry=0xb4bc5a38,
event=event@entry=4, old_state=old_state@entry=0xbfef08a0,
new_state=new_state@entry=0xbfef089c,
unref=unref@entry=0xbfef08a4) at dlg_hash.c:952<br>
952 LM_DBG("dialog %p changed from state %d to "<br>
(gdb) <br>
955 }<br>
(gdb) <br>
dlg_onreply (t=0xb4bc5ce8, type=1048576, param=0xbfef097c)
at dlg_handlers.c:470<br>
470 dlg_run_event_route(dlg,
(rpl==FAKED_REPLY)?NULL:rpl, old_state, new_state);<br>
(gdb) <br>
472 if (new_state==DLG_STATE_EARLY) {<br>
(gdb) <br>
479 if (new_state==DLG_STATE_CONFIRMED_NA
&&<br>
(gdb) <br>
540 if ( new_state==DLG_STATE_DELETED<br>
(gdb) <br>
542 || old_state==DLG_STATE_EARLY)
) {<br>
(gdb) <br>
541 &&
(old_state==DLG_STATE_UNCONFIRMED<br>
(gdb) <br>
559 if (unref) dlg_unref(dlg, unref);<br>
(gdb) <br>
563 dlg_release(dlg);<br>
(gdb) <br>
565 }<br>
(gdb) <br>
run_trans_callbacks_internal (cb_lst=0xb4bc5d28,
type=type@entry=1048576, trans=0xb4bc5ce8,
params=params@entry=0xbfef097c) at t_hooks.c:292<br>
292 cbp=cbp->next;<br>
(gdb) <br>
283 while(cbp){<br>
(gdb) <br>
286 if ( (cbp->types)&type ) {<br>
(gdb) <br>
292 cbp=cbp->next;<br>
(gdb) <br>
283 while(cbp){<br>
(gdb) <br>
286 if ( (cbp->types)&type ) {<br>
(gdb) <br>
287 DBG("DBG: trans=%p, callback type %d,
id %d entered\n",<br>
(gdb) <br>
290 cbp->callback( trans, type, params
);<br>
(gdb) <br>
289 params->param =
&(cbp->param);<br>
(gdb) <br>
290 cbp->callback( trans, type, params
);<br>
(gdb) <br>
292 cbp=cbp->next;<br>
(gdb) <br>
283 while(cbp){<br>
(gdb) <br>
286 if ( (cbp->types)&type ) {<br>
(gdb) <br>
292 cbp=cbp->next;<br>
(gdb) <br>
283 while(cbp){<br>
(gdb) <br>
286 if ( (cbp->types)&type ) {<br>
(gdb) <br>
287 DBG("DBG: trans=%p, callback type %d,
id %d entered\n",<br>
(gdb) <br>
290 cbp->callback( trans, type, params
);<br>
(gdb) <br>
289 params->param =
&(cbp->param);<br>
(gdb) <br>
290 cbp->callback( trans, type, params
);<br>
(gdb) <br>
292 cbp=cbp->next;<br>
(gdb) <br>
283 while(cbp){<br>
<br>
</div>
<div>Regards<br>
<br>
</div>
<div>Efelin<br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">2013/11/20 Daniel-Constantin Mierla <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
<br>
I will investigate more -- meanwhile had some traveling.
It would speed up if you can send the backtrace of one
process that blocks when you applied the patch.<br>
<br>
You need to connect with gdb to it:<br>
<br>
gdb /path/to/kamailio _PID_<br>
<br>
replace _PID_ with the pid of blocked kamailio process.<br>
<br>
Cheers,<br>
Daniel
<div>
<div class="h5"><br>
<br>
<div>On 11/15/13 5:50 PM, Efelin Novak wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>Hi Daniel,<br>
<br>
</div>
thanks for a reply. I applied the patch
and now the Kamailio just prints <br>
<br>
WARNING: tm [t_lookup.c:1564]:
t_unref(): WARNING: script writer didn't
release transaction<br>
<br>
</div>
and than freezes without any log. It does
not resend the incoming "winning" failure
reply neither response to any other
messages, not even to a new calls. It just
freezes. All kamailio processes are
running and eating the whole 4-core
processor.<br>
<br>
</div>
Restart of Kamailio solves this problem.<br>
<br>
</div>
Any ideas how to continue with debug?<br>
<br>
</div>
<div>Thanks <br>
</div>
<div><br>
</div>
Efelin<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote"> 2013/11/15
Daniel-Constantin Mierla <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:miconda@gmail.com"
target="_blank">miconda@gmail.com</a>></span><br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hello,<br>
<br>
can you try attached patch?<br>
<br>
Let me know if all goes fine and I will
commit it to the repository.<br>
<br>
Cheers,<br>
Daniel
<div>
<div><br>
<br>
<div>On 11/15/13 10:25 AM, Efelin
Novak wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>
<div dir="ltr">
<div>
<div>Hi,<br>
<br>
</div>
when I use t_reply("505",
"Error"); in my failure route,
the response is not forwarded
and following is written into a
log:<br>
<br>
kamailio[26216]: WARNING: tm
[t_lookup.c:1559]: t_unref():
WARNING: script writer didn't
release transaction<br>
<br>
plus next line is written
exactly 416000 times into a log
afterwards:<br>
<br>
kamailio[32685]: CRITICAL:
dialog [dlg_hash.c:794]:
log_next_state_dlg(): bogus
event 4 in state 5 for dlg
0xb4af6588 [2575:7017] with clid
'121d44f0-6555f4c8' and tags
'd12546d053aadc68o2' ''<br>
<br>
</div>
<div>My point is to change the
incoming code from users and
append a Q.850 reason code.<br>
</div>
<div>Is there any other way how to
do this or a way how to fix
this?<br>
</div>
<div>I'm using Kamilio 4.0.4 on
Debian 7.1<br>
</div>
<div><br>
<br>
</div>
The code is as follows:<br>
failure_route[MANAGE_FAILURE]<br>
{<br>
if (t_is_canceled()) {<br>
exit;<br>
}<br>
if($T_reply_code == 408
&& isflagset(10))<br>
{<br>
xlog("Ringing
timeout");<br>
append_to_reply("Reason:
Q.850;cause=28\r\n");<br>
t_reply("505",
"Error");<br>
}<br>
}<br>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a moz-do-not-send="true" href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a moz-do-not-send="true" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><span><font color="#888888">
</font></span></pre>
<span><font color="#888888"> </font></span></blockquote>
<span><font color="#888888"> <br>
<pre cols="72">--
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28
- more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
</font></span></div>
<br>
_______________________________________________<br>
SIP Express Router (SER) and Kamailio
(OpenSER) - sr-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:sr-users@lists.sip-router.org"
target="_blank">sr-users@lists.sip-router.org</a><br>
<a moz-do-not-send="true"
href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users"
target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28
- more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla - <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://twitter.com/#%21/miconda">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a></pre>
</body>
</html>