<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    Hello,<br>
    <br>
    the backtrace is ok for the moment. I will look over it and come
    back with results.<br>
    <br>
    Cheers,<br>
    Daniel<br>
    <br>
    <div class="moz-cite-prefix">On 11/25/13 12:10 PM, Efelin Novak
      wrote:<br>
    </div>
    <blockquote
cite="mid:CABKTgArt2i=Sk38RTeZjUKs===3YPYh_boMb6jjy_LrSdDYhaA@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div>
          <div>Hi Daniel,<br>
            <br>
          </div>
          sorry it took me more than I expected. Is this sufficient?
          Meanwhile I found out that this happens when fr_inv_timer
          triggers and dialog module is turned on.<br>
          <br>
        </div>
        Backtrace:<br>
        <div>
          <div><br>
            bt<br>
            #0  futex_release (lock=0xb4b90798) at
            ../../mem/../futexlock.h:137<br>
            #1  next_state_dlg (dlg=dlg@entry=0xb4bc5a38,
            event=event@entry=4, old_state=old_state@entry=0xbfef08a0,
            new_state=new_state@entry=0xbfef089c,
            unref=unref@entry=0xbfef08a4) at dlg_hash.c:950<br>
            #2  0xb6b05479 in dlg_onreply (t=0xb4bc5ce8, type=1048576,
            param=0xbfef097c) at dlg_handlers.c:469<br>
            #3  0xb6cc2336 in run_trans_callbacks_internal
            (cb_lst=0xb4bc5d28, type=type@entry=1048576,
            trans=0xb4bc5ce8, params=params@entry=0xbfef097c) at
            t_hooks.c:290<br>
            #4  0xb6cc25da in run_trans_callbacks_with_buf
            (type=type@entry=1048576, rbuf=rbuf@entry=0xb4bc5d54,
            req=req@entry=0xb4bc6a08, repl=repl@entry=0xffffffff,
            flags=flags@entry=505) at t_hooks.c:336<br>
            #5  0xb6cee953 in _reply_light
            (trans=trans@entry=0xb4bc5ce8, <br>
                buf=0xb731f588 "SIP/2.0 505 Error\r\nVia: SIP/2.0/UDP
            192.168.21.4;branch=z9hG4bKa6f5.8ca73657.0\r\nVia:
            SIP/2.0/UDP
            10.0.0.95;rport=5060;branch=z9hG4bKBc143aFe81jDj\r\nFrom:
            \"USER\" <<a moz-do-not-send="true"
              href="mailto:sip%3AUSER@10.0.0.95">sip:USER@10.0.0.95</a>>;t"...,
            len=len@entry=434, code=code@entry=505, lock=0,
            bm=0xbfef0e48, to_tag_len=<error reading variable:
            Unhandled dwarf expression opcode 0xfa>, <br>
                to_tag=<error reading variable: Unhandled dwarf
            expression opcode 0xfa>) at t_reply.c:628<br>
            #6  0xb6ceedeb in _reply (trans=0xb4bc5ce8,
            trans@entry=0x1f9, p_msg=p_msg@entry=0xb731f568, code=505,
            text=0xb731f568 "Error", lock=0) at t_reply.c:782<br>
            #7  0xb6ceefd7 in t_reply_unsafe (t=0x1f9,
            t@entry=0xb4bc5ce8, p_msg=0xb731f568,
            p_msg@entry=0xb6d196c0, code=0, text=text@entry=0xb731f568
            "Error") at t_reply.c:1531<br>
            #8  0xb6cd6904 in w_t_reply (msg=0xb6d196c0, p1=0xb731c988
            "8]0\267\001", p2=0xb731c9b8 "8P0\267 ") at tm.c:1283<br>
            #9  0x080651cf in do_action (h=h@entry=0xbfef1258,
            a=a@entry=0xb7305d98, msg=msg@entry=0xb6d196c0) at
            action.c:1080<br>
            #10 0x08064137 in run_actions (h=h@entry=0xbfef1258,
            a=a@entry=0xb73053c8, msg=msg@entry=0xb6d196c0) at
            action.c:1573<br>
            #11 0x0806cedd in run_top_route (a=0xb73053c8,
            msg=msg@entry=0xb6d196c0, c=c@entry=0x0) at action.c:1658<br>
            #12 0xb6ceba98 in run_failure_handlers
            (t=t@entry=0xb4bc5ce8, rpl=0xffffffff, code=408,
            extra_flags=96) at t_reply.c:1028<br>
            #13 0xb6cece22 in t_should_relay_response
            (Trans=Trans@entry=0xb4bc5ce8, new_code=new_code@entry=408,
            branch=branch@entry=0,
            should_store=should_store@entry=0xbfef1464, <br>
                should_relay=should_relay@entry=0xbfef1460,
            cancel_data=cancel_data@entry=0xbfef1510,
            reply=reply@entry=0xffffffff) at t_reply.c:1304<br>
            #14 0xb6cef076 in relay_reply (t=t@entry=0xb4bc5ce8,
            p_msg=p_msg@entry=0xffffffff, branch=branch@entry=0,
            msg_status=msg_status@entry=408,
            cancel_data=cancel_data@entry=0xbfef1510, <br>
                do_put_on_wait=do_put_on_wait@entry=0) at t_reply.c:1707<br>
            #15 0xb6cc28b4 in fake_reply (t=t@entry=0xb4bc5ce8,
            branch=0, code=code@entry=408) at timer.c:354<br>
            #16 0xb6cc34b0 in final_response_handler (t=0xb4bc5ce8,
            r_buf=0xb4bc5dc8) at timer.c:526<br>
            #17 retr_buf_handler (ticks=272914717, tl=0xb4bc5ddc,
            p=0x3e8) at timer.c:584<br>
            #18 0x08183325 in timer_list_expire (slow_mark=12,
            slow_l=0xb49f2438, h=0xb49f2338, t=272914717) at timer.c:894<br>
            #19 timer_handler () at timer.c:959<br>
            #20 timer_main () at timer.c:998<br>
            #21 0x080b8c65 in main_loop () at main.c:1709<br>
            #22 0x08063bfe in main (argc=1, argv=0xbfef1844) at
            main.c:2566<br>
            <br>
          </div>
          <div>(I have changed  USER and IP address to hide my network
            :))<br>
            <br>
          </div>
          <div>After this when I hit 'n' multiple times I got following
            output in infinite loop:<br>
            <br>
            22 0x08063bfe in main (argc=1, argv=0xbfef1844) at
            main.c:2566<br>
            (gdb) n<br>
            next_state_dlg (dlg=dlg@entry=0xb4bc5a38,
            event=event@entry=4, old_state=old_state@entry=0xbfef08a0,
            new_state=new_state@entry=0xbfef089c,
            unref=unref@entry=0xbfef08a4) at dlg_hash.c:952<br>
            952        LM_DBG("dialog %p changed from state %d to "<br>
            (gdb) <br>
            955    }<br>
            (gdb) <br>
            dlg_onreply (t=0xb4bc5ce8, type=1048576, param=0xbfef097c)
            at dlg_handlers.c:470<br>
            470        dlg_run_event_route(dlg,
            (rpl==FAKED_REPLY)?NULL:rpl, old_state, new_state);<br>
            (gdb) <br>
            472        if (new_state==DLG_STATE_EARLY) {<br>
            (gdb) <br>
            479        if (new_state==DLG_STATE_CONFIRMED_NA &&<br>
            (gdb) <br>
            540        if ( new_state==DLG_STATE_DELETED<br>
            (gdb) <br>
            542                        || old_state==DLG_STATE_EARLY) )
            {<br>
            (gdb) <br>
            541                    &&
            (old_state==DLG_STATE_UNCONFIRMED<br>
            (gdb) <br>
            559        if (unref) dlg_unref(dlg, unref);<br>
            (gdb) <br>
            563        dlg_release(dlg);<br>
            (gdb) <br>
            565    }<br>
            (gdb) <br>
            run_trans_callbacks_internal (cb_lst=0xb4bc5d28,
            type=type@entry=1048576, trans=0xb4bc5ce8,
            params=params@entry=0xbfef097c) at t_hooks.c:292<br>
            292            cbp=cbp->next;<br>
            (gdb) <br>
            283        while(cbp){<br>
            (gdb) <br>
            286            if ( (cbp->types)&type ) {<br>
            (gdb) <br>
            292            cbp=cbp->next;<br>
            (gdb) <br>
            283        while(cbp){<br>
            (gdb) <br>
            286            if ( (cbp->types)&type ) {<br>
            (gdb) <br>
            287                DBG("DBG: trans=%p, callback type %d, id
            %d entered\n",<br>
            (gdb) <br>
            290                cbp->callback( trans, type, params );<br>
            (gdb) <br>
            289                params->param = &(cbp->param);<br>
            (gdb) <br>
            290                cbp->callback( trans, type, params );<br>
            (gdb) <br>
            292            cbp=cbp->next;<br>
            (gdb) <br>
            283        while(cbp){<br>
            (gdb) <br>
            286            if ( (cbp->types)&type ) {<br>
            (gdb) <br>
            292            cbp=cbp->next;<br>
            (gdb) <br>
            283        while(cbp){<br>
            (gdb) <br>
            286            if ( (cbp->types)&type ) {<br>
            (gdb) <br>
            287                DBG("DBG: trans=%p, callback type %d, id
            %d entered\n",<br>
            (gdb) <br>
            290                cbp->callback( trans, type, params );<br>
            (gdb) <br>
            289                params->param = &(cbp->param);<br>
            (gdb) <br>
            290                cbp->callback( trans, type, params );<br>
            (gdb) <br>
            292            cbp=cbp->next;<br>
            (gdb) <br>
            283        while(cbp){<br>
            <br>
          </div>
          <div>Regards<br>
            <br>
          </div>
          <div>Efelin<br>
          </div>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <br>
        <div class="gmail_quote">2013/11/20 Daniel-Constantin Mierla <span
            dir="ltr"><<a moz-do-not-send="true"
              href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span><br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
              <br>
              I will investigate more -- meanwhile had some traveling.
              It would speed up if you can send the backtrace of one
              process that blocks when you applied the patch.<br>
              <br>
              You need to connect with gdb to it:<br>
              <br>
              gdb /path/to/kamailio _PID_<br>
              <br>
              replace _PID_ with the pid of blocked kamailio process.<br>
              <br>
              Cheers,<br>
              Daniel
              <div>
                <div class="h5"><br>
                  <br>
                  <div>On 11/15/13 5:50 PM, Efelin Novak wrote:<br>
                  </div>
                  <blockquote type="cite">
                    <div dir="ltr">
                      <div>
                        <div>
                          <div>
                            <div>
                              <div>Hi Daniel,<br>
                                <br>
                              </div>
                              thanks for a reply. I applied the patch
                              and now the Kamailio just prints <br>
                              <br>
                              WARNING: tm [t_lookup.c:1564]: t_unref():
                              WARNING: script writer didn't release
                              transaction<br>
                              <br>
                            </div>
                            and than freezes without any log. It does
                            not resend the incoming "winning" failure
                            reply neither response to any other
                            messages, not even to a new calls. It just
                            freezes. All kamailio processes are running
                            and eating the whole 4-core processor.<br>
                            <br>
                          </div>
                          Restart of Kamailio solves this problem.<br>
                          <br>
                        </div>
                        Any ideas how to continue with debug?<br>
                        <br>
                      </div>
                      <div>Thanks <br>
                      </div>
                      <div><br>
                      </div>
                      Efelin<br>
                    </div>
                    <div class="gmail_extra"><br>
                      <br>
                      <div class="gmail_quote"> 2013/11/15
                        Daniel-Constantin Mierla <span dir="ltr"><<a
                            moz-do-not-send="true"
                            href="mailto:miconda@gmail.com"
                            target="_blank">miconda@gmail.com</a>></span><br>
                        <blockquote class="gmail_quote" style="margin:0
                          0 0 .8ex;border-left:1px #ccc
                          solid;padding-left:1ex">
                          <div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
                            <br>
                            can you try attached patch?<br>
                            <br>
                            Let me know if all goes fine and I will
                            commit it to the repository.<br>
                            <br>
                            Cheers,<br>
                            Daniel
                            <div>
                              <div><br>
                                <br>
                                <div>On 11/15/13 10:25 AM, Efelin Novak
                                  wrote:<br>
                                </div>
                              </div>
                            </div>
                            <blockquote type="cite">
                              <div>
                                <div>
                                  <div dir="ltr">
                                    <div>
                                      <div>Hi,<br>
                                        <br>
                                      </div>
                                      when I use t_reply("505",
                                      "Error"); in my failure route, the
                                      response is not forwarded and
                                      following is written into a log:<br>
                                      <br>
                                      kamailio[26216]: WARNING: tm
                                      [t_lookup.c:1559]: t_unref():
                                      WARNING: script writer didn't
                                      release transaction<br>
                                      <br>
                                      plus next line is written exactly
                                      416000 times into a log
                                      afterwards:<br>
                                      <br>
                                      kamailio[32685]: CRITICAL: dialog
                                      [dlg_hash.c:794]:
                                      log_next_state_dlg(): bogus event
                                      4 in state 5 for dlg 0xb4af6588
                                      [2575:7017] with clid
                                      '121d44f0-6555f4c8' and tags
                                      'd12546d053aadc68o2' ''<br>
                                      <br>
                                    </div>
                                    <div>My point is to change the
                                      incoming code from users and
                                      append a Q.850 reason code.<br>
                                    </div>
                                    <div>Is there any other way how to
                                      do this or a way how to fix this?<br>
                                    </div>
                                    <div>I'm using Kamilio 4.0.4 on
                                      Debian 7.1<br>
                                    </div>
                                    <div><br>
                                      <br>
                                    </div>
                                    The code is as follows:<br>
                                    failure_route[MANAGE_FAILURE]<br>
                                    {<br>
                                            if (t_is_canceled()) {<br>
                                                    exit;<br>
                                            }<br>
                                            if($T_reply_code == 408
                                    && isflagset(10))<br>
                                            {<br>
                                                    xlog("Ringing
                                    timeout");<br>
                                                   
                                    append_to_reply("Reason:
                                    Q.850;cause=28\r\n");<br>
                                                    t_reply("505",
                                    "Error");<br>
                                            }<br>
                                    }<br>
                                  </div>
                                  <br>
                                  <fieldset></fieldset>
                                  <br>
                                </div>
                              </div>
                              <pre>_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a moz-do-not-send="true" href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a moz-do-not-send="true" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><span><font color="#888888">
</font></span></pre>
                              <span><font color="#888888"> </font></span></blockquote>
                            <span><font color="#888888"> <br>
                                <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28
  - more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
                              </font></span></div>
                          <br>
_______________________________________________<br>
                          SIP Express Router (SER) and Kamailio
                          (OpenSER) - sr-users mailing list<br>
                          <a moz-do-not-send="true"
                            href="mailto:sr-users@lists.sip-router.org"
                            target="_blank">sr-users@lists.sip-router.org</a><br>
                          <a moz-do-not-send="true"
                            href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users"
                            target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
                          <br>
                        </blockquote>
                      </div>
                      <br>
                    </div>
                  </blockquote>
                  <br>
                  <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28
  - more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Daniel-Constantin Mierla - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a></pre>
  </body>
</html>