<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    Hello,<br>
    <br>
    I added a patch to fix allocation of 0 size in dialog, can you give
    it a try:<br>
    <br>
    -
<a class="moz-txt-link-freetext" href="http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=082a6c43938cf8e3839d46fd070e391bd522d4ed">http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=082a6c43938cf8e3839d46fd070e391bd522d4ed</a><br>
    <br>
    It is rather small, should be easy to backport.<br>
    <br>
    I will look further in memory manager, but for now I thought worth
    fixing dialog. If the issues doesn't appear any longer, then I know
    where to look properly.<br>
    <br>
    Cheers,<br>
    Daniel<br>
    <br>
    <div class="moz-cite-prefix">On 11/4/13 11:52 PM, Daniel-Constantin
      Mierla wrote:<br>
    </div>
    <blockquote cite="mid:52782524.5090702@gmail.com" type="cite">
      <meta content="text/html; charset=ISO-8859-1"
        http-equiv="Content-Type">
      Hello,<br>
      <br>
      it might be related to the malloc of a size 0 done for one of the
      cseq value (which doesn't exist at the respective time). I need to
      investigate a bit more, I will come with updates as I get.<br>
      <br>
      Cheers,<br>
      Daniel<br>
      <br>
      <div class="moz-cite-prefix">On 10/29/13 10:33 AM, Morten Isaksen
        wrote:<br>
      </div>
      <blockquote
cite="mid:CAAyFOkH=MVoAJOf0+Vd7wFaEyYd9YMuxDf4Kr8rTss0KxS2tGQ@mail.gmail.com"
        type="cite">
        <div dir="ltr">
          <div>
            <div>Hi Daniel,<br>
              <br>
            </div>
            I still have the core file. You can find the output of p
            *shm_block here <a moz-do-not-send="true"
              href="http://pastebin.com/mgh4ed44">http://pastebin.com/mgh4ed44</a>
            - it was too big to poste on the list.<br>
            <br>
          </div>
          /Morten<br>
        </div>
        <div class="gmail_extra"><br>
          <br>
          <div class="gmail_quote">On Tue, Oct 29, 2013 at 9:56 AM,
            Daniel-Constantin Mierla <span dir="ltr"><<a
                moz-do-not-send="true" href="mailto:miconda@gmail.com"
                target="_blank">miconda@gmail.com</a>></span> wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
                <br>
                going to look over it. Do you still have the core file?
                If yes, provide the output of:<br>
                <br>
                p *shm_block<br>
                <br>
                from inside gdb.<br>
                <br>
                Cheers,<br>
                Daniel
                <div>
                  <div class="h5"><br>
                    <br>
                    <div>On 10/28/13 1:00 PM, Morten Isaksen wrote:<br>
                    </div>
                    <blockquote type="cite">
                      <div dir="ltr">
                        <div>Hi,<br>
                          <br>
                          We upgraded to 4.0.4 and now we got a new core
                          dump in q_malloc.c:269. It has been stable for
                          about 5 days after the upgrade.<br>
                          <br>
                        </div>
                        /Morten<br>
                        <div><br>
                          The full backtrace is here:<br>
                          <br>
                          Loaded symbols for /lib64/libnss_files.so.2<br>
                          Core was generated by
                          `/usr/local/sbin/kamailio -P
                          /var/run/kamailio/kamailio.pid -m 256 -M 8 -u
                          kamai'.<br>
                          Program terminated with signal 11,
                          Segmentation fault.<br>
                          #0  qm_detach_free (qm=0x7f5394de9000, size=0)
                          at mem/q_malloc.c:269<br>
                          269            
                          FRAG_END(next)->prev_free=prev;<br>
                          Missing separate debuginfos, use:
                          debuginfo-install glibc-2.12-1.107.el6.x86_64
                          hiredis-0.10.1-3.el6.x86_64
                          keyutils-libs-1.4-4.el6.x86_64
                          krb5-libs-1.10.3-10.el6.x86_64
                          libcom_err-1.41.12-14.el6.x86_64
                          libselinux-2.0.94-5.3.el6.x86_64
                          libxml2-2.7.6-12.el6_4.1.x86_64
                          mysql-libs-5.1.67-1.el6_3.x86_64
                          nss-softokn-freebl-3.12.9-11.el6.x86_64
                          openssl-1.0.0-27.el6_4.2.x86_64
                          zlib-1.2.3-29.el6.x86_64<br>
                          (gdb) bt full<br>
                          #0  qm_detach_free (qm=0x7f5394de9000, size=0)
                          at mem/q_malloc.c:269<br>
                                  prev = 0x7f539524ed60<br>
                                  next = 0x0<br>
                          #1  qm_malloc (qm=0x7f5394de9000, size=0) at
                          mem/q_malloc.c:386<br>
                                  f = 0x7f539524ed60<br>
                                  hash = 0<br>
                          #2  0x00007f53a5662950 in shm_malloc (size=0)
                          at ../../mem/shm_mem.h:262<br>
                                  p = <value optimized out><br>
                          #3  0x00007f53a566b49b in dlg_set_leg_info
                          (dlg=<value optimized out>,
                          tag=0x7ffff680b9a0, rr=0x7ffff680b860,
                          contact=0x7ffff680b870, cseq=0x7ffff680b880,
                          leg=1) at dlg_hash.c:498<br>
                                  p = <value optimized out><br>
                                  __FUNCTION__ = "dlg_set_leg_info"<br>
                          #4  0x00007f53a5659e88 in populate_leg_info
                          (dlg=0x7f53952312b0, msg=<value optimized
                          out>, t=<value optimized out>, leg=1,
                          tag=0x7ffff680b9a0) at dlg_handlers.c:276<br>
                                  skip_recs = 4<br>
                                  cseq = {s = 0x0, len = 0}<br>
                                  contact = {<br>
                                    s = 0x8da2c0 "<a
                            moz-do-not-send="true"
                            href="http://sip:65993777@178.21.249.22:5060"
                            target="_blank">sip:65993777@178.21.249.22:5060</a>>\r\nContent-Type:


                          application/sdp\r\nRequire:
                          timer\r\nContent-Length: 252\r\n\r\nv=0\r\no=-
                          <a moz-do-not-send="true"
                            href="tel:2042480911" value="+12042480911"
                            target="_blank">2042480911</a> <a
                            moz-do-not-send="true" href="tel:2042480911"
                            value="+12042480911" target="_blank">2042480911</a>
                          IN IP4 178.21.249.22\r\ns=Hosted PBX\r\nc=IN
                          IP4 178.21.249.22\r\nt=0 0"..., len = 31}<br>
                                  rr_set = {s = 0x7f53ab927c20 
                          <incomplete sequence \361\237\215>, len
                          = 0}<br>
                                  __FUNCTION__ = "populate_leg_info"<br>
                          #5  0x00007f53a565b949 in dlg_onreply
                          (t=0x7f5395245030, type=<value optimized
                          out>, param=<value optimized out>) at
                          dlg_handlers.c:500<br>
                                  dlg = 0x7f53952312b0<br>
                                  iuid = <value optimized out><br>
                                  new_state = 3<br>
                                  old_state = 1<br>
                                  unref = 0<br>
                                  event = 3<br>
                                  tag = {<br>
                                    s = 0x8da1bc "as77c791c9\r\nCall-ID:
                          0dd71e51-ba5d-1231-c79f-00095200eb1e\r\nCSeq:
                          1206140 INVITE\r\nServer:
                          one-connect\r\nAllow: INVITE, ACK, CANCEL,
                          OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO,
                          PUBLISH\r\nSupported: repl"..., len = 10}<br>
                                  req = 0x7f5395640090<br>
                                  rpl = 0x7f53ab8cd7a0<br>
                                  __FUNCTION__ = "dlg_onreply"<br>
                          #6  0x00007f53aa1f58ad in
                          run_trans_callbacks_internal (cb_lst=<value
                          optimized out>, type=1048576,
                          trans=0x7f5395245030, params=0x7ffff680baa0)
                          at t_hooks.c:290<br>
                                  cbp = 0x7f5395311260<br>
                                  backup_from = 0x8e9ed0<br>
                                  backup_to = 0x8e9ed8<br>
                                  backup_dom_from = 0x8e9ee0<br>
                                  backup_dom_to = 0x8e9ee8<br>
                                  backup_uri_from = 0x8e9ec0<br>
                                  backup_uri_to = 0x8e9ec8<br>
                                  backup_xavps = 0x8e9fd0<br>
                                  __FUNCTION__ =
                          "run_trans_callbacks_internal"<br>
                          #7  0x00007f53aa1f5a94 in
                          run_trans_callbacks_with_buf (type=<value
                          optimized out>, rbuf=0x7f53952450f0,
                          req=<value optimized out>,
                          repl=<value optimized out>,
                          flags=<value optimized out>) at
                          t_hooks.c:336<br>
                                  params = {req = 0x7f5395640090, rpl =
                          0x7f53ab8cd7a0, param = 0x7f5395311270, code =
                          200, flags = 200, branch = 0, t_rbuf =
                          0x7f53952450f0, dst = 0x7f5395245140, send_buf
                          = {<br>
                                      s = 0x7f53953ac460 "SIP/2.0 200
                          OK\r\nVia: SIP/2.0/UDP
                          178.21.248.30;rport=5060;branch=z9hG4bKd24f.32134635.0\r\nVia:
                          SIP/2.0/UDP
                          178.21.248.8;rport=5060;branch=z9hG4bKd24f.b20f4bb3.0\r\nVia:
                          SIP/2.0/UDP 178.21.248.20;branch=z"..., len =
                          1341}}<br>
                                  trans = <value optimized out><br>
                          #8  0x00007f53aa22728c in relay_reply
                          (t=<value optimized out>,
                          p_msg=<value optimized out>, branch=0,
                          msg_status=200, cancel_data=0x7ffff680be40,
                          do_put_on_wait=1) at t_reply.c:1867<br>
                                  relay = 0<br>
                                  save_clone = 0<br>
                                  buf = 0x7f53ab9361a0 "SIP/2.0 200
                          OK\r\nVia: SIP/2.0/UDP
                          178.21.248.30;rport=5060;branch=z9hG4bKd24f.32134635.0\r\nVia:
                          SIP/2.0/UDP
                          178.21.248.8;rport=5060;branch=z9hG4bKd24f.b20f4bb3.0\r\nVia:
                          SIP/2.0/UDP 178.21.248.20;branch=z"...<br>
                                  res_len = 1341<br>
                                  relayed_code = 200<br>
                                  relayed_msg = 0x7f53ab8cd7a0<br>
                                  reply_bak = <value optimized
                          out><br>
                                  bm = {to_tag_val = {s = 0x7f5395600c50
                          "P\233\365\224S\177", len = -1440728525}}<br>
                                  totag_retr = 0<br>
                                  reply_status = RPS_COMPLETED<br>
                                  uas_rb = <value optimized out><br>
                                  to_tag = <value optimized out><br>
                                  reason = {s = 0x3 <Address 0x3 out
                          of bounds>, len = -159334788}<br>
                                  onsend_params = {req = 0x7f53aa45e594,
                          rpl = 0x7ffff680be78, param = 0x0, code = 0,
                          flags = 0, branch = 0, t_rbuf = 0x10, dst =
                          0x7f53aa200a91, send_buf = {s = 0x7f5395601f38
                          "", len = -1440601726}}<br>
                          ---Type <return> to continue, or q
                          <return> to quit---<br>
                                  __FUNCTION__ = "relay_reply"<br>
                          #9  0x00007f53aa227da6 in reply_received
                          (p_msg=0x7f53ab8cd7a0) at t_reply.c:2374<br>
                                  msg_status = 200<br>
                                  last_uac_status = 100<br>
                                  ack = 0x1 <Address 0x1 out of
                          bounds><br>
                                  ack_len = 0<br>
                                  branch = 0<br>
                                  reply_status = <value optimized
                          out><br>
                                  onreply_route = <value optimized
                          out><br>
                                  cancel_data = {cancel_bitmap = 0,
                          reason = {cause = 200, u = {text = {s = 0x0,
                          len = -1416571248}, e2e_cancel = 0x0,
                          packed_hdrs = {s = 0x0, len = -1416571248}}}}<br>
                                  uac = 0x7f5395245198<br>
                                  t = 0x7f5395245030<br>
                                  lack_dst = {send_sock = 0x0, to = {s =
                          {sa_family = 48720, sa_data =
                          "\200\366\377\177\000\000\000\000\000\000\000\000\000"},
                          sin = {sin_family = 48720, sin_port = 63104,
                          sin_addr = {<br>
                                          s_addr = 32767}, sin_zero =
                          "\000\000\000\000\000\000\000"}, sin6 =
                          {sin6_family = 48720, sin6_port = 63104,
                          sin6_flowinfo = 32767, sin6_addr = {__in6_u =
                          {<br>
                                            __u6_addr8 =
                          "\000\000\000\000\000\000\000\000\260kq\253S\177\000",
                          __u6_addr16 = {0, 0, 0, 0, 27568, 43889,
                          32595, 0}, __u6_addr32 = {0, 0, 2876337072,
                          32595}}},<br>
                                        sin6_scope_id = 2878134176}}, id
                          = 32595, proto = -76 '\264', send_flags = {f =
                          5 '\005', blst_imask = 0 '\000'}}<br>
                                  backup_user_from = <value optimized
                          out><br>
                                  backup_user_to = <value optimized
                          out><br>
                                  backup_domain_from = <value
                          optimized out><br>
                                  backup_domain_to = <value optimized
                          out><br>
                                  backup_uri_from = <value optimized
                          out><br>
                                  backup_uri_to = <value optimized
                          out><br>
                                  backup_xavps = <value optimized
                          out><br>
                                  replies_locked = <value optimized
                          out><br>
                                  branch_ret = <value optimized
                          out><br>
                                  prev_branch = <value optimized
                          out><br>
                                  blst_503_timeout = <value optimized
                          out><br>
                                  hf = <value optimized out><br>
                                  onsend_params = {req = 0x8da424, rpl =
                          0x543274, param = 0x10, code = -1416833120,
                          flags = 32595, branch = 0, t_rbuf =
                          0x7ffff680be50, dst = 0x7f53a566690b, send_buf
                          = {<br>
                                      s = 0x4ff1e7b00000000 <Address
                          0x4ff1e7b00000000 out of bounds>, len = 0}}<br>
                                  ctx = {rec_lev = 0, run_flags = 0,
                          last_retcode = 1, jmp_env = {{__jmpbuf =
                          {140737329020144, -5577311537021373234,
                          139997336902320, 139997337147296, 200,
                          139996961202968, 5577292203380126926,<br>
                                          -5577312076254333746},
                          __mask_was_saved = 0, __saved_mask = {__val =
                          {139997337420128, 9281488, 139997335445253, 1,
                          140737329020496, 6191275, 5978073, 8591616,
                          9281252, 68728758110,<br>
                                            5548121, 9281656, 9281173,
                          139997337409168, 9282596, 139997337519904}}}}}<br>
                                  __FUNCTION__ = "reply_received"<br>
                          #10 0x0000000000456d54 in do_forward_reply
                          (msg=0x7f53ab8cd7a0, mode=<value optimized
                          out>) at forward.c:799<br>
                                  new_buf = 0x0<br>
                                  dst = {send_sock = 0x0, to = {s =
                          {sa_family = 0, sa_data = '\000' <repeats
                          13 times>}, sin = {sin_family = 0, sin_port
                          = 0, sin_addr = {s_addr = 0}, sin_zero =
                          "\000\000\000\000\000\000\000"},<br>
                                      sin6 = {sin6_family = 0, sin6_port
                          = 0, sin6_flowinfo = 0, sin6_addr = {__in6_u =
                          {__u6_addr8 = '\000' <repeats 15 times>,
                          __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0},
                          __u6_addr32 = {0, 0, 0, 0}}},<br>
                                        sin6_scope_id = 0}}, id = 0,
                          proto = 0 '\000', send_flags = {f = 0 '\000',
                          blst_imask = 0 '\000'}}<br>
                                  new_len = <value optimized out><br>
                                  r = <value optimized out><br>
                                  s = <value optimized out><br>
                                  len = <value optimized out><br>
                                  __FUNCTION__ = "do_forward_reply"<br>
                          #11 0x000000000049f4be in receive_msg (<br>
                              buf=0x8d9e80 "SIP/2.0 200 OK\r\nVia:
                          SIP/2.0/UDP
                          178.21.249.20;branch=z9hG4bKd24f.20eeb764.0;received=178.21.249.20;rport=5060\r\nVia:


                          SIP/2.0/UDP
                          178.21.248.30;rport=5060;branch=z9hG4bKd24f.32134635.0\r\nVia:
                          SIP/2.0/UD"..., len=<value optimized
                          out>, rcv_info=0x7ffff680c100) at
                          receive.c:272<br>
                                  msg = 0x7f53ab8cd7a0<br>
                                  ctx = {rec_lev = 12, run_flags = 0,
                          last_retcode = -1517756399, jmp_env =
                          {{__jmpbuf = {139996960369776, 12,
                          219309716216, 139997343567872,
                          140737329021424, 4294967295, 140737329021551,
                          1},<br>
                                        __mask_was_saved = 8581896,
                          __saved_mask = {__val = {0, 28, 16, 0,
                          219305533392, 1, 0, 139997335309296,
                          219309716216, 139996960369776,
                          139997337398864, 139997343565480,
                          139997337398872,<br>
                                            140737329021320,
                          219305559701, 140737329021192}}}}}<br>
                                  ret = <value optimized out><br>
                                  inb = {<br>
                                    s = 0x8d9e80 "SIP/2.0 200 OK\r\nVia:
                          SIP/2.0/UDP
                          178.21.249.20;branch=z9hG4bKd24f.20eeb764.0;received=178.21.249.20;rport=5060\r\nVia:


                          SIP/2.0/UDP
                          178.21.248.30;rport=5060;branch=z9hG4bKd24f.32134635.0\r\nVia:
                          SIP/2.0/UD"..., len = 1444}<br>
                                  __FUNCTION__ = "receive_msg"<br>
                          #12 0x0000000000532306 in udp_rcv_loop () at
                          udp_server.c:557<br>
                                  len = 1444<br>
                                  buf = "SIP/2.0 200 OK\r\nVia:
                          SIP/2.0/UDP
                          178.21.249.20;branch=z9hG4bKd24f.20eeb764.0;received=178.21.249.20;rport=5060\r\nVia:


                          SIP/2.0/UDP
                          178.21.248.30;rport=5060;branch=z9hG4bKd24f.32134635.0\r\nVia:
                          SIP/2.0/UD"...<br>
                                  from = 0x7f53ab90ae50<br>
                                  fromlen = 16<br>
                                  ri = {src_ip = {af = 2, len = 4, u =
                          {addrl = {385422770, 0}, addr32 = {385422770,
                          0, 0, 0}, addr16 = {5554, 5881, 0, 0, 0, 0, 0,
                          0}, addr = "\262\025\371\026", '\000'
                          <repeats 11 times>}},<br>
                          ---Type <return> to continue, or q
                          <return> to quit---<br>
                                    dst_ip = {af = 2, len = 4, u =
                          {addrl = {351868338, 0}, addr32 = {351868338,
                          0, 0, 0}, addr16 = {5554, 5369, 0, 0, 0, 0, 0,
                          0}, addr = "\262\025\371\024", '\000'
                          <repeats 11 times>}},<br>
                                    src_port = 5060, dst_port = 5060,
                          proto_reserved1 = 0, proto_reserved2 = 0,
                          src_su = {s = {sa_family = 2, sa_data =
                          "\023Ä&sup2;\025\371\026\000\000\000\000\000\000\000"},
                          sin = {sin_family = 2,<br>
                                        sin_port = 50195, sin_addr =
                          {s_addr = 385422770}, sin_zero =
                          "\000\000\000\000\000\000\000"}, sin6 =
                          {sin6_family = 2, sin6_port = 50195,
                          sin6_flowinfo = 385422770, sin6_addr =
                          {__in6_u = {<br>
                                            __u6_addr8 = '\000'
                          <repeats 15 times>, __u6_addr16 = {0, 0,
                          0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0,
                          0}}}, sin6_scope_id = 0}}, bind_address =
                          0x7f53ab8cec10, proto = 1 '\001'}<br>
                                  __FUNCTION__ = "udp_rcv_loop"<br>
                          #13 0x0000000000467a7a in main_loop () at
                          main.c:1638<br>
                                  i = <value optimized out><br>
                                  pid = <value optimized out><br>
                                  si = <value optimized out><br>
                                  si_desc = "udp receiver child=3 sock=<a
                            moz-do-not-send="true"
                            href="http://178.21.249.20:5060"
                            target="_blank">178.21.249.20:5060</a>\000\000\000\000@\\\220\253S\177\000\000\000\000\000\000\000\000\000\000\003\000\000\000\000\000\000\000\001\000\000\000\001\000\000\000\300\375\216\000\000\000\000\000\001\000\000\000\000\000\000\000\000\376\216\000\000\000\000\000\000\000\200\020",


                          '\000' <repeats 12 times>,
                          "\005\000\000\000\000\000\000"<br>
                                  nrprocs = <value optimized out><br>
                                  __FUNCTION__ = "main_loop"<br>
                          #14 0x000000000046a912 in main (argc=<value
                          optimized out>, argv=<value optimized
                          out>) at main.c:2566<br>
                                  cfg_stream = <value optimized
                          out><br>
                                  c = <value optimized out><br>
                                  r = <value optimized out><br>
                                  tmp = 0x7ffff680d89a ""<br>
                                  tmp_len = 0<br>
                                  options = 0x5c1de8
                          ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"<br>
                                  ret = -1<br>
                                  seed = 1043769636<br>
                                  rfd = <value optimized out><br>
                                  debug_save = <value optimized
                          out><br>
                                  debug_flag = <value optimized
                          out><br>
                                  dont_fork_cnt = <value optimized
                          out><br>
                                  n_lst = <value optimized out><br>
                                  p = <value optimized out><br>
                                  __FUNCTION__ = "main"<br>
                          <br>
                        </div>
                      </div>
                      <div class="gmail_extra"><br>
                        <br>
                        <div class="gmail_quote">On Mon, Oct 21, 2013 at
                          8:15 PM, Daniel-Constantin Mierla <span
                            dir="ltr"><<a moz-do-not-send="true"
                              href="mailto:miconda@gmail.com"
                              target="_blank">miconda@gmail.com</a>></span>
                          wrote:<br>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex">
                            <div bgcolor="#FFFFFF" text="#000000">
                              Hello,<br>
                              <br>
                              can you upgrade to 4.0.4? there were some
                              fixes related to new fields in sip message
                              structure that resulted in some out bound
                              memory inside transactions.<br>
                              <br>
                              The config file and database is the same,
                              so just re-install over the previous
                              installation.<br>
                              <br>
                              Cheers,<br>
                              Daniel
                              <div>
                                <div><br>
                                  <br>
                                  <div>On 10/21/13 11:10 AM, Morten
                                    Isaksen wrote:<br>
                                  </div>
                                </div>
                              </div>
                              <blockquote type="cite">
                                <div>
                                  <div>
                                    <div dir="ltr">
                                      <div>
                                        <div>
                                          <div>Hi,<br>
                                            <br>
                                          </div>
                                          Our Kamailio has stopped with
                                          a segmentation fault 4 times
                                          the last week.<br>
                                          <br>
                                        </div>
                                        There has been no changes to the
                                        configuration file the last 15
                                        days, so I suspect a SIP phone
                                        is sending a SIP packet that
                                        Kamailio does not like.<br>
                                        <br>
                                      </div>
                                      <div>We have a core dump file but
                                        I cannot read anything usefull
                                        from the backtrace. Can you see
                                        what is wrong from the
                                        backtrace?<br>
                                        <br>
                                      </div>
                                      <div>Regards<br>
                                      </div>
                                      <div>Morten<br>
                                      </div>
                                      <div><br>
                                      </div>
                                      The output from gdb (bt full):<br>
                                      <br>
                                      Core was generated by
                                      `/usr/local/sbin/kamailio -P
                                      /var/run/kamailio/kamailio.pid -m
                                      256 -M 8 -u kamai'.<br>
                                      Program terminated with signal 11,
                                      Segmentation fault.<br>
                                      #0  0x00007f0e10de17b2 in
                                      cancel_branch (t=0x7f0dfbf38e10,
                                      branch=0, reason=<value
                                      optimized out>, flags=4) at
                                      t_cancel.c:284<br>
                                      284             if (cfg_get(tm,
                                      tm_cfg, reparse_invite) ||<br>
                                      Missing separate debuginfos, use:
                                      debuginfo-install
                                      glibc-2.12-1.107.el6.x86_64
                                      hiredis-0.10.1-3.el6.x86_64
                                      keyutils-libs-1.4-4.el6.x86_64
                                      krb5-libs-1.10.3-10.el6.x86_64
                                      libcom_err-1.41.12-14.el6.x86_64
                                      libselinux-2.0.94-5.3.el6.x86_64
                                      libxml2-2.7.6-12.el6_4.1.x86_64
                                      mysql-libs-5.1.67-1.el6_3.x86_64
                                      nss-softokn-freebl-3.12.9-11.el6.x86_64
                                      openssl-1.0.0-27.el6_4.2.x86_64
                                      zlib-1.2.3-29.el6.x86_64<br>
                                      (gdb) bt full<br>
                                      #0  0x00007f0e10de17b2 in
                                      cancel_branch (t=0x7f0dfbf38e10,
                                      branch=0, reason=<value
                                      optimized out>, flags=4) at
                                      t_cancel.c:284<br>
                                              cancel = <value
                                      optimized out><br>
                                              len = <value optimized
                                      out><br>
                                              crb = 0x7f0dfbf39008<br>
                                              irb = 0x7f0dfbf38f80<br>
                                              ret = 1<br>
                                              tmp_cd = {cancel_bitmap =
                                      0, reason = {cause = 0, u = {text
                                      = {s = 0x0, len = 0}, e2e_cancel =
                                      0x0, packed_hdrs = {s = 0x0, len =
                                      0}}}}<br>
                                              pcbuf = <value
                                      optimized out><br>
                                              __FUNCTION__ =
                                      "cancel_branch"<br>
                                      #1  0x00007f0e10e298ab in
                                      reply_received
                                      (p_msg=0x7f0e124ce760) at
                                      t_reply.c:2194<br>
                                              msg_status = <value
                                      optimized out><br>
                                              last_uac_status = 408<br>
                                              ack = 0x7f0dfbf38e10 "Ð<br>
                                                                    
                                      \267\373\r\177"<br>
                                              ack_len = <value
                                      optimized out><br>
                                              branch = 0<br>
                                              reply_status = <value
                                      optimized out><br>
                                              onreply_route = <value
                                      optimized out><br>
                                              cancel_data =
                                      {cancel_bitmap = 0, reason =
                                      {cause = 408, u = {text = {s =
                                      0x0, len = 307468800}, e2e_cancel
                                      = 0x0, packed_hdrs = {s = 0x0, len
                                      = 307468800}}}}<br>
                                              uac = <value optimized
                                      out><br>
                                              t = 0x7f0dfbf38e10<br>
                                              lack_dst = {send_sock =
                                      0x0, to = {s = {sa_family = 6704,
                                      sa_data =
                                      "\\#\377\177\000\000\000\000\000\000\000\000\000"},
                                      sin = {sin_family = 6704, sin_port
                                      = 9052, sin_addr = {s_addr =
                                      32767},<br>
                                                    sin_zero =
                                      "\000\000\000\000\000\000\000"},
                                      sin6 = {sin6_family = 6704,
                                      sin6_port = 9052, sin6_flowinfo =
                                      32767, sin6_addr = {__in6_u = {<br>
                                                        __u6_addr8 =
                                      "\000\000\000\000\000\000\000\000p{1\022\016\177\000",
                                      __u6_addr16 = {0, 0, 0, 0, 31600,
                                      4657, 32526, 0}, __u6_addr32 = {0,
                                      0, 305232752, 32526}}},<br>
                                                    sin6_scope_id =
                                      307029856}}, id = 32526, proto =
                                      28 '\034', send_flags = {f = 0
                                      '\000', blst_imask = 0 '\000'}}<br>
                                              backup_user_from =
                                      <value optimized out><br>
                                              backup_user_to = <value
                                      optimized out><br>
                                              backup_domain_from =
                                      <value optimized out><br>
                                              backup_domain_to =
                                      <value optimized out><br>
                                              backup_uri_from =
                                      <value optimized out><br>
                                              backup_uri_to = <value
                                      optimized out><br>
                                              backup_xavps = <value
                                      optimized out><br>
                                              replies_locked = 0<br>
                                              branch_ret = <value
                                      optimized out><br>
                                              prev_branch = <value
                                      optimized out><br>
                                              blst_503_timeout =
                                      <value optimized out><br>
                                              hf = <value optimized
                                      out><br>
                                              onsend_params = {req =
                                      0x8d8a39, rpl = 0x541db4, param =
                                      0x7f0e1253e528, code = 307029856,
                                      flags = 32526, branch = 0, t_rbuf
                                      = 0x7fff235c1a30, dst =
                                      0x7f0e12317b70, send_buf = {<br>
                                                  s = 0x375311000000000
                                      <Address 0x375311000000000 out
                                      of bounds>, len = 0}}<br>
                                              ctx = {rec_lev =
                                      307491008, run_flags = 32526,
                                      last_retcode = 5674412, jmp_env =
                                      {{__jmpbuf = {140733786626256,
                                      63331951475841423,
                                      139698413054576, 139698413299552,
                                      9276465, 139698039855608,<br>
                                                     
                                      -63254168797292145,
                                      63332490682325391},
                                      __mask_was_saved = 0, __saved_mask
                                      = {__val = {139698413734928, 0,
                                      139698411522821, 1,
                                      140733786626608, 6185835, 5972697,
                                      8586176, 9275699,<br>
                                                        69026945952, 3,
                                      9276465, 9275673, 139698413738496,
                                      9275961, 139698413760704}}}}}<br>
                                              __FUNCTION__ =
                                      "reply_received"<br>
                                      #2  0x0000000000456444 in
                                      do_forward_reply
                                      (msg=0x7f0e124ce760,
                                      mode=<value optimized out>)
                                      at forward.c:799<br>
                                              new_buf = 0x0<br>
                                              dst = {send_sock = 0x0, to
                                      = {s = {sa_family = 0, sa_data =
                                      '\000' <repeats 13 times>},
                                      sin = {sin_family = 0, sin_port =
                                      0, sin_addr = {s_addr = 0},
                                      sin_zero =
                                      "\000\000\000\000\000\000\000"},<br>
                                                  sin6 = {sin6_family =
                                      0, sin6_port = 0, sin6_flowinfo =
                                      0, sin6_addr = {__in6_u =
                                      {__u6_addr8 = '\000' <repeats
                                      15 times>, __u6_addr16 = {0, 0,
                                      0, 0, 0, 0, 0, 0}, __u6_addr32 =
                                      {0, 0, 0, 0}}},<br>
                                                    sin6_scope_id = 0}},
                                      id = 0, proto = 0 '\000',
                                      send_flags = {f = 0 '\000',
                                      blst_imask = 0 '\000'}}<br>
                                              new_len = <value
                                      optimized out><br>
                                              r = <value optimized
                                      out><br>
                                              s = <value optimized
                                      out><br>
                                              len = <value optimized
                                      out><br>
                                              __FUNCTION__ =
                                      "do_forward_reply"<br>
                                      #3  0x000000000049e15e in
                                      receive_msg (buf=<value
                                      optimized out>, len=313,
                                      rcv_info=0x7fff235c1cd0) at
                                      receive.c:270<br>
                                              msg = 0x7f0e124ce760<br>
                                              ctx = {rec_lev = 11,
                                      run_flags = 0, last_retcode =
                                      206110737, jmp_env = {{__jmpbuf =
                                      {139698036884436, 11,
                                      219309716216, 139698419720192,
                                      140733786627520, 4294967295,
                                      140733786627647, 1},<br>
                                                    __mask_was_saved =
                                      8576456, __saved_mask = {__val =
                                      {0, 28, 16, 0, 219305533392, 1, 0,
                                      139698411461552, 219309716216,
                                      139698036884436, 139698413732672,
                                      139698419717800, 139698413732680,<br>
                                                        140733786627416,
                                      219305559701, 140733786627288}}}}}<br>
                                              ret = <value optimized
                                      out><br>
                                              inb = {<br>
                                                s = 0x8d8900 "SIP/2.0
                                      100 Trying\r\nVia: SIP/2.0/UDP
                                      178.21.249.20;branch=z9hG4bK8149.c6575a95.0\r\nTo:
                                      <a moz-do-not-send="true"
                                        href="mailto:sip%3A201@78799865.pbx.one-connect.dk"
                                        target="_blank">sip:201@78799865.pbx.one-connect.dk</a>;tag=07c44e68\r\nFrom:



                                      <a moz-do-not-send="true"
                                        href="mailto:sip%3A201@78799865.pbx.one-connect.dk"
                                        target="_blank">sip:201@78799865.pbx.one-connect.dk</a>;tag=a6a1c5f60faecf035a"...,



                                      len = 313}<br>
                                              __FUNCTION__ =
                                      "receive_msg"<br>
                                      #4  0x0000000000530e46 in
                                      udp_rcv_loop () at
                                      udp_server.c:557<br>
                                      ---Type <return> to
                                      continue, or q <return> to
                                      quit---<br>
                                              len = 313<br>
                                              buf = "SIP/2.0 100
                                      Trying\r\nVia: SIP/2.0/UDP
                                      178.21.249.20;branch=z9hG4bK8149.c6575a95.0\r\nTo:
                                      <a moz-do-not-send="true"
                                        href="mailto:sip%3A201@78799865.pbx.one-connect.dk"
                                        target="_blank">sip:201@78799865.pbx.one-connect.dk</a>;tag=07c44e68\r\nFrom:



                                      <a moz-do-not-send="true"
                                        href="mailto:sip%3A201@78799865.pbx.one-connect.dk"
                                        target="_blank">sip:201@78799865.pbx.one-connect.dk</a>;tag=a6a1c5f60faecf035a"...<br>
                                              from = 0x7f0e12538340<br>
                                              fromlen = 16<br>
                                              ri = {src_ip = {af = 2,
                                      len = 4, u = {addrl = {2993962576,
                                      0}, addr32 = {2993962576, 0, 0,
                                      0}, addr16 = {15952, 45684, 0, 0,
                                      0, 0, 0, 0}, addr = "P>t\262",
                                      '\000' <repeats 11 times>}},
                                      dst_ip = {<br>
                                                  af = 2, len = 4, u =
                                      {addrl = {351868338, 0}, addr32 =
                                      {351868338, 0, 0, 0}, addr16 =
                                      {5554, 5369, 0, 0, 0, 0, 0, 0},
                                      addr = "\262\025\371\024", '\000'
                                      <repeats 11 times>}},
                                      src_port = 35754,<br>
                                                dst_port = 5060,
                                      proto_reserved1 = 0,
                                      proto_reserved2 = 0, src_su = {s =
                                      {sa_family = 2, sa_data =
                                      "\213\252P>t\262\000\000\000\000\000\000\000"},
                                      sin = {sin_family = 2, sin_port =
                                      43659,<br>
                                                    sin_addr = {s_addr =
                                      2993962576}, sin_zero =
                                      "\000\000\000\000\000\000\000"},
                                      sin6 = {sin6_family = 2, sin6_port
                                      = 43659, sin6_flowinfo =
                                      2993962576, sin6_addr = {__in6_u =
                                      {<br>
                                                        __u6_addr8 =
                                      '\000' <repeats 15 times>,
                                      __u6_addr16 = {0, 0, 0, 0, 0, 0,
                                      0, 0}, __u6_addr32 = {0, 0, 0,
                                      0}}}, sin6_scope_id = 0}},
                                      bind_address = 0x7f0e124cfbd0,
                                      proto = 1 '\001'}<br>
                                              __FUNCTION__ =
                                      "udp_rcv_loop"<br>
                                      #5  0x000000000046716a in
                                      main_loop () at main.c:1638<br>
                                              i = <value optimized
                                      out><br>
                                              pid = <value optimized
                                      out><br>
                                              si = <value optimized
                                      out><br>
                                              si_desc = "udp receiver
                                      child=2 sock=<a
                                        moz-do-not-send="true"
                                        href="http://178.21.249.20:5060"
                                        target="_blank">178.21.249.20:5060</a>\000\000\000\000\200\303P\022\016\177\000\000\000\000\000\000\000\000\000\000\003\000\000\000\000\000\000\000\001\000\000\000\001\000\000\000@\350\216\000\000\000\000\000\001\000\000\000\000\000\000\000\200\350\216\000\000\000\000\000\000\000\200\020",



                                      '\000' <repeats 12 times>,
                                      "\005\000\000\000\000\000\000"<br>
                                              nrprocs = <value
                                      optimized out><br>
                                              __FUNCTION__ = "main_loop"<br>
                                      #6  0x000000000046a002 in main
                                      (argc=<value optimized out>,
                                      argv=<value optimized out>)
                                      at main.c:2566<br>
                                              cfg_stream = <value
                                      optimized out><br>
                                              c = <value optimized
                                      out><br>
                                              r = <value optimized
                                      out><br>
                                              tmp = 0x7fff235c377f ""<br>
                                              tmp_len = 0<br>
                                              options = 0x5c08c8
                                      ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"<br>
                                              ret = -1<br>
                                              seed = 1722854551<br>
                                              rfd = <value optimized
                                      out><br>
                                              debug_save = <value
                                      optimized out><br>
                                              debug_flag = <value
                                      optimized out><br>
                                              dont_fork_cnt = <value
                                      optimized out><br>
                                              n_lst = <value
                                      optimized out><br>
                                              p = <value optimized
                                      out><br>
                                              __FUNCTION__ = "main"<br>
                                      (gdb)<br>
                                      <br>
                                      <br>
                                      <br>
                                      <div>
                                        <div><br clear="all">
                                          <div>
                                            <div>
                                              <div><br>
                                                -- <br>
                                                Morten Isaksen<br>
                                              </div>
                                            </div>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                    <br>
                                    <fieldset></fieldset>
                                    <br>
                                  </div>
                                </div>
                                <pre>_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a moz-do-not-send="true" href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a moz-do-not-send="true" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><span><font color="#888888">
</font></span></pre>
                                <span><font color="#888888"> </font></span></blockquote>
                              <span><font color="#888888"> <br>
                                  <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
  - more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
                                </font></span></div>
                            <br>
_______________________________________________<br>
                            SIP Express Router (SER) and Kamailio
                            (OpenSER) - sr-users mailing list<br>
                            <a moz-do-not-send="true"
                              href="mailto:sr-users@lists.sip-router.org"
                              target="_blank">sr-users@lists.sip-router.org</a><br>
                            <a moz-do-not-send="true"
                              href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users"
                              target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
                            <br>
                          </blockquote>
                        </div>
                        <br>
                        <br clear="all">
                        <br>
                        -- <br>
                        Morten Isaksen<br>
                      </div>
                      <br>
                      <fieldset></fieldset>
                      <br>
                      <pre>_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a moz-do-not-send="true" href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a moz-do-not-send="true" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
                    </blockquote>
                    <br>
                    <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28
  - more details about Kamailio trainings at <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a> -
</pre>
                  </div>
                </div>
              </div>
              <br>
              _______________________________________________<br>
              SIP Express Router (SER) and Kamailio (OpenSER) - sr-users
              mailing list<br>
              <a moz-do-not-send="true"
                href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
              <a moz-do-not-send="true"
                href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users"
                target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
              <br>
            </blockquote>
          </div>
          <br>
          <br clear="all">
          <br>
          -- <br>
          Morten Isaksen<br>
        </div>
        <br>
        <fieldset class="mimeAttachmentHeader"></fieldset>
        <br>
        <pre wrap="">_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
      </blockquote>
      <br>
      <pre class="moz-signature" cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://twitter.com/#%21/miconda">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28
  - more details about Kamailio trainings at <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a> -
</pre>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Daniel-Constantin Mierla - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
Kamailio Advanced Trainings - Berlin, Nov 25-28
  - more details about Kamailio trainings at <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a> -
</pre>
  </body>
</html>