<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hello,<br>
<br>
this one is also from the cleanup process at shut down -- if you
look at the backtrace, if frames are executing shutdown_children(),
cleanup(), destory_modules()... then it is the shutdown. There
should be another core that is the reason for the runtime problem --
if there was something wrong with the memory structures at runtime,
then shutdown can end in it as well.<br>
<br>
Do you have other core files from where to extract the bt full?<br>
<br>
Just to rule it out, are you having any private modules or patches?<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<div class="moz-cite-prefix">On 11/09/15 16:03, Igor Potjevlesch
wrote:<br>
</div>
<blockquote cite="mid:008f01d0ec9a$958f9c80$c0aed580$@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"Préformaté HTML Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.PrformatHTMLCar
{mso-style-name:"Préformaté HTML Car";
mso-style-priority:99;
mso-style-link:"Préformaté HTML";
font-family:Consolas;
color:black;
mso-fareast-language:EN-US;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:#1F497D;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:windowtext;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Verdana",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Hello,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">I got two more crashes but they seem to be
different.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">The logs mentioned:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">kernel: kamailio[27096] general protection
ip:7f0b3e0bae55 sp:7fffdbce5fd0 error:0 in
dialog.so[7f0b3e077000+8f000]<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">/usr/local/sbin/kamailio[27056]: ALERT:
<core> [main.c:784]: handle_sigs(): child process
27096 exited by a signal 11<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Both gdb are related to dialog module:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">(gdb) bt full<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#0 0x00007fb6a8984c0e in
remove_dialog_timer_unsafe (tl=0x7fb6978e9060) at
dlg_timer.c:156<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">No locals.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#1 0x00007fb6a8985001 in remove_dialog_timer
(tl=0x7fb6978e9060) at dlg_timer.c:182<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "remove_dialog_timer"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#2 0x00007fb6a8966bb7 in destroy_dlg
(dlg=0x7fb6978e9008) at dlg_hash.c:357<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> ret = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> var = 0x7fb6976154b0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "destroy_dlg"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#3 0x00007fb6a8967b35 in destroy_dlg_table ()
at dlg_hash.c:438<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> dlg = 0xb02030a01201001<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> l_dlg = 0x7fb6978e9008<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> i = 2087<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "destroy_dlg_table"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#4 0x00007fb6a8933263 in mod_destroy () at
dialog.c:783<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">No locals.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#5 0x0000000000590d79 in destroy_modules () at
sr_module.c:811<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> t = 0x7fb6af43d670<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> foo = 0x7fb6af43d440<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "destroy_modules"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#6 0x000000000049bb43 in cleanup
(show_status=1) at main.c:569<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> memlog = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "cleanup"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#7 0x000000000049d10b in shutdown_children
(sig=15, show_status=1) at main.c:711<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "shutdown_children"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#8 0x000000000049f6e1 in handle_sigs () at
main.c:802<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> chld = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> chld_status = 139<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> memlog = -1755228944<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "handle_sigs"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#9 0x00000000004a6fbf in main_loop () at
main.c:1757<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> i = 8<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> pid = 4424<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> si = 0x0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> si_desc = "udp receiver child=7
sock=91.213.145.60:5060\000\000\000\000\016\b\000\000\377\177\000\000\260Ta\227\266\177\000\000\000\000\000\020\004\000\000\000\260Ta\227\266\177\000\000\060SA\000\000\000\000\000\240\177\207\b\001\000\000\000\060}\207\b\377\177\000\000\032dN\000\000\000\000\000h\261@\257z\000\000\000\276}p\000\000\000\000"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> nrprocs = 8<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "main_loop"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#10 0x00000000004ab8bf in main (argc=7,
argv=0x7fff08877fa8) at main.c:2578<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> cfg_stream = 0x18b4010<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> c = -1<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> r = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">tmp
= 0x7fff08879f70 ""<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">
tmp_len = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">
port = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">
proto = 32767<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">
options = 0x6fcc00
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">
</span><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">ret = -1<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> seed = 2249241156<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> rfd = 4<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> debug_save = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> debug_flag = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> dont_fork_cnt = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> n_lst = 0xc2<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> p = 0x7fff08877e7e ""<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "main"<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Igor.<o:p></o:p></span></p>
<p class="MsoNormal"><a moz-do-not-send="true"
name="_MailEndCompose"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></a></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="color:windowtext;mso-fareast-language:FR">De :</span></b><span
style="color:windowtext;mso-fareast-language:FR"> Igor
Potjevlesch [<a class="moz-txt-link-freetext" href="mailto:igor.potjevlesch@gmail.com">mailto:igor.potjevlesch@gmail.com</a>] <br>
<b>Envoyé :</b> jeudi 10 septembre 2015 16:43<br>
<b>À :</b> <a class="moz-txt-link-abbreviated" href="mailto:miconda@gmail.com">miconda@gmail.com</a>; 'Kamailio (SER) - Users
Mailing List' <a class="moz-txt-link-rfc2396E" href="mailto:sr-users@lists.sip-router.org"><sr-users@lists.sip-router.org></a><br>
<b>Objet :</b> RE: [SR-Users] Multiple crashes of
Kamailio 4.2.1<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Hello
Daniel,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">There is no particular reason. It's just that
our others Kamailio are running 4.2.3. So, we try to have
the same version on all nodes.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Regarding the backtrace, I'm surprised, because
there was no manual shutdown.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Unfortunately, for this occurrence of the
crash, it's the only coredump available. There was another
crash later this day with two coredumps and the following
backtrace:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">[…]<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Igor.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="color:windowtext;mso-fareast-language:FR">De :</span></b><span
style="color:windowtext;mso-fareast-language:FR">
sr-users [<a moz-do-not-send="true"
href="mailto:sr-users-bounces@lists.sip-router.org">mailto:sr-users-bounces@lists.sip-router.org</a>]
<b>De la part de</b> Daniel-Constantin Mierla<br>
<b>Envoyé :</b> jeudi 10 septembre 2015 12:57<br>
<b>À :</b> Kamailio (SER) - Users Mailing List <<a
moz-do-not-send="true"
href="mailto:sr-users@lists.sip-router.org"><a class="moz-txt-link-abbreviated" href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a></a>><br>
<b>Objet :</b> Re: [SR-Users] Multiple crashes of
Kamailio 4.2.1<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hello,<br>
<br>
there are newer releases in branch 4.2, was there any reason
to stop at 4.2.3?<br>
<br>
Back to the backtrace, this one is from shutdown, happening
during the cleanup, not showing the reason of the crash at
runtime, unless there was a manual shut down triggered at that
moment. Do you have another core file? <br>
<br>
Cheers,<br>
Daniel<span style="font-size:12.0pt;mso-fareast-language:FR"><o:p></o:p></span></p>
<div>
<p class="MsoNormal">On 10/09/15 12:36, Igor Potjevlesch
wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Hello,</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">We got some others crashes even after updated
to 4.2.3.</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">From the last one, I got the following into
"bt full":</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Core was generated by
`/usr/local/sbin/kamailio -P /var/run/kamailio.pid -m 256
-M 64'.</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Program terminated with signal 11,
Segmentation fault.</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#0 0x0000000000617612 in fm_insert_free
(qm=0x7f7d4206d000, frag=0x7f7d422ea368) at
mem/f_malloc.c:245</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">245 if (*f) (*f)->prv_free =
&(frag->u.nxt_free);</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Missing separate debuginfos, use:
debuginfo-install bzip2-libs-1.0.5-7.el6_0.x86_64
db4-4.7.25-18.el6_4.x86_64
elfutils-libelf-0.152-1.el6.x86_64
glibc-2.12-1.132.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64
krb5-libs-1.10.3-10.el6_4.6.x86_64
libacl-2.2.49-6.el6.x86_64 libattr-2.4.44-7.el6.x86_64
libcap-2.16-5.5.el6.x86_64
libcom_err-1.41.12-18.el6.x86_64
libgcc-4.4.7-11.el6.x86_64
libselinux-2.0.94-5.3.el6_4.1.x86_64
lm_sensors-libs-3.1.1-17.el6.x86_64
lua-5.1.4-4.1.el6.x86_64 mysql-libs-5.1.73-3.el6_5.x86_64
net-snmp-libs-5.5-50.el6_6.1.x86_64
nspr-4.10.0-1.el6.x86_64 nss-3.15.1-15.el6.x86_64
nss-softokn-freebl-3.14.3-9.el6.x86_64
nss-util-3.15.1-3.el6.x86_64
openssl-1.0.1e-30.el6_6.4.x86_64 pcre-7.8-6.el6.x86_64
perl-libs-5.10.1-136.el6.x86_64 popt-1.13-7.el6.x86_64
rpm-libs-4.8.0-37.el6.x86_64
tcp_wrappers-libs-7.6-57.el6.x86_64
xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64
zlib-1.2.3-29.el6.x86_64</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">(gdb) bt full</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#0 0x0000000000617612 in fm_insert_free
(qm=0x7f7d4206d000, frag=0x7f7d422ea368) at
mem/f_malloc.c:245</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> f = 0x7f7d4206d188</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> hash = 4</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#1 0x000000000061a2fe in fm_free
(qm=0x7f7d4206d000, p=0x7f7d422ea3a0, file=0x7f7d53408ff0
"dialog: dlg_timer.c", func=0x7f7d53409d00
"destroy_dlg_timer", line=95)</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> at mem/f_malloc.c:614</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> f = 0x7f7d422ea368</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "fm_free"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#2 0x00007f7d533e441f in destroy_dlg_timer
() at dlg_timer.c:95</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "destroy_dlg_timer"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#3 0x00007f7d53393268 in mod_destroy () at
dialog.c:784</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">No locals.</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#4 0x0000000000590d79 in destroy_modules ()
at sr_module.c:811</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> t = 0x7f7d59e9d670</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> foo = 0x7f7d59e9d440</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "destroy_modules"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#5 0x000000000049bb43 in cleanup
(show_status=1) at main.c:569</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> memlog = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "cleanup"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#6 0x000000000049d10b in shutdown_children
(sig=15, show_status=1) at main.c:711</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "shutdown_children"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#7 0x000000000049f6e1 in handle_sigs () at
main.c:802</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> chld = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> chld_status = 139</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> memlog = 1107776752</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "handle_sigs"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#8 0x00000000004a6fbf in main_loop () at
main.c:1757</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> i = 8</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> pid = 4147</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> si = 0x0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> si_desc = "udp receiver child=7
sock=91.213.145.60:5060\000\000\000\000\016\b\000\000\377\177\000\000\260T\aB}\177\000\000\000\000\000\020\004\000\000\000\260T\aB}\177\000\000\060SA\000\000\000\000\000\260:\240\343\001\000\000\000@8\240\343\377\177\000\000\032dN\000\000\000\000\000h\261\346Yz\000\000\000\276}p\000\000\000\000"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> nrprocs = 8</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "main_loop"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">#9 0x00000000004ab8bf in main (argc=7,
argv=0x7fffe3a03ab8) at main.c:2578</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> cfg_stream = 0x1a04010</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> c = -1</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> r = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> tmp = 0x7fffe3a04f70 ""</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> tmp_len = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> port = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> proto = 32767</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> options = 0x6fcc00
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> ret = -1</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> seed = 3320376299</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> rfd = 4</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> debug_save = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> debug_flag = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> dont_fork_cnt = 0</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> n_lst = 0xc2</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> p = 0x7fffe3a0398e ""</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> __FUNCTION__ = "main"</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB">Igor.</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#1F497D"
lang="EN-GB"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="mso-fareast-language:FR">De :</span></b><span
style="mso-fareast-language:FR"> Igor Potjevlesch [<a
moz-do-not-send="true"
href="mailto:igor.potjevlesch@gmail.com"><a class="moz-txt-link-freetext" href="mailto:igor.potjevlesch@gmail.com">mailto:igor.potjevlesch@gmail.com</a></a>]
<br>
<b>Envoyé :</b> lundi 20 juillet 2015 16:22<br>
<b>À :</b> <a moz-do-not-send="true"
href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
<b>Objet :</b> Multiple crashes of Kamailio 4.2.1</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">Hello,</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">3 crashes occurred today (we never seen
crashes on this server before) during a period of 20
minutes.</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">I had a look to the coredumps. Here is a
quick overview of the first entries of "bt full":</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.27671 : #0 0x00007f5577048d65 in
run_trans_callbacks_internal (cb_lst=0x7f5560abbd50,
type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at
t_hooks.c:286</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.27656 : #0 0x00007f557700a048 in
free_cell (dead_cell=0x7f5560abbce0) at h_table.c:175</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.27685 : #0 0x00007f5577048d65 in
run_trans_callbacks_internal (cb_lst=0x7f5560abbd50,
type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at
t_hooks.c:286</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.27689 : #0 0x00007f5577048d65 in
run_trans_callbacks_internal (cb_lst=0x7f5560abbd50,
type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at
t_hooks.c:286</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.27691 : #0 0x00007f5577048d65 in
run_trans_callbacks_internal (cb_lst=0x7f5560abbd50,
type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at
t_hooks.c:286</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.31977 : #0 0x00007f038e66fdee in
free_cell (dead_cell=0x7f037811ebe0) at h_table.c:157</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.31994 : #0 0x00007f038e6ab65a in
t_forward_nonack (t=0x7f037811ebe0, p_msg=0x7f038fdca438,
proxy=0x0, proto=0) at t_fwd.c:1762</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.3700 : #0 0x000000000061697a in
fm_insert_free (qm=0x7f19861fa000, frag=0x7f19864d53e8) at
mem/f_malloc.c:245</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">core.3727 : #0 0x00000000006167a4 in
fm_extract_free (qm=0x7f19861fa000, frag=0x7f19864f4cc8)
at mem/f_malloc.c:208</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">I don't know how to proceed now. Can you help
me to highlight the issue here? Thank you.<br>
<br>
Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:9.0pt;font-family:"Verdana",sans-serif"
lang="EN-GB">Igor.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif;mso-fareast-language:FR"><br>
<br>
<o:p></o:p></span></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif;mso-fareast-language:FR"><o:p> </o:p></span></p>
<pre>-- <o:p></o:p></pre>
<pre>Daniel-Constantin Mierla<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://twitter.com/#%21/miconda">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a><o:p></o:p></pre>
<pre>Book: SIP Routing With Kamailio - <a moz-do-not-send="true" href="http://www.asipto.com">http://www.asipto.com</a><o:p></o:p></pre>
<pre>Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - <a moz-do-not-send="true" href="http://asipto.com/u/kat">http://asipto.com/u/kat</a><o:p></o:p></pre>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
Book: SIP Routing With Kamailio - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - <a class="moz-txt-link-freetext" href="http://asipto.com/u/kat">http://asipto.com/u/kat</a></pre>
</body>
</html>