<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hello,<br>
<br>
the backtrace shows a crash in tm module, not sqlops.<br>
<br>
Can you tell which of the log files correspond to the instance that
produced the core file from where you took the back trace you
attached?<br>
<br>
Can you give the backtraces from all cores? You say there were 3
crashes. It is important to see if the backtrace is the same
everywhere.<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<div class="moz-cite-prefix">On 09/01/14 13:28, Uri Shacked wrote:<br>
</div>
<blockquote
cite="mid:CAMMbDhRmMH=iKQHABArxP9NA=5HzeB1uAsBMGhn7Ajd0mkvAeA@mail.gmail.com"
type="cite">
<div dir="ltr">
<p class="MsoNormal" style="line-height:150%"><font
face="courier new, monospace"><span style="line-height:150%"
lang="HE">Hi, </span><span style="line-height:normal"> </span></font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">Our system had crashed again.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">Here are the exact details and there are
logs and corefile attached.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">We use 6 kamailio servers over 3 sites.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">All the servers are connecting to 1 DB
server running MYSQL 5.6.12.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">The sql server supplies 2 things for
kamailio servers.</font></p>
<p class="" style="margin:0cm 0cm 0.0001pt 36pt;direction:ltr"><font
face="courier new, monospace">1.
<span dir="LTR"></span>Some SQLOPS
that the servers need.</font></p>
<p class="" style="margin:0cm 0cm 0.0001pt 36pt;direction:ltr"><font
face="courier new, monospace">2.
<span dir="LTR"></span>The
data for the MTREE, HTABLE, CARRIERROUTE etc..</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">We use "kamctl" command in shell script to
reload the data
for the modules every 1 minute. In normal times, the reload
takes about 2-3 sec
each minute.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">When the sql server is down, the queries and
reloads fail and it is
OK.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">Yesterday, we had a different failure on the
sql server – The server
is set ona centos 6.2 that is under a VM cluster. The VM
cluster had some disk
writing errors, and the sql server could not write or read
fast enough.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">On 3 of the 6 servers (one at each site)
kamailio crashed.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">I have no idea why. When the sql server is
down, it does not happen.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">I attached all 6 servers log, 1 server core
file, 1 server
var/log/messages and the mysql slow-query log.</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">Any ideas how to handle this?</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"><br>
</font></p>
<p class="MsoNormal" style="">
<font face="courier new, monospace">kamailio version 3.3.2</font></p>
<p class="MsoNormal" style=""><br>
</p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace"> </font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">BR,</font></p>
<p class="MsoNormal" style="direction:ltr"><font face="courier
new, monospace">Uri</font></p>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla
<a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a></pre>
</body>
</html>