<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Candara;
panose-1:2 14 5 2 3 3 3 2 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas","serif";
color:black;
mso-fareast-language:EN-US;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Daniel<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">There is no load on the system as I’m developing the implementation at the moment, I have a softphone and hardphone registered and that is it.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Im using Centos 6.5 32bit as the OS and have disabled SElinux whilst developing.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">There is no rhyme or reason as far as I can tell, sometime’s registration hangs (using auth_db against my Asterisk RT tables), sometimes it’s the dispatcher, and sometimes just calling ‘kamctl dispatcher reload’.
One thing I have tried to help narrow the search focus down was run this when I experienced loss of connectivity<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">‘date && kamctl dispatcher reload && date’ which would give the start and end time of the query, where I seem to experience a 30min delay between start and finish (I have 2 entires in the dispatcher table, so
its not a lack of index or anything like that).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I have a nasty feeling this may be caused by SQL Azures idle connection handling which is quite different to the standard server implementation (which works frustratingly fine, i.e. if I work from home the kamailio
process handles vpn disconnections and resumes operations as soon as I connect).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Im going to try disabling connection pooling in unixodbc, see if that helps, if that doesn’t help then I fear I am going to have to brush off my c skills and dive into the db_unixodbc module and general DB logic
to get a better understanding of how Kamailio is handling db connections.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Tim.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="color:windowtext;mso-fareast-language:EN-GB">From:</span></b><span lang="EN-US" style="color:windowtext;mso-fareast-language:EN-GB"> sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org]
<b>On Behalf Of </b>Daniel-Constantin Mierla<br>
<b>Sent:</b> 26 June 2014 09:08<br>
<b>To:</b> Kamailio (SER) - Users Mailing List<br>
<b>Subject:</b> Re: [SR-Users] Strange application hang querying DB<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hello,<br>
<br>
have you identified the config function that hangs? Is a sqlops function or something related to authentication, location, etc?<br>
<br>
If not, then you can use benchmark module to measure the duration of execution of various parts of config, in this way identifying the function that takes too long.<br>
<br>
You should eventually run directly with debug=3 and watch the logs in syslog file.<br>
<br>
Once you tell the function taking too long, I will try to dig further to see if I can spot any issue.<br>
<br>
Some other details would help:<br>
- is the system under load or it happens only for one request and always?<br>
- what is the operating system? I encountered many issues with selinux in the past, under high load or too many connections opened<br>
<br>
Cheers,<br>
Daniel<span style="font-size:12.0pt;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<div>
<p class="MsoNormal">On 25/06/14 20:06, Tim Chubb wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">I’m currently wracking my brains trying to deduce where an application hang can be coming from.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">I have ported the PostGres schema to be compatible with MS Sql Server and have had no issues getting a working dev environment working. However when I try and use SQL Azure as the backend db as opposed to SQL Server 2012 (running locally
on my dev machine) Kamailio will seem to hang after about 10 – 15 mins of flawless operation. I have recreated the SQL Azure schema from the working schema running on my dev machine and copied all data in a 1:1 manner so I can be certain it’s not a data quality
issue. I am using the same FreeTDS/UnixODBC config that I am using for Asterisk realtime (which doesn’t exhibit this behaviour). The really strange thing is that I can see the queries being sent by Kmailio in the freetds trace log, and the response from
SQL Azure, however its as if the db response handler is deadlocked/hung. The kamailio process continues to work fine, and as long as its not required to commiunicate with the DB all appears well i.e. no error messages in the log file.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">This behaviour only happens when im using SQL Azure as opposed to SQL Server, so there must be something causing the issue, that’s unique to SQL azure as opposed to SQL Server. As far as I can tell this is not something I can create a
backtrace for as no core dumps are produced, as far as I can tell there is no easy way debug what the lock is. If anyone has any suggestions I would be very grateful to hear them.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">One thing that occurs to me is that the db_unixodbc module doesn’t have half as many parameters as its equilent in Asterisk, and that connection pooling may be in use, this could be a bit of a show stopper AFAIK you have to disable connection
pooling with SQL server, certainly any of the guides I have come across configuring Asterisk to work with SQL server suggests that.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">Tim Chubb</span></b><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">Developer</span><o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Candara","sans-serif";color:#404040;mso-fareast-language:EN-GB"><img width="208" height="42" id="Picture_x0020_1" src="cid:image001.png@01CF9124.D1B194C0" alt="VS-logo-email"></span><o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">The Coach House, Heywood House, Park Lane, Heywood, Westbury, BA13 4NA</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">Direct: +44 (0)333 0110164</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">Email:
<a href="mailto:tim.chubb@voicesimplified.com">tim.chubb@voicesimplified.com</a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">Web:
<a href="http://www.voicesimplified.com/">www.VoiceSimplified.com</a></span><o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:6.0pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:8.0pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">Voice Simplified Ltd (registered in England & Wales: 07171825) registered office: Curzon House, 2nd Floor,
24 High Street, Banstead, Surrey. SM7 2LJ.</span><o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:8.0pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">The information in this email is confidential and may be legally privileged. It is intended solely for
the addressee. Access to this email by anyone else </span><o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:8.0pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action
taken or omitted to be taken in reliance on it,</span><o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:8.0pt;font-family:"Candara","sans-serif";color:#44546A;mso-fareast-language:EN-GB">is prohibited and may be unlawful. If you are not the intended recipient, please return the message to
the sender and delete it from your records. </span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:EN-GB"><br>
<br>
<br>
<o:p></o:p></span></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list<o:p></o:p></pre>
<pre><a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><o:p></o:p></pre>
<pre><a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:EN-GB"><br>
<br>
<o:p></o:p></span></p>
<pre>-- <o:p></o:p></pre>
<pre>Daniel-Constantin Mierla - <a href="http://www.asipto.com">http://www.asipto.com</a><o:p></o:p></pre>
<pre><a href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a><o:p></o:p></pre>
</div>
</body>
</html>