<div dir="ltr"><div><div>Hi Daniel<br><br></div>Thanks for the reply! Here are things we use:<br>* keepalived for HA<br></div>* Ubuntu 12.04<br><div>There seems to be problems when Kamailio operates on both UDP and TCP at the same time. But I'll upgrade Kamailio from v4.0.0. to v4.0.7 first and see what we get, since there are a few bug fixes to do with TCP between these two versions.<br><br></div><div>Cheers,<br></div><div>Yufei<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 4 April 2015 at 11:00, <span dir="ltr"><<a href="mailto:sr-users-request@lists.sip-router.org" target="_blank">sr-users-request@lists.sip-router.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send sr-users mailing list submissions to<br>
<a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:sr-users-request@lists.sip-router.org">sr-users-request@lists.sip-router.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:sr-users-owner@lists.sip-router.org">sr-users-owner@lists.sip-router.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of sr-users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: SIP messages over UDP with sizes over MTU (Yufei Tao)<br>
2. Re: SIP messages over UDP with sizes over MTU<br>
(Daniel-Constantin Mierla)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Fri, 3 Apr 2015 12:04:49 +0100<br>
From: Yufei Tao <<a href="mailto:yufei.tao@gmail.com">yufei.tao@gmail.com</a>><br>
To: <a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
Subject: Re: [SR-Users] SIP messages over UDP with sizes over MTU<br>
Message-ID:<br>
<<a href="mailto:CAJwP0iTBGk8S91CheS%2BxAON2fJgyKZH9HFcwTbvcLygmzLNd%2Bg@mail.gmail.com">CAJwP0iTBGk8S91CheS+xAON2fJgyKZH9HFcwTbvcLygmzLNd+g@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi Daniel<br>
<br>
We don't have pike module enabled.<br>
<br>
When the problem occurred on an VIP, we observed these:<br>
* Kamailio stopped responding to any messages that were sent to the VIP,<br>
not just OPTIONS<br>
* The OPTIONS messages are not big. But the other SIP messages, e.g. some<br>
of the INVITE/OK that came from some SBCs can be big<br>
* netstat showed the Recv-Q on that VIP had a lot of bytes accumulated,<br>
while Kamailio was not seeing/reading them<br>
* Kamailio responded fine on its real IP, when I sent OPTIONS pings to it<br>
using sipsak<br>
* After restarting Kamailio it started working. But after 1 week or so the<br>
same problem happened again<br>
<br>
Since this only happened after running for 1 week or so, we didn't have any<br>
traces to show what exactly happened at the particular time when it<br>
happened. It is possible some SIP messages may have come in fragmented and<br>
some are just too big, depending on the route they came from etc. So I was<br>
wonder if it was possible that the UDP receive buffer was filled somehow<br>
with messed-up messages. Is there anyway to check this? Any suggestions<br>
where I should start looking please? Or is it generally a bad idea to use<br>
UDP when there are messages that may be too big, either fragmented or not?<br>
<br>
Since the it is running in the production environment, I'd like to get some<br>
confidence that a Kamailio upgrade will fix the problem first before I<br>
change anything there.<br>
<br>
Cheers,<br>
Yufei<br>
<br>
<br>
<br>
Date: Wed, 01 Apr 2015 16:27:44 +0200<br>
From: Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com">miconda@gmail.com</a>><br>
To: "Kamailio (SER) - Users Mailing List"<br>
<<a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
><br>
Subject: Re: [SR-Users] SIP messages over UDP with sizes over MTU<br>
Message-ID: <<a href="mailto:551C0060.3010002@gmail.com">551C0060.3010002@gmail.com</a>><br>
Content-Type: text/plain; charset="windows-1252"<br>
<br>
Hello,<br>
<br>
first, not related to the topic you reported, I would recommend at least<br>
upgrading to latest 4.0.x, there were many fixes from v4.0.0 and there<br>
are no changes to be done to config or database -- simply deploy latest<br>
4.0.x and restart.<br>
<br>
Back to this topic. Is all the other traffic handled apart of big OPTIONS?<br>
<br>
Do you have pike module enabled? If yes, can you double check and be<br>
sure that the SBC is not blacklisted (traffic from it should not be<br>
handled via pike_check_req()).<br>
<br>
Cheers,<br>
Daniel<br>
<br>
On 01/04/15 15:42, Yufei Tao wrote:<br>
> Hi<br>
><br>
> We've got Kamailio (v4.0.0) connected to some SBC, which sends SIP<br>
> traffic and periodic OPTIONS pings to Kamailio's VIP. Kamailio<br>
> responds to the OPTIONS pings with OK, i.e. in the main route block:<br>
> if (is_method("OPTIONS"))<br>
> {<br>
> sl_send_reply("200","OK");<br>
> exit;<br>
> }<br>
><br>
> All works fine for a week or so, then Kamailio stopped responding to<br>
> the OPTIONS pings on the VIP it listens to. But it still respond to<br>
> OPTIONS pings that are sent to its real IP. The real IP is not used<br>
> for receiving/sending any traffic while only the VIP is. So it seems<br>
> that Kamailio is still working, but maybe having problems with the<br>
> receiving buffer for the VIP?<br>
><br>
> We do see that some SIP messages sent to Kamailio's VIP are too big<br>
> (sometimes over 1500 bytes). My question is, in this case, what would<br>
> be expected to happen? Is it possible somehow the receiving buffer for<br>
> the VIP got messed up by the big UDP messages? Any one seen similar<br>
> problems? What is the suggested solution?<br>
><br>
> We're considering moving to TCP. But since this is production<br>
> environment, I want to get some confidence that the problem we saw was<br>
> likely to have been caused by the UDP message being too large.<br>
><br>
> Cheers,<br>
> Yufei<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.sip-router.org/pipermail/sr-users/attachments/20150403/99203242/attachment-0001.html" target="_blank">http://lists.sip-router.org/pipermail/sr-users/attachments/20150403/99203242/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Fri, 03 Apr 2015 19:23:35 +0200<br>
From: Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com">miconda@gmail.com</a>><br>
To: "Kamailio (SER) - Users Mailing List"<br>
<<a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a>><br>
Subject: Re: [SR-Users] SIP messages over UDP with sizes over MTU<br>
Message-ID: <<a href="mailto:551ECC97.7060600@gmail.com">551ECC97.7060600@gmail.com</a>><br>
Content-Type: text/plain; charset="windows-1252"<br>
<br>
Hello,<br>
<br>
I don't remember any similar reported issues in 4.0, but in general, it<br>
is easier to troubleshoot with the latest version in the branch. Using<br>
the latest 4.0.x will solve other issues that might hit you.<br>
<br>
I still have couple of 4.0 in production, not started for more than half<br>
an year, with all traffic on a virtual ip managed by ucarp.<br>
<br>
Kamailio itself cannot mess the buffer, because it reads an UDP packet<br>
at a time, which is one sip packet and then if it is broken, it is<br>
discarded. There is no queuing inside kamailio for udp packets. Also,<br>
the UDP fragmentation is handled by the OS/Kernel, not by kamailio. So<br>
it might worth checking there are no barriers in the OS.<br>
<br>
I have to say that the OSes I mainly work with are Debian based (Debian<br>
or Ubuntu). I know that CentOS enables selinux by default which has tons<br>
of limits -- if you use it, consider disabling selinux and see if goes<br>
better.<br>
<br>
Cheers,<br>
Daniel<br>
<br>
On 03/04/15 13:04, Yufei Tao wrote:<br>
> Hi Daniel<br>
><br>
> We don't have pike module enabled.<br>
><br>
> When the problem occurred on an VIP, we observed these:<br>
> * Kamailio stopped responding to any messages that were sent to the<br>
> VIP, not just OPTIONS<br>
> * The OPTIONS messages are not big. But the other SIP messages, e.g.<br>
> some of the INVITE/OK that came from some SBCs can be big<br>
> * netstat showed the Recv-Q on that VIP had a lot of bytes<br>
> accumulated, while Kamailio was not seeing/reading them<br>
> * Kamailio responded fine on its real IP, when I sent OPTIONS pings to<br>
> it using sipsak<br>
> * After restarting Kamailio it started working. But after 1 week or so<br>
> the same problem happened again<br>
><br>
> Since this only happened after running for 1 week or so, we didn't<br>
> have any traces to show what exactly happened at the particular time<br>
> when it happened. It is possible some SIP messages may have come in<br>
> fragmented and some are just too big, depending on the route they came<br>
> from etc. So I was wonder if it was possible that the UDP receive<br>
> buffer was filled somehow with messed-up messages. Is there anyway to<br>
> check this? Any suggestions where I should start looking please? Or is<br>
> it generally a bad idea to use UDP when there are messages that may be<br>
> too big, either fragmented or not?<br>
><br>
> Since the it is running in the production environment, I'd like to get<br>
> some confidence that a Kamailio upgrade will fix the problem first<br>
> before I change anything there.<br>
><br>
> Cheers,<br>
> Yufei<br>
><br>
><br>
><br>
> Date: Wed, 01 Apr 2015 16:27:44 +0200<br>
> From: Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com">miconda@gmail.com</a><br>
> <mailto:<a href="mailto:miconda@gmail.com">miconda@gmail.com</a>>><br>
> To: "Kamailio (SER) - Users Mailing List"<br>
> <<a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
> <mailto:<a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a>><br>
> ><br>
> Subject: Re: [SR-Users] SIP messages over UDP with sizes over MTU<br>
> Message-ID: <<a href="mailto:551C0060.3010002@gmail.com">551C0060.3010002@gmail.com</a><br>
> <mailto:<a href="mailto:551C0060.3010002@gmail.com">551C0060.3010002@gmail.com</a>>><br>
> Content-Type: text/plain; charset="windows-1252"<br>
><br>
> Hello,<br>
><br>
> first, not related to the topic you reported, I would recommend at least<br>
> upgrading to latest 4.0.x, there were many fixes from v4.0.0 and there<br>
> are no changes to be done to config or database -- simply deploy latest<br>
> 4.0.x and restart.<br>
><br>
> Back to this topic. Is all the other traffic handled apart of big OPTIONS?<br>
><br>
> Do you have pike module enabled? If yes, can you double check and be<br>
> sure that the SBC is not blacklisted (traffic from it should not be<br>
> handled via pike_check_req()).<br>
><br>
> Cheers,<br>
> Daniel<br>
><br>
> On 01/04/15 15:42, Yufei Tao wrote:<br>
> > Hi<br>
> ><br>
> > We've got Kamailio (v4.0.0) connected to some SBC, which sends SIP<br>
> > traffic and periodic OPTIONS pings to Kamailio's VIP. Kamailio<br>
> > responds to the OPTIONS pings with OK, i.e. in the main route block:<br>
> > if (is_method("OPTIONS"))<br>
> > {<br>
> > sl_send_reply("200","OK");<br>
> > exit;<br>
> > }<br>
> ><br>
> > All works fine for a week or so, then Kamailio stopped responding to<br>
> > the OPTIONS pings on the VIP it listens to. But it still respond to<br>
> > OPTIONS pings that are sent to its real IP. The real IP is not used<br>
> > for receiving/sending any traffic while only the VIP is. So it seems<br>
> > that Kamailio is still working, but maybe having problems with the<br>
> > receiving buffer for the VIP?<br>
> ><br>
> > We do see that some SIP messages sent to Kamailio's VIP are too big<br>
> > (sometimes over 1500 bytes). My question is, in this case, what would<br>
> > be expected to happen? Is it possible somehow the receiving buffer for<br>
> > the VIP got messed up by the big UDP messages? Any one seen similar<br>
> > problems? What is the suggested solution?<br>
> ><br>
> > We're considering moving to TCP. But since this is production<br>
> > environment, I want to get some confidence that the problem we saw was<br>
> > likely to have been caused by the UDP message being too large.<br>
> ><br>
> > Cheers,<br>
> > Yufei<br>
><br>
><br>
> _______________________________________________<br>
> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list<br>
> <a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
> <a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
<br>
--<br>
Daniel-Constantin Mierla<br>
<a href="http://twitter.com/#!/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a><br>
Kamailio World Conference, May 27-29, 2015<br>
Berlin, Germany - <a href="http://www.kamailioworld.com" target="_blank">http://www.kamailioworld.com</a><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.sip-router.org/pipermail/sr-users/attachments/20150403/a32b7130/attachment-0001.html" target="_blank">http://lists.sip-router.org/pipermail/sr-users/attachments/20150403/a32b7130/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
sr-users mailing list<br>
<a href="mailto:sr-users@lists.sip-router.org">sr-users@lists.sip-router.org</a><br>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
<br>
<br>
------------------------------<br>
<br>
End of sr-users Digest, Vol 119, Issue 4<br>
****************************************<br>
</blockquote></div><br></div>