<div dir="ltr"><div><div>Hello.<br><br></div>Thanks for the quick reply.<br></div><div><br></div><div>The internal ports were different in the testing I was doing. Client A was using 5061 as the local TCP port for the persistent TLS connection for SIP signalling. Client B was using ephemeral high TCP ports for the brief test connections.<br><br></div><div>The best way to reproduce the problem is to configure iptables for SNAT with a very small port range. Here are the iptables commands I used for my test. Notice I use a range of only 100 ports. 192.168.128.151 is the IP address of the NAT box, which is running Ubuntu server 14.04.<br><br>sudo iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT<br>sudo iptables -A FORWARD -i eth0 -o eth1 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT<br>sudo iptables -t nat -A POSTROUTING -p tcp -o eth0 -j SNAT --to 192.168.128.151:6000-6099<br>sudo iptables -t nat -A POSTROUTING -p udp -o eth0 -j SNAT --to 192.168.128.151:6000-6099<br>sudo iptables -t nat -A POSTROUTING -p icmp -o eth0 -j SNAT --to 192.168.128.151<br><br></div><div>I believe that iptables doesn't mind reusing the same external IP:port, so long as the external destination server IP or port is different. It must maintain enough state to route the responses back to the correct internal target by examining the external server IP and port.<br><br></div><div>Do you agree that setting 'new_conn_alias_flags = 1' seems like a reasonable way to address this issue?<br><br></div><div>If so, then we will likely make a local modification to 'cfg.y' so that we can set that option in the config file. Currently, it seems that it can only be set at runtime.<br><br></div><div>Is it planned to make ''new_conn_alias_flags' available in the config file?<br><br></div><div>Thanks.<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 18, 2016 at 11:24 PM, Daniel-Constantin Mierla <span dir="ltr"><<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hello,<br>
<br>
interesting to see a case when the same ip:port is given by the
firewall for the source of a connection.<br>
<br>
I haven't gone to the code to track the new_conn_alias_flags, but if
tests don't reveal any issue, then all should be good.<br>
<br>
Are the internal ports for the A and B different, or both use the
same (5060/5061)?<br>
<br>
Cheers,<br>
Daniel<div><div class="h5"><br>
<br>
<div>On 18/01/16 19:55, Cody Herzog wrote:<br>
</div>
</div></div><blockquote type="cite"><div><div class="h5">
<div dir="ltr">
<p class="MsoNormal">Hello.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">In a test environment, I've been able to
use the
"new_conn_alias_flags" option to solve a NAT problem, but I'm
concerned that the option is not supported/safe.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">It seems the option is not documented, and
cannot be used in
the config file, because 'cfg.y' doesn't support parsing it.
However, it can be
set at runtime using a command such as the following:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">kamctl kamcmd cfg.set_now_int tcp
new_conn_alias_flags 1</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">**Question #1**</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Is this option experimental and/or risky?</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">As background, I will now try to describe
my NAT problem.
Perhaps there is an alternate way to solve my problem which
doesn't require
using "new_conn_alias_flags".</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">My server architecture uses multiple
Kamailio edge proxies,
and a single central Kamailio registrar. The edge proxies
listen on multiple
TLS ports. All servers are on version 4.3.3.
</p>
<p class="MsoNormal"><br>
</p>
<p class="MsoNormal">My client app includes a port tester, which
periodically
tests whether certain SIP proxy targets are reachable. These
test connections
are very brief, and don’t persist.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">The problem seems to relate to a behavior
of iptables as
follows:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Client A and client B are both behind the
same iptables NAT.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Client A has a persistent TLS connection to
one of the
proxies.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Client B is doing periodic port testing,
and sometimes, the
itpables NAT will assign exactly the same external IP and port
for a test
connection as is already being used by client A for its
persistent connection,
to the same SIP proxy IP. Only the SIP proxy target port is
different.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">To better explain, I will list out examples
for all the
relevant IPs and ports along the paths. The IPs and ports I've
selected are
arbitrary.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Client A persistent TLS connection</p>
<p class="MsoNormal">----------------------------------</p>
<p class="MsoNormal">Internal IP:Port = <a href="http://10.10.10.100:30000" target="_blank">10.10.10.100:30000</a></p>
<p class="MsoNormal">External IP:Port = <a href="http://88.88.88.88:10000" target="_blank">88.88.88.88:10000</a></p>
<p class="MsoNormal">SIP proxy IP:Port = <a href="http://66.66.66.66:5061" target="_blank">66.66.66.66:5061</a></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Client B port test connection</p>
<p class="MsoNormal">------------------------------</p>
<p class="MsoNormal">Internal IP:Port = <a href="http://10.10.10.101:35000" target="_blank">10.10.10.101:35000</a></p>
<p class="MsoNormal">External IP:Port = <a href="http://88.88.88.88:10000" target="_blank">88.88.88.88:10000</a>
*** Same as above!</p>
<p class="MsoNormal">SIP proxy IP:Port = <a href="http://66.66.66.66:443" target="_blank">66.66.66.66:443</a>
*** SIP proxy port being
different is the only thing that makes this a distinct TCP
connection from
above.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">When this happens, it seems that some of
the TCP
connection/alias hash tables inside Kamailio are modified such
that future
attempts to call client A may fail. Client B's port test
connection seems to overwrite some of
the state which was important for connections into Client A.
After client B's test connection has stomped on client A's
state, this is what happens:</p>
<p class="MsoNormal"><br>
</p>
<p class="MsoNormal">When an INVITE sent to client A arrives at
the proxy, the
proxy fails to find the matching persistent TLS connection
which already exists,
so it tries to open a new outbound TLS connection to client A,
but that always fails
because client A's NAT doesn't allow it. Such calls end up
failing with a 408
timeout error.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I added some extra logging to the TCP
connection/alias hash
code paths, and I can see some of the client A entries being
overwritten when
client B makes its test connection.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Did I explain that well enough? I know it's
a bit confusing.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Anyway, after doing some code inspection, I
noticed that the
"new_conn_alias_flags" option seemed like it might improve
this
problem, or at least change the behavior. It turns out that
setting
"new_conn_alias_flags" equal to 1 seems to solve my problem.
With
that setting, client B's test connections do not overwrite any
of the TCP
connection hashes/aliases for client A's persistent TLS
connection, and calls
into client A never seem to fail.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">**Question #2**</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Does setting "new_conn_alias_flags" to 1
seem like
a good way to address my type of problem? If not, is there an
alternate way to
solve my problem? Perhaps there are some things I should be
doing with the NAT
helper module that could fix the issue without relying on
"new_conn_alias_flags"?</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I realize that I may need to provide more
information to
answer these questions fully, but I’m initially hoping to just
get some
high-level impressions, without going into a ton of details.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Thanks.</p>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><pre>_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
<a href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><span class="HOEnZb"><font color="#888888">
</font></span></pre><span class="HOEnZb"><font color="#888888">
</font></span></blockquote><span class="HOEnZb"><font color="#888888">
<br>
<pre cols="72">--
Daniel-Constantin Mierla
<a href="http://twitter.com/#!/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
Book: SIP Routing With Kamailio - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a href="http://miconda.eu" target="_blank">http://miconda.eu</a></pre>
</font></span></div>
</blockquote></div><br></div>