iptables postrouting SNAT not working

web_janitor · 08-09-2009, 03:59 AM

I am trying to alter the source IP of my outgoing http packets through iptables. This should be simple enough, right? Regardless, I'm cracking my head trying to understand what I am doing wrong. Wireshark shows that the outgoing packets do not have the source IP modified at all. I want this to happen for TCP packets sent on Port 80 (http traffic).

I am using the nat table in iptables to attempt POSTROUTING. I've tried several different rules at separate times:

iptables -t nat -A POSTROUTING -p tcp --dport 80 -j SNAT --to-source 172.16.8.50 <--- Still sends out TCP packets as originating from 172.16.8.100

iptables -t nat -A POSTROUTING -s 172.16.8.100 -p tcp --sport 80 -o eth0 -j SNAT --to-source 172.16.8.50:80 <--- Still sends out TCP packets as originating from 172.16.8.100

iptables -t nat -A POSTROUTING -d 172.16.10.71 -j SNAT --to-source 172.16.8.50 <--- This rule was a last-ditch effort. It is supposed to rewrite the source IP for ALL packets going to the single client machine. Still doesn't work - the packets have the source IP unchanged.

(The background, possibly irrelevant, is this: I have another server - let's call it serverB/172.16.8.50 - that forwards all http TCP packets on port 80 to serverA/172.16.8.100. This part works. But when ServerA responds, it responds with its source IP for ServerA. I need it to respond with the source IP of ServerB. Otherwise, the client gets confused and ignores the packets (because the client sent an HTTP request to ServerB, but the response comes from ServerA).

web_janitor · 08-09-2009, 04:29 AM

Also, I tried using the LOG functionality of iptables. The POSTROUTING chain for the nat table is not being reached.

But the OUTPUT chain for the filter table does get reached. I can see outgoing packets in the log file. Still can't modify the source IP through the filter table's OUTPUT chain... or can I?

jschiwal · 08-09-2009, 07:39 AM

Don't you want the mangle table instead of the nat table?

web_janitor · 08-09-2009, 10:58 AM

Unfortunately, using the mangle table did not work for me. I did try those same rules, by substituting "-t mangle" for "-t nat". The syntax was not allowed. I received this error: iptables: Unknown error 18446744073709551615

Also, according to the man pages, if I want to do SNAT I need to do it in the nat table: "This target is only valid in the nat table, in the POSTROUTING chain". And according to a netfilter HOWTO on changing the Source IP (see here: http://www.netfilter.org/documentati...T-HOWTO-6.html ), it sounds like this should be done in the "nat" table.

web_janitor · 08-10-2009, 09:57 AM

Still not working, but here is some new information. Out of frustration, I removed all of my iptables rules. Then I added logging statements to see which chains get executed on a normal web server request. The nat POSTROUTING chain is never reached. After mangle POSTROUTING, the response seems to go back to the client.

Can anyone explain why nat POSTROUTING never gets reached? If the nat POSTROUTING never gets reached, then it cannot do SNAT.

Here's an excerpt from my system log (names have been changed to protect the innocent - client is 172.16.10.71, serverB is 172.16.8.100):

Code:

Aug  9 16:05:35 serverB kernel: nat PREROUTING: IN=eth0 OUT= MAC=MAC1 SRC=172.16.10.71 DST=172.16.8.100 LEN=48 TOS=0x00 PREC=0x00 TTL=128 ID=17089 DF PROTO=TCP SPT=1782 DPT=80 WINDOW=64512 RES=0x00 SYN URGP=0
Aug  9 16:05:35 serverB kernel: filter INPUT: IN=eth0 OUT= MAC=MAC1 SRC=172.16.10.71 DST=172.16.8.100 LEN=48 TOS=0x00 PREC=0x00 TTL=128 ID=17089 DF PROTO=TCP SPT=1782 DPT=80 WINDOW=64512 RES=0x00 SYN URGP=0

Aug  9 16:05:35 serverB kernel: filter OUTPUT: IN= OUT=eth0 SRC=172.16.8.100 DST=172.16.10.71 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=80 DPT=1782 WINDOW=5840 RES=0x00 ACK SYN URGP=0
Aug  9 16:05:35 serverB kernel: mangle POSTROUTING: IN= OUT=eth0 SRC=172.16.8.100 DST=172.16.10.71 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=80 DPT=1782 WINDOW=5840 RES=0x00 ACK SYN URGP=0

Aug  9 16:05:35 serverB kernel: filter INPUT: IN=eth0 OUT= MAC=MAC1 SRC=172.16.10.71 DST=172.16.8.100 LEN=40 TOS=0x00 PREC=0x00 TTL=128 ID=17092 DF PROTO=TCP SPT=1782 DPT=80 WINDOW=64512 RES=0x00 ACK URGP=0
Aug  9 16:05:35 serverB kernel: filter INPUT: IN=eth0 OUT= MAC=MAC1 SRC=172.16.10.71 DST=172.16.8.100 LEN=488 TOS=0x00 PREC=0x00 TTL=128 ID=17093 DF PROTO=TCP SPT=1782 DPT=80 WINDOW=64512 RES=0x00 ACK PSH URGP=0

Aug  9 16:05:35 serverB kernel: filter OUTPUT: IN= OUT=eth0 SRC=172.16.8.100 DST=172.16.10.71 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=29498 DF PROTO=TCP SPT=80 DPT=1782 WINDOW=6432 RES=0x00 ACK URGP=0
Aug  9 16:05:35 serverB kernel: mangle POSTROUTING: IN= OUT=eth0 SRC=172.16.8.100 DST=172.16.10.71 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=29498 DF PROTO=TCP SPT=80 DPT=1782 WINDOW=6432 RES=0x00 ACK URGP=0
Aug  9 16:05:35 serverB kernel: filter OUTPUT: IN= OUT=eth0 SRC=172.16.8.100 DST=172.16.10.71 LEN=228 TOS=0x00 PREC=0x00 TTL=64 ID=29499 DF PROTO=TCP SPT=80 DPT=1782 WINDOW=6432 RES=0x00 ACK PSH URGP=0
Aug  9 16:05:35 serverB kernel: mangle POSTROUTING: IN= OUT=eth0 SRC=172.16.8.100 DST=172.16.10.71 LEN=228 TOS=0x00 PREC=0x00 TTL=64 ID=29499 DF PROTO=TCP SPT=80 DPT=1782 WINDOW=6432 RES=0x00 ACK PSH URGP=0

Aug  9 16:05:35 serverB kernel: filter INPUT: IN=eth0 OUT= MAC=MAC1 SRC=172.16.10.71 DST=172.16.8.100 LEN=40 TOS=0x00 PREC=0x00 TTL=128 ID=17098 DF PROTO=TCP SPT=1782 DPT=80 WINDOW=64324 RES=0x00 ACK URGP=0

(And these were the logging statements I used):

Code:

iptables -t mangle -A POSTROUTING -p tcp -j LOG --log-prefix "mangle POSTROUTING: "
iptables -t nat -A POSTROUTING -p tcp -j LOG --log-prefix "nat POSTROUTING: "

iptables -t filter -A INPUT -p tcp -j LOG --log-prefix "filter INPUT: "
iptables -t filter -A FORWARD -p tcp -j LOG --log-prefix "filter FORWARD: "
iptables -t filter -A OUTPUT -p tcp -j LOG --log-prefix "filter OUTPUT: "
iptables -t nat -A PREROUTING -p tcp -j LOG --log-prefix "nat PREROUTING: "

lievendp · 02-01-2010, 03:10 AM

Did you get this working? I'm experiencing the same thing.

Everything worked and then all of a sudden, the snat in postrouting is not working anymore. On the outgoing interface I still see the internal ip as source ip.

Looks like postrouting is never reached.

thanks.

web_janitor · 02-03-2010, 03:28 PM

I did get my problem solved. What I needed to do was set up rules for a "transparent proxy". Once I applied those rules, I had what I needed.

Is sounds like your setup is a little bit different? What are your rules?

lievendp · 02-04-2010, 05:30 AM

Hi, It was solved on my side to, it was in fact a dns issue.

What lead me the wrong way:

- when I monitor networktraffic from the internal lan to the outside internet on my external interface, and this is natted, then I usually see the ip of the external interface as the source and not the ip of the lan clients.

- In this particular case, it was also traffic from the lan going to the outside internet and it should be natted thus I expected to see the external ip of my router as the source.

I don't understand why but I still see the internal lan ip's on the outside interface and there really is a snat rule in the postrouting chain.
BUT... it is working, I get responses from the internet. (strange?)

The dns problem: the setup was changed, there is now an additional internet interface and this interface is now used as default gw. (there is also config for advanced routing so we can use the other interface for specific traffic)

But the dns was still set to the old isp and our dns requests were rejected by him ofcourse because they didn't come from their own network.

Still don't understand why I see the lan ip's on the external interface as the source, there really is a natting rule. (no masquerade, it's a fixed ip)

cheers.

devwatchdog · 02-04-2010, 09:22 AM

Eh, something somewhere is doing some NAT/masquerading for you. I've seen plenty of outbound traffic with private addresses as source -- and it goes _nowhere_. That traffic, by design, cannot be routed through public address space.

lievendp · 02-05-2010, 02:24 AM

Yeah, strange indeed, there only is a router from the isp before but that should not do any natting.
Anyway, If there is a natting rule without restrictions, I should see the external ip address as the source ip on the outbound interface, right?

When I have some time, I'll look into it but for now, it is working.

thanks.

Dando Real · 02-10-2010, 02:47 AM

Quote:

Originally Posted by devwatchdog

Eh, something somewhere is doing some NAT/masquerading for you. I've seen plenty of outbound traffic with private addresses as source -- and it goes _nowhere_. That traffic, by design, cannot be routed through public address space.

Sorry, can you explain me what you mean?
Maybe you mean that if, for example, I have 2 linux routers which should nat their subnets, connected to another linux router which does nat too, the natting of the first machines is bypassed and only the last natting works? And that's way there is no reply to pings made in the subnets, because the last linux router don't see the ip of the middle routers but of the subnet themselves, so it is not able to reply or is not able to nat at all?

However, I have the same problem described here:
I need to nat a subnet with only one fixed ip address. I tried both masquerade and SNAT (ipforward is set to 1).
Whit iptables log, When I ping I see a request made with the subnet ip, both in FORWARD and OUTPUT tables.
with tcpdump, i see the request with my external interface ip, so SNAT/MASQUERADE seems to work. But, there is no reply.
If I ping from the machine wich nats, it works. It's just the subnet which doesn't get answers.
What I see, with two tcpdump, one with src the address I want to ping and the other with dst (i am pinging www.google.com):
tcpdump dst shows me an ICMP echo request, with source ip the external interface ip. After failure, I see an ARP request for 74.125.39.99, tell "to my external ip"
tcpdump src shows me as ICMP echo reply:
nothing until an ARP reply
ARP, Ethernet (len 6), IPv4 (len 4), Reply 74.125.39.99 is-at fe:ff:ff:ff:ff:ff, length 28
and this is non-sense...

I hope someone can help me...

lievendp · 02-11-2010, 04:16 AM

@Dando Real: Are you sure that you have a policy on your firewall/natting device that allows ping from within the subnet?

If your natting is working (as you can see with tcpdump on the external interface) and you can ping google from the device itself, then it probably is a policy or a routing problem. Check your subnet masks and routing table. (Also on the pc on the lan, wrong subnetmasks can cause this behaviour in my experience.)

cheers,
Lieven

devwatchdog · 02-11-2010, 05:34 AM

Quote:

Originally Posted by Dando Real

Sorry, can you explain me what you mean?
Maybe you mean that if, for example, I have 2 linux routers which should nat their subnets, connected to another linux router which does nat too, the natting of the first machines is bypassed and only the last natting works?

If you're doing source nat on every device, then every router in your chain could be affecting the source IP on the packets flowing through it.
If you were to check egress traffic on each router, you should see whatever your source nat rules are doing on that particular device. Should, that is. Heh.

Quote:

And that's way there is no reply to pings made in the subnets, because the last linux router don't see the ip of the middle routers but of the subnet themselves, so it is not able to reply or is not able to nat at all?

If you have your source nat rule set up correctly on each device, you should be able to ping whatever you want, as long as you allow icmp traffic. (as lievendp mentioned)

Quote:

However, I have the same problem described here:
I need to nat a subnet with only one fixed ip address. I tried both masquerade and SNAT (ipforward is set to 1).
Whit iptables log, When I ping I see a request made with the subnet ip, both in FORWARD and OUTPUT tables.
with tcpdump, i see the request with my external interface ip, so SNAT/MASQUERADE seems to work. But, there is no reply.
If I ping from the machine wich nats, it works. It's just the subnet which doesn't get answers.
What I see, with two tcpdump, one with src the address I want to ping and the other with dst (i am pinging www.google.com):
tcpdump dst shows me an ICMP echo request, with source ip the external interface ip. After failure, I see an ARP request for 74.125.39.99, tell "to my external ip"
tcpdump src shows me as ICMP echo reply:
nothing until an ARP reply
ARP, Ethernet (len 6), IPv4 (len 4), Reply 74.125.39.99 is-at fe:ff:ff:ff:ff:ff, length 28
and this is non-sense...

I hope someone can help me...

Soooo...you can ping google.com from the system that nats your traffic to public space, but if you try to ping google.com from a system on your private network, you see arp traffic on the public interface?

That's rather strange. If your default route was set up incorrectly, or missing altogether on the system that nats to public space, I would expect to see arp traffic like that. But, from what I understand, you can ping google.com from your internet router. Is this correct?

We should probably do a few traffic captures, and look at your interfaces as well as your routing table, and your iptables nat rules.

Change your ip information so it doesn't represent your actual setup, of course, which it appears you already do.

We'll need to see the results for:

ifconfig
route -n
iptables -t nat -L

Plus captures for icmp traffic when you ping from an internal system.

You need to ping from a system that is connected to your private network -- an actual client machine. It sounds like you were doing that already.

(internal & external, simultaneously)

tcpdump -nni eth(x) icmp

Dando Real · 02-11-2010, 06:26 AM

Don't worry. Thanks for every reply!
There were no errors on my side.
Know what?
I discovered that openvpn + NAT(Masquerade) + Ubuntu Karmic 9.10 x64 DON'T work.
I installed debian 5 Lenny x64 and everything is as expected.
I can say this for sure, because I first did a fresh install of ubuntu, and it was not working;
then i switched to debian, and by doing the EXACTLY same passages, it works.

I posted on ubuntu forums this problem.

lievendp · 02-13-2010, 12:56 AM

Well,
I'm using openvpn all the time and even with karmic, it has always worked for me.
But if you're using openvpn, you should als make appropriate policies and natting on the firewall too. Depends if you're running your vpn in bridged or routed mode.

But anyhow, glad it works for you now.

rgds,
Lieven