Selective Routing [to bypass VPN for most]

ChuckTSI · 04-16-2014, 11:03 AM

Hello all. Looking to do some selective routing. I am having a hell of a time trying to figure out why things do not work.

I would like all my media pcs (192.168.0.100 & 192.168.0.101) to access internet via VPN (10.173.xx.xx) while everyone else (192.168.0.10-192.168.0.99) bypasses the VPN. (all LAN PCs (192.168.0.xx) must see each other)

So here goes:

Internet: The service is Bell 50/50 but also provides TV service. (meaning can’t get rid of the sagemcom router/modem)
Public IP: 67.63.xx.xx LAN IP Range is 192.168.2.xx Router IP: 192.168.2.1
I have setup the router’s DHCP server to give the Ubuntu 12.04 server (w/ two physical NICS) the ip of 192.168.2.200

eth0 – INTERNET (IP: 192.168.2.200 GW: 192.168.2.1)
eth1 – LAN (IP: 192.168.0.1) (DHCP SERVER)

With this, if I simply do a quick "iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE"
I have access to the internet on all LAN PCs through Bell.

Now I realize this is double NAT, but I’ll deal with that later.

Here is the routing table (main) before VPN

Code:

169.254.0.0/16 dev eth1  scope link  metric 1000
192.168.0.0/24 dev eth1  proto kernel  scope link  src 192.168.0.1
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.200  metric 1

I installed OpenVPN w/ following config

Code:

client
dev tun
proto udp
remote 209.222.18.19 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
tls-client
remote-cert-tls server
auth-user-pass /etc/login.conf
comp-lzo
verb 1
reneg-sec 0

Connection Established

tun0 – VPN (IP: 10.173.1.6 P-t-P: 10.173.1.5) I am guessing that the GW is 10.173.1.1 (I see it in the routing table).

OpenVPN changes the routing table

Code:

0.0.0.0/1 via 10.112.1.5 dev tun0
default via 192.168.2.1 dev eth0  proto static
10.112.1.1 via 10.112.1.5 dev tun0
10.112.1.5 dev tun0  proto kernel  scope link  src 10.112.1.6
128.0.0.0/1 via 10.112.1.5 dev tun0
169.254.0.0/16 dev eth1  scope link  metric 1000
192.168.0.0/24 dev eth1  proto kernel  scope link  src 192.168.0.1
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.200  metric 1
209.222.18.19 via 192.168.2.1 dev eth0

But all of a sudden, all LAN PCS no longer have internet.
so I flush iptables and do: "iptables --table nat --append POSTROUTING --out-interface tun0 -j MASQUERADE"

Voila! LAN traffic leaving through VPN. But I only want two media pcs to go through VPN (tun0). Everyone else through (eth0)

Funny that wget --bind-address=192.168.2.200 can still leave via that interface and find the web url but MASQUERADE loses that ability?

I have been trying to use "ip rule", "ip route" and "iptables" to shape the traffic the way I want
but in all cases 0.0.0.0/1 in the main table is taking all traffic.

Tried nuking 0.0.0.0/1 in the main table and mark packets with two new routing tables w/ 0.0.0.0/1 assignments, but most examples I find do not show how to implement with MASQUERADE.

Any help would be greatly appreciated.

ipconfig

Code:

eth0      Link encap:Ethernet  HWaddr 00:1a:6b:54:8a:9a
          inet addr:192.168.2.200  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::21a:6bff:fe54:8a9a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:20600006 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12766007 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:12085117969 (12.0 GB)  TX bytes:7503088442 (7.5 GB)
          Interrupt:16

eth1      Link encap:Ethernet  HWaddr 00:50:04:04:00:7b
          inet addr:192.168.0.1  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::250:4ff:fe04:7b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10328418 errors:0 dropped:0 overruns:100 frame:0
          TX packets:14925328 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2576196115 (2.5 GB)  TX bytes:4285829908 (4.2 GB)
          Interrupt:21 Base address:0xc000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:55307 errors:0 dropped:0 overruns:0 frame:0
          TX packets:55307 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:36049029 (36.0 MB)  TX bytes:36049029 (36.0 MB)

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:10.112.1.6  P-t-P:10.112.1.5  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:10541098 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8224613 errors:0 dropped:514 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:1780986830 (1.7 GB)  TX bytes:2296687068 (2.2 GB)

netstat -nr

Code:

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.112.1.5      128.0.0.0       UG        0 0          0 tun0
0.0.0.0         192.168.2.1     0.0.0.0         UG        0 0          0 eth0
10.112.1.1      10.112.1.5      255.255.255.255 UGH       0 0          0 tun0
10.112.1.5      0.0.0.0         255.255.255.255 UH        0 0          0 tun0
128.0.0.0       10.112.1.5      128.0.0.0       UG        0 0          0 tun0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.2.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0
209.222.18.19   192.168.2.1     255.255.255.255 UGH       0 0          0 eth0

ip route show

Code:

0.0.0.0/1 via 10.112.1.5 dev tun0
default via 192.168.2.1 dev eth0  proto static
10.112.1.1 via 10.112.1.5 dev tun0
10.112.1.5 dev tun0  proto kernel  scope link  src 10.112.1.6
128.0.0.0/1 via 10.112.1.5 dev tun0
169.254.0.0/16 dev eth1  scope link  metric 1000
192.168.0.0/24 dev eth1  proto kernel  scope link  src 192.168.0.1
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.200  metric 1
209.222.18.19 via 192.168.2.1 dev eth0

iptables -L -t nat -v

Code:

Chain PREROUTING (policy ACCEPT 3370 packets, 388K bytes)
pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 347 packets, 45276 bytes)
pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 882 packets, 58911 bytes)
pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 198 packets, 16782 bytes)
pkts bytes target     prot opt in     out     source               destination
3255  272K MASQUERADE  all  --  any    tun0    anywhere             anywhere

ChuckTSI · 04-16-2014, 12:37 PM

I am an idiot.

I kept specifying the --out-interface on the MASQUERADE
Removed it.

Code:

echo "Clearing firewall and allowing everyone..."
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

#
# First it is necessary to disable Reverse Path Filtering on all
# current and future network interfaces:
#
for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
  echo 0 > $i
done

#
# Delete and table custom and flush any existing rules if they exist.
#
ip route flush table custom
echo "100 custom" >> /etc/iproute2/rt_tables
ip route del default table custom
ip rule del fwmark 1 table custom
ip route flush cache
iptables -t mangle -F PREROUTING

ip route show table main | grep -Ev ^default | grep -Ev tun0 | while read ROUTE ; do ip route add table 100 $ROUTE
done
ip route add default table custom via 192.168.2.1
ip rule add fwmark 1 table custom
ip route flush cache


# Define the routing policies for the traffic. The rules will be applied in the order that they
# are listed. In the end, packets with MARK set to "0" will pass through the VPN. If MARK is set
# to "1" it will bypass the VPN.
#
# EXAMPLES:
#
#  All traffic from a particular computer on the LAN will use the VPN
iptables -t mangle -A PREROUTING -i eth1 -m iprange --src-range 192.168.0.100-192.168.0.110 -j MARK --set-mark 0

# All other clients bypass VPN
iptables -t mangle -A PREROUTING -i eth1 -m iprange --src-range 192.168.0.10-192.168.0.99 -j MARK --set-mark 1

iptables --table nat --append POSTROUTING -j MASQUERADE

echo "Done \n"