[SOLVED] How does network stack work?

mabo1 · 09-24-2015, 09:13 PM

Hi,

I have been doing a lot reading on the subject recently, mainly because I have network problems, but I still don't think I understand what is happening in the stack.

So, I hope someone may be able try and explain what is happening with my packets?

Example setup;

I have a program that wants to send a packet to an outside internet address, and then on to my server ...

The packet needs to flow through a vpn program and then flow through a local proxy program before it exits to the outside internet.

The flow would look something like this ...

[client ping (1)-> vpn (2)-> proxy (3)->] internet [(4)-> server proxy (5)-> vpn (6)->] internet -> remote host.

Some communication happens via the kernel and some via the stack?

Example; information from ping to vpn does not show up on the stack so kernel handles it?

Now if I watch this happen with tcpdump then what should I expect see?

Rinndalir · 09-25-2015, 05:50 PM

Quote:

Originally Posted by mabo1

I have a program that wants to send a packet to an outside internet address, and then on to my server ...

What program? What protocol?

Usually the program wouldn't care if the network is configured. The route from client to "remote host" should be transparent to the program.

But VPN is all about the network/routing so if it's not configured or blocking your program's protocol then your program won't work.

mabo1 · 09-25-2015, 07:10 PM

The program is ping 8.8.8.8 directed through an openvpn tunnel which is then directed through an obfuscation proxy before leaving my laptop on its way to my server.

Once the tunnel is up I lose contact with the outside internet addresses.

I checked firewall rules, routing, conf files but can not identify what the problem is ...

I started watching packets with tcpdump to try and identify where the problem is, but I am not sure what I should expect to see in the tcpdump output, and this is the reason for my question?

The vpn tunnel has worked previously without the obfsproxy but it is difficult to know for sure when it is and isn't configured correctly because vpn is very unreliable in my host country.

frankbell · 09-25-2015, 09:28 PM

Try a traceroute. It should tell you where the ping is failing.

See man traceroute for details.

mabo1 · 09-25-2015, 11:16 PM

Output from traceroute.

condition is openvpn client/server connected.

Code:

mabo@debian:~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

Rinndalir · 09-26-2015, 12:05 PM

Quote:

Originally Posted by mabo1

Output from traceroute.

What's running on that host? What output do you expect to see and why?

Can you connect to it using a browser, ssh, etc.?

mabo1 · 09-26-2015, 06:02 PM

No programs will connect after the vpn tunnel is connected.

I have been testing with this command; ping 8.8.8.8 and running my browser to load bing.com

I would expect if this command; ping 8.8.8.8 is executed, then I would see something like this in the tcpdump output.

packet src/dst from ping to openvpn

packet src/dst from openvpn to obfsproxy

packet src/dst obfsproxy to router gateway.

Also, I would expect to see some ack packets along the the way.

Here is example of what I do see; obfsproxyproxy 127.0.0.1, tun0 is 10.8.0.33.

I would have also expected a packet directed to the router gateway ip as the last step?

Code:

mabo@debian:~$ sudo tcpdump -n -i any
[sudo] password for mabo: 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes

07:05:58.938902 IP 10.8.0.33 > 8.8.8.8: ICMP echo request, id 2408, seq 16, length 64
07:05:58.938992 IP 127.0.0.1.49164 > 127.0.0.1.10194: Flags [P.], seq 3881:4016, ack 71, win 32748, options [nop,nop,TS val 286532 ecr 286280], length 135
07:05:58.939005 IP 127.0.0.1.10194 > 127.0.0.1.49164: Flags [.], ack 4016, win 802, options [nop,nop,TS val 286532 ecr 286532], length 0
07:05:59.946970 IP 10.8.0.33 > 8.8.8.8: ICMP echo request, id 2408, seq 17, length 64
07:05:59.947138 IP 127.0.0.1.49164 > 127.0.0.1.10194: Flags [P.], seq 4016:4151, ack 71, win 32748, options [nop,nop,TS val 286784 ecr 286532], length 135
07:05:59.947168 IP 127.0.0.1.10194 > 127.0.0.1.49164: Flags [.], ack 4151, win 811, options [nop,nop,TS val 286784 ecr 286784], length 0
07:06:00.954911 IP 10.8.0.33 > 8.8.8.8: ICMP echo request, id 2408, seq 18, length 64
07:06:00.955011 IP 127.0.0.1.49164 > 127.0.0.1.10194: Flags [P.], seq 4151:4286, ack 71, win 32748, options [nop,nop,TS val 287036 ecr 286784], length 135
07:06:00.955027 IP 127.0.0.1.10194 > 127.0.0.1.49164: Flags [.], ack 4286, win 819, options [nop,nop,TS val 287036 ecr 287036], length 0
07:06:01.963001 IP 10.8.0.33 > 8.8.8.8: ICMP echo request, id 2408, seq 19, length 64
07:06:01.963153 IP 127.0.0.1.49164 > 127.0.0.1.10194: Flags [P.], seq 4286:4421, ack 71, win 32748, options [nop,nop,TS val 287288 ecr 287036], length 135
07:06:01.963178 IP 127.0.0.1.10194 > 127.0.0.1.49164: Flags [.], ack 4421, win 828, options [nop,nop,TS val 287288 ecr 287288], length 0

mabo1 · 09-26-2015, 06:31 PM

Here is the output from netstat ...

I think it shows me connections are established and packets have been sent but not received.

Obfsproxy is a python program.

Code:

mabo@debian:~$ sudo netstat -natupvv
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name          
tcp        0      0 127.0.0.1:16            0.0.0.0:*               LISTEN      4844/openvpn        
tcp        0      0 127.0.0.1:10194         0.0.0.0:*               LISTEN      4369/python         
tcp        0   3025 192.168.1.124:50630     VPN_Server_IP:21194     ESTABLISHED 4369/python         
tcp        0      0 127.0.0.1:10194         127.0.0.1:59560         ESTABLISHED 4369/python         
tcp        0      0 127.0.0.1:59560         127.0.0.1:10194         ESTABLISHED 4844/openvpn

frankbell · 09-26-2015, 08:03 PM

I must say, that's the first traceroute I've seen where every device along the way seems to have ICMP reporting turned off.

mabo1 · 09-26-2015, 08:11 PM

If I traceroute or ping 8.8.8.8 without the vpn connected then I do get responses.

When vpn is connected the only ip that I have found that gives me ping reply's is localhost/127.0.0.1.

wildwizard · 09-27-2015, 03:34 AM

I'm surprise no one has asked the obvious.

What does your routing table look like with the VPN down and then when it is up?

mabo1 · 09-28-2015, 06:51 PM

Here is the routing table info.

Before connecting vpn;

Code:

mabo@debian:~$ sudo ip route list table all
default via 192.168.20.1 dev wlan0  proto static  metric 600 
169.254.0.0/16 dev wlan0  scope link  metric 1000 
192.168.20.0/24 dev wlan0  proto kernel  scope link  src 192.168.20.103  metric 600 
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
broadcast 192.168.20.0 dev wlan0  table local  proto kernel  scope link  src 192.168.20.103 
local 192.168.20.103 dev wlan0  table local  proto kernel  scope host  src 192.168.20.103 
broadcast 192.168.20.255 dev wlan0  table local  proto kernel  scope link  src 192.168.20.103 
fe80::/64 dev wlan0  proto kernel  metric 256  pref medium
unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101 pref medium
local ::1 dev lo  table local  proto none  metric 0  pref medium
local fe80::3ea9:f4ff:fea7:da8c dev lo  table local  proto none  metric 0  pref medium
ff00::/8 dev wlan0  table local  metric 256  pref medium
unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101 pref medium

After connecting vpn;

Code:

mabo@debian:~$ sudo ip route list table all
default via 10.8.0.34 dev tun0  proto static  metric 50 
default via 192.168.20.1 dev wlan0  proto static  metric 600 
10.8.0.1 via 10.8.0.34 dev tun0  proto static  metric 50 
10.8.0.33 dev tun0  proto kernel  scope link  src 10.8.0.33  metric 50 
10.8.0.34 dev tun0  proto kernel  scope link  src 10.8.0.33 
127.0.0.1 via 192.168.20.1 dev wlan0  proto static  metric 600 
169.254.0.0/16 dev wlan0  scope link  metric 1000 
192.168.20.0/24 dev wlan0  proto kernel  scope link  src 192.168.20.103  metric 600 
local 10.8.0.33 dev tun0  table local  proto kernel  scope host  src 10.8.0.33 
broadcast 10.8.0.33 dev tun0  table local  proto kernel  scope link  src 10.8.0.33 
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
broadcast 192.168.20.0 dev wlan0  table local  proto kernel  scope link  src 192.168.20.103 
local 192.168.20.103 dev wlan0  table local  proto kernel  scope host  src 192.168.20.103 
broadcast 192.168.20.255 dev wlan0  table local  proto kernel  scope link  src 192.168.20.103 
fe80::/64 dev wlan0  proto kernel  metric 256  pref medium
unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101 pref medium
local ::1 dev lo  table local  proto none  metric 0  pref medium
local fe80::3ea9:f4ff:fea7:da8c dev lo  table local  proto none  metric 0  pref medium
ff00::/8 dev wlan0  table local  metric 256  pref medium
unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101 pref medium

extra info;

Code:

mabo@debian:~$ ip route get 8.8.8.8
8.8.8.8 via 10.8.0.34 dev tun0  src 10.8.0.33 
    cache

mabo1 · 09-29-2015, 01:32 AM

Here is routing table with vpn connected without the obfsproxy;

VPN works without obfsproxy in some geographic areas.

Code:

mabo@debian:~$ sudo ip route list table all
default via 10.8.0.34 dev tun0  proto static  metric 50 
default via 192.168.20.1 dev wlan0  proto static  metric 600 
10.8.0.1 via 10.8.0.34 dev tun0  proto static  metric 50 
10.8.0.33 dev tun0  proto kernel  scope link  src 10.8.0.33  metric 50 
10.8.0.34 dev tun0  proto kernel  scope link  src 10.8.0.33 
VPN_Server_IP via 192.168.20.1 dev wlan0  proto static  metric 600 
169.254.0.0/16 dev wlan0  scope link  metric 1000 
192.168.20.0/24 dev wlan0  proto kernel  scope link  src 192.168.20.103  metric 600 
local 10.8.0.33 dev tun0  table local  proto kernel  scope host  src 10.8.0.33 
broadcast 10.8.0.33 dev tun0  table local  proto kernel  scope link  src 10.8.0.33 
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
broadcast 192.168.20.0 dev wlan0  table local  proto kernel  scope link  src 192.168.20.103 
local 192.168.20.103 dev wlan0  table local  proto kernel  scope host  src 192.168.20.103 
broadcast 192.168.20.255 dev wlan0  table local  proto kernel  scope link  src 192.168.20.103 
fe80::/64 dev wlan0  proto kernel  metric 256  pref medium
unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101 pref medium
local ::1 dev lo  table local  proto none  metric 0  pref medium
local fe80::3ea9:f4ff:fea7:da8c dev lo  table local  proto none  metric 0  pref medium
ff00::/8 dev wlan0  table local  metric 256  pref medium
unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101 pref medium

Code:

mabo@debian:~$ ip route get 8.8.8.8
8.8.8.8 via 10.8.0.34 dev tun0  src 10.8.0.33 
    cache

mabo1 · 09-29-2015, 01:52 AM

The difference I see between the routing tables ...

VPN connected without obfsproxy;

Code:

VPN_Server_IP via 192.168.20.1 dev wlan0  proto static  metric 600

VPN Connected with obfsproxy;

Code:

127.0.0.1 via 192.168.20.1 dev wlan0  proto static  metric 600

Routing tables are identical except for this line.

mabo1 · 09-29-2015, 02:26 AM

When I replace the route with this command then it is working.

Code:

sudo ip route replace VPN_Server_IP via 192.168.20.1 dev wlan0  proto static  metric 600

VPN is connected with obfsproxy running ..

So, next step is to understand why the incorrect route is being added ?