issue with XEN network device
 

Hi All,

I have following issue when I am trying to setup XEN virtualization on RHEL52.

I have server with several network interfaces:
- 2 NICs are on board (Broadcom Corporation NetXtreme II BCM5708 Gigabit)
- 4 NICs on external card (Intel Corporation 82575GB Gigabit)

Two onboards cards use vlan 40 and external NICs use different vlan 37

When I use any broadcom card (eth0, eth1) everything goes OK. I can create XEN network bridges, I can add network devices to RHEL Guest and network on RHEL Guest id operating properly. I can ping other hosts or create ssh sesion.

The issue occurs when I use any Intel card (eth2, eth3, eth4, eth5). I can create network bridges and add network device to RHEL Guest. Network interface on RHEL guest is initialized properly and I can ping remote hosts. Also remote hosts are able to ping Guest. Problem is that I cannot create ssh session or telnet session to remote host or from the remote host to Guest. Ssh is waiting for something and after timeout the session failed.

But communication between Host (dom0) and Guest is working properly. I can connect using telnet or ssh both ways.


dom0 INFO: - only eth5 is up

brctl show:
bridge name bridge id STP enabled interfaces
virbr0 8000.000000000000 yes
xenbr5 8000.feffffffffff no vif3.0
peth5
vif0.5

ifconfig:
eth5 Link encap:Ethernet HWaddr 00:1B:21:22:49:FD
inet addr:10.250.37.34 Bcast:10.250.37.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe22:49fd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:198413 errors:0 dropped:0 overruns:0 frame:0
TX packets:75829 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:13949943 (13.3 MiB) TX bytes:519080397 (495.0 MiB)

peth5 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:199455 errors:0 dropped:0 overruns:0 frame:0
TX packets:382067 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:14023568 (13.3 MiB) TX bytes:539288505 (514.3 MiB)
Memory:d58a0000-d58c0000

vif0.5 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:75836 errors:0 dropped:0 overruns:0 frame:0
TX packets:198416 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:519082727 (495.0 MiB) TX bytes:13950123 (13.3 MiB)

vif3.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:218 errors:0 dropped:0 overruns:0 frame:0
TX packets:27588 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:20958 (20.4 KiB) TX bytes:50564873 (48.2 MiB)

virbr0 Link encap:Ethernet HWaddr 00:00:00:00:00:00
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:468 (468.0 b)

xenbr5 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:8637 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:956151 (933.7 KiB) TX bytes:0 (0.0 b)


Guest INFO

ifconfig:
eth0 Link encap:Ethernet HWaddr 00:16:3E:0C:ED:EF
inet addr:10.250.37.53 Bcast:10.250.37.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:28518 errors:0 dropped:0 overruns:0 frame:0
TX packets:266 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:50684948 (48.3 MiB) TX bytes:30300 (29.5 KiB)

ip link show:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:16:3e:0c:ed:ef brd ff:ff:ff:ff:ff:ff

In Guest's modprobe.conf I have line: alias eth0 xennet
Is it correct?

Please could you help with this issue?

thank you very much.

regards,
Stefan

solution
 

Hi all,

I already find the root cause of issue.

The problem is that Intel NIC use module igb. RHEL52 still use the older version which has a bug.

There are two way how to fix issue:

1. Download the latest version of igb module and compile it to system.
2. Use workaround by changing configuration of Intel cards:

ethtool -K <eth_n> tx off

I try the second option and change configuration for all Intel cards and everything is working properly.


regards,
Stefan