[SOLVED] Network loss on host when running VMs that use tap devices on the same bridge as main physical interface

saria_chic · 03-07-2024, 02:39 PM

Hello there.

I'm trying to run a simple setup with qemu: run several qemu-kvm virtual machines that have a public (DHCP) IP. I.e., I'm not running any NAT in this network.

I'm using Alpine Linux 3.19.1

My current setup is:

I have a "bridge_ext" bridge that I connect my main interface (eth0). My /etc/network/interfaces file content is:

Code:

auto lo
iface lo inet loopback

iface eth0 inet manual

auto bridge_ext
iface bridge_ext inet dhcp
	bridge_ports eth0
	bridge_stp 0

It works just fine. Then, I create a tap interface for my machine:

Code:

# tunctl -t tap5 -u myself
# brctl addif bridge_ext tap5
# ifconfig tap5 up

And finally, I call qemu:

Code:

$ qemu-system-x86_64 -accel kvm -display vnc=:2 -hda vm.qcow2 -m 2g -netdev tap,ifname=tap5,id=n1,script=no,downscript=no -device e1000,netdev=n1,mac=DE:AD:BE:EF:00:02

At the exact moment I run this command, network is lost in the host with no explanation. But the virtual machine can acquire a new IP by DHCP! After I turn off the virtual machine I still have no access to the network on the host side. I can only access the network after I remove my tap device from the bridge (with brctl) or remove it:

Code:

# tunctl -d tap5

In dmesg, I see stuff like "tap5: entered promiscuous mode". I have no idea if it is related to this problem or not.

Is there something wrong with my setup?

BTW, a very strange thing: even after the tap device (in this example, tap5) is removed from the bridge and deleted, if I create it again and attach it to the bridge, I lost control to the network again. But it doesn't happen if the created tap device was not used by qemu at all. So, right now I have six "unused" tap devices (tap0...tap5) that I cannot use even if I delete and create them again.

Have you ever seen anything like this?

Thank you very much.

jayjwa · 03-10-2024, 11:42 PM

Yes, I had similar happen. How many ip4 addresses is your ISP giving you? Probably one. If you are letting your guest take it, your host is getting disconnected. Run a dhcpd server for the guests. Similar setup: https://www.linuxquestions.org/quest...8/#post6404001 The post mentions SIMH but I use Qemu with it just the same. You will have to do ip4 masquerading if you want the guest virtual machines to reach outside.

saria_chic · 03-25-2024, 06:44 AM

Hi! Thanks for the reply. It happens that I didn't realize that bridge MAC address was cloned from the last interface attached to the bridge. In my configuration, eth0 was the only interface attached to the bridge at the boot (with /etc/network/interfaces configuration). After acquiring an IP address with DHCP, I was manually creating and attaching another interface (a tap interface) to the bridge, which caused the bridge to change the MAC address, hence stopping network from working.

The solution I found was to create all tap devices I need at boot time and attaching them to the bridge before acquiring an IP from DHCP. So I now don't attach or remove no more interfaces from the bridge after it has a valid IP.