Updated: June 25, 2011
All right, by now, you're fairly versed with KVM. We've had a handful of tutorials, including the basic introduction and intermediate setup and use, network and storage configurations and setting up KVM alongside VirtualBox. The next logical step is to learn about bridged networking so that we can give our virtual machines usable external IP addresses, in case we need to forward ports and whatnot.
What I'm going to show you today is going to be a little different from what you have seen in most tutorials elsewhere. Namely, I will try to work around the brute-force approach you might have seen in other tutorials. I'll show you two possible methods of exposing your virtual machines to the external world. One of the method is a hack, but it's absolutely genius. You've never seen it before! And we'll talk a little about limitations and possible problems.
Let us begin with a reality check. Bridged networking is a little ugly and require some understanding of the Linux command line. However, sometimes bridging is necessary. If you want to run some kind of a server inside a virtual machine, you may want give it an external IP address, so that other hosts can communicate with it directly.
Example: Your router grants addresses in the range 10.0.0.X. Your physical hosts uses 10.0.0.4, but your virtual machine runs with 192.168.2.133. Without some changes to how the system interprets packets sent back and forth, clients on the 10.0.0 network will not be able to talk to clients on the 192.168.2 network.
You can work around the problem by using bridging. This will allow your virtual machines to connect to the router and gain external IP addresses. In this context, external means whatever your physical hosts are using, not necessarily non-private addresses.
By default, KVM creates its own bridge, 192.168.122.1. This device acts as a virtual router for your virtual machines and will grant them IP addresses in this range. However, we want our machines to lease external addresses. In this tutorial, this is the 192.168.2 range.
Two alternative solutions
There are two ways we can try doing this.
One, we will create a bridge. Two devices will be bridged. Our physical device eth1 and the virtual device called vnet0. After we ascertain that our bridge works properly, we will commit the changes to a network configuration file on the disk. Virtual machines will lease their IP addresses from the router. This is similar to what we did with VirtualBox. This is the method we will cover here.The second method is not guaranteed to work, but it just might. If your network adapter supports bridging out of the box, then you may be able to use it directly. The only change you will need to do is assign a small subnet of addresses to your virtual bridge in KVM, which does not overlap with the addresses used by physical hosts on the network.
There are many things that can go wrong. Your router may not support bridging. Your network interface may not support bridging. Your available pool of IP address leases may be limited. You may have a firewall interfering. Lastly, specifically for Ubuntu, you might encounter problems and conflicts if the Network Manager is managing the bridged interface.
For more reading, please take a look at the following documentation:
All right, now you know what lies ahead of us. Bridging is definitely not a simple thing. There are several things you need to take into account. First, does your network adapter support bridging, at all? For instance, on some laptops, Wireless adapters do not really support this mode of work. Second, are you comfortable fiddling with network settings? Third, what are you trying to achieve? Perhaps NAT is good enough for you?
While my test box is Ubuntu, I would like this tutorial to be distro-agnostic. Following Ubuntu KVM community documentation is a good thing, but it will only work for Debian based systems. Therefore, let's deviate a little and introduce networking concepts that are applicable for both RedHat and SUSE systems.
On Debian-based systems like Ubuntu, persistent network configurations are kept under /etc/network/interfaces. On RedHat-based systems plus SUSE friends and family, network configurations are stored in individual files under /etc/sysconfig/network, usually prefixed with ifcfg-ethX or ifcfg-eth-MAC-address. Do replace the generic eth with corresponding interface name as you see them on your system using the ifconfig command.
You can also create bridges using the brctl utility on the command line, without committing changes to your configuration files, but the settings will be lost every time you restart the network service. This is a recommended way of testing things first, before editing any system files. And don't forget to backup your configurations before fiddling. For more details on network setups, please take a look at my Linux commands article.
Now, we're ready to play. I suggest you start by using brctl, ifconfig and dhclient commands. You will excuse me if I don't go into too many details on how these are used. If there's popular demand, there's going to be an extra tutorial. In general, the sequence is as follows (with sudo or as root) - let's assume br0, physical and/or virtual devices eth0/1 and DHCP lease from the switch or the router:
ifconfig eth0 0.0.0.0
ifconfig eth1 0.0.0.0
brctl addbr br0
brctl addif br0 eth0
brctl addif br0 eth1
Please note that the virtual adapter may not be available until you create and start the virtual machine. Therefore, you need not add the virtual adapter just yet. KVM will do that automatically for you if you choose bridged networking in the advanced options.
If you see your bridge interface leasing the external address from its DHCP server, then you are on the right track. For home users, the DHCP server will be the router. If you have a direct Internet connection, this might not be possible, as your ISP might not allow more than one IP address per user.
Once the bridged interface comes up, make sure your network functionality remains unchanged. You may need to edit your /etc/resolv.conf and possibly fix the routing table.
If this works, you may want to commit the changes to the network configuration file. Please note that my example is generic, very similar to the one shown in the community documentation. For example, you may need to enable the Spanning Tree Protocol (STP) on your bridge, to properly handle multiple hops or cyclic routes.
Specifically, in our case, we want to edit the /etc/network/interface and add br0 there. If you require STP, then please change bridge_stp off to bridge_stp on. You can verify what your bridge does by issuing btctl show command.
And some STP fun - sounds naughty, doesn't it.
We learned how to do this previously, so it's not a new thing. In the final stage, under Advanced options, change the network address assignment and make sure the virtual adapter has a unique MAC address that does not conflict with any other.
If you're lucky, you might not even need setting up special bridging; your network adapter could support it out of the box, and then you're fine! But you're probably not going to be lucky, so expect a handful of hard keyboard labor.To make sure everything works:
If you can ping it, you can hack it. This is almost identical to what we did with VirtualBox. Now, you can expose services, share data and whatever you need. Enjoy.
You might not be so lucky. The Network Manager may refuse to cooperate and there might be a ton of other issues with your setup. To this end, you may want to create a separate virtual network adapter and route it to a physical device that you believe supports bridging out of the box. Mind the conflicting IP addresses.
Finally, if this works for you, then you can try a very dirty hack. Create a virtual network that shares the same /24 space with your network. For example, if your router is on the 192.168.2.0 network, create a virtual network that starts at 192.168.2.160 and give it only a small range, maybe 16 addresses altogether. Make sure your router never gives out those addresses to other clients on the network.
Here's an example: I created a network 192.168.2.160/28. I'm not expecting my router to ever lease an address in that range, but I can limit its DHCP assignment. Then, route the virtual network to a physical device, in this case, a Wireless adapter eth1.
And now, create your virtual machine and route to eth1, just like above. Once it's online, notice the lovely cooperation. The host has .102 address, the virtual NIC virbr1 has .161 address, while the virtual machine came up with .168 address.
And here's a ping from a physical Windows 7 box on the same network, with .103 address, pinging the virtual adapter with .161 address on the Ubuntu Lucid box, both Wireless, both connected through the same router, showing how cute and smart this setup really is. Now, it won't always work, but when it does, it's genius.
For virtual machines that already exist, you can also edit the settings by changing directives under /etc/libvirt/. Each virtual machine has a separate .xml configuration file. You will need to power cycle your virtual machines for any change to take effect. For instance:
KVM bridged networking setup is not really intuitive. It's about as friendly as VirtualBox was some two or three years ago. But it's manageable, although your real limitations will be the understanding and the functionality of the network topology. It will surely get better with time.
If you're lucky, your network adapters will support bridging out of the box. If not, you will need to create a bridge adapter and assign physical and virtual network devices. Then, resolve any potential issues with DNS, DHCP or routing. Finally, as a dirty hack, you may want to try abusing your network ranges and creating a subnet within a subnet, make sure the IP addresses never overlap, and then route your virtual adapter to your physical device.
This tutorial is fairly basic yet with a cunning twist, but there might be numerous issues left uncovered. If you have questions, suggestions or need tips on overcoming difficult network setups in KVM, send me a mail and I might concoct a sequel howto addressing these new problems.