KVM vs QEMU vs libvirt: how Linux virtualization fits together

People say “KVM” when they mean the whole Linux virtualization stack, but it is really three layers. Knowing which layer does what makes error messages and documentation much easier to follow.

KVM: the kernel part

KVM (Kernel-based Virtual Machine) is built into the Linux kernel. It uses the CPU’s hardware virtualization extensions (Intel VT or AMD-V on x86) so guest code can run directly on the real CPU at near-native speed. It is a core module, kvm, plus a CPU-specific module, kvm_intel or kvm_amd. When it is loaded you get the device /dev/kvm.

lsmod | grep kvm
ls -l /dev/kvm
grep -Ec '(vmx|svm)' /proc/cpuinfo   # >0 means the CPU advertises VT-x/AMD-V

KVM alone does not give you a virtual machine: it has no disks, network cards or display. Something in user space has to build those.

QEMU: the machine emulator

QEMU describes itself as a generic, open-source machine emulator and virtualizer. In system emulation it provides a virtual model of an entire machine (CPU, memory and devices such as disks, NICs and graphics). It can emulate the CPU in software, which is slow but works for other architectures, or work with a hypervisor such as KVM, in which case the guest runs on the host CPU. Each running VM is one qemu process on the host.

You can start QEMU by hand, but its command lines get very long. That is where libvirt comes in.

libvirt: the management layer

libvirt is a daemon and API that manages VMs, virtual networks and storage pools. Each VM is described by an XML definition; libvirt turns it into the right QEMU command, starts the process, sets permissions and SELinux/AppArmor labels, and creates networks such as the default NAT bridge virbr0. Tools build on libvirt:

• virsh: the command-line shell for libvirt.
• virt-install: creates new VMs from install media or disk images.
• virt-manager: a desktop GUI.
• Other platforms, such as OpenStack’s compute service, can also use libvirt underneath.

System vs session connections

libvirt’s QEMU driver has two kinds of connection:

• qemu:///system: a privileged instance for servers. VMs here can use host networking such as bridges and are usually what administrators want.
• qemu:///session: a per-user instance where QEMU runs with your own user’s privileges. It is aimed at desktop use and has limited networking.

A classic confusion is creating a VM as root (system) and then not seeing it as a normal user (session). Be explicit:

virsh -c qemu:///system list --all
export LIBVIRT_DEFAULT_URI=qemu:///system

Check that a host is ready

libvirt ships virt-host-validate, which checks whether the host is set up for its hypervisor drivers, including hardware virtualization and /dev/kvm:

virt-host-validate qemu

Putting it together

When you run virsh start web01, virsh asks libvirt, libvirt reads the XML for web01 and launches a qemu process with matching options, and QEMU uses /dev/kvm so the guest CPU runs on the hardware. Errors can come from any of these layers; the message usually tells you which one (for example, a QEMU error is often quoted inside a libvirt error).

Official documentation

• KVM project main page
• About QEMU (qemu.org)
• libvirt QEMU/KVM driver