Linux: Hardware Accelerated Virtualization with QEMU and KVM

In this post I will take you through a series of steps for setting up hardware accelerated virtualization on your PC, if it is supported, along with some background on virtualization. The assumption that I make here is that you already have a Intel/ AMD based processor with virtualization support with a GNU/Linux installation (preferably Debian).

Although it may sound little tricky but believe me, it is not as difficult to set it up, as it seems, given you follow the right steps.

A Must Read for those are planning to buy new PC hardware

Even if you don’t intend to set up virtualization on your PC, it will be a good read and probably you will think about Virtualization support before buying a new PC.

Why should You care?

Usually people prefer to stick with an Operating System which they are more comfortable with and even though they may setup a dual boot machine, they hate to switch between them frequently. At the same time, once in a while you require the other OS for certain things…for example to test some software, to learn an OS (for example if you are a Windows user and want to familiarize yourself with GNU/Linux), to isolate an environment for a software (say a crack for a software which you do not want to infect your windows installation

) etc. , Thus having Hardware Accelerated Virtualization will definitely make the experience of using a Virtual or Guest OS a breeze and will avoid any kind of frustation due to slow emualation without h/w accelerated virtualization.

What is Virtualization?

Virtualization is a mechanism which allows running one kind of environment inside another environment. It can be of several types:

1. API Virtualization:

It is quite like cygwin, where GNU/Linux API calls are translated to equivalent windows API calls. But the apps need recompilation to run on cygwin. Another example is WINE (windows emulator) which does the reverse, i.e it runs unmodified windows applications to be run on Linux.

2. OS Virtualization / Full Virtualization:

It is the kind of virtualization where a complete Operating System runs within another Operating System. It could be any combination of OS (Linux inside Linux, Windows inside Linux, Windows inside Windows with 32, b4 bit combinations as well). it is most popular form of virtualization and is usually required by a lot of people for testing softwares. Examples of this kind of virtualization are: QEMU, VMWare workstation

3. OS Level Virtualization:

It is the application level virtualization provided by the OS itself for the applications. Examples of this include ‘chroot’ jail, where an application runs inside a ‘jail’ with access to the OS resources only in that jail. This is also useful for application testing.

4. Emulators:

This kind of virtualization is usually referred, if not limited to, to the kind of virtualization where a complete environment is replicated. An example of this is Palm OS emulator, Android emulator, Nintendo Emulator etc. Even QEMU is an example of Emulator but it is usually referred to in the context of x86 instruction emulation or OS /Full Virtualization.

5. Desktop Virtualization:

In this kind of virtualization the desktop of a user is emulated / exported to another remote system from where it can be manipulated. Examples of this are: VNC viewer, Windows Remote Desktop. I would even consider eyeOS to be a kind of desktop virtualization.

For a detailed and complete list of different types of Virtualization, refer to Wikipedia article http://en.wikipedia.org/wiki/Virtualization

Hardware Accelerated Virtualization

Hardware Accelerated Virtualization is a kind of Full Virtualization which is accelerated with some hardware support. Not too long ago, it was difficult to do emulation of certain kind of instructions, without modifications to the Guest OS. The requirement for a system to be really called Full Virtualization system is that the Guest OS shouldn’t be modified. Apparently doing this was not so easy and even if implemented, the instructions were trapped and then emulated in software itself. This made the performance of a Virtual OS really slow in comparison to the native OS running on the system.

Hardware accelerated virtualization, solved this problem by proving certain instructions / extensions so that the ‘trap and emulate’ of instructions could be done in hardware. AMD and Intel processor added extensions called vmx or svm respectively to provide support for hardware assisted virtualization. To know if your processor supports these extensions type the following on a shell prompt:

$cat /proc/cpuinfo | grep "vmx \| svm"

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority

Essentially the command above lists the info about your CPU and greps for flag which tells if virtualization extension is supported or not.

In short Hardware Accelerated Virtualization dramatically increases the speed of a ‘Guest OS’, so that it is as much responsive and usable as the Native / Host OS.

Hardware Requirements for Hardware Accelerated virtualization

The requirements for hardware accelerated virtualization are listed below:

CPU support (found using cat /proc/cpuinfo | grep "vmx \| svm")
BIOS support –> Usually most recent Motherboards which support Core2 Duo processors have virtualization support but cross check this information by reading the motherboard manual (available online for most popular motherboard manufacturers).

Software Requirements for Hardware Accelerated virtualization

QEMU –> Qemu is the emulator which acts as the emulator on which the virtualized or ‘Guest’ OS runs. It is similar to VMware client
Linux kernel supporting kvm
kvm.ko and kvm_intel.ko or kvm_amd.ko kernel modules which utilize the vmx | svm capability for h/w acceleration
An OS image to run / test.

How to set up hardware virtualization:

Assuming you have hardware which is capable of accelerating virtualization,

Warning: Since the exact steps to setup hardware virtualization could vary from one Linux distribution to another, the steps below are only indicative and you may need to figure out how to set it up for your distribution. But the principle remains the same. The configuration on which I have been able to setup virtualization is listed below:

Intel Core2 Duo E8400 (3.0 Ghz). –> supports vmx extension

Asus P5P43TD motherboard –> BIOS for this motherboard does support virtualization and is enabled by default)

Debian 5.0.3 (lenny)

At the time of writing this post, kvm support in Qemu downloaded from Debian repository (0.9.1-10lenny1) is broken. Ideally one should be able to download and install Qemu and run it without any issues and Qemu will automatically use h/w acceleration, but that is not the case and hence you require some changes to your setup.

Compile 2.6.32 kernel, with Virtualization support enabled. Please note that the kernel that we build here is not the official Debian kernel wherein certain code is removed from mainline kernel to keep up with the Debian software policy.
- Download ‘Full Source’ of kernel 2.6.32 from http://www.kernel.org/ to some folder. Then:
- $tar -jvxf linux-2.6.32.2.tar.bz2
- cd linux-2.6.32.2
- Copy your existing kernel config to untarred folder i.e. linux-2.6.32.2/.config
- Run menuconfig, which is a ncurses based configuration chooser (lets you choose various features to be enabled / disabled in the Linux kernel). If it asks for questions for enabling / disabling of features, just choose the default, unless you know what you are doing.
- make menuconfig
- Select Virtualization support in the menuconfig (see the screenshots below).
- Install any build dependencies. This will install g++
- sudo apt-get install build-essential fakeroot
- Compile and install the kernel the Debian way or you can compile it the usual way and install it (in that case skip this step). The following step will build the kernel and automatically do the changes to your bootloader (grub or lilo), copy the kernel modules into /lib/modules/2.6.32.2/, setup links in / pointing to appropriate vmlinuz, prepare initrd automatically. This actually makes building kernel very easy on a debian system.
- fakeroot make-kpkg –initrd –revision=custom.1.0 kernel_image
- sudo dpkg -i ../linux-image-2.6.18_custom.1.0_i386.deb

Download, build and install the qemu-kvm package
- Download the qemu-kvm package from http://sourceforge.net/projects/kvm/files/. In my case I downloaded qemu-kvm-0.12.1.1.tar.gz, which works fine for me.
- Ensure that package checkinstall is already installed on your system. This is not essential as you can install a package on your system using usual ‘make install’ which will simply copy the required files in appropriate folders. But installing it through checkinstall ensures that you can get the status of your installed package and you can remove it like any standard debian package. To install ‘checkinstall’:
- sudo apt-get install checkinstall
- Do
- $qemu-kvm-0.12.1.1.tar.gz
- cd qemu-kvm-0.12.1.1
- ./configure
- make
- checkinstall
- Chekinstall asks some questions like the name of the package etc. which you can provide so as to keep track of the installed packages.
- Once you do dpkg –list it shows the following:
  - dpkg –list qemu-kvm
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
    ||/ Name Version Description
    +++-=================================-=================================-==================================================================================
    ii qemu-kvm 0.12.1.1-1 qemu-self-compiled
- To see the list of files installed by this package, run the folowing command:
- dpkg –list qemu-kvm
- This lists a lot of files of which the following is important
- ….
- …..
- …..
- /usr/local/bin/qemu-system-x86_64
- /usr/local/bin/qemu-nbd
- /usr/local/bin/qemu-io
- /usr/local/bin/qemu-img
- /etc/init.d/qemu-kvm

Now reboot your system
Add the user to kvm group. This is essential so that the qemu can read the device /dev/kvm, which by default has permissions 600. Simply changing the permission to 666 is not a good option as this will be reset to 600 on next reboot.
- sudo adduser divkis01 kvm
Before you can us QEMU + KVM, you need to have an OS image to be able to boot it with QEMU. To prepare images,
First make an ISO image of your XP, Ubuntoo CD or alternatively if you already have an Iso Image skip this step
- dd if=/dev/cdrom of=~/MyMachines/winxp.iso
Now create an image in which the OS installation will be done.
- qemu-img create -f qcow2 winxp_x86.img 10G
Install the OS on the image created in last step
- qemu-system-x86_64 -hda winxp_x86.img -cdrom ~/MyMachines/winxp.iso -boot d -m 512
Once the installation is complete, you are ready to run Qemu with hardware acceleration. You can boot the OS using the following command:
- qemu-system-x86_64 ~/MyMachines/windows/winxp_x86.img

See the screen shots below which shows Windows XP and Ubuntoo booted on a Debian GNU/Linux host

Caveats

If you see that your native OS (on which you are running QMU) is fast enough but the guest OS runing on QEMU is too slow, then it is possible that hardware acceleration is not being used. There are several ways to figure out if that is the case or not.
- Listing the currently loaded module and grepping for kvm, should show non zero entries in the right most column. For example on my machine, it shows:
  - lsmod | grep kvm
    kvm_intel 31979 6
    kvm 181149 1 kvm_intel
- On running QEMU, if QEMU complains with the following message
  - qemu-system-x86_64 ~/MyMachines/Linux/ubuntoo-9.10.x86.img &
    divkis01@divkis01-pc:/home/data/downloads/isos/debian-x86$ open /dev/kvm: No such file or directory
    Could not initialize KVM, will disable KVM support
- The performance is too low of the guest OS. I have found drastic difference at least 10x perceived difference with and without hardware acceleration

The reason for h/w acceleration not available could be due to any of the following reasons:
- kvm not properly installed
- user running QEMU is not part of kvm group i.e. the user does not have access to device /dev/kvm
- kvm is installed but the kernel modules for kvm (kvm.ko and kvm_intel.ko / kvm_amd.ko) are not properly loaded
- The installed kernel does not have kvm built as module
- Or probably h/w acceleration is not supported by your processor or motherboard.

Useful Links

http://en.wikipedia.org/wiki/Hardware-assisted_virtualization

http://www.linux-kvm.org/page/Main_Page

http://www.nongnu.org/qemu/user-doc.html

http://kernel-handbook.alioth.debian.org/

http://wiki.debian.org/QEMU

If you face any difficulty setting up kvm or QEMU, don’t hesitate to send me questions. I will be glad to help. And if you find this page useful, please feel free to drop a comment.

Happy QEMUing!!!!

Linux

Tuesday, February 22, 2011

Hardware Accelerated Virtualization with QEMU and KVM