While benchmarks are a very difficult exercice and as such the results must be considered carefully, they are still interesting to compare the respective performance between the solutions for different workloads.

This page will try to keep an eye on the relative performance of Xen and KVM. Until 2.6.25, virtio was not part of the mainstream linux kernel, so the tests compared performance of HVM systems (the guest system was an "unmodified OS").

29 Jul 2008

Setup is the same as the tests from 22 Jul 2008 below. Please note that unless specified otherwise, the VMs use a disk image on the host.

Test consist of a single run of bonnie++ v1.03b (default package from ubuntu 8.04).

  • Sequential Output (Size is 6GB)
Per Chr Block Rewrite
platform KB/sec %CPU KB/sec %CPU KB/sec %CPU
host (native) 52149 87 58412 40 29247 13
kvm-71 vm cache=on 23666 63 23968 12 17436 9
kvm-71 vm cache=on using lvm 28147 70 27708 15 26988 13
kvm-71 vm cache=off 41046 82 43866 21 20055 8
kvm-71 vm cache=off using lvm 53492 93 60277 29 28931 12
kvm-71 scsi disk cache=on using lvm 23528 60 28160 14 27864 11
kvm-71 scsi disk cache=off using lvm 35793 76 55243 26 29397 11
kvm virtio disk cache=on 24138 66 24698 13 17989 11
kvm virtio disk cache=off 42277 83 45366 24 20605 9
xen hvm 19245 28 20700 3 17973 4
xen pv 19078 28 20364 3 18698 4
  • Sequential Input (Size is 6GB)
Per Chr Block
platform KB/sec %CPU KB/sec %CPU
host (native) 59967 89 67919 11
kvm-71 vm cache=on 23993 69 37927 10
kvm-71 vm cache=on using lvm 20684 70 65742 19
kvm-71 vm cache=off 21059 63 43648 12
kvm-71 vm cache=off using lvm 23325 69 67107 18
kvm-71 scsi disk cache=on using lvm 20579 67 66032 13
kvm-71 scsi disk cache=off using lvm 23536 64 66527 11
kvm virtio disk cache=on 25058 78 37872 10
kvm virtio disk cache=off 25090 74 41338 10
xen hvm 38117 45 44853 10
xen pv 38087 44 44091 9
  • random (Size is 6GB)
Seeks
platform /sec %CPU
host (native) 211.9 1
kvm-71 vm cache=on 171.2 0
kvm-71 vm cache=on using lvm 198.8 1
kvm-71 vm cache=off 202.7 0
kvm-71 vm cache=off using lvm 206.1 1
kvm-71 scsi disk cache=on using lvm 227.0 1
kvm-71 scsi disk cache=off using lvm 219.5 1
kvm virtio disk cache=on 182.5 1
kvm virtio disk cache=off 224.4 1
xen hvm 150.1 0
xen pv 147.0 0
  • Sequential Create (16 files)
Create Read Delete
platform /sec %CPU /sec %CPU /sec %CPU
host (native) +++++ +++ +++++ +++ +++++ +++
kvm-71 vm cache=on +++++ +++ +++++ +++ +++++ +++
kvm-71 vm cache=on using lvm 32588 99 +++++ +++ +++++ +++
kvm-71 vm cache=off 31645 94 +++++ +++ +++++ +++
kvm-71 vm cache=off using lvm 30843 96 +++++ +++ +++++ +++
kvm-71 scsi disk cache=on using lvm 32077 92 +++++ +++ +++++ +++
kvm-71 scsi disk cache=off using lvm +++++ +++ +++++ +++ +++++ +++
kvm virtio disk cache=on +++++ +++ +++++ +++ +++++ +++
kvm virtio disk cache=off +++++ +++ +++++ +++ +++++ +++
xen hvm +++++ +++ +++++ +++ +++++ +++
xen pv +++++ +++ +++++ +++ +++++ +++
  • Random Create (16 files)
Create Read Delete
platform /sec %CPU /sec %CPU /sec %CPU
host (native) +++++ +++ +++++ +++ +++++ +++
kvm-71 vm cache=on +++++ +++ +++++ +++ +++++ +++
kvm-71 vm cache=on using lvm +++++ +++ +++++ +++ +++++ +++
kvm-71 vm cache=off +++++ +++ +++++ +++ +++++ +++
kvm-71 vm cache=off using lvm +++++ +++ +++++ +++ +++++ +++
kvm-71 scsi disk cache=on using lvm 31669 88 +++++ +++ +++++ +++
kvm-71 scsi disk cache=off using lvm +++++ +++ +++++ +++ +++++ +++
kvm virtio disk cache=on 31648 90 +++++ +++ +++++ +++
kvm virtio disk cache=off +++++ +++ +++++ +++ +++++ +++
xen hvm +++++ +++ +++++ +++ +++++ +++
xen pv +++++ +++ +++++ +++ +++++ +++

comparison to native

(normalised against native performances, rounded to nearest)

Sequential Output Sequential Input Seeks
platform Per Chr Block Rewrite Per Chr Block /sec
host (native) 100 100 100 100 100 100
kvm-71 vm cache=on 45 41 60 40 56 81
kvm-71 vm cache=on using lvm 54 47 92 34 97 94
kvm-71 vm cache=off 79 75 69 35 64 96
kvm-71 vm cache=off using lvm 103 103 99 39 99 97
kvm-71 scsi disk cache=on using lvm 45 48 95 34 97 107
kvm-71 scsi disk cache=off using lvm 69 95 101 39 98 104
kvm virtio disk cache=on 46 42 62 42 56 86
kvm virtio disk cache=off 81 78 70 42 61 106
xen hvm 37 35 61 64 66 71
xen pv 37 35 64 64 65 70

22 Jul 2008

All tests where realised on the same system: AMD X2 4400+/4GB RAM/sata drive.

Host is running Ubuntu Desktop 8.04 amd64, kernel 2.6.24-19-generic (when values are given for the host, it was started with mem=3072M) All the runs with the guests use the same image based on a basic install of ubuntu server 8.04 amd64, with 3072M and smp 2.

  • /!\ The VM image use a disk image on the host. It was reported to me that this severly impacts IO performances.

Versions of the KVM module (manually compiled against host kernel) is either 70 or 71 (default is 71 when non specified)
Version of Xen is 3.3 unstable retrieved from git (7 Jul 2008)

kvm hvm kernel 2.6.24-16-server
kvm hvm virtio same as above, booted using -drive file=<image>,if=virtio,boot=on (initrd contains modules virtio_pci, virtio_blk)
xen hvm kernel 2.6.24-16-server, same as kvm hvm above
xen pv kernel 2.6.18.8-xen (same as Dom0), same image as all vm above

dbench 3.04

The best of 2 runs for dbench 10

platform Throughput in MB/sec % of native
host (native) 544.403 100
kvm vm 543.041 100
kvm virtio disk cache=on 567.623 104
kvm virtio disk cache=off 574.813 106
xen hvm 522.167 96
xen pv 365.44 67

kernbench 0.42

One run of kernbench -M (available memory being below 4GB)

Results present time in s at load -j 8 with standard deviation being in parenthesis

platform Elapsed Time System Time User Time Percent CPU Context Switches Sleeps Elapsed time /native
host (native) 54.214 (0.174442) 93.348 (0.266964) 10.634 (0.265104) 191 (0.707107) 17898.4 (65.9758) 13341.8 (39.8522) 1.00
kvm vm (kvm 70) 84.664 (0.397781) 107.48 (0.501348) 53.692 (0.234883) 190 (0.707107) 15756.4 (66.4139) 16633.6 (92.4868) 1.56
kvm vm (kvm 71) 84.158 (0.16769) 107.826 (0.307782) 52.924 (0.33894) 190.6 (0.547723) 15754 (50.2544) 16518.6 (71.8213) 1.57
kvm vm (kvm 72) 86.678 (0.5205) 108.982 (0.214056) 54.302 (0.628387) 188.2 (0.83666) 15696.8 (49.8618) 16619.8 (99.911) 1.60
kvm virtio disk cache=on 83.676 (0.461877) 107.838 (0.766988) 52.13 (1.02242) 190.8 (0.447214) 15650 (68.4142) 16515.2 (107.183) 1.54
kvm virtio disk cache=off 89.41 (0.91169) 110.73 (0.976627) 57.85 (1.06104) 188 (0.707107) 16143.6 (55.9178) 16746.2 (38.2583) 1.65
xen hvm 101.046 (6.3364) 127.176 (5.13701) 55.446 (1.5455) 181.2 (16.331) 17114.4 (613.934) 16993.8 (388.987) 1.86
xen pv 62.812 (0.458443) 94.474 (0.592182) 20.082 (0.387001) 182 (1.22474) 9545.2 (39.6573) 10872.2 (78.1326) 1.12
xen dom0 60.924 (0.43558) 94.616 (0.261687) 20.294 (0.282011) 188.4 (1.34164) 10330 (232.204) 10976.2 (86.9897) 1.12

13 Feb 2008

Intel system

Tests on an Intel based system with an Intel Core2 2Ghz and 2GB RAM. The test VM for both Xen and KVM is the same image: an unmodified Debian Lenny kernel 2.6.22 amd64 w/512 MB RAM

The test is still the "make clean && make allnoconfig && time make" of a 2.6.24 kernel

  • Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18.8-xen, Dom0 userspace is a 64bits Ubuntu Feisty
run 1 2 3
real 2m34 2m24 2m25
user 1m23 1m22 1m22
sys 1m0 1m0 1m1

dbench 10: 243 MB/s

  • Host is a Ubuntu feisty amd64 kernel 2.6.20 and KVM 60
run 1 2 3
real 2m12 2m13 2m12
user 1m27 1m28 1m27
sys 0m43 0m43 0m43

dbench 10: 255 MB/s

  • Result on host Ubuntu feisty amd64 kernel 2.6.20 (not in a VM)
real 1m31
user 1m11
sys 0m10

dbench 10: 505 MB/s

AMD system

Tests on an AMD based system with an AMD X2 4400+ and 4GB RAM. The test VM for both Xen and KVM is the same image: an unmodified Debian Lenny kernel 2.6.22 amd64 w/512 MB RAM

The test is still the "make clean && make allnoconfig && time make" of a 2.6.24 kernel

  • Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Ubuntu Gutsy
run 1 2 3
real 2m28 2m17 2m16
user 1m27 1m27 1m27
sys 0m51 0m49 0m49
  • Host in Ubuntu gutsy amd64 kernel 2.6.22-14-generic with KVM 60
run 1 2 3
real 2m31 2m29 2m29
user 1m36 1m36 1m36
sys 0m52 0m53 0m52

12 Feb 2008

Thanks for Dan Berrange who pointed me on xen.org instead of xensource.com as a source of download for the latest Xen.

The tests consists of the compilation of a linux kernel 2.6.24 with allnoconfig (make allnoconfig && time make). The host hardware is a 1.86Ghz Intel Core2duo.

unmodified VM: 64bits Debian Etch

The first two tests run the same Virtual Machine with an unmodified 64bits Debian Etch.

  • Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Debian Lenny
run 1 2 3
real 2m26 2m19 2m20
user 1m21 1m22 1m21
sys 1m02 0m58 1m00
  • Host is a 64bits Debian Lenny with a 2.6.24 kernel and KVM-60
run 1 2 3
real 2m11 2m07 2m07
user 1m26 1m26 1m26
sys 0m45 0m40 0m40

paravirtualised VM: 64bits Debian Etch with xenified kernel

The following test runs on the same hardware than the previous one. The kernel used in the DomU is the same as in Dom0:

  • Host is an opensource Xen 3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Debian Lenny
run 1 2 3
real 1m34 1m33 1m33
user 1m17 1m18 1m18
sys 0m15 0m15 0m15

11 Feb 2008

For the following 2 tests, the system is an Intel core 2 duo 2.4Ghz w/ 2GB RAM running the latest available versions:

  • for Xen: opensource Xen 3.1.0 (from xen.xensource.com) + Dom0 kernel based on kernel.org 2.6.18
  • for KVM: kvm-60 on a 2.6.23

The guest used in both case is the exact same VM (non-paravirtualised 64 bit Debian etch with 256 MB RAM).

Guest system boot-up time

platform time in seconds
Xen 23
KVM 20

Kernel compilation benchmark

Time to compile a 2.6.24 allnoconfig

  • first try - cold cache
platform real time user time sys time
Xen 1m53 1m06 0m43
KVM 1m03 1m06 0m33
  • second try - w/ cache
platform real time user time sys time
Xen 1m52 1m07 0m43
KVM 0m56 1m07 0m31