While benchmarks are a very difficult exercice and as such the results must be considered carefully, they are still interesting to compare the respective performance between the solutions for different workloads.

This page will try to keep an eye on the relative performance of Xen and KVM. Until 2.6.25, virtio was not part of the mainstream linux kernel, so the tests compared performance of HVM systems (the guest system was an "unmodified OS").

29 Jul 2008

Setup is the same as the tests from 22 Jul 2008 below. Please note that unless specified otherwise, the VMs use a disk image on the host.

Test consist of a single run of bonnie++ v1.03b (default package from ubuntu 8.04).

Sequential Output (Size is 6GB)

	Per Chr		Block		Rewrite
platform	KB/sec	%CPU	KB/sec	%CPU	KB/sec	%CPU
host (native)	52149	87	58412	40	29247	13
kvm-71 vm cache=on	23666	63	23968	12	17436	9
kvm-71 vm cache=on using lvm	28147	70	27708	15	26988	13
kvm-71 vm cache=off	41046	82	43866	21	20055	8
kvm-71 vm cache=off using lvm	53492	93	60277	29	28931	12
kvm-71 scsi disk cache=on using lvm	23528	60	28160	14	27864	11
kvm-71 scsi disk cache=off using lvm	35793	76	55243	26	29397	11
kvm virtio disk cache=on	24138	66	24698	13	17989	11
kvm virtio disk cache=off	42277	83	45366	24	20605	9
xen hvm	19245	28	20700	3	17973	4
xen pv	19078	28	20364	3	18698	4

Sequential Input (Size is 6GB)

	Per Chr		Block
platform	KB/sec	%CPU	KB/sec	%CPU
host (native)	59967	89	67919	11
kvm-71 vm cache=on	23993	69	37927	10
kvm-71 vm cache=on using lvm	20684	70	65742	19
kvm-71 vm cache=off	21059	63	43648	12
kvm-71 vm cache=off using lvm	23325	69	67107	18
kvm-71 scsi disk cache=on using lvm	20579	67	66032	13
kvm-71 scsi disk cache=off using lvm	23536	64	66527	11
kvm virtio disk cache=on	25058	78	37872	10
kvm virtio disk cache=off	25090	74	41338	10
xen hvm	38117	45	44853	10
xen pv	38087	44	44091	9

random (Size is 6GB)

	Seeks
platform	/sec	%CPU
host (native)	211.9	1
kvm-71 vm cache=on	171.2	0
kvm-71 vm cache=on using lvm	198.8	1
kvm-71 vm cache=off	202.7	0
kvm-71 vm cache=off using lvm	206.1	1
kvm-71 scsi disk cache=on using lvm	227.0	1
kvm-71 scsi disk cache=off using lvm	219.5	1
kvm virtio disk cache=on	182.5	1
kvm virtio disk cache=off	224.4	1
xen hvm	150.1	0
xen pv	147.0	0

Sequential Create (16 files)

	Create		Read		Delete
platform	/sec	%CPU	/sec	%CPU	/sec	%CPU
host (native)	+++++	+++	+++++	+++	+++++	+++
kvm-71 vm cache=on	+++++	+++	+++++	+++	+++++	+++
kvm-71 vm cache=on using lvm	32588	99	+++++	+++	+++++	+++
kvm-71 vm cache=off	31645	94	+++++	+++	+++++	+++
kvm-71 vm cache=off using lvm	30843	96	+++++	+++	+++++	+++
kvm-71 scsi disk cache=on using lvm	32077	92	+++++	+++	+++++	+++
kvm-71 scsi disk cache=off using lvm	+++++	+++	+++++	+++	+++++	+++
kvm virtio disk cache=on	+++++	+++	+++++	+++	+++++	+++
kvm virtio disk cache=off	+++++	+++	+++++	+++	+++++	+++
xen hvm	+++++	+++	+++++	+++	+++++	+++
xen pv	+++++	+++	+++++	+++	+++++	+++

Random Create (16 files)

	Create		Read		Delete
platform	/sec	%CPU	/sec	%CPU	/sec	%CPU
host (native)	+++++	+++	+++++	+++	+++++	+++
kvm-71 vm cache=on	+++++	+++	+++++	+++	+++++	+++
kvm-71 vm cache=on using lvm	+++++	+++	+++++	+++	+++++	+++
kvm-71 vm cache=off	+++++	+++	+++++	+++	+++++	+++
kvm-71 vm cache=off using lvm	+++++	+++	+++++	+++	+++++	+++
kvm-71 scsi disk cache=on using lvm	31669	88	+++++	+++	+++++	+++
kvm-71 scsi disk cache=off using lvm	+++++	+++	+++++	+++	+++++	+++
kvm virtio disk cache=on	31648	90	+++++	+++	+++++	+++
kvm virtio disk cache=off	+++++	+++	+++++	+++	+++++	+++
xen hvm	+++++	+++	+++++	+++	+++++	+++
xen pv	+++++	+++	+++++	+++	+++++	+++

comparison to native

(normalised against native performances, rounded to nearest)

	Sequential Output			Sequential Input		Seeks
platform	Per Chr	Block	Rewrite	Per Chr	Block	/sec
host (native)	100	100	100	100	100	100
kvm-71 vm cache=on	45	41	60	40	56	81
kvm-71 vm cache=on using lvm	54	47	92	34	97	94
kvm-71 vm cache=off	79	75	69	35	64	96
kvm-71 vm cache=off using lvm	103	103	99	39	99	97
kvm-71 scsi disk cache=on using lvm	45	48	95	34	97	107
kvm-71 scsi disk cache=off using lvm	69	95	101	39	98	104
kvm virtio disk cache=on	46	42	62	42	56	86
kvm virtio disk cache=off	81	78	70	42	61	106
xen hvm	37	35	61	64	66	71
xen pv	37	35	64	64	65	70

22 Jul 2008

All tests where realised on the same system: AMD X2 4400+/4GB RAM/sata drive.

Host is running Ubuntu Desktop 8.04 amd64, kernel 2.6.24-19-generic (when values are given for the host, it was started with mem=3072M) All the runs with the guests use the same image based on a basic install of ubuntu server 8.04 amd64, with 3072M and smp 2.

$/!\$ The VM image use a disk image on the host. It was reported to me that this severly impacts IO performances.

Versions of the KVM module (manually compiled against host kernel) is either 70 or 71 (default is 71 when non specified)
Version of Xen is 3.3 unstable retrieved from git (7 Jul 2008)

kvm hvm	kernel 2.6.24-16-server
kvm hvm virtio	same as above, booted using `-drive file=<image>,if=virtio,boot=on` (initrd contains modules virtio_pci, virtio_blk)
xen hvm	kernel 2.6.24-16-server, same as kvm hvm above
xen pv	kernel 2.6.18.8-xen (same as Dom0), same image as all vm above

dbench 3.04

The best of 2 runs for dbench 10

platform	Throughput in MB/sec	% of native
host (native)	544.403	100
kvm vm	543.041	100
kvm virtio disk cache=on	567.623	104
kvm virtio disk cache=off	574.813	106
xen hvm	522.167	96
xen pv	365.44	67

kernbench 0.42

One run of kernbench -M (available memory being below 4GB)

Results present time in s at load -j 8 with standard deviation being in parenthesis

platform	Elapsed Time	System Time	User Time	Percent CPU	Context Switches	Sleeps	Elapsed time /native
host (native)	54.214 (0.174442)	93.348 (0.266964)	10.634 (0.265104)	191 (0.707107)	17898.4 (65.9758)	13341.8 (39.8522)	1.00
kvm vm (kvm 70)	84.664 (0.397781)	107.48 (0.501348)	53.692 (0.234883)	190 (0.707107)	15756.4 (66.4139)	16633.6 (92.4868)	1.56
kvm vm (kvm 71)	84.158 (0.16769)	107.826 (0.307782)	52.924 (0.33894)	190.6 (0.547723)	15754 (50.2544)	16518.6 (71.8213)	1.57
kvm vm (kvm 72)	86.678 (0.5205)	108.982 (0.214056)	54.302 (0.628387)	188.2 (0.83666)	15696.8 (49.8618)	16619.8 (99.911)	1.60
kvm virtio disk cache=on	83.676 (0.461877)	107.838 (0.766988)	52.13 (1.02242)	190.8 (0.447214)	15650 (68.4142)	16515.2 (107.183)	1.54
kvm virtio disk cache=off	89.41 (0.91169)	110.73 (0.976627)	57.85 (1.06104)	188 (0.707107)	16143.6 (55.9178)	16746.2 (38.2583)	1.65
xen hvm	101.046 (6.3364)	127.176 (5.13701)	55.446 (1.5455)	181.2 (16.331)	17114.4 (613.934)	16993.8 (388.987)	1.86
xen pv	62.812 (0.458443)	94.474 (0.592182)	20.082 (0.387001)	182 (1.22474)	9545.2 (39.6573)	10872.2 (78.1326)	1.12
xen dom0	60.924 (0.43558)	94.616 (0.261687)	20.294 (0.282011)	188.4 (1.34164)	10330 (232.204)	10976.2 (86.9897)	1.12

13 Feb 2008

Intel system

Tests on an Intel based system with an Intel Core2 2Ghz and 2GB RAM. The test VM for both Xen and KVM is the same image: an unmodified Debian Lenny kernel 2.6.22 amd64 w/512 MB RAM

The test is still the "make clean && make allnoconfig && time make" of a 2.6.24 kernel

Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18.8-xen, Dom0 userspace is a 64bits Ubuntu Feisty

run	1	2	3
real	2m34	2m24	2m25
user	1m23	1m22	1m22
sys	1m0	1m0	1m1

dbench 10: 243 MB/s

Host is a Ubuntu feisty amd64 kernel 2.6.20 and KVM 60

run	1	2	3
real	2m12	2m13	2m12
user	1m27	1m28	1m27
sys	0m43	0m43	0m43

dbench 10: 255 MB/s

Result on host Ubuntu feisty amd64 kernel 2.6.20 (not in a VM)

real	1m31
user	1m11
sys	0m10

dbench 10: 505 MB/s

AMD system

Tests on an AMD based system with an AMD X2 4400+ and 4GB RAM. The test VM for both Xen and KVM is the same image: an unmodified Debian Lenny kernel 2.6.22 amd64 w/512 MB RAM

The test is still the "make clean && make allnoconfig && time make" of a 2.6.24 kernel

Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Ubuntu Gutsy

run	1	2	3
real	2m28	2m17	2m16
user	1m27	1m27	1m27
sys	0m51	0m49	0m49

Host in Ubuntu gutsy amd64 kernel 2.6.22-14-generic with KVM 60

run	1	2	3
real	2m31	2m29	2m29
user	1m36	1m36	1m36
sys	0m52	0m53	0m52

12 Feb 2008

Thanks for Dan Berrange who pointed me on xen.org instead of xensource.com as a source of download for the latest Xen.

The tests consists of the compilation of a linux kernel 2.6.24 with allnoconfig (make allnoconfig && time make). The host hardware is a 1.86Ghz Intel Core2duo.

unmodified VM: 64bits Debian Etch

The first two tests run the same Virtual Machine with an unmodified 64bits Debian Etch.

Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Debian Lenny

run	1	2	3
real	2m26	2m19	2m20
user	1m21	1m22	1m21
sys	1m02	0m58	1m00

Host is a 64bits Debian Lenny with a 2.6.24 kernel and KVM-60

run	1	2	3
real	2m11	2m07	2m07
user	1m26	1m26	1m26
sys	0m45	0m40	0m40

paravirtualised VM: 64bits Debian Etch with xenified kernel

The following test runs on the same hardware than the previous one. The kernel used in the DomU is the same as in Dom0:

Host is an opensource Xen 3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Debian Lenny

run	1	2	3
real	1m34	1m33	1m33
user	1m17	1m18	1m18
sys	0m15	0m15	0m15

11 Feb 2008

For the following 2 tests, the system is an Intel core 2 duo 2.4Ghz w/ 2GB RAM running the latest available versions:

for Xen: opensource Xen 3.1.0 (from xen.xensource.com) + Dom0 kernel based on kernel.org 2.6.18
for KVM: kvm-60 on a 2.6.23

The guest used in both case is the exact same VM (non-paravirtualised 64 bit Debian etch with 256 MB RAM).

Guest system boot-up time

platform	time in seconds
Xen	23
KVM	20

Kernel compilation benchmark

Time to compile a 2.6.24 allnoconfig

first try - cold cache

platform	real time	user time	sys time
Xen	1m53	1m06	0m43
KVM	1m03	1m06	0m33

second try - w/ cache

platform	real time	user time	sys time
Xen	1m52	1m07	0m43
KVM	0m56	1m07	0m31