• Immutable Page
  • Info
  • Attachments

XenVsKVM

While benchmarks are a very difficult exercice and as such the results must be considered carefully, they are still interesting to compare the respective performance between the solutions for different workloads.

This page will try to keep an eye on the relative performance of Xen and KVM. Until 2.6.25, virtio was not part of the mainstream linux kernel, so the tests compared performance of HVM systems (the guest system was an "unmodified OS").

29 Jul 2008

Setup is the same as the tests from 22 Jul 2008 below. Please note that unless specified otherwise, the VMs use a disk image on the host.

Test consist of a single run of bonnie++ v1.03b (default package from ubuntu 8.04).

  • Sequential Output (Size is 6GB)

Per Chr

Block

Rewrite

platform

KB/sec

%CPU

KB/sec

%CPU

KB/sec

%CPU

host (native)

52149

87

58412

40

29247

13

kvm-71 vm cache=on

23666

63

23968

12

17436

9

kvm-71 vm cache=on using lvm

28147

70

27708

15

26988

13

kvm-71 vm cache=off

41046

82

43866

21

20055

8

kvm-71 vm cache=off using lvm

53492

93

60277

29

28931

12

kvm-71 scsi disk cache=on using lvm

23528

60

28160

14

27864

11

kvm-71 scsi disk cache=off using lvm

35793

76

55243

26

29397

11

kvm virtio disk cache=on

24138

66

24698

13

17989

11

kvm virtio disk cache=off

42277

83

45366

24

20605

9

xen hvm

19245

28

20700

3

17973

4

xen pv

19078

28

20364

3

18698

4

  • Sequential Input (Size is 6GB)

Per Chr

Block

platform

KB/sec

%CPU

KB/sec

%CPU

host (native)

59967

89

67919

11

kvm-71 vm cache=on

23993

69

37927

10

kvm-71 vm cache=on using lvm

20684

70

65742

19

kvm-71 vm cache=off

21059

63

43648

12

kvm-71 vm cache=off using lvm

23325

69

67107

18

kvm-71 scsi disk cache=on using lvm

20579

67

66032

13

kvm-71 scsi disk cache=off using lvm

23536

64

66527

11

kvm virtio disk cache=on

25058

78

37872

10

kvm virtio disk cache=off

25090

74

41338

10

xen hvm

38117

45

44853

10

xen pv

38087

44

44091

9

  • random (Size is 6GB)

Seeks

platform

/sec

%CPU

host (native)

211.9

1

kvm-71 vm cache=on

171.2

0

kvm-71 vm cache=on using lvm

198.8

1

kvm-71 vm cache=off

202.7

0

kvm-71 vm cache=off using lvm

206.1

1

kvm-71 scsi disk cache=on using lvm

227.0

1

kvm-71 scsi disk cache=off using lvm

219.5

1

kvm virtio disk cache=on

182.5

1

kvm virtio disk cache=off

224.4

1

xen hvm

150.1

0

xen pv

147.0

0

  • Sequential Create (16 files)

Create

Read

Delete

platform

/sec

%CPU

/sec

%CPU

/sec

%CPU

host (native)

+++++

+++

+++++

+++

+++++

+++

kvm-71 vm cache=on

+++++

+++

+++++

+++

+++++

+++

kvm-71 vm cache=on using lvm

32588

99

+++++

+++

+++++

+++

kvm-71 vm cache=off

31645

94

+++++

+++

+++++

+++

kvm-71 vm cache=off using lvm

30843

96

+++++

+++

+++++

+++

kvm-71 scsi disk cache=on using lvm

32077

92

+++++

+++

+++++

+++

kvm-71 scsi disk cache=off using lvm

+++++

+++

+++++

+++

+++++

+++

kvm virtio disk cache=on

+++++

+++

+++++

+++

+++++

+++

kvm virtio disk cache=off

+++++

+++

+++++

+++

+++++

+++

xen hvm

+++++

+++

+++++

+++

+++++

+++

xen pv

+++++

+++

+++++

+++

+++++

+++

  • Random Create (16 files)

Create

Read

Delete

platform

/sec

%CPU

/sec

%CPU

/sec

%CPU

host (native)

+++++

+++

+++++

+++

+++++

+++

kvm-71 vm cache=on

+++++

+++

+++++

+++

+++++

+++

kvm-71 vm cache=on using lvm

+++++

+++

+++++

+++

+++++

+++

kvm-71 vm cache=off

+++++

+++

+++++

+++

+++++

+++

kvm-71 vm cache=off using lvm

+++++

+++

+++++

+++

+++++

+++

kvm-71 scsi disk cache=on using lvm

31669

88

+++++

+++

+++++

+++

kvm-71 scsi disk cache=off using lvm

+++++

+++

+++++

+++

+++++

+++

kvm virtio disk cache=on

31648

90

+++++

+++

+++++

+++

kvm virtio disk cache=off

+++++

+++

+++++

+++

+++++

+++

xen hvm

+++++

+++

+++++

+++

+++++

+++

xen pv

+++++

+++

+++++

+++

+++++

+++

comparison to native

(normalised against native performances, rounded to nearest)

Sequential Output

Sequential Input

Seeks

platform

Per Chr

Block

Rewrite

Per Chr

Block

/sec

host (native)

100

100

100

100

100

100

kvm-71 vm cache=on

45

41

60

40

56

81

kvm-71 vm cache=on using lvm

54

47

92

34

97

94

kvm-71 vm cache=off

79

75

69

35

64

96

kvm-71 vm cache=off using lvm

103

103

99

39

99

97

kvm-71 scsi disk cache=on using lvm

45

48

95

34

97

107

kvm-71 scsi disk cache=off using lvm

69

95

101

39

98

104

kvm virtio disk cache=on

46

42

62

42

56

86

kvm virtio disk cache=off

81

78

70

42

61

106

xen hvm

37

35

61

64

66

71

xen pv

37

35

64

64

65

70

22 Jul 2008

All tests where realised on the same system: AMD X2 4400+/4GB RAM/sata drive.

Host is running Ubuntu Desktop 8.04 amd64, kernel 2.6.24-19-generic (when values are given for the host, it was started with mem=3072M) All the runs with the guests use the same image based on a basic install of ubuntu server 8.04 amd64, with 3072M and smp 2.

  • . /!\ The VM image use a disk image on the host. It was reported to me that this severly impacts IO performances.

Versions of the KVM module (manually compiled against host kernel) is either 70 or 71 (default is 71 when non specified)
Version of Xen is 3.3 unstable retrieved from git (7 Jul 2008)

kvm hvm

kernel 2.6.24-16-server

kvm hvm virtio

same as above, booted using -drive file=<image>,if=virtio,boot=on (initrd contains modules virtio_pci, virtio_blk)

xen hvm

kernel 2.6.24-16-server, same as kvm hvm above

xen pv

kernel 2.6.18.8-xen (same as Dom0), same image as all vm above

dbench 3.04

The best of 2 runs for dbench 10

platform

Throughput in MB/sec

% of native

host (native)

544.403

100

kvm vm

543.041

100

kvm virtio disk cache=on

567.623

104

kvm virtio disk cache=off

574.813

106

xen hvm

522.167

96

xen pv

365.44

67

kernbench 0.42

One run of kernbench -M (available memory being below 4GB)

Results present time in s at load -j 8 with standard deviation being in parenthesis

platform

Elapsed Time

System Time

User Time

Percent CPU

Context Switches

Sleeps

Elapsed time /native

host (native)

54.214 (0.174442)

93.348 (0.266964)

10.634 (0.265104)

191 (0.707107)

17898.4 (65.9758)

13341.8 (39.8522)

1.00

kvm vm (kvm 70)

84.664 (0.397781)

107.48 (0.501348)

53.692 (0.234883)

190 (0.707107)

15756.4 (66.4139)

16633.6 (92.4868)

1.56

kvm vm (kvm 71)

84.158 (0.16769)

107.826 (0.307782)

52.924 (0.33894)

190.6 (0.547723)

15754 (50.2544)

16518.6 (71.8213)

1.57

kvm vm (kvm 72)

86.678 (0.5205)

108.982 (0.214056)

54.302 (0.628387)

188.2 (0.83666)

15696.8 (49.8618)

16619.8 (99.911)

1.60

kvm virtio disk cache=on

83.676 (0.461877)

107.838 (0.766988)

52.13 (1.02242)

190.8 (0.447214)

15650 (68.4142)

16515.2 (107.183)

1.54

kvm virtio disk cache=off

89.41 (0.91169)

110.73 (0.976627)

57.85 (1.06104)

188 (0.707107)

16143.6 (55.9178)

16746.2 (38.2583)

1.65

xen hvm

101.046 (6.3364)

127.176 (5.13701)

55.446 (1.5455)

181.2 (16.331)

17114.4 (613.934)

16993.8 (388.987)

1.86

xen pv

62.812 (0.458443)

94.474 (0.592182)

20.082 (0.387001)

182 (1.22474)

9545.2 (39.6573)

10872.2 (78.1326)

1.12

xen dom0

60.924 (0.43558)

94.616 (0.261687)

20.294 (0.282011)

188.4 (1.34164)

10330 (232.204)

10976.2 (86.9897)

1.12

13 Feb 2008

Intel system

Tests on an Intel based system with an Intel Core2 2Ghz and 2GB RAM. The test VM for both Xen and KVM is the same image: an unmodified Debian Lenny kernel 2.6.22 amd64 w/512 MB RAM

The test is still the "make clean && make allnoconfig && time make" of a 2.6.24 kernel

  • Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18.8-xen, Dom0 userspace is a 64bits Ubuntu Feisty

run

1

2

3

real

2m34

2m24

2m25

user

1m23

1m22

1m22

sys

1m0

1m0

1m1

dbench 10: 243 MB/s

  • Host is a Ubuntu feisty amd64 kernel 2.6.20 and KVM 60

run

1

2

3

real

2m12

2m13

2m12

user

1m27

1m28

1m27

sys

0m43

0m43

0m43

dbench 10: 255 MB/s

  • Result on host Ubuntu feisty amd64 kernel 2.6.20 (not in a VM)

real

1m31

user

1m11

sys

0m10

dbench 10: 505 MB/s

AMD system

Tests on an AMD based system with an AMD X2 4400+ and 4GB RAM. The test VM for both Xen and KVM is the same image: an unmodified Debian Lenny kernel 2.6.22 amd64 w/512 MB RAM

The test is still the "make clean && make allnoconfig && time make" of a 2.6.24 kernel

  • Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Ubuntu Gutsy

run

1

2

3

real

2m28

2m17

2m16

user

1m27

1m27

1m27

sys

0m51

0m49

0m49

  • Host in Ubuntu gutsy amd64 kernel 2.6.22-14-generic with KVM 60

run

1

2

3

real

2m31

2m29

2m29

user

1m36

1m36

1m36

sys

0m52

0m53

0m52

12 Feb 2008

Thanks for Dan Berrange who pointed me on xen.org instead of xensource.com as a source of download for the latest Xen.

The tests consists of the compilation of a linux kernel 2.6.24 with allnoconfig (make allnoconfig && time make). The host hardware is a 1.86Ghz Intel Core2duo.

unmodified VM: 64bits Debian Etch

The first two tests run the same Virtual Machine with an unmodified 64bits Debian Etch.

  • Host is an opensource Xen-3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Debian Lenny

run

1

2

3

real

2m26

2m19

2m20

user

1m21

1m22

1m21

sys

1m02

0m58

1m00

  • Host is a 64bits Debian Lenny with a 2.6.24 kernel and KVM-60

run

1

2

3

real

2m11

2m07

2m07

user

1m26

1m26

1m26

sys

0m45

0m40

0m40

paravirtualised VM: 64bits Debian Etch with xenified kernel

The following test runs on the same hardware than the previous one. The kernel used in the DomU is the same as in Dom0:

  • Host is an opensource Xen 3.2 (from xen.org) with a Dom0 running a kernel-2.6.18-xen, Dom0 userspace is a 64bits Debian Lenny

run

1

2

3

real

1m34

1m33

1m33

user

1m17

1m18

1m18

sys

0m15

0m15

0m15

11 Feb 2008

For the following 2 tests, the system is an Intel core 2 duo 2.4Ghz w/ 2GB RAM running the latest available versions:

  • for Xen: opensource Xen 3.1.0 (from xen.xensource.com) + Dom0 kernel based on kernel.org 2.6.18

  • for KVM: kvm-60 on a 2.6.23

The guest used in both case is the exact same VM (non-paravirtualised 64 bit Debian etch with 256 MB RAM).

Guest system boot-up time

platform

time in seconds

Xen

23

KVM

20

Kernel compilation benchmark

Time to compile a 2.6.24 allnoconfig

  • first try - cold cache

platform

real time

user time

sys time

Xen

1m53

1m06

0m43

KVM

1m03

1m06

0m33

  • second try - w/ cache

platform

real time

user time

sys time

Xen

1m52

1m07

0m43

KVM

0m56

1m07

0m31

Tell others about this page:

last edited 2008-08-16 15:04:08 by GildasLeNadan