A wise server migration

Yesterday I migrated two servers at work to a new machine. The old machines were pretty underpowered: a Pentium III 1 Ghz system (with 384 MB RAM if I remember correctly) and a Pentium 4 system. The new machine is a Dell Poweredge R410 with a Xeon E5504 processor, 12 GB RAM and a PERC H700 RAID controller (which is actually a LSI MegaSAS 9260) with 3 750 GB SATA disks in RAID 5. The hardware is pretty nice (although I am not happy at all with the way Dell treats its customers, so I will rather prefer other vendors in the future, but maybe more about that in a later post).

The old servers were running Debian Lenny, while the new server runs Debian Squeeze, which is frozen since yesterday. That was a nice coincidence. The new server actually hosts three virtual machines using the Linux KVM virtualization system. One of them hosts the website http://wise.vub.ac.be, a second VM hosts a file server, and a third VM is the gateway for a small internal network.

The migration went pretty well, except for the smaller problems which you can always expect with such things, for example a Java web application which had hard-coded host names which had changed. The update to PHP 5.3 caused some compatibility problems which I already fixed before the migration itself: Dokuwiki had to be updated to a recent version in order to not show any warnings. There was also a website based on Joomla 1.0, which is actually not supported anymore. As updating it to a recent version was not really an option, I found some posts on the web on how to fix the errors I was seeing and cooked up a simple patch.

The new server is using 140 Watts most of the time. Keeping in mind that in the future it will also replace a third old server, that will probably be a nice reduction of power consumption compared to the old situation.

Migration to virtual machine

I just finished migrating this website to a new KVM virtual machine. The virtual machine is running Mandriva Cooker (2010.0) now with kernel 2.6.31-rc9 and the virtio drivers. The host machine is a HP DL185 G5 running Debian Lenny (kernel 2.6.26) and kvm 85.

During the next days, there might be some short downtime now and then while I continue configuring things

Server migration

Since two days, I have merged the main servers used by two research laboratories at work. One server was an old Linux server which really needed a hardware upgrade, and the other one was a Mac Pro machine running a flaky OS X Leopard. The new server is of course running Linux: Debian Lenny.

It was a very interesting experience: working out procedures to migrate the mailboxes (from Dovecot on the Linux server and Cyrus on the Mac server to Cyrus on the new server), finding out how to set up one NIC in two different subnets (especially the routing is a little bit tricky), getting all services hooked up to LDAP and managed by GOSA, getting dhcpd to do exactly what we want in a shared-network set up, and much more.

The new server is a HP DL185 G5 with an AMD Opteron quad core CPU and 8 GB of RAM and hosts two KVM virtual machines, one for public services and another one running internal services. You can visit the two websites, which are also hosted on this machine of course, of the concerned research labs:

Maybe in the not too far away future, I should try to move the services hosted on the underpowered desktop machine running this website, also to a virtual machine…

Using KVM on Mandriva 2009.0

Introduction

My new Dell Latitude E6400 laptop, is my first system which support hardware visualization. It also has 4 GB RAM, making this an ideal machine to start experimenting with KVM, the kernel based virtual machine system which is now integrated in Linux itself.

In the past I had already used different virtualization systems. VMWare Server is annoying because you have to rebuild the kernel modules by hand each time you install a new kernel and it’s not Free (as in Free Speech). Virtualbox handles the modules rebuild nicely thanks to the use of DKMS in Mandriva, but always randomly crashes on an assert, making it totally unusable for serious work. Also it does not really include a server mode, where virtual machines just run in the background. Xen is especially nice if you have a system which does not support hardware virtualization and you just want to run a paravirtualized Linux in it. However, the host system (Dom0) needs to run a very specific kernel version (normally version 2.6.18, although Debian has a 2.6.26 based one now). OpenVZ is nice for lightweight Linux server containers (some kind of chroot on steroids), but is not a complete virtualization solution.

Because KVM is built into the standard Linux kernel, this means there won’t be any problems with kernel modules which have to be installed separately. It’s a complete virtualization system which means that you can run all kind of operating systems on it, like for example also Windows. KVM is evolving very quickly and its performance and features are constantly improving. Thanks to libvirt and virt-manager, it also has a nice GUI front-end which make configuring and managing virtual machines very easy (libvirt and virt-manager also include support for Xen and OpenVZ, but I don’t know how well that works).

Installing and using KVM on Mandriva 2009.0

Before installing KVM, it’s important to verify that the CPU on your system has hardware virtualization support. Start up a terminal, and run


$ cat /proc/cpuinfo

If you have an Intel CPU, you should have vmx in the flags. If you are using an AMD CPU, you should have svm.

It’s best to use KVM, libvirt and virt-manager from Mandriva’s backports repositories. The versions included there, are the most recent ones, and fix a few problems which made networking not work out of the box with the ones included in the standard repositories. So start up the Mandriva Control Center (Tools – System Tools – Configure Your Computer), and in the Software category, launch the media sources configuration tool and enable the Main Backports and Contrib Backports sources by checking the checkboxes.

Before we install the necessary packages, it’s necessary to make a change to udev’s configuration file to prevent the virbr* and vnet* virtual ethernet devices and bridges being configured automatically by Mandriva’s configuration tools. To do so, edit the file /etc/sysconfig/udev_net and add virbr* and vnet* to the BLACKLIST_ALL line if they are not there yet. The line should look like this:

BLACKLIST_ALL="ppp*|ippp*|isdn*|plip*|hso*|lo*|irda*|dummy*|ipsec*|vmnet*|wifi*|
wmaster*|br*|vbd*|vtpm*|vif*|ax*|nr*|rose*|bce*|scc*|virbr*|vnet*"

Then install the kvm and virt-manager packages with the graphical software installation utility or with urpmi. Some extra dependencies will be installed, such as dnsmasq-base, libvirt-utils and python-virtinst.

After the installation, make sure that the libvirtd service is running. You can use the graphical tool in the Mandriva Control Center in the category System – Manager system services, or you can run “service libvirtd status” to verify whether it’s running and “service libvirtd start” to start it.

Now start the Virtual Machine Manager from Tools – Emulators. Choose File – Add Connection and set up a Local QEMU connection. Double click on the localhost/qemu connection in the virtual machine manager, and enter your root password. You can now create new virtual machines by clicking on the New button. To use KVM, choose a fully virtualized system, and choose KVM as hypervisor. If you want to run Mandriva 2009.0 or later in a KVM based virtual machine, you can probably choose Linux as OS type and Ubuntu Hardy as OS variant, because that should give better performance than the generic options.

I currently have Windows Vista and Debian Lenny running in KVM. It works very fast and very reliably. The only problem I encounter, is that sometimes I cannot switch back from my Windows virtual machine to my host system: the mouse and keyboard cannot leave the VNC window anymore and also the Ctrl-Alt keystroke does not release the mouse pointer. The only solution is to shut down the Windows system.

Virtualbox 1.4

Only one week after I had no success with running Virtualbox on my Athlon 64 system, a new version was announced. One of the important changes in Virtualbox 1.4 is support for AMD64 hosts, so this seemed exactly what I was looking for! To test new distributions and software, I have already been using VMWare Server for some time, which is free (read: it costs nothing), but a real Free (as in free speech) virtualisation solution always sounds interesting, especially as Fedora 7 always crashed VMWare Server and my host.

Installation of Virtualbox was very easy. It has been packaged and integrated in Mandriva, so a simple “urpmi virtualbox” sufficed to install it. Already a lot easier than VMWare Server, which comes in different RPM and ZIP files you have to download and extract. There was no hassle with licences, as Virtualbox is released under the GPL unlike VMWare Server for which you need to register on the site to request a licence key.

The kernel modules for Virtualbox were automatically built with dkms. This time, there were no problems with my x86_64 2.6.21-tmb kernel! Again this was easier than in VMWare, which often needs the installation of an extra patch if you are running a recent kernel.

Configuration is a bit different than VMWare, but actually very easy. The only thing which seems more complex than VMWare, is configuration of bridged networking, i.e. if you want to integrate your virtual machine directly in your real network like a real physical machine. According to the documentation it requires some manual bridge configuration on the host, but I did not try this. For simple NAT networking, I had not to do anything, this worked out of the box and was sufficient for me.

Virtualbox supports everything you would expect from a modern virtualisation system: ACPI, networking, cd/dvd drives (you can access a physical drive or use an ISO file, like VMWare) and sound. The sound implementation in Virtualbox is even better than VMWare, as it can use both OSS and Alsa. With VMWare I never succeeded in having working sound, because I’m using Alsa, and VMWare always complained that /dev/snd was in use. With Virtualbox and Alsa, everything is working great now. Virtualbox also supports creation of snapshots. In VMWare Server you can only create one snapshot, if you need to create more, you have to pay for another edition. Did I say that Virtualbox has everything you would expect? Well, maybe that’s not true. There’s one important thing missing: unfortunately it does not have USB support. This is an important omission which I hope will be added soon, as this works great in VMWare.

Unlike VMWare, Virtualbox does not have any problem with the fact that I am using frequency scaling on my processor (AMD’s Cool’n’Quiet with the powernowd daemon in Linux). In VMWare I had to disable frequency scaling, otherwise the clock of the virtual machine went too fast or too slow most of the time. But not with Virtualbox!

Virtualbox uses a nice QT interface, which integrates very well in a KDE environment. I don’t like QT’s open and save dialogs too much, but as this is a virtualisation product, and not a document editor, you won’t need these too much, so I can live with that. Virtualbox can use VMWare images, but unfortunately it is still not so easy to import your VMWare virtual machines as the virtual hardware is different. My Mandriva 2007 Spring installation in VMWare did not succeed to mount the root partition in Virtualbox, because of the different hard drive controller. With a rescue CD and some manual regeneration of the initrd, it should be possible to overcome this problem though.

Performance of Virtualbox is good. It feels at least as fast as VMWare, so there are no bad surprises here. Virtualbox is more of a workstation virtualisation product though. Unlike VMWare Server, you cannot run virtual machines in the background, and connect to the virtualisation server from the network. At least, I did not see this functionality.

So, in the end I have to say I like Virtualbox a lot! It has a lot of advantages to VMWare Server: it has better sound support, better time keeping, creation of snapshots and generally is a bit easier to install and configure. And it installs Fedora 7 without crashing my machine! If you need USB support or a client-server virtualisation solution, you still have to take a look at VMWare Server though.

Good things ahead!

Today I got an account on Mandriva’s build cluster! This means it will be easier for me to submit RPM packages for inclusion in the distribution. I’ve still got a lot to learn, but with some reading on the wiki and the greatly appreciated help from Dvalin, this will work out fine in the end. Currently working on a package for DrScheme, which is a Scheme IDE also used at university here.

Virtualbox released version 1.4.0 of their virtualisation software today. Especially interesting is that they added AMD64 support according to the changelog. This will probably fix the problems I was experiencing a few days ago when trying Virtualbox on my Athlon 64 machine.

At work, I’m currently installing a nice new server consisting of four dual core Opteron CPUs with 16 GB of memory :-) It will be used for running virtual machines (not with Virtualbox, but OpenVZ). Also a new version of the Linux clustering software Kerrighed was released, which I should definitely try out on one of the clusters at work, because the previous version was not much of a success (it just crashed when activating the cluster).

Other good news, I finally fixed my summer holidays. Now I really should start planning what I will do then. Gentse Feesten will of course be high on the list :-)

Virtualisation mess

I have downloaded the Fedora 7 installation DVD ISO and wanted to give it a try in a virtual machine. I am using VMWare Server already for some time as it was the first free (as in free beer) available feature-complete and fast virtualisation software. I Created a virtual disk, configured the ISO as source for the CD device, and started up the virtual machine. But then while booting the Fedora 7 kernel, VMWare just crashed, also making my host OS unstable, so I had to do a hard reset. I was still using VMWare Server 1.0.1, so I tried an upgrade to 1.0.2 with latest vmware-any-any patch, but all to no avail: VMWare just keeps on crashing.

Now there’s also Virtualbox, which is freely (as in free speech!) available, so this seemed like an excellent time to give it a try. Virtualbox is packaged for Mandriva, so urpmi virtualbox should suffice to install it. It automatically installs some dkms-virtualbox package, probably containing drivers for virtual network cards and such, like VMWare does too. But while compiling these modules, it bombed out with some compilation errors, and a warning that Virtualbox is not tested with kernels > 2.6.17. As I’m using 2.6.21 x86_64 tmb kernel, and I did not immediately find a reference to this error on Google, I’m stuck here I’m afraid. Let’s hope new versions of VMWare Server or Virtualbox fix these issues soon. In the meantime, I’ll continue to use VMWare Server.