Clamav is great

Posted on by Frederik

Like a lot of people, I use the free anti-virus program Clamav on my mail server. Last week, I was seriously impressed with its performance.

It started last wednesday, 25 July. At about noon, I received a mail by amavisd-new that it had blocked an e-mail containing a virus, Trojan.Downloader-11827. What was strange, is that I received this message on an e-mail account which is protected by my ISPs proprietary anti-virus solution. So it had not caught this virus, while Clamav did. Then I submitted the file to virustotal.com, and apparently only a few (about five) anti-virus programs detected the virus. Amongst others, Kaspersky, F-Secure, NOD32, Bitdefender, Symantec and of course Clamav. In the clamav-virusdb mailing list archives, I found that Clamav had detection for this virus since 7h21 CEST, so it was really amongs the first to detect this virus.

Then friday evening, I was looking at the blocked spam messages (I use spamassassin too on this server), and noticed that it had blocked an e-mail message containing an exe file. A spam message with an exe file, that sounded suspicious, but Clamav could not detect a virus. Again I submitted the file to virustotal, and there was one positive result: Ikarus detected it as a trojan horse. I submitted the file via clamav’s website at around 19h CEST. About half an hour later, I received a message that detection for this virus had been added. I updated Clamav, and indeed, it was recognized. I checked the file again on virustotal, around 20h, and then there were 4 anti-virus programs recognizing it: Clamav, F-Prot Ikarus and Virusbuster. 1.5h later, Antivir, AVG and Kaspersky had also added detection. Other well-known anti-virus vendors, still did not detect it at that moment such as Bitdefender, F-Secure, NOD32, Panda, Sophos and Symantec.

In the meantime, F-Secure blogged about these two virus outbreaks: funny.zip and fungame.zip

Two conclusions:

  • Clamav has an excellent response time, which is comparable to the best proprietary anti-virus solutions. If you have an e-mail server, you definitely want to integrate Clamav in it, even if you already have a proprietary solution (Clamav is particulary good in detecting phishing mails too!)
  • No anti-virus program is perfect. If you receive an e-mail message at the start of a virus outbreak, it’s quite possible that your anti-virus solution will not detect it yet, no matter which anti-virus you have.

Mandriva is not negotiating a patent deal with Microsoft

Posted on by Frederik

Several sites are spreading unfounded rumours that Mandriva will be the next one signing a patent deal with Microsoft, after Novell, Xandros and Linspire. Adamw, a Mandriva employee answered to the rumours on the Mandriva Cooker IRC channel today:

09:32 < AdamW> sander85: there are no plans to do a deal with microsoft,
and that comes from the top (fb)

(fb is probably François Bancilhon, Mandriva’s CEO).

Can we please stop spreading pointless rumours, and get back to real work and news please?

Update: Official statement

The end of the CK kernel patch set

Posted on by Frederik

Today kernel developer Con Kolivas announced that he will stop developing his Linux patch which improves desktop performance. For people who have followed recent discussions about his SD CPU scheduler and about the inclusion of his swap prefetching patches in the Linux kernel this will not come as a surprise.

The CK patch set was popular especially amongst desktop users who want to get maximum performance out of their machine. The CK kernel came with a different CPU scheduler (first Staircase, later SD), which improves the smoothness of desktop applications (for example no more sound stuttering), the mapped watermark patches, which makes the OS use less swap, and the swap prefetching patches, which makes the system more responsive after a memory hungry application caused others to be temporarily swapped out. The CK patch set was also used in several distro kernels, such as the Mandriva’s tmb kernel and kernels in Gentoo and Arch Linux.

The decision to completely stop kernel development, came after the critical reactions by other kernel developers about the SD scheduler and swap prefetching. After the first releases of the SD kernel, some developers preferred trolling instead of helping out to fix the problems which existed at that time. While the SD scheduler slowly became more and more stable, only thanks to Con Kolivas efforts, a competing scheduler (CFS) which was based on the same concepts, was started. Now that both schedulers are mature and stable, a lot of CK kernel users and Con Kolivas himself are left wondering why it was even necessary to start competing with SD, instead of uniting all powers to make one great scheduler.

Swap prefetching was already proposed for inclusion in the Linux kernel a long time ago. But several developers remained critical, while a lots of users reported improvements by these patches. The patches were included in the mm kernel, but developers did not really review it and proposed it for the mainline kernel. Until Ingo Molnar finally stepped up recently, and gave some positive comments after a code review. Again some developers started criticizing the patch, and the future of this patch became again unclear.

With all this in mind, it’s normal that Con Kolivas got fed up with Linux kernel development. It seems some Linux developers really need to do something to improve their communication, and need to be a bit more reasonable and constructive, instead of immediately criticizing one’s efforts. This is at least the second kernel developer who got fed up with the way the Linux kernel development goes in a short time.

Developers come and go, that’s a normal process. Still I think Con Kolivas’ departure could have been avoided. In the end, we can only thank him for his great work, which certainly was not useless. In the end, the CFS scheduler which will be included in Linux owes a lot to Con Kolivas’ ideas, and I hope the other patches will find their way to inclusion in other patch sets in way or another.

Resistance is futile, you will be packaged!

Posted on by Frederik

Today, I had again the honour to work with an operating system which is not based around a package manager. The victim: Mac OS X Server. It’s a brand new Mac Pro machine being used as a mail and web server.

Mac OS X Server already comes with most software for configuring a web and mail server included, and has graphical configuration tools. Postfix, Cyrus, Amavisd-new, Spamassassin, Clamav, Apache HTTPD, etc, are all there by default, and easy to configure. Sounds great? Wait a minute…

The problem is that the versions included, are really old, even completely outdated. Let’s take Apache. The version included is some 1.3 version. If you need Subversion running on Apache with Webdav support, then you need at least Apache 2.0… Clamav? the included version is some 0.88 version, which cannot use today’s virusdb updates anymore. That makes Clamav completely useless… Spamassassin? You have the outdated version 3.0.1, hardly impressive if you need to filter’s todays spammer’s creations.

So to make your system really useful, you have to compile a lot of programs by hand. On the system there was Macports installed, and Perl version 5.8 (not sure if it came like this by default, or someone else installed these on this machine before I touched it). So I installed Spamassassin with Perl 5.8 CPAN. All went fine. Let’s restart amavisd, and it will be using the new spamassassin, right? Wrong! Amavisd-new itself is a Perl program, and does not use the spamassassin or spamc binaries, but directly accesses the Spamassassin Perl module. amavisd-new was still using Perl 5.6 as installed by default in Mac OS X, while Spamassassin used Perl 5.8 from Macports, so amavisd-new only found the old Spamassassin in @INC. That should not be too difficult to fix: let’s just change the shebang in amavisd-new, so it uses Perl 5.8 in /opt/local/bin. I restart amavisd, and got a lot of errors of missing Perl modules. By trial and error (read: installing dependency, trying to start amavisd, getting new error, installing dependency,….), I succeed in the end in installing all its dependencies, and amavisd starts fine now. A bit later, new mail starts arriving, and this causes weird errors in the amavisd log (something about wrong file handles). Huh? Well, the amavisd-new included in Mac OS X is very old (from 2004 or 2005 if I remember correctly). Maybe it simply does not work with Perl 5.8?

So now I had to upgrade amavisd-new too… Fortunately some great documentation on the web helped me a lot. Again I had to install some Perl dependencies with cpan, I had to patch amavisd-new for Mac OS X as instructed in the guide, and I had to recreate a new amavisd.conf file. But in the end, I finally had a working amavisd-new installation.

But we don’t have finished yet! Now it seems mails are not scanned anymore with Razor2, although it is installed by default in Mac OS X and I have activated it in my Spamassassin 3.2 configuration… Well it’s the same problem again: Razor2 is installed in Perl 5.6 @INC, but not in Perl 5.8. So again I had to grab the sources and install it by hand, to make it work. While at it, I also compiled Pyzor and dcc-client. And I created a little cronjob wich uses sa-update to grab new rules from SARE.

So, after several hours of work, I think I finally have an adequate working spam filtering system on Mac OS X Server. On an operating system with a good package manager and enough available packages, such as Debian or Mandriva, this would have cost me about an hour at most. Operating systems like Mac OS X, Slackware and others which lack a complete and well integrated packaging system and ditto repositories, really make this a terrible experience. Avoid them if you can!

Virtualbox 1.4

Posted on by Frederik

Only one week after I had no success with running Virtualbox on my Athlon 64 system, a new version was announced. One of the important changes in Virtualbox 1.4 is support for AMD64 hosts, so this seemed exactly what I was looking for! To test new distributions and software, I have already been using VMWare Server for some time, which is free (read: it costs nothing), but a real Free (as in free speech) virtualisation solution always sounds interesting, especially as Fedora 7 always crashed VMWare Server and my host.

Installation of Virtualbox was very easy. It has been packaged and integrated in Mandriva, so a simple “urpmi virtualbox” sufficed to install it. Already a lot easier than VMWare Server, which comes in different RPM and ZIP files you have to download and extract. There was no hassle with licences, as Virtualbox is released under the GPL unlike VMWare Server for which you need to register on the site to request a licence key.

The kernel modules for Virtualbox were automatically built with dkms. This time, there were no problems with my x86_64 2.6.21-tmb kernel! Again this was easier than in VMWare, which often needs the installation of an extra patch if you are running a recent kernel.

Configuration is a bit different than VMWare, but actually very easy. The only thing which seems more complex than VMWare, is configuration of bridged networking, i.e. if you want to integrate your virtual machine directly in your real network like a real physical machine. According to the documentation it requires some manual bridge configuration on the host, but I did not try this. For simple NAT networking, I had not to do anything, this worked out of the box and was sufficient for me.

Virtualbox supports everything you would expect from a modern virtualisation system: ACPI, networking, cd/dvd drives (you can access a physical drive or use an ISO file, like VMWare) and sound. The sound implementation in Virtualbox is even better than VMWare, as it can use both OSS and Alsa. With VMWare I never succeeded in having working sound, because I’m using Alsa, and VMWare always complained that /dev/snd was in use. With Virtualbox and Alsa, everything is working great now. Virtualbox also supports creation of snapshots. In VMWare Server you can only create one snapshot, if you need to create more, you have to pay for another edition. Did I say that Virtualbox has everything you would expect? Well, maybe that’s not true. There’s one important thing missing: unfortunately it does not have USB support. This is an important omission which I hope will be added soon, as this works great in VMWare.

Unlike VMWare, Virtualbox does not have any problem with the fact that I am using frequency scaling on my processor (AMD’s Cool’n’Quiet with the powernowd daemon in Linux). In VMWare I had to disable frequency scaling, otherwise the clock of the virtual machine went too fast or too slow most of the time. But not with Virtualbox!

Virtualbox uses a nice QT interface, which integrates very well in a KDE environment. I don’t like QT’s open and save dialogs too much, but as this is a virtualisation product, and not a document editor, you won’t need these too much, so I can live with that. Virtualbox can use VMWare images, but unfortunately it is still not so easy to import your VMWare virtual machines as the virtual hardware is different. My Mandriva 2007 Spring installation in VMWare did not succeed to mount the root partition in Virtualbox, because of the different hard drive controller. With a rescue CD and some manual regeneration of the initrd, it should be possible to overcome this problem though.

Performance of Virtualbox is good. It feels at least as fast as VMWare, so there are no bad surprises here. Virtualbox is more of a workstation virtualisation product though. Unlike VMWare Server, you cannot run virtual machines in the background, and connect to the virtualisation server from the network. At least, I did not see this functionality.

So, in the end I have to say I like Virtualbox a lot! It has a lot of advantages to VMWare Server: it has better sound support, better time keeping, creation of snapshots and generally is a bit easier to install and configure. And it installs Fedora 7 without crashing my machine! If you need USB support or a client-server virtualisation solution, you still have to take a look at VMWare Server though.

Good things ahead!

Posted on by Frederik

Today I got an account on Mandriva’s build cluster! This means it will be easier for me to submit RPM packages for inclusion in the distribution. I’ve still got a lot to learn, but with some reading on the wiki and the greatly appreciated help from Dvalin, this will work out fine in the end. Currently working on a package for DrScheme, which is a Scheme IDE also used at university here.

Virtualbox released version 1.4.0 of their virtualisation software today. Especially interesting is that they added AMD64 support according to the changelog. This will probably fix the problems I was experiencing a few days ago when trying Virtualbox on my Athlon 64 machine.

At work, I’m currently installing a nice new server consisting of four dual core Opteron CPUs with 16 GB of memory :-) It will be used for running virtual machines (not with Virtualbox, but OpenVZ). Also a new version of the Linux clustering software Kerrighed was released, which I should definitely try out on one of the clusters at work, because the previous version was not much of a success (it just crashed when activating the cluster).

Other good news, I finally fixed my summer holidays. Now I really should start planning what I will do then. Gentse Feesten will of course be high on the list :-)

Liberation fonts

Posted on by Frederik

I quickly redid some of the font settings in the CSS file of this blog. This blog is now using the Liberation fonts!

Packages for these new True Type Fonts are available for all kind of OSes. Those using Mandriva Cooker, can install the fonts-ttf-liberation package with their favourite package manager (urpmi, rpmdrake, smart). Mandriva 2007.1 Spring users, can download this backported RPM package (SRPM available too).

I also changed the font size used in the blog a bit. The template was often using small, x-small and even xx-small fonts, which was a bit too small for my taste. Now it should all be a bit more readable I hope. Let me know what you think of it!

Virtualisation mess

Posted on by Frederik

I have downloaded the Fedora 7 installation DVD ISO and wanted to give it a try in a virtual machine. I am using VMWare Server already for some time as it was the first free (as in free beer) available feature-complete and fast virtualisation software. I Created a virtual disk, configured the ISO as source for the CD device, and started up the virtual machine. But then while booting the Fedora 7 kernel, VMWare just crashed, also making my host OS unstable, so I had to do a hard reset. I was still using VMWare Server 1.0.1, so I tried an upgrade to 1.0.2 with latest vmware-any-any patch, but all to no avail: VMWare just keeps on crashing.

Now there’s also Virtualbox, which is freely (as in free speech!) available, so this seemed like an excellent time to give it a try. Virtualbox is packaged for Mandriva, so urpmi virtualbox should suffice to install it. It automatically installs some dkms-virtualbox package, probably containing drivers for virtual network cards and such, like VMWare does too. But while compiling these modules, it bombed out with some compilation errors, and a warning that Virtualbox is not tested with kernels > 2.6.17. As I’m using 2.6.21 x86_64 tmb kernel, and I did not immediately find a reference to this error on Google, I’m stuck here I’m afraid. Let’s hope new versions of VMWare Server or Virtualbox fix these issues soon. In the meantime, I’ll continue to use VMWare Server.

Bug fixing progress

Posted on by Frederik

Since a few days, the bug triaging process is in full force. The purpose is to review all old, unconfirmed bugs, and verify if they are still valid. And of course, making sure those valid bugs are fixed. I have the impression that it’s really a great success. Lots of Bugzilla e-mails are arriving in my mailbox, and looking at changelog mailing list, several old bugs are finally getting fixed.

I started to concentrate on bugs in Kaffeine and KMplayer. I think Kaffeine is now as stable as is possible now. That means: far from perfect, but I think all remaining problems should now be handled by its upstream authors. Already since some time, Kaffeine has XCB support, which should make it a lot more stable when viewing embedded videos in the Konqueror web browser. And since this weekend, Kaffeine correctly disables the screen saver when you are viewing a full screen video, and won’t copy the whole media file you open from media:/ URIs to your home directory. I proposed an update for Mandriva 2007.0 and 2007.1 Spring to fix the last two issues. Fixing the first one, is unfortunately a lot more difficult for older Mandriva versions, because it requires a more recent version of xine, and depends on libxcb, which is in contribs…

KMplayer was actually in a much better state. The only real serious issue, is that KMplayer does not add itself to KDEs service menus which appear when you insert a DVD disk. This is rather serious as KMplayer is Mandriva’s default video player, and thus user’s don’t get an option to actually play the movie when they insert a DVD disk. This should not be too difficult to fix though, so let’s hope we’ll have a fix soon.

I also reviewed Kopete bugs. About all of them are fixed in current Cooker, possibly we’ll also have udpated packages for 2007.0 and 2007.1 Spring implementing a higher framerate for MSN webcam support, fixing decryption of gpg encrypted messages (2007.1 only) and fixing errors when chatting on conference.jabber.org (2007.0 only).

For my own use (and pleasure), I also recompile a lot of Cooker packages for 2007.1. While recompiling a more recent version of libxml, I had problems with python crashing. After a lot of trying to find out what causes it, I found out that python-reportlab as shipped in both 2007.1 and Cooker makes python crash. Python-reportlab is a dependency of the hplip printer drivers, so if you don’t have these installed, you’ll probably never notice this problem. Ia also makes certain hplip utilities crash in 2007.1 and Cooker. Compiling a more recent version of python-reportlab, fixed all problems, so I hope we will see this as an update for 2007.1 soon.

Next on the bug review list, will be cd burning applications. Thanks to the article I wrote on this blog about audio cd burning applications, I already tested some of them, which always helps in reviewing bugs.

If you always wanted to get involved in free software development, this is actually a great opportunity to join the Mandriva bug reviewing effort! You don’t really need to have development skills, just some willingness to compare different bugs, to try to reproduce them, and to look them up in other bug trackers. Often those bugs are really already fixed, it’s just that someone has to confirm they actually are, or provide the good patch or solution which has been created by the upstream developers. If you are interested, do not hesitate to contact us on the bugteam mailing list!

Disk encryption in Mandriva

Posted on by Frederik

Last weekend, I bough an external USB hard drive to finally start regularly making back-ups of my computers at home. For security reasons, I wanted to store back-ups on an encrypted partition, because one never knows what may happen. Using an encrypted partition was not too hard, but still some bugs prevented it from being newbie proof.

For the encrypted partition, I chose to use LUKS. It seems to be some kind of standard, widely supported (by Hal for example), and it has some graphical utilities available which should make it a no-brainer.

First problem, I’m using x86_64, and apparently luks-tools only exists for i586 in Mandriva. I filed a bug, and a new luks-tools package for x86_64 should already appear very shortly on a Cooker mirror near you. If you have luks-tools installed, it should be as simply as running gnome-luks-format to set up an existing partition.

So I did it at the console. Of course, make sure you have the package cryptsetup installed, otherwise you won’t have the necessary utilities. First we’ll add a header to the partition, indicating that this is a LUKS encrypted partition, and which encryption type we are using:

cryptsetup -c aes-cbc-essiv:sha256 luksFormat /dev/sdb1

(as I already mentioned with gnome-luks-format, you should create the partition first, for example with diskdrake or cfdisk).
Once the header is in place, we’ll open the encrypted partition:

cryptsetup luksOpen /dev/sdb1 encbackup

Encback is a name you can freely replace by your own choice of course. Once you have done this, your encrypted partition will be mapped to /dev/mapper/encback. You can use this like any normal partition for example to create an ext3 file system on it:

mkfs.ext3 /dev/mapper/encback

Then we can mount it:

mount /dev/mapper/encback /media/encrypted-backup

And we’re ready to go. To close the encrypted device after I have umount it, I run

cryptsetup luksClose encback

When you plug in the USB disk, hall will automatically detect you have a LUKS encrypted partition. A dialog will appear, where you can enter your passphrase, and after that it is mounted automatically. At least, that’s the theory. Unfortunately, this was not working, probably because of bug #30015. Let’s hope this gets fixed soon! Also note that KDE does not seem to support this at all, so even then this is Gnome only. For now I have created two small scripts, which will run cryptsetup and mount or umount, so I don’t have to retype these commands by hand each time.

I opened another bug to request LUKS support in diskdrake. Disk encryption is becoming more and more common these days with all those portable storage media, so Mandriva’s default partition tool should have this support built in. I guess it should not be too hard to implement this.

Some nice documentation which helped me a lot during this exploration, can be found in Red Hat Magazine.

Update 22 May 2007: I updated the howto to use aes-cbc-essiv:sha256 instead of aes-cbc-plain. Reader David Crick pointed me to the fact that aes-cbc-plain is vulnerable to a cryptography weakness, which is called the “watermarking attack”. Thanks!