Linux kernel development thoughts

One week ago, kernel hacker Ingo Molnar reviewed Con Kolivas’ swap prefetch patches and approved them to be added to the official Linux kernel. Swap prefetching is a technique, which will load swapped out memory pages back in memory if the system is idle and memory has become available. This is useful for people starting temporary jobs which use a lot of memory, which makes other processes move to swap. Once the memory hungry process is finished, swap prefetching will kick in and slowly reload the swapped out pages back to memory, so that the system potentially does not need to do this anymore when the user again uses one of the swapped out processes. Tjos functionality can be enabled and disabled at will during compilation of the kernel.

Swap prefetching has been available for a long time in Con Kolivas patch set, and was also added to the mm development kernels some time ago. In that period, no bugs have been reported, and it seems people are happy with this feature. So it was hoped that the push by Ingo Molnar, would finally make swap prefetching available for all Linux users in version 2.6.22.

Developer Nick Piggin seemed rather critical of the swap prefetching feature. I I have understood the thread correctly, he discovered a problem (actually swap prefetching did not seem to work anymore because of some unrelated changes in Linux), and there was some serious disagreement how it should be dealt with. Con Kolivas’ got fed up with the unreasonable objections to the patch and proposed to dump the patch completely. So this is yet another performance improving feature we won’t be seeing in the Linux kernel for some time…

Update 12 May 2007: Con Kolivas posted an updated version of swap prefetch, addressing some problems, and the patch has not been dropped from the mm kernel. Maybe things do not look so bad after all?

This whole story makes me remind of the recent RSDL/SD scheduler. Con Kolivas implements a feature which is clearly working very good for most people, and then someone comes up who vetoes it for whatever reason, and in the case of SD other implementations are started (which still do not have the same maturity as SD). In the end a lot of work seems to duplicated and interesting features are delayed or even cancelled completely. Is it me, or is the behaviour of some kernel developers really hurting Linux development?

Also in related news, kernel hacker Adrian Bunk decided to not track regressions anymore. He was tired of Linux 2.6.21 being released with lots of known regressions. It seems Linux kernel development has really some issues these days…

Mandriva work this week

This week I was contacting the mailing lists for various upstream projects to find fixes for some problems.

First there was the problem that F-Spot does not automatically handle the upgrade from sqlite2 to sqlite3. In the past, F-spot used sqlite2 as its database, today it uses sqlite3 by default, but you can still continue to use your sqlite2 database. That is, if sqlite2 libraries are installed on your system. And one can imagine that in the future, distributions will stop shipping sqlite2, because it is not maintained anymore and because sqlite3 is far superior. It is possible to upgrade from sqlite2 to sqlite3 by upgrading the db at the command line, but we really cannot expect normal users to do this, do we?

In Pan, I had the problem that new headers were not appearing in a newsgroup, until I killed the “get new headers”-task by hand. I found out it was caused by a news server I had configured in Pan, which was not reachable. I filed a bug.

I experienced a crash in ogginfo when some wrongly encoded characters are found in the Vorbis tag, for which I found a patch with Google. I filed a bug but I’ll need to investigate if the tag is really invalid: Easytag shows it correctly without any problem.

Another problem I experienced already some time ago, but which I never really investigated, was that Amarok did not add files on a read-only Samba share to its library. Thanks to Debian changelog mailing list, I discovered today that there was a patch for libtag fixing this. This is a real serendipity!

I did again some testing with Kaffeine. I contact its mailing list for the problem that it copied everything to ~/tmp when using system:/ URI in Konqueror. I found a fix now, but I’m still not impressed by Kaffeine’s stability. For now, you’re better of continuing to use kmplayer.

I contacted Mandriva’s web discuss mailing list (which does not seem to be archived unfortunately) for a new structure of the End user documentation on the Mandriva wiki and posted a proposed for some documentation about the tmb kernel. I’m a bit disappointed by the reactions: like this we won’t succeed in creating a very active wiki community with lots of useful user documentation. I’ll try to implement a proposal in my sandbox this weekend: practical work is often better than theoretical discussion

Mandriva 2007 Spring released, looking forward to 2008

Now that Mandriva 2007 Spring has been released and the corrupted Mandriva Cooker subversion repository has been fixed, development for Mandriva 2008 has started now.

Some things which I have on my wishlist for Mandriva 2008:

  • New kernel, Linux 2.6.22 or later which I hope will give us these things: CFQ IO scheduler as default (since Linux 2.6.18), better task scheduler, probably CFS, better wireless drivers (bcm43xx is stable in 2.6.20, but it’s not in Mandriva’s 2.6.17, finally good Ralink drivers: complete documentation exists, but drivers are still not production quality)
  • Mandriva still uses OSS drivers for some sound cards which are perfectly supported with Alsa (for example for some Intel ICH2 810 AC97 cards, which was not fixed in 2007.1 as far as I know). This is annoying as OSS does not have sound mixing, and can cause applications complain about occupied /dev/dsp and it also causes missing sound in Flash. Fedora even does not ship OSS anymore for some time already. A maximum of cards should be switched to Alsa now early in Cooker development , so that things get tested.
  • slocate is still being installed by default by rpmsrate instead of the much performance friendlier mlocate (already in 2007.1 Main). We should make the switch now in rpmsrate so it’s the default in 2008.
  • Almost a year ago, I proposed a new program menu structure for Mandriva. According to Gnome and menu maintainer Frédéric Crozat, it would be implemented for 2008. Let’s hope this becomes true, as it will make the menu at the same time friendly for new users, and more practical for power users. Unfortunately, I lost the complete original document (what a shame!). If somebody still has it somewhere, I would be grateful if you could send it to me!
  • Drop esound, and use Pulseaudio instead. Pulseaudio is a more modern and actively maintained sound server. There was some discussion about it some time ago, and I still hope Pulseaudio will not be forced on all users (personally I prefer not to use a sound server, and let all the mixing be done by Alsa. For this reason, I still hope more and more applications will use GStreamer, as GStreamer makes it easy to choose your preferred output method, such as Alsa or Pulseaudio for all applications. But Pulseaudio should certainly replace esound in main.
  • Test new Kaffeine: In Mandriva 2007.0, Kaffeine was replaced by kmplayer as the default video player in KDE, because Kaffeine was unstable paying embedded videos in Konqueror. Now that both Xine and Kaffeine support XCB, this problem should be fixed. We should test if it is indeed stable now, and if Kaffeine plays supports all kinds of embedded videos in web pages like kmplayer. If this is the case, we should consider moving back to Kaffeine, as its interface is a bit better than KMplayer’s.
  • Mandriva’s net_applet and mdkonline system tray utilities still poll too much, which causes unnecessary CPU usage, and which also eats battery life on laptops. These applications should be scrutinized a bit more, so they are completely idle most of the time.
  • Up to date printing support: Unfortunately printing support did not really advance in Mandriva 2007 Spring. The drivers are mostly still the same as in 2007.0, and some newer printer models supported by newer versions of hpijs and other models which appeared on linuxprinting.org, are not support in 2007 Spring. Mandriva 2008 should really have up to date printing support, like was always the case in previous versions.
  • Cleaning up the repositories: the Main repository now contains a lot of programs which are not used by a lot of people (things like flphoto, uucp, uw-imapd, xpdf,…). On the other hand, some very interesting packages are in Contribs (Transmission, dovecot,…). The contents of both repositories should be looked at, in order to clean up, and put packages in the appropriate repository.
  • We need a simple application installation program: rpmdrake has improved a lot in 2007.0 and now in 2007.1 Spring. Still it is too complex for new users. Suppose a user wanting to search a DTP application. He should easily find Scribus. Well, Scribus does not even appear in the Publishing category in rpmdrake. Instead, these category contains mostly technnical console programs not suitable for beginners. We should have something like Ubuntu’s gnome-app-install which only shows userfriendly graphical applications.
  • More and easier to find documentation: Mandriva already comes with a lot of documentation, but the included handbooks are difficult to find (being hidden in More Applications). They should be in a more obvious place. On the other hand, we should make sure more documentation becomes available in different languages on the wiki, and the wiki should be easy to find (include a link on the desktop?). This way, users will know where to look for help.
  • Related to the menu structure and the reorganization of the repositories, we should constantly be evaluating which applications should be in the default installation and which not. We had such discussion shortly before 2007.1 Spring went out, but this was a bit too little too late. We should make sure we start with this process before the first beta is released, so we have more time to evaluate and stress test programs.

Currently I already started by taking a quick look at the contents of the main repository. Maybe other people are interested in participating in this effort, so that it can happen a bit more organized?