Last week-end, Debian Lenny 5.0 was finally released. I use Debian on most servers I manage at work. A few of them were already using Lenny when it was still the testing branch, but most are still on Debian Etch. So this morning I decided to test upgrade one of the less critical Etch systems to Lenny. That system is only used to store back-up files from other systems, so it would not be a problem if that machine was off line for a couple of hours.
According to the release notes, you should rather use aptitude instead of apt to upgrade, so that’s what I did. All went well, until suddenly the package upgrade hung while installing new udev configuration files. I could Ctrl-C the process to continue, but from that moment on, more and more post installation scripts started hanging and had to be interrupted.
I noticed that also simple commands, such as ps and getent passwd were hanging too and that I could not log in via SSH anymore. Fortunately, the existing SSH connections continued to work, so I was not locked out yet.
I straced getent passwd and noticed that it hung while trying to connect to the remote LDAP server. The problem was apparently that Lenny’s libnns_ldap tried to connect via LDAPS to port 389, while LDAPS uses port 636 by default. It seems that you need to specify the port number 636 now to make LDAPS work right, so for example: ldaps://remote.host:636. I fixed this in libnss-ldap.conf and pam_ldap.conf, and then I could finish the upgrade without any problem. Apparently this is a known problem.
So definitely modify your configuration first if you are using LDAP authentication and want to upgrade to Lenny. I should probably also fix my nsswitch.conf so that applications don’t start to hang if the LDAP server is unreachable…
In spite of this problem, the whole upgrade was done in less than 1 hour. Without that problem, I guess it would have taken about 20 minutes less. Quite impressive!
Good hacking there! I’m kind of surprise that no one has sent a patch out for this yet.
I don’t mean to advertise, but this is a great piece of software if you have largescale LDAP deployments on Linux: http://code.google.com/p/nsscache/ .. it makes my workstation a happier place.
You just made my day with nsscache.
I guess I should have mentioned what nsscache is.. it’s like nscd, but a bit more aggressive with caching and a lot less buggy.