[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: news from the Debian Beowulf project?

Adam C Powell IV a écrit :
On Fri, 2007-04-20 at 00:41 +0200, Fabrice Lorrain wrote:
Adam C Powell IV a écrit :

On Sun, 2007-04-15 at 09:35 +0200, karel lang wrote:
i'd like to ask, if there are some howto's or materials like that of newer date about installing and maintaining the Beowulf project on Debian Linux.
As far as I know, the most current Debian-specific howtos are on the
Debian Wiki: start at http://wiki.debian.org/?DebianBeowulf .  There are
many Debian-specific enhancements such as update-cluster, FAI, dsh, etc.
which make facilitate cluster management on Debian.  There is of course
still room for improvement, e.g. generating dhcp server config files
using update-cluster (bug 156810); ideas and patches are always welcome.

Unfortunately, since the Beowulf Wiki pages were written (mostly by
yours truly), the diskless package has been orphaned, so that section is
valid through sarge but not for etch.  Would someone be interested in
redoing that section for lessdisks -- or in re-introducing diskless?
(Though the latter will not help with etch.)

I would suggest to have a look at Daniel Baumann & all work on debian-live [1] before (re)starting any work on {less}disks{less} packages.

The infrastructure (debian-live + etch initrd + unionfs) provided to build live cd makes it trivial to build diskless environnements. See the wiki for the technical informations.

As debian-live is here to stay, regrouping human effort there seems more effective than splitting it over several overlapping package.

Excellent point Fabrice.  It seems there's also an LTSP howto on the
Wiki.  I've marked DiskLess as deprecated.

On the other hand, diskless (and I presume lessdisks and LTSP, though
I'm still using sarge diskless packages on etch) has a lot of very nice
features for minimizing the hard drive space of a cluster (by not
repeating /usr, /bin, /sbin, /var/lib/dpkg etc. over the various client
images), nice maintenance tools, etc.  For example, chroot into one
place, apt-get update dist-upgrade install remove etc. to update the
main chroot, then exit the chroot, and run a simple script which updates
all of the images' /etc and /var directories (outside of /var/lib/dpkg).

<disclamer: I'm not a user of the cited packages, but I've a descent understanding of how they work/>

I agree with you that minimizing disk storage is important. Chroot + aptitude {update,upgrade} (apt-get should die). is a must have
( = simplicity of maintenance).

Debian-live is using unionfs [1] which allow to mount-merge a ro branch with a rw one. Concretly, it meens you export your /srv/etch-root_chrooted to your cluster nodes and in the initrd you merge it with a rw tmpfs where you can put the few files you need to modify.

In short, you have a single image for all your client. You can maintaine it with the chroot/aptitude upgrade trick.

Debian Live seems geared toward providing an image for a single client
machine for a static image, so I don't know if that's what Beowulf
people really want.  Is it possible to update?  How can one copy over
multiple images, while preserving their separate configurations?

My personnal experience, but for workstations (not a cluster environnement) is that there is very few files to manage hardware/environnement differences.

If you really need to keep separate configuration for each node, you can maintain them on the server-side (ip or hostname directory tree) and merge them in the tmpfs/unionfs on the client during the initrd initialisation.

FYI, LTSP, knoppix and quite a lot of other livecd-distrib have been using unionfs or it's half-fork AUFS for more than a year know.


[1] http://en.wikipedia.org/wiki/UnionFS
[2] http://aufs.sourceforge.net/

Reply to: