Re: news from the Debian Beowulf project?
On Tue, 2007-04-24 at 00:28 +0200, Fabrice Lorrain wrote:
> Adam C Powell IV a écrit :
> > On Fri, 2007-04-20 at 00:41 +0200, Fabrice Lorrain wrote:
> >> Adam C Powell IV a écrit :
> >>> Greetings,
> >>>
> >>> On Sun, 2007-04-15 at 09:35 +0200, karel lang wrote:
> >>>> hello,
> >>>> i'd like to ask, if there are some howto's or materials like that of
> >>>> newer date about installing and maintaining the Beowulf project on
> >>>> Debian Linux.
> >>> As far as I know, the most current Debian-specific howtos are on the
> >>> Debian Wiki: start at http://wiki.debian.org/?DebianBeowulf . There are
> >>> many Debian-specific enhancements such as update-cluster, FAI, dsh, etc.
> >>> which make facilitate cluster management on Debian. There is of course
> >>> still room for improvement, e.g. generating dhcp server config files
> >>> using update-cluster (bug 156810); ideas and patches are always welcome.
> >>>
> >>> Unfortunately, since the Beowulf Wiki pages were written (mostly by
> >>> yours truly), the diskless package has been orphaned, so that section is
> >>> valid through sarge but not for etch. Would someone be interested in
> >>> redoing that section for lessdisks -- or in re-introducing diskless?
> >>> (Though the latter will not help with etch.)
> >> Hello,
> >>
> >> I would suggest to have a look at Daniel Baumann & all work on
> >> debian-live [1] before (re)starting any work on {less}disks{less} packages.
> >>
> >> The infrastructure (debian-live + etch initrd + unionfs) provided to
> >> build live cd makes it trivial to build diskless environnements. See the
> >> wiki for the technical informations.
> >>
> >> As debian-live is here to stay, regrouping human effort there seems more
> >> effective than splitting it over several overlapping package.
> >
> > Excellent point Fabrice. It seems there's also an LTSP howto on the
> > Wiki. I've marked DiskLess as deprecated.
> >
> > On the other hand, diskless (and I presume lessdisks and LTSP, though
> > I'm still using sarge diskless packages on etch) has a lot of very nice
> > features for minimizing the hard drive space of a cluster (by not
> > repeating /usr, /bin, /sbin, /var/lib/dpkg etc. over the various client
> > images), nice maintenance tools, etc. For example, chroot into one
> > place, apt-get update dist-upgrade install remove etc. to update the
> > main chroot, then exit the chroot, and run a simple script which updates
> > all of the images' /etc and /var directories (outside of /var/lib/dpkg).
>
> <disclamer: I'm not a user of the cited packages, but I've a descent
> understanding of how they work/>
>
> I agree with you that minimizing disk storage is important. Chroot +
> aptitude {update,upgrade} (apt-get should die). is a must have
> ( = simplicity of maintenance).
>
> Debian-live is using unionfs [1] which allow to mount-merge a ro branch
> with a rw one. Concretly, it meens you export your
> /srv/etch-root_chrooted to your cluster nodes and in the initrd you
> merge it with a rw tmpfs where you can put the few files you need to modify.
>
> In short, you have a single image for all your client. You can maintaine
> it with the chroot/aptitude upgrade trick.
This is interesting for /tmp, but because /etc and /var need to have
persistent files, and I'm afraid won't work as well. For example, it's
helpful on a cluster to have persistent /var/log .
> > Debian Live seems geared toward providing an image for a single client
> > machine for a static image, so I don't know if that's what Beowulf
> > people really want. Is it possible to update? How can one copy over
> > multiple images, while preserving their separate configurations?
>
> My personnal experience, but for workstations (not a cluster
> environnement) is that there is very few files to manage
> hardware/environnement differences.
>
> If you really need to keep separate configuration for each node, you can
> maintain them on the server-side (ip or hostname directory tree) and
> merge them in the tmpfs/unionfs on the client during the initrd
> initialisation.
Right, then you need to put the merge in the initrd, and have tools to
manage the server-side directories. That's exactly what diskless and
lessdisks used to do (note no lessdisks uploads since October 2005, and
it doesn't seem to be in etch)-:, and what drbl apparently does (thank
you Jaehyung). This is the goal of LTSP (Linux Terminal Server Project)
as well, though that doesn't work with Debian at this point.
Debian-Live sounds like a very promising approach, and certainly one
with a broader user base and better prospects for long-term maintenance
than diskless. But it's not usable for a cluster at this point, and
will take quite a bit of work to get it there.
Thanks,
-Adam
--
GPG fingerprint: D54D 1AEE B11C CE9B A02B C5DD 526F 01E8 564E E4B6
Welcome to the best software in the world today cafe!
http://www.take6.com/albums/greatesthits.html
Reply to: