Dialog on a feature of netboot=nfs:// cifs:// http:// https://

[whadar at gmail.com (Hadar)]

Hi,

I'll try to describe my efforts to integrate http filesystem into casper.
There is more work to be done and help is welcome.

First, bear in mind the following when dealing with http filesystems:
1. AFAIK there are two implementations for http filesystems:
httpfs<http://httpfs.sourceforge.net/>and
http-fuse-cloop<http://unit.aist.go.jp/itri/knoppix/http-fuse/index-en.html>.
Both projects are not part of Debian and IMHO are still immature and not
ready for production use. There are several filesystems out there that can
be used as well, such as davfs and curlftpfs.

2. DHCP is a must, similar to nfsroot and cifsroot.

3. As you can imagine, the performance can be really bad when the network
connection is not fast. There are few ways to optimize the performance such
as local disk caching, preloading of blocks/files and compression. httpfs
lacks the disk caching and preloading.

4. These filesystems are bloat because of their dependencies, namely glibc
and fuse. Inserting them into the initramfs, makes the cpio image bigger
(important in some scenarios).

5. Both filesystem does not support plain directories (this is another
feature I would like to see sometime). httpfs hooks to a single image file
such as iso or squashfs, while http-fuse-cloop uses an index file and a set
of blocks files.

6. This feature does not replace the boot itself. The system must be booted
from a local device or a PXE server! I don't know any BIOS that fetches boot
images over http. Maybe linuxbios? :)

We (neocleus.com) decided to use http fuse cloop (fs_wrapper) for the first
version of the product, so we can have disk caching and preloading. In the
future we hope to use a better httpfs + squashfs. Here's the code we've used
in casper:

* Added this function:
do_httpmount() {
    rc=1
    if [ -x "/sbin/fs_wrapper" ]; then
        [ "$quiet" != "y" ] && log_begin_msg "Trying fs_wrapper -f
${NFSROOT} ${mountpoint}"
        modprobe "${MP_QUIET}" fuse
        mkdir -p /cow/blocks
        fs_wrapper -f "${mountpoint}" "${NFSROOT}"  &

        for i in 0 1 2 3 4 5 6 7 8 9 a b c d e f 10 11 12 13; do
            if [ -f "${mountpoint}"/* ]; then
                rc=0
                break
            else
                log_begin_msg "waiting for fs_wrapper:"$i
                sleep 1
            fi
        done
    fi
    return ${rc}
}

* Changed netboot function to call do_httpmount instead of cifs - an ugly
hack ;)

* Added support for cloop which is necessary when using http-fuse-cloop, see
the attached patch.

* Added this code to the end of casper:

+++    #ugly hack to keep fs_wrapper running...
+++    sed -i 's/killall5/ps axwww \| grep -v fs_wrapper \| grep -v PID \|
grep -v rc \| grep -v init \| awk "\{print \\\$1\}" \|  xargs +++ -n1
kill/g' /root/etc/init.d/sendsigs
+++    sed -i 's/ES\=\$?/ES\=\$?;\/usr\/bin\/killall -15 fs_wrapper;
\/bin\/sleep 3/' /root/etc/init.d/umountroot

    exec 1>&6 6>&-
    exec 2>&7 7>&-
    cp casper.log "${rootmnt}/var/log/"
 }

* Switched losetup! The current casper uses "losetup" which is the one
supplied from busybox. This losetup didn't work well for me. Therefore, I
switch to /sbin/losetup which is included in the first place. Attached a
patch for that change. Again this patch is required only for http-fuse
cloop.

* Last change was to add cloop, fuse and fs_wrapper modules to the
hooks/casper file. Make sure to have those modules in the live image.

As I wrote earlier, when the kernel switches from initramfs to the real
filesystem, run-init kills all the processes.
This hacky sed suppose to prevent the kill of the http filesystem.