About hostid and ZFS (was Re: [Pkg-zfsonlinux-devel] ZFS on Linux and native ZFS on BSD)

[Carlos Alberto Lopez Perez <clopez@igalia.com>]

On 17/02/13 15:19, Arno Töll wrote:
>> > There is also a question about /etc/hostid handling, do you know how
>> > is it handled in kBSD? Existing packaging work of Fedora ZoL makes
>> > hostid static, but I doubt it's desired.
> We do not define any hostid, in fact (and Debian/Linux neither ships
> /etc/hostid. In glibc, which kfreebsd uses as well, the hostid is
> determined by gethostid(2) which does some magic based on
> gethostbyname(2) when that file does not exist).

From spl/module/spl/spl-generic.c

/*
 * Read the unique system identifier from the /etc/hostid file.
 *
 * The behavior of /usr/bin/hostid on Linux systems with the
 * regular eglibc and coreutils is:
 *
 *   1. Generate the value if the /etc/hostid file does not exist
 *      or if the /etc/hostid file is less than four bytes in size.
 *
 *   2. If the /etc/hostid file is at least 4 bytes, then return
 *      the first four bytes [0..3] in native endian order.
 *
 *   3. Always ignore bytes [4..] if they exist in the file.
 *
 * Only the first four bytes are significant, even on systems that
 * have a 64-bit word size.
 *
 * See:
 *
 *   eglibc: sysdeps/unix/sysv/linux/gethostid.c
 *   coreutils: src/hostid.c
 *
 * Notes:
 *
 * The /etc/hostid file on Solaris is a text file that often reads:
 *
 *   # DO NOT EDIT
 *   "0123456789"
 *
 * Directly copying this file to Linux results in a constant
 * hostid of 4f442023 because the default comment constitutes
 * the first four bytes of the file.
 *
 */


The behavior of /usr/bin/hostid on Linux systems with the regular eglibc
and coreutils is:

# eglibc: sysdeps/unix/sysv/linux/gethostid.c
  http://paste.debian.net/235758/

if ((exists /etc/hostid) && (sizeof(/etc/hostid) >= 4 bytes) {
    hostid = read_4bytes(/etc/hostid);
else {
    if ((defined(hostname) && defined(getip(hostname))) {
        hostid = getip(hostname);
        hostid = (hostid << 16 | hostid >> 16);
    else
        hostid = 0;
    }
}

Therefore there is only one way to ensure the value of hostid preserves
across reboots and is to write it to /etc/hostid.

However, most of times the hostid of the machine will be 007f0101
because the IP address of $(hostname) on Debian systems is typically
127.0.1.1

## gethostip available on package syslinux
# gethostip $(hostname)
localhost.localdomain 127.0.1.1 7F000101

But gethostid() swaps the bytes

/* For the return value to be not exactly the IP address we do some
     bit fiddling.  */
return (int32_t) (in.s_addr << 16 | in.s_addr >> 16);

So 0x7F000101 turns into 0x007F0101



So, it turns out that most of times, hostid will return:
 * 007f0101 (01017f00 in case of big-endian machines)


So this invalidates the original purpose of gethostid(). Which was meant
to be a value unique for each host, has turned into a value that is
shared by most of the hosts.

http://bugs.debian.org/595790

Which then invalidates the original purpose of storing the hostid value
into the ZFS pool
http://distfiles.scode.org/tmp/zfs-handbook/zfs-hostid.htmle


So I propose that we can do the following in the postinst of SPL:

* If /etc/hostid already exits do nothing
* If the value returned by hostid is other than 007f0101, 01017f00 or
00000000 then write it on /etc/hostid. Otherwise generate a random value
and write it on /etc/hostid.

Code: http://paste.debian.net/235773/


In FreeBSD systems, the hostid is unique for each machine. This value is
set to a random one on the first boot.

https://gitorious.org/freebsd/freebsd/blobs/HEAD/etc/rc.d/hostid


Another idea (more in line with FreeBSD) could be just generate always a
random value for /etc/hostid when this file don't exists on the system.
Perhaps this is the better approach.

if [ ! -f /etc/hostid ]; then
	dd if=/dev/urandom bs=1 count=4 of=/etc/hostid 2>/dev/null
fi



Thoughts?

Attachment: signature.asc
Description: OpenPGP digital signature