[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Hardware compatibility test: draft proposal


As I mentioned in my blog[1], I kindof like the suggestion that Bdale
came up with during Debconf that we write a hardware compatibility test
of sorts that hardware vendors could run on their own hardware to test
whether Debian works on their system. The rationale for such a test:
while most of us know how Debian works and how one should test whether
their hardware actually works with Debian, it is not reasonable to
expect the same of hardware vendors for /all/ GNU/Linux distributions
out there. According to Bdale, currently all other major operating
system vendors, including the commercial Linux distributions, already
provide such a test to the hardware vendors. For this purpose, it is
okay if such a test is interactive to some extent (after all, it is
something that you would run on a hardware prototype in a lab, not on
each and every production machine), although I'm thinking a hardware
compatibility test could be useful in more cases, where it might be
better if it wasn't interactive.

So, after more than twelve hours of boredom on an airplane and half a
night of not-being-able-to-sleep-due-to-jetlag, which is certainly
enough to think about this problem, I came up with the following things
such a system could need:

- It should be modular. People who maintain driver packages for
  particular hardware may want to write additional tests that a vendor
  may want to run; and if this particular package supports it, the
  driver package maintainer may want to provide pointers to a particular
  package so that an inexperienced user may be able to configure their
  hardware after running the test themselves.
- The different tests should each be able to communicate what type of
  hardware they test for, whether that particular piece of hardware has
  been found, and whether it actually works. The test of whether
  something works may require that network connectivity, hard disks, or
  other similar things are available in or to the system, as applicable.
- It should support a notion of what I'll call 'profiles'. A 'server'
  profile should check for different things than a 'desktop' or 'laptop'
  profile; e.g., it's usually okay if a server doesn't have graphical
  support or wireless drivers, while the same isn't true for a laptop.
- The vendor should be able to influence the score of a test by
  explicitly stating that particular hardware isn't available. If the
  vendor really wants to build a laptop without wireless drivers in this
  day and age, then it's obviously okay if no wireless drivers were
  detected. If, however, the vendor is not insane, then the failure to
  detect a wireless chipset should clearly influence the score.

So, that's probably it at this point. I should have a look at bdale's
talk once it becomes available at meetings-archive.debian.net, but that
doesn't yet seem to be the case. If someone who was at bdale's talk has
anything to add, that'd be welcome. If someone could think of anything
even without being at bdale's talk, that's welcome too, of course.

Now, with the above in mind, and after having considered Holger's
proposal to do this with Debian Live[2], I think the following generic
spec should cut it, but I'm open to other suggestions at this point.
It's also not very detailed yet, but since no code has been written yet,
that doesn't really matter at this point.

- A base package 'debian-hct' will provide a basic infrastructure for
  these tests to run in and an initscript that actually runs them. It
  will also contain some tests that are useful for /any/ system, such as
  "do we find something that looks like a harddisk controller" etc.
- Additional packages may provide tests. Packages that do so should say
  'Provides: hardware-compatibility-test' in their control file.
- Tests are found in /etc/hw-compat-tests. This directory will have
  subdirectories, one for each of 'hard disk controller', 'wired network
  interfaces', 'wireless network interfaces', etc. The scripts in this
  directory will run in asciibetical order, so that, e.g., drivers that
  need firmware to be loaded can ensure this firmware is actually loaded
  before allowing the generic test for this class of hardware to be ran.
- Scripts in the subdirectories, when ran, should end their output with
  on a single line the letter 'S', followed by a colon, followed by two
  numbers separated by a '/'; the first is the score, the second the
  maximum possible score for this script. All output that precedes the
  score is redirected to a file for the user to inspect later. Scripts
  should make sure to avoid having a line that starts with 'S:' in this
  preceding output, perhaps by escaping it with a leading space or some
  such (this leading space will not be stripped).
  - If no hardware is found that would be supported by the driver this
    script checks for, then the second number, the maximum possible
    score for this script is, by definition, 0.
  - If the hardware being tested for can only be supported by the use of
    non-free drivers or firmware, then the script must mention this fact
    through the output of the letter 'n' after the actual score/maximum
    possible score; e.g., it would say something like 'S:1/1n'. I still
    need to flesh out how to best handle this particular bit of
    information (it doesn't have to be all the same way; e.g., the
    requirement of non-free drivers for the wireless interface is less
    problematic when one wants to install than is the requirement of
    non-free drivers for a hard disk). Suggestions are welcome.
  - Scripts may use debconf to ask the user for input if they need it.
  - Scripts should not just test for availability of hardware. Instead,
    they should test the actual functionality; e.g., tests for a network
    interface should be done by trying to DHCP off that interface, X.org
    drivers should try to start X and ask for input using a graphical
    dialog, and tests for a hard disk should be done by trying to read
    some data from the disk.
  - If multiple scripts in a particular subsystem directory offer
    maximum possible scores, their values will be added to eachother. If
    a script offers 0 as maximum possible score, it will be ignored,
    even if it also outputs 'n' to signal non-free usage.
  - A script must not assume input to be available from the user, nor
    can it expect a controlling terminal. It can only assume that stdout
    works and that debconf is available.
- If none of the scripts in a particular directory indicates it sees
  some hardware (i.e., all scripts offer 0 as maximum possible score),
  the system will ask the user whether this is expected. If this is
  expected, the subsystem will be ignored for the final score; if not,
  the final score for this subsystem will be 0/1.
- After all the scores have been collected, the framework will multiply
  or divide the scores for each of the different subsystem so they fit
  within a predefined scheme that ends up giving the system a score on a
  scale of 1 to 100, which is then presented to the user along with
  details about the tests that were run and the. It is not
  necessary that every piece of hardware is given the same amount of
  consideration in this final score; e.g., we will want to consider the
  proper functioning of a hard disk to be of much more value than the
  proper functioning of a fingerprint reader.
- A Debian Live image will be provided that will install the
  'debian-hct' package plus all packages that say 'Provides:
  hardware-compatibility-test' plus all their dependencies. This will be
  the hardware compatibility test that we can give to vendors.
- Vendors who pass this test on a particular bit of hardware are allowed
  to advertise that fact; it might be nice to have provide them with a
  logo that they may use for this purpose.


(Back to bed now. Jet lag, gotta love it)

[1] http://grep.be/blog/en/computer/debian/hardware_test and
[2] http://debian-live.alioth.debian.org/

<Lo-lan-do> Home is where you have to wash the dishes.
  -- #debian-devel, Freenode, 2004-09-22

Reply to: