[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

draft "Testing on SI" document

This is a first cut at my "Testing on the SI" document.
Please feed back any comments/suggestions/improvements.


Instructions for performing LSB Platform testing on the LSB-si

Note:  this document is subject to considerable change, as
the final composition of the LSB-si and any add-ons is still
evolving.  This information is correct as of July 2002 -
which means it may already be out of date with respect to
the materials in the CVS tree for the LSB-si.  Open questions
may be flagged, if so look for [XXX].


The LSB-si is a minimal LSB-conforming runtime environment
which runs as a chroot inside a hosting platform.  Optionally,
the LSB-si could be made standalone bootable if there's any
need - there are no current plans in this area.  Since it's
a chroot on a hosted system, testing results will be impacted
by the underlying kernel - if the test results are to be
fully conforming, the kernel must be one which does not have
LSB conformance faults.

LSB Platform testing consists of several components. The
largest piece is the binary platform test suite.  Most
of this document will concentrate on running this test
suite.  The other pieces are:

* lsblibchk - verify libraries
* lsbcmdchk - verify presence of required commands
* application battery - run at least two of the provided
  example programs, and sign off that they pass the FVT
  (Functional Verification Test)

The LSB-si itself aims to provide only the minimal set
of commands and libraries required by the LSB.  However,
running the test suites will require some additional
materials that are not part of the LSB.

Installing the LSB-si:

The LSB-si is currently distributed as a tarball, which
should simply be unpacked, as root, in /usr/src. 

[XXX - ought this to be built to instal under /opt?
 ought there to be some kind of a more formal install?]

[XXX - are we likely to have problems with owners/permissions
 on the host, esp. if the password file is different than
 on the build machine?]

[XXX - will what's currently called lsbsi-intermediate
 also be available to provide a developer environment,
 or is that to be considered internal-only?]

Preparing to run the LSB-si:

Since the LSB-si will be run in a chroot, it's not going
to have access to things outside its' own tree.  One way
to work around that is to set up bind mounts before
starting the chroot, if there's a need to access other
files.  Usually this is only neeed for the preparatory
steps of installing software.  For example, to set up
access to the files in the home directory of "username":

# mkdir /usr/src/lsbsi/home/username
# mount -o bind /home/username /usr/src/lsbsi/home/username

An alternative approach is to identify all files that
will be needed and copy them into a directory inside
the LSB-si tree

Installing the tests:

The LSB-si is equipped with a copy of rpm, to meet the
requirement that a conforming platform be able to
install rpm packages of the specified format.  Note that
rpm itself is not required by the LSB spec.  The rpm
database inside the LSB-si is not initilized.  Start
by entering the LSB-si:

    # /sbin/chroot /usr/src/lsbsi

Install the binary runtime test package inside the LSB-si:

   # rpm -i --nodeps <path-to>/lsb-runtime-test-1.2.1-1.i386.rpm

Substitue in the proper path, the architecture, and the
appropriate version for that architecture. 
For an unreleased ar

[XXX the --nodeps part is a bug that remains to be resolved]

Install the other test packages inside the LSB-si:

    # rpm -i --nodeps <path-to>/lsblibchk-1.2.2-1.i386.rpm
    # rpm -i --nodeps <path-to>/lsbappchk-1.2.2-1.i386.rpm
    # rpm -i --nodeps <path-to>/lsbcmdchk-1.2.1-1.i386.rpm

Install the Application Battery packages inside the LSB-si:

    # rpm -i --nodeps <path-to>/lsb-apache-2.0.39-1.i486.rpm
    # rpm -i --nodeps <path-to>/lsb-rsync-2.5.5-1.i486.rpm
    # rpm -i --nodeps <path-to>/lsb-xpaint-2.6.2-1.i486.rpm

Preparation for LSB runtime testing:

The runtime tests require some preparation.  These are
all one-time actions; there is a separate section
describing steps for starting the LSB-si in test mode,
it assumes these steps are already done.

For the steps which need to be performed inside the
LSB-si, run the chroot command (as root) to start:

    # /sbin/chroot /usr/src/lsbsi

1. Dynamic linker. If running an *unreleased* platform,
the runtime tests won't be linked against the LSB dynamic
linker, and the standard Linux dynamic linker won't be
found inside the LSB-si.  This can be worked around with
a symbolic link:

    # cd /usr/src/lsbsi/lib
    # ln -s ld-2.2.5.so ld-linux.so.2

On Itanium, the dynamic linker has a different name, use:

    # ln -s ld-2.2.5.so ld-linux-ia64.so.2

Do not to install the symbolic link unless it is actually

2. mounts.  The chroot will require /proc to be mounted
for many reasons.  Certain tests also require other setup
of mounts.  A minimal /etc/fstab will be present inside
the LSB-si, but it should be updated to reflect the realities
of the host system.
[XXX it may be possible to automate this with a script]

The fstab in the LSB-si should have entries for the pseudo
devices.  Copy from the host system the entry for /proc,
and, if present, for /dev/pts and/or /dev/shm.  

Also check all of the regular file systems mounted in the
host, and make sure the mount point for each exists inside
the LSB-si.  This seems like a strange requirement, but
there's a reason. If the 'df' command returns failure,
some tests will fail.  To test that you've got it set up
right, run the 'df' command inside the LSB-si and make sure
there are no error messages.

Note:  if using devpts, make sure the mount options will
set the default mode to 0620.  Here's the line from my
/etc/fstab to do this:

    none	/dev/pts	devpts	gid=5,mode=620  0  0

It may be necessary to manually create /dev/pts (using mkdir)
inside the LSB-si.

3.  mke2fs/tune2fs.  The test setup needs to make a filesystem
in a file, which will be bound to a loopback device and used
for certain tests on filesystem conditions. The LSB-si
does not include the /sbin/mke2fs program that will be used
(nor /sbin/tune2fs which is used immediately after).
The easiest solution for now is to copy in a working file
from the host system.  Inside the chroot, the file will be
named /home/tet/test_sets/loopback_disk.ext2.  The setup
step will create this file, but it will be empty, since
the command is not found.  Here's how to create one and
copy it in; the setup can then be restarted.

[XXX need details on how this occurs - I've forgotten. Is
the setup done by installing the rpm or when you log in
as vsx0 and start run_tests the first time?]

Outside the chroot:

    # dd if=/dev/zero of=loopback_disk.ext2 bs=1k count=2000
    # /sbin/mke2fs -F loopback_disk.ext2
    # /sbin/tune2fs -s 0 -i 0 loopback_disk.ext2
    # cp loopback_disk.ext2 /usr/src/lsbsi/home/tet/test_sets

[XXX the missing commands may be provided soon as an "extras" package]

4. syslogd.  The LSB spec does not require a syslog daemon
running, but the tests of the syslog ABI do require one
to respond to logging requests, and will fail if one is not
already running when the tests are started.  A tarball with
the necessary pieces is available.  Inside the chroot,

    # cd /
    # tar -xvf <path-to>/lsb-si-syslogd-i386.tar

[XXX this will be provided soon in an "extras" package]

5. pax.  The tests will run best if an LSB-conforming version
of the pax command (for tar and/or cpio testing) is installed.
Inside the chroot:

    # tar -xvf <path-to>/lsb-si-pax-i386.tar

[XXX this may be provided soon as an "extras" package]

Note that the pax RPM package pointed to by the LSB website
is not usable inside the LSB-si, although it's fine for testing
a native system.

6. telnetd.  Some tests will fail if the LSB-si is not entered
via a true login.  Tests will run as the user "vsx0", and it's
not sufficient to run the chroot command, then 'su -' to vsx0.
Inside the chroot:

    # tar -xvf <path-to>/lsb-si-telnet-i386.tar

[XXX this may be provided soon as an "extras" package]

7. Devfs.  The 1.2 version of the LSB specification does not
officially allow the symbolic links in /dev that devfs will
create, so if your host system is equipped with devfs it
needs to be disabled for best test results.  This will change
with the 1.3 specification, at which point the tests will
update also.  This was accomplished on one system using
LILO by adding this line to the boot specification in lilo.conf:


Entering/Setting up the LSB-si for Testing:

These steps assume the one-time setup steps have
previously been completed.

1. Disable conflicting services on the host.

a. If the host is enabled to run the telnetd service, it
should be disabled.  This may be possible through the
chkconfig command, or it may be necessary to manually
change the configuration of /etc/xinetd.d/telnet or

b. If the Apache webserver (application battery) is to
be tested, shut down any running webserver that is listening
on port 80, and make sure the isn't one started on-demand
from xinetd/inetd (this is not a normal configuration so it's
unlikely this is the case). Here's a possible command,
modify as necessary:

    # sh /etc/init.d/httpd stop

c. Shut down the syslogd daemon. Here's a possible command,
modify as necessary:

    # sh /etc/init.d/syslog stop

2. Enter the chroot

    # /sbin/chroot /usr/src/lsbsi

3. Mount file systems:

    # mount /proc

    And, as necessary (do the same as your host system does):

    # mount /dev/pts
    # mount /dev/shm

4.  Start syslogd:

    # /usr/sbin/syslogd -m 0

5.  Start telnetd:

    # /usr/sbin/telnetd -debug

    Note that this is a single-shot invocation, it will quit
    when the telnet session closes. If necessary, the same
    command can be used to enable a new session.

Running the LSB runtime test suite:

>From a terminal (x window, console session, etc.) log is as
the test user ('vsx0') and follow the prompts:

    $ telnet localhost
    Connected to localhost.
    Escape character is '^]'.

    Linux 2.4.18 (haddock) (08:55 on Wednesday, 07 August 2002)

    haddock login: vsx0
    Welcome to the Linux Standard Base Sample Implementation 1.2
    No mail.

    To execute the LSB test suites run "run_tests"


Start the tests, answer the questions, accepting the defaults
except where it doesn't seem appropriate, by running
"run_tests" (as in the prompt).

Reply to: