[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

piuparts output format: a proposal

I propose the following as the new output format of piuparts:

* Standard output will report tests that have been run, and their
  results. This output will be similar to the lintian output. This
  output should usually be pretty short, and to the point.

* Additionally, a log file will be written, similar to the current
  output. It will be named after the (first) package being tested
  (or simply piuparts.log?), and placed in the current working
  directory by default.

* Failed tests may _optionally_ also write dumps of the virtual
  environment (chroot, eventually maybe kvm images) so things may
  be debugged.

For example:

    I: scenario install-remove-purge 
    I: environment minimal
    I: test working-symlinks PASS
    E: test leaves-no-extra-files FAIL /etc/foo.conf
    E: test leaves-no-extra-files FAIL /var/lib/foo/
    E: test leaves-no-extra-users FAIL foo
    I: test install-remove PASS

What piuparts does is run a few different scenarios, and perform
tests for each scenario. The current scenarios are:

    * install package, then remove it, and finally also purge it
    * install package from repository, upgrade to new version provided 
      by user, then rmeove and finally purge

Additionally, these scenarios can be run in different kinds of environments,
such as a clean, minimal system, or one with lots of other junk installed.
The test output needs to record these.

Then there's the actual tests. They need to get a name, so they may
be easily referred to. When a test fails, it needs to report something
to make it clear what is wrong. In the sample output above, the failed
tests report the filename or user that caused the failure. Note that if
there are multiple problematic files, they are all reported, one by one
(for easy parsing).

How does this sound? It is important that those who might use piuparts
be able to understand the output.

Reply to: