Usability test: Installation of a Debian desktop
1 Description of the test
a) Short description of the test user
The female test user is a 30 year old usability engineer with
a master in media technology and design.
b) Short description of the test setup
The test setup is on a home network with a Linux box providing
Nat and dhcp (no proxy). The install box is a dual boot machine
where a GNU Linux system (Slackware) is to be replaced, an old
Lexmark Z31 printer is attached. This system is on the second
partition of the second hard disc.
For hardware details look at bug #273568.
2 Question on the test user about his level of experience
a) Linux experience of the test user
The test user had already installed a Suse box for testing
purpose 5 years ago, then used Red Hat.
Due to dislikes of its logo and a friend recommendation
switched to Slackware.
This installation were dual boot and rarely used.
Due to the "Unix severity" of Slackware and not working
hardware (sound and printer) it was rarely used.
Sometimes for surfing and for latex typesetting.
b) Debian experience of the test user
Zero.
c) Free software experience of the test user
Daily use of Mozilla as browser and mailer.
Sometimes Openoffice, Latex and Gimp.
d) Expectations regarding Debian
Easy upgrades and installations of software.
Easier to maintain than Slackware.
Hopefully to be able to print.
e) Expectations of the Debian-installer (short d-i)
"Better than what i already know",
as usability test have already influenced its user interface.
Never the less the test user expected problems.
Not everything working from the start.
3 d-i installation manual.
The test user is given time to read the manual in order to help
her on her decisions, remarks are noted below. The manual was
available during installation on a nearby laptop.
The user goes directly to chapter 3,
as she's already running Linux thus assuming that the
"System Requirements" will be no problem.
in chapter 3.1 user stumbles over
"Back up any existing data or documents on the hard disk where you plan to
install."
The user thinks only the one partition to be affected and
that there will be no consequences for other partitions on
the same disc.
as she is thinking to be using the same partition,
where there is already a Linux she thinks no backup is needed.
The user is asked to reread this phrase, but doesn't understand
implications.
In 3.2 the "non-native operating system" is not understood.
The test user doesn't know the UNIX device naming.
Has problems to understand the order of hard disc and partitions.
The concerning explanation is not linked from 3.3.2
"Finding Sources of Hardware Information"
(Pamela Jones maintains a nice chapter on the topic of partitions.
maybe her ideas could enhance current manual:
http://www.grokdoc.net/index.php/General_Installation )
After recommandation by test observer data is backuped.
The user writes down the needed information.
4. The installation process with d-i
The test user chooses english, as country austria,
german as keyboard layout.
Stumbles over configuring the hostname.
The default entry "Debian" seems misleading, wants to append
a string to that.
Takes some time to clear this default and to type in her wish.
Stumbles on the screen concerning the domainname.
Doesn't understand what it is used for.
There is no explanation that for a home network
it may be a good idea to leave it blank.
User tries with blank.
The partitioner comes up with the choice of erasing each
of the 2 hard discs or to manually edit the partition table.
Well the first 2 choices don't fit the users task.
So she has to manually edit it's table. The user is upset
that the installer doesn't have an automatic guided
partitioning based on the choice of her 3 partitions.
The user doesn't really feel up to edit the partition table.
Finally as she doesn't want to erase any whole disc,
she goes further. hdb2 is chosen in partman.
"Do not use it" is presented as option.
"What shall i do to change that?"
"Shall i press enter?" "Oooh"
"yes i want to format that partition. Ok choosing ext3"
It may be a good choice but no information on the top
of the screen regarding the fs choice.
The next step is to choose the mount point of the partition.
The user doesn't understand its meaning.
This template has again no explanations on its top.
The manual 6.3 is consulted. It has no explanations about
a mountpoint. It explains that partman assigns mountpoints,
but not what they are for.
The observers help is necessary to continue the installation,
although user guesses right choice: '/'.
The template regarding the label is again without any
explanations on its top, nor does manual 6.3 have any
explanation.
The top choice is chosen.
The chosen partition is formatted and the partitioner
continues with a successful base install.
Grub says to have detected another Operating System,
but its install fails (bug #273568).
The user tries it a second time, but it doesn't work.
Therefore lilo is installed on the mbr of the fist disc.
As the Debconf priority is lowered by aboves error,
the observer asked the test user to rise it's priority
level. The tester would have wished to land directly
on this choice as the failed step was successfully
overcome with the lower priority.
Machine reboots successfully.
The user does not understand the timezone screen: her
computer clock is set to gmt as the bios is reporting
utc timezone, but the shown time is the current
West European summer time.
The user asks the observer what to do.
Again this step has to be regarded as failed.
The screen has nice explanations, but they seem not to
have helped in this conflicting case.
The root password and user is successfully created.
But after its creation the user times lots of <tab>
before next template shows up.
The user is confused as the next template has no
default choice.
Nevertheless apt is configured successfully.
The user is a bit upset to be reasked the same question
he already answered for the initial apt configuration.
In tasksel the user does not understand how to choose
the tasks. The manual is consulted, but 7.2.5 does
not explain how to select a task.
A short notice on top of the screen concerning the
<space> key to select a task would be very helpful.
Again the install can be counted as failed as the
test user has to ask the observer to fulfill this step.
Manual 7.2.5 is consulted again in order to know if
the selected collection of software will be retained
on "manual package selection". The manual has no entry
on that question.
"Desktop environment", "Web Server", "Print Server",
"SQL database" and "manual package selection" was chosen.
In aptitude the strings "Download/Install/Remove Pkgs"
invites the user to press g in order to remove kdebase,
as the user prefers Gnome as desktop and saw no need
of the second. Aptitudes reactions to perform all pending
actions was not the anticipated action. The user is upset
about the misleading string.
The user performs the short exim configuration for local
delivery and leaves the name for the samba workstation blank.
The xfree configuration doesn't ask for a keyboard,
which is improperly set, but does ask for the video card.
The user has no idea that mga stands for Matrox.
This step of the install can be counted as failed.
The automatic mouse detection fails, and a very nice screen
about the difference of mouse connectors shows up.
The user is very pleased. The next screen puts the user
in front of all the different UNIX mice devices without any
further explanation, which the user regards as rude and
disappointing.
The user does not know what the options mean.
This step can be again counted as failed.
For the mouse protocol the user chooses the one
that matches it's logitech serial mouse manufacturer.
In fact the mouse does use another one, so it did not work.
Aptitude asked to push enter to continue after having
installed all packages. "Not a very shiny screen."
The templates of aptitude are perceived as unpolished
and very different to the previous screens of d-i.
After the welcome of the successful install,
gnome starts with a completely broken resolution and broken mouse.
The xfree config is fixed up manually.
The user is successfully using it's fresh Debian install.
The printer was sucessfully configured with the help of foomatic-gui.
Software installs were noted to be quicker than on windows.
The user found it very compelling that Debian is unifying
all the different software in its package archives.
5. Conclusions
The manual was quite good, but yet left many questions
unanswered.
The d-i did a very good job concerning
network setup, ide and cd detection on that install, but failed
on the package selection and on the xfree configuration. *)
The partitioner was found to be very hard to understand,
as many templates had no explanations.
The level of needed expertise of the user (human-readable
information and helpful explanations vs. technical terms
and abbreviations, missing background information)
differs considerably between the screens.
Short meaningful explanations and an updated manual would
already help a lot.
The test person and the test observer hope that aboves
usability test helps the developers to further confine d-i
and to make an even better user experience for novel users.
The test was done in the manner of well established usability
testing methods. Some may follow the next months
and i will link this posting to the usability community
effort at grokdoc.
Further questions are welcome.
--
maks
kernel janitor http://janitor.kernelnewbies.org/
*) I know that the current xfree setup is not part of d-i itself,
nevertheless, it would be great if the current hooks of d-i
(keyboard setup, ..) would be extended to the other important
parts of Debian (apt, xfree86) as from a user point of view
they are perceived to be part of the installer.
Reply to: