Bug#274935: Usability test: Installation of a Debian desktop
Package: debian-installer-manual
Here is something that deserves more that just a place in ML archive.
On Mon, Sep 27, 2004 at 02:00:22AM +0200, maks attems wrote:
>
> 1 Description of the test
> a) Short description of the test user
>
> The female test user is a 30 year old usability engineer with
> a master in media technology and design.
>
> b) Short description of the test setup
>
> The test setup is on a home network with a Linux box providing
> Nat and dhcp (no proxy). The install box is a dual boot machine
> where a GNU Linux system (Slackware) is to be replaced, an old
> Lexmark Z31 printer is attached. This system is on the second
> partition of the second hard disc.
> For hardware details look at bug #273568.
>
>
> 2 Question on the test user about his level of experience
> a) Linux experience of the test user
>
> The test user had already installed a Suse box for testing
> purpose 5 years ago, then used Red Hat.
> Due to dislikes of its logo and a friend recommendation
> switched to Slackware.
> This installation were dual boot and rarely used.
> Due to the "Unix severity" of Slackware and not working
> hardware (sound and printer) it was rarely used.
> Sometimes for surfing and for latex typesetting.
>
> b) Debian experience of the test user
>
> Zero.
>
> c) Free software experience of the test user
>
> Daily use of Mozilla as browser and mailer.
> Sometimes Openoffice, Latex and Gimp.
>
> d) Expectations regarding Debian
>
> Easy upgrades and installations of software.
> Easier to maintain than Slackware.
> Hopefully to be able to print.
>
> e) Expectations of the Debian-installer (short d-i)
>
> "Better than what i already know",
> as usability test have already influenced its user interface.
>
> Never the less the test user expected problems.
> Not everything working from the start.
>
>
> 3 d-i installation manual.
>
> The test user is given time to read the manual in order to help
> her on her decisions, remarks are noted below. The manual was
> available during installation on a nearby laptop.
>
> The user goes directly to chapter 3,
> as she's already running Linux thus assuming that the
> "System Requirements" will be no problem.
>
> in chapter 3.1 user stumbles over
> "Back up any existing data or documents on the hard disk where you plan to
> install."
> The user thinks only the one partition to be affected and
> that there will be no consequences for other partitions on
> the same disc.
> as she is thinking to be using the same partition,
> where there is already a Linux she thinks no backup is needed.
> The user is asked to reread this phrase, but doesn't understand
> implications.
>
> In 3.2 the "non-native operating system" is not understood.
>
> The test user doesn't know the UNIX device naming.
> Has problems to understand the order of hard disc and partitions.
> The concerning explanation is not linked from 3.3.2
> "Finding Sources of Hardware Information"
>
> (Pamela Jones maintains a nice chapter on the topic of partitions.
> maybe her ideas could enhance current manual:
> http://www.grokdoc.net/index.php/General_Installation )
>
> After recommandation by test observer data is backuped.
>
> The user writes down the needed information.
>
>
> 4. The installation process with d-i
>
> The test user chooses english, as country austria,
> german as keyboard layout.
>
> Stumbles over configuring the hostname.
> The default entry "Debian" seems misleading, wants to append
> a string to that.
> Takes some time to clear this default and to type in her wish.
>
> Stumbles on the screen concerning the domainname.
> Doesn't understand what it is used for.
> There is no explanation that for a home network
> it may be a good idea to leave it blank.
> User tries with blank.
>
> The partitioner comes up with the choice of erasing each
> of the 2 hard discs or to manually edit the partition table.
> Well the first 2 choices don't fit the users task.
> So she has to manually edit it's table. The user is upset
> that the installer doesn't have an automatic guided
> partitioning based on the choice of her 3 partitions.
> The user doesn't really feel up to edit the partition table.
>
> Finally as she doesn't want to erase any whole disc,
> she goes further. hdb2 is chosen in partman.
> "Do not use it" is presented as option.
> "What shall i do to change that?"
> "Shall i press enter?" "Oooh"
> "yes i want to format that partition. Ok choosing ext3"
> It may be a good choice but no information on the top
> of the screen regarding the fs choice.
>
> The next step is to choose the mount point of the partition.
> The user doesn't understand its meaning.
> This template has again no explanations on its top.
> The manual 6.3 is consulted. It has no explanations about
> a mountpoint. It explains that partman assigns mountpoints,
> but not what they are for.
> The observers help is necessary to continue the installation,
> although user guesses right choice: '/'.
>
> The template regarding the label is again without any
> explanations on its top, nor does manual 6.3 have any
> explanation.
> The top choice is chosen.
>
> The chosen partition is formatted and the partitioner
> continues with a successful base install.
> Grub says to have detected another Operating System,
> but its install fails (bug #273568).
> The user tries it a second time, but it doesn't work.
> Therefore lilo is installed on the mbr of the fist disc.
>
> As the Debconf priority is lowered by aboves error,
> the observer asked the test user to rise it's priority
> level. The tester would have wished to land directly
> on this choice as the failed step was successfully
> overcome with the lower priority.
> Machine reboots successfully.
>
> The user does not understand the timezone screen: her
> computer clock is set to gmt as the bios is reporting
> utc timezone, but the shown time is the current
> West European summer time.
> The user asks the observer what to do.
> Again this step has to be regarded as failed.
> The screen has nice explanations, but they seem not to
> have helped in this conflicting case.
>
> The root password and user is successfully created.
> But after its creation the user times lots of <tab>
> before next template shows up.
> The user is confused as the next template has no
> default choice.
> Nevertheless apt is configured successfully.
> The user is a bit upset to be reasked the same question
> he already answered for the initial apt configuration.
>
> In tasksel the user does not understand how to choose
> the tasks. The manual is consulted, but 7.2.5 does
> not explain how to select a task.
> A short notice on top of the screen concerning the
> <space> key to select a task would be very helpful.
> Again the install can be counted as failed as the
> test user has to ask the observer to fulfill this step.
>
> Manual 7.2.5 is consulted again in order to know if
> the selected collection of software will be retained
> on "manual package selection". The manual has no entry
> on that question.
>
> "Desktop environment", "Web Server", "Print Server",
> "SQL database" and "manual package selection" was chosen.
>
> In aptitude the strings "Download/Install/Remove Pkgs"
> invites the user to press g in order to remove kdebase,
> as the user prefers Gnome as desktop and saw no need
> of the second. Aptitudes reactions to perform all pending
> actions was not the anticipated action. The user is upset
> about the misleading string.
>
> The user performs the short exim configuration for local
> delivery and leaves the name for the samba workstation blank.
>
> The xfree configuration doesn't ask for a keyboard,
> which is improperly set, but does ask for the video card.
> The user has no idea that mga stands for Matrox.
> This step of the install can be counted as failed.
> The automatic mouse detection fails, and a very nice screen
> about the difference of mouse connectors shows up.
> The user is very pleased. The next screen puts the user
> in front of all the different UNIX mice devices without any
> further explanation, which the user regards as rude and
> disappointing.
> The user does not know what the options mean.
> This step can be again counted as failed.
> For the mouse protocol the user chooses the one
> that matches it's logitech serial mouse manufacturer.
> In fact the mouse does use another one, so it did not work.
>
> Aptitude asked to push enter to continue after having
> installed all packages. "Not a very shiny screen."
> The templates of aptitude are perceived as unpolished
> and very different to the previous screens of d-i.
>
> After the welcome of the successful install,
> gnome starts with a completely broken resolution and broken mouse.
> The xfree config is fixed up manually.
>
> The user is successfully using it's fresh Debian install.
> The printer was sucessfully configured with the help of foomatic-gui.
> Software installs were noted to be quicker than on windows.
> The user found it very compelling that Debian is unifying
> all the different software in its package archives.
>
>
> 5. Conclusions
>
> The manual was quite good, but yet left many questions
> unanswered.
> The d-i did a very good job concerning
> network setup, ide and cd detection on that install, but failed
> on the package selection and on the xfree configuration. *)
> The partitioner was found to be very hard to understand,
> as many templates had no explanations.
> The level of needed expertise of the user (human-readable
> information and helpful explanations vs. technical terms
> and abbreviations, missing background information)
> differs considerably between the screens.
> Short meaningful explanations and an updated manual would
> already help a lot.
>
>
>
> The test person and the test observer hope that aboves
> usability test helps the developers to further confine d-i
> and to make an even better user experience for novel users.
> The test was done in the manner of well established usability
> testing methods. Some may follow the next months
> and i will link this posting to the usability community
> effort at grokdoc.
> Further questions are welcome.
>
>
>
>
> --
> maks
> kernel janitor http://janitor.kernelnewbies.org/
>
>
> *) I know that the current xfree setup is not part of d-i itself,
> nevertheless, it would be great if the current hooks of d-i
> (keyboard setup, ..) would be extended to the other important
> parts of Debian (apt, xfree86) as from a user point of view
> they are perceived to be part of the installer.
I hope that some one picks the good parts from it.
Geert Stappers
Reply to: