[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

What about some user analysis?


I still don't know if the idea of making a separate working group for
usability issues has been well accepted; however, I see that there is
interest in working with usability issues.

While I see many practical proposals about Debian improvements, I rarely
see them backed with an analysis identifying the needs and the goals
that they are supposed to address.

It doesn't mean they are stupid proposals: we, as users, felt the hitch
and were able to design the appropriate scratch.  If it works, it's
useful and the system evolves.

In the meantime, however, the system and the user community becomes more
complicate, and there might be itches we can't perceive, or we could
design scratches for the wrong itch.  While we continue to solve
problems in the traditional, practical way, I'd like to start some
parallel studies to cast some light on that big, itchy back we're
working on.

By the book, first thing to do for such a study is to start with the
users.  For that reason, I'm looking for volunteers to join me in
designing and performing some user analysis for the Debian project.

Here's a draft plan for the analysis.  Suggestions?  Who's in?


 * User analysis  (what is all this about?)

First thing I feel the need of doing is some user analysis, that is learning
about the people that use Debian.

The social contract clearly says that "Our Priorities are Our Users and Free
Software", but I don't know of a serious effort to try and know what those
users that make our priority are.  If there has been one, I'd really like to
have references to it.

To give a quick idea of what a user analysis is, I'd like to quote from the
book "User and Task Analysis for Interface Design", by JoAnn T. Hackos and
Janice C. Redish:

  "You want to learn about the characteristics that may influence how
  people use your product, how they approach learning new tasks, how
  physical differences among them may affect your design decisions, and
  how motivated they are to change their ways of working.  Some of these
  user characteristics will influence your design thinking, while others
  will prove interesting but irrelevant.  But to decide that a user
  characteristic is irrelevant, you must first know that it exists and
  evaluate its impact on your work."

I think it gives the idea.

 * Goal  (what do I want to have?)

We're not designing a specific software tool: we are creating the universal
operating system.  We can't be too specific about our users.

Doing a user analysis for such a system can be dangerous, because you risk
giving our user community a shape and limit our perception of it.

Such an analysis is however essential, to avoid the risk of leaving some kind
of user out because we never had an idea (s)he existed.

As a consequence, the goal for the study should be to provide the developers
with means to stretch their perception of who the users are.

 * Method  (what do I need for it?)

One such possible mean is a collection of data about users.  I have identified
four types of useful data: user profiles, scenarios, interviews and Flanagan's
"critical-incident technique":

User profile
	A brief narrative and/or visual description of the individual users
	we've met.

	A story about Debian.  For example it could tell about a user, tell of
	a place where Debian is used, tell about a task that is performed or
	about the need for a task.

	Interviews are difficult to plan, but allow investigating specific
	topics.  One that I see very important is to probe what is the users'
	perception of Debian, to see if they see it differently than the
	These are questions that can be investigated through interviews:
	 - how do users perceive Debian?
	 - what do they know about Debian?
	 - why do they use Debian?

Flanagan's critical incident technique
	A technique to collect interesting stories of real situations.
	It consists of asking people to recall a critical incident that
	happened to them, and then get informations about it.

	For example, we could ask about the last incident or frustrating
	situation that recently happened to them when using Debian, then about
	the second-to-last, or about the most embarassing or frustrating
	incident that they recall.

 * Presentation of collected data  (how do I want the results?)

For collected data to be useful, it must be presented in the most effective
way.  Since the goal is to help developers think about the users, it is
important to give structure to the data to help properties and relationships
emerge.  Groupings, taxonomies and numerable data are important in building
structures, but should be used with care to avoid creating unexistant
relationships as presentation "artifacts".

Keeping this warning in mind, here are some variables that can be useful to
give structure to the collected data:

 - age
 - nationality
 - job
 - knowledge of the english language
 - available bandwidth
 - computing power
 - how long a Debian user
 - how long a computer user
 - is a debian developer?
 - stage of use

All of these variables should be quite obvious, except for "stage of use", and
so I'll include some lines further a short explanation of them.

 * How to gather the data  (how do I intend to work?)

We are the developers, but we are users and we know users.  If we can pay
attention on collecting our friends' voices unchanged (it might not be as easy
as it seems), we as developers can start collecting data on our own.  User
profiles, scenarios and critical incidents can already be collected by these

For the interviews we can set up a web form for people to compile and submit.

Via web forms we could also gather user profiles, scenarios and critical
incidents by people external to the project or not in direct contact with a
Debian developer.  We could also consider writing and packaging a polling

We could then setup a mail address or mailing list where we could receive the
data.  I volunteer for collecting and organizing it, altough I'd like some
help, mainly to avoid the errors you are likely to make when working alone.

I'd like to work in a group also for planning the collecting phase, since I
don't have the experience required to do such a thing alone.  In fact, it's the
first time I'm seriously doing something like this.

 * Appendix A: What are the stages of use

"Stages of use" is a model to characterize users in a given moment in time.
The version I present here is the model developed by Hubert and Stuart Dreyfus
and successively refined by Hackos and Redish.

It consists of four stages, which I try to summarize from the very good
description given in the book:

 - Novice
   The new user of an artifact.  All users are novices at first: a novice user
   has goals, but does not know exactly what to do to reach them.  The main
   focus is on accomplishing real work, rather than learning concepts.

   At this phase, one faces the unknown, needing to figure things out, and
   risking frustration, embarassment, anger or annoyance, in front of others or

   A novice is not a newborn baby, but brings his/her past experiences from
   other fields.  (S)he can be expert of similar systems, or a theorical expert
   in the matter.  Or maybe not.

   They may be interested to quickly become non-novices, or maybe not, such as
   in high-turnover workplaces or when using systems such as the information
   system in a museum or the automated ticket machines of foreign cities.

   They may also always be novice because they rarely use the artifact, and
   have to re-learn it every time.
 - Advanced beginner

   Once a novice starts using the program to get his/her job done, (s)he
   becomes an advanced beginner.  This stage is still goal oriented, and the
   artifact is used as a black-box tool.  We are usually in the advanced
   beginner stage with regards to an ATM machine, a microwave oven or most
   applications we use, and we are usually not interested to progress further.

   An advanced beginner is more interested in performing tasks than in spending
   time in learning new concepts.

   When an advanced beginner keeps performing tasks, (s)he develops an
   empirical mental model of the artifact, that is probably different to how
   the artifact really works, but that is sufficient to get the job done.

   The difference with a novice is that the advanced beginner knows how to
   perform several tasks well.  However, an advanced beginner has difficulties
   in handling problems, due to the empirical mental model and the
   task-oriented approach.

 - Competent performer

   A competent performer is a user that has built up experience and has formed
   a good mental model of the artifact.  (S)he starts to see the big picture,
   and starts to be able to foresee a behaviour, plan how to perform a new task
   or solve simple problems.

   A competent performer is also likely to access task-oriented documentation
   to gain a better knowledge about how to more efficiently perform a task and
   how to diagnose and correct errors.
 - Expert performer

   Experts use the artifact frequently and know how it works.  They are able to
   figure out how to solve their problems and the problem of others.  They are
   interested in learning about concepts and theories about the artifact they

   They also have interest in interacting with other experts, reading tecnical
   reviews or participating in online communities.

   They can often be identified as the ones other people refer to when they
   have a problem or don't know how to do a complex task. (sounds familiar,
   isn't it? :)

 * Appendix B: Bibliography

JoAnn T. Hackos, Janice C. Redish, "User and Task Analysis for Interface
Design", Wiley Computer Publishing


Yours truly,


GPG key: 1024D/797EBFAB 2000-12-05 Enrico Zini <enrico@debian.org>

Attachment: pgpG9pHo_bUw2.pgp
Description: PGP signature

Reply to: