Comment 37 for bug 882274

On 31/10/11 16:05, Richard Gaskin wrote:
> There are many opinions about design, but one way to move beyond opinion
> to an implementation we can all have greater confidence in is to broaden
> the scope and frequency of usability testing.

Agreed. We do in fact do a great deal of regular independent testing,
with a professional researcher and user testing expert present, and with
independently recruited test users who have no affiliation or
association with the product or the team. What you see reported is some
of the benchmark testing, there are other more fine-grained tests done
to evaluate particular directions or ideas.

> Usability testing can be expensive, but one way to raise the return-on-
> investment for such testing is to consider adopting A/B prototyping
> early on, probably best done with paper prototypes in those early
> stages, long before a single line of code is written. User Interface
> Engineering and other excellent teams have published extensively about
> the methods and usefulness of paper prototyping, and it may even be the
> case that Jared Spool and his team at UIE might be willing to contribute
> consultation to such a high-profile project as this if asked.

All of these techniques are in active use. We'd be glad to have other
independent agencies contribute testing feedback and assessment.

> In summary, may I humbly suggest an increased role for usability testing
> as one way to move forward beyond debates centered around opinion, to
> unite the community around designs whose benefits can be quantified.
The acting head of design in Canonical was appointed specifically
because of their strength and background in independent testing across a
variety of design disciplines. The recruitment job description to fill
that role permanently specifically requires deep insight and experience
with user testing. We're on the same page.

If you've done this before you know how it goes - test results can be
surprising, and products designed that way won't always suit everyone.
Nevertheless, that's the way we do it.

Mark