OpenSource Usability Labs

Today I closed the 2nd KMail icon test. This time 1945 people took part. Much less people than the last time, but also the drop-out was much less than last time.

So here are some basic statistics:

• Selected language: English: 1002, German: 476, Spanish: 334, Italian: 76, Polish: 57
• Gender: Male: 99%, Female: 1%
• Average Age: 28,6 years (from 11 to 93)
• Most of the participants are strong KDE users. More than 80% use KDE very frequently. Next is Windows (TM) with more than 25% of frequent users, GNome with about 10% and Mac OS (TM) with about 7%.
• The participants are almost half split in their use of KMail, more than 50% use KMail frequently and about 30% say they never use KMail (the rest in somewhere in between).

In this test we included the icons of two elements of KMail: The icons used in the window and in the message menu of the composer window. These are icons of different size in the standard configuration of KDE, so we presented the icons in the according size and used the KUbuntu translations of the terms associated with them. The overall findings are very good. Most icon-term-relations seem to work out fine. We did not find find any explicitly not working relations, but some still show the potential for improvement.

Icon-Term relations working really well

The following icon-term relations seem to work well. (more…)

The second icon test for KMail Icons is still running. If you have not participated yet, please do so now!

Jochen S. asked me to publish the distribution of the age of the participants of the test. I am happy to do so (Thanks to rKward!)…

All participants of the test:

Mean:  27,9 years
Standard deviation: 7,5 years

Frequent users of KMail:

(more…)

It is great to see how many people participated in the first icon test for KMail: 3327 started the test and 2399 finished. This is a drop-out of about 28% across the whole study and fairly good for this kind of study - especially seeing the technical difficulties we had when we started the study.

Here is a first brief summary of the results:

• Selected language: English: 2244, German: 659, Spanish: 313, Polish: 109
• Gender: Male: 98%, Female: 2%
• Average Age: 29 years (from 10 to 88)

It is great to see that we got sufficient participants for reliable results in all languages!

As you might know, our test combines multiple indicators and calculates a single value for each icon-term relationship. The maximum value an icon-term-relation can reach is 10,0. Following I split up the results into 3 groups: (more…)

It is really great to see how many of you already took part in our KMail-Icon-Test. A lot of questions arrived my by mail or in my blog, so I thought I would just explain a little about testing icons…

We do this testing, because icons are useful and beautiful - they save place and people can recognize them faster than they can read text. Practically icons work via a visual-metaphor. If that metaphor, however, is not understood by the user, (more…)

We planned it for the Linux-Tag, but then there wasn’t a good Internet connection -

So we ask you now: Which are the best icons for Tine?

Please take part in this very short test where we would like to find out which icon fits best for some of the Tine parts like “Ressources” or “Today”.

To help us make Tine 2.0 even more user friendly, please click here and choose you preferred language:

http://tiny.cc/icontest

Plus: Calendar testers needed!

We still need testers for our Usability test in two weeks. If you live in Berlin, or happen to be there in two weeks, please give me a shout at: tine.calendar@gmail.com

We are looking for Tine-experienced people but newbies are also great.

The test itself will take place in Kreuzberg for about an hour and your effort will be rewarded, of course.

Thank you very much!

Wow. Today we closed the Icon Test for TV-Browser. And man - this has been a real test for our Beta-Service. The result: Within one week more than 22000 individual tests have been conducted in our 18 experimental conditions, filling our database with 123404 rows of data. OpenOffice Calc gives up on this amount of data (As an interesting insight: MS Excel on the other hand can read that many rows of data, but not save them).

So it will take us some time to actually get valid (in term of significant) results, as this sheer amount of data first has to be handled. But I will uncover not too much when I say: a first look at the data is promising.

I guess we have found a valid way to actually have users judge the usability of icons of an application. The icon test is easy to set up, it is easy (ok, it will be easy once the platform is finished up…) to get the results and it is fast - in the best meaning of agile usability. You need valid feedback on your icons till tomorrow? No problem anymore!

With our icon test we are able to spot the good and the better-find-another icons in an interface, and we can easily decide which of a set of alternative icons for one term is the one users prefer most.

More soon to be seen on www.icon-test.com!

We just started an online-survey together with the guys from TV-Browser. The question we want to answer is twofold:

1. Which Icons the TV-Browser uses are easy to understand, helpful or in just one word: appropriate? Which Icons should TV-Browser use in future releases?
2. Does the method and the online-Tool we have developed for testing the quality of icons work as good, as we hope it does?

If you are curious: The study is online until the 1st of September 2008. You can participate even if you do not know much about the TV-Browser - as long as you use Firefox Webbrowser, because this study is still beta: Study for checking the quality of icons in TV-Browser.

After starting the study first problems became obvious immediately and most of them got solved quite quickly. I have learned, e.g. that a switch can be “half-broken” eating up lots of, but not all data-packages.

Summing it up: the study runs quite well and we are getting tons of answers by the TV-Browser-Users. It is always fun to have this great community involved into new studies.