OpenSource Usability Labs

This summer I participated in the SoU project with Gallery, the open source photo sharing software. The broad goal of the project was to conduct a survey to learn more about Gallery’s users in moving toward the release of the 3rd version of the software. The project was a success in terms of garnering useful data for Gallery, but, importantly, it was an incredibly valuable learning experience for me.

As a student working towards a masters in Human-Computer Interaction, I have learned about and conducted user research and usability studies in my coursework. But, what SoU provided me that no school project ever could was the opportunity to engage in research that has an actual impact on improving a product— and the opportunity to experience all the challenges that go along with that.

At the start of the project, I have to say I was a bit overwhelmed with what direction to take, but with the guidance of Björn as my mentor, the help of Jakob, a former SoU student who continues to work on the Gallery project, and the awesome tools of Usability-Methods.com, I soon found my way. Here are some of the lessons I learned along the way.

(more…)

Today I found an article by Sam Dean who asks for a Centralized, State-of-the-Art Open Source Usability Lab. He refers to a cnet article by Matt Asay in which he postulates what Open Source can learn from Apple. Both articles point out that there is need for a shifting view in Open Source. Open Source needs to be more user-driven and less developer-centric - in other words there is a need for usability in Open Source work.

Well, there are good and bad news for both of them:
It is not as easy as they think but we have already come much further than they think!

To explain this I would like to clarify some myths on Open Source Usability:

Usability plays no role in Open Source development.

The OpenUsability.org initiative has provided Usability guidance to hundreds of Open Source projects for more than 5 years. We have worked with various  projects from big ones like Wikipedia or KDE to very small ones. Many projects have developed their own Usability-Community like the OpenOffice Renaissance or the KDE Usability project. Celeste Paul - one of the members of OpenUsability and KDE Usability - has just recently been elected into the KDE e.V. board.

So there is a community willing to assist Open Source projects on the user front and their work is been widely accepted.

Additionally our service, the OpenSource Usability Labs,  provides professional usability support to commercial Open-Source projects and traditional usability companys have detected Open-Source as a market by now.

Commercial Software always has a better Usability than Open Source Software.

First of all: the quality in commercial software varies as much as in open source. There are products with excellent usability around and there is just the opposite. In both cases the bad products die sooner or later.

So what we need to think about is: “What is possible for Usability in Open Source development?”

There are numerous Open Source projects around that provide excellent usability. Firefox challenges the Microsoft Internet Explorer. Think of projects like gallery, KDE4 or Tine2.0. All have undergone rewrites in order to enhance their usability and all have proven to be successful in relation to the age of the project.

So there is prove that Open Source projects are capable of a really good user experience.

Open-source software ends up being written for other developers.

This argument used to be true. Back in those good old days Open Source was successful, because developers could directly influence and change the software. If the software did not match their needs, they simply took the code and changed whatever they did not like. Projects split up, died, new ones were started - they evoluted. And by this they also evoluted a perfect usability - perfect for software developers which happened to be the main target group. In other words: those products evoluted perfect usability.

Nowadays that the user-base shifts, the goals in development differentiate. Projects that need to be used by average Joes and Janes build up user-feedback channels, integrate usability experts into the development and do regular user testing. They get designed for the average Joes and Janes.

Usability is a matter of a centralized lab.

This is actually not a special Open Source myth, but it is nevertheless wrong. Good usability can only be reached through a user-centric development process. A lab can be very handy during this process, but it is not the backbone. Usability experts need to be tightly integrated into the processes - from the definition of requirements, the evaluation of user goal, setting the information architecture to actually testing the products.

This is possible even in those distributed  and self-motivated development-teams you usually find in Open-Source projects. By tying it all into a single, centralized lab, as Sam Dean suggests, you would loose the strengths of this distributed development - just think of the requirements arising from different cultural needs.

Even more: Open-Source software is much more capable to integrate their users then a single lab would allow. Our experience is that Open-Source user are very willing to give feedback to the developers. While customers of commercial projects often ask: “What do I get, when I contribute?”, Open-Source users feel it is a good chance to say “Thank you” to the developers for providing a great piece of software.

By collaborating via the Internet it is not only a dream to activate this potential - it is reality. For example, we have just started an Icon Usability-Test for the new Oxygen k3b-Icons and we got more then 2000 participants within just 2 days.

Summing it up

The evolutionary process that stands behind Open Source development has already adopted to the idea of user centric development. Just as it will adopt to any other upcoming need in software development. And Usability on the other hand has started to understand the needs and the potentials of the Open Source idea, and makes great advances in activating them for the good of the projects.

For sure we are just at the beginning of a long journey. But we are already on the road. Articles like the ones from Sam and Matt show the increasing public demand for more usable Open Source products. I am sure the community notices these signals and will just speed up. The foundations are being laid…

Anne and I will be at the LinuxTag in Berlin. Hope it will be as much fun as in the last years!

The plans are at the moment that at least one of us will be there on Wednesday and Thursday afternoon, as well as Friday the whole day. We will be mainly at the Tine 2.0 booth (Halle 7.2a Stand 102a). You will have the possibility to participate in a small Icon-Usabilitytest to identify the best icons for the new Tine calender-app there!

We are looking forward to discuss any issue with you concerning the Usability of Tine 2.0 or any other Open Source Software.

Good news for all students of usability, user-interface design, interaction design,…: The initiative Open Usability has started the call for participation for the Season of Usability 2009 (a very big THANK YOU to Celeste for organizing and finding sponsors!!!).

The Season of Usability provides students the opportunity to practice what they have learned in university on a real Free/Libre/Open-source software project. The list of projects this year is great again:

• Amarok
• Drupal
• Gallery
• GeneMANIA
• Ubuntu - GNOME User Management Tool
• Kadu
• KOrganizer
• OLM
• SemNotes
• KUbuntu Add/Remove… Software

Of course I will again work as a usability-mentor for the SoU 2009. This year I am mentor for the Gallery project. This project is interested in learning more about its users. Project activities will include user research activities such as surveys, interviews, creating user groups and personas, and competitive analysis.

I will be really enjoying this job, as we have worked hard on our survey-platform Icon Test and the connected questionnaire methods. They will be of great use for the student and the gallery project.

Are any of those projects interesting for you? Then don’t hesitate to apply.

Hurray! Tine 2.0 has qualified as a finalist in the  Trophées du Libre. Congratulations to the whole team! Good work gets appreciated

Footnote:
I am in the jury of Trophées du Libre, but as I am juror in the category “Education”, I had no influence on the nomination of Tine in the category “Professional”.

Our family of companies (OpenSource-Usability-Labs, Apliki and binaere bauten) has finally moved from eGroupware to Tine 2.0 as our leading groupware system. Only for the in Tine 2.0 still missing calender we continue to use the eGroupware calendar.

I was quite nervous promoting and pushing the decision to do so. The dialogues and workflows of Tine show to a very great extent my handwriting. Therefore everything that does not yet work as expected falls back on me. And one thing is for sure: Tine 2.0 is still a young project, having many bugs and lacking features at this point of time.

So why did I promote the change? First of all I was always very unhappy about eGroupware. It is feature rich, but - many complex technical product have this problem - most users could not use more than perhaps 20% of these features. We had eGroupware running for more than 2 years now - still whenever I talked to Conny or Lars telling them that I failed to do this or that with eGroupware the answer was always the same: It is possible. Just this did not help me in my daily life. So in my eyes the lacking features of Tine 2.0 compared to eGroupware are to a great extent only virtually missing, because for the normal user eGroupware is not feature-rich, it is confusing.

The second reason to change to Tine 2.0 in productive use is of course to get feedback. I am in charge for the usability part of the development - but until now we did not have the chance to actually test the concepts we have thought out. So now is the time to get back to earth and see whether things work out the way we hope. Next to using Tine 2.0 ourselves, we will start doing active usability testing from February on.

So what is my intermediate result after about 3 weeks of active Tine usage?

Honestly, I am overwhelmed. There are a lot of bugs. To a great extent these are not severe bugs - they just get in your ways here and there - but hey: we are working on a development snapshot. What should one expect? But the integration of different aspects of groupware is just great. The filter-list-system is working well (of course: it still needs a little polishing). Still more than this: Using Tine is just fun. Working with eGroupware I always felt like: ok, this is the application we use, so teeth together and do what you need to do. So I always did what I really had to do, but never more than that. With Tine 2.0 it is just the opposite. I really enjoy working with Tine. I have caught myself just playing around with Tine, exploring the possibilities. This way I did what is most important for a groupware: I used it. I put all the information in Tine that never found their way into eGroupware.

Summing it up: except for the painfully missing calendar, I think Tine 2.0 is a great groupware. It has less features than eGroupware, but overcompensates this by being straight forward and fun to use. The decision to change from eGroupware to Tine 2.0 was a good one! I hope the usability tests we conduct in the next time will confirm this picture.

The Icon Usability Test of the TV-Browser icons has been closed for some time now. We have been busy learning from that study and working on further improvements for the icon test. So I totally forgot to write about the actual results…

Just as a reminder - the aim of the study was twofold:

First, we wanted to prove the method, which worked very well and we are going to publish the results in some scientific magazine sometime.

The second goal was to reveal the good and the not so good icons in TV-Browser. For the test we used some icons that are visible in TV-Browser and the tool-tips provided for them. We then asked the user of TV-Browser (they should be used to the icons!) to attend the survey - and they did indeed! A great “Thank you” to all of you who have participated!

And these are the results in short:

The following icons proved to be just great and really helpful for the user:

Print, Reminder Function, Favorite program, Search, Filter

A second set of icons worked alright, but has the potential to be improved:

List of currently running programs, Simple marker plug in

And finally some icons showed the definite necessity for an improvement:

Start external Program, Tool for translators, Get recent TV listings, Rate program

Now we are working on an second study, in which we will test alternative icons for those last functions. In this study we are going to present alterative icons to the users, which then have to pick the icons that fit best for he function. At last we are going to test the new set of icons to prove the improvement, that we hopefully will have achieved by then!

Next to TV-Browser, we are also working on a study for the testing of Tine 2.0 icons. Hopefully I will be back to you with some news on that issues soon!

Wow. Today we closed the Icon Test for TV-Browser. And man - this has been a real test for our Beta-Service. The result: Within one week more than 22000 individual tests have been conducted in our 18 experimental conditions, filling our database with 123404 rows of data. OpenOffice Calc gives up on this amount of data (As an interesting insight: MS Excel on the other hand can read that many rows of data, but not save them).

So it will take us some time to actually get valid (in term of significant) results, as this sheer amount of data first has to be handled. But I will uncover not too much when I say: a first look at the data is promising.

I guess we have found a valid way to actually have users judge the usability of icons of an application. The icon test is easy to set up, it is easy (ok, it will be easy once the platform is finished up…) to get the results and it is fast - in the best meaning of agile usability. You need valid feedback on your icons till tomorrow? No problem anymore!

With our icon test we are able to spot the good and the better-find-another icons in an interface, and we can easily decide which of a set of alternative icons for one term is the one users prefer most.

More soon to be seen on www.icon-test.com!

We just started an online-survey together with the guys from TV-Browser. The question we want to answer is twofold:

1. Which Icons the TV-Browser uses are easy to understand, helpful or in just one word: appropriate? Which Icons should TV-Browser use in future releases?
2. Does the method and the online-Tool we have developed for testing the quality of icons work as good, as we hope it does?

If you are curious: The study is online until the 1st of September 2008. You can participate even if you do not know much about the TV-Browser - as long as you use Firefox Webbrowser, because this study is still beta: Study for checking the quality of icons in TV-Browser.

After starting the study first problems became obvious immediately and most of them got solved quite quickly. I have learned, e.g. that a switch can be “half-broken” eating up lots of, but not all data-packages.

Summing it up: the study runs quite well and we are getting tons of answers by the TV-Browser-Users. It is always fun to have this great community involved into new studies.

On LinuxTag 2008 in Berlin the OpenSource Usability Labs were invited by the Tine 2.0 project. So I was at their booth a fair bit of the time, discussing the concepts, screens and ideas we have. The feedback was great - a lot of people encouraged us in our work. The main feedback was: there is no easy-to-use OpenSource groupware-solution for small and medium-sized enterprises around but there is definitely a need for one! So hey-ho - let’s go, Tine!

We demonstrated what we call a technology preview. This preview reveals some of the basic functionality and some of our interaction-concepts. The visitors were impressed by the ease of interaction made possible by newest Web 2.0 technology. But make up your own mind and try out what we have achieved so far.

Additionally we made a web-based icon-understandability test on the booth. So visitors could help us to understand what icons are already well understood - and which icons still need to be improved. This way visitors had the possibility to directly influence the usability of Tine 2.0. We will provide this facility for all of you, who could not attend LinuxTag on the Tine-Website soon. Tools like this are extremely important for achieving the certification as being developed according to ISO 13407.

Next to Tine I promoted the idea of the OpenSource Usability Labs. There were quite a few commercial OpenSource Projects around on LinuxTag. I talked to a lot of people about our ideas of professional OpenSource Usability, our philosophy and our business models - and most of them found the ideas very interesting.

I really enjoyed LinuxTag and already look forward to next year’s edition! Thanks to all the guys making this possible. We should not forget that this event is mainly the result of , oluntary work.