OpenSource Usability Labs

I am very happy that our blog is now aggregated on planet KDE!

I am Björn, usability professional for more than 10 years and one of the founders of OpenUsability.org. I have been working in and with open source for a very long time now. Therefore I am very happy that we now found a way how we as a usability consulting company can actually contribute to KDE.

And this is what we are going to do first: We are supporting Nuno Pinhero and the other artists of KDE by measuring the quality or usability of the icons used in the oxygen icon set.We do this in order to continuously improve the quality of the standard icon set used for the KDE SC - and make KDE SC rock even more!

To achieve this goal we will need your help. We will regularly ask you to participate in a short icon test survey. We will try to keep all surveys shorter than 5 minutes, so it is not too much of a hassle for you. We plan to set up a new survey every 2-4 weeks. The first will start in the next couple of days.

Also I am personally interested in the intercultural quality of icons or, say it differently: the necessity to internationalize not only text, but icons as well. To achieve this we will on the long run need some people willing to help us translating the surveys. If you should be interested, please just send me a mail.

We are able to do what we do here, because we are running a service that helps developers to understand what their users actually want. Testing icons is only one part of the game - there are many more possibilities. We are - additionally to what we do with the artists team - very happy to support anyone from the KDE community in getting to know their users and develop even better products. Same as above: If you are interested, please send me a mail.

First of all:
A BIG thank you to all the 143 participants that took all the way of our survey! Your feedback helps us a lot, because we are doing Tine 2.0 only for you! If you missed participation this time: join us on our mailing-list for Tine 2.0 surveys!

Let’s take a look at the main results:

What shall we do next?

We asked you to sort a couple of ideas we have on what features we could work on next. It showed no real winner, so I guess all our ideas have to be integrated in Tine 2.0. Still some points that seem to be more important are:

• Dashboard
• Birthday in Calendar
• Attachments
• Editable Notes
• Linking

These 5 were the winners and we will gladly consider your voting in our feature-plan.

Also we got a lot of comments on features you would like to see. Of course, we will discuss your ideas and  in the next survey we will ask how relevant they are for all of you!

How do you like and use Tine 2.0?

Read the rest of this entry »

This summer I participated in the SoU project with Gallery, the open source photo sharing software. The broad goal of the project was to conduct a survey to learn more about Gallery’s users in moving toward the release of the 3rd version of the software. The project was a success in terms of garnering useful data for Gallery, but, importantly, it was an incredibly valuable learning experience for me.

As a student working towards a masters in Human-Computer Interaction, I have learned about and conducted user research and usability studies in my coursework. But, what SoU provided me that no school project ever could was the opportunity to engage in research that has an actual impact on improving a product— and the opportunity to experience all the challenges that go along with that.

At the start of the project, I have to say I was a bit overwhelmed with what direction to take, but with the guidance of Björn as my mentor, the help of Jakob, a former SoU student who continues to work on the Gallery project, and the awesome tools of Usability-Methods.com, I soon found my way. Here are some of the lessons I learned along the way.

Read the rest of this entry »

There are many directions for the Tine 2.0 development we are discussing at the moment. But in the good tradition of our development we really want to know what you - our users and potential user - want us to do. So we decided to start a short survey and kindly ask you to participate - it will cost you less then 5 minutes and helps you and us to make Tine 2.0 the most rocking groupware around!

Tine 2.0 user survey

(The survey is powered by usability-methods.com)

I want to thank everyone who was interested in taking part in the Usability calendar tests. Unfortunately I could only invite people from Berlin, because the tests had to be conducted in person. So a special thanks to all the testers that came!

In the test I focused on the Tine 2.0 calendar. The tasks included creating a shared calendar, creating whole day and recurring events, inviting, copying data from the address book to the description of an event, accepting an invitation and moving an appointment to another calendar.

These are the top 5 problems the testers had:

1. For moving an appointment to another calendar, nearly everyone tried to drag and drop this appointment. (#1460 in the Bugtracker)
2. When inviting people, 2/3 of the testers expected to find the accounts, they shared the calendar with, to be first in the invitation menu. (#1522)
3. When creating a recurring event, 2/3 of all testers found “Every 1. month” irritating. A proposal for redesign is attached to the issue in the Bugtracker. (#1524)
4. Accepting or declinig an invitation is not obvious and fast enough. Two thirds of everyone struggled to complete the task. They couldn’t find said invitation without knowing at which day it was and then didn’t expect to have to open the Edit window to accept. (#1440)
5. When copying data from the business card area in the address book, more than half of all testers tried highlighting and then right clicking to copy. This didn’t work. (#1422)

Including the above problems there will be a priority list handed over to the great guys at Metaways and we’ll hopefully see most of the things already included in the next release of Tine 2.0.

If you have questions, please email me or write a comment.

To the nearly 200 (173, to be exact) participants in our icon test: Thank you all for taking part! We got great results in terms of the quality of the Tine 2.0 icons (which are, in fact from the Oxygen Iconset). Also, we would like to thank you for all the comments you gave: Many of them were really encouraging, we got many compliments, but also some valuable criticism and tipps.

How did the icon test work?

We not only recorded which icons were chosen for which terms, but also how long it took you to decide.

From that, we could calculate three values:

• “Strength of association” (Indicates what percentage of the users assigned this icon with this term.),
• “Discriminatory power” (Indicates wether and how often this icon was assigned to other terms as well.) and
• “Conspicuity” (Indicates how fast the icon was chosen by the user in proportion to the average.)

In the pictures below you can find the three values on the left in the above order. The three values together give an overall Rating between 0 and 10. After a few tests we conducted, we can say that only icons with a rating above 8 can be seen as perfect for a term.

What is very apparent in all the results: The Oxygen Icons are not made for Tine! From the few terms we wanted to find the perfect icon for, only “User” got a rating above 8:

The weakest rating was achieved for “Today”, which is very interesting, because I expected this to be the most evident one:

Here are the results of all the other terms:

Add Appointment:

Add Calendar:

And Resource:

What do we learn from that?

First of all, we need icons that fit better to the special Tine 2.0 terms. Of course, the Oxygen Icons are a great start, but they just don’t fit all of our needs.

That’s why we want you, our community, to contribute!

• If you use Tine 2.0 and you see an icon which you think doesn’t fit: Tell us!
• If you are an icon designer and love to play with pixels: We need you!
• If you know how to visualize complicated things with as little complexity as possible: Write us!
• And even if you just know someone of the above (or know someone that knows someone :), we would like to hear from you!

Please write your ideas and suggestions to: tine.calendar@gmail.com or post in the Tine forum.

We planned it for the Linux-Tag, but then there wasn’t a good Internet connection -

So we ask you now: Which are the best icons for Tine?

Please take part in this very short test where we would like to find out which icon fits best for some of the Tine parts like “Ressources” or “Today”.

To help us make Tine 2.0 even more user friendly, please click here and choose you preferred language:

http://tiny.cc/icontest

Plus: Calendar testers needed!

We still need testers for our Usability test in two weeks. If you live in Berlin, or happen to be there in two weeks, please give me a shout at: tine.calendar@gmail.com

We are looking for Tine-experienced people but newbies are also great.

The test itself will take place in Kreuzberg for about an hour and your effort will be rewarded, of course.

Thank you very much!

Today I found an article by Sam Dean who asks for a Centralized, State-of-the-Art Open Source Usability Lab. He refers to a cnet article by Matt Asay in which he postulates what Open Source can learn from Apple. Both articles point out that there is need for a shifting view in Open Source. Open Source needs to be more user-driven and less developer-centric - in other words there is a need for usability in Open Source work.

Well, there are good and bad news for both of them:
It is not as easy as they think but we have already come much further than they think!

To explain this I would like to clarify some myths on Open Source Usability:

Usability plays no role in Open Source development.

The OpenUsability.org initiative has provided Usability guidance to hundreds of Open Source projects for more than 5 years. We have worked with various  projects from big ones like Wikipedia or KDE to very small ones. Many projects have developed their own Usability-Community like the OpenOffice Renaissance or the KDE Usability project. Celeste Paul - one of the members of OpenUsability and KDE Usability - has just recently been elected into the KDE e.V. board.

So there is a community willing to assist Open Source projects on the user front and their work is been widely accepted.

Additionally our service, the OpenSource Usability Labs,  provides professional usability support to commercial Open-Source projects and traditional usability companys have detected Open-Source as a market by now.

Commercial Software always has a better Usability than Open Source Software.

First of all: the quality in commercial software varies as much as in open source. There are products with excellent usability around and there is just the opposite. In both cases the bad products die sooner or later.

So what we need to think about is: “What is possible for Usability in Open Source development?”

There are numerous Open Source projects around that provide excellent usability. Firefox challenges the Microsoft Internet Explorer. Think of projects like gallery, KDE4 or Tine2.0. All have undergone rewrites in order to enhance their usability and all have proven to be successful in relation to the age of the project.

So there is prove that Open Source projects are capable of a really good user experience.

Open-source software ends up being written for other developers.

This argument used to be true. Back in those good old days Open Source was successful, because developers could directly influence and change the software. If the software did not match their needs, they simply took the code and changed whatever they did not like. Projects split up, died, new ones were started - they evoluted. And by this they also evoluted a perfect usability - perfect for software developers which happened to be the main target group. In other words: those products evoluted perfect usability.

Nowadays that the user-base shifts, the goals in development differentiate. Projects that need to be used by average Joes and Janes build up user-feedback channels, integrate usability experts into the development and do regular user testing. They get designed for the average Joes and Janes.

Usability is a matter of a centralized lab.

This is actually not a special Open Source myth, but it is nevertheless wrong. Good usability can only be reached through a user-centric development process. A lab can be very handy during this process, but it is not the backbone. Usability experts need to be tightly integrated into the processes - from the definition of requirements, the evaluation of user goal, setting the information architecture to actually testing the products.

This is possible even in those distributed  and self-motivated development-teams you usually find in Open-Source projects. By tying it all into a single, centralized lab, as Sam Dean suggests, you would loose the strengths of this distributed development - just think of the requirements arising from different cultural needs.

Even more: Open-Source software is much more capable to integrate their users then a single lab would allow. Our experience is that Open-Source user are very willing to give feedback to the developers. While customers of commercial projects often ask: “What do I get, when I contribute?”, Open-Source users feel it is a good chance to say “Thank you” to the developers for providing a great piece of software.

By collaborating via the Internet it is not only a dream to activate this potential - it is reality. For example, we have just started an Icon Usability-Test for the new Oxygen k3b-Icons and we got more then 2000 participants within just 2 days.

Summing it up

The evolutionary process that stands behind Open Source development has already adopted to the idea of user centric development. Just as it will adopt to any other upcoming need in software development. And Usability on the other hand has started to understand the needs and the potentials of the Open Source idea, and makes great advances in activating them for the good of the projects.

For sure we are just at the beginning of a long journey. But we are already on the road. Articles like the ones from Sam and Matt show the increasing public demand for more usable Open Source products. I am sure the community notices these signals and will just speed up. The foundations are being laid…

Anne and I will be at the LinuxTag in Berlin. Hope it will be as much fun as in the last years!

The plans are at the moment that at least one of us will be there on Wednesday and Thursday afternoon, as well as Friday the whole day. We will be mainly at the Tine 2.0 booth (Halle 7.2a Stand 102a). You will have the possibility to participate in a small Icon-Usabilitytest to identify the best icons for the new Tine calender-app there!

We are looking forward to discuss any issue with you concerning the Usability of Tine 2.0 or any other Open Source Software.

Thank you for taking the time answer our survey!
We had a great outcome with a total of 365 successfully completed surveys and 106 partially finished ones.

A few things that came out that you might find interesting:

• Overall, 346 people took the survey in German, 68 in English
• 30% were women, 70% were men
• on average, people were 31 years old
• 85% come from Germany, 10% from the Rest of Europe, the remaining 5% are from other parts of the world including China and South Africa (!)
• Professional appointments: women mostly use papercalendars (65%) and locally installed calendar applications (44%), men mostly use locally installed calendar applications (54%) and Smartphone calendars (46%)
• Private appointments: women mostly use paper calendars (83%) and locally installed calendar applications (24%), men mostly use Smartphone calendars (49%) and locally installed calendar applications (45%)
• The calendar application used the most is Microsoft Outlook (45%) followed by others (25%) and eGroupware (23%)
• 43% of all people asked use a smartphone, the biggest smartphone brand being Nokia (30%) followed by Apple iPhone/iPod touch (27%)
• 1/3 of smartphone users synchronize daily between phone and computer
• Per week, 3/4 of all respondents make one or no all day appointment, 2-10 appointments shorter than one day, none with international partners and 1-5 recurring appointments
• Half of all people who answered don’t make any multi-day appointments per week
• Nearly half of all people (45%) don’t make any public appointments, while 2/3 of them make 2-5 private appointments per week
• Nearly two thirds (65%) of the people asked find it very important or important to include tasks and milestones into their calendar
• Slightly more than two thirds of all respondents find it very important or important to make appointments distinguishable by colour
• But there no noticeable tendency when it comes to accessing other calendars to coordinate appointments: 39% say, that this is absolutely not important or not important to them, while 34% say the opposite. 20% are inbetween, though.
• An clear majority of everyone (86%) who answered wants to have a “quick save possibility”, with start (98%), name (92%), place (67%) and end (58%) being the most important information for creating an appointment.
• As for the calendar view, people considered the week view to be the most important one followed by the day view.
• Also, Pop-Up seems to be the most favourite way of all the respondents to be reminded of an appointment, 2/3 (67%) voted for that, followed by sound signal (48%).
• And for the last question: As you can see in the picture below, synchronization is what people miss most in their current calendar, followed by reminders and groupware functionality:

If you have any questions regarding the survey, please ask!