OpenSource Usability Labs

To the nearly 200 (173, to be exact) participants in our icon test: Thank you all for taking part! We got great results in terms of the quality of the Tine 2.0 icons (which are, in fact from the Oxygen Iconset). Also, we would like to thank you for all the comments you gave: Many of them were really encouraging, we got many compliments, but also some valuable criticism and tipps.

How did the icon test work?

We not only recorded which icons were chosen for which terms, but also how long it took you to decide.

From that, we could calculate three values:

• “Strength of association” (Indicates what percentage of the users assigned this icon with this term.),
• “Discriminatory power” (Indicates wether and how often this icon was assigned to other terms as well.) and
• “Conspicuity” (Indicates how fast the icon was chosen by the user in proportion to the average.)

In the pictures below you can find the three values on the left in the above order. The three values together give an overall Rating between 0 and 10. After a few tests we conducted, we can say that only icons with a rating above 8 can be seen as perfect for a term.

What is very apparent in all the results: The Oxygen Icons are not made for Tine! From the few terms we wanted to find the perfect icon for, only “User” got a rating above 8:

The weakest rating was achieved for “Today”, which is very interesting, because I expected this to be the most evident one:

Here are the results of all the other terms:

Add Appointment:

Add Calendar:

And Resource:

What do we learn from that?

First of all, we need icons that fit better to the special Tine 2.0 terms. Of course, the Oxygen Icons are a great start, but they just don’t fit all of our needs.

That’s why we want you, our community, to contribute!

• If you use Tine 2.0 and you see an icon which you think doesn’t fit: Tell us!
• If you are an icon designer and love to play with pixels: We need you!
• If you know how to visualize complicated things with as little complexity as possible: Write us!
• And even if you just know someone of the above (or know someone that knows someone :), we would like to hear from you!

Please write your ideas and suggestions to: tine.calendar@gmail.com or post in the Tine forum.

We planned it for the Linux-Tag, but then there wasn’t a good Internet connection -

So we ask you now: Which are the best icons for Tine?

Please take part in this very short test where we would like to find out which icon fits best for some of the Tine parts like “Ressources” or “Today”.

To help us make Tine 2.0 even more user friendly, please click here and choose you preferred language:

http://tiny.cc/icontest

Plus: Calendar testers needed!

We still need testers for our Usability test in two weeks. If you live in Berlin, or happen to be there in two weeks, please give me a shout at: tine.calendar@gmail.com

We are looking for Tine-experienced people but newbies are also great.

The test itself will take place in Kreuzberg for about an hour and your effort will be rewarded, of course.

Thank you very much!

Today I found an article by Sam Dean who asks for a Centralized, State-of-the-Art Open Source Usability Lab. He refers to a cnet article by Matt Asay in which he postulates what Open Source can learn from Apple. Both articles point out that there is need for a shifting view in Open Source. Open Source needs to be more user-driven and less developer-centric - in other words there is a need for usability in Open Source work.

Well, there are good and bad news for both of them:
It is not as easy as they think but we have already come much further than they think!

To explain this I would like to clarify some myths on Open Source Usability:

Usability plays no role in Open Source development.

The OpenUsability.org initiative has provided Usability guidance to hundreds of Open Source projects for more than 5 years. We have worked with various  projects from big ones like Wikipedia or KDE to very small ones. Many projects have developed their own Usability-Community like the OpenOffice Renaissance or the KDE Usability project. Celeste Paul - one of the members of OpenUsability and KDE Usability - has just recently been elected into the KDE e.V. board.

So there is a community willing to assist Open Source projects on the user front and their work is been widely accepted.

Additionally our service, the OpenSource Usability Labs,  provides professional usability support to commercial Open-Source projects and traditional usability companys have detected Open-Source as a market by now.

Commercial Software always has a better Usability than Open Source Software.

First of all: the quality in commercial software varies as much as in open source. There are products with excellent usability around and there is just the opposite. In both cases the bad products die sooner or later.

So what we need to think about is: “What is possible for Usability in Open Source development?”

There are numerous Open Source projects around that provide excellent usability. Firefox challenges the Microsoft Internet Explorer. Think of projects like gallery, KDE4 or Tine2.0. All have undergone rewrites in order to enhance their usability and all have proven to be successful in relation to the age of the project.

So there is prove that Open Source projects are capable of a really good user experience.

Open-source software ends up being written for other developers.

This argument used to be true. Back in those good old days Open Source was successful, because developers could directly influence and change the software. If the software did not match their needs, they simply took the code and changed whatever they did not like. Projects split up, died, new ones were started - they evoluted. And by this they also evoluted a perfect usability - perfect for software developers which happened to be the main target group. In other words: those products evoluted perfect usability.

Nowadays that the user-base shifts, the goals in development differentiate. Projects that need to be used by average Joes and Janes build up user-feedback channels, integrate usability experts into the development and do regular user testing. They get designed for the average Joes and Janes.

Usability is a matter of a centralized lab.

This is actually not a special Open Source myth, but it is nevertheless wrong. Good usability can only be reached through a user-centric development process. A lab can be very handy during this process, but it is not the backbone. Usability experts need to be tightly integrated into the processes - from the definition of requirements, the evaluation of user goal, setting the information architecture to actually testing the products.

This is possible even in those distributed  and self-motivated development-teams you usually find in Open-Source projects. By tying it all into a single, centralized lab, as Sam Dean suggests, you would loose the strengths of this distributed development - just think of the requirements arising from different cultural needs.

Even more: Open-Source software is much more capable to integrate their users then a single lab would allow. Our experience is that Open-Source user are very willing to give feedback to the developers. While customers of commercial projects often ask: “What do I get, when I contribute?”, Open-Source users feel it is a good chance to say “Thank you” to the developers for providing a great piece of software.

By collaborating via the Internet it is not only a dream to activate this potential - it is reality. For example, we have just started an Icon Usability-Test for the new Oxygen k3b-Icons and we got more then 2000 participants within just 2 days.

Summing it up

The evolutionary process that stands behind Open Source development has already adopted to the idea of user centric development. Just as it will adopt to any other upcoming need in software development. And Usability on the other hand has started to understand the needs and the potentials of the Open Source idea, and makes great advances in activating them for the good of the projects.

For sure we are just at the beginning of a long journey. But we are already on the road. Articles like the ones from Sam and Matt show the increasing public demand for more usable Open Source products. I am sure the community notices these signals and will just speed up. The foundations are being laid…

Anne and I will be at the LinuxTag in Berlin. Hope it will be as much fun as in the last years!

The plans are at the moment that at least one of us will be there on Wednesday and Thursday afternoon, as well as Friday the whole day. We will be mainly at the Tine 2.0 booth (Halle 7.2a Stand 102a). You will have the possibility to participate in a small Icon-Usabilitytest to identify the best icons for the new Tine calender-app there!

We are looking forward to discuss any issue with you concerning the Usability of Tine 2.0 or any other Open Source Software.

Thank you for taking the time answer our survey!
We had a great outcome with a total of 365 successfully completed surveys and 106 partially finished ones.

A few things that came out that you might find interesting:

• Overall, 346 people took the survey in German, 68 in English
• 30% were women, 70% were men
• on average, people were 31 years old
• 85% come from Germany, 10% from the Rest of Europe, the remaining 5% are from other parts of the world including China and South Africa (!)
• Professional appointments: women mostly use papercalendars (65%) and locally installed calendar applications (44%), men mostly use locally installed calendar applications (54%) and Smartphone calendars (46%)
• Private appointments: women mostly use paper calendars (83%) and locally installed calendar applications (24%), men mostly use Smartphone calendars (49%) and locally installed calendar applications (45%)
• The calendar application used the most is Microsoft Outlook (45%) followed by others (25%) and eGroupware (23%)
• 43% of all people asked use a smartphone, the biggest smartphone brand being Nokia (30%) followed by Apple iPhone/iPod touch (27%)
• 1/3 of smartphone users synchronize daily between phone and computer
• Per week, 3/4 of all respondents make one or no all day appointment, 2-10 appointments shorter than one day, none with international partners and 1-5 recurring appointments
• Half of all people who answered don’t make any multi-day appointments per week
• Nearly half of all people (45%) don’t make any public appointments, while 2/3 of them make 2-5 private appointments per week
• Nearly two thirds (65%) of the people asked find it very important or important to include tasks and milestones into their calendar
• Slightly more than two thirds of all respondents find it very important or important to make appointments distinguishable by colour
• But there no noticeable tendency when it comes to accessing other calendars to coordinate appointments: 39% say, that this is absolutely not important or not important to them, while 34% say the opposite. 20% are inbetween, though.
• An clear majority of everyone (86%) who answered wants to have a “quick save possibility”, with start (98%), name (92%), place (67%) and end (58%) being the most important information for creating an appointment.
• As for the calendar view, people considered the week view to be the most important one followed by the day view.
• Also, Pop-Up seems to be the most favourite way of all the respondents to be reminded of an appointment, 2/3 (67%) voted for that, followed by sound signal (48%).
• And for the last question: As you can see in the picture below, synchronization is what people miss most in their current calendar, followed by reminders and groupware functionality:

If you have any questions regarding the survey, please ask!

Based on your great responses to the survey I have been busy building a calendar prototype in Axure.

If you want, you can take a look at it. (German version only.)

Comments appreciated!

We need your answers!

Please take 10 minutes to answer a few questions about how you use calendars.

All answers will be used to make the new Tine calendar fit everbody’s needs perfectly!

The survey is in two languages:

English version: http://tiny.cc/calendarsurvey_en

German version: http://tiny.cc/calendarsurvey_de

We really appreciate your contribution.

Hurray! Tine 2.0 has qualified as a finalist in the  Trophées du Libre. Congratulations to the whole team! Good work gets appreciated

Footnote:
I am in the jury of Trophées du Libre, but as I am juror in the category “Education”, I had no influence on the nomination of Tine in the category “Professional”.

Today I have started an introduction on the main concepts of Tine 2.0. I hope this will be helpful to new and experienced Tine-users.

I have chosen not to post this introduction as blog-post, since I will try to keep them up-to-date. So you will have to take a look at the static articles in our blog.

So far there is:

• Tine 2.0: concepts and ideas
  • Choosing an application
  • Filtering Lists
  • Places - or how to manage visibility
  • Working with records

I am very happy about comments since I will try to keep working and improving the articles.

We were in the lucky situation to conduct a usability-test with seven volunteers of a German company. They are using Tine 2.0 mainly as CRM-Tool to coordinate their sales-department. Consequently they are using contacts, CRM and tasks, but not the other apps of Tine 2.0. All user were regular, but not frequent users of Tine 2.0 and moderately comfortable with computers in general.

The tests were concepted as semi-structured, task-oriented interviews. I lead all interviews, while Conny Weiß was participating as an active observer. He was allowed to ask and answer any questions during the interview.

The scenario was as follows:

1. Users had to log-in and explain their general impressions and understanding of Tine 2.0.
2. Users had to find and modify an existing contact. Focus of this task was the use of the filter-system as a quick-filter, the short-view of a contact and the modification of an item.
3. Users then had to export a list of contacts to .pdf, that had to be composed by a rather complex filter-setting.
4. Next they were asked to create a to-do-list which only a certain group of people was allowed to see.
5. Finally they were asked to show how they manage a typical task of their daily work with Tine 2.0.

The interviews were very informative. The results were aggregated and transformed into tasks for further development. The detailed results can be found in the bug-tracking-systems of Tine 2.0. Following I will give an overview of the most important findings:

1. General understanding of Tine 2.0 is good
   All users were able to explain the general concepts of Tine 2.0 and were competent to solve even complex tasks. This encourages us to stay with the chosen general approach of Tine 2.0, namely keeping the general screen-estate, the filter-list-view, the application-pile etc.
2. Sometimes Tine 2.0 gets in the users’ way
   In some details Tine 2.0 is not enough consistent, predictive or supportive for the user. We have found heaps of these small Usability-Bugs, showing us that we have to be even more precise with all those little things. Examples for this category are:
   * The addressbook-container has not been correctly pre-selected when adding a new contact
   * Some wording was not understood
   * Not sufficient feedback when filtering caused no results
   * Number of elements in lists are not configurable
   * …
3. Notes should be editable
   The concept of notes was appreciated by the users, but they wanted notes to be editable and they wanted notes to show up at more then one place.
4. No connection between CRM and Tasks
   Users had the problem that they cannot see which CRM-Lead a certain task belongs to.
5. Use of Right-Mouse-Buttons is confusing
   Users were not able to predict where and how they could use the right-mouse-button-menu.
6. Selection of a person is confusing
   The dialogue to select e.g. the responsible person for a task lead to some irritation for the user.

This is just a short summary of the most important findings. A lot of the issues have already been fixed for the upcoming Tine 2.0 Lara release. All other issues are integrated into the further development schedule.

Summing it up: the user test was well worth it and helped us a lot to loose our expert view and once again see Tine 2.0 through the eyes of a “normal user”.

We want you - for Tine 2.0:

If you are interested to support Tine 2.0 development, then join our mailing list for voluntary usability-testers. Your help is very much appreciated and taking part in these tests only takes a little time and is fun! Take a look at our Icon Test demo to get a feeling what these usability-tests could look like.