OpenSource Usability Labs

I want to thank everyone who was interested in taking part in the Usability calendar tests. Unfortunately I could only invite people from Berlin, because the tests had to be conducted in person. So a special thanks to all the testers that came!

In the test I focused on the Tine 2.0 calendar. The tasks included creating a shared calendar, creating whole day and recurring events, inviting, copying data from the address book to the description of an event, accepting an invitation and moving an appointment to another calendar.

These are the top 5 problems the testers had:

1. For moving an appointment to another calendar, nearly everyone tried to drag and drop this appointment. (#1460 in the Bugtracker)
2. When inviting people, 2/3 of the testers expected to find the accounts, they shared the calendar with, to be first in the invitation menu. (#1522)
3. When creating a recurring event, 2/3 of all testers found “Every 1. month” irritating. A proposal for redesign is attached to the issue in the Bugtracker. (#1524)
4. Accepting or declinig an invitation is not obvious and fast enough. Two thirds of everyone struggled to complete the task. They couldn’t find said invitation without knowing at which day it was and then didn’t expect to have to open the Edit window to accept. (#1440)
5. When copying data from the business card area in the address book, more than half of all testers tried highlighting and then right clicking to copy. This didn’t work. (#1422)

Including the above problems there will be a priority list handed over to the great guys at Metaways and we’ll hopefully see most of the things already included in the next release of Tine 2.0.

If you have questions, please email me or write a comment.

To the nearly 200 (173, to be exact) participants in our icon test: Thank you all for taking part! We got great results in terms of the quality of the Tine 2.0 icons (which are, in fact from the Oxygen Iconset). Also, we would like to thank you for all the comments you gave: Many of them were really encouraging, we got many compliments, but also some valuable criticism and tipps.

How did the icon test work?

We not only recorded which icons were chosen for which terms, but also how long it took you to decide.

From that, we could calculate three values:

• “Strength of association” (Indicates what percentage of the users assigned this icon with this term.),
• “Discriminatory power” (Indicates wether and how often this icon was assigned to other terms as well.) and
• “Conspicuity” (Indicates how fast the icon was chosen by the user in proportion to the average.)

In the pictures below you can find the three values on the left in the above order. The three values together give an overall Rating between 0 and 10. After a few tests we conducted, we can say that only icons with a rating above 8 can be seen as perfect for a term.

What is very apparent in all the results: The Oxygen Icons are not made for Tine! From the few terms we wanted to find the perfect icon for, only “User” got a rating above 8:

The weakest rating was achieved for “Today”, which is very interesting, because I expected this to be the most evident one:

Here are the results of all the other terms:

Add Appointment:

Add Calendar:

And Resource:

What do we learn from that?

First of all, we need icons that fit better to the special Tine 2.0 terms. Of course, the Oxygen Icons are a great start, but they just don’t fit all of our needs.

That’s why we want you, our community, to contribute!

• If you use Tine 2.0 and you see an icon which you think doesn’t fit: Tell us!
• If you are an icon designer and love to play with pixels: We need you!
• If you know how to visualize complicated things with as little complexity as possible: Write us!
• And even if you just know someone of the above (or know someone that knows someone , we would like to hear from you!

Please write your ideas and suggestions to: tine.calendar@gmail.com or post in the Tine forum.

Anne and I will be at the LinuxTag in Berlin. Hope it will be as much fun as in the last years!

The plans are at the moment that at least one of us will be there on Wednesday and Thursday afternoon, as well as Friday the whole day. We will be mainly at the Tine 2.0 booth (Halle 7.2a Stand 102a). You will have the possibility to participate in a small Icon-Usabilitytest to identify the best icons for the new Tine calender-app there!

We are looking forward to discuss any issue with you concerning the Usability of Tine 2.0 or any other Open Source Software.

Thank you for taking the time answer our survey!
We had a great outcome with a total of 365 successfully completed surveys and 106 partially finished ones.

A few things that came out that you might find interesting:

• Overall, 346 people took the survey in German, 68 in English
• 30% were women, 70% were men
• on average, people were 31 years old
• 85% come from Germany, 10% from the Rest of Europe, the remaining 5% are from other parts of the world including China and South Africa (!)
• Professional appointments: women mostly use papercalendars (65%) and locally installed calendar applications (44%), men mostly use locally installed calendar applications (54%) and Smartphone calendars (46%)
• Private appointments: women mostly use paper calendars (83%) and locally installed calendar applications (24%), men mostly use Smartphone calendars (49%) and locally installed calendar applications (45%)
• The calendar application used the most is Microsoft Outlook (45%) followed by others (25%) and eGroupware (23%)
• 43% of all people asked use a smartphone, the biggest smartphone brand being Nokia (30%) followed by Apple iPhone/iPod touch (27%)
• 1/3 of smartphone users synchronize daily between phone and computer
• Per week, 3/4 of all respondents make one or no all day appointment, 2-10 appointments shorter than one day, none with international partners and 1-5 recurring appointments
• Half of all people who answered don’t make any multi-day appointments per week
• Nearly half of all people (45%) don’t make any public appointments, while 2/3 of them make 2-5 private appointments per week
• Nearly two thirds (65%) of the people asked find it very important or important to include tasks and milestones into their calendar
• Slightly more than two thirds of all respondents find it very important or important to make appointments distinguishable by colour
• But there no noticeable tendency when it comes to accessing other calendars to coordinate appointments: 39% say, that this is absolutely not important or not important to them, while 34% say the opposite. 20% are inbetween, though.
• An clear majority of everyone (86%) who answered wants to have a “quick save possibility”, with start (98%), name (92%), place (67%) and end (58%) being the most important information for creating an appointment.
• As for the calendar view, people considered the week view to be the most important one followed by the day view.
• Also, Pop-Up seems to be the most favourite way of all the respondents to be reminded of an appointment, 2/3 (67%) voted for that, followed by sound signal (48%).
• And for the last question: As you can see in the picture below, synchronization is what people miss most in their current calendar, followed by reminders and groupware functionality:

If you have any questions regarding the survey, please ask!

We need your answers!

Please take 10 minutes to answer a few questions about how you use calendars.

All answers will be used to make the new Tine calendar fit everbody’s needs perfectly!

The survey is in two languages:

English version: http://tiny.cc/calendarsurvey_en

German version: http://tiny.cc/calendarsurvey_de

We really appreciate your contribution.

Hurray! Tine 2.0 has qualified as a finalist in the  Trophées du Libre. Congratulations to the whole team! Good work gets appreciated

Footnote:
I am in the jury of Trophées du Libre, but as I am juror in the category “Education”, I had no influence on the nomination of Tine in the category “Professional”.

I am very happy that Anne decided to join the Tine 2.0 Team. Anne studies international media informatics at the FHTW Berlin. She is doing her bachelor work on the Tine 2.0 calendar. In this scope she will analyse personas and use-cases for the calendar as well as other existing calendars. She will develop the interaction- and interface-design and do some surveys and usability-tests. Of course she will regularly blog about her work.

Welcome, Anne!

Seeing the upcoming surveys: if you haven’t registered for participating in these tests yet, you shouldn’t wait any longer. This is your chance to feedback on the Tine 2.0 calendar and other Tine 2.0 development! Also feel invited to discuss any issue on the calendar in our forum!

After Conny asked me to, I moved the introduction of Tine 2.0 Concepts into the Tine 2.0 Wiki today. When I did this I was shocked to see how old most information in the Users-section was. So additionally to adding the introduction I ended up updating some more pages in the wiki. But as time is always a little short there is still some work to do. So if you feel competent, please add some features to the description of the Tine 2.0 applications.

Today I have started an introduction on the main concepts of Tine 2.0. I hope this will be helpful to new and experienced Tine-users.

I have chosen not to post this introduction as blog-post, since I will try to keep them up-to-date. So you will have to take a look at the static articles in our blog.

So far there is:

• Tine 2.0: concepts and ideas
  • Choosing an application
  • Filtering Lists
  • Places – or how to manage visibility
  • Working with records

I am very happy about comments since I will try to keep working and improving the articles.

We were in the lucky situation to conduct a usability-test with seven volunteers of a German company. They are using Tine 2.0 mainly as CRM-Tool to coordinate their sales-department. Consequently they are using contacts, CRM and tasks, but not the other apps of Tine 2.0. All user were regular, but not frequent users of Tine 2.0 and moderately comfortable with computers in general.

The tests were concepted as semi-structured, task-oriented interviews. I lead all interviews, while Conny Weiß was participating as an active observer. He was allowed to ask and answer any questions during the interview.

The scenario was as follows:

1. Users had to log-in and explain their general impressions and understanding of Tine 2.0.
2. Users had to find and modify an existing contact. Focus of this task was the use of the filter-system as a quick-filter, the short-view of a contact and the modification of an item.
3. Users then had to export a list of contacts to .pdf, that had to be composed by a rather complex filter-setting.
4. Next they were asked to create a to-do-list which only a certain group of people was allowed to see.
5. Finally they were asked to show how they manage a typical task of their daily work with Tine 2.0.

The interviews were very informative. The results were aggregated and transformed into tasks for further development. The detailed results can be found in the bug-tracking-systems of Tine 2.0. Following I will give an overview of the most important findings:

1. General understanding of Tine 2.0 is good
   All users were able to explain the general concepts of Tine 2.0 and were competent to solve even complex tasks. This encourages us to stay with the chosen general approach of Tine 2.0, namely keeping the general screen-estate, the filter-list-view, the application-pile etc.
2. Sometimes Tine 2.0 gets in the users’ way
   In some details Tine 2.0 is not enough consistent, predictive or supportive for the user. We have found heaps of these small Usability-Bugs, showing us that we have to be even more precise with all those little things. Examples for this category are:
   * The addressbook-container has not been correctly pre-selected when adding a new contact
   * Some wording was not understood
   * Not sufficient feedback when filtering caused no results
   * Number of elements in lists are not configurable
   * …
3. Notes should be editable
   The concept of notes was appreciated by the users, but they wanted notes to be editable and they wanted notes to show up at more then one place.
4. No connection between CRM and Tasks
   Users had the problem that they cannot see which CRM-Lead a certain task belongs to.
5. Use of Right-Mouse-Buttons is confusing
   Users were not able to predict where and how they could use the right-mouse-button-menu.
6. Selection of a person is confusing
   The dialogue to select e.g. the responsible person for a task lead to some irritation for the user.

This is just a short summary of the most important findings. A lot of the issues have already been fixed for the upcoming Tine 2.0 Lara release. All other issues are integrated into the further development schedule.

Summing it up: the user test was well worth it and helped us a lot to loose our expert view and once again see Tine 2.0 through the eyes of a “normal user”.

We want you – for Tine 2.0:

If you are interested to support Tine 2.0 development, then join our mailing list for voluntary usability-testers. Your help is very much appreciated and taking part in these tests only takes a little time and is fun! Take a look at our Icon Test demo to get a feeling what these usability-tests could look like.