Why could that be of interest for you?

• I will be much more engaged in the usability of Free Software projects in future.
  Next to the work I have been doing and will continue to do for Tine 2.0, I already teamed up much closer with the KDE and the LibreOffice community. And I am actually still looking for more challenging projects from the Free Software World.
• I will try to re-animate OpenUsability.org.
  I care a lot about this initiative. It tries to accomplish two things: First it helps to provide Usability Competence to Free Software projects. Secondly it tries to make process and work done in usability more transparent, so more people can benefit from it. Sad enough, this project is basically dead at the moment, but I want to work to re-animate it again (And you are happily invited to help re-building the structures we need to fulfill our duty!).
• I will free a software for user integration.
  Over the years at my old company, we have been developing a web based tool that helps to integrate users into the development process of software. I am currently working to release this software under the aGPL (Again, you are happily invited to join and support me in this effort).

The first and most important result: For the first time we reached all our Usability goals with Tine 2.0!

Read on to get the whole picture.

Demographics

The answers we got were from male (97%) people with an avarage age of 36 (SD=8.4). Most of them came from Germany (82%) followed by Austria and Italy (3% each), Sweden, Switzerland and China (2% each). Altogether there were answers from 13 different countries speaking 11 different languages. German of course was the most prominent one (84%), followed by Italian (4%), Chinese (3%), Russian and Swedish (2% each). Seeing this, the vast majority of Tine 2.0 user still is in Germany. This is not surprising as this project actually started in Germany. But it is great to see the increasing numbers not only of users taking part in the survey, but also the increasing numbers of countries our users come from.

To stress the fact that we do have a quickly growing community, 45% of the answers were from new users of Tine 2.0, using it less than 3 months. Interestingly enough, our surveys seem to be encouraging even for long term users: 25% reported to be using Tine 2.0 for more than 1 year. Apart from this indicator 90% of the participants stated they would like to take part in further surveys. So our surveys seem to be on the right track for you.

In the past we have been struggling with the fact that we reached too little non-technical or non-administrative users in our surveys. Again, the results of this survey are very promising: About 50% of the participants  stated that they were primarily end-users of Tine 2.0.

Still seeing the amount of male participants, we do not reach a real representative sample of our users yet. This is something we will continue to work on.

Benchmarking

In every Tine 2.0 survey we include benchmarking questions: We ask the participants to rate the state of Tine 2.0 and the individual components on a scale from ‘needs improvements’ (1) to ‘excellent’ (7). Looking at the answers of all users we find that the estimation of untouched components as well as the overall impression of Tine 2.0 stays stable compared to previous results. Looking at the values we can clearly see that all users are satisfied (values above 5) with the general impression (5.47) as well as with the state of the the most used components address book (average=5.22, usage rate=91%) and calendar (5,09, 94%). Our work in the administration area (5.06, 82%) pays off, and for the first time we get a value of above 5 for this component as well.

The effort we put in on the mid-used component email (4.6, 77%) also pays off. It improved by 0.3 points and for the first first time reached our internal goal for mid-used components of a rating above 4.5.  Untouched tasks (4.83, 66%) stays about were it was.

For the less used components time-tracker (4.25, 44%) and CRM (4.00, 35%) we target above average values. We reached these in the past and as we did not touch these components, the rating was stable compared to last surveys.

So on the first glance this is a result to open a bottle of champagne. For the first time we reached all Usability metrics we have set ourselves as goals. Congratulations to the team – and a big thank you to all our users, who helped us to get there! Without your participation in our surveys and our forum or you buying support to solve urgent features, we couldn’t have gotten here.

But where there is sun, there also is shade. So taking a closer look at the results and focussing only on the end-users, we see that Tine 2.0 obviously is more loved by non-end users, than by end-users. What do I mean by that? If you are primarily using Tine 2.0 and not setting it up for others, you are more likely to like it a little less. The avarage rating of an end-user is about 0.2 lower than that of all users. This is true for all applications except for time-tracking. End-users rate time-tracking slightly better than non-end-users.

We promise to work on this result. Of course we want to achieve our Usability goals for every population of Tine 2.0 users.

Missing features

We also asked for suggestions and missing features. Obviously I do not want to bore you with detailed results of this, because we got a lot of long and detailed answers. So I keep it short: There were many interesting ideas in the results, quite some we could easily fix, some that we cannot even put on the horizon. But by far the most wanted features were a filemanager, and CalDAV / CardDAV support. And hey, what should I say: we listen to you. In the next version of Tine 2.0 (Milan) all of these will be included (and much more). Perhaps you like to try it out and give us some feedback on it?

As a short note on how you can influence the development: Next to stating what you think Tine 2.0 is missing in our surveys, you always have the possibility to code it yourself or to pay someone to code it for you.

Speed of Tine 2.0

We have been doing a lot to improve the speed of Tine 2.0. So we wanted to know how you experience the velocity of Tine 2.0. Of course, your impression is not only influenced by factors we can influence. So the speed of your server, the browser you use and your Internet connections brings in a big source of variance to this question. The happier we were when we saw the results: 78% of you reported that the speed is at least ‘OK’ and 55% reported that speed improved over time. So we can be happy with these results too, seeing that our work in this area has paid off.

Email usage in Tine 2.0

One of the focus point of the last release was to improve emailing. As I said above, we are happy to have reached our Usability goal for emailing with this release, but we wanted to know more. Now we understand, that the access to your emails varies widely. Most of you use a desktop as well as a mobile mailer, some even other web mailers next to Tine 2.0. So Tine 2.0 basically has to integrate with that. Most people only have one email account, but the reported numbers going up to 150 with a median of 3. We are very pleased that you like way we provide filter for emails in Tine 2.0 (4.64, scale see above).

Filtering in Tine 2.0

Last but not least we wanted to know how satisfied you are with the way we are doing filtering in Tine 2.0. As you know the powerful filter concept is one of the core parts of Tine 2.0. So it was more than time to get some feedback on it. And the result was surprising: 51% of you reported that would like to have more filtering options. Only 1 person said that less options are needed. Again, we will take this result and your detailed comments serious and promise to further extend the filtering system.

I gave a talk on ‘Detecting signals in the feedback noise’. You can find my .pdf slides here. I was told there would also be a stream / video of the talk, but I haven’t found it yet.

Unfortunately all the usability and user experience related talks were in a room almost impossible to find. At least I hope this was the reason for so few people attending my talk . I do hope these topics will get more focus and attention in future!

I am really looking forward to meeting all of you next year again!

Users want to contribute

It’s remarkable that a huge number of people (n=5419) answered our questions in about one week, with a really low drop-out (~75% completed the survey).

Usability is a major issue of modern applications. We believe that a user-centric development can help to create usable software, e.g. make easier and more enjoyable to create better results. For a user-centric development users need to willing to join us. The survey is the prove that LibreOffice users are willing to do so – so let’s start to make it possible for our users to efficiently and enjoyably participate in the future development.

Motifs: Describing users beyond Personas

One main purpose of this study was to investigate a more psychological approach to describe the typical LibreOffice users. Currently the most common method for doing so is called Persona. A Persona basically is an aggregation of typical user characteristics described in the form of a fictional user of a product that is used to unify the view of the development team on the user base.

As psychologists we were always dissatisfied about the lack of a scientific, psychological ground to this methodology. Therefore we adopted the psychological concept of ‘motifs’ for our study.

The scientific grounding was developed by Norbert Bischof, known as Zurich Model of Social Motivation, and is based on ethology. It postulates three general systems: security, arousal, and autonomy. The latter again is divided into power, prestige, and achievement. The model describes  a regulatory loop that combines all motifs.

Motifs can be used to explain and predict human behavior, e.g. people with a higher motif on arousal will in the same situation more likely do something that excites them than those who have low arousal value.

In our study we found first hints that using motifs could be well suited for describing typical users in a user-centric design process.

Security

Security is a more basic motif. We picked the question “I need familiar people around me” to access it. Because it is more relevant for children we did not find a significant relation to our dependent variables or the use of LibreOffice.

Arousal

Arousal deals with new situations, search for excitations, adrenalin. We asked the participants about their self-estimation on the statement “Unknown situations are very attractive to me”. The results combined with questions concerning the use of LibreOffice and software in general show that people that have a higher arousal:

• utilize Impress and Calc excessively and
• make use of the possibility of self individualization within the software.

Power

The motive of Power was assessed by accordance with the statement “In a work team I like to take the lead”. The higher people scored on the power motif,

• the higher was the self-estimated relevance of computers,
• the higher was the self-estimated expertise,
• more intensively all LibreOffice tools were utilized,
• the higher was the interest to interact fast and graphical instead of precise and textual.

Prestige

The question “I enjoy it, when other people pay special attention to me” was used to assess the prestige motif. People with high values on this question need social integration (perhaps they would have more friends on social networks too) and avoid to make failures in contrast to such with high values in arousal. In particular high values correlate with:

• preference of assistance and active support,
• preference of graphical interactions,
• praise of innovative designs (like ribbons),
• higher utilization only of Impress.

Achievement

Achievement, assessed as “It is important for me, to deliver good performances”, turned out to be highly relevant for almost all of the responders. Due to this extreme distribution with roof effects we cannot reveal a particular relation to LibreOffice or UI in general. We will have to rephrase this question for future surveys.

Conclusion

We believe in motifs as replacement of or extension to personas. Perhaps not all of the postulated motifs are relevant in context of UI design but if we can create standard users based on this approach the results would be valid and reproducible. In our next studies we will integrate a revised questionnaire for assessing motifs, usage preferences and the use of LibreOffice, so we can re-assure our findings.

Acknowledgments

I want to say a special ‘Thank you!’ to Heiko Tietze and Isabel Menz from Apliki, who have worked hard on the data and the postings. And of course I want to say ‘Thank you!’ to everyone who took part in the survey. Keep up the good spirit!

To understand the preferences our users have when using software, we created nine questions on a six-step semantic differential. The questions are about preferences in menu navigation, workflow, individualization and the use of templates.

We found, that …

• the sample divides up in those who like ribbon style menus and those who don’t
• older people prefer classic menus while younger people go for ribbons
• younger people prefer to be actively supported by the software while older people only want support after asking for it
• older people more often use individual configurations and self-formatting than younger people
• people with a strong prestige motif prefer active support, to be led by assistants and they use graphical dialogs instead of textual ones
• people with a strong arousal motif prefer individual configurations and self-formatting

Part 1: Menu navigation

Structure

The first question was about menu navigation. We wanted to know whether user prefer to have:

• many menu items in fewer levels of a menu or
• less menu items in more levels.

Obviously the more functions a software has, the more menu items are there. Even with a lot of functions, the menus should still be neat and clear.

We did not find a specific pattern in the answers. Although there are some people on either extreme side, we could not find any moderating effect.

Organization aka ribbons

This next question was about preference of a ribbon style toolbar on the one side (low values) versus a classic menu and a set of toolbars on the other side (high values). Microsoft invented Ribbons as a solution to the problem above: which functions should be directly accessible?

This question about the preference turns out to be of very controversy matter. People either love or hate ribbons, which is shown in the histogram.

We also found that there are strong effects between this question and the rating on the motif prestige (graphic on the right). People who rate higher on prestige prefer ribbons (F_(1,4176)=48.61; eta=0.601). This is an interesting result as it wasn’t expected. One possible reason could be that people, who want to be popular and admired by others also try to keep up on latest developments and to be always up to date. Are there more ideas to this?

There is an even stronger effect of the variable age on answering this question.   Older people are more likely to prefer classic menus while younger people prefer ribbons (F_(1,4011)=496.28; eta=0.992). As ribbons are newer inventions, they are easier to understand and handle for people, who never got too used to the classic menus. This result is consistent to the findings of other studies.

To summarize one can say that people either like or dislike ribbons. Younger people and such with high values on prestige are more likely to enjoy ribbon-like controls. On the other hand there are a lot of users that would refuse to use these ‘gimmicks’. We think the direction of interface design taken with the invention of ribbons is good but they still need to better elaborated and understood.

Optimization

The last one of the questions about menu navigation deals with menu control. We asked about the preference to use a mouse on the one side of the semantic differential vs. a keyboard (shortcuts and keystroke combinations) to control menus on the other side. Mouse control is easier and more intuitive. Keystroke combinations are faster but harder to find out and they have to be memorized.

We did expect effects of motifs, age or expertise to take an effect on this preference statement. But again, this question turned out to be not selective for the sample. Obviously, there are people on every step between the options.

Part 2: Workflow

Assistance

The first question dealing with the preferences of handling workflows was about preferring either being led by an assistant or free editing without being led.

It shows, that there is an obvious tendency to free editing and only very few people prefer to be led by an assistant (wizard). This could be a result of the fact that people nowadays are often irritated by assistants (like “Clippy” from Microsoft Office). They rather edit free than be annoyed by a bad assistant.

There is an effect between this question and the motif prestige (graphic on the right). People with stronger motifs for being admired aka with high prestige prefer to be led (F_(1,4099)=35.82; eta=0.488). A possible reason could be that mistakes are more unlikely if you are being led by an assistant. People with greater intentions for prestige would want to avoid mistakes and show good results. Do you have another plausible idea for this result?

Access to functions

The second question was about the quantity of functions in direct access (many toolbars or no toolbars). We could not find a clear result. This does not mean people don’t care about toolbar cluttered interfaces. User interface design should carefully investigate whether or not additional toolbars should be introduced or replaced by other controls. The question probably simply could not be answered seriously by the users.

Interaction

Question three deals with the preference for a configuration via graphical elements (e.g. create a table by dragging rows and columns in a preview) or via textual dialogs (e.g. create a table by entering the number of rows and columns). Graphical dialogs look better and are more intuitive to use by mouse control. Textual dialogs are – once learned – faster by only using the keyboard and offer full functionality.

Although in most software you can configure using textual dialogs, it seems people strongly prefer graphical dialogs (as you can see in the histogram on the left). They are easier to handle than abstract digits.

As shown in the graphs, there is a relation between this question and the motifs power and prestige. People with an average to low power motif mostly prefer textual dialogs (F_(1,4098)=32.84; eta=0.456). What could be a reason for this finding?

People with a higher prestige motif prefer graphical dialogs while people with a lower prestige motif prefer textual dialogs (F_(1,4104)=28.33; eta=0.405). Yet again, this could be a result of the fact that the people with high prestige motif are worried about doing things wrong and therefore go for the easier solution.

A very strong relation can be found with the participant’s age. Older people significantly prefer textual dialogs. The effect is with an effect size of eta=0.742 very relevant. Could this be because older users are more used to textual dialogs? Do you have any other interpretation?

Support

The last question to workflow was about whether one like to get active support through the software or rather only after asking for it. This question is quite close to the question above about wizards and probably reminds people again of annoying pop-ups. 

Again there is a significant effect of age on this preference. Younger people prefer to be actively supported by the software. The older the people are, the more they only want support after asking for it. The effect size of eta=0.762 highlights the relevance of this result. Can you think of a reason for this finding?

As in some other preferences before, again we find a relation between this preference and the motif prestige. People who rate higher on prestige favour active support (F_(1,4108)=44.12; eta=0.567).

Part 3: Individualization

Settings

The next question was about the preferences of individualization when using software. Participants were asked to indicate whether they use the software’s default setting on the one side or configure it individually in accordance to personal preferences on the other side. We asked this question as it not only shows if we should provide more options to individualize: it also tells us something about the real expertise of our participants/users. We would expect more expert users to make more extensive use of individualisation options.

All in all, there is a tendency towards an individual configuration. Surprisingly it shows, that older people seem to be more interested in individual configuration than younger people (F_(1,4027)=85.12; eta=0.802) while we also found that younger people see themselves more as experts. Perhaps younger people overrate their skills or – take it the other way around – older people have excepted the computer to be more complex than one would hope?

Another interesting significant effect is that people with a higher motif for arousal prefer individual configuration (F_(1,4070)=45.12; eta=0.577). Obviously users needs to be a bit adventurous to dare to individualise their software.

Templates

Last but not least we asked, whether the participants prefer to use default formatting or self-created formatting. Again we would expect more advanced users to prefer self-created formatting. So, like in the question above, the answer should tell us something about the expertise of the users. But again, this question does not seem to be easy to answer.

In result it shows that older people prefer self-created formatting (see graph on the left), mostly people between 50 and 60 (F_(1,4020)=74.3; eta=0.761). Did they make bad experience in the past or do the simply have a different need?

Again it shows – like in the question above – that people with high arousal motif use self-created formatting more willingly. The effect size eta=0.476 (F_(1,4060)=34.45) is not overly high, but points at the effect. The reason could be the same as for individual configuration: those people like to be aroused by trying out functions of the software instead of using the same defaults again.

That’s it folks.

These are all results of our survey for now! We will soon give the long awaited management-summary of the findings – hopefully in time for the LibreOffice Conference in Paris.

So we are looking forward to exchange with you – either in person on the conference or in the comments below. Did you like what we did? What should we improve in the next survey? Are there any questions you have and we could try to answer using the data from this survey?

We plan to work on power and network management in KDE and we would be very happy if you would share your experiences with us. We have set-up a small survey that will help us to hopefully do the right things the right way!

The survey in only active until Wednesday, 28th of September. It should take you no more than 5 minutes to fill it out:

CLICK HERE AND PARTICIPATE!
Update: Sorry, I put the wrong link in before – thanks Markus for noting this. The correct link is this one:

http://www.usability-methods.com/survey/fbabc2517f95418ebc84005185abc086/

You can find more information about it in the Tine 2.0 forum (in English or in German).

Writer is the tool which is used most often, followed by Calc. For us surprisingly infrequently used is Impress. The distance to the least used tools Draw, Base and Math is much smaller than expected.

Summing the results up, we found interesting relations between age, expertise and motifs and the frequency individual applications are used:

• Older people use Base and Calc more often than younger people
• Unemployed and students use Calc less frequently than everyone else
• Calc and Impress are more often used by people with high arousal and power motifs
• People with higher ratings on prestige use Impress more frequently

Knowing these facts can help to improve the different applications. As Base for example is more frequently used by older people it should be designed for older people’s needs and preferences.

Read on to get all details.

Demographics

As discussed in the last post, due to the large amount of data a lot of tests reveal statistical significance. Therefore we use the effect size ‘eta’ to identify the relevant results (as a rule of thumb, eta should be greater than 0.5 to spot a highly relevant finding).

Age

Age shows some interesting effects on the use of the LibreOffice applications. Older people seem to use Base about as often as they use Impress. And the older people are the more often they use Base. This effect size is rather high (eta=0.94). The same seems to be true for Calc. Young people use Calc much less often than older ones. This result could be related to a business need for databases and calculating data sheets. Draw and Writer also show relevant effects (see table 1) that cannot easily be explained. Perhaps the different usage pattern of students (see below) takes an impact here.

Table 1: Statistical analysis of LibreOffice usage by age. Results are sorted by effect size, significant effects in red.

Business

Obviously, unemployed people and students use Calc less frequently. This is probably related to the fact, that Calc is often needed in offices. Many people use it for their jobs (especially people who work in economics, as shown in the graph) rather than privately. So people with no job (and students) don’t need it as often. Another less apparent effect is that people from education and health seem to use Impress more often than people with other professions. Especially for the educational work this is comprehensible, as multimedia presentation are used regularly in schools and universities.

For all other applications there are no major differences between the businesses. This might be influenced by limitations in the question. For example a great amount of participants (almost 20%) indicated that they work in IT which was not present as a single selection option. For the next survey we will improve the options for this question.

Motifs

Most significant findings when looking at the motives and the usage of LibreOffice applications is that higher ratings on the dimensions power and arousal are accompanied by a more frequent use of LibreOffice in general, and Impress and Calc in detail.

Most likely this is related to work issues. People with a higher power motif could be more career-minded, get to higher positions and hence need to use Impress and Calc (and LibreOffice in general) more often. On the other hand it is interesting and does not seem very clear to us why people with a higher arousal motif also use LibreOffice more often. Perhaps this is related to the fact that these people are more outgoing and hence need tools to create and present.

Quite interesting is the fact that people with a high score on prestige use Impress more often. Although the effect size is only medium high (eta=0,39), it shows that presenting is important for people who want to gain recognition. This finding is a good evidence for the validity of motifs as we measured them. As a more humorist comment: people with a higher prestige do not seem to validate the presented results in a greater depth – at least they do not do that with LibreOffice applications.

Table 2: Statistical analysis of LibreOffice usage by motifs. Results are sorted by effect size ‘eta’, significant effects in red.

Our next episode will be about the user’s preferences in using software in general and how they are related to the other variables.

The main results show that

• motifs security and prestige decline linear with age,
• asking users for a self-estimation relevance or expertize does not work out,
• motifs can be used for characterization and description of users.

Read on to get the full details.

Motifs

A very common approach in usability is to create fictional characters to represent different user types within a targeted user group. This methodology is called Personas, promoted e.g. by Alan Cooper [1]. Thus a Persona is meant to stand for a set of users that are expected to use the product in a similar way. These Personas are usually described on dimensions like demographic, attitude, goals or behavior.

Unfortunately Personas are not scientifically proved in any way. They seem to work somehow, but are far away from the insights of the science of psychology in general or differential psychology more specifically. Differential Psychology studies the ways in which individual people differ in their behavior. At the Psychological IT-Consultancy Apliki we want to bring the findings of Psychology into the development of IT products, thus we cannot be satisfied with this situation.

We therefore apply well-established psychological approaches to the field of IT. In this study we are targeting a theory called ‘Zürcher Modell’ by Norbert Bischof [2], better known as ‘motifs’:

Motifs are stable personality traits that show the importance of certain goals for the person. To find out more about our users, we asked them to self-estimate themselves on the five basic motifs that are fundamental to every human being:

• Arousal is the motif of experiencing new and exciting situations.
• Prestige is the motif that describes the intention of being popular and admired by other people.
• (Social) Security is the need of being in secure relationships and not left alone.
• Achievement is the motif of being very fierce to deliver good performances.
• Power is the motif of wanting to take the lead in teams.

To put it short: motifs try to explain why people act at all. Motifs promote a constant set-actual comparison. If e.g. a person with a low set-value in social security is in a crowded situation, this person will most likely try to leave this setting. For more specific reading the book from Bischof [2] is recommended.

In this study we are collecting data on motifs to find out how motifs influence our IT preferences. This addresses questions like: Do people with a high motif for achievement have other preferences in individualization of a software than people with a lower motif for achievement?

There is a well-perceived questionnaire with 25 items for the self-assessment on the five motifs. This is obviously too long for a survey, so we decided to use only one question per motif. We chose the one that rated highest on the motif factor and we considered as most reasonable.

Relations between motifs and independent variables

To check whether the motifs – as we raised them in the survey – are actually usable, we first set them in relation to the independent variables introduced in the last post.

First, we dichotomised the variables expertise, relevance and usage to have a sufficient number of answers for each option. As already mentioned in the last posting, e.g. for expertise, there are only few people consider themselves as beginners and average users, so we put them together in a group we call “less expertise” and those, who consider themselves as experts make up the second group “more expertise”. We did the same for relevance and usage, where the problems were similar in order to the given answer options.

In this article the effect size ‘eta’ will be important. Due to the large data set a lot of statistical tests reveal significance. Therefore, primary attention has to be drawn on the effect size  - an indicator for the relevance or strength of an effect. Commonly, an effect is treated as medium strong if eta is above 0.3 and as high if it exceeds 0.5. We will only discuss results with an effect size over 0.5 to make sure the effects we are talking about are relevant.

Age

The graph (click to enlarge) shows that the variable age is related to the motifs prestige and security. The older the participant is, the lower (s)he rates on these two scales. Thus, younger people (in our sample) have higher motifs of prestige and security than older people. Statistical analysis reveals a strong evidence of these findings with effect sizes of eta=0.87 for security and eta=0.55 for prestige. All numerical results are summarized in table 1. The other three motifs – arousal, achievement and power – prove not to dependent on the age of the participant.

Overall, people seem to relax more when they get older.

Expertise

In the graphic you can see a slight difference between expertise on the motifs arousal and power. People with lower expertise seem to have higher motifs for being aroused and having power.This result is contradictory to the expectation. On the one hand people who intend to have a lot of power could see themselves not as much as experts as they do need more experience (or power) to feel powerful over the computer.

Summing it up,  we believe the question does not work out. People find it difficult to estimate their own expertise and, more relevant, no one wants to be a beginner. Therefore questions of this kind should not be used in general and will more specifically be excluded from further inspection of our data.

Relevance

Relevance describes whether people consider working on a computer more or less important. It turns out to take a strong effect on all motifs except for security. Detailed statistical analysis prove the strongest effects for achievement (eta=0.85) and power (eta=0.68). The direction of this effect is the same for all four motifs: People for whom working with computers is more important show lower ratings on the motifs.

Again, this is actually a very debatable finding. Do people need lower motives to allow or at least report computer to be important for their lives? Again, we believe this kind of question does not work out in a user survey and thus the data is excluded from further inspection.

Usage

The time spent on the computer shows a slight effect on achievement, power and arousal. People who spend less time using a computer report higher values on these motifs (comparable to expertise).

These effects are not strong (eta < 0.3). Further investigations on the effect of usage on other constructs also proves this variable to be irrelevant. This shows that the time people spend in front of a computer does take a relevant influence. This result should be obvious: You can spend a lot of time playing, reading e-mails or just surfing the web, without gaining any expertise in how a computer works. This variable will therefore be excluded from further investigations.

Statistical analysis

Statistical analysis were computed as linear regression respectively ANOVA between motifs and independent variables. Based on resulting F-value the effect size ‘eta’ was calculated to estimate the relevance of a finding. The explained variance is eta². For example the relation between security and age reveals 76% of the overall variance. All calculations (and graphics) were conducted using R.

Table 1: Statistical analysis of motifs. Red color shows significant results. Eta denotes the effect size.

Last but not least, it is worth to mention – as you can see in all of the graphs – that our sample in general rates above average on all motifs, but especially on achievement. This does not necessarily mean that all of our participants are extremely ambitious. We rather assume the question we chose to assess achievement has been formulated a little too strong. Because of the ceiling effect we decided to exclude this motif from further investigations. We will modify this question in future surveys, in order to move the results more towards the middle of the scale.

Relations between motifs

In the next step of our analysis, we conducted an analysis of configurational frequencies (CFA). The idea of a CFA is to compare the expected frequency of a certain combination of factors with the actually observed frequency. This way we are able to identify patterns in the data that occur significantly more often (they are called ‘types’) and such that occur less often (they are called ‘antitypes’).

In order to do so, we dichotomized the remaining four motifs by average (marked as + for above and − below average values in the following table) for the calculations. Although motif dimensions are assumed as  independent, the question we are trying to answer in the context of motifs is: Do people’s answers show any patterns? A pattern could be something like: high arousal often goes along with low prestige.

The results, as shown in table 2, reveal combinations that characterize about 6% of the population each in average. The deviation from that average is about ±5%. The data thus proves to be suited for a CFA.

In theory the motifs are considered to be independent from each other. The results however show types of low-raters (−−−−) and high-raters (++++). To explain these findings, we assume some people did not answer the motif questions  honestly, but clicked constantly on the left- or right-hand side. As the motif questions feel very personal, this behavior is not too surprising. The following significant antitypes are a consequence of the two strong types.

Summing it up, we expect some unwanted variance in the motifs, which might superimpose other effects in follow-up examinations. But we are confident to have assessed the motifs in a sane way and to be able to continue to work with them.

Table 2: Analysis of configurational frequencies. Minus denotes values below, plus above median. Red color shows significant results.

In the next post we will reveal our findings about the usage of LibreOffice tools.

References

[1]: Cooper, A. (1999). The Inmates Are Running the Asylum. Sams: Indianapolis.

[2]: Bischof, N. (1975). A systems approach toward the functional connections of attachment and fear. Child Development, 46(4), 801-817.

As promised, this article is about the first results from the latest LibreOffice (LO) survey. To start with, we will give you an impression about the variables we raised and how they are distributed in the sample. Following on, we will tell you more about the general effects found in the survey and how that can be used to improve LibreOffice.

There will be a couple of posts in this blog dealing with this survey, starting today with a description of the independent variables. Next we will present results of motifs and thereafter such of LO usage. It’s remarkable that a huge number of people (n=5419) answered our questions in about one week, with a fairly good responder quote (~75% completed the survey). Many of them are students with a high self-estimated expertise.

Obviously our users are more than willing to bring in their experience and their wishes into the development of LibreOffice. This is an experience I also gained in many other FLOSS projects. As I discussed with Michael Meeks on the Desktop Summit, we will give LibreOffice users this chance to participate in surveys more easily in future. Read on to get an insight into the results. If you want to analyse anything yourself, we are more than happy to give you access to the data.

Aims

The aims of the study were twofold:

• First, we started usability research for LibreOffice by investigating the target group. Decisions about the direction LO is going to take, need to be based upon knowledge about our users, how they use LO and where they need more support.
• To get a more scientific groundwork into the user research work we introduced the so called “motifs” which are well established in psychology to describe people’s goals [1,2]. Research on motifs was the second purpose of this study.

Methodology

There are independent variables and dependent variables in every survey:

• Independent variables are variables that are given and do not change when other parameters differ (e.g. your origin is fixed and does not change with your preference for keyboard combinations or mouse usage).
• Dependent variables are expected to change in relation to these independent variables.

For the present study we collected some demographic data as independent variable that could be used to divide users into groups. E.g. elder users might have a different approach to use software compared to younger. The demographic data we collected was origin, age, business, expertise in terms of computer use in hours per day, self estimated know-how, and valuation of personnel computer relevance.

As a further independent variable we also raised the participant’s basic motifs. They will be discussed in a follow-up article.

Results: Demographics

Today we present the analysis of the independent demographic variables.

Origin

The study was translated in four languages. Most participants answered in English, second and third was Portuguese and German and the smallest group were French speaking participants. The great amount of English answers doesn’t tell us a lot about the participant’s origin though. As we had only four languages, everyone who is neither French nor German or Portuguese would have answered in English. Hopefully for the next surveys, we get a bigger range of translations. I do count on the LO translation community there!

Age

Most of our participants were quite young (between 20-30). Only very little were older than 50 (7%), which is surely a result of the fact, that the survey was advertised only via social media. But maybe, we do have mainly young people amongst our users?

Business

Considering the young age of our participants, many of them are still students (27%), we also have many employees from the service sector (15%) and the IT sector (20%). Smaller groups are working in education (11%) or craft (10%). The other sectors like health, economics and right & law were much smaller represented in our sample (5-1%). Do these sectors prefer closed-source solutions?

Expertise

Expertise in using a computer, experience (duration of computer usage per day) and relevance of using a computer are the three variables that we summarize as “expertise”. Unfortunately, responders did not self-estimate as newbies. The relevance of computer using is only for 1% low (“unimportant” n=8 and “rather unimportant” n=41), but for 99% rather important (n=957) or even important (n=3554). We can see the same effect in self rated knowledge, where only 0,4% rated themselves as beginners (n=18), 23,6% as average user (n=1079), but 76% see themselves as expert in using computers (n=3470). 66% spend more than 6 hours daily on a computer, 25,3% of them even more than 10 hours.

Correlations between independent variables

One would expect no correlation between the independent variables. As you can see in the correlogram most values are low to not existent. Usage, expertise and relevance do correlate moderately due to ceiling effects. We will have to alter these questions for further surveys to avoid these artifacts.

The relatively high correlation between age and business is due to the fact that the majority of responders are students less than 30 years old. A different methods of recruiting people to take part in the next surveys will help to lower this bias.

Next on LibreOffice user research: Descriptives for motifs and the relation to independent variables.

Special thanks to Isabel Menz and Heiko Tietze for fiddling around with the data and R and supporting me with this post.

References:

[1]: Bischof, N. (1975). A systems approach toward the functional connections of attachment and fear. Child Development, 46(4), 801-817.
[2]: Schönbrodt, F.D., Unkelbach, S.R., Spinath, F.M. (2009). Broad Motives in Short Scales – A Questionnaire for the Zurich Model of Social Motivation. European Journal of Psychological Assessment, 25, 141-149.