OpenSource Usability Labs

I haven’t been blogging for a long time. To give you some short reason for that: After almost 6 years I have decided to leave Apliki, the company I once co-founded. This process was a little time-consuming but is finished now.

Why could that be of interest for you?

• I will be much more engaged in the usability of Free Software projects in future.
  Next to the work I have been doing and will continue to do for Tine 2.0, I already teamed up much closer with the KDE and the LibreOffice community. And I am actually still looking for more challenging projects from the Free Software World.
• I will try to re-animate OpenUsability.org.
  I care a lot about this initiative. It tries to accomplish two things: First it helps to provide Usability Competence to Free Software projects. Secondly it tries to make process and work done in usability more transparent, so more people can benefit from it. Sad enough, this project is basically dead at the moment, but I want to work to re-animate it again (And you are happily invited to help re-building the structures we need to fulfill our duty!).
• I will free a software for user integration.
  Over the years at my old company, we have been developing a web based tool that helps to integrate users into the development process of software. I am currently working to release this software under the aGPL (Again, you are happily invited to join and support me in this effort).

First thing I want to say is ‘Thank you!’ to all the people who have been organizing the conference. It was a great experience for me – full of contrasts, inspiration and networking. I was very happy to meet so many of you LibreOffice enthusiasts personally. There have been a lot of detailed descriptions on various topics (e.g. I really liked Christophs post, also check the LibreOffice Planet). I do not want to add much more to these – only one little thing:

I gave a talk on ‘Detecting signals in the feedback noise’. You can find my .pdf slides here. I was told there would also be a stream / video of the talk, but I haven’t found it yet.

Unfortunately all the usability and user experience related talks were in a room almost impossible to find. At least I hope this was the reason for so few people attending my talk . I do hope these topics will get more focus and attention in future!

I am really looking forward to meeting all of you next year again!

This is the last post about our LibreOffice survey and we simply like to sum up the main findings. So if you are interested in more detailed background information, please have a look at the past postings.

Users want to contribute

It’s remarkable that a huge number of people (n=5419) answered our questions in about one week, with a really low drop-out (~75% completed the survey).

Usability is a major issue of modern applications. We believe that a user-centric development can help to create usable software, e.g. make easier and more enjoyable to create better results. For a user-centric development users need to willing to join us. The survey is the prove that LibreOffice users are willing to do so – so let’s start to make it possible for our users to efficiently and enjoyably participate in the future development.

Motifs: Describing users beyond Personas

One main purpose of this study was to investigate a more psychological approach to describe the typical LibreOffice users. Currently the most common method for doing so is called Persona. A Persona basically is an aggregation of typical user characteristics described in the form of a fictional user of a product that is used to unify the view of the development team on the user base.

As psychologists we were always dissatisfied about the lack of a scientific, psychological ground to this methodology. Therefore we adopted the psychological concept of ‘motifs’ for our study.

The scientific grounding was developed by Norbert Bischof, known as Zurich Model of Social Motivation, and is based on ethology. It postulates three general systems: security, arousal, and autonomy. The latter again is divided into power, prestige, and achievement. The model describes  a regulatory loop that combines all motifs.

Motifs can be used to explain and predict human behavior, e.g. people with a higher motif on arousal will in the same situation more likely do something that excites them than those who have low arousal value.

In our study we found first hints that using motifs could be well suited for describing typical users in a user-centric design process.

Security

Security is a more basic motif. We picked the question “I need familiar people around me” to access it. Because it is more relevant for children we did not find a significant relation to our dependent variables or the use of LibreOffice.

Arousal

Arousal deals with new situations, search for excitations, adrenalin. We asked the participants about their self-estimation on the statement “Unknown situations are very attractive to me”. The results combined with questions concerning the use of LibreOffice and software in general show that people that have a higher arousal:

• utilize Impress and Calc excessively and
• make use of the possibility of self individualization within the software.

Power

The motive of Power was assessed by accordance with the statement “In a work team I like to take the lead”. The higher people scored on the power motif,

• the higher was the self-estimated relevance of computers,
• the higher was the self-estimated expertise,
• more intensively all LibreOffice tools were utilized,
• the higher was the interest to interact fast and graphical instead of precise and textual.

Prestige

The question “I enjoy it, when other people pay special attention to me” was used to assess the prestige motif. People with high values on this question need social integration (perhaps they would have more friends on social networks too) and avoid to make failures in contrast to such with high values in arousal. In particular high values correlate with:

• preference of assistance and active support,
• preference of graphical interactions,
• praise of innovative designs (like ribbons),
• higher utilization only of Impress.

Achievement

Achievement, assessed as “It is important for me, to deliver good performances”, turned out to be highly relevant for almost all of the responders. Due to this extreme distribution with roof effects we cannot reveal a particular relation to LibreOffice or UI in general. We will have to rephrase this question for future surveys.

Conclusion

We believe in motifs as replacement of or extension to personas. Perhaps not all of the postulated motifs are relevant in context of UI design but if we can create standard users based on this approach the results would be valid and reproducible. In our next studies we will integrate a revised questionnaire for assessing motifs, usage preferences and the use of LibreOffice, so we can re-assure our findings.

Acknowledgments

I want to say a special ‘Thank you!’ to Heiko Tietze and Isabel Menz from Apliki, who have worked hard on the data and the postings. And of course I want to say ‘Thank you!’ to everyone who took part in the survey. Keep up the good spirit!

This post is about software preferences. Yes, we still haven’t written about all results from our LibreOffice user research – but I promise, this will be the last of the long posts about the results.

To understand the preferences our users have when using software, we created nine questions on a six-step semantic differential. The questions are about preferences in menu navigation, workflow, individualization and the use of templates.

We found, that …

• the sample divides up in those who like ribbon style menus and those who don’t
• older people prefer classic menus while younger people go for ribbons
• younger people prefer to be actively supported by the software while older people only want support after asking for it
• older people more often use individual configurations and self-formatting than younger people
• people with a strong prestige motif prefer active support, to be led by assistants and they use graphical dialogs instead of textual ones
• people with a strong arousal motif prefer individual configurations and self-formatting

Part 1: Menu navigation

Structure

The first question was about menu navigation. We wanted to know whether user prefer to have:

• many menu items in fewer levels of a menu or
• less menu items in more levels.

Obviously the more functions a software has, the more menu items are there. Even with a lot of functions, the menus should still be neat and clear.

We did not find a specific pattern in the answers. Although there are some people on either extreme side, we could not find any moderating effect.

Organization aka ribbons

This next question was about preference of a ribbon style toolbar on the one side (low values) versus a classic menu and a set of toolbars on the other side (high values). Microsoft invented Ribbons as a solution to the problem above: which functions should be directly accessible?

This question about the preference turns out to be of very controversy matter. People either love or hate ribbons, which is shown in the histogram.

We also found that there are strong effects between this question and the rating on the motif prestige (graphic on the right). People who rate higher on prestige prefer ribbons (F_(1,4176)=48.61; eta=0.601). This is an interesting result as it wasn’t expected. One possible reason could be that people, who want to be popular and admired by others also try to keep up on latest developments and to be always up to date. Are there more ideas to this?

There is an even stronger effect of the variable age on answering this question.   Older people are more likely to prefer classic menus while younger people prefer ribbons (F_(1,4011)=496.28; eta=0.992). As ribbons are newer inventions, they are easier to understand and handle for people, who never got too used to the classic menus. This result is consistent to the findings of other studies.

To summarize one can say that people either like or dislike ribbons. Younger people and such with high values on prestige are more likely to enjoy ribbon-like controls. On the other hand there are a lot of users that would refuse to use these ‘gimmicks’. We think the direction of interface design taken with the invention of ribbons is good but they still need to better elaborated and understood.

Optimization

The last one of the questions about menu navigation deals with menu control. We asked about the preference to use a mouse on the one side of the semantic differential vs. a keyboard (shortcuts and keystroke combinations) to control menus on the other side. Mouse control is easier and more intuitive. Keystroke combinations are faster but harder to find out and they have to be memorized.

We did expect effects of motifs, age or expertise to take an effect on this preference statement. But again, this question turned out to be not selective for the sample. Obviously, there are people on every step between the options.

Part 2: Workflow

Assistance

The first question dealing with the preferences of handling workflows was about preferring either being led by an assistant or free editing without being led.

It shows, that there is an obvious tendency to free editing and only very few people prefer to be led by an assistant (wizard). This could be a result of the fact that people nowadays are often irritated by assistants (like “Clippy” from Microsoft Office). They rather edit free than be annoyed by a bad assistant.

There is an effect between this question and the motif prestige (graphic on the right). People with stronger motifs for being admired aka with high prestige prefer to be led (F_(1,4099)=35.82; eta=0.488). A possible reason could be that mistakes are more unlikely if you are being led by an assistant. People with greater intentions for prestige would want to avoid mistakes and show good results. Do you have another plausible idea for this result?

Access to functions

The second question was about the quantity of functions in direct access (many toolbars or no toolbars). We could not find a clear result. This does not mean people don’t care about toolbar cluttered interfaces. User interface design should carefully investigate whether or not additional toolbars should be introduced or replaced by other controls. The question probably simply could not be answered seriously by the users.

Interaction

Question three deals with the preference for a configuration via graphical elements (e.g. create a table by dragging rows and columns in a preview) or via textual dialogs (e.g. create a table by entering the number of rows and columns). Graphical dialogs look better and are more intuitive to use by mouse control. Textual dialogs are – once learned – faster by only using the keyboard and offer full functionality.

Although in most software you can configure using textual dialogs, it seems people strongly prefer graphical dialogs (as you can see in the histogram on the left). They are easier to handle than abstract digits.

As shown in the graphs, there is a relation between this question and the motifs power and prestige. People with an average to low power motif mostly prefer textual dialogs (F_(1,4098)=32.84; eta=0.456). What could be a reason for this finding?

People with a higher prestige motif prefer graphical dialogs while people with a lower prestige motif prefer textual dialogs (F_(1,4104)=28.33; eta=0.405). Yet again, this could be a result of the fact that the people with high prestige motif are worried about doing things wrong and therefore go for the easier solution.

A very strong relation can be found with the participant’s age. Older people significantly prefer textual dialogs. The effect is with an effect size of eta=0.742 very relevant. Could this be because older users are more used to textual dialogs? Do you have any other interpretation?

Support

The last question to workflow was about whether one like to get active support through the software or rather only after asking for it. This question is quite close to the question above about wizards and probably reminds people again of annoying pop-ups. 

Again there is a significant effect of age on this preference. Younger people prefer to be actively supported by the software. The older the people are, the more they only want support after asking for it. The effect size of eta=0.762 highlights the relevance of this result. Can you think of a reason for this finding?

As in some other preferences before, again we find a relation between this preference and the motif prestige. People who rate higher on prestige favour active support (F_(1,4108)=44.12; eta=0.567).

Part 3: Individualization

Settings

The next question was about the preferences of individualization when using software. Participants were asked to indicate whether they use the software’s default setting on the one side or configure it individually in accordance to personal preferences on the other side. We asked this question as it not only shows if we should provide more options to individualize: it also tells us something about the real expertise of our participants/users. We would expect more expert users to make more extensive use of individualisation options.

All in all, there is a tendency towards an individual configuration. Surprisingly it shows, that older people seem to be more interested in individual configuration than younger people (F_(1,4027)=85.12; eta=0.802) while we also found that younger people see themselves more as experts. Perhaps younger people overrate their skills or – take it the other way around – older people have excepted the computer to be more complex than one would hope?

Another interesting significant effect is that people with a higher motif for arousal prefer individual configuration (F_(1,4070)=45.12; eta=0.577). Obviously users needs to be a bit adventurous to dare to individualise their software.

Templates

Last but not least we asked, whether the participants prefer to use default formatting or self-created formatting. Again we would expect more advanced users to prefer self-created formatting. So, like in the question above, the answer should tell us something about the expertise of the users. But again, this question does not seem to be easy to answer.

In result it shows that older people prefer self-created formatting (see graph on the left), mostly people between 50 and 60 (F_(1,4020)=74.3; eta=0.761). Did they make bad experience in the past or do the simply have a different need?

Again it shows – like in the question above – that people with high arousal motif use self-created formatting more willingly. The effect size eta=0.476 (F_(1,4060)=34.45) is not overly high, but points at the effect. The reason could be the same as for individual configuration: those people like to be aroused by trying out functions of the software instead of using the same defaults again.

That’s it folks.

These are all results of our survey for now! We will soon give the long awaited management-summary of the findings – hopefully in time for the LibreOffice Conference in Paris.

So we are looking forward to exchange with you – either in person on the conference or in the comments below. Did you like what we did? What should we improve in the next survey? Are there any questions you have and we could try to answer using the data from this survey?

Today we discuss the results of our survey concerning the usage of the six LibreOffice applications. We also show how usage is related to other variables we talked about in the last posts (see volume 1 and volume 2).

Writer is the tool which is used most often, followed by Calc. For us surprisingly infrequently used is Impress. The distance to the least used tools Draw, Base and Math is much smaller than expected.

Summing the results up, we found interesting relations between age, expertise and motifs and the frequency individual applications are used:

• Older people use Base and Calc more often than younger people
• Unemployed and students use Calc less frequently than everyone else
• Calc and Impress are more often used by people with high arousal and power motifs
• People with higher ratings on prestige use Impress more frequently

Knowing these facts can help to improve the different applications. As Base for example is more frequently used by older people it should be designed for older people’s needs and preferences.

Read on to get all details.

Demographics

As discussed in the last post, due to the large amount of data a lot of tests reveal statistical significance. Therefore we use the effect size ‘eta’ to identify the relevant results (as a rule of thumb, eta should be greater than 0.5 to spot a highly relevant finding).

Age

Age shows some interesting effects on the use of the LibreOffice applications. Older people seem to use Base about as often as they use Impress. And the older people are the more often they use Base. This effect size is rather high (eta=0.94). The same seems to be true for Calc. Young people use Calc much less often than older ones. This result could be related to a business need for databases and calculating data sheets. Draw and Writer also show relevant effects (see table 1) that cannot easily be explained. Perhaps the different usage pattern of students (see below) takes an impact here.

Table 1: Statistical analysis of LibreOffice usage by age. Results are sorted by effect size, significant effects in red.

Business

Obviously, unemployed people and students use Calc less frequently. This is probably related to the fact, that Calc is often needed in offices. Many people use it for their jobs (especially people who work in economics, as shown in the graph) rather than privately. So people with no job (and students) don’t need it as often. Another less apparent effect is that people from education and health seem to use Impress more often than people with other professions. Especially for the educational work this is comprehensible, as multimedia presentation are used regularly in schools and universities.

For all other applications there are no major differences between the businesses. This might be influenced by limitations in the question. For example a great amount of participants (almost 20%) indicated that they work in IT which was not present as a single selection option. For the next survey we will improve the options for this question.

Motifs

Most significant findings when looking at the motives and the usage of LibreOffice applications is that higher ratings on the dimensions power and arousal are accompanied by a more frequent use of LibreOffice in general, and Impress and Calc in detail.

Most likely this is related to work issues. People with a higher power motif could be more career-minded, get to higher positions and hence need to use Impress and Calc (and LibreOffice in general) more often. On the other hand it is interesting and does not seem very clear to us why people with a higher arousal motif also use LibreOffice more often. Perhaps this is related to the fact that these people are more outgoing and hence need tools to create and present.

Quite interesting is the fact that people with a high score on prestige use Impress more often. Although the effect size is only medium high (eta=0,39), it shows that presenting is important for people who want to gain recognition. This finding is a good evidence for the validity of motifs as we measured them. As a more humorist comment: people with a higher prestige do not seem to validate the presented results in a greater depth – at least they do not do that with LibreOffice applications.

Table 2: Statistical analysis of LibreOffice usage by motifs. Results are sorted by effect size ‘eta’, significant effects in red.

Our next episode will be about the user’s preferences in using software in general and how they are related to the other variables.

Today you can learn more about the psychological construct of motifs. We included them as variables in the LO survey similar to the independent variables introduced in our last post.

The main results show that

• motifs security and prestige decline linear with age,
• asking users for a self-estimation relevance or expertize does not work out,
• motifs can be used for characterization and description of users.

Read on to get the full details.

Motifs

A very common approach in usability is to create fictional characters to represent different user types within a targeted user group. This methodology is called Personas, promoted e.g. by Alan Cooper [1]. Thus a Persona is meant to stand for a set of users that are expected to use the product in a similar way. These Personas are usually described on dimensions like demographic, attitude, goals or behavior.

Unfortunately Personas are not scientifically proved in any way. They seem to work somehow, but are far away from the insights of the science of psychology in general or differential psychology more specifically. Differential Psychology studies the ways in which individual people differ in their behavior. At the Psychological IT-Consultancy Apliki we want to bring the findings of Psychology into the development of IT products, thus we cannot be satisfied with this situation.

We therefore apply well-established psychological approaches to the field of IT. In this study we are targeting a theory called ‘Zürcher Modell’ by Norbert Bischof [2], better known as ‘motifs’:

Motifs are stable personality traits that show the importance of certain goals for the person. To find out more about our users, we asked them to self-estimate themselves on the five basic motifs that are fundamental to every human being:

• Arousal is the motif of experiencing new and exciting situations.
• Prestige is the motif that describes the intention of being popular and admired by other people.
• (Social) Security is the need of being in secure relationships and not left alone.
• Achievement is the motif of being very fierce to deliver good performances.
• Power is the motif of wanting to take the lead in teams.

To put it short: motifs try to explain why people act at all. Motifs promote a constant set-actual comparison. If e.g. a person with a low set-value in social security is in a crowded situation, this person will most likely try to leave this setting. For more specific reading the book from Bischof [2] is recommended.

In this study we are collecting data on motifs to find out how motifs influence our IT preferences. This addresses questions like: Do people with a high motif for achievement have other preferences in individualization of a software than people with a lower motif for achievement?

There is a well-perceived questionnaire with 25 items for the self-assessment on the five motifs. This is obviously too long for a survey, so we decided to use only one question per motif. We chose the one that rated highest on the motif factor and we considered as most reasonable.

Relations between motifs and independent variables

To check whether the motifs – as we raised them in the survey – are actually usable, we first set them in relation to the independent variables introduced in the last post.

First, we dichotomised the variables expertise, relevance and usage to have a sufficient number of answers for each option. As already mentioned in the last posting, e.g. for expertise, there are only few people consider themselves as beginners and average users, so we put them together in a group we call “less expertise” and those, who consider themselves as experts make up the second group “more expertise”. We did the same for relevance and usage, where the problems were similar in order to the given answer options.

In this article the effect size ‘eta’ will be important. Due to the large data set a lot of statistical tests reveal significance. Therefore, primary attention has to be drawn on the effect size  - an indicator for the relevance or strength of an effect. Commonly, an effect is treated as medium strong if eta is above 0.3 and as high if it exceeds 0.5. We will only discuss results with an effect size over 0.5 to make sure the effects we are talking about are relevant.

Age

The graph (click to enlarge) shows that the variable age is related to the motifs prestige and security. The older the participant is, the lower (s)he rates on these two scales. Thus, younger people (in our sample) have higher motifs of prestige and security than older people. Statistical analysis reveals a strong evidence of these findings with effect sizes of eta=0.87 for security and eta=0.55 for prestige. All numerical results are summarized in table 1. The other three motifs – arousal, achievement and power – prove not to dependent on the age of the participant.

Overall, people seem to relax more when they get older.

Expertise

In the graphic you can see a slight difference between expertise on the motifs arousal and power. People with lower expertise seem to have higher motifs for being aroused and having power.This result is contradictory to the expectation. On the one hand people who intend to have a lot of power could see themselves not as much as experts as they do need more experience (or power) to feel powerful over the computer.

Summing it up,  we believe the question does not work out. People find it difficult to estimate their own expertise and, more relevant, no one wants to be a beginner. Therefore questions of this kind should not be used in general and will more specifically be excluded from further inspection of our data.

Relevance

Relevance describes whether people consider working on a computer more or less important. It turns out to take a strong effect on all motifs except for security. Detailed statistical analysis prove the strongest effects for achievement (eta=0.85) and power (eta=0.68). The direction of this effect is the same for all four motifs: People for whom working with computers is more important show lower ratings on the motifs.

Again, this is actually a very debatable finding. Do people need lower motives to allow or at least report computer to be important for their lives? Again, we believe this kind of question does not work out in a user survey and thus the data is excluded from further inspection.

Usage

The time spent on the computer shows a slight effect on achievement, power and arousal. People who spend less time using a computer report higher values on these motifs (comparable to expertise).

These effects are not strong (eta < 0.3). Further investigations on the effect of usage on other constructs also proves this variable to be irrelevant. This shows that the time people spend in front of a computer does take a relevant influence. This result should be obvious: You can spend a lot of time playing, reading e-mails or just surfing the web, without gaining any expertise in how a computer works. This variable will therefore be excluded from further investigations.

Statistical analysis

Statistical analysis were computed as linear regression respectively ANOVA between motifs and independent variables. Based on resulting F-value the effect size ‘eta’ was calculated to estimate the relevance of a finding. The explained variance is eta². For example the relation between security and age reveals 76% of the overall variance. All calculations (and graphics) were conducted using R.

Table 1: Statistical analysis of motifs. Red color shows significant results. Eta denotes the effect size.

Last but not least, it is worth to mention – as you can see in all of the graphs – that our sample in general rates above average on all motifs, but especially on achievement. This does not necessarily mean that all of our participants are extremely ambitious. We rather assume the question we chose to assess achievement has been formulated a little too strong. Because of the ceiling effect we decided to exclude this motif from further investigations. We will modify this question in future surveys, in order to move the results more towards the middle of the scale.

Relations between motifs

In the next step of our analysis, we conducted an analysis of configurational frequencies (CFA). The idea of a CFA is to compare the expected frequency of a certain combination of factors with the actually observed frequency. This way we are able to identify patterns in the data that occur significantly more often (they are called ‘types’) and such that occur less often (they are called ‘antitypes’).

In order to do so, we dichotomized the remaining four motifs by average (marked as + for above and − below average values in the following table) for the calculations. Although motif dimensions are assumed as  independent, the question we are trying to answer in the context of motifs is: Do people’s answers show any patterns? A pattern could be something like: high arousal often goes along with low prestige.

The results, as shown in table 2, reveal combinations that characterize about 6% of the population each in average. The deviation from that average is about ±5%. The data thus proves to be suited for a CFA.

In theory the motifs are considered to be independent from each other. The results however show types of low-raters (−−−−) and high-raters (++++). To explain these findings, we assume some people did not answer the motif questions  honestly, but clicked constantly on the left- or right-hand side. As the motif questions feel very personal, this behavior is not too surprising. The following significant antitypes are a consequence of the two strong types.

Summing it up, we expect some unwanted variance in the motifs, which might superimpose other effects in follow-up examinations. But we are confident to have assessed the motifs in a sane way and to be able to continue to work with them.

Table 2: Analysis of configurational frequencies. Minus denotes values below, plus above median. Red color shows significant results.

In the next post we will reveal our findings about the usage of LibreOffice tools.

References

[1]: Cooper, A. (1999). The Inmates Are Running the Asylum. Sams: Indianapolis.

[2]: Bischof, N. (1975). A systems approach toward the functional connections of attachment and fear. Child Development, 46(4), 801-817.

Summary

As promised, this article is about the first results from the latest LibreOffice (LO) survey. To start with, we will give you an impression about the variables we raised and how they are distributed in the sample. Following on, we will tell you more about the general effects found in the survey and how that can be used to improve LibreOffice.

There will be a couple of posts in this blog dealing with this survey, starting today with a description of the independent variables. Next we will present results of motifs and thereafter such of LO usage. It’s remarkable that a huge number of people (n=5419) answered our questions in about one week, with a fairly good responder quote (~75% completed the survey). Many of them are students with a high self-estimated expertise.

Obviously our users are more than willing to bring in their experience and their wishes into the development of LibreOffice. This is an experience I also gained in many other FLOSS projects. As I discussed with Michael Meeks on the Desktop Summit, we will give LibreOffice users this chance to participate in surveys more easily in future. Read on to get an insight into the results. If you want to analyse anything yourself, we are more than happy to give you access to the data.

Aims

The aims of the study were twofold:

• First, we started usability research for LibreOffice by investigating the target group. Decisions about the direction LO is going to take, need to be based upon knowledge about our users, how they use LO and where they need more support.
• To get a more scientific groundwork into the user research work we introduced the so called “motifs” which are well established in psychology to describe people’s goals [1,2]. Research on motifs was the second purpose of this study.

Methodology

There are independent variables and dependent variables in every survey:

• Independent variables are variables that are given and do not change when other parameters differ (e.g. your origin is fixed and does not change with your preference for keyboard combinations or mouse usage).
• Dependent variables are expected to change in relation to these independent variables.

For the present study we collected some demographic data as independent variable that could be used to divide users into groups. E.g. elder users might have a different approach to use software compared to younger. The demographic data we collected was origin, age, business, expertise in terms of computer use in hours per day, self estimated know-how, and valuation of personnel computer relevance.

As a further independent variable we also raised the participant’s basic motifs. They will be discussed in a follow-up article.

Results: Demographics

Today we present the analysis of the independent demographic variables.

Origin

The study was translated in four languages. Most participants answered in English, second and third was Portuguese and German and the smallest group were French speaking participants. The great amount of English answers doesn’t tell us a lot about the participant’s origin though. As we had only four languages, everyone who is neither French nor German or Portuguese would have answered in English. Hopefully for the next surveys, we get a bigger range of translations. I do count on the LO translation community there!

Age

Most of our participants were quite young (between 20-30). Only very little were older than 50 (7%), which is surely a result of the fact, that the survey was advertised only via social media. But maybe, we do have mainly young people amongst our users?

Business

Considering the young age of our participants, many of them are still students (27%), we also have many employees from the service sector (15%) and the IT sector (20%). Smaller groups are working in education (11%) or craft (10%). The other sectors like health, economics and right & law were much smaller represented in our sample (5-1%). Do these sectors prefer closed-source solutions?

Expertise

Expertise in using a computer, experience (duration of computer usage per day) and relevance of using a computer are the three variables that we summarize as “expertise”. Unfortunately, responders did not self-estimate as newbies. The relevance of computer using is only for 1% low (“unimportant” n=8 and “rather unimportant” n=41), but for 99% rather important (n=957) or even important (n=3554). We can see the same effect in self rated knowledge, where only 0,4% rated themselves as beginners (n=18), 23,6% as average user (n=1079), but 76% see themselves as expert in using computers (n=3470). 66% spend more than 6 hours daily on a computer, 25,3% of them even more than 10 hours.

Correlations between independent variables

One would expect no correlation between the independent variables. As you can see in the correlogram most values are low to not existent. Usage, expertise and relevance do correlate moderately due to ceiling effects. We will have to alter these questions for further surveys to avoid these artifacts.

The relatively high correlation between age and business is due to the fact that the majority of responders are students less than 30 years old. A different methods of recruiting people to take part in the next surveys will help to lower this bias.

Next on LibreOffice user research: Descriptives for motifs and the relation to independent variables.

Special thanks to Isabel Menz and Heiko Tietze for fiddling around with the data and R and supporting me with this post.

References:

[1]: Bischof, N. (1975). A systems approach toward the functional connections of attachment and fear. Child Development, 46(4), 801-817.
[2]: Schönbrodt, F.D., Unkelbach, S.R., Spinath, F.M. (2009). Broad Motives in Short Scales – A Questionnaire for the Zurich Model of Social Motivation. European Journal of Psychological Assessment, 25, 141-149.

As Akademy / Desktop Summit was in Berlin this year, I took the chance to participate. I probably wouldn’t if it had been somewhere else – and I have to admit: It was a great decision to take part (and a big mistake not to do so in the years before). A big praise to the organisers!

Everyone with a slight interest in KDE or Gnome should join this conference. I will hopefully be able to attend next akademies again. And hopefully there will be more people with a similar background to mine, doing user resaerch, usability or UX work. We do need you!

Personally I have been talking to dozens of people about better ways of integrating users into the development. My thesis was basically:

The way users are integrated into the projects at the moment is frustrating and (trying to find a nice word) ineffective for both, users and developers. It thus does not bring in the desired amount of new blood nor innovation into the projects.

Interestingly enough, I found no-one who really disagreed with this. On this basis I did talk to developers of a lot of different projects, but also to representatives of the KDE e.V. and user representatives about the problem and how we could use the usability and user-driven-innovation tool we are currently developing to improve the situation. I also managed to do a BoF about it, to get in touch with some people from Gnome and I talked to Michael Meeks from LibreOffice about it.

As a result of these talks, we will start creating user panels for individual projects, by simply popping up a layer that will be asking users to join the panel after some time of usage. We will use this panel to ask usability and demographic questions, but also to motivate users to get – step by step – involved deeper into the projects. A lot work to be done…

As we are entering virgin soil here, we will obviously (need to) gain experiences. If you want to join this ride with your project or just your experience or opinion, you are happily invited to mail me or comment below.

The first user research survey for LibreOffice has just been closed. About 5400 participants exceeded our hopes and expectations by far.

A big THANK YOU to everyone who participated!

This success shows us that the LibreOffice users are more than interested in giving feedback. They want to take part in the further improvement of LibreOffice. We can, should and will take this seriously! A series of follow-up surveys is planned and will be coordinated with the development.

At a first glance it shows that the most used applications (on an activity scale from 1=never to 5=daily) are LibreOffice Writer (3,92), Calc (3,22) and Impress (2,41). A lot of the participants (76%) see themselves as experts in dealing with computers and spend a lot of time at the computer during the day. Thus, using the computer is very important for most of them (78%).

We will analyse the results in greater detail now and keep you informed about the findings. A big part of the survey dealt with the preferences in using software. It looks like we found quite some interesting results. For example, a question about ribbon interfaces polarizes the users: while 23% of the users strongly prefer ribbon interfaces, 20% reject them (extreme values on a semantic differential). A fact we will evaluate further, so stay tuned!

If you want to join us in the user research team, simply subscribe to the LibreOffice Design Mailinglist. We can give you the opportunity to look at the results personally on Usability-Methods.com, where we conducted the survey.

Read more about the results soon. Thanks again to everyone for your support!

The relationship of developers and designers is full of misunderstandings. Projects fail, e. g. due to the struggle among different views on the same subject. Communication can be frustrating for both sides. But there is hope. In this article I want to share and discuss factors that facilitate a ‘Designer with Developer’ rather than a ‘Designer vs. Developer’ in FLOSS projects.

Background

You need to know that I am a psychologist. I see myself rather as a designer than a developer, because I do not actually write code. But I also do not do graphic-design. My approach to design is not to create pretty pictures – it is to understand people and design solutions for their needs.

My company Apliki is intentionally called “Psychological IT Consultancy”. In our work we build the necessary foundations for projects and bridge the gap among the different worlds in a project (developer and designer are not the only relevant groups in a project – e. g. understanding the user is another central part of our work). In my non-commercial FLOSS work for OpenUsability.org I do the same. After more than ten years in the business I have worked with very different people and projects.

Recently there have been some discussions on the LibreOffice Design mailing list about the rights and duties of developers and designers. I actually do not want to contribute anything specific to this discussion. It is already valuable and fruitful but sometimes I need external impulses to do something I wanted to do for a long time: share my experiences about factors that facilitate good cooperation in projects especially from the design perspective and hence lead to successful and satisfying projects.

So here we go:

1. Respect

Successful projects are based on respect between developers and designers. They show respect by taking time to talk to one another and explain their ideas.

Often designers will say something like: “This solution has the better usability”. Developers tend to answer: “But it is too expensive to code”. This kind of conversation is the best starting point to build up an oppositional feeling in a project  – and I have seen many good ideas failing this way.

My grandfather sometimes says: “You didn’t experience war so you cannot understand it”. This always upsets me. I do understand a lot of things, but my dialogue partner needs to take the time to explain them to me. Taking time shows respect and helps establishing mutual trust.

2. Reasons

In a successful project developers and designers explain why something is better or more expensive, and take the time needed to address the objections of one another.

Giving a good reason is not easy. Neither hierarchical position (or the FLOSS equivalent ‘length of contribution to the project’) nor a vote (“5 out of 8 think this is good”) are good reasons. In Non-Free projects they are more likely to be accepted, but in FLOSS projects most people invest free time.  There is a psychological concept called cognitive dissonance. It helps to understand why people volunteering in FLOSS projects need not only good, but excellent reasons to continue with their involvement (Because they have to self-justify their involvement all the time).

The worst, but still often heard reason is, of course, extortion: “If you do not accept this, I will stop my contribution”. The fact that it is still heard quite often, shows how tiring it is to discuss the relevant issues. Especially in FLOSS projects, where non-native speaker discuss mainly via mail, chat or the like.

3. Foundations

In successful projects design is understood as an engineering discipline, with a common and clearly articulated set of values for all relevant parameters. This sets the frame for the optimization of interfaces.

To be able to address objections, a team needs a common understanding. In my experience, developers are much more sophisticated in this matter than designers. Programmers tend to make fundamental decisions very early. They agree on technical frameworks, architecture of the software, code repositories and so on. This allows them to provide good reasons why, for example, something is expensive to do. This usually leads to the situation that different developers will provide similar answers for a question (as long as this question is technical).

The situation in design could not be any more different from that. Discussions occur about even the smallest issues. This is partially due to everyone being an expert when it comes to using things (see Parkinson’s Law of Triviality for some interesting thoughts on that). But the main reason is that the common ground is missing.

Interface design is often misunderstood as an art. While it definitely has artistic aspects, it mainly stays an engineering discipline. Unfortunately most people working on interfaces think they are doing arts. The problem is immediately evident: Art may be interpreted, but never needs to be justified. And this is where the arts approach to interfaces almost surely fails. If various people work on the same interface, they will need to justify what they do, convince others why their ideas are so great and so on. The more people, with a variety of backgrounds, are involved into this process the more problems will occur.

The only solution is to focus on the engineering aspects of interface design. Engineering is the opposite of art. All it does is:

1. Find out all the parameters that are relevant,
2. try to find reasonable values for them, and
3. iteratively optimize the solution within the given parameters.

This looks like a very boring approach for most people working in design, and it is hence very hard to establish. In the end it is the only way to lay the foundations on top of which problems can be solved. Additionally it is very close to the developer’s thinking and eases communication among different parties.

4. Trust

In successful projects an aura of trust allows new ideas and new people to grow to the benefit of the project.

Once the foundation for the design work is laid, the team can start to build up trust. Development teams rely on code reviews. Good development teams only check whether the code complies with the common foundations. Even though the more experienced developer will sometimes know more elegant ways of solving a problem, generally any improvement will be accepted. The team shows the trust that  all members will learn and accept that they are not perfect yet. They accept that the product is always “in process”.

Again, design teams tend to miss this generosity. As a result of missing explicit foundations, details are discussed in great depth involving a large group of people. Also these discussions often try to find the ultimate, all-time-end-of-discussion solution for a problem (that has not been described in full detail). These discussions can be frustrating, especially for willing new design contributors.

Since usually some sort of interface already exists, an implicit agreement on the foundation exists as well. New contributors therefore either do marginal work or will most likely fail with anything more ambitious. Both ways do not facilitate to build up trust in the new members. The old ones feel responsible for everything. New ideas and fresh personnel has a hard way to get into the project.

5. Communication

In successful projects communication is well structured, facilitates work and does not prevent it.

A potential lack of trust inevitably leads to extended communication. In FLOSS and therefore often voluntary projects people do not find the time to work anymore, because they are stuck in administrative issues (following the different discussions). This is frustrating and will lead to people leaving the project (see concept of cognitive dissonance above).

Also developers are not willing to participate in the debate, or even look at the discussions, because they indeed have better things to do. The gap between design and development increases.

Additionally we should not forget that developers and designers speak different languages. Developers talk about technical issues sometimes with the same words, designer use for describing interactions. Add the fact that people contributing often do not use their mother tongue and the communication chaos is perfect. Astonishing for a psychologist to say, but talking about it is not always the best solution.

6. Facts

In successful FLOSS projects all contributions are welcome (they may be rejected though) and discussion always takes place on the current objective, not on what single people did.

To reduce communication it will be necessary to accept facts. FLOSS development evolves gradually and not always in the shortest linear way. Sometimes things even get worse. This is not really a problem, as such things will get corrected after a while. These faulty developments can also help to clarify the foundations or to show how conclusions can be drawn on them.

It should not be subject of any discussion when someone has done something. Remember, people are spending their spare time. If they have to justify that they have done something they are likely to leave the project (did I mention cognitive dissonance before?). And what is worse? A wrong turn in the project or the loss of a motivated contributor?

7. Structure

In successful projects structure facilitates the cooperation of different disciplines.

The acceptance of facts is a non-commercial FLOSS-only approach. In every commercial project the opposite is true, because in commercial project structures are more natural. FLOSS projects tend to refuse structure with reference to the bazaar. But even on a bazaar not everyone trades everything and there are rules and structure.

Structure prevents turning facts into the wrong direction in the first place. Design has to provide a structure in two ways:

1. Contact and support for developers willing to work on a certain topic, and
2. a road map of topics the design team considers urgent.

If this structure is provided, development does not need to follow the design team very closely. Points of contact are clearly defined and help getting the work done.

Closing words

Design in FLOSS projects still is in its childhood. Unlike in development, structure is often seen as an opponent to or a restriction of freedom. The opposite is true.

Structure is a necessary enabler of freedom. It enables people to decide into which project they will invest their time. It enables consistency, the prerequisite of any usable or even desirable product. So do not waste your time in discussions about problems – invest it into discussions about a suitable structure for your project.

What do you think? Do you agree to my point of view? Let me know in the comments below.