Diary study guide: how to get the best results
A diary study is a valuable tool for learning about user behaviour. Here’s how to get the best outcomes.
What is a diary study?
A diary study is a user research method that involves asking a number of people to record their experiences relating to a particular subject over a defined period of time. It's a really useful tool for learning about user behaviour, providing you with a record of thoughts and actions in context.
Diary studies aren't always the right choice, but they're well-suited for some use cases, for example if you're studying the use of mobile devices, which can be challenging given the location-independent nature of mobiles.
Field studies are another option, but often prove to be too impractical, particularly for a service or product that is used all day. Lab-based tests can give good insight, but for some areas of interest, they just won't give you real-world accuracy.
A diary study is flexible and needs no special materials or equipment. However, this can also be a potential pitfall. Without careful planning, it's easy to lose track of exactly what it is you want to get out of the process, and the data can then be incorrectly focused.
Different diary study methods
There are many ways in which you can ask participants to capture diary entries.
1. Paper diaries
Paper diaries are the most traditional type of diary recording method and are considered the most natural and personal. They're good because they can be used by participants with all levels of technical ability, they can be used at the same time as digital devices (for example, while browsing the internet), and they're very portable, lending themselves to the timely capture of events, as and when they happen.
However, they do have some downfalls. Firstly, the researcher must wait until the diary is returned from the participant before starting analysis. Also, rich experiences can't be captured in the same way that's possible with audio or video; handwritten notes aren't always easy to read and transcribing, if required, can be time-consuming.
Quick tip: paper diaries should include the participant's name, instructions for capturing entries, and details for when and where to return the diary, so participants are continually reminded of this during the process.
2. Email diaries
Participants can be asked to email all diary entries to the researcher (usually at the end of each day) in a similar manner to a blog. This is useful as the researcher can view the diaries each day, rather than having to wait until the end of the process.
The user can choose how they record their thoughts during the day, i.e. notes on mobile, pen and paper, or just recalling the day straight into the email.
Usually, the latter is what the user opts for, which means diary entries are usually made somewhat after the event. This is an issue since participants may not remember the exact details of what happened, what they thought, and how they felt at the time of the event.
3. Twitter feeds
Using a twitter feed allows participants to send a text message or tweet online, making entries to a private twitter account.
This is a much newer method and fits in with modern living, making it natural for many users. The researcher can view the diary entries as they occur and can adapt the focus of the study as it goes since insights can be gained straight away.
Short entries can be focussed and to the point, and a user may be more inclined to write them more often. However, Twitter's character limit may reduce the amount of information captured and its success will depend on the audience (think: is this is a comfortable way of communicating for all your users?).
Quick tip: Twitter feeds allow you to reply to participants if they require advice, but beware of interacting too much with them as it may alter their thoughts and actions.
4. A blended approach
Being flexible and allowing different participants to use different diary capture methods is better for the participants. However, bear in mind that having to analyse different types of diaries can complicate the analysis process.
Your diary study checklist
Deciding on the type of diary study is an important step in the process, but there is a lot to consider, so here are some key questions to consider.
1. What needs to be prepared in advance?
There are a few items that should be prepared in advance to ensure the smooth and successful running of a diary study.
Firstly, you'll need an analysis guide – a set of criteria against which each diary will be analysed. Be sure to include sections for everything you wish to observe.
You'll also need to create an interview guide outlining questions about the activities you're studying that you can ask to all participants. You can add points from an individual's diary which you want to expand on or clarify when you are reading the diaries.
Lastly, a management schedule is crucial for planning agreed dates/times for check-points, noting progress, and managing who has been contacted and when.
2. How should participant recruitment be handled?
Always recruit more participants than you need. A 15-20% drop-out rate is quite normal because of the longer-term nature of this user research, and the greater involvement required.
Also, some participants might not carry out the activities you're interested in and people's plans can change, so it's best to include some extra participants. A rule of thumb is to recruit about a third more participants than you need to get good data from.
Provide a participant study reference letter which outlines what's required of participants in general, as well as the specifics of what they'll need to do each day.
3. How long should the study be?
The period of time can either be set, or you can monitor the collected data until you have what you need. In practice, a set period of time often works better as it may be difficult to find participants to sign up to an open-ended study.
4. Is the diary content always quality data?
Make sure participants know what's expected of them in terms of the kind of information to record and when to record it. It is important though to avoid leading the participants by being too specific. Make sure they only record their natural behaviour and don't deliberately do things they feel would be more interesting to you. It's a difficult skill to master, but it's important you get this balance right otherwise the results may end up being skewed.
5. Are progress check points required?
Participants should be required to be available at a set number of check-points during the study. This is important to check their progress and helps minimise any drop-outs by incentivising the completion of each check-point (and, if possible, having increasing incentives for each check-point).
The number of check-points depends on what is being studied, but a minimum of three is recommended, for instance one at the start, one halfway through, and one at the end which includes an interview.
If using paper diaries, near the end of the study, remind participants to return their diary as soon as possible. Incentivising the receipt of completed diary content will help ensure content is returned as soon as possible. Also try to keep the time delay between completing the diary and coming to interview to a minimum to make sure the experiences are still fresh in their mind.
6. How should the diaries be analysed?
Record key observations from the analysis of diary content using the analysis guide you prepared in advance. Start by summarising the key steps taken by the participant throughout the study. Next, try and identify any patterns between participants.
Patterns to look for include: common behaviour (what they did), differences in behaviour, context of use, situated action, tasks, user needs, information requirements, successes/failures, and barriers/problems encountered.
7. What should interview follow-up entail?
Talk the participant through the completed analysis guide. Start by going through the steps of each diary, asking why each step was done will help clarify user behaviour. Try to clarify any patterns of behaviour and expand on areas you found interesting.
Where possible, try to cover the same topics in each interview so that you can compare experiences across participants. People generally love to talk about their experiences, so ask as many questions as you need to.
Make sure the atmosphere is conversational rather than formal. Trying to formalise a discussion around something which is everyday behaviour will negatively impact the responses you receive. Interviewing participants' face-to-face works better if possible, as it is more relaxed and less structured than via the telephone.
8. What happens next?
Having completed the diary study you will now have a wealth of information about your target group which you can report on or feed into a design process. Make sure you keep everything created during the study, from the diaries themselves, to your analysis of the diary content and interview notes. This will act as a handy reference to help answer subsequent questions about your target group.
A diary study is a quick and inexpensive way of obtaining real-world data about user behaviour. Careful management of your target group together with studying these guidelines will enable your diary study to run smoothly and provide useful results.