How to conduct a research "Audit"

Written by 

Luke Kelly

 

September 11, 2020

What was the first deliverable of your last project? Hopefully it wasn’t the final report. One of the earliest lessons I had to learn as a researcher was how to get comfortable sharing things that weren’t finished. A lack of confidence made me want to polish every paragraph and make every video edit immaculate - when all my boss wanted was reassurance that things were progressing in at least vaguely the right direction.

Often the first thing you’ll share is a plan, but even here it’s worth being careful. Every minute spent glossing a gant chart is one which delays the arrival of actionable results, and if you’re in a particularly dysfunctional environment you can end up bogged down in meeting after meeting discussing work that you could have finished already. Don’t try and plan your way to the promised land: start work as quickly as you can and then adjust your course accordingly. A complete U-turn after a week is better than a month spent arguing about the right route. 

Every project needs some kind of structure put in place at the outset, but ideally this should be something that’s useful in and of itself. This is what led us to start conducting “audits”: a review of what’s already been done that’s aimed at defining what should happen next. An audit is the best way to achieve the four things you need to do in order to kick off a project successfully - and also has the advantage of being a deliverable in it’s own right. 

Obviously you don’t always need to complete one - the manager who’s asked you to moderate the weekly usability lab is going to be slightly nonplussed if you insist on completing the below steps beforehand - but for any strategic or discovery piece of work, they’re a great way to get the ball rolling. And getting started is easy:


Step 1: Get hold of anything and everything relating to your topic

And we really do mean anything: obviously you should get hold of any research already done in this area - whether that’s an agency report, some market research, a workshop transcript or even a training manual for front-line staff - but never be afraid to also do the basics. Google will quickly flag up any specialist news websites operating in your topic area, and even sites like Quora can help with the fundamentals.

Don’t worry too much about what you’re looking for. When I started researching at Farfetch, I knew nothing about fashion. When I started at Shell, I knew nothing about energy. Or petrol stations. Or where they got their sausage rolls from. When you know nothing about an area, everything you read will fill in some of the blanks. Take advantage of the complete blank slate you’re bringing to the topic and spend at least a day doing nothing but reading whilst taking rough notes for yourself. 

After a day or so, start writing these notes up as a series of discrete points. Keep the format simple: a couple of sentences and a supporting quote (if you have a nice one) is fine. 

All you’re doing at this point is trying to get a picture of where the boundaries of your research question are (sausage rolls ended up out of scope for me, which was obviously devastating). To make this explicit, it can help to start processing your insights into a shareable format. 


Step 2: Write up what you know as a “day in the life”

You should have at least a vague idea by this point about who your customer is. I often ended up focusing on the B2B side of things, so my “customers” at Farfetch and Shell were fashion boutique owners and petrol station owners (who have a lot more in common than you might think). Once you have an understanding of who these people are, the best way to articulate what you think might be relevant is to describe a typical day for them: what tasks do they need to complete? What challenges do they face? What pork-based savoury snacks do they need to source in order to keep their customers happy?

At Filo we structure this as a simple journey map, and we’ve designed the way journeys work in the repository to support this. At the top level you have stages: “arriving at work”; “the lunchtime rush”; “cashing up” and so on. These are then broken down into a series of discrete tasks: “the first thing Jenny does when she arrives at work is check the site to make sure it’s safe”. Anything we don’t know which feels relevant - “What happens if she finds something wrong?”; “Who trained her how to do this?” - is added as an “insight” to each task. 

Sometimes you might want to make a separate map that represents a different perspective. Maybe you want to “zoom out” and talk about how petrol stations work at the country level (“How does petrol get from the refinery to the forecourt?”). Depending on your topic you might also want to “zoom in” and focus on the step-by-step process needed to complete a particular task (“How does the software used for cashing up work?”). 

Don’t be afraid if you end up with more questions than answers - that’s the whole purpose of the exercise. What you’re effectively doing is contextualising what you don’t know - and the next step is to start filling in some of these blanks.


Step 3: Talk to colleagues who know more than you do

If you’re just starting off in a new industry, this means pretty much anyone and everyone. A great thing to do in any new job is to ask people if you can grab an hour of their time and hear about their experiences. If you bring your “day in the life” journey along then you can structure this conversation around whatever gaps you’ve flagged in your rapidly growing knowledge. It’s much easier for people to understand what you need to know if you’ve already visualised this for them.

There are a couple of other benefits to this approach too. Firstly, you’re introducing your team to the sort of thing you’re going to produce for them as a researcher. They might know a lot more about the topic than you, but they also might not have seen this information structured into a journey map before. Introducing yourself this way means they’ll have a much better idea of what you actually do - which is never a bad thing to establish with people.

Secondly, after working on your map with 3 or 4 different colleagues you might well have ended up with something that’s a nice deliverable in its own right. You don’t need to formally present it as such - you can always explain that it’s just something you made to help your own understanding - but a nicely formatted journey map which explains your industry and highlights the knowledge gaps you’re trying to fill is a great thing to have at the end of your first week on the job. There’s also one more thing you can add to it:


Step 4: Add a “data” lane to your map

This is something I’ve learned the hard way. For many UX Researchers, quantitative data is something outside of their expertise. In bigger companies it’s often treated as a separate discipline with dedicated teams of analysts or data scientists. Whilst it might not be your responsibility to apply quantitative metrics to your qualitative findings, you're limiting the impact they can have if you ignore them altogether.

It can help to think of it this way: when looking at your “day in the life” how would you measure “success”? What sort of metrics could turn the experience your map describes from a good one to a great one? In a straightforward ecommerce environment this might be more sales; more customers; or more repeat visits. In B2B environments it might involve having more time - so spending less time on the tasks you’ve identified, or being able to complete more of them.

The specifics don’t matter as much as the principle: be the first to start asking the question “what does good look like?”. The whole purpose of your research is that your company is going to do something as a result - change a product; develop a new one; improve a service. Asking up front “how will we know if this has worked?” makes defining success something everyone has to think about. Even if you can’t collect the data you need yourself, flagging it up as relevant shows people that you’ve thought about it. You’ve identified another negative space which needs to be filled - even if someone else will end up filling it. And now it’s time to pull everything together.


Step 5: Highlight the gaps, and make a plan for filling them 

By now you’ve gone through all the material you can get your hands on, shown it to anyone who will talk to you, and tied it all up into the most complete story you can currently tell about your customers. The next step is to look at where the gaps are, and explicitly agree with your stakeholders how you’re going to go about filling these. It should be obvious to anyone at this point who you need to talk to and what you need to ask, but if you’ve flagged up lots of outstanding questions then it’s worth prioritising which are the most important.

It’s not uncommon for projects we work on to pivot wildly at this point. Sometimes companies realise that they have everything they need to start working on some ideas, so we won’t do any additional fieldwork at all. Either way it’s always a useful exercise to get everyone in a room and say “this is what we know, this is what we don’t know, and so this is where we’ll focus”. Being able to physically point at a space on a map and say “this is where we’re working” is by far the best way for teams to articulate what they need.

Think of it like a jigsaw. You’ve gathered as many pieces as you can, and assembled as much of the puzzle as possible. This gives you a good enough idea of the finished picture to prioritise the pieces you’ll go out and collect next - but if the image is complete enough to proceed without even doing that, then you’ve just saved everyone a great deal of time and effort. 

Assuming more work is needed, you’re now all set for the next step: recruiting your participants. 


This article is part of our "Beginners Guide to Research" series, which sees a new article uploaded every week. Drop us a line at team@filo.io if you'd like to know more.