Research automation: how to do more with the same team

The work that fills most of the day

Ask a researcher how they spend their time and the answer is usually some version of: less on the interesting parts than I would like.

The interesting parts are analysis, interpretation, synthesis. Understanding what the data actually means. Identifying the insight that changes how a client thinks about a problem. Writing the narrative that makes findings land.

The parts that fill the day are everything else. Exporting data from one tool, cleaning it up, reformatting it to work with the next one. Running searches, copying content into spreadsheets, reading through it to find what is actually useful. Building slide templates, formatting tables, chasing respondents who have not answered yet.

In my experience running research projects, the mechanical work, the collection, cleaning, structuring, and formatting layer, accounted for a large part of the project time. The thinking work, designing the right questions, interpreting the findings, shaping the narrative, got whatever was left.

That ratio is not inevitable. It is a consequence of how research workflows have historically been structured, not a feature of research itself.

The SurveyMonkey problem

One example that will be familiar to anyone who has run survey-based research: getting data out of a survey platform and into a format you can actually work with.

Data collected in SurveyMonkey, or any similar tool, requires a manual export. The export format is rarely what you need. Columns need renaming. Response scales need recoding. Open-ended responses sit alongside quantitative data in the same sheet, but getting them into a usable format for analysis requires more manipulation than it should. By the time the data is in a state where analysis can begin, a meaningful chunk of the project time is already gone.

This is not a criticism of any particular tool. It is a description of what happens when research workflows are built from components that were not designed to work together, and when the connecting tissue between those components is manual human effort.

The same pattern is even more pronounced in desk research and online monitoring. Searching for relevant content online, reading through results to find what is genuinely useful, organising what remains into something an analyst can work with. Each step requires effort that does not require expertise. Together they consume time that expertise could be applied to instead.

What automation should and should not mean

The word automation makes some researchers uncomfortable, and understandably so. Research quality depends on human judgment. The concern is that automating parts of the process means removing that judgment, and that the output suffers as a result.

That concern is valid when applied to the wrong parts of the process. Automating the decision about what is a meaningful insight is a bad idea. Automating the manual export and reformatting of survey data is not.

The distinction worth making is between tasks that require expertise and tasks that require effort. Expertise means knowing which questions matter, understanding what good evidence looks like, spotting nuance in how people describe a problem, and translating raw findings into something a client can act on. Effort means running a search query, copying content into a spreadsheet, reformatting a data export.

Only one of those categories benefits from a researcher’s experience. The other is just time. And time spent on effort is time not spent on expertise.

What the automated layer looks like

For desk research and continuous monitoring, the mechanical layer is primarily collection and filtering. Defining the sources to search, running queries systematically, retrieving content, applying filters to remove noise, and organising what remains for analysis.

When this layer is automated, a researcher defines the parameters once: which sources to monitor, which topics to track, what filtering criteria to apply. The system handles the ongoing collection. The researcher’s time goes into reviewing what has been surfaced, identifying what is significant, and turning it into something useful.

The volume of content a researcher can work with increases significantly, because the bottleneck shifts from collection to analysis. Instead of spending three days searching and filtering before analysis can begin, the researcher arrives at a clean, relevant dataset and starts the interesting work immediately.

For survey-based research, the equivalent is better integration between data collection and analysis tools, and better automation of the formatting and cleaning steps that currently happen manually. This is a more fragmented problem because it involves a wider range of tools, but the principle is the same: the steps that require effort rather than expertise should not require a researcher to do them.

What this means for a small team

The practical implication for a small research team or a solo researcher is significant. If a large proportion of project time is mechanical work, and that work can be substantially automated, the same researcher can handle a meaningfully larger volume of work without a proportional increase in hours.

This is not about replacing researchers. It is about changing what researchers spend their time on. A researcher whose collection and filtering layer is automated is not doing less research. They are doing more analysis, more interpretation, more of the work that produces the findings clients are paying for.

For an agency, this changes the economics of research delivery. For an in-house team, it changes what is possible with a fixed headcount. For a solo researcher, it changes what kind of clients and briefs are viable.

The constraint was never intelligence or expertise. It was the time consumed by work that did not require either.

If you’re thinking about where automation could change the economics of your research work, we’d love to hear what you’re working with. Get in touch.