A new AI to analyze public opinion

When you’re a policymaker trying to make sense of thousands of public comments on a new environmental policy, you worry about comments concerning costs and fairness, or hear personal stories on the topic. Whatever those responses entail, the sheer volume of them is overwhelming. Traditionally, you’d need a team of researchers working for weeks or months, combing through text line by line to identify common themes. However, with a machine performing the same work in hours, these jobs can be completed at a fraction of the cost.

That’s the promise of DECOTA, the Deep Computational Text Analyzer developed by a team of researchers from the University of Bath and University College London, and published on Psychological Methods.

The balance between quantitative and qualitative data

Numbers tell one kind of story. Polls might say “60% of people support a new tax,” but comments from people explain why. The comments can reveal that maybe people like the idea in principle, but fear it will hit low-income households hardest. This kind of qualitative data is gold for researchers and policymakers, offering context and nuance that numbers alone can’t capture. However, analyzing this data is slow, expensive, and heavily dependent on human expertise.

Traditionally, social scientists use a method called thematic analysis, which involves identifying recurring “codes” (like subtopics) and grouping them into broader “themes.” While powerful, thematic analysis usually works best on a few hundred responses. Once you get into the thousands (or tens of thousands), the process becomes unmanageable.

Enter DECOTA

DECOTA is designed to tackle this problem head-on. Think of it as an automated research assistant that can scan massive text datasets, sort them into meaningful themes, and even provide illustrative quotes, all without requiring weeks of human labor. DECOTA begins with a method called structural topic modeling, which identifies clusters of related words across responses. Then, an AI large language model based on GPT-3.5 interprets those clusters and turns them into clear “codes” (concise labels for what people are talking about). Similar codes are grouped using a sentence transformer, a tool that measures semantic similarity. Another model then names these groups of codes as overarching “themes.” However, DECOTA also shows how often they appear among different groups (say, younger vs. older respondents) and offers example quotes. By doing this, the structure of DECOTA produces a streamlined analysis that mirrors what human researchers do, but at a much faster speed.

In tests, DECOTA proved to be 378 times faster and 1,920 times cheaper than human coding. And it wasn’t just about speed; its outputs matched or complemented human analyses more than 90% of the time. That level of accuracy makes DECOTA more than a toy; it’s a serious tool for research and policy.

The team that developed DECOTA also validated the system on four different datasets. These datasets included people’s views on saving water, local climate policies, and even feedback on a public health website during the COVID-19 pandemic. In each case, the machine-generated themes aligned closely with the human-coded ones, and sometimes even captured nuances that humans had overlooked.

Currently, vast amounts of public input go unanalyzed. In the UK alone, government consultations generate mountains of free-text responses that often sit unused due to a lack of time and staff. In healthcare, over 80% of data is unstructured text, such as from patient notes and feedback forms, and most of which is never systematically analyzed. DECOTA could help unlock those insights. DECOTA isn’t meant to replace human researchers. Instead, it’s a way to make qualitative insights accessible when traditional analysis isn’t feasible. For policymakers, that could mean faster, evidence-based decisions. For nonprofits and businesses, it could reveal what communities or customers are really saying. And for academics, it could free up time for deeper interpretation rather than repetitive coding.

Of course, no AI system is perfect. There are risks of bias, misinterpretation, or overreliance on machine outputs. Still, the vision is clear. As large language models continue to improve, tools like DECOTA could democratize access to qualitative research, making it possible for city councils, NGOs, and even small community groups to analyze voices that would otherwise go unheard. In a world where public trust in policy depends on listening to opinions as well as measuring them, that’s no small achievement.

If you want to learn more, the original article titled "The Use of Large Language Models for Qualitative Research: The Deep Computational Text Analyser (DECOTA)" on Psychological Methods at https://doi.org/10.1037/met0000753.