Research slop is human slop.
A response to recent articles on AI research tools and the future of research.
I read two articles last week about “research slop.” Judd Antin worries AI tools let anyone produce sexy, terrible research at lightning speed. Jess Holbrook calls these “blurry, AI-shaped objects.” Insights that look real but lack substance.
What bothered me was that their framing lets us off the hook. The research slop problem is mostly about human responsibility, not AI capabilities.
Yes, Antin and Holbrook acknowledge that “bad research has always been a thing.” But neither article pushes this far enough.
If we’re going to solve the research slop problem, we need brutal honesty about its origins. Research slop has always been a human creation. AI just makes our existing failures more visible at scale.
1. We’ve always generated slop. Let’s own it
The history of research is not a story of methodological excellence corrupted by technology. It’s a story of humans repeatedly failing at rigor and ethics.
Let’s not go too far back and start with my field of psychology. The replication crisis that exploded around 2011 revealed that between 50% and 75% of published psychology studies couldn’t be replicated. Consider Diederik Stapel, the Dutch social psychologist who fabricated data for at least 55 studies over nearly two decades. He didn’t need an LLM; just Excel and a willingness to make up numbers.
The crisis goes deeper than outright fabrication. P-hacking (running multiple analyses until you find a significant result) became so normalized, it was barely considered misconduct. The file drawer problem (hiding negative results because journals wouldn’t publish them) was standard practice. Humans systematically corrupting their own field.
Psychology wasn’t unique. John Ioannidis’s 2005 paper “Why Most Published Research Findings Are False” demonstrated that statistical power issues, bias, and selective reporting plagued medical research. Economics faced its own reckoning after the 2007-2008 financial crisis, because its models treated humans as predictable, machine-like entities rather than inventive, creative beings who can modify the future. Physics found that Jan Hendrik Schön fabricated data in at least 16 papers in top journals. The Reproducibility Project in cancer biology found that most foundational studies couldn’t be reproduced. Andrew Wakefield, whose fraudulent research linking vaccines to autism sparked a public health crisis we’re still dealing with.
No AI required, just human shenanigans. Human slop.
And UX research? We’ve been plenty sloppy.
Jakob Nielsen’s “magic number 5” became gospel, despite being based on heuristic evaluations of specific interfaces rather than generalized usability testing. Nielsen himself acknowledged major limitations, but the industry embraced it anyway because it justified doing less work while claiming rigor.
Personas divorced from actual data, becoming creative writing exercises. “Insights” presentations where correlation was sold as causation. Cherry-picked quotes becoming “themes.” Leading questions, confirmation bias, and predetermined conclusions dressed up as research.
And don’t even get me started on NPS.
Slop.
Antin shares a story about a PM who “ran dozens of AI-moderated interviews” that produced “flaws, mistakes, and hallucinations.” But I’ve seen my share (and I know Judd has seen his) of human-moderated interviews produce identical problems: leading questions, confirmation bias, lack of documentation, and predetermined conclusions.
The only difference? The human-led disasters took longer and cost more.
The pattern is clear: research methodology crises aren’t new, and they’re not caused by technology. They’re caused by humans cutting corners, seeking validation for existing beliefs, and producing work that appears rigorous but is dangerously flawed.
2. The real problem: AI lacks context, and humans lack the skills to deliver it
Holbrook gets remarkably close to the core issue when he describes how context in organizations is:
“distributed across people, organizations, and documents. Some context is digitized. Most resides within people.”
He’s exactly right. Humans produce slop when they lack context and skills, and AI is no different.
Think about what makes any researcher effective. They need to:
Understand the product roadmap and business constraints.
Access previous research findings and institutional knowledge.
Know their specific user base and actual behaviors.
Build relationships with stakeholders that surface real questions, and grasp what’s been tried and why it failed.
Without this context, what happens? Wrong questions. Misinterpreted answers. Insights that ignore crucial constraints.
Or consider the “vendorizing” of qual research at many companies. If you’ve spent high five-figures on external research only to get back insights that make you dread your next 1:1 with your manager, you know exactly what I’m talking about. The vendor lacks domain knowledge, organizational context, and connection to your specific user base. Is the vendor to blame? Or did we fail to provide proper context?
When AI tools fail for exactly the same reasons (i.e. lack of context) we blame the AI.
The PM in Antin’s story who ran AI-moderated interviews that produced flawed outputs? I guarantee the AI moderator wasn’t connected to customer support data showing actual pain points, product analytics revealing user behavior patterns, previous research highlighting known issues, or the competitive landscape and market positioning.
This isn’t an AI problem. It’s a context engineering problem. Holbrook’s recommendation to “always conduct some sessions yourself” and use AI as “adversary and accelerant, not author” is a good start. Antin’s call for “expert-in-the-loop” approaches is also helpful.
But the solution isn’t just better AI guardrails or human-in-the-loop. It’s about context and building human skills to deliver it.
We need to teach people how to shape AI tools that serve research needs: engineering the right context into systems, determining when AI is appropriate and when it’s not, designing validation frameworks against domain knowledge, connecting AI to organizational data and knowledge systems, and building recognition of what credible substance looks like versus credible appearance.
3. The super-exponential curve of AI improvement
AI is improving faster than any previous technology, and we’re not talking about this enough.
When electricity transformed industries, it took 40-50 years to fully penetrate manufacturing. The internet took about 20 years to become ubiquitous. AI? We’re seeing capability improvements not on yearly timescales, but weekly and monthly ones.
Recent research describes not just exponential growth, but “superexponential” growth. The rate of acceleration itself is increasing. The technical term is “jolting”: a positive third derivative of capability metrics. AI isn’t just getting better exponentially.
It’s getting better at getting better.
Model capabilities that seemed impossible in 2023 became standard in 2024. Model capabilities that seemed insane in 2024 became standard in 2025.
Today’s limitations are not tomorrow’s limitations.
Yes, current AI-moderated interviews sometimes miss nuance. Yes, AI synthesis can be shallow without proper context. Yes, the technology makes mistakes.
But it made more mistakes six months ago than today. And it will make far fewer mistakes six months from now.
When Garry Kasparov lost to Deep Blue in 1997, chess experts insisted Go would never fall because it was too complex, too intuitive. AlphaGo won in 2016. When radiologists saw early AI attempts at reading X-rays, they emphasized the contextual knowledge machines lacked. Now, AI-augmented radiology systems detect certain cancers as well as, or better than, humans. And radiologists who’ve learned to use these tools have become massively more effective.
The pattern repeats: experts identify current limitations, declare them fundamental rather than temporary, then are surprised when the technology leaps past supposedly insurmountable barriers.
AI research tools in late 2025 are not the ceiling. They’re the floor.
Conversations about research slop should focus less on current AI limitations and more on building human capabilities that will remain valuable as AI vastly improves. The skills Holbrook and Antin outline—building intuition through direct contact, using AI as an adversarial reviewer, and taking confidence passes—will become increasingly valuable as AI improves.
What this means for research
Both Antin and Holbrook end their pieces with calls for researchers to shape how AI tools are integrated and maintain our standards. I completely agree. Three points:
Build better context infrastructure
Stop treating AI tools as standalone products. Integrate them with your research repositories, product data, user databases, and institutional knowledge. As Holbrook notes, context is distributed. Our job is to make it accessible.
The gap between “research slop” and “valuable research” is almost entirely about context engineering—enabling AI to work with the full picture.
To be clear, this requires guardrails: data privacy controls, security protocols, and clear boundaries around what information AI systems can access and how they handle sensitive user data. But we need to hustle, define the guardrails, and set AI platforms up for success, as we would human researchers.
Invest in human skills, not just AI guardrails
The problem isn’t that non-researchers can access research tools. As both authors acknowledge, democratizing research is fundamentally a good thing. The problem is we haven’t taught researchers or non-researchers how to use these tools well.
We must create frameworks for human-AI partnership, define the contexts AI needs to produce valuable outputs, establish how to validate outputs against domain knowledge, teach people to recognize when something appears credible but lacks substance, and build our AI evals skills and tooling.
Embrace the exponential curve
The researchers who thrive won’t be the ones who remain outside of the AI arena. Researchers who thrive will be those who put in the work to become exceptionally good at leveraging AI tools to conduct research that was previously impossible. Research at scales we couldn’t reach, at speeds we couldn’t match, and with patterns we couldn’t see.
We must match the pace of technological change with fervent effort to keep up and leverage these technologies to their fullest.
A necessary reframing
Research slop isn’t new. Research slop is human. AI didn’t create it, and AI won’t fix it. The solution isn’t better AI constraints, it’s better context and human skills. Current AI limitations will be short-lived. Human judgment and context engineering are longer-standing necessities.
Both articles acknowledge these points, but don’t go far enough. If we’re going to solve this problem, we need to stop treating AI as the variable we need to control and start treating human capabilities as the variable we need to improve.
If we can be honest about our own failures and invest in the skills to use these tools well, human-AI collaboration will make research less sloppy.
Massive thanks to Inna Tsirlin, Ryan Anderson, and Andy Warr for reviewing this post and providing feedback.
P.S. This article was written in collaboration with Claude. And if you think it’s slop, blame me.
P.P.S. Want to learn more about how to leverage AI for research? Join the AI Insights Lab subreddit here.