Kairo — AI-assisted research automation on Claude Code
A personal research automation pipeline built entirely on Claude Code that runs literature synthesis, adversarial hypothesis critique, preregistration, and real experiments as a disciplined, auditable process instead of a single chat conversation.
Kairo is a personal AI-assisted research automation tool built entirely on Claude Code — it runs on a Claude subscription with no separate API keys — that treats the scientific method as a disciplined, auditable pipeline instead of a single chat conversation: literature search and synthesis, hypothesis generation under adversarial critique, mandatory preregistration, real experiment execution, and confidence updating, plus a bottom-up path that lets engineering or product work spawn a formal hypothesis the moment it hits genuine technical uncertainty. Every stage is grounded in an actual methodology rather than invented from scratch: Popper, Mayo, and Meehl's philosophy of severe testing shapes the critique cycle; Simmons, Nelson, and Simonsohn's 2011 work on researcher degrees of freedom is why preregistration is mandatory before any code runs; the Cochrane Handbook governs how evidence from replications gets combined; PRISMA and Wohlin's citation-snowballing method structure literature search. I benchmarked the design against documented failure modes of frontier AI research systems — DeepMind's AI Co-Scientist, Sakana AI's 'The AI Scientist', FutureHouse's Robin/Kosmos — and built in rigor mechanisms those systems lack: a hypothesis only reaches 'confirmed' after an independent replication, never a single experiment; analysis is applied mechanically by versioned scripts rather than an LLM's subjective read of a result; every experiment freezes its prediction, metric, and decision thresholds before any code runs, with a full environment manifest (dependency hashes, seeds, dataset hashes) for exact reproducibility; and a dedicated validity gate refuses to produce any verdict when an experiment's own sanity checks fail, instead of fabricating a conclusion from broken data. It's built on Claude Code's Skills, Subagents, Agent Teams, MCP servers, and Hooks, keeps its knowledge in an Obsidian vault with local, no-API-key semantic search, and does literature discovery through free public APIs (arXiv, Semantic Scholar, PatentsView) with citation-graph snowballing — packaged as an installable open-source Claude Code plugin, cleanly separated from any private research data.
The problem
AI research assistants tend to collapse the scientific method into a single chat turn: an LLM proposes a hypothesis and reports on it in the same breath, with no separation between generating a claim and trying to break it, no commitment to a prediction before seeing results, and no mechanism to admit an experiment's data is too broken to support any conclusion. That's indistinguishable from a chatbot with search access, and it fails exactly where rigor matters most — it will still produce a confident-sounding verdict even when it shouldn't.
The solution
I built Kairo as a pipeline instead of a chat: literature synthesis feeds hypothesis generation, which goes through adversarial critique before being locked into a preregistration — prediction, metric, and decision threshold frozen before any code runs — then into real experiment execution with a full reproducibility manifest, then through a validity gate that can refuse to produce a verdict at all, and finally into confidence updating that only promotes a hypothesis to 'confirmed' after an independent replication. I validated it end-to-end on a known-answer case — a well-documented ML phenomenon I deliberately didn't name to the tool, so its search and reasoning had to rediscover it independently. The literature search connected two distinct bodies of research the prepared answer key hadn't anticipated; the critique pipeline caught and refuted a plausible-but-false hypothesis, then refined it into a genuinely testable claim through a real multi-round loop; and two real GPU experiments run on Kaggle were both correctly flagged as invalid by the integrity checks instead of producing a false verdict, even after the compute cost had already been spent — that refusal to fabricate a conclusion under real pressure was the most important result of the whole evaluation.