On November 2, 2026, Stanford researchers Owen Queen and James Zou published a study with an uncomfortable message for everyone who builds, buys or funds artificial intelligence. They took 50 widely cited findings about large language models, reproduced each experiment, and ran the tests on 20 models from eight developers, released between July 2024 and September 2026. The result: 44% of the findings had expired, meaning their effect had fallen to one-fifth or less of its original size. Another 34% had weakened significantly. Only 16% still held at their original strength, and 6% had grown stronger.
The headline number needs careful reading. The 50 findings were hand-picked by the authors, not randomly sampled, so the figures do not mean that 44% of all AI research is obsolete. The study is an interactive project page with open code and data, not yet a peer-reviewed paper. But even with those caveats, the direction is clear: claims about language models age fast. For the 22 findings where an expiry date could be estimated, the average lifespan was 15.7 months and the median was 14.7 months. About 4% of claims had expired within six months of publication, 16% within a year and 39% within two years.
Which Findings Expire, and Which Survive
The pattern of what faded and what lasted is the most useful part of the study. Capability failures expired fastest. Five of nine reasoning-related findings expired, and none of them held unchanged or grew stronger. The examples are familiar to anyone who followed early language-model research: models failing to count letters in a word, struggling with arithmetic in unfamiliar number systems, missing facts buried in the middle of long documents, inventing citations, and abandoning a correct answer when challenged. These were real weaknesses of the models of 2023. They are, increasingly, not weaknesses of the models of 2026.
The citation case study makes the point vividly. A 2023 study found that ChatGPT, running on GPT-3.5, fabricated 55% of the academic references it produced. Repeating those prompts, the Stanford team found GPT-3.5 Turbo fabricated 59% of 210 references, while Claude Opus 4.6 fabricated only 4% of 302. Other coverage reported fabrication rates as low as 0% for GPT-6.1 Sol and 1% for Claude Sonnet 5.5. The problem has not vanished: one model in the panel, Qwen3.7-Flash, still fabricated 69% of its references.
What survived? Mostly findings about preferences and judgement rather than raw capability. Sycophancy, the tendency of models to agree with the user rather than tell the truth, persisted. So did evaluator bias, the tendency of a model used as a judge to favour its own outputs. These are tendencies baked into how models are trained and aligned, not bugs that a bigger model automatically fixes. That distinction matters: capabilities improve with scale, but habits of judgement may not.
Why This Matters for Founders
For founders building AI products, the study is a warning against building a company on a borrowed claim. A startup whose pitch is "models cannot do X, so we do X for them" is standing on ground that moves. If the weakness it exploits disappears in the next model release, the product's reason to exist disappears with it. Founders should test every assumption against the models they actually ship on, and re-test on every major release. A benchmark result that is two years old is not evidence; it is history.
The study also changes how founders should talk about their own products. Claims such as "our model hallucinates 90% less" are only meaningful if the model version, the test date and the prompts are stated. Queen and Zou recommend exactly that: document the model, the version and the date, and publish prompts and code so others can repeat the test.
Why This Matters for Investors
For investors, the study reframes technical diligence. A pitch deck that cites a 2023 paper to argue that a capability gap will persist is making a bet that progress has stopped. The Stanford data suggest that bet usually loses within about fifteen months. Diligence should therefore focus less on what models cannot do today and more on what is structurally hard: proprietary data, distribution, workflow integration, or a network effect that survives even as the underlying model improves.
Investors should also be sceptical of vendor benchmarks. A benchmark result is a snapshot of a specific model on a specific test on a specific date. Ask which model version was tested, when, on what prompts, and whether an independent party can reproduce the result. If the answer is vague, the claim should be discounted.
Finally, the study points to a new diligence step: independent replication. Just as investors commission code reviews and customer calls, they should increasingly commission fresh benchmark runs on the current models before writing a cheque. The cost is small compared with the investment, and the Stanford study shows the alternative can be wrong nearly half the time.
The reproducibility crisis is not a reason to distrust AI research. The original studies were mostly right about the models of their day. It is a reason to treat every AI claim as perishable, with a shelf life measured in months.
Angeled.com connects founders with investors.
Sources: Owen Queen and James Zou, Stanford University, "The shelf life of LLM findings," November 2, 2026; The Decoder, November 3, 2026; Superpower Daily and EMSI, November 2026.
Read Next
OpenAI Releases Codex Security as Open-Source CLI and SDK
Databricks Unveils Always-On Genie, an AI Agent That Stops Waiting to Be Asked
DeepMind Launches AlphaGenome 2, a DNA Model That Reads Genomes Cell by Cell
ElevenLabs Unveils 13 New Products and Upgrades at Its New York Summit
Google Launches Gemini Decision 1, a Model That Answers Instead of Writing
Runway Open-Sources Praxis-1, Its Robot Model That Learned by Watching Video



