Can Science Predict When a Study Won’t Hold Up?

Artificial

Intelligence

OpenAI Shutting Down Sora

Data Centers

A.I. Agents

White House Policy

Pentagon-Anthropic Dispute

Advertisement

SKIP ADVERTISEMENT

You have a preview view of this article while we are checking your access. When we have confirmed access, the full article content will load.

Supported by

SKIP ADVERTISEMENT

Can Science Predict When a Study Won’t Hold Up?

Conducting research is hard; confirming the results is, too. And artificial intelligence isn’t yet ready to help, a major new study finds.

Listen

· 9:50 min

Share full article

Brian Nosek, an executive director at the Center for Open Science. In the 2010s, Dr. Nosek and colleagues replicated 100 psychology papers — and matched the original results only 39 percent of the time.

Credit...

Kirsten Luce for The New York Times

By

Carl Zimmer

Carl Zimmer has reported on replication in science since 2011.

April 1, 2026,

11:10 a.m. ET

See more of our coverage in your search results.

Encuentra más de nuestra cobertura en los resultados de búsqueda.

Add The New York Times on Google

Agrega The New York Times en Google

Scientists publish more than 10 million studies and other publications a year. Some of those findings will add to humanity’s storehouse of knowledge. But some will be wrong.

To assess a study, scientists can replicate it to see if they get the same result. But seven years ago, a team of hundreds of scientists set out to find a faster way to judge new scientific literature. They built artificial intelligence systems to predict whether studies would hold up to scrutiny.

The project, funded by the Defense Advanced Research Projects Agency, or DARPA, was called Systematizing Confidence in Open Research and Evidence — SCORE, for short. The idea came from Adam Russell, then a program manager for the agency. He envisioned generating a kind of credit score for science.

“People can say, ‘Hey, this is likely to be robust, we can premise a policy on it,’” said Dr. Russell, who is now at the University of Southern California. “‘But this? Nah, this might make for a book in the airport.’”

The SCORE team inspected hundreds of studies, running many of them again, to better understand what makes research hold up. Now it is publishing

a raft of papers

on those efforts.

For now, a scientific credit score remains a dream, the researchers say. Artificial intelligence cannot make reliable predictions.

Thank you for your patience while we verify access. If you are in Reader mode please exit and

log into

your Times account, or

subscribe

for all of The Times.

Thank you for your patience while we verify access.

Already a subscriber?

Log in

.

Want all of The Times?

Subscribe

.

Related Content

Advertisement

SKIP ADVERTISEMENT