Newsletters
The company behind some of the world’s most powerful AI systems is now warning that these same systems could soon build their own successors, with no humans at the keyboard.
That is not a line from a science fiction screenplay. It is, more or less, what Anthropic said in a blog post co-authored by co-founder Jack Clark and Anthropic Institute President Marina Favaro.
The company acknowledged that AI systems are already taking over an increasing share of the software development and research processes that it once relied on humans to run. If that trend continues, the post said, we may arrive at what researchers call "recursive self-improvement"—a point at which AI designs and builds the next generation of AI.
"We are not there yet," it read, "and recursive self-improvement is not inevitable. But it may happen sooner than most institutions are prepared for."
That idea is worth pausing on.
The company that likely understands AI’s progress best is warning that AI advancements could outrun our capacity to keep pace. And as more research shows, they are not alone in this view.
The idea of AI building other AI isn’t new, but it’s speeding up in ways even its earliest creators didn’t expect.
In May 2017, Google Brain researchers introduced AutoML, an AI system built to create other AI systems. They equipped AutoML with a controller neural network that could suggest "child" AI designs, train them for specific tasks, test their performance, and improve its suggestions. This cycle repeated thousands of times, all without humans involved in each round.
The resulting child AI, called NASNet, was built to recognize objects in real-time video. When tested on a leaading image classification dataset, it reached 82.7% accuracy. That was 1.2% better than any previous human-designed system and 4% more efficient.
"If we succeed," Google researchers wrote at the time, "we think this can inspire new types of neural nets and make it possible for non-experts to create neural nets tailored to their particular needs."
That was back in 2017. The real question became: what happens if we succeed beyond our expectations?
The capabilities demonstrated by AutoML were narrow and controlled. What researchers documented more recently is something considerably less contained.
A collaboration between Aizip Inc. and scientists at the Massachusetts Institute of Technology (MIT) and several University of California (UC) campuses demonstrated that large AI models can now create smaller, specialized AI systems from start to finish—from data generation through model deployment and testing—without human involvement at any stage.
"This is the first step in the path to show that AI models can build AI models," Yan Sun, CEO of Aizip, told Fox News. Yubei Chen, a UC Davis professor and Aizip co-founder, described the dynamic using a sibling metaphor: "Right now, we're using bigger models to build the smaller models, like a bigger brother helping its smaller brother to improve. That's the first step towards a bigger job of self-evolving AI."
The practical applications being discussed—hearing aids that detect human voices, pipeline sensors that flag integrity failures before they occur, and wildlife trackers that use satellite data—sound reassuring.
They are specialized. They are useful. They are small. But the structural question the research raises is large: if a bigger model can design a smaller one without human oversight, what happens when the bigger model turns that same capacity on itself?
That question has moved from philosophy to active engineering.
A separate study by Chinese researchers, published in late 2024 and reported by Live Science, found that two popular large language models were capable of cloning themselves. Across 10 trials, one model successfully replicated itself in 50% of attempts. The other did so in 90%.
"Successful self-replication under no human assistance is the essential step for AI to outsmart humans," the researchers wrote. "It is an early signal for rogue AIs."
The paper hasn’t been peer-reviewed yet, and its findings are still debated. But the fact that this research exists at all shows that the community is starting to see self-replication as a real engineering goal, not just a far-off idea.
In May 2026, Richard Socher, a well-known AI researcher and founder of the chatbot startup You.com, launched a new company, Recursive Superintelligence, in San Francisco. Backed by $650 million, the startup aims to build an AI model that can spot its own weaknesses and redesign itself to improve.
"Our main focus is to build truly recursive, self-improving superintelligence at scale," Socher told TechCrunch. "This means that the entire process of ideation, implementation, and validation of research ideas would be automatic," he said.
The project includes researchers like Peter Norvig, a leading figure in AI, and Tim Shi, co-founder of Cresta. Socher compared their approach to biological evolution: just as animals adapt and respond to changes over time, the goal is to create a system that keeps making better versions of itself, with no set limit.
He is not alone. OpenAI has stated ambitions to build an automated AI research system by September 2026. Anthropic is publishing research on automated alignment researchers. DeepMind has said automation of alignment research "should be done when feasible." A startup called Mirendil has declared its goal as "building systems that excel at AI research and development."
Now, hundreds of billions of dollars are being invested in companies whose main goal is to automate the creation of smarter AI.
To understand why this is happening so fast, it helps to step back to where AI stood just 18 months ago and follow the thread.
Not long ago, AI coding tools were curiosities—useful for autocomplete, occasionally impressive for boilerplate, but not trusted with anything consequential.
That reputation has eroded. Two structural properties explain why software development became the first major field to feel the full weight of AI's productive capacity.
First, code is based on patterns. Most software development involves applying familiar structures to new situations, such as REST endpoints, authentication, or data checks. AI models trained on lots of open-source code have seen these patterns many times.
Second, software is easy to verify. Unlike legal briefs or medical diagnoses, code either works or it doesn’t. Tests pass or fail. This clear feedback lets AI check its own work without needing a human.
These two factors have led to a real jump in productivity. Some developers say they now finish in hours what used to take days. Small teams are launching products that once needed dozens of engineers.
1 min read
One reason things are moving so fast is that software engineers are driving the pace. If a developer finds a new AI technique on Monday, it’s in a blog post by Wednesday, and by Friday, several teams are already building tools around it. Within weeks, it becomes part of everyday work.
This feedback loop—discovery, sharing, adoption, and improvement—moves faster in software than in almost any other field.
This rapid pace is hard to manage. Jack Clark from Anthropic said the company wants lawmakers involved in recursive self-improvement before it becomes a crisis.
"As organizations, and eventually probably as societies, we need to figure out the tools to validate and verify that the stuff being done by these AI systems is correct and is aligned with human intentions," Clark told Axios.
All these changes add up to a clear trend.
In 2017, Google built an AI capable of designing other AIs. By late 2024, AI models were cloning themselves in controlled experiments. By mid-2026, a well-funded startup had declared recursive self-improvement its core product. And the company that runs some of the world's most capable AI systems is publishing blog posts urging governments to prepare for the consequences.
The question Anthropic raised in its warning is not whether this is possible. It is a question of whether our institutions will be ready when it arrives. "If systems are capable of fully building their successors," the post said, "then our methods for securing, monitoring, and shaping their behavior become far more important."
The company went even further, suggesting that slowing down or pausing advanced AI development across several major labs and countries might be worth considering.
This would give social institutions time to adapt. It’s a strong statement for a top AI company to make about its own industry.
It’s also the kind of warning that often gets overlooked right when it matters most.
From Google’s AutoML in 2017 to Anthropic’s warning about recursive self-improvement in 2026, the story has moved in a steady direction. AI has been building AI for years. Now, the scale, independence, and ambition of this process are outpacing our institutions' capacity to handle them.
None of this means disaster is coming soon. Humans are still involved for now. Regulated industries are moving slowly. The toughest parts of engineering still need the kind of judgment that takes years to develop, and that hasn’t been automated yet.
The uncomfortable part isn’t that AI might eventually build its own successor. It’s that no one in charge has had to answer what happens the day after it does.
So here’s the question that still has no answer: when does being "in the loop" stop meaning in control and start meaning just being there?
Set as preferred source
Set as preferred source
1 min read
