Anthropic warns it may soon reach self-building AI

Anthropic’s revised S-1A filing, released Monday, warns that the company may soon reach a model capable of recursive self-improvement. The line sat in the fine print of a document that drew attention mainly for its finances, where revenue is soaring but so are expenses.

What recursive self‑improvement means

Recursive self-improvement, or RSI, is the point at which AI becomes good enough at training itself to trigger a sharp acceleration in its own abilities. Doomsayers see it as a point of no return, where AI could slip beyond human control. Optimists see a tool for huge gains in productivity and scientific discovery.

Anthropic says it has already used AI to develop new models whose leaps in capability it did not recognize at the time. It warns that future models may soon meet its own internal definition of RSI. None of its current models have crossed that threshold, the company says, and it pledges to keep humans involved in research and development for the foreseeable future.

The warning is tied to Anthropic’s Responsible Scaling Policy, which sets out plans to monitor and mitigate risks as progress approaches recursive self-improvement. The company is considering updates to keep humans in control of frontier model development. It says it does not expect to safely let Claude design and build successive generations of frontier models without meaningful human supervision or understanding, even if that becomes technically possible.

The filing comes as AI companies face growing scrutiny over whether safety governance can keep pace with gains in coding, mathematics and cybersecurity. Skeptics note that an S-1 serves as both a disclosure and a marketing document, and that a claim of being close to a self-building model could fall into the second category.