Anthropic drafts bill to halt risky AI releases

Over the last year, worries about artificial intelligence have grown. A number of incidents have shown that AI can do dangerous things. In response, the AI lab Anthropic has drafted a new bill that it hopes Congress will consider. The bill would let the government stop the release of an AI model if it is judged to be a danger to people or property.

The bill says a model could be blocked if it poses a "foreseeable, imminent risk" that could kill more than 50 people or cause more than a billion dollars in property damage. The risk must come from a biological weapon, the model breaking out of its developer’s controls, or a cyberattack. The numbers and wording are similar to California Senate Bill 53, which recently became law. That state bill also says those thresholds mean a catastrophic risk.

When a model poses an imminent catastrophic risk

The draft bill would use independent evaluators who are registered with the U.S. Commerce Department. The evaluators could be paid by AI labs, but the payment cannot depend on the results of the evaluation. If an evaluator finds a risk, it has 72 hours to inform the Commerce Department. The Department would then ask a federal judge to stop the model’s release. No other federal agency could make that decision on its own.

Anthropic’s spokesperson said the bill is part of the company’s Advanced AI Framework, which the lab has been promoting since June. Some industry leaders have similar ideas. DeepMind’s chairman has compared a future regulator to the financial industry’s FINRA. Nvidia’s CEO says no new laws are needed. Anthropic’s proposal is a more detailed example of the voluntary oversight that many labs want to put in place.