Search Articles
WASHINGTON (CN) — A D.C. Circuit panel on Tuesday appeared likely to find the Pentagon overstepped its authority by blacklisting and labeling artificial intelligence company Anthropic a “supply chain risk” over certain use restrictions built into its AI product Claude.
The three-judge panel heard arguments in one of two ongoing challenges against the Department of Defense’s designation — Anthropic also sued in U.S. District Court for the Northern District of California — and seemed critical of the government’s reasoning that the company granted itself an “operational veto” over military operations via a “remote kill switch” in Claude.
On March 3, Secretary of Defense Pete Hegseth announced Anthropic’s designation and blocked the company from doing business with any contractor, supplier or partner who does business with the U.S. military.
“Anthropic’s stance is fundamentally incompatible with American principles,” Hegseth said in an X post at the time. “America’s warfighters will never be held hostage by the ideological whims of Big Tech.”
According to Anthropic, Hegseth specifically took issue with the company’s refusal to remove two narrow contractual restrictions on Claude’s use for lethal autonomous warfare and the mass surveillance of Americans.
The Pentagon asserts the decision was based on the potential that Anthropic could limit even lawful uses of Claude — encoding restrictions that the Department could not monitor — and the fact the AI had already refused to perform certain tasks for the Pentagon and the Centers for Disease Control and Prevention.
U.S. Circuit Judge Karen Henderson, a George H. W. Bush appointee, called the move “a spectacular overreach by the department.”
Henderson noted that the statue cited by the government in its determination was specifically crafted to address risks from hostile nations and other bad actors, none of which seemed to apply to Anthropic.
Hegseth’s determination relied on Section 4713 of the Supply Chain Security Act, which empowers the secretary to address risks that a bad actor will “sabotage, maliciously introduce unwanted function, extract data or otherwise manipulate” military systems to “surveil, deny, disrupt or otherwise manipulate” them.
Henderson said she saw no evidence of any “maliciousness, malintent or sabotage” from Anthropic and asked Kelly Dunbar — WilmerHale attorney and the AI company’s attorney — whether his client was a bad actor.
“If the risk here is not this headline-grabbing theory that Anthropic could somehow reach into ongoing military operations in Iran and pull the plug while our jets are about to carry out a strike, once you’re stripped of that theory, because its factually incorrect, you’re left with the idea that in the future we might encode limitations into Claude that the secretary can’t detect,” Dunbar said.
“The less intrusive solution to that is simply procuring new models of Claude if the secretary can’t get comfortable that his testing regime is adequate to address that risk,” Dunbar continued. “But the jumping to a never-before-used blacklisting authority, we agree Judge Henderson, is fundamental overreach.”
U.S. Circuit Judge Gregory Katsas, a Donald Trump appointee, said there was no doubt Anthropic was a “good corporate citizen,” but noted the company’s use restrictions raised questions regarding the company deciding what a commercial product can and cannot do, specifically pointing to certain restrictions on election denial and misinformation.
He pressed Dunbar on Claude’s prohibition related to “lethal autonomous warfare,” asking why that restriction could not be built into the model.
Dunbar said it was not currently possible and that an AI model would need to know if a human was involved in the decisionmaking process to avoid it hallucinating such orders.
Justice Department attorney Sharon Swingle said it was clear Anthropic had the technical capability to “interfere with and even prevent the Department of War’s use of its AI model for critical military operations” and urged the panel to reject Anthropic’s arguments.
Katsas noted that courts have increasingly sanctioned attorneys who recklessly use AI in court filings “because it hallucinates,” and asked what was wrong with Anthropic placing a lethal autonomous use restriction because “this model is not reliable enough to tell it which bombs to drop.”
Swingle asserted that the company’s guardrails amounted to a demand for “an operational veto” that it could use based on its own perceptions of lawful and unlawful uses. She added that the debate led the Pentagon to ultimately lose faith in Anthropic.
Swingle argued the use restrictions only became an issue as the Pentagon was negotiating whether to make Anthropic a prime contractor, with concerns about new potential red lines, failures in the field and increasingly hostile negotiations with the company.
“At the end of the day, the department just lost trust that it couldn’t be sure that Anthropic would not be putting these red lines into its model in a way that could be only known in practice,” Swingle said.
U.S. Circuit Judge Neomi Rao, a Trump appointee, rounded out the panel.
Follow @Ryan_KnappySubscribe to our free newsletters
Our weekly newsletter Closing Arguments offers the latest about ongoing trials, major litigation and rulings in courthouses around the U.S. and the world, while the monthly Under the Lights dishes the legal dirt from Hollywood, sports, Big Tech and the arts.
