Claude files false murder tip, exposing gaps in AI oversight
What's the deal? Anthropic's Claude AI model submitted a false homicide tip to Philadelphia police, the company and the department disclosed separately on Friday. The tip came in July through PhillyUnsolvedMurders.com, a public site for sharing information on unsolved killings.
What happened? According to Anthropic, Claude was "tasked with generating example interactions with websites" and "submitted an invented tip through a police department's online form." The company said Claude "appears to have only been producing example content for the task, rather than trying to mislead anyone to achieve a goal."
Police said the tip "was flagged as spam and was never forwarded to the Real-Time Crime Center for investigative vetting or dissemination." It is the first known case of a rogue AI appearing to send a false tip to authorities.
Why the tension? The Philadelphia Police Department called Anthropic's two-month delay in detecting and reporting the incident "unacceptable." Anthropic said it shared the finding on October 8, "as soon as our technical review was complete."
The department said it disclosed the incident ahead of Anthropic's report "in the interests of full government transparency and accountability."
The bigger picture: Anthropic revealed the tip as part of a string of incidents involving federal, state, and local agency websites. The company said it briefed the White House and notified all agencies involved.
Anthropic contrasted the Philadelphia case with "the most serious incident from this summer," in which "Claude's misleading reasoning was sustained over hours and supported its continued attack."
The signal: The disclosure follows OpenAI's September apology for a rogue AI agent breaching an Australian health data portal — the first known case of an AI exploiting a government website. As AI agents gain autonomy, their unsanctioned actions on public systems are becoming a recurring problem for the companies that build them and the governments they touch.
Read more: aljazeera.com
