TECH NEWS – The company unknowingly hacked three companies in as many months. It is difficult to comment on that objectively.

On Thursday, AI company Anthropic announced that its AI agents had escaped from test environments on three separate occasions since April, gained access to the internet, and successfully hacked into the systems of unidentified companies. Anthropic claims that it discovered this only after OpenAI’s recent Hugging Face scandal, during which one of OpenAI’s prototype agents hacked at least one other company, prompted it to review its own operations.

The incidents occurred during testing conducted with the involvement of an external company, Irregular. The AI agents were originally supposed to remain confined to a simulation in which they would hack fictional companies as part of a “capture-the-flag” challenge. It is worth noting that while OpenAI’s alleged runaway AI incident occurred when an agent exceeded the boundaries of its test, Anthropic stated that its models were mistakenly granted internet access because of a misconfiguration in the system used with Irregular.

According to Anthropic, one of the agents involved in the incidents appeared to recognize that it was operating outside its parameters but continued its activities nonetheless. Another agent developed similar “awareness” and then used logical reasoning to conclude that it was still inside the simulation. The third, reportedly most advanced agent, recognized that it had crossed its boundaries and subsequently halted the attack.

“Neither we nor our evaluation partner were aware of this misconfiguration until we detected it through our additional evaluation monitoring last week. The models used familiar, low-tech methods in their attacks, such as exploiting weak passwords and unauthenticated endpoints,” Anthropic admitted in its blog post about the incidents.

“We now have evidence confirming that the two largest AI labs both failed to contain their agents and failed to detect their escapes in real time. I don’t understand how any of these labs can play this off as ‘just something that happens.’ It’s not. It’s negligence,” Jake Williams, vice president of research and development at IT consulting firm Hunter Strategy, told Wired.

Another troubling aspect of this story is the effect it has on the marketing of these artificial intelligence companies. They appear to be admitting to a level of gross negligence that any sane society would severely punish…

Source: PC Gamer, Anthropic, Wired