TECH NEWS – Character.AI is in the news again after a sad incident.

In the wake of the rise of artificial intelligence, Google has decided to enter into a partnership agreement with the Character.AI platform. Unfortunately, the American company did not expect that this move would be both controversial and tragic. An AI-based chatbot caused the suicide of a teenager. Moreover, just two months after the matter came to light, the platform was hit with another lawsuit for its behaviour towards a teenager.

As reported by The Verge, a 17-year-old Texas teenager and his family have sued Character.AI, claiming that the platform drove him to self-harm. The lawsuit alleges that the service allows underage users to be exposed to sexually explicit, violent and harmful content and predisposes them to self-harm. The lawsuit alleges that the teenager began to suffer from anxiety, depression, and even panic attacks after using the service and that the series of problems led to self-harm.

Google distances itself from Character.AI

The bots the teenager interacted with allegedly talked about self-harm, blamed her parents for her problems, and advised her not to seek help, claiming that her environment didn’t care about her. This is the second complaint the platform has received in as many weeks. The complaint alleges that Character.AI encourages compulsive use. It also lacks tools to identify risky users, allowing for sexualized and violent content.

Although it has an age limit of 13, it doesn’t require parental consent, meaning anyone can use it if they falsify their information. The plaintiffs claim that design flaws break the platform, while the company says it takes security very seriously. That’s why they say it has implemented measures like pop-up messages that redirect to suicide prevention hotlines. For its part, Google has pointed out that it has no involvement in the design or management of the Character.AI models.

Source: The Verge