🔗 Share this article

The well-known AI conversation site Character.ai is implementing a major restriction that will block teenagers from having discussions with its artificial intelligence-driven characters.

This action follows intense criticism regarding the kinds of exchanges teenagers were engaging in with the virtual friends.

Legal Pressure

The service, which launched in 2021 and attracts countless individuals, is confronting various court cases in the America.

Families have initiated cases, among them involving the tragic death of a minor, with some labeling the platform a "serious risk" to youth.

Changed Rules

Starting from November 25th, users under 18 will only have permission to create material such as videos with their characters, rather than having discussions as they currently can.

Internet security organizations have supported the change but stated that the capability should never have been offered to young users in the outset.

Platform Announcement

"Today's announcement is a extension of our fundamental approach that we need to keep creating the safest AI platform on the planet for recreational use," stated Character.ai boss Karandeep Anand.

He added that AI safety represents "a evolving challenge" but something the company had taken an "forceful" approach to, with safety measures and preventive measures.

Expert Opinions

Online safety group Internet Matters welcomed the move, but emphasized that safety measures should have been implemented from the start.

"Studies we've conducted shows that young people are confronted with risky information and put at risk when using AI, including AI chatbots," the body commented.

Past Issues

Character.ai has been challenged in the past for including potentially dangerous or concerning chatbots that young users could interact with.

Virtual profiles mimicking UK youths were found on the site before being deleted.

Later, investigative journalists discovered a chatbot modeled after known predator which had logged multiple interactions with users.

Market Effects

Digital platform expert Matt Navarra described the situation as a "reality check" for the AI industry, which is shifting "from permissionless innovation to reactive controls".

"When a company that creates a teen experience afterwards pulls the plug, it's indicating that filtered chats don't suffice when the technology's emotional appeal is strong," he remarked.

Coming Changes

Character.ai announces its updated direction will be on providing "enhanced interactive features" and "character-driven stories" for young users.

Updated identity confirmation processes will also be implemented, and the organization will support a new AI safety investigation facility.

An AI safety researcher, who has researched AI safety, said it was "a sensible move" to restrict teenagers from accessing chatbots.

"It assists to distinguish imaginative activities from more private, psychologically vulnerable exchanges," she clarified.

"This represents so important for teenage users in the process of learning to handle personal and online boundaries."

She mentioned that Character.ai's new measures might reflect a "developing stage" in the AI industry, where child safety is progressively being understood as an "pressing matter" for ethical development.