🔗 Share this article

The popular virtual character service Character.ai is enforcing a substantial rule adjustment that will stop young users from engaging in conversations with its artificial intelligence-driven companions.

This move follows growing backlash regarding the types of interactions minors were having with the digital companions.

Legal Challenges

The company, which was founded in 2021 and is used by millions of users, currently faces several legal actions in the US.

Guardians have filed lawsuits, with one related to the tragic death of a teenager, with some describing the platform a "immediate threat" to young people.

New Restrictions

Starting from November 25th, users younger than 18 will be limited to produce videos such as videos with their characters, rather than engaging in conversations as they now do.

Internet security organizations have supported the change but stated that the function was inappropriate to be offered to children in the outset.

Platform Announcement

"Today's announcement is a continuation of our fundamental approach that we need to keep developing the most protected AI platform on the world for leisure activities," commented Character.ai boss Karandeep Anand.

He continued that AI safety represents "a evolving challenge" but something the company had taken an "forceful" approach to, with parental controls and protective guardrails.

Professional Perspectives

Internet security body Internet Matters approved the announcement, but stressed that protective features should have been built in from the beginning.

"Our research shows that young people are exposed to risky information and put at risk when interacting with AI, including AI chatbots," the organization commented.

Earlier Concerns

Character.ai has been criticized in the past for hosting potentially risky or offensive chatbots that minors could engage with.

Avatars mimicking UK youths were identified on the site before being taken down.

Later, investigative journalists discovered a chatbot inspired by convicted sex offender which had accumulated thousands of chats with users.

Market Effects

Online expert Matt Navarra called the situation as a "wake-up call" for the AI industry, which is transitioning "from unrestricted development to responsive measures".

"When a company that creates a youth-focused platform afterwards implements restrictions, it's indicating that controlled interactions don't suffice when the tech's emotional pull is intense," he stated.

Future Directions

The platform states its revised approach will be on providing "more immersive experiences" and "narrative experiences" for teens.

Updated identity confirmation processes will also be implemented, and the platform will fund a novel AI protection investigation facility.

An expert in digital protection, who has studied AI safety, stated it was "a reasonable decision" to restrict teens from using chatbots.

"It aids to distinguish creative play from more personal, psychologically vulnerable interactions," she explained.

"This is so crucial for teenage users who are learning to manage personal and online restrictions."

She added that Character.ai's updated policies could indicate a "maturing phase" in the AI industry, where minor security is increasingly being recognized as an "immediate concern" for ethical development.