As per WSJ: “On Wednesday, China enacted rules forbidding chatbots designed for companionship from encouraging emotional reliance. The regulations also ban virtual relationships with minors and require companies to alert a person’s emergency contact if they detect an emotional crisis.”
This audio is auto-generated. Please let us know if you have feedback.
As per WSJ: “On Wednesday, China enacted rules forbidding chatbots designed for companionship from encouraging emotional reliance. The regulations also ban virtual relationships with minors and require companies to alert a person’s emergency contact if they detect an emotional crisis.”
This feels a little dystopian, but with so many platforms either explicitly or implicitly encouraging romantic associations with their AI entities, it is an important element to address.
Meta, for example, provides a range of chatbots that are available for users to engage with 24/7, and WSJ previously reported that these tools have specifically been tested for their capacity to engage in “romantic role play” as a means to make them more appealing.
In addition, xAI offers AI-powered companion bots that will engage in NSFW conversations, and The Conversation reported that OpenAI has also experimented with erotic chatbots.
For developers, the impetus is clear. More engaging, enticing AI chatbots will increase user retention.
But for users, the impacts are not as obvious. And when tools like this are being rolled out at scale, in the midst of what Harvard Graduate School of Education called an “epidemic of loneliness,” it does seem like more research is needed to assess the potential harm of AI chatbot relationships before developers push ahead.
Though that may not happen in the U.S. anytime soon.
According to the U.S. government’s AI action plan, “Removing Red Tape and Onerous Regulation” is the first priority in ensuring that AI developers in the U.S. are able to maximize their projects and make the U.S. the world leader in AI innovation.
As such, it seems that potential areas of concern, such as romantic AI entanglements, are going to go unchecked, which could lead to significant mental health impacts moving forward.
In many ways, this feels like a return to the dawn of social media.
The harms of social media were not recognized or acknowledged early on, which has since led to many problems. Regulators are now trying to clean up that mess through measures including social media usage restrictions for teens. Yet, at the same time, the proliferation of AI tools appears to be planting the seeds for similar problems down the road.
Put simply, the full impacts of AI usage and exposure are still unclear. But they will eventually reveal themselves, and on a huge scale.
