For many people, AI chatbots are simply tools for accessing information and researching various topics. However, these AI tools also have a strong capacity to influence human emotions, which could potentially lead to serious consequences, such as self-harm or suicide. As a result, Chinese regulators have now moved to restrict AI chatbots’ ability to affect human emotions.
The Cyberspace Administration of China has proposed new legislation to prevent AI chatbots from exerting a significant influence on users’ emotions through text, images, audio, or video. The bill targets “human-like interactive AI services” available within the territory of China, where using them could potentially encourage users to commit suicide or self-harm.
The bill proposes that AI chatbots should not generate content that encourages suicide or self-harm, or engage in verbal abuse or emotional manipulation. In addition, these tools should not produce gambling-related or violent content. Minors are also required to have guardian consent to use AI for emotional companionship.
But one of the most unusual proposed measures is that companies behind AI chatbots must have a human intervene in the conversation and immediately contact the user’s guardian or a designated individual if a user explicitly expresses suicidal intent. Considering the large number of users of AI chatbots in China and the possibility of errors in any system, this requirement may be difficult to implement.
The bill also includes several other requirements, including that tech providers must remind users after 2 hours of continuous interaction with AI, and that platforms must be able to determine whether a user is a minor.
China is among the first countries in the world to regulate emotional interactions between AI tools and humans. As more and more users develop deep emotional bonds with chatbots, and these tools even begin to replace therapists, this is a move with good intentions. However, its implementation may face significant challenges.
Via: CNBC