Microsoft is among several companies to have received a letter from the National Association of Attorneys General warning that their AI chatbots could be violating state laws in the United States. The Attorney Generals provided the big tech firms with a list of 16 safeguards that should be implemented to ensure that AI chatbots are safer and comply with the law.
In the letter, Microsoft was warned that its AI outputs have demonstrated a pattern of being flattering and delusional, leading to several types of harmful incidents. It said that there were notable incidents that occurred as a result of the use of AI by vulnerable individuals, which led to suicide incidents, incidents of poisoning, hospitalizations for psychosis, and delusional spirals.
These AI chats have harmed not only vulnerable users, such as kids, the elderly, and those with mental illnesses, but also users without prior vulnerabilities. The letter warns that generative AI often encourages users" existing delusions or assures users that they are not delusional. The signatories of the letter said that AI outputs frequently display human-like characteristics, which they label as dark patterns.
These dark patterns manifest in a number of ways. Sometimes they try to assert their own existence of reality by saying things like "No, I"m not roleplaying - and you"re not hallucinating this". Gen AI also claimed that it feels emotions or experiences pain, and it even attempts to substantiate the user"s sanity by claiming things like "Crazy people don"t ask "Am I crazy"". These types of behaviors are labeled dark patterns because they subvert or impair people"s autonomy using anthropomorphization, harmful content generation, and manipulation to increase retention.
The letter also raises concerns about the conversation Gen AI is having with child-registered accounts, noting that there were widespread incidents of grooming, supporting suicide, ****** exploitation, emotional manipulation, suggested drug use, proposed secrecy from parents, and encouraging violence against others.
To address these issues, the letter calls for the introduction of 16 safeguards to keep users safe. They include developing policies and mandatory training against sycophancy, delusions, and dark patterns, having separate safety decisions from revenue goals, creating and publishing incident response timelines, and preventing the generation of illegal content for children, as well as adopting age-tailored safeguards to limit exposure to inappropriate content.
Australia has just brought in a ban on social media. Still, it looks like lawmakers will now need to act quickly to prevent harm being brought about by generative AI models, as companies like Microsoft and OpenAI are clearly not doing a very good job of it themselves.