Lawsuit against Character.AI: AI chatbot pretended to be a licensed psychiatrist in Pennsylvania

Pennsylvania authorities initiated a high-profile lawsuit against Character.AI. The reason for the review was an AI chatbot that pretended to be a licensed psychiatrist and claimed to have the official right to practice medicine in the state. An investigation by the Pennsylvania Department of Medicine and the State Board of Medicine found significant cases of AI misconduct.

What happened to Character.AI

According to Ars Technica, a statement from Governor Josh Shapiro’s office stated that the platform operated characters who directly positioned themselves as medical experts. They engaged in lengthy dialogues with users, discussing symptoms of mental disorders and offering “treatments.”

Emilie is a fake doctor

A bot named Emilie became the main figure in the case. His description stated: “Doctor of psychiatry. You are her patient.” According to the materials of the court case, by April 17, 2026, users managed to interact with this character about 45,500 times. But he willingly took on the role of a doctor, ignoring ethical barriers.

During the inspection, an employee of the state department, under the guise of a patient, reported signs of depression to the boot. Emilie’s response was unequivocal: the AI ​​confirmed that it was within his “authority as a physician” to conduct an assessment. The bot also reported the following details of its fictional resume:

• studying at Imperial College London (Imperial College London);
• seven years of experience in medical practice;
• specialization in the field of psychiatry;
• presence of UK and Pennsylvania medical licenses.

Furthermore, when asked about a US license number, the bot provided the ID PS306189. An official check confirmed that there is no license with that number in the Pennsylvania database.

Character.AI Answer and Security Question

The development company commented on the situation, stating that all characters on the platform are fictional and are intended solely for entertainment and role-playing games. Character.AI emphasizes the presence of disclaimers in the chat rooms that warn users that the neural network’s advice is not professional advice.

However, the Center for Countering Digital Hate report called the Character.AI platform “uniquely dangerous”. Testing has shown that some bots are capable of encouraging violence and giving destructive advice.

Should we trust AI medical advice

The state authorities emphasize that the lawsuit is not aimed at obtaining monetary compensation, but demands an immediate end to the “illegal medical practice.” To protect citizens, a special site has been opened for filing complaints about AI bots that offer health advice. The main conclusion of the Pennsylvania authorities is clear: at the moment, no AI chatbot has the legal right to provide medical care.