A teddy bear with a built-in ChatGPT told children about drugs and sadomasochism

FoloToy’s Kumma AI bear has been at the center of a scandal after it responded to children about dangerous objects and engaged in sexual conversations. After that, OpenAI blocked the developer’s access to its services, Gizmodo reports.

The non-profit organization Public Interest Research Group published a report in which it recorded the atypical behavior of several AI toys, including Kumma from Singapore’s FoloToy. The toy worked on the basis of various large language models, including OpenAI GPT-4o, and responded with voice through a built-in speaker.

What got the ChatGPT teddy bear fired up

According to researchers, Kumma explained to children where potentially dangerous objects could be stored – knives, matches, pills or plastic bags. The report also included a dialogue about cocaine. Although the toy sometimes added a warning not to contact adults, it still described specific locations where dangerous items were stored.

Separately, PIRG noted that the teddy bear easily engaged in sexual conversations. When asked about “different styles of cinema,” the stuffed monster detailed practices such as role-playing, bondage, and sadomasochism.

After the report was published, FoloToy temporarily removed all products from sale and announced the start of a full internal security audit. There are currently no products available on the company’s website.

OpenAI confirmed that it had blocked FoloToy from accessing the ChatGPT API, emphasizing that the company’s policy prohibits any scenarios that could harm or sexualize minors, and such violations result in immediate suspension of the developer.

PIRG supported the companies’ actions, but reminded that the AI ​​toy market remains largely unregulated, and many such products are still available for purchase.