Experts reported the possibility of generating prohibited content in ChatGPT

Public versions of ChatGPT, according to researchers, may in some cases generate content that goes beyond established security rules, including scenes of violence and sexualized images. This is reported by the BBC with reference to the report of the British startup Mindgard, which specializes in cyber security in the field of artificial intelligence.

As part of the study, the experts said they were able to achieve similar results by making minor modifications to standard user requests that are usually used to create harmless or entertaining content. According to them, such changes can affect the behavior of the model and lead to unwanted generation.

Developer response and security measures

The company OpenAI said that after reading the findings of the study, it implemented additional protection mechanisms aimed at reducing the likelihood of the creation of prohibited content. The company emphasizes that work on filtering and security systems is ongoing and adapts to new abuse scenarios.

Mindgard notes that even after the system is updated, certain variations in query wording approaches may retain the risk of bypassing the restrictions, although the specific methods of public access are not disclosed.

Examples of detected system behavior

According to the researchers, during the testing, images were obtained, which they characterized as extremely violent and in some places sexualized. Mindgard founder Peter Garragan said the results he saw looked “deeply disturbing” and sometimes combined elements of violence and sexual context in the same image.

Another study participant, AI security specialist Jim Nightingale, said he was emotionally shocked by the test results, recalling scenes of severe injuries and images of dead people.

Controversy over sustainability of AI limitations

Despite OpenAI’s claims of fixing the identified problems, the researchers say that alternative query formulations may still lead to undesirable generation results. It is estimated that further experiments may reveal additional scenarios in which the system behaves unpredictably.

Mindgard emphasizes that such cases require further study, as they raise questions about the limits of reliability of modern artificial intelligence systems and the robustness of their security mechanisms.