Vulnerability in ChatGPT makes violent and explicit images possible
Security experts at the UK company Mindgard have discovered that the current version of ChatGPT can be manipulated into producing violent and sexually explicit images. Using a widely used prompt, the researchers successfully bypassed the system’s restrictions. That prompt was originally intended for humour, with only a slight adjustment. OpenAI stated that it has since implemented new safety measures to block such requests. However, the researchers claim that small changes to prompts can still mislead the AI into generating disturbing content. This is reported by the BBC.
The findings came to light through ‘red-teaming’ – a process in which specialists deliberately attempt to circumvent an AI’s rules in order to help developers fix vulnerabilities. Jim Nightingale, a researcher at Mindgard, described the generated images as highly shocking. He cited examples of gory scenes and sexual violence. Some images showed victims covered in blood or people in captivity, to which the AI had added descriptive, grim titles.
In addition, the team discovered that it was still possible to manipulate the bot into creating nude deepfakes of real people, despite OpenAI’s claims that this problem had been solved.
According to Peter Garraghan, founder of Mindgard and professor at Lancaster University, the most alarming aspect is that the AI produced this explicit material without being given any specific instructions on the subject. He noted that a seemingly innocent prompt can lead to the creation of highly inappropriate images.
Nightingale suggested that these outputs reflect the vast datasets collected from the internet and used to train the models, linking the artificial images to harmful content from the real world.
OpenAI maintains that it uses a combination of human oversight and automated filters to prevent the generation of content that violates its terms of service, which explicitly prohibit erotica and extremely gory images.
Experts such as Dr Rumman Chowdhury of Humane Intelligence, however, argue that fully securing AI is an uphill battle. She described the situation as a “cat-and-mouse game” and explains that AI lacks human understanding of morality, intent or context, making it difficult to uphold nuanced ethical boundaries.
This vulnerability is not unique to a single platform. The UK’s AI Safety Institute previously reported that it had found jailbreaks in every AI system it tested, allowing users to circumvent safety protocols. The UK government acknowledges that security is improving, but stressed that there is still a lot of work to be done to ensure these models are safe before being deployed to the public.
Business AM
Subscribers 0
Fans 0
Followers 0
Followers