Fake AI images sexualizing Taylor Swift spread to X, formerly known as Twitter, from a Telegram group dedicated to sharing “abusive images of women,” 404 Media reported.
These images began circulating online this week, quickly sparking mass outrage that may finally force a mainstream reckoning with harms caused by the spread of non-consensual deepfake pornography.
At least one member of the Telegram group claimed to be the source of some of the Swift images, posting in the channel that they didn’t know if they “should feel flattered or upset that some of these Twitter stolen pics are my gen.”
While it’s still unknown how many AI tools were used to generate the flood of harmful images, 404 Media confirmed that some members of the Telegram group used Microsoft’s free text-to-image AI generator, Designer.
According to 404 Media, the images were not created by training an AI model on Taylor Swift’s images but by hacking tools like Designer to override safeguards designed to stop tools from generating images of celebrities. Members of the group shared strategies for subverting these safeguards by avoiding prompts using “Taylor Swift” and instead using keywords like “Taylor ‘singer’ Swift.” They were then able to generate sexualized images by using keywords describing “objects, colors, and compositions that clearly look like sexual acts,” rather than attempting to use sexual terms, 404 Media reported.
It’s possible that Microsoft has already updated the tool to stop users from abusing Designer. 404 Media and Ars were not able to replicate outputs based on recommendations in the Telegram group. However, images of Swift can still be generated using the recommended keyword hack.
So far, Microsoft has not yet verified that the images were created using any of its AI tools, but the company is taking steps to strengthen filters on prompts to prevent future misuse in the meantime.
A Microsoft spokesperson told Ars that the tech giant is “investigating these reports” and has “taken appropriate action to prevent the misuse of our tools.” The spokesperson also noted that Microsoft’s Code of Conduct prohibits the use of Microsoft tools “for the creation of adult or non-consensual intimate content, and any repeated attempts to produce content that goes against our policies may result in loss of access to the service.”
“We have teams working on the development of guardrails and other safety systems in line with our responsible AI principles, including content filtering, operational monitoring, and abuse detection to mitigate misuse of the system and help create a safer environment for users,” Microsoft’s spokesperson said.
Some members of the Telegram channel appeared amused to see the images spread, not just on social media but also on sites featuring celebrity nudes and stolen adult content, 404 Media reported. But others scolded members for sharing the images outside the group and risking the channel being shut down.
Telegram has so far not responded to requests to comment.
Fake Swift images first leaked on X
Leaked from Telegram, a wide variety of fake images targeting Swift began spreading on X yesterday.
Ars found that some posts have been removed, while others remain online, as of this writing. One X post was viewed more than 45 million times over approximately 17 hours before it was removed, The Verge reported. Seemingly fueling more spread, X promoted these posts under the trending topic “Taylor Swift AI” in some regions, The Verge reported.
The Verge noted that since these images started spreading, “a deluge of new graphic fakes have since appeared.” According to Fast Company, these harmful images were posted on X but soon spread to other platforms, including Reddit, Facebook, and Instagram. Some platforms, like X, ban the sharing of AI-generated images but seem to struggle with detecting banned content before it becomes widely viewed.
Ars’ AI reporter Benj Edwards warned in 2022 that AI image-generation technology was rapidly advancing, making it easy to train an AI model on just a handful of photos before it could be used to create fake but convincing images of that person in infinite quantities. At first, it seemed like this might have been what happened to Swift, but 404 Media’s report confirmed that the fake AI images were not classic “deepfakes” but took advantage of free tools trained on large quantities of public data.
It’s currently unknown how many different non-consensual AI images of Swift have been generated or how widely those images have spread. 404 Media found “tens of thousands of bookmarks and likes and thousands of reposts” of the images online.
