Can AI art generators be ‘accidentally’ racist? Seems like they sure can.
A gamer nicknamed Jacques Tamagotchi recently used an unspecified AI art generator, and asked it to produce a meme of Homer Simpson. This is no ordinary meme, though. To add a bit more flavour, Mr. Tamagotchi entered the prompt: “guy with swords pointed at him meme except they’re pointing the swords at Homer Simpson”.
Some context for those who are not extremely online: the ‘guy with swords pointed at him’ meme most likely refers to this widely-used image on social media. The meme originates from the Disney animated film Tangled (2010).
So, our friend Jacques Tamagotchi wanted a variation of the meme that featured Homer Simpson. However, what was generated by the AI was a black Homer Simpson with the name tag ‘Ethnically Ambigaus’.
“I have no idea where ‘Ethnically Ambigaus’ came from,” wrote Mr. Tamagotchi.
AI art generators feat. bias
AI art generators like Midjourney, Stable Diffusion, DALL-E and so on have long been documented to contain racial bias in their models. Thus, in an attempt to combat their tendency of only generating images of white individuals, programmers have added “prompt augmentation” efforts to de-bias these systems.
However, albeit well-intentioned, an unexpected side effect is the above ‘accident’ that backfires.
“It is a way of trying to patch a problem that is much deeper… if a dataset is unbiased to begin with, then you would not have these ‘cheap’ solutions to counter them,” explained Dr. Liam Magee, Principal Research Fellow at the Institute for Culture and Society at Western Sydney University.
“There are teams of engineers reviewing results before such tools are released to the public, but there is only so much they can look at.
“When named individuals are mentioned… the model has no internal knowledge that, for example, ‘Homer Simpson’ must always correspond with the colour and term ‘yellow’,” he added.
Alexa, hit shuffle
In some way, what AI art generators are currently doing is the equivalent of Spotify shuffling songs and landing on a track based on your music taste. It is not entirely randomised, but there is a certain underlying pattern or trend here that is difficult to pick apart.
“These [AI art generators] begin with a completely noisy image. If you watch the image generation process until an image emerges, you will see that the ‘randomness’ goes away. What takes place is the form of an object in an image,” Dr. Magee said.
This could help explain why, the Homer Simpson image is not made up of Homer’s face conjoined with an alpaca’s body and donkey ears, for example. The AI art tool has probably never watched an episode of The Simpsons and does not understand the cultural significance of the TV show, so it thinks that playing mix-and-match with Homer Simpson’s skin colour is okay!
Testing ‘Homer Simpson AI’ memes
We tested Tamagotchi’s prompts on Bing Image Generator. The results were also… strange. One of the four images generated was the iconic ‘Spider Man pointing fingers’ meme. But in between both Spider Men there’s an image of a brown person.
Next, we tested the same prompt on Wombo Dream, an AI art generator that launched an app where users can get multiple AI-generated photos of themselves by just uploading a single selfie. We ran the prompt thrice with two different filters: ‘Dreamland’, ‘Baroque’ and ‘Realistic’.
At least all three didn’t contain bizarre depictions of people of colour…
Other X users who replied under Tamagotchi’s post also made similar claims on the supposed ‘accidental bias’ of AI art generators. For example, one user claimed they entered a prompt to produce an image of “the 45th [US] president eating 100 Big Macs”. The result was an image of Donald Trump in dark skin munching on Maccas.
Next, another user shared an AI-generated image that they created of comedian Conan O’Brien and pop culture legend Guy Fieri. The image is of both celebrities smiling and having sushi at a Japanese restaurant – but Guy Fieri is, amusingly, yellow-skinned.
“The one time they randomly made Guy Fieri Asian,” the user joked.
Not all art generators
It’s important to note here, however, that this issue differs across AI models. For some like ChatGPT – in which DALL-E 3 is integrated into – generating an image of Homer Simpson is not permitted at all. In our correspondence, professor Liam Magee attempted to have ChatGPT reproduce Mr. Tamagotchi’s prompt, but his request was denied.
Midjourney, on the other hand, reproduces comic strips of Homer Simpson in various period outfits wielding a sword. Homer’s skin colour is in the original yellow.
“If the training set is biased, AI outputs will be biased too,” said Dr. Seyedali Mirjalili, Professor of AI at Torrens University Australia tells The Chainsaw.
“The goal is to ensure that the data used to train AI models is diverse and representative of different groups. This helps in reducing biases that might arise due to underrepresentation of certain groups in the data,” he added.
