Yahoo Finance Image-generating AI could be creating its own strange ‘secret language,’ research suggests
OpenAI’s text-to-image generating artificial intelligence (AI) Dalle-2 may be creating its own “hidden vocabulary”, according to a new study that sheds more light on how AI systems process information.
The yet-to-be peer-reviewed research, posted as a preprint on the platform arXiv, suggests that Dalle-2 may have its own strange “secret language”.
Dalle-2 made news earlier this year for its unique ability to combine text prompts and concepts into styled images.
But when Giannis Daras, a computer science PhD student at the University of Texas in Austin, asked the AI system to generate an image of “farmers talking about vegetables”, it generated text with the unknown word “vicootes”.
In a series of tweets on Tuesday, Mr Daras said that the strange text may not be as random as it appears.
When researchers fed the text “Vicootes” from the previous image to Dalle-2, the output was, surprisingly, an image of dishes with vegetables.
“We then feed the words: “Apoploe vesrreaitars” and we get birds. It seems that the farmers are talking about birds, messing with their vegetables!” he added.
“We discover that Dalle-2 seems to have a hidden vocabulary that can be used to generate images with absurd prompts,” researchers wrote in the yet-to-be-reviewed study, describing the new findings.
They suspect that the advanced image-generating AI may have generated a “hidden vocabulary” that could be working in parallel with its primary function.
A known limitation of DALLE-2 is that it struggles with text. For example, the prompt: "Two farmers talking about vegetables, with subtitles" gives an image that appears to have gibberish text on it.
However, the text is not as random as it initially appears... (2/n) pic.twitter.com/B3e5qVsTKu— Giannis Daras (@giannis_daras) May 31, 2022
The findings point to the many complexities of understanding the inner workings of AI systems, known as the black-box problem.
“Dalle-2 has a secret language,” Mr Daras said, adding that the “discovery of the Dalle-2 language creates many interesting security and interpretability challenges”.
However, researchers acknowledge that the study and its interpretations do have some limitations, adding that results to some of the prompts are not always consistent.
Citing an example, they said the prompt “contarra ccetnxniams luryca tanniounons” sometimes generates pictures of bugs, while at other times it generates images of “mostly animals”.
For the prompt “two whales talking about food, with subtitles,” the AI returned an image with the text “wa ch zod ahaakes rea” written on it.
Another example: "Two whales talking about food, with subtitles". We get an image with the text "Wa ch zod rea" written on it. Apparently, the whales are actually talking about their food in the DALLE-2 language. (4/n) pic.twitter.com/cqlUYXlLvf
— Giannis Daras (@giannis_daras) May 31, 2022
When the gibberish words “Wa ch zod rea,” were fed to the model, it displayed images of sea food.
“It seems that the gibberish text indeed has a meaning that is sometimes aligned with the text-conditioning that produced it,” researchers wrote.
However, not everyone seems convinced.
Research analyst Benjamin Hilton at London-based non-profit 80,000hours suspects the image results for the prompts could be random and not indicative of a hidden vocabulary.
“The key to claims of a Dalle ‘secret language’ is that these terms apply across Dalle prompts – including when used in more complex prompts, like asking Dalle to output in other styles. But if I add ‘3d render’ to the prompt, I get sea-related things, not bugs,” he explained in a series of tweets.
Mr Hilton said the researchers “got lucky” when the whales said “Wa ch zod rea”, and that “happened to generate pictures of food”.
“To me this is all starting to look a lot more like stochastic, random noise, than a secret Dalle language,” the analyst added.
Mr Hilton agreed that there’s something “definitely weird” that the gibberish prompt “apoploe vesrreaitais” gives bird images as the result every time “despite seeming nonsense”, adding that “there’s for sure something to this”.
But he said there’s not enough evidence there’s a “secret language” involved, or that the text in Dalle images “means anything.”
Reacting to Mr Hilton’s comment that the researchers got lucky with the whales example, Mr Daras said they could find another similar example to bolster the claims.
What do I think? "Evve waeles" is either nonsense, or a corruption of the word "whales". Giannis got lucky when his whales said "Wa ch zod rea" and that happened to generate pictures of food.
(13/15)— Benjamin Hilton (@benjamin_hilton) May 31, 2022
“Two men talking about soccer, with subtitles” gives the word ‘tiboer’. This seems to give sports in ~4/10 images,” he tweeted.
Researchers say the discovery of the Dalle-2 language also creates some security and interpretability challenges.
Currently, Natural Language Processing systems filter text prompts that violate the policy rules, but Mr Daras says gibberish prompts, such as those used in the study, may be used to “bypass these filters”.
“If a system behaves in an unpredictable way, even if that happens 1/10 times, that is still a massive security and interpretability issue, worth understanding,” Mr Daras explained.
However, researchers agree that “more foundational research is needed” to understand these phenomena, and to create robust language and image generation models “that behave as humans would expect”.
