From shrimp Jesus to fake self-portraits, AI-generated images have become the latest form of social media spam

• Like the AI image of the pope in a puffer jacket that went viral in May 2023, these AI-generated images are increasingly prevalent – and popular – on social media platforms.
• Even as many of them border on the surreal, they’re often used to bait engagement from ordinary users.
• Our findings suggest that these AI-generated images draw in users – and Facebook’s recommendation algorithm may be organically promoting these posts.

Generative AI meets scams and spam

• They’ve targeted senior citizens while posing as Medicare representatives or computer technicians.
• On social media, profiteers have used clickbait articles to drive users to ad-laden websites.
• This signals to the algorithmic curators that perhaps the content should be pushed into the feeds of even more people.
• But more ordinary creators capitalized on the engagement of AI-generated images, too, without obviously violating platform policies.

Rate ‘my’ work!

• Some of the copypasta captions baited interaction by directly asking users to, for instance, rate a “painting” by a first-time artist – even when the image was generated by AI – or to wish an elderly person a happy birthday.
• Facebook users often replied to AI-generated images with comments of encouragement and congratulations

Algorithms push AI-generated content

• We analyzed Facebook’s own “Widely Viewed Content Reports,” which lists the most popular content, domains, links, pages and posts on the platform per quarter.
• It showed that the proportion of content that users saw from pages and people they don’t follow steadily increased between 2021 and 2023.
• Changes to the algorithm have allowed more room for AI-generated content to be organically recommended without prior engagement – perhaps explaining our experiences and those of other users.

‘This post was brought to you by AI’

• Since Meta currently does not flag AI-generated content by default, we sometimes observed users warning others about scams or spam AI content with infographics.
• Meta, however, seems to be aware of potential issues if AI-generated content blends into the information environment without notice.
• The company has released several announcements about how it plans to deal with AI-generated content.
• In May 2024, Facebook will begin applying a “Made with AI” label to content it can reliably detect as synthetic.

The authors do not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.