An AI-generated image of Pope Francis wearing a stylish white puffer jacket has been circulating on social media, tricking many users into believing it is real. The image, which shows the Pope holding what appears to be a coffee cup in one hand and wearing a bejeweled crucifix, was generated by an artificial intelligence program called Midjourney. Founded by David Holz, who also co-founded Leap Motion, Midjourney uses AI to create images based on textual descriptions provided by users, similar to OpenAI’s DALL-E.

While the image of the Pope looks realistic at first, there are telltale signs that it is fake, according to AI expert Henry Ajder. The hand holding the coffee cup appears to be deformed, and the pope’s glasses are not rendered properly, creating inconsistencies in the shadowing on his face. Ajder warns that Midjourney’s progress since it was founded has been “startling,” and that the platform can create very convincing fake news reports and images of politicians.

Ajder has worked with Partnership in AI, a non-profit organization promoting responsible AI use, to set out guidelines for people using AI tools as well as creators of the tool. This includes building a direct or indirect disclosure mechanism so that users are alerted when they are engaging with an AI-generated piece of content. This could include a watermark or a pre-rolled disclaimer for videos.

In the competitive arms race environment that exists in the AI industry, concerns around safety or ethics may be put to the side. It is important for users and creators to be aware of the potential consequences of AI-generated content and to act responsibly in its use.