ChatGPT's New Image Generator Sparks Concerns Over Fake Receipts

This month, ChatGPT introduced a new image generator as part of its 4o model, which has demonstrated significant improvements in generating text within images. However, its capabilities have already raised concerns about potential misuse.

Users are leveraging the tool to create fake restaurant receipts, adding to the growing arsenal of AI-powered deepfakes used for fraudulent purposes. Venture capitalist and frequent social media user Deedy Das shared an image on X of a fabricated receipt for a real San Francisco steakhouse, claiming it was generated using ChatGPT 4o. Others soon replicated similar results, with some even adding simulated food or drink stains to enhance authenticity.

Despite these convincing attempts, some signs still reveal the receipts as fake. In one case, the total amount displayed a comma instead of a period, and the calculation was incorrect — a common issue with large language models that struggle with basic math. While these errors might seem obvious, a determined fraudster could easily make minor adjustments using photo editing software or refined prompts.

The accessibility of tools for generating fake receipts has heightened fears of potential misuse. Fraudsters could use these convincing replicas to submit false expense reports, seeking reimbursements for nonexistent purchases.

In response to these concerns, OpenAI spokesperson Taya Christianson told TechCrunch that all images generated using ChatGPT include metadata identifying their AI origin. She added that OpenAI actively enforces its usage policies, taking action when violations occur, and continuously learns from real-world use and feedback.

When TechCrunch questioned why ChatGPT permits the creation of fake receipts despite its policies against fraud, Christianson explained that OpenAI’s primary objective is to provide users with creative freedom. She noted that AI-generated receipts could serve legitimate purposes, such as financial literacy education, artistic projects, or product advertisements.

While OpenAI maintains safeguards to mitigate misuse, the company faces ongoing challenges in preventing malicious applications of its technology.