When JPEGs Start Giving Orders: A Journey into Multi-modal Prompt Injection

Hello World! During a recent security assessment, I came across an interesting finding in an AI-powered application that used a vision-language model to generate captions from user-supplied images. What initially appeared to be a straightforward image-captioning feature eventually led me down the rabbit hole of multi-modal prompt injection.

About the application

The application includes a feature that allows users to either upload an image or provide an image URL. Once an image is submitted, the backend retrieves it, processes it using a vision-language model, and generates captions describing its contents. These AI-generated captions are then used in subsequent workflows within the application.

Initial Testing

When I started testing this feature, my initial focus was on testing for Server-Side Request Forgery (SSRF), as the application accepted user-supplied image URLs. I experimented with various payloads, URL manipulations, and different attack scenarios to understand how the backend retrieved and processed the provided images. However, none of my initial attempts revealed any SSRF issues.

To better understand its behavior, I observed how the feature worked under normal conditions. The workflow was straightforward: the user supplied an image (or an image URL), the backend processed it using the vision-language model, and the application returned AI-generated captions describing the contents of the image.

Since the application's other AI-powered features had implemented effective guardrails, I initially assumed that simple prompt injection attempts would also be mitigated in this module. Still, out of curiosity, I decided to test a simple prompt injection payload. I created a simple image containing prompt injection instructions written in Notepad and submitted it to the image captioning feature.

As it turned out, my assumption was incorrect.

The prompt injection attempt succeeded. Unlike the application's other AI-powered features, this module did not appear to effectively mitigate prompt injection. My payload, which instructed the model to output "Prompt Injection Successful" was followed exactly and the model returned exactly the phrase I had instructed it to output.

At this point, I had confirmed the application was vulnerable to multi-modal prompt injection. Unlike traditional prompt injection, where malicious instructions are supplied directly as text, multi-modal prompt injection embeds attacker controlled instructions within another input modality (such as an image) which is then interpreted by a vision-language model.

This also explained why my earlier SSRF testing hadn't produced any interesting results. The real attack surface wasn't the image retrieval itself, but how the retrieved image was interpreted by the vision model.

While the initial payload simply manipulated the generated captions, the next obvious question was whether the model could be convinced to disclose information beyond its intended output. The results were far more interesting than I had expected.

After confirming the initial prompt injection, I continued experimenting with additional prompt injection payloads. The model responded with internal prompt instructions, details about the application's functionality, and other application-specific information. It also returned what appeared to be environment variables and runtime details. While I couldn't independently verify the authenticity of all the disclosed values, the responses demonstrated that the model could be influenced into revealing information beyond its intended purpose.

Although the feature was intended to generate captions for user-supplied images, it also interpreted and followed attacker controlled instructions embedded within those images. By embedding prompt injection instructions within an image, I was able to manipulate the model's responses instead of receiving genuine captions.

This highlights an important security consideration for AI-powered applications: user-controlled content should be treated as untrusted input, regardless of whether it is text, an image, or another file type. Without appropriate guardrails and validation, even a seemingly harmless image can become a path for influencing a model's behavior in unintended ways.