Image Inputs

The Responses API accepts images alongside text in a single turn, using the OpenAI-compatible input_image content part. Images are stored in the Data Platform rather than in the Responses API, so a conversation that contains images spans two services, which matters for how you export and delete them.

Supplying an image

Supported image types are PNG, JPEG and WebP. The type is detected from the image bytes; the data: header is ignored. Bytes that are not a supported image fail with 415 image_mime_not_allowed, whatever the header claims, and so does any other type (GIF, SVG, PDF…​). A supported image with the wrong header is accepted under its real type: PNG bytes sent as data:image/jpeg are treated as PNG.

The same check, including the IMAGE_ALLOWED_MIME_TYPES allowlist, applies to a file referenced by file_id, using the type Sherlock recorded at upload (compared case-insensitively, ignoring parameters such as ; charset=binary). A referenced file with no recorded type is rejected.

An input_image part takes exactly one source:

Field Meaning

image_url

A data: URI carrying base64 image bytes. The service uploads it to the Data Platform on your behalf.

file_id

A reference to a file that already exists in the Data Platform.

detail

Requested vision fidelity: low, high, or auto (default). Forwarded to the backend, which decides for auto.
External URLs are rejected

A fully-qualified http(s) image_url is not supported and is refused. The service never fetches an arbitrary caller-supplied URL.

  • curl

  • Python (OpenAI SDK)

curl -X POST https://your-deployment-url/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AA_TOKEN" \
  -d '{
    "model": "qwen3-32b-tool",
    "input": [
      {
        "role": "user",
        "content": [
          { "type": "input_text", "text": "What is in this image?" },
          {
            "type": "input_image",
            "image_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg...",
            "detail": "auto"
          }
        ]
      }
    ]
  }'
import base64

from openai import OpenAI

client = OpenAI(base_url="https://your-deployment-url/v1", api_key=AA_TOKEN)

with open("chart.png", "rb") as fh:
    encoded = base64.b64encode(fh.read()).decode()

response = client.responses.create(
    model="qwen3-32b-tool",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "What is in this image?"},
                {
                    "type": "input_image",
                    "image_url": f"data:image/png;base64,{encoded}",
                    "detail": "auto",
                },
            ],
        }
    ],
)

print(response.output_text)

Using an image already stored in Sherlock

Instead of sending base64 bytes on every request, you can upload an image to the Data Platform (Sherlock) once and pass its file_id. The service checks the file exists and is readable by you, and never uploads, copies, or deletes it: a referenced file belongs to you.

Both calls must use the same identity. The Responses API forwards your Authorization bearer to Sherlock, so the file must live in a search store you can view (can_view).

1. Create a search store (or reuse one)

curl -X POST "$SHERLOCK_URL/api/v1/search-stores" \
  -H "Authorization: Bearer $AA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-images",
    "models": [
      {
        "embedding_model": "text-embedding-3-small",
        "type": "dense",
        "dimension": 1536,
        "distance": "cosine",
        "modality": "text"
      }
    ]
  }'

Keep the returned id; it is the search_store_id. The store is private by default.

2. Upload the image

curl -X POST "$SHERLOCK_URL/api/v1/search-stores/$STORE_ID/files" \
  -H "Authorization: Bearer $AA_TOKEN" \
  -F "files=@dog.png;type=image/png"

The response is a list with one entry per uploaded file; keep its id. Uploads are deduplicated by content hash per store, so uploading the same bytes again returns the existing file.

3. Reference it from the Responses API

Pass the Sherlock file id as file_id and name its store in the x-alephalpha-search-store-id header:

  • curl

  • Python (OpenAI SDK)

curl -X POST https://your-deployment-url/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AA_TOKEN" \
  -H "x-alephalpha-search-store-id: $STORE_ID" \
  -d '{
    "model": "qwen3-32b-tool",
    "input": [
      {
        "role": "user",
        "content": [
          { "type": "input_text", "text": "What animal is this?" },
          { "type": "input_image", "file_id": "'"$FILE_ID"'" }
        ]
      }
    ]
  }'
response = client.responses.create(
    model="qwen3-32b-tool",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "What animal is this?"},
                {"type": "input_image", "file_id": file_id},
            ],
        }
    ],
    extra_headers={"x-alephalpha-search-store-id": store_id},
)

Without the header, a bare file_id cannot be located and the request fails with 400 (image_search_store_not_specified). The header also applies to inline data: images: when it is set, uploads go into that store instead of an auto-created one.

You can skip the header by passing the structured reference the service itself uses, a JSON string naming both the store and the file:

{
  "type": "input_image",
  "file_id": "{\"search_store_id\":\"<store id>\",\"file_id\":\"<file id>\"}"
}

Stored responses always carry file_id in this structured form, so you can copy an input_image part from a previous response into a new request unchanged.

Error Cause

400 image_search_store_not_specified

A file_id that is not a structured reference, without the x-alephalpha-search-store-id header.

400 image_not_found

The file does not exist in that store, or you cannot view it.

415 image_mime_not_allowed

The file is not a supported image type, or its type is not in IMAGE_ALLOWED_MIME_TYPES.

A file_id that does not parse as a structured reference (not JSON, or missing search_store_id / file_id) is treated as a bare file ID. With the x-alephalpha-search-store-id header, the whole string is looked up in that store, which normally fails with 400 image_not_found. Without the header, the request fails with 400 image_search_store_not_specified.

Deleting your response does not delete your file

Only images the service uploaded itself (from data: URLs) are removed when a response is deleted or purged by retention. Files you referenced by file_id stay in Sherlock until you delete them.

Sharing an image with a Sherlock public URL

Sherlock can mint a time-limited download link for any file you can view:

curl -X POST "$SHERLOCK_URL/api/v1/search-stores/$STORE_ID/files/$FILE_ID/public-urls?ttl=1h" \
  -H "Authorization: Bearer $AA_TOKEN"
{
  "url": "https://<sherlock-host>/api/v1/search-stores/<store>/files/<file>/download?token=<opaque>",
  "expires_at": "2026-08-25T13:00:00Z"
}

  • ttl is optional: it defaults to 1h and must be between 1m and 24h.

  • The token is a bearer credential bound to that one file: anyone holding the URL can download it until it expires or is revoked. Treat it as a secret.

  • Downloading still needs an authenticated call (Sherlock requires an identity on every request). A plain browser navigation without credentials fails with 401.

Do not pass a public URL as image_url

The Responses API refuses every non-data: image_url, including Sherlock public URLs. To use a Sherlock file as model input, pass its file_id as shown above. A public URL is for sharing the file with people or other systems, not for the Responses API.

See the Sherlock API reference (docs/sherlock/api.md in the sherlock-holmes repository, sections Upload Files and Create a Public Download URL) for the full upload and public URL contracts.

How It Works

  1. A data: image is uploaded to the Data Platform, and the service records which response, conversation and user it belongs to.

  2. On each turn the worker fetches the image from the Data Platform and inlines it for the backend as a data: URL. The backend never receives a URL to fetch.

  3. A file_id supplied directly is resolved the same way, with no upload step.

Images and compaction

Images are never described on upload. When a chain is compacted (manually via /v1/responses/compact or automatically), the service:

  1. attaches an ImageAI workflow named image-description to every search store referenced by the chain’s images (auto-created and caller-provided alike; if it is already attached, it is reused) and runs it over those images, one store per run, in parallel. Each run writes one description document per image into its store;

  2. copies the projected context and removes the image parts from the copy;

  3. puts each image’s description in its place as an input_text part;

  4. summarises the copy. The stored history is not changed.

Every Data Platform call authenticates with the caller’s Authorization bearer. For automatic compaction, the bearer of the request that crossed the threshold travels with the compaction job (stored in the Redis compaction stream, like request headers on the work stream), so a token that has expired by the time the worker runs makes those images fall back to the placeholder.

Inline images that were never stored, and images in a store whose attach or run fails or times out, are replaced by a short "no description available" placeholder, so compaction still completes (a failed run is logged at WARNING). Set IMAGE_DESCRIBE_FOR_COMPACTION=false to send the images to the summariser unchanged instead.

Where the bytes live

Image bytes are held by the Data Platform, not by the Responses API (see Data Flows). The Responses API stores only the provenance, which response, conversation, user, store and file an image belongs to.

Two consequences worth planning for:

  • Export. Retrieving a conversation through GET /v1/conversations/{id}/responses returns the image references, not the bytes. A complete export needs the corresponding Data Platform files as well.

  • Deletion. Images are purged by the retention sweep in step with the conversation hard-delete, so their lifetime is bounded by the same retention policy (see Managing Responses). The sweep authenticates with a service-account token, because it runs on a schedule and has no caller Authorization to forward.

Retention requires a service-account token

If RETENTION_SHERLOCK_TOKEN is unset, the sweep has no credential to delete with and skips the image purge. Images then outlive the conversations that referenced them. The service logs a warning at startup of the first affected sweep; deployments that accept image input should set the token.

Configuration

Image input is enabled implicitly whenever an image store base URL resolves, a deployment that already has file search configured gets image input without further settings.

When it is not enabled, any request carrying an input_image part (a data: image or a file_id) is refused with 400 image_input_disabled, before anything is stored or sent to the model. Without an image store the image could not be replaced by a Data Platform reference, so it would reach the model raw and no guardrail image stage could check it.

Variable Default Description

IMAGE_STORE_BASE_URL

falls back to FILE_SEARCH_BASE_URL, then SHERLOCK_API_URL

Data Platform base URL for image files. Image input is on when this resolves to a non-empty value.

IMAGE_MAX_BYTES

20971520 (20 MiB)

Per-image size cap.

IMAGE_INLINE_MAX_BYTES

1048576 (1 MiB)

Reserved for a future public-URL path; currently unused, the worker always inlines.

IMAGE_ALLOWED_MIME_TYPES

image/jpeg,image/png,image/webp

Comma-separated MIME allowlist for both inline uploads and file_id references. It can only narrow the supported types; listing another type (such as image/gif) does not enable it.

IMAGE_DESCRIBE_FOR_COMPACTION

true

Describe images via the store’s workflow before compaction and summarise the descriptions instead of the images.

IMAGE_DESCRIPTION_TIMEOUT_SECONDS

120

Max wait for one store’s image-description workflow run.

IMAGE_DESCRIPTION_POLL_INTERVAL_SECONDS

1

Interval between workflow-run status polls.

RETENTION_SHERLOCK_TOKEN

-

Service-account bearer the retention sweep uses to delete Data Platform images. Unset, images are not purged.