vLLM : DoS via unvalidated mm_processor_kwargs in multimodal image processors (Isaac, InternVL, H2OVL, etc.)

Summary

API clients can supply arbitrary mm_processor_kwargs per request. vLLM forwards these values through the multimodal rendering and processing pipeline to Hugging Face and custom image/video processor constructors and calls. Resource-sensitive values such as image size, patch size, patch count, crop count, or frame count can therefore be controlled by a remote client.

For processors that upscale inputs, a single request can force an allocation large enough to terminate the API server or worker with an out-of-memory condition.

Affected Components

The per-request field was introduced in commit 041e29471671a3d43afedb6a749421d56ef4d3e7 and first shipped in v0.7.3. The generic forwarding path remains present in current main.

The Isaac path is a concrete high-amplification example:

• ChatCompletionRequest.mm_processor_kwargs
• OnlineRenderer request extras
• MMProcessorInputs.hf_processor_mm_kwargs
• BaseMultiModalProcessor.info.get_hf_processor(**hf_processor_mm_kwargs)
• IsaacProcessingInfo.get_image_processor(**kwargs)
• IsaacImageProcessor(**kwargs)
• process_vision_for_patches()

Equivalent constructor forwarding exists in InternVL-derived processors, H2OVL, Step3VL, NemotronVL, and other multimodal processors. The issue is at the shared remote-input boundary and is not limited to a fixed model list.

Impact

• Availability: one request can exhaust CPU or GPU memory and terminate a shared service process.
• Scope: all users of the affected service instance lose availability.
• Authentication: vLLM's API key is optional, so deployments without an external authentication layer are remotely reachable without privileges.

Remediation

Reject non-empty per-request mm_processor_kwargs at every API-server request schema by default. Operators serving only trusted clients can explicitly opt in to the existing override behavior. Server-level --mm-processor-kwargs and offline LLM overrides remain available.

This shared boundary covers current and future multimodal processors instead of maintaining a model-specific list or a denylist that can miss custom processor arguments.

A patch for this was merged here: https://github.com/vllm-project/vllm/pull/58830