(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3997, in execute_model
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] deferred_state_corrections_fn = self._update_states(scheduler_output)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1239, in _update_states
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] self._init_mrope_positions(req_state)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1582, in _init_mrope_positions
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] assert req_state.prompt_token_ids is not None, (
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] AssertionError: M-RoPE requires prompt_token_ids to be available.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] AsyncLLM output_handler failed.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] Traceback (most recent call last):
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 660, in output_handler
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] outputs = await engine_core.get_output_async()
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 1030, in get_output_async
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] raise self._format_exception(outputs) from None
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
All requests using the /v1/chat/completions endpoints will have text/prompt_token_ids parts (corresponding to the chat template), and prompt_embeds parts are handled as mm_features. This method (rightly) filters out those prompt_embeds content parts as they are treated as text positions.
A sufficient solution to type narrowing here without raising a fatal assertion is to instead replace the assertion with a using dummy token ids:
def _init_mrope_positions(self, req_state: CachedRequestState):
model = self.get_model()
assert supports_mrope(model), "M-RoPE support is not implemented."
mrope_model = cast(SupportsMRoPE, model)
# Filter out prompt_embeds modality (text-only position info)
mrope_features = [
f for f in req_state.mm_features if f.modality != "prompt_embeds"
]
# Handle both token_ids and embeddings-only inputs
if req_state.prompt_token_ids is not None:
input_tokens = req_state.prompt_token_ids
elif req_state.prompt_embeds is not None:
# For text-only embeddings, dummy token IDs are safe since
# get_mrope_input_positions only uses len(input_tokens) when mm_features is empty
seq_len = req_state.prompt_embeds.shape[0]
input_tokens = list(range(seq_len))
# Verify no mm_features remain (should be true after prompt_embeds filter)
assert len(mrope_features) == 0, (
"M-RoPE with prompt_embeds-only input should have no multimodal features"
)
else:
raise ValueError(
"M-RoPE requires either prompt_token_ids or prompt_embeds."
)
req_state.mrope_positions, req_state.mrope_position_delta = (
mrope_model.get_mrope_input_positions(
input_tokens,
mrope_features,
)
)
Technically, in isolation, this method still crashes in the case where req_state.prompt_token_ids is None and req_state.mm_features, so the solution above still leaves that potential vector open. As far as can be determined, however, such a req_state is impossible in the first place in online mode, because it would require a /v1/completions request with prompt_embeds AND multimodal features, but the /v1/completions request schema does not expose multimodal inputs in any discernible way. Today, those are the only two endpoints with prompt_embeds support.
When in offline mode, it is technically possible to directly create an EngineCoreRequest that has prompt_embeds and not prompt_token_ids and mm_features, and pass that to LLM.generate. That would trigger this same assertion, and no validation would prevent that combination. It is strongly suspected, though, that this combination would be undefined in any model that support M-RoPE, because it would not be possible to determine which token positions correspond to mm_features. The proposed solution above would end up not setting req_state.mrope_positions and req_state.mrope_position_delta in this scenario, which could result in undefined behavior.
prompt_embeds is far more familiar here than M-RoPE, and it is understood that each model that supports it is responsible for defining its own get_mrope_input_positions which have varying implementations. There is insufficient knowledge to be prescriptive in how the two features should interact in the offline case, other than possibly raising a validation error earlier on preventing that combination (which would emulate the current assertion behavior). Regardless, in offline mode, the chances of a remote user being able to exploit this are slim-to-nil compared to the online case which is incredibly straightforward.
Impact
What kind of vulnerability is it? Who is impacted?
Denial of Service caused by an incorrect assertion inside of the GPUModelRunner which causes a fatal EngineCore exception
Any configuration with --enable-prompt-embeds and M-RoPE-supported model is vulnerable
The attack is extremely easy from the remote attacker's perspective (copying the official prompt_embeds online mode docs examples almost-verbatim, accounting for model-name and connection details, of course, will induce a guaranteed shutdown)