The GitHub merge conflict resolver concatenated both test sets but left
`from litellm.proxy._lazy_openapi_snapshot import _normalize_operation_ids`
stranded between functions instead of at the top of the file.
* fix: add expected_base_url origin check to get_litellm_gateway_api_key
* fix: scope get_stored_api_key and save base_url on login
* fix: pass base_url to get_stored_api_key in CLI entrypoint
* fix: scope ProxyClient stored key to base_url
* test: add expected_base_url coverage for get_stored_api_key
* fix: initialize self.http with resolved api_key not raw param
* fix: black formatting in client.py and test_auth_commands.py
* refactor: scope /health response to caller's models and tidy display fields
Two small consistency changes to the /health response:
1. health_endpoint() now narrows _llm_model_list to deployments whose
model_name is in user_api_key_dict.models, matching how other model
listing endpoints already scope their output. The same narrowing applies
to the cached health_check_results dict when background_health_checks is
enabled, via a new _filter_health_check_results_by_model_ids helper.
2. ILLEGAL_DISPLAY_PARAMS in health_check.py picks up api_base and
api_version, which are provider routing fields and not part of the
health response shape.
Tests in tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py
pin both behaviors so future changes do not widen the response shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* address greptile review feedback (greploop iteration 1)
- tests: extend background-cache test with model_id on cached entries plus
positive assertions that model-a's deployment is the one returned, so
the test is no longer satisfied by an empty result.
- _health_endpoints.py: add a verbose_proxy_logger.debug line when a scoped
key has accessible model_names but the matching deployments have no
model_info.id, so the empty cache-result case is observable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* make background-cache test's non-vacuity explicit
Restructure test_health_endpoint_filters_background_cache_by_user_access
so the assertions positively pin the post-scoping result (one entry,
model_id == "id-a", api_base == https://example-a.test) and add fixture
sanity checks that confirm the source cache had two entries and every
cached entry carries a model_id.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* gate api_base in /health response on proxy-admin role
Replace the blanket strip of api_base / api_version with a role-aware
post-processor:
- api_base is now left in the cleaned per-deployment dict that
_clean_endpoint_data produces (api_version stays in the denylist).
- health_endpoint() removes api_base from each endpoint entry before
returning when the caller's user_role is not PROXY_ADMIN /
PROXY_ADMIN_VIEW_ONLY. The strip uses a copy so the shared
health_check_results cache still carries api_base for subsequent
admin reads.
Net effect: a proxy admin can still see which Vertex region or Azure
resource is healthy in the /health output, while non-admin keys (and
read-only keys) only see model / model_id / status fields.
Tests:
- test_health_endpoint_admin_sees_api_base_non_admin_does_not pins both
branches and verifies the cache is not mutated.
- test_clean_endpoint_data_strips_credentials_but_keeps_api_base
replaces the previous mask/drop tests now that the cleaning helper
no longer touches api_base.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* address review feedback: api_version symmetry, missing-id warnings, deprecation header
Three blockers raised in review:
1. api_version asymmetry — api_base was role-gated for proxy admins, but
api_version was unconditionally stripped via ILLEGAL_DISPLAY_PARAMS.
Move api_version out of the credential denylist and into a new
ADMIN_ONLY_HEALTH_DISPLAY_PARAMS tuple alongside api_base, so admins
keep both routing fields and non-admins lose both. Useful for telling
apart Vertex regions or Azure api-versions from the /health response.
2. Silent empty results when scoped key's deployments lack model_info.id —
raise the existing log from .debug to .warning, and add a structured
"warnings" field to the response so the caller can distinguish "no
deployments configured" from "deployments excluded due to missing
model_info.id".
3. Migration signal for the api_base / api_version removal — when a
non-admin caller hits /health, set a "Litellm-Health-Field-Notice"
response header so existing dashboards or scripts that parsed those
fields can detect the change programmatically rather than silently
seeing absent keys.
Tests adjusted: existing background-cache test injects a Response stub,
admin-vs-non-admin test now asserts both api_base and api_version are
gated and asserts the notice header. New test covers the warnings field
when a scoped key's deployments are missing model_info.id.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* defensive copies + clarifying comments in /health filter
- _filter_health_check_results_by_model_ids now shallow-copies each
retained endpoint dict before returning. The shared module-level
health_check_results cache should never be mutated by downstream
transforms, even though _strip_admin_only_fields_from_health_result
already builds new dicts today.
- Document the live (model_name) vs cache (model_id) scoping asymmetry
so future readers do not have to derive it from the warnings field.
- Document why _PROXY_ADMIN_ROLES includes PROXY_ADMIN_VIEW_ONLY (read-
only operators need routing fields to diagnose health).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile follow-ups on the prior commit:
- (P1) ``is_request_body_safe`` recursed into ``litellm_embedding_config``
with no depth bound, so a request body 1000 levels deep could exhaust
Python's call stack and surface a 500 ``RecursionError``. Refactored
the check to be iterative (single-level descent into a fixed list of
nested-config keys) and extracted the per-dict banned-param scan into
a helper that's shared between the root and the nested call sites.
Also fixes the ``recursive_detector`` CI job that was triggered by
the recursive-by-name pattern.
- (P2) ``assert_same_origin`` error messages identified the mismatching
component but echoed the ``expected`` host and the candidate
hostname back to the caller. In the SSRF threat model the caller is
the attacker, so reflecting that information was a secondary leak of
operator infrastructure. Messages now identify only *which*
component mismatched (scheme / host / port) without naming names.
- (P2) ``_NESTED_CONFIG_KEYS`` was defined after the function that used
it. Hoisted the constant (and the new ``_BANNED_REQUEST_BODY_PARAMS``
tuple) above the function for readability.
Adds a 1000-level-deep nested config test that asserts no
``RecursionError`` and a hostname-leak test that asserts no operator
host appears in the rejection message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The async CustomStreamWrapper.__anext__ caught httpx.TimeoutException
and re-raised it raw, bypassing _handle_stream_fallback_error. The
Router's FallbackStreamWrapper only reacts to MidStreamFallbackError,
so stream_timeout firing mid-stream never triggered fallbacks the way
connection-phase timeout does.
Route timeouts through _handle_stream_fallback_error like every other
exception so they wrap into MidStreamFallbackError and the Router can
switch to a fallback model.
Two SSRF findings were OPEN with no in-flight fix; both are closed
now using narrow defenses that key off existing trust boundaries.
VERIA-6 (Milvus ``litellm_embedding_config``):
``is_request_body_safe`` already blocks ``api_base`` / ``api_key`` /
``langfuse_host`` / ``s3_endpoint_url`` / etc. at the *root* of the
request body, gated by an admin opt-in (``allow_client_side_credentials``
or per-deployment ``configurable_clientside_auth_params``). The bug is
that the Milvus vector-store transformer unpacks
``litellm_embedding_config`` into ``litellm.embedding(**embedding_config)``,
so a caller can smuggle the same banned params in via nesting and bypass
the check. Fix: ``is_request_body_safe`` now recurses into a known list
of nested-config dicts (``litellm_embedding_config`` for now) and applies
the same banned-param check with the same admin opt-in. Admin-side
vector-store config flows through ``litellm_params`` rather than the
request body, so it's unaffected.
VERIA-51 (polling URLs returned by upstream APIs):
Azure DALL-E 2, Azure Document Intelligence, and Black Forest Labs
all blindly fetched a polling URL returned by the upstream and
attached the operator's API key to the request. A compromised upstream
or a future API contract change could redirect credentials anywhere.
New ``url_utils.assert_same_origin(candidate, expected)`` helper checks
scheme, host (case-insensitive), and port (with default-port
normalization). Applied at all five polling sites: Azure DALL-E
sync+async, Azure DI sync+async, BFL image generation sync+async, BFL
image edit sync+async. Cross-origin polling URLs now raise rather than
forward credentials. The Azure DALL-E ``Expected 'status' in response``
exception no longer reflects the raw response body — that path turned
Blind SSRF into Full-Read SSRF for the limited window before the
origin check fully closed it.
Tests: 7 ``assert_same_origin`` unit tests, 6 ``is_request_body_safe``
nested-config tests, 5 polling-site rejection tests + 1 same-origin
sanity check.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Route publisher/model ids (e.g. xai/grok) to .../endpoints/openapi; keep model in JSON body
- Add model_prices keys for vertex_ai/openai/xai/grok-*
- Document xAI Grok on vertex_partner (aligned with GPT-OSS)
- Add tests for create_vertex_url and body-model heuristic
Made-with: Cursor
Success handlers already run when CustomStreamWrapper or
CachedResponsesAPIStreamingIterator finishes replay. Logging at
cache-hit time for acompletion/completion streaming duplicated spend
and callbacks. Align tests with deferred behavior.
Made-with: Cursor
_base_process_chunk only encodes response IDs when parsed_chunk contains
a top-level "response" key. Align test_process_chunk_completed_response_
updates_id_and_usage_cost with that contract and test_base_responses_api_streaming_iterator.
Made-with: Cursor
The test supplies a minimal PDF base64 payload but expected the wrong
constant (base64 for "test"). Assert against the same pdf_b64 value
and drop the unused import.
Made-with: Cursor
* fix(bedrock): handle document content blocks in Converse API message conversion
Document content blocks (used for PDF support) were silently dropped
during message conversion for Bedrock's Converse API. The content block
processing loop only handled text, image_url, and file types — document
blocks were skipped without warning, causing the model to respond as if
no document was provided.
Adds document block handling in three locations:
- Sync user message processing (_bedrock_converse_messages_pt)
- Async user message processing (_bedrock_converse_messages_pt_async)
- Tool result conversion (_convert_to_bedrock_tool_call_result)
Fixes#24641
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use _validate_format for proper MIME type to Bedrock format mapping
Address Greptile review: naive media_type.split("/")[1] produced invalid
Bedrock format names for complex MIME types (e.g. OOXML → docx, text/plain
→ txt, text/markdown → md). Now reuses BedrockImageProcessor._validate_format
which handles all MIME types correctly via mimetypes + fallback.
Also fixes test assertions to expect correct Bedrock format values and adds
text/plain and text/markdown test cases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reject non-base64 document sources with a clear error
URL-type document sources (e.g. {"type": "url", "url": "..."}) would
crash with an opaque KeyError on missing 'media_type'. Guard at the top
of _process_document_message and raise a clear ValueError since Bedrock
Converse only supports base64-encoded document sources.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The /v2/model/info endpoint (used by the UI's Models + Endpoints page)
was not resolving access group names when filtering models by team.
When a team has models: ["Group-A"] where "Group-A" is an access group,
_filter_models_by_team_id() passed it as a literal model name to
get_model_list(), which found no deployments with that name. This caused
the UI to show all models instead of only team-accessible ones.
The request-time auth path (model_in_access_group in auth_checks.py)
correctly resolves access groups via get_model_access_groups(). This
fix applies the same resolution in _filter_models_by_team_id() for both
the in-memory router lookup and the database fallback query.
Tests added:
- test_filter_resolves_access_group_names
- test_filter_resolves_mix_of_access_groups_and_literal_names
- test_filter_excludes_models_from_other_access_group
- test_filter_db_fallback_receives_resolved_model_names
