Adds litellm.proxy_auth to automatically obtain and refresh OAuth2/JWT
tokens when connecting to LiteLLM Proxy or any OAuth2-protected endpoint.
- Add ProxyAuthHandler for token lifecycle (obtain, cache, refresh)
- Add AzureADCredential wrapper for azure-identity credentials
- Add GenericOAuth2Credential for any OAuth2 provider (Okta, Auth0, etc)
- Auto-inject Authorization headers in completion() and embedding()
Closes#19834
Add the following SCIM v2 discovery endpoints per RFC 7643/7644:
- GET /scim/v2 - Base resource discovery (ListResponse of ResourceTypes)
- GET /scim/v2/ResourceTypes - List all supported resource types
- GET /scim/v2/ResourceTypes/{id} - Get a specific resource type (User/Group)
- GET /scim/v2/Schemas - List all supported schemas
- GET /scim/v2/Schemas/{uri} - Get a specific schema by URI
These endpoints are required by identity providers (Okta, Azure AD, etc.)
for SCIM resource discovery. Previously, GET /scim/v2 returned 404.
Also adds SCIMResourceType, SCIMSchema, and SCIMSchemaAttribute Pydantic
models to the SCIM types module.
Fixes#20295
* fix: strip stale mcp-session-id header to prevent 'Session not found' error loop
When VSCode reconnects to LiteLLM's MCP endpoint after a reload, it sends
a stale mcp-session-id header. The session was already cleaned up, causing
a 404 'Session not found' error. VSCode retries with the same stale ID,
creating an infinite error loop.
Before forwarding requests to the StreamableHTTP session manager, check if
the mcp-session-id header references a valid session. If the session doesn't
exist, strip the header so a new session is created automatically.
Fixes#20292
* refactor: extract stale session handling into _strip_stale_mcp_session_header helper
* perf: cache _get_relevant_args_to_use_for_logging() as module-level frozenset
The set of valid LLM API parameter names for logging was being rebuilt
on every request from 8 OpenAI SDK type annotations + set operations.
Since these are static TypedDict annotations that never change at
runtime, compute once at import time and store as a class-level
frozenset.
Line profiler: get_standard_logging_model_parameters() dropped from
774ms to 77ms across 12K calls (90% reduction, ~25µs/req saved).
* test: add tests for cached ModelParamHelper logging args
Verify cached frozenset matches dynamic computation and that
prompt content keys (messages, prompt, input) are excluded from
logged model parameters.
Pass-through endpoints (like vLLM classify) were not setting
standard_logging_object because _get_assembled_streaming_response
returns None for non-ModelResponse results.
This caused model_max_budget_limiter.async_log_success_event to raise
ValueError('standard_logging_payload is required').
The fix adds an elif branch in async_success_handler that mirrors the
non-pass-through code path.
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Add x-api-key header to CountTokens handler to match chat completion
authentication. Azure AI Anthropic requires this header per Microsoft's
native API format.
Bedrock rejects requests when toolResult or toolUse blocks within a
single message contain duplicate IDs. The Converse message transformer
merges consecutive tool/assistant messages without checking for
duplicate toolUseId values, causing BedrockException errors.
Add _deduplicate_bedrock_content_blocks() — a generalized helper that
removes duplicate blocks by ID, logs a warning for each dropped
duplicate via verbose_logger, and preserves non-tool blocks (e.g.
cachePoint). Apply it at all four merge sites (sync/async × toolResult/
toolUse).
The Anthropic /messages path was fixed in PR #19324; this applies the
equivalent fix to the Bedrock Converse path.
Fixes#20048
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Previously, when Model Armor guardrail blocked a request/response,
the `applied_guardrails` field was not populated in the logs because
`add_guardrail_to_applied_guardrails_header()` was called after the
HTTPException was raised.
This fix moves the `add_guardrail_to_applied_guardrails_header()` call
to before the blocking check in all hooks:
- async_pre_call_hook (pre_call mode)
- async_moderation_hook (during_call mode)
- async_post_call_success_hook (post_call mode)
- async_post_call_streaming_iterator_hook (streaming)
This ensures that even when a guardrail blocks content, the guardrail
name is properly recorded in the logs for observability.
Added regression tests to verify applied_guardrails is populated when
content is blocked.
Co-authored-by: Cursor <cursoragent@cursor.com>
When using LiteLLM's Anthropic /v1/messages endpoint to route requests to
OpenAI models, requests fail if any tool name exceeds OpenAI's 64-character
limit. Anthropic API has no such limit, causing compatibility issues.
Changes:
- Add truncate_tool_name() function using {55-char-prefix}_{8-char-hash} format
- Modify translate_anthropic_tools_to_openai() to truncate and return mapping
- Modify translate_anthropic_tool_choice_to_openai() to truncate tool name
- Restore original tool names in responses using the mapping
- Support tool name restoration in streaming responses
- Add backwards-compatible API (existing methods still work)
The fix only applies when routing Anthropic requests to OpenAI models.
Native Anthropic/Claude requests pass through unchanged.
* fix: models loadbalancing billing issue by filter (#18891)
* fix: models loadbalancing billing issue by filter
* fix: separate key and team access groups in metadata
* fix: lint issues
Fixes#19788
- Add `supported_regions: ["global"]` to Qwen MaaS models in model_prices_and_context_window.json
- Update `get_supported_regions()` to read directly from `model_cost` dict
- Update `get_complete_vertex_url()` to use `get_vertex_region()` for global-only models
- Update `create_vertex_url()` to generate correct URL for global location (without region prefix)
- Add tests for Qwen global endpoint support
Per review feedback, thought_signature should not be a root-level
param on ImageObject as it's not OpenAI compatible. Moved to
provider_specific_fields dict to match the pattern used in chat
completions (Message, Delta, Choices, etc).
Fixes#17184 - Gemini 3 Pro image preview model returns a thoughtSignature
field required for interactive image editing. This change:
- Adds thought_signature field to ImageObject class
- Updates Gemini and Vertex AI transformations to extract thoughtSignature
- Adds test for thought_signature in response transformation
- Check for both litellm_proxy_failed_requests_metric_total and the deprecated litellm_llm_api_failed_requests_metric_total
- The proxy-level failure hook may not always be called depending on where the exception occurs
- Simplify total_requests check to only verify key fields
Co-authored-by: Cursor <cursoragent@cursor.com>
