The default timeout (600s) from AsyncHTTPHandler is sufficient.
Removes the explicit timeout param to keep the call simple.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-request `httpx.AsyncClient` with `get_async_httpx_client`
to avoid the +500ms latency penalty from creating new clients per
request. Updates tests to mock the cached client factory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The documentation test checks that all env vars used in code are
documented. The Vantage integration added 5 new env vars without
updating the reference table.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix genuine regression in responses_api_bridge_check where the second
call assigned to `model_info` instead of `responses_api_model_info`,
preventing gpt-5.4 + tools + reasoning_effort from routing to the
Responses API bridge.
Also update outdated tests:
- Vantage tests: match "csv" file key and use supported column names
- Anthropic caching test: add "type": "custom" to expected tool payload
- Claude Agent SDK test: remove non-deterministic LLM content assertion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): bump tar to 7.5.11 and tornado to 6.5.5
- tar >=7.5.11: fixes CVE-2026-31802 (HIGH) in node-pkg
- tornado >=6.5.5: fixes CVE-2026-31958 (HIGH) and GHSA-78cv-mqj4-43f7 (MEDIUM) in python-pkg
Addresses vulnerabilities found in ghcr.io/berriai/litellm:main-v1.82.0-stable Trivy scan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: document tar override is enforced via Dockerfile, not npm
* fix: revert invalid JSON comment in package.json tar override
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Handle response.failed, response.incomplete, and response.cancelled terminal events in background streaming
Previously the background streaming task only handled response.completed and
hardcoded the final status to "completed". This missed three other terminal
event types from the OpenAI streaming spec, causing failed/incomplete/cancelled
responses to be incorrectly marked as completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove unused terminal_response_data variable
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Address code review: derive fallback status from event type, rewrite tests as integration tests
1. Replace hardcoded "completed" fallback in response_data.get("status")
with _event_to_status lookup so that response.incomplete and
response.cancelled events get the correct fallback if the response
body ever omits the status field.
2. Replace duplicated-logic unit tests with integration tests that
exercise background_streaming_task directly using mocked streaming
responses and assert on the final update_state call arguments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove dead mock_processor and unused mock_response parameter from test helper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove FastAPI and UserAPIKeyAuth imports from test file
These types were only used as Mock(spec=...) arguments. Drop the spec
constraints and remove the top-level imports to avoid pulling FastAPI
into test files outside litellm/proxy/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Log warning when streaming response has no body_iterator
If base_process_llm_request returns a non-streaming response (no
body_iterator), log a warning since this likely indicates a
misconfiguration or provider error rather than a successful completion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."
* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."
* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
inherit it, and removed the now-redundant copy from the Azure AI subclass.
* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
inherit it, and removed the now-redundant copy from the Azure AI subclass.
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
The model dropdown menus in the Add Fallbacks modal were rendering behind
the modal overlay because Ant Design portals Select dropdowns to document.body
by default. By setting getPopupContainer to attach the dropdown to its parent
element, the dropdown inherits the modal's stacking context and renders above
the modal.
Fixes#17895
Fixes#23502
The huggingface_embed.embedding() call was not receiving the headers
parameter, causing extra_headers (e.g., X-HF-Bill-To) to be silently
dropped. Other providers (openrouter, vercel_ai_gateway, bedrock) already
pass headers correctly. This fix adds headers=headers to match the
behavior of other providers.
Co-authored-by: Jah-yee <sparklab@outlook.com>
Extract permission checks and constraint validation from update_key_fn
into _validate_update_key_data helper to reduce statement count below
the 50-statement limit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests were outdated after #23472 added pagination (take/order) to find_many
and stale-row cleanup via update_many. Updated assertions to match new call
signatures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The self-exclusion filter compared raw key strings against SHA-256
hashed tokens from the DB, so keys were never excluded and
double-counting persisted. Now hash data.key before comparison.
Also add tpm_limit_type/rpm_limit_type to _throughput_fields_changed
guard, fall back to existing_key_row.team_id for team limit checks
(matching the org pattern), and add team self-exclusion test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only run org validation (get_org_object + _check_org_key_limits) when
the update actually touches throughput-related fields (tpm_limit,
rpm_limit, or organization_id). Previously, any update to a key
belonging to an org would trigger the check, which would fail with a
400 if the org had been deleted — blocking unrelated field changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When updating a key, _check_org_key_limits and _check_team_key_limits
would include the key being updated in the find_many results, causing
its current limits to be counted twice (once from the DB query, once
from the new requested limits). This caused false 400 errors on valid
limit adjustments.
Fix: exclude the key being updated (by matching token) from the
allocated totals before checking limits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of hardcoding SPEND_PER_REQUEST (which broke when the model
changed from gpt-3.5-turbo-0301 to gpt-3.5-turbo), make a single
calibration request first, poll for its spend, and use that as the
per-request cost. Fails fast with pytest.fail() after 5 retries if
calibration cannot determine the cost.
Also fixes a bug in test_basic_spend_accuracy where the user spend
assertion error message referenced user_info['info'] instead of
user_info['user_info'].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Same deprecated model fix as proxy_server_config.yaml — these two CI
configs also referenced gpt-3.5-turbo-0301 which has no pricing data.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase wait timeout to 90s and pytest.fail() instead of silently
continuing, so the failure message points at the real cause.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gpt-3.5-turbo-0301 was removed from the model cost map, so every call
had response_cost=0 and team member spend never increased. The wait
helper also returned True after 3s regardless of whether spend updated.
- Switch fake-openai-endpoint to gpt-3.5-turbo (has pricing in cost map)
- Remove premature early-return in wait_for_team_member_spend_update
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase max_retries from 6 to 9 and retry_delay from 10s to 20s
(180s total wait, up from 60s) to give batch cost tracking more time
to finish before cleanup attempts file deletion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The terminal-state DB shortcut in retrieve_batch returned a LiteLLMBatch
with empty _hidden_params, causing the managed_files hook to skip encoding
output_file_id into a unified ID. This adds the same model_id extraction
from unified_batch_id that the non-terminal path already has.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- realtime_api/main.py: Revert param types to Dict, explicitly construct
RealtimeSessionConfig/RealtimeExpiresAfter before passing to
RealtimeClientSecretRequest
- presidio.py: Move type:ignore[override] to def line where mypy reports it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_e2e_managed_batch test intermittently fails during cleanup
when deleting the input file — the batch cost tracking hasn't finished
processing yet (batch_processed=true not set), causing a 400 error.
This is a timing race condition unrelated to batch retrieval logic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Union[RealtimeClientSecretResponse, Response] annotation breaks
FastAPI's response model generation. Revert to the original annotation
and suppress mypy on the error-path return instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The terminal-state DB shortcut in retrieve_batch returned a LiteLLMBatch
with empty _hidden_params, causing the managed_files hook to skip encoding
output_file_id into a unified ID. This adds the same model_id extraction
from unified_batch_id that the non-terminal path already has.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
