466f06df6d
* fix(mcp): surface upstream 401 for token-forwarding MCP servers For MCP servers configured with extra_headers: [Authorization], the gateway forwards the client token directly to the upstream. When that token is rejected (expired or invalid) the upstream returns 401, but the MCP SDK starts the SSE stream with 200 OK before calling handlers, so the 401 can't be returned mid-stream. Fix: add a pre-flight httpx probe in handle_streamable_http_mcp — before the SDK opens the session — so the gateway can still return HTTP 401 with WWW-Authenticate: Bearer authorization_uri=<gateway-discovery-url> when the upstream rejects the token. The probe fails-open (returns 200) on network errors so a transient hiccup does not block valid requests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): parallelize pre-flight auth probes and use HEAD to avoid side effects - Extract forwarded_auth outside the pass-through server loop (was called N times for the same scope value) - Gather all upstream auth probes concurrently with asyncio.gather instead of sequentially; eliminates N×5 s worst-case latency - Switch probe from POST+initialize JSON-RPC body to HEAD request; HEAD carries the Authorization header so the upstream rejects invalid tokens with 401 but never allocates a session or writes an audit entry Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): use get_async_httpx_client in _probe_upstream_auth Replaces bare httpx.AsyncClient with the project-standard get_async_httpx_client(httpxSpecialProvider.MCP) to satisfy the ensure_async_clients_test code coverage check and avoid the +500 ms per-request overhead of creating a new client on every probe call. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(mcp): extract pre-flight probe into _check_passthrough_upstream_auth Moves the parallel upstream auth probe logic out of handle_streamable_http_mcp into a dedicated helper to satisfy Ruff PLR0915 (Too many statements > 50). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): gate pre-flight probes on authorized server set to prevent bypass _check_passthrough_upstream_auth was resolving user-supplied server names directly before authorization ran, letting any permitted LiteLLM key trigger an upstream HEAD probe to a server it was not allowed to use. Changes: - Call _get_allowed_mcp_servers inside the helper so only servers the caller's key is authorized for are probed. - Move the call site to after toolset scoping so the auth context is fully resolved before the probe list is built. - Thread user_api_key_auth into the helper signature (replaces the raw mcp_servers name list). Co-authored-by: Cursor <cursoragent@cursor.com> * Add async HTTP HEAD support Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): use Scope type annotation in _get_forwarded_auth_from_scope Co-authored-by: Cursor <cursoragent@cursor.com> * Fix MCP upstream auth probe method Co-authored-by: Yassin Kortam <yassin@berri.ai> * Remove unused AsyncHTTPHandler head method Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): exclude has_client_credentials servers from pre-flight auth probe _prepare_mcp_server_headers skips caller Authorization when the server uses OAuth client-credentials (M2M), but the pre-flight probe was still selecting those servers and forwarding the caller's raw token in the HEAD request. Exclude servers with has_client_credentials from the probe list to match the actual downstream header-preparation logic. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): propagate upstream 403 as 403, not 401 with WWW-Authenticate Per RFC 9110, 401 means "go get new credentials." Mapping an upstream 403 to a gateway 401 causes OAuth clients to restart the authorization flow, obtain a fresh token with identical scopes, hit 403 again, and loop indefinitely. 401 from upstream → gateway 401 + WWW-Authenticate (re-authorize) 403 from upstream → gateway 403 (no WWW-Authenticate hint) Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): skip auth probe when Authorization may be the LiteLLM proxy key The pre-flight upstream probe must not forward the caller's Authorization header when it could itself be the LiteLLM proxy API key. Restrict the probe to requests that supply x-litellm-api-key explicitly — only then is the Authorization header unambiguously the upstream OAuth token the caller wants forwarded. * Fix MCP ASGI HTTPException propagation Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): use public AsyncHTTPHandler.post() in auth probe Use AsyncHTTPHandler.post() and catch httpx.HTTPStatusError explicitly so the 401/403 we want to surface is not silently swallowed by the broad fail-open except Exception block. Avoids reaching into the handler's private client attribute, which would silently regress to fail-open if AsyncHTTPHandler is ever refactored. * Fix MCP auth probe tests Co-authored-by: Yassin Kortam <yassin@berri.ai> * test(mcp): add coverage for httpx.HTTPStatusError path in auth probe AsyncHTTPHandler.post() calls raise_for_status() internally, so a real upstream 401/403 lands as httpx.HTTPStatusError. Add a test that exercises that specific exception path so a regression that swallows the error in the broad fail-open except Exception would be caught. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: claude-bot <claude-bot@anthropic.com> |
||
|---|---|---|
| .. | ||
| a2a_protocol | ||
| anthropic_interface/exceptions | ||
| caching | ||
| completion_extras | ||
| containers | ||
| enterprise | ||
| expected_fine_tuning_api | ||
| expected_responses_api_request | ||
| experimental_mcp_client | ||
| google_genai | ||
| images | ||
| integrations | ||
| interactions | ||
| litellm_core_utils | ||
| llms | ||
| ocr | ||
| passthrough | ||
| proxy | ||
| responses | ||
| router_strategy | ||
| router_utils | ||
| secret_managers | ||
| test_router | ||
| types | ||
| vector_stores | ||
| __init__.py | ||
| conftest.py | ||
| log.txt | ||
| readme.md | ||
| test_a2a_registry_lookup.py | ||
| test_acompletion_session_reuse_e2e.py | ||
| test_add_deployment_no_master_key.py | ||
| test_aembedding_session_reuse_e2e.py | ||
| test_anthropic_beta_headers_filtering.py | ||
| test_anthropic_skills_transformation.py | ||
| test_azure_video_router.py | ||
| test_bedrock_anthropic_1hr_cache_pricing.py | ||
| test_chat_ui_responses_session.py | ||
| test_claude_haiku_4_5_config.py | ||
| test_claude_opus_4_6_config.py | ||
| test_completion_timeout_resolution.py | ||
| test_compression.py | ||
| test_constants.py | ||
| test_container_router.py | ||
| test_cost_calculation_log_level.py | ||
| test_cost_calculator.py | ||
| test_count_tokens_public_api.py | ||
| test_dashscope_image_generation.py | ||
| test_deepseek_model_metadata.py | ||
| test_dockerfile_non_root.py | ||
| test_eager_tiktoken_load.py | ||
| test_exception_exports.py | ||
| test_exception_header_preservation.py | ||
| test_exception_mapping_request_attribute.py | ||
| test_filter_out_litellm_params.py | ||
| test_get_blog_posts.py | ||
| test_gpt_image_cost_calculator.py | ||
| test_groq_streaming_encoding.py | ||
| test_lazy_imports.py | ||
| test_litellm_params_reserved_keys.py | ||
| test_logging.py | ||
| test_lowest_latency_zero_tokens.py | ||
| test_main_module_header.py | ||
| test_main.py | ||
| test_model_cost_aliases.py | ||
| test_model_param_helper.py | ||
| test_model_response_normalization.py | ||
| test_nested_drop_params.py | ||
| test_openai_embedding_encoding_format_default.py | ||
| test_project_alias_tracking.py | ||
| test_project_tags_pydantic.py | ||
| test_redact_string_in_error_paths.py | ||
| test_redis.py | ||
| test_register_model_custom_pricing.py | ||
| test_responses_api_bridge_non_stream.py | ||
| test_responses_id_security.py | ||
| test_retrieve_batch_bedrock_dispatch.py | ||
| test_router_google_genai.py | ||
| test_router_model_cost_isolation.py | ||
| test_router_order_fallback.py | ||
| test_router_per_deployment_num_retries.py | ||
| test_router_redis_init.py | ||
| test_router_retry_non_retryable_errors.py | ||
| test_router_silent_experiment.py | ||
| test_router.py | ||
| test_secret_redaction.py | ||
| test_service_logger.py | ||
| test_setup_wizard.py | ||
| test_shared_session_integration.py | ||
| test_ssl_verify_unit.py | ||
| test_stream_chunk_builder_annotations.py | ||
| test_streaming_connection_cleanup.py | ||
| test_system_message_format_bug.py | ||
| test_utils_module_docstring.py | ||
| test_utils.py | ||
| test_uuid_helper.py | ||
| test_vcr_safe_body_matcher.py | ||
| test_video_generation.py | ||
| test_xai_grok_4_3_model_metadata.py | ||
| test_xai_responses_auto_routing.py | ||
Testing for litellm/
This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.
The point of this is to:
- Increase test coverage of
litellm/ - Make it easy for contributors to add tests for the
litellm/package and easily run tests without needing LLM API keys.
File name conventions
litellm/proxy/test_caching_routes.pymaps tolitellm/proxy/caching_routes.pytest_<filename>.pymaps tolitellm/<filename>.py