b14e1d7d6a
* refactor: scope /health response to caller's models and tidy display fields Two small consistency changes to the /health response: 1. health_endpoint() now narrows _llm_model_list to deployments whose model_name is in user_api_key_dict.models, matching how other model listing endpoints already scope their output. The same narrowing applies to the cached health_check_results dict when background_health_checks is enabled, via a new _filter_health_check_results_by_model_ids helper. 2. ILLEGAL_DISPLAY_PARAMS in health_check.py picks up api_base and api_version, which are provider routing fields and not part of the health response shape. Tests in tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py pin both behaviors so future changes do not widen the response shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * address greptile review feedback (greploop iteration 1) - tests: extend background-cache test with model_id on cached entries plus positive assertions that model-a's deployment is the one returned, so the test is no longer satisfied by an empty result. - _health_endpoints.py: add a verbose_proxy_logger.debug line when a scoped key has accessible model_names but the matching deployments have no model_info.id, so the empty cache-result case is observable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * make background-cache test's non-vacuity explicit Restructure test_health_endpoint_filters_background_cache_by_user_access so the assertions positively pin the post-scoping result (one entry, model_id == "id-a", api_base == https://example-a.test) and add fixture sanity checks that confirm the source cache had two entries and every cached entry carries a model_id. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * gate api_base in /health response on proxy-admin role Replace the blanket strip of api_base / api_version with a role-aware post-processor: - api_base is now left in the cleaned per-deployment dict that _clean_endpoint_data produces (api_version stays in the denylist). - health_endpoint() removes api_base from each endpoint entry before returning when the caller's user_role is not PROXY_ADMIN / PROXY_ADMIN_VIEW_ONLY. The strip uses a copy so the shared health_check_results cache still carries api_base for subsequent admin reads. Net effect: a proxy admin can still see which Vertex region or Azure resource is healthy in the /health output, while non-admin keys (and read-only keys) only see model / model_id / status fields. Tests: - test_health_endpoint_admin_sees_api_base_non_admin_does_not pins both branches and verifies the cache is not mutated. - test_clean_endpoint_data_strips_credentials_but_keeps_api_base replaces the previous mask/drop tests now that the cleaning helper no longer touches api_base. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * address review feedback: api_version symmetry, missing-id warnings, deprecation header Three blockers raised in review: 1. api_version asymmetry — api_base was role-gated for proxy admins, but api_version was unconditionally stripped via ILLEGAL_DISPLAY_PARAMS. Move api_version out of the credential denylist and into a new ADMIN_ONLY_HEALTH_DISPLAY_PARAMS tuple alongside api_base, so admins keep both routing fields and non-admins lose both. Useful for telling apart Vertex regions or Azure api-versions from the /health response. 2. Silent empty results when scoped key's deployments lack model_info.id — raise the existing log from .debug to .warning, and add a structured "warnings" field to the response so the caller can distinguish "no deployments configured" from "deployments excluded due to missing model_info.id". 3. Migration signal for the api_base / api_version removal — when a non-admin caller hits /health, set a "Litellm-Health-Field-Notice" response header so existing dashboards or scripts that parsed those fields can detect the change programmatically rather than silently seeing absent keys. Tests adjusted: existing background-cache test injects a Response stub, admin-vs-non-admin test now asserts both api_base and api_version are gated and asserts the notice header. New test covers the warnings field when a scoped key's deployments are missing model_info.id. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * defensive copies + clarifying comments in /health filter - _filter_health_check_results_by_model_ids now shallow-copies each retained endpoint dict before returning. The shared module-level health_check_results cache should never be mutated by downstream transforms, even though _strip_admin_only_fields_from_health_result already builds new dicts today. - Document the live (model_name) vs cache (model_id) scoping asymmetry so future readers do not have to derive it from the warnings field. - Document why _PROXY_ADMIN_ROLES includes PROXY_ADMIN_VIEW_ONLY (read- only operators need routing fields to diagnose health). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| litellm | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| litellm-proxy-extras | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| eval_swe_bench.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| README.MD | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_team.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.