litellm/tests/test_litellm/proxy
Sameer Kankute 95e3d136e1
test(google): add google-genai SDK proxy integration tests (#29781)
* test(google): add google-genai SDK proxy integration tests for Gemini and Vertex

Pin google-genai in the CI dependency group and exercise streaming/non-streaming
generate_content through the LiteLLM proxy in the existing unified_google_tests suite.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(test): address Greptile review for google-genai proxy SDK tests

Restore GOOGLE_APPLICATION_CREDENTIALS after the module proxy fixture tears down,
initialize temp-file tracking on the proxy SDK base class, and skip litellm reload
for proxy_genai_sdk tests so the module-scoped proxy server stays consistent.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(test): only load Vertex credentials when keys exist for proxy SDK tests

Avoid writing empty GOOGLE_APPLICATION_CREDENTIALS temp files so Vertex tests
skip cleanly without credentials, use a session-scoped proxy fixture, and clean up
per-test credential temp files.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore(test): scope google-genai pin to unified_google_tests only

Remove google-genai from the ci dependency group and pin it in
tests/unified_google_tests/requirements.txt for local test installs.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(google): tie litellm reload skip to proxy fixture dependency

Replace the name-based reload guard with a check on whether the test
requests the google_genai_proxy_url fixture, so the skip stays correct
if the proxy SDK tests are renamed.

* fix(test): stop DatabaseURLSettings tests leaking DATABASE_URL into os.environ

The autouse env scrubber relied on monkeypatch.delenv, but apply_to_env
writes DATABASE_URL straight into os.environ, which monkeypatch never
tracks and therefore never undoes. The synthesized writer.example.com URL
leaked past the last test in this module and into proxy-infra tests that
read DATABASE_URL to decide whether to hit a real database, e.g.
test_deprecated_key_grace_period_cache_hit_path, turning an intended skip
into a ConnectError. Snapshot and restore the managed vars directly so the
original environment is reinstated regardless of how it was mutated.

* test(google): drop redundant per-test vertex credential setup

The session-scoped google_genai_proxy_url fixture already configures
GOOGLE_APPLICATION_CREDENTIALS before the proxy starts, and
_require_proxy_sdk skips when credentials are missing, so the per-test
_setup_vertex_credentials_if_needed helper and its temp-file tracking
never did any work. Remove it to keep the ABC self-contained.

* test(google): declare model_config contract on proxy SDK ABC

_skip_reason_if_credentials_missing reads self.model_config to pick the
provider, but that property was only declared on the sibling
BaseGoogleGenAITest. Make the dependency explicit by adding model_config
as an abstract property on BaseGoogleGenAIProxySDKTest so the ABC is
self-contained and a standalone subclass fails fast instead of hitting an
AttributeError.

* test(google): narrow streaming error catch to Exception

Catching BaseException in the streaming assertion swallowed
KeyboardInterrupt and SystemExit, turning a Ctrl-C into a test failure
message instead of letting pytest interrupt cleanly. Only genuine runtime
errors should be recorded as stream failures, so catch Exception.

* test(google): initialize proxy on the same loop that serves it

The proxy was initialized via asyncio.run() on the main thread, which
creates and tears down a throwaway event loop, while requests were served
on a separate loop in the worker thread. Any asyncio primitive bound to
the init loop would be unusable once serving started. Run initialize()
on the worker thread's loop right before server.serve() so setup and
request handling share a single event loop.

* test(google): drop redundant google-genai requirements pin

google-genai>=1.37.0,<2.0 is already declared in the proxy-runtime extra,
which the google_generate_content_endpoint_testing CI job installs via
uv sync --all-extras. The standalone tests/unified_google_tests/requirements.txt
duplicated that pin with a narrower ==1.37.0 specifier and was never
installed by CI, so it added a second source of truth without changing
what gets installed. Drop it and rely on the proxy-runtime extra.

* chore: revert incidental uv.lock exclude-newer bump

The google-genai ci pin was added and then dropped (it is already
provided by the proxy-runtime group), but each uv lock recomputed the
relative exclude-newer span, leaving only a timestamp bump in uv.lock.
Restore it to the base value so this test-only PR carries no lockfile
change.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-05 21:05:32 +00:00
..
_experimental/mcp_server Litellm oss staging 050626 (#29774) 2026-06-05 13:51:51 -07:00
a2a feat(a2a): well-known agent-card discovery + LangGraph Platform mode (#28860) 2026-05-29 20:50:42 -07:00
agent_endpoints Support OAuth M2M for Databricks Apps A2A agents (#29586) 2026-06-04 23:03:37 -07:00
anthropic_endpoints
auth fix(auth): expand all-team-models sentinel in can_key_call_model for batch validation (#29746) 2026-06-05 09:04:45 -07:00
client fix: scope CLI stored token to base_url to prevent cross-domain credential leakage (#26945) 2026-05-01 12:11:32 -07:00
common_utils fix(reset_budget): write only {spend, budget_reset_at} and stop pre-zeroing counter (#29358) 2026-05-30 17:48:16 -07:00
db test(google): add google-genai SDK proxy integration tests (#29781) 2026-06-05 21:05:32 +00:00
discovery_endpoints
experimental/mcp_server
google_endpoints Gemini managed agents support (#28270) 2026-05-19 16:02:03 -07:00
guardrails Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
health_endpoints feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439) 2026-06-02 16:30:44 -07:00
hooks fix(auth): expand all-team-models sentinel in can_key_call_model for batch validation (#29746) 2026-06-05 09:04:45 -07:00
image_endpoints
management_endpoints fix(galileo): use ingest traces API and standard logging payload (#29651) 2026-06-05 09:03:17 -07:00
management_helpers feat(mcp/auth): additive key access-group grants + opt-in member assignment (#29313) 2026-05-30 17:35:31 -07:00
memory Litellm memory improvements v2 (#26541) 2026-04-25 19:03:43 -07:00
middleware fix(proxy): point /metrics 401 at the opt-out flag 2026-05-08 18:09:14 -07:00
openai_files_endpoint
pass_through_endpoints fix(proxy): passthrough 404 when SERVER_ROOT_PATH is set (#29658) 2026-06-04 07:44:51 -07:00
policy_engine fix(guardrails): return HTTP 400 for litellm content filter blocks (#28418) 2026-05-30 14:28:46 -07:00
prompts
proxy_server fix(proxy): stop team BYOK model name corruption on model edit (#29731) 2026-06-04 20:40:40 -07:00
public_endpoints fix(mcp): gate /public/mcp_hub strictly on litellm.public_mcp_servers (#27764) 2026-06-04 17:26:59 +05:30
rag_endpoints fix: harden /key/update authorization checks (#27878) 2026-05-14 04:16:04 +00:00
realtime_endpoints
response_api_endpoints Litellm websocket improvements (#29563) 2026-06-03 11:48:35 -07:00
shutdown feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439) 2026-06-02 16:30:44 -07:00
spend_tracking fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 (#29515) 2026-06-02 12:07:11 -07:00
test_configs
types_utils chore(proxy): also scrub guardrail callbacks / module paths from DB overlay 2026-05-14 01:24:51 +00:00
ui_crud_endpoints chore(ci): merge dev branch (#28657) 2026-05-25 13:48:47 -07:00
utils test(proxy/utils): pin ProxyLogging behavior (#29485) 2026-06-02 17:45:39 -07:00
vector_store_endpoints fix(proxy): restrict vector store index create/delete to proxy admins (#29202) 2026-05-30 15:10:21 -07:00
__init__.py
conftest.py feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439) 2026-06-02 16:30:44 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py
test_batch_expiry.py fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata (#28425) 2026-05-26 11:35:42 -07:00
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py fix(proxy/batches): forward model to retrieve_batch for bedrock 2026-04-29 22:48:03 +02:00
test_budget_reservation.py fix(proxy): gate image-gen reservation strictly on model mode 2026-05-09 09:16:27 -07:00
test_caching_routes.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_chat_completion_metadata.py
test_common_request_processing.py fix(proxy): disable proxy buffering on streaming SSE responses (#29557) 2026-06-04 17:23:14 +05:30
test_component_allowlists.py fix(anthropic/adapter): emit thinking block for reasoning_content-only streaming chunks (#29600) 2026-06-03 11:26:13 -07:00
test_cors_config.py
test_custom_proxy.py
test_dynamic_mcp_route.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_empty_model_list.py
test_enforce_user_param.py
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py fix: resolve access group names in _filter_models_by_team_id (#25224) 2026-05-01 11:55:36 +05:30
test_health_check_functions.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_health_check_max_tokens.py Litellm agent oss staging 05 11 2026 (#27733) 2026-05-13 14:09:12 -07:00
test_langfuse_passthrough_security.py chore(callbacks): guard dynamic integration hosts 2026-04-30 14:27:19 -07:00
test_lazy_openapi_snapshot.py [Fix] Tests: Move Misplaced Import in Lazy OpenAPI Snapshot Test 2026-05-01 14:24:53 -07:00
test_litellm_pre_call_utils.py fix(proxy): enforce tag budgets for key-level tags (#29108) 2026-05-28 11:39:02 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847) 2026-05-13 12:03:36 -07:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py
test_model_info_default_limits.py
test_model_level_guardrails.py fix(proxy): run model-level post_call guardrails on streaming requests (#26922) 2026-05-07 11:53:03 -07:00
test_openapi_schema_validation.py fix(proxy): preserve HTTP operations when injecting WebSocket stubs into OpenAPI schema 2026-05-06 00:28:42 +02:00
test_pricing_field_strip.py fix(proxy): move pricing strip below the litellm_metadata JSON-string parse 2026-05-03 02:04:05 +00:00
test_prometheus_cleanup.py
test_provider_url_destination_guard.py chore(proxy): move URL-valued model/file_id guard from SDK to proxy 2026-05-02 02:17:46 +00:00
test_proxy_cli.py Add granian as a ASGI compliant web server. Provider better throughput stability, (#26027) 2026-05-21 19:08:37 -07:00
test_proxy_logging_hook_detection.py perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths (#28289) 2026-05-23 12:15:59 -07:00
test_proxy_server.py fix(proxy): omit OpenAI [DONE] on google-genai streamGenerateContent (#29426) 2026-06-01 14:38:19 -07:00
test_proxy_types.py fix(proxy): add default=None to LiteLLM_TeamMembership.litellm_budget_table (#29684) 2026-06-04 12:13:11 -07:00
test_proxy_utils.py feat(otel): typed semconv-aligned OpenTelemetry instrumentation (#28909) 2026-05-29 23:15:27 -07:00
test_pyroscope.py Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. 2026-05-04 18:17:31 +00:00
test_redis_auth_cache_flag.py refactor(proxy): replace DualCache with UserApiKeyCache for user API key management 2026-04-28 19:15:03 +00:00
test_response_model_sanitization.py perf: eliminate per-request callback scanning on proxy hot path (#27858) 2026-05-14 09:28:31 -07:00
test_route_a2a_models.py
test_route_llm_request.py feat(proxy): fix vector store retrieve/list/update/delete without model (#27929) 2026-05-15 04:37:59 -07:00
test_sensitive_route_auth.py chore(proxy): guard sensitive public endpoints 2026-04-30 11:52:47 -07:00
test_shared_health_check.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_spend_log_cleanup.py fix(proxy): keep spend log cleanup running after batch failures and surface DB errors (#27303) 2026-05-06 18:39:15 +00:00
test_swagger_chat_completions.py fix(proxy): avoid misleading multi-method operation ids 2026-04-30 20:44:14 -07:00
test_team_member_update.py
test_team_org_move.py
test_tools_allowlist_enforcement.py
test_update_llm_router_resilience.py