litellm/tests/llm_translation
yuneng-jiang 3604b600d3
[Infra] Merge internal dev branch with main (#25036)
* fix(proxy): enforce key-level model allowlist for custom auth

custom_auth_run_common_checks only runs common_checks (team/user/project model checks).
Custom auth now also enforces key-level model restrictions via can_key_call_model.

Move the custom-auth key-access regression tests to test_user_api_key_auth.py and keep test_custom_auth_end_user_budget.py focused on end-user budget behavior.

Made-with: Cursor

* fix(proxy): gate custom-auth key model checks behind opt-in

Keep key-level model allowlist enforcement in custom auth behind `custom_auth_run_common_checks` to preserve backwards compatibility, and update tests to verify default non-enforcement and opt-in enforcement behavior.

Made-with: Cursor

* test(proxy): isolate custom auth default check from shared settings state

Patch `proxy_server.general_settings` to an empty dict in the default custom-auth key-access test so it remains deterministic under shared module state.

Made-with: Cursor

* test(proxy): strengthen custom auth post-check assertions

Tighten custom auth regression tests by asserting exact can_key_call_model args and remove an unused common_checks mock from the default behavior path.

Made-with: Cursor

* fix(agentcore): parse A2A JSON-RPC responses in AgentCore provider

* fix(prompt-templates): ensure_alternating_roles handles tool-call chains

* feat(auth): add JWT claim routing overrides for OAuth2 validation

Made-with: Cursor

* docs(auth): document JWT-to-OAuth2 routing overrides

Add generic docs for running JWT and OAuth2 together, including routing_overrides YAML examples and list-based selector behavior for iss/client_id/aud.

Made-with: Cursor

---------

Co-authored-by: Milan <milan@berri.ai>
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
2026-04-02 16:38:01 -07:00
..
fixtures
realtime [Fix] Update realtime websocket tests for websockets v15+ API 2026-03-12 14:40:48 -07:00
test_llm_response_utils fix(types): normalize null top_logprobs to [] in ChatCompletionTokenLogprob (#22245) 2026-02-28 15:59:29 -08:00
test_skills_data Remove Apache 2 license from SKILL.md (#22322) 2026-02-27 19:33:55 -08:00
test-skill [Feat] New API - Claude Skills API (Anthropic) (#17042) 2025-11-24 15:01:40 -08:00
base_audio_transcription_unit_tests.py
base_embedding_unit_tests.py
base_llm_unit_tests.py Fix mistral ocr failing test 2026-03-09 11:29:33 +05:30
base_rerank_unit_tests.py
conftest.py [Fix] Restore unconditional importlib.reload for llm_translation conftest 2026-03-16 00:35:02 -07:00
dog.wav
duck.png
gettysburg.wav
guinea.png
log.xt
Readme.md
test_a2a.py [Feat] Allow calling A2A agents through LiteLLM /chat/completions API (#20358) 2026-02-03 12:52:33 -08:00
test_anthropic_completion.py Merge pull request #24445 from quora/fix/missing-content-part-added 2026-03-30 17:52:40 -07:00
test_aws_base_llm.py
test_azure_agents.py fix(azure-ai-agents): accumulate annotations from multiple text items in streaming 2026-03-18 09:04:00 +05:30
test_azure_ai.py test_completion_azure 2026-03-30 21:54:27 -07:00
test_azure_o_series.py test: test 2026-03-28 19:17:38 -07:00
test_azure_openai.py Announce April Townhall (#25021) 2026-04-02 14:10:49 -07:00
test_bedrock_agentcore.py fix(agentcore): simplify agentcore streaming (#17141) 2026-01-19 05:20:24 -08:00
test_bedrock_agents.py
test_bedrock_anthropic_regression.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_common_utils.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_completion.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_dynamic_auth_params_unit_tests.py
test_bedrock_embedding.py
test_bedrock_govcloud.py [Test Fix] fix gov pricing tests (#25022) 2026-04-02 15:55:55 -07:00
test_bedrock_gpt_oss.py
test_bedrock_invoke_tests.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_llama.py
test_bedrock_moonshot.py Add tests for kimi 2 bedrock model 2026-01-08 10:07:33 +05:30
test_bedrock_nova_embedding.py fix(cost-calc): use per-image pricing for Bedrock multimodal embeddings (#21646) 2026-02-20 08:51:21 -08:00
test_bedrock_nova_json.py
test_cloudflare.py test_completion_cloudflare 2026-03-30 16:40:37 -07:00
test_cohere.py Revert "Fix xdist test isolation: capture true defaults and poll instead of sleep" 2026-03-15 22:57:39 -07:00
test_containers_api.py [Feat] Containers API - add new container API file management + UI Interface (#17745) 2025-12-09 17:33:26 -08:00
test_convert_dict_to_image.py
test_databricks.py fix: test_databricks_embeddings[True] 2026-01-06 14:20:30 +05:30
test_deepgram.py
test_deepseek_completion.py test_deepseek_mock_completion 2026-01-17 16:36:42 -08:00
test_elevenlabs.py Integrate eleven labs text-to-speech (#16573) 2025-11-24 18:49:30 -08:00
test_evals_api.py fix(tests): add flaky retries to flaky CI tests (#21795) 2026-02-21 11:56:29 -08:00
test_fireworks_ai_translation.py fix: fix setup wizard 2026-03-30 09:06:31 -07:00
test_gemini_image_usage.py fix(gemini): resolve image token undercounting in usage metadata (#22608) 2026-03-04 18:52:08 -08:00
test_gemini.py [Fix] Update gemini-2.0-flash to gemini-2.5-flash in test_gemini integration tests 2026-03-30 13:53:38 -07:00
test_gigachat.py bugfix: Remove user messages merging 2026-02-03 12:58:28 +00:00
test_gpt4o_audio.py
test_groq.py Add grok reasoning content 2026-01-27 16:34:57 +05:30
test_hosted_vllm_embedding_e2e.py move e2e to llm translation 2026-02-17 22:33:13 +05:30
test_huggingface_chat_completion.py
test_hyperbolic.py
test_infinity.py
test_jina_ai.py
test_lambda_ai.py
test_langgraph.py [Feat] New Provider - add langgraph (#17783) 2025-12-10 12:30:35 -08:00
test_litellm_proxy_provider.py
test_minimax_tts.py Add support for minmax tts 2025-12-23 21:53:53 +05:30
test_mistral_api.py
test_model_cost_map_resilience.py [Stability] Investigate + fix issue where model cost map became poorly formatted (#20895) 2026-02-10 15:17:01 -08:00
test_morph.py
test_nvidia_nim.py TestNvidiaNim 2025-12-13 16:38:11 -08:00
test_openai_o1.py fix(tests): update outdated model names in o1 and gemini tests 2026-03-12 22:50:52 -07:00
test_openai.py Fix _supports_reasoning_effort_level for responses bridge 2026-03-13 13:29:39 +05:30
test_openrouter.py fix(test): update openrouter image generation assertion for gemini-2.5-flash-image (#23070) 2026-03-07 16:52:04 -08:00
test_optional_params.py test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 (#24931) 2026-04-01 11:01:03 -07:00
test_perplexity_reasoning.py
test_prompt_caching.py
test_prompt_factory.py [Infra] Merge internal dev branch with main (#25036) 2026-04-02 16:38:01 -07:00
test_replicate.py Add all replicate models in model cost map 2026-01-12 15:19:49 +05:30
test_rerank.py refactor: refactor testing 2026-03-28 18:39:32 -07:00
test_router_llm_translation_tests.py test: test 2026-03-28 19:17:38 -07:00
test_sambanova_chat_transformation.py
test_skills_api.py Reapply "feat: add model_cost aliases expansion support" 2026-03-12 13:36:57 -03:00
test_skills_e2e.py test_slack_gif_skill_creates_gif 2025-12-20 21:35:25 +05:30
test_snowflake.py Merge main and resolve Snowflake test conflict 2026-03-30 18:06:37 -07:00
test_text_completion_unit_tests.py fix(openai): use optimized async http client for text completions (#17831) 2025-12-11 14:54:26 -08:00
test_text_completion.py
test_together_ai.py
test_triton.py
test_unit_test_bedrock_invoke.py
test_v0.py
test_voyage_ai.py
test_watsonx.py [Fix] WatsonX tests failing on CI due to missing env vars 2026-03-30 13:27:03 -07:00
test_xai.py Fix usage in xai 2026-02-19 18:48:30 +05:30

Unit tests for individual LLM providers.

Name of the test file is the name of the LLM provider - e.g. test_openai.py is for OpenAI.