litellm/tests
shin-bot-litellm 1d92968e17
Fix test isolation for test_watsonx_gpt_oss_prompt_transformation (#20474)
Set cached tokenizer config directly and mock both sync and async
tokenizer functions to avoid race conditions when running with
parallel test execution (-n 16).

The issue was that parallel tests could populate the
litellm.known_tokenizer_config cache between clearing it and
when the code checked it. This caused the sync code path to be
used instead of the async path, bypassing the mocked async functions.

Fix:
1. Set cache directly instead of clearing it
2. Also mock sync versions _get_tokenizer_config and _get_chat_template_file

This ensures the test is deterministic regardless of test execution order.
2026-02-05 12:57:20 -08:00
..
agent_tests
audio_tests
basic_proxy_startup_tests
batches_tests Add cost tacking and usage info in call_type=aretrieve_batch 2026-01-29 15:27:41 +05:30
code_coverage_tests Custom Code Guardrails UI Playground (#20377) 2026-02-03 19:57:24 -08:00
documentation_tests fix: test_env_keys 2026-01-20 18:37:56 +05:30
enterprise Merge pull request #20402 from BerriAI/litellm_responses_tools_file_ids 2026-02-04 18:09:39 +05:30
guardrails_tests [Release Day] - Fixed CI/CD issues & changed processes (#19902) 2026-01-28 17:57:24 -08:00
image_gen_tests Fix: imagegeneration@006 has been deprecated 2026-01-22 18:24:59 +05:30
litellm feat(sdk): add proxy_auth for auto OAuth2/JWT token management (#20238) 2026-02-02 22:04:08 -08:00
litellm_core_utils Fix: empty assistant message for converse API 2026-02-04 09:50:58 +05:30
litellm_utils_tests feat: hashicorp vault rotate support 2026-01-23 17:32:55 +09:00
litellm-proxy-extras fix: resolve 'does not exist' migration errors as applied in setup_database (#19281) 2026-01-26 22:11:36 -08:00
llm_responses_api_testing Fix: Responses API logging error for StopIteration 2026-01-23 18:42:33 +05:30
llm_translation Merge branch 'main' into litellm_oss_staging_02_04_2026 2026-02-05 12:19:03 +05:30
load_tests
local_testing fix(test): accept both AuthenticationError and InternalServerError in batch_completion test (#20186) 2026-01-31 13:36:27 -08:00
logging_callback_tests Litellm fix langfuse otel trace (#20382) 2026-02-03 22:40:19 -08:00
mcp_tests Skip test_e2e_semantic_filter when OPENAI_API_KEY is not set (#20387) 2026-02-03 22:09:47 -08:00
multi_instance_e2e_tests
ocr_tests
old_proxy_tests/tests
openai_endpoints_tests Fix: Managed Batches: Inconsistent State Management for list and cancel batches 2026-02-03 14:47:28 +05:30
otel_tests test fix 2026-01-31 19:08:07 -08:00
pass_through_tests
pass_through_unit_tests [Fix] inconsistent response format in anthropic.messages.acreate() when using non anthropic providers (#20442) 2026-02-04 16:37:40 -08:00
proxy_admin_ui_tests Make test_get_users_key_count deterministic by creating dedicated test user (#19795) 2026-01-26 10:13:15 -08:00
proxy_e2e_anthropic_messages_tests fix: bedrock-converse-claude-sonnet-4.5 2026-01-31 14:41:02 -08:00
proxy_security_tests
proxy_unit_tests Add support for 0 cost models 2026-02-02 13:29:37 +05:30
router_unit_tests fix(test): add router.acancel_batch coverage (#20183) 2026-01-31 12:39:19 -08:00
scim_tests
search_tests skip brave tests 2026-01-22 10:50:23 -08:00
spend_tracking_tests increasing time for spend tracking 2026-01-20 16:31:25 -08:00
store_model_in_db_tests
test_litellm Fix test isolation for test_watsonx_gpt_oss_prompt_transformation (#20474) 2026-02-05 12:57:20 -08:00
unified_google_tests Revert "[Infra] Changing Google Tests to use Gemini 3 Flash Preview" 2026-01-20 17:32:10 -08:00
vector_store_tests [Feat] RAG API - Add s3_vectors as provider on /vector_store/search API + UI for creating + PDF support for /rag/ingest (#19895) 2026-01-27 16:30:59 -08:00
windows_tests
__init__.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py fix: resolve Read-only file system error in non-root images (#19449) 2026-01-20 19:00:52 -08:00
test_end_users.py
test_entrypoint.py
test_fallbacks.py Revert "fix: prevent error when max_fallbacks exceeds available models (#20071)" 2026-02-03 15:15:30 +05:30
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py Adding retries to flaky tests 2026-01-22 15:21:44 -08:00
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py
test_openai_endpoints.py
test_organizations.py Adding retries to flaky tests 2026-01-22 15:21:44 -08:00
test_otel_thread_leak.py Fix thread leak in OpenTelemetry dynamic header path (#19946) 2026-01-28 10:35:37 -08:00
test_passthrough_endpoints.py
test_presidio_latency.py fix(presidio): reuse HTTP connections to prevent OOMs (#19964) 2026-01-28 16:08:53 -08:00
test_proxy_server_non_root.py deactivating non root tests 2026-01-23 22:55:36 -08:00
test_ratelimit.py
test_resource_cleanup.py
test_service_logger_otel.py fix(langfuse_otel): ignore service logs and fix callback shadowing (#19298) 2026-01-19 05:53:47 -08:00
test_spend_logs.py
test_team_logging.py
test_team_members.py
test_team.py Fix: test_team_update_sc_2 2026-02-05 09:40:21 +05:30
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.