litellm

History

yuneng-jiang ff869e91b0 Fix flaky caching tests: use mock_response, add parallelism, remove fail-fast - Replace real OpenAI/Anthropic/Bedrock API calls with mock_response in ~20 cache tests to eliminate network-dependent flakiness - Remove -x (fail-fast) from caching_unit_tests so all failures are reported - Add parallelism: 2 with circleci tests run --split-by=timings - Improve pip dependency cache key (v2-caching-deps) with fallback key Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>		2026-03-15 15:23:19 -07:00
..
agent_tests
audio_tests
basic_proxy_startup_tests
batches_tests
code_coverage_tests	[Fix] Add max_depth guard to BFL _read_image_bytes recursive function	2026-03-12 13:22:56 -07:00
documentation_tests
enterprise
guardrails_tests	Optimize CI: parallelize router and guardrails test jobs, fix test isolation	2026-03-14 22:54:44 -07:00
image_gen_tests	fix(ci): remove unused is_expired variable (ruff F841) and handle ModelDeprecated in image gen test	2026-03-12 01:27:42 +00:00
litellm	fix(proxy): Windows compatibility for Prisma engine watchdog (#23494 )	2026-03-12 16:02:37 -07:00
litellm_core_utils
litellm_utils_tests	fix(tests): remove test_oidc_circle_v1_with_amazon_fips that depends on external infra	2026-03-13 10:33:52 -07:00
litellm-proxy-extras
llm_responses_api_testing
llm_translation	fix(tests): remove test_completion_bedrock_claude_sts_oidc_auth and test_completion_bedrock_httpx_command_r_sts_oidc_auth that depend on external infra	2026-03-13 11:53:53 -07:00
load_tests
local_testing	Fix flaky caching tests: use mock_response, add parallelism, remove fail-fast	2026-03-15 15:23:19 -07:00
logging_callback_tests	Merge pull request #23483 from BerriAI/litellm_update_deprecated_test_models	2026-03-12 14:16:52 -07:00
mcp_tests	Fix test_get_tools_for_single_server	2026-03-12 18:33:14 +05:30
multi_instance_e2e_tests
ocr_tests
old_proxy_tests/tests
openai_endpoints_tests
otel_tests	fix(tests): increase MAX_CALLS and reduce sleep in flaky e2e budget test	2026-03-13 00:04:31 -07:00
pass_through_tests	fix(ci): stabilize CI tests - conditional import, mock fixes, timing adjustments	2026-03-13 00:01:25 +00:00
pass_through_unit_tests
proxy_admin_ui_tests
proxy_e2e_anthropic_messages_tests	Merge branch 'main' into litellm_fix_nova_pro_max_tokens	2026-03-09 15:47:53 -07:00
proxy_e2e_azure_batches_tests	Fix flaky proxy_e2e_azure_batches_tests: populate _hidden_params for DB-cached batch retrieval	2026-03-15 15:13:30 -07:00
proxy_security_tests
proxy_unit_tests	Handle response.failed, response.incomplete, and response.cancelled (#23492 )	2026-03-13 23:02:09 -07:00
router_unit_tests
scim_tests
search_tests	Merge remote main, resolve conflict keeping new unit tests	2026-03-09 15:20:20 -07:00
spend_tracking_tests	[Fix] Derive SPEND_PER_REQUEST dynamically in spend accuracy tests	2026-03-13 13:27:06 -07:00
store_model_in_db_tests
test_litellm	Fix flaky encrypted_content_affinity tests: clear HTTP client cache, disable retries	2026-03-15 15:18:34 -07:00
unified_google_tests
vector_store_tests
windows_tests
__init__.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py	fix: tiktoken cache nonroot offline (#23498 )	2026-03-14 10:48:36 -07:00
test_end_users.py
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py	[Fix] Remove deprecated o1-preview from O-series test and deduplicate is_o_series check	2026-03-12 13:22:00 -07:00
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py	fix(ci): stabilize CI tests - conditional import, mock fixes, timing adjustments	2026-03-13 00:01:25 +00:00
test_new_vector_store_endpoints.py	fix(ci): stabilize CI tests - conditional import, mock fixes, timing adjustments	2026-03-13 00:01:25 +00:00
test_openai_endpoints.py
test_organizations.py
test_otel_thread_leak.py
test_passthrough_endpoints.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py
test_team_logging.py
test_team_members.py
test_team.py	[Fix] Fail fast when team member spend not flushed in time	2026-03-13 12:40:17 -07:00
test_users.py

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.