litellm/tests/local_testing
yuneng-jiang 002d64b321 fix(tests): increase MAX_CALLS and reduce sleep in flaky e2e budget test
The test_chat_completion_low_budget test was flaky because async spend
tracking couldn't reliably catch up within 50 calls with 0.5s sleeps.
Increased to 200 calls with 0.1s sleeps (same total time budget) to
give more opportunities for budget enforcement to trigger.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 00:04:31 -07:00
..
.litellm_cache
auto_router [Feat] Backend Router - Add Auto-Router powered by semantic-router (#12955) 2025-07-24 18:32:56 -07:00
example_config_yaml fix tests 2025-10-25 10:19:24 -07:00
test_configs fix tests 2025-10-25 10:19:24 -07:00
test_model_response_typing
adroit-crow-413218-bc47f303efc9.json
azure_fine_tune.jsonl
azure_speech.mp3 [Feat] Add Azure AVA TTS integration (#15749) 2025-10-20 16:52:23 -07:00
batch_job_results_furniture.jsonl
cache_unit_tests.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
conftest.py [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 2025-08-23 13:13:23 -07:00
create_mock_standard_logging_payload.py
data_map.txt
eagle.wav
example.jsonl VertexAI non-jsonl file storage support (#9781) 2025-04-09 14:01:48 -07:00
gettysburg.wav
large_text.py
model_cost.json
openai_batch_completions_router.jsonl
openai_batch_completions.jsonl
speech_vertex.mp3
stream_chunk_testdata.py
test_acompletion_fallbacks.py
test_acompletion.py
test_acooldowns_router.py fix tests 2025-10-25 10:19:24 -07:00
test_add_function_to_prompt.py
test_add_update_models.py fix(tests): skip remaining real prisma DB tests in CI and related test suites 2026-02-20 13:25:42 -03:00
test_aim_guardrails.py Fix aim security guardrail tests (#17499) 2025-12-04 13:29:20 -08:00
test_alangfuse.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
test_amazing_vertex_completion.py fix(tests): update deprecated gemini-1.5-pro model refs in vertex tests 2026-03-12 23:00:17 -07:00
test_anthropic_prompt_caching.py [Fix] Fix web search model info regression, deprecated prompt caching model, undocumented env keys 2026-02-20 17:26:58 -08:00
test_arize_ai.py feat: allow configuring project name for OpenTelemetry service name 2026-01-07 14:20:10 +09:00
test_arize_phoenix.py ArizePhoenixConfig 2025-11-22 09:28:37 -08:00
test_assistants.py _add_azure_related_dynamic_params 2025-10-25 11:11:36 -07:00
test_async_fn.py test_completion_cost_databricks_embedding 2025-10-25 11:47:03 -07:00
test_auth_utils.py test(proxy): add regression tests for vertex passthrough model names with slashes (#19855) 2026-01-27 17:34:40 -08:00
test_azure_content_safety.py
test_azure_openai.py Fix:test_aaaaazure_tenant_id_auth 2026-01-20 17:39:08 +05:30
test_azure_perf.py fix tests 2025-10-25 10:19:24 -07:00
test_basic_python_version.py [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
test_batch_completion_return_exceptions.py fix(test): accept both AuthenticationError and InternalServerError in batch_completion test (#20186) 2026-01-31 13:36:27 -08:00
test_batch_completions.py test fix: gcp deprecated gemini-1.5-flash 2025-08-06 08:43:45 -07:00
test_blocked_user_list.py fix(tests): skip remaining real prisma DB tests in CI and related test suites 2026-02-20 13:25:42 -03:00
test_braintrust.py [Performance] Improve LiteLLM Python SDK RPS by +200 RPS (#13839) 2025-08-20 21:46:33 -07:00
test_budget_manager.py
test_caching_handler.py Add support for caching for responses API 2026-01-14 13:33:07 +05:30
test_caching_ssl.py fix tests 2025-10-25 10:19:24 -07:00
test_caching.py Fix sonnet 3.7 tests 2026-02-20 17:29:16 -08:00
test_class.py fix tests 2025-10-25 10:19:24 -07:00
test_completion_cost.py fix(tests): increase MAX_CALLS and reduce sleep in flaky e2e budget test 2026-03-13 00:04:31 -07:00
test_completion_with_retries.py inspect BadRequestError after all other policy types (#19878) 2026-01-27 18:15:04 -08:00
test_completion.py [Fix] Replace deprecated claude-3-7-sonnet-20250219 with claude-sonnet-4-5-20250929 in test_completion 2026-02-20 17:26:02 -08:00
test_config.py fix tests 2025-10-25 10:19:24 -07:00
test_cost_calc.py
test_custom_api_logger.py
test_custom_callback_input.py Fix flaky audio streaming cost assertion in test_standard_logging_payload_audio 2026-03-12 23:23:10 -07:00
test_custom_llm.py feat(custom_llm): add image_edit and aimage_edit support (#17999) 2025-12-15 14:16:36 -08:00
test_custom_logger.py fix(ci): stabilize CI tests - conditional import, mock fixes, timing adjustments 2026-03-13 00:01:25 +00:00
test_disk_cache_unit_tests.py
test_docker_no_network_on_deploy.py fix: update docker test file to right path 2026-02-19 18:44:28 +05:30
test_dual_cache.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_dynamic_rate_limit_handler.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_dynamodb_logs.py
test_embedding.py fix(test): update deprecated gemini embedding model (#20621) 2026-02-06 18:35:40 -08:00
test_exceptions.py Fix update deprecated model test 2026-03-12 18:34:20 +05:30
test_file_types.py
test_function_call_parsing.py [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
test_function_calling.py FIx: replace deprecated claude-3-7-sonnet-20250219 with claude-4-sonnet-20250514 2026-02-20 17:27:59 -08:00
test_function_setup.py Add test for removal of thought signature 2025-12-23 14:56:42 +05:30
test_gcs_bucket.py Fixes test_aaabasic_gcs_logger 2026-01-21 08:58:16 +05:30
test_gcs_cache_unit_tests.py Add GCS bucket caching support (#13122) 2025-08-04 16:09:33 -07:00
test_gemini_reasoning_content.py Add thought signature for non tool call requests 2026-01-02 12:31:31 +05:30
test_get_llm_provider.py Fix test_default_api_base failing because of chatgpt as provider 2026-01-21 09:32:38 +05:30
test_get_model_file.py
test_get_model_info.py [Release - 02/10/2026] v1.81.10-nightly 2026-02-10 16:26:30 -08:00
test_get_optional_params_embeddings.py fix(embeddings): allow dimensions param passthrough via allowed_openai_params for non-text-embedding-3 OpenAI models 2026-02-26 09:59:37 +05:30
test_get_optional_params_functions_not_supported.py
test_google_ai_studio_gemini.py
test_guardrails_ai.py
test_helicone_integration.py fix tests 2025-10-25 10:19:24 -07:00
test_http_parsing_utils.py test_http_parsing_utils.py 2025-07-10 18:20:41 -07:00
test_img_resize.py
test_lakera_ai_prompt_injection.py
test_langchain_ChatLiteLLM.py
test_langsmith.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_least_busy_routing.py test: fixes because azure deactivated our account 2025-10-25 15:10:45 -07:00
test_litellm_max_budget.py
test_llm_guard.py [Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea (#10782) 2025-05-13 09:42:22 -07:00
test_load_test_router_s3.py fix tests 2025-10-25 10:19:24 -07:00
test_loadtest_router.py fix tests 2025-10-25 10:19:24 -07:00
test_logfire.py
test_logging.py
test_longer_context_fallback.py
test_lowest_cost_routing.py fix tests 2025-10-25 10:19:24 -07:00
test_lowest_latency_routing.py fix tests 2025-10-25 10:19:24 -07:00
test_lunary.py
test_max_tpm_rpm_limiter.py
test_mem_leak.py
test_mem_usage.py fix tests 2025-10-25 10:19:24 -07:00
test_mock_request.py test_router_mock_request_with_mock_timeout_with_fallbacks 2025-09-27 13:57:43 -07:00
test_model_alias_map.py test_model_alias_map 2025-09-01 17:59:40 -07:00
test_model_max_token_adjust.py
test_multiple_deployments.py
test_ollama_local_chat.py
test_ollama_local.py
test_ollama.py [Fix] CI/CD - litellm_mapped_tests_llms | litellm_mapped_tests_core | caching_unit_tests (#18197) 2025-12-18 08:49:23 -08:00
test_openai_moderations_hook.py [Fix] Guardrails API - Ensure OpenAI Moderations Guard works with OpenAI Embeddings (#20523) 2026-02-05 14:40:15 -08:00
test_opik.py OpikLogger: fix the bug with not incorrect attachment to existing trace & refactor (#15529) 2025-11-05 16:29:50 -08:00
test_pass_through_endpoints.py fix(tests): make RPM limit test sequential to avoid race condition 2026-02-23 16:34:52 -03:00
test_profiling_router.py
test_prometheus_service.py [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
test_prompt_caching.py claude-sonnet-4-5-20250929 fix 2025-10-31 18:20:52 -07:00
test_prompt_injection_detection.py fix tests 2025-10-25 10:19:24 -07:00
test_promptlayer_integration.py
test_provider_specific_config.py fix tests 2025-10-25 10:19:24 -07:00
test_pydantic_namespaces.py
test_pydantic.py
test_redis_batch_optimizations.py [Perf] Alexsander fixes round 2 - Oct 18th (#15695) 2025-10-18 11:12:00 -07:00
test_register_model.py Revert "test_update_model_cost_map_url" 2025-12-22 12:41:30 +05:30
test_router_auto_router.py test fix claude-sonnet-4-5-20250929 2025-10-28 19:05:13 -07:00
test_router_batch_completion.py test fix 2025-09-01 17:04:47 -07:00
test_router_budget_limiter.py fix(tests): stabilize 5 flaky/outdated router integration tests 2026-03-12 22:54:31 -07:00
test_router_caching.py fix tests 2025-10-25 10:19:24 -07:00
test_router_client_init.py test fix 2025-10-25 16:46:29 -07:00
test_router_cooldown_handlers.py fix(tests): stabilize 5 flaky/outdated router integration tests 2026-03-12 22:54:31 -07:00
test_router_custom_routing.py
test_router_debug_logs.py fix tests 2025-10-25 10:19:24 -07:00
test_router_fallback_handlers.py
test_router_fallbacks.py fix(tests): stabilize 5 flaky/outdated router integration tests 2026-03-12 22:54:31 -07:00
test_router_get_deployments.py Fix:add async_get_available_deployment_for_pass_through in code tests 2026-01-16 16:37:44 +05:30
test_router_init.py fix tests 2025-10-25 10:19:24 -07:00
test_router_max_parallel_requests.py Fix Azure RPM calculation formula (#19513) 2026-01-21 21:03:23 -08:00
test_router_pattern_matching.py Fix: Model matching priority in configuration 2026-01-13 11:44:48 +05:30
test_router_retries.py fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var instead of hardcoding model (#21781) 2026-02-21 10:46:49 -08:00
test_router_timeout.py [Fix] Fix flaky and outdated router integration tests 2026-03-12 15:20:55 -07:00
test_router_utils.py Add support for wildcards models for files api 2026-03-04 11:48:42 +05:30
test_router_with_fallbacks.py
test_router.py fix(tests): move test_router_azure_acompletion to llm_translation testing (#21837) 2026-02-21 14:41:53 -08:00
test_rules.py
test_sagemaker.py test: mock sagemaker tests 2025-03-21 16:21:18 -07:00
test_scheduler.py fix(scheduler): remove orphan entries from queue - causing memory leak. (#20866) 2026-02-10 22:34:52 -08:00
test_secret_detect_hook.py [Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea (#10782) 2025-05-13 09:42:22 -07:00
test_simple_shuffle.py
test_spend_calculate_endpoint.py test fix 2025-09-01 17:04:47 -07:00
test_stream_chunk_builder.py Fix flaky test_stream_chunk_builder_openai_audio_output_usage 2026-03-09 17:18:00 -07:00
test_streaming.py Replace deprecated model names in tests that were removed from remote model cost map 2026-03-12 14:12:07 -07:00
test_supabase_integration.py
test_team_config.py
test_text_completion.py [LLM Translation] Fix Realtime API endpoint for no intent (#13476) 2025-08-14 16:24:14 -07:00
test_timeout.py Replace deprecated model names in tests that were removed from remote model cost map 2026-03-12 14:12:07 -07:00
test_together_ai.py
test_tpm_rpm_routing_v2.py fix tests 2025-10-25 10:19:24 -07:00
test_traceloop.py
test_ui_sso_helper_utils.py
test_unit_test_caching.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_update_spend.py fix(tests): skip remaining real prisma DB tests in CI and related test suites 2026-02-20 13:25:42 -03:00
test_validate_environment.py
test_wandb.py
user_cost.json
vertex_ai.jsonl
vertex_batch_completions.jsonl
vertex_key.json
whitelisted_bedrock_models.txt Add supports_pdf_input: true to Claude 3.7 bedrock models (#9917) 2025-05-01 14:56:54 -07:00