ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
212b249e38
litellm/tests/local_testing
History
hatim-ez 17bfa420e4
fix(router): discard oldest entry when trimming latency list in lowest_latency strategy (#25548) 
* fix(router): discard oldest entry when trimming latency list in lowest_latency strategy

The lowest_latency routing strategy keeps a rolling window of the most
recent latency and time-to-first-token measurements per deployment. When
the window is full, the strategy was discarding the *newest* value
instead of the oldest, because the trim used
`[: max_latency_list_size - 1]` (keeping indices 0..N-2) rather than
`[1:]` (dropping index 0 and keeping indices 1..N-1).

Since new values are appended at the end, the bug meant the most recent
measurement was always dropped once the list reached capacity. The
routing decisions then relied on stale data (including any early-spike
values that never aged out), and timeout penalties written via
`async_log_failure_event` were silently discarded as well.

Fix the slice in all five call sites (sync + async log_success_event for
both latency and time_to_first_token, and async_log_failure_event for
the timeout penalty) and add regression tests covering each path.

* test(router): cover async TTFT trim path in lowest_latency regression tests

Adds test_ttft_list_trimming_discards_oldest_entry_async, an async
counterpart to test_ttft_list_trimming_discards_oldest_entry that drives
async_log_success_event with a ModelResponse and completion_start_time so
the async time_to_first_token trim branch is actually exercised.

Previously no test touched that code path: the sync TTFT test used
log_success_event, and the async latency test passed a plain dict
response_obj without stream/completion_start_time, so TTFT was never
computed and the async trim was unreached. Verified load-bearing by
reverting only the async TTFT slice — the new test fails and all others
pass.

* format
2026-04-14 23:37:49 +05:30
..
.litellm_cache
auto_router [Feat] Backend Router - Add Auto-Router powered by semantic-router (#12955) 2025-07-24 18:32:56 -07:00
example_config_yaml test: test 2026-03-28 19:17:38 -07:00
test_configs test: test 2026-03-28 19:17:38 -07:00
test_model_response_typing
azure_fine_tune.jsonl
azure_speech.mp3 [Feat] Add Azure AVA TTS integration (#15749) 2025-10-20 16:52:23 -07:00
batch_job_results_furniture.jsonl
cache_unit_tests.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
conftest.py [Fix] Reset api_base/api_key in xdist conftest to prevent cross-test leakage 2026-03-15 23:55:44 -07:00
create_mock_standard_logging_payload.py
data_map.txt
eagle.wav
example.jsonl
gettysburg.wav
large_text.py
model_cost.json
openai_batch_completions_router.jsonl
openai_batch_completions.jsonl
speech_vertex.mp3
stream_chunk_testdata.py
test_acompletion_fallbacks.py
test_acompletion.py
test_acooldowns_router.py test: test 2026-03-28 19:17:38 -07:00
test_add_function_to_prompt.py
test_add_update_models.py fix(tests): skip remaining real prisma DB tests in CI and related test suites 2026-02-20 13:25:42 -03:00
test_aim_guardrails.py Fix aim security guardrail tests (#17499) 2025-12-04 13:29:20 -08:00
test_alangfuse.py test: update key names 2026-03-28 21:13:16 -07:00
test_amazing_vertex_completion.py format vertex test file 2026-04-09 23:58:35 -07:00
test_anthropic_prompt_caching.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_arize_ai.py test: rename env var 2026-03-28 20:27:39 -07:00
test_arize_phoenix.py ArizePhoenixConfig 2025-11-22 09:28:37 -08:00
test_assistants.py fix 2026-03-30 21:33:47 -07:00
test_async_fn.py test_completion_cost_databricks_embedding 2025-10-25 11:47:03 -07:00
test_auth_utils.py Support multiple headers mapped to the customer user role (#23664) 2026-03-15 14:20:45 +05:30
test_azure_content_safety.py
test_azure_openai.py test: test 2026-03-28 19:17:38 -07:00
test_azure_perf.py test: test 2026-03-28 19:17:38 -07:00
test_basic_python_version.py [Fix] Pin uv/pip versions and fix bare prisma calls in CI 2026-04-10 00:04:32 -07:00
test_batch_completion_return_exceptions.py fix(test): accept both AuthenticationError and InternalServerError in batch_completion test (#20186) 2026-01-31 13:36:27 -08:00
test_batch_completions.py test fix: gcp deprecated gemini-1.5-flash 2025-08-06 08:43:45 -07:00
test_blocked_user_list.py fix(tests): skip remaining real prisma DB tests in CI and related test suites 2026-02-20 13:25:42 -03:00
test_braintrust.py [Performance] Improve LiteLLM Python SDK RPS by +200 RPS (#13839) 2025-08-20 21:46:33 -07:00
test_budget_manager.py
test_caching_handler.py Add support for caching for responses API 2026-01-14 13:33:07 +05:30
test_caching_ssl.py Merge main and resolve conflict in test_router_client_init.py 2026-03-30 18:44:33 -07:00
test_caching.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_class.py test: test 2026-03-28 19:17:38 -07:00
test_completion_cost.py fix(tests): replace deprecated model refs in cost and model_info tests 2026-03-13 00:39:35 -07:00
test_completion_with_retries.py inspect BadRequestError after all other policy types (#19878) 2026-01-27 18:15:04 -08:00
test_completion.py test_completion_azure 2026-03-30 21:07:49 -07:00
test_config.py test: test 2026-03-28 19:17:38 -07:00
test_cost_calc.py Revert "Fix xdist test isolation: capture true defaults and poll instead of sleep" 2026-03-15 22:57:39 -07:00
test_custom_api_logger.py
test_custom_callback_input.py Fix flaky audio streaming cost assertion in test_standard_logging_payload_audio 2026-03-12 23:23:10 -07:00
test_custom_llm.py feat(custom_llm): add image_edit and aimage_edit support (#17999) 2025-12-15 14:16:36 -08:00
test_custom_logger.py Mark test_redis_cache_completion_stream as flaky with retries 2026-03-15 20:44:18 -07:00
test_disk_cache_unit_tests.py
test_docker_no_network_on_deploy.py build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
test_dual_cache.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_dynamic_rate_limit_handler.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_dynamodb_logs.py
test_embedding.py fix: batch-limit stale managed object cleanup to prevent 300K row UPD… (#25258) 2026-04-06 19:11:55 -07:00
test_exceptions.py fix: cleanup tests 2026-03-30 16:24:35 -07:00
test_file_types.py
test_function_call_parsing.py [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
test_function_calling.py Fix flaky tests: anthropic error_msg state leak and vertex llama 404 2026-03-15 17:59:25 -07:00
test_function_setup.py Add test for removal of thought signature 2025-12-23 14:56:42 +05:30
test_gcs_bucket.py test GCS BUCKET 2026-03-30 17:53:26 -07:00
test_gcs_cache_unit_tests.py Add GCS bucket caching support (#13122) 2025-08-04 16:09:33 -07:00
test_gemini_reasoning_content.py Add thought signature for non tool call requests 2026-01-02 12:31:31 +05:30
test_get_llm_provider.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_get_model_file.py
test_get_model_info.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_get_optional_params_embeddings.py fix(embeddings): allow dimensions param passthrough via allowed_openai_params for non-text-embedding-3 OpenAI models 2026-02-26 09:59:37 +05:30
test_get_optional_params_functions_not_supported.py
test_google_ai_studio_gemini.py
test_guardrails_ai.py
test_helicone_integration.py fix tests 2025-10-25 10:19:24 -07:00
test_http_parsing_utils.py test_http_parsing_utils.py 2025-07-10 18:20:41 -07:00
test_img_resize.py
test_lakera_ai_prompt_injection.py
test_langchain_ChatLiteLLM.py
test_langsmith.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_least_busy_routing.py test: fixes because azure deactivated our account 2025-10-25 15:10:45 -07:00
test_litellm_max_budget.py
test_llm_guard.py
test_load_test_router_s3.py fix tests 2025-10-25 10:19:24 -07:00
test_loadtest_router.py test: test 2026-03-28 19:17:38 -07:00
test_logfire.py
test_logging.py
test_longer_context_fallback.py
test_lowest_cost_routing.py fix tests 2025-10-25 10:19:24 -07:00
test_lowest_latency_routing.py fix(router): discard oldest entry when trimming latency list in lowest_latency strategy (#25548) 2026-04-14 23:37:49 +05:30
test_lunary.py
test_max_tpm_rpm_limiter.py
test_mem_leak.py
test_mem_usage.py fix tests 2025-10-25 10:19:24 -07:00
test_mock_request.py test_router_mock_request_with_mock_timeout_with_fallbacks 2025-09-27 13:57:43 -07:00
test_model_alias_map.py test_model_alias_map 2025-09-01 17:59:40 -07:00
test_model_max_token_adjust.py
test_multiple_deployments.py
test_ollama_local_chat.py
test_ollama_local.py
test_ollama.py [Fix] CI/CD - litellm_mapped_tests_llms | litellm_mapped_tests_core | caching_unit_tests (#18197) 2025-12-18 08:49:23 -08:00
test_openai_moderations_hook.py [Fix] Guardrails API - Ensure OpenAI Moderations Guard works with OpenAI Embeddings (#20523) 2026-02-05 14:40:15 -08:00
test_opik.py OpikLogger: fix the bug with not incorrect attachment to existing trace & refactor (#15529) 2025-11-05 16:29:50 -08:00
test_pass_through_endpoints.py fix(tests): make RPM limit test sequential to avoid race condition 2026-02-23 16:34:52 -03:00
test_profiling_router.py
test_prometheus_service.py [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
test_prompt_caching.py claude-sonnet-4-5-20250929 fix 2025-10-31 18:20:52 -07:00
test_prompt_injection_detection.py test: test 2026-03-28 19:17:38 -07:00
test_promptlayer_integration.py
test_provider_specific_config.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_pydantic_namespaces.py
test_pydantic.py
test_redis_batch_optimizations.py [Perf] Alexsander fixes round 2 - Oct 18th (#15695) 2025-10-18 11:12:00 -07:00
test_register_model.py Revert "test_update_model_cost_map_url" 2025-12-22 12:41:30 +05:30
test_router_auto_router.py test fix claude-sonnet-4-5-20250929 2025-10-28 19:05:13 -07:00
test_router_batch_completion.py test fix 2025-09-01 17:04:47 -07:00
test_router_budget_limiter.py test: test 2026-03-28 19:17:38 -07:00
test_router_caching.py test: test 2026-03-28 19:17:38 -07:00
test_router_client_init.py test_router_init_azure_service_principal_with_secret_with_environment_variables 2026-03-30 21:15:53 -07:00
test_router_cooldown_handlers.py test: test 2026-03-28 19:17:38 -07:00
test_router_custom_routing.py Optimize CI: parallelize router and guardrails test jobs, fix test isolation 2026-03-14 22:54:44 -07:00
test_router_debug_logs.py test: test 2026-03-28 19:17:38 -07:00
test_router_fallback_handlers.py Optimize CI: parallelize router and guardrails test jobs, fix test isolation 2026-03-14 22:54:44 -07:00
test_router_fallbacks.py test: test 2026-03-28 19:17:38 -07:00
test_router_get_deployments.py Fix:add async_get_available_deployment_for_pass_through in code tests 2026-01-16 16:37:44 +05:30
test_router_max_parallel_requests.py Revert "Fix xdist test isolation: capture true defaults and poll instead of sleep" 2026-03-15 22:57:39 -07:00
test_router_pattern_matching.py Fix: Model matching priority in configuration 2026-01-13 11:44:48 +05:30
test_router_retries.py fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var instead of hardcoding model (#21781) 2026-02-21 10:46:49 -08:00
test_router_timeout.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_router_utils.py test: test 2026-03-28 19:17:38 -07:00
test_router_with_fallbacks.py
test_router.py refactor: cleanup deprecated models 2026-03-28 19:39:11 -07:00
test_rules.py
test_sagemaker_nova_integration.py feat: add sagemaker_nova provider for Amazon Nova models on SageMaker (#21542) 2026-03-14 15:10:01 -07:00
test_sagemaker.py Fix test_completion_sagemaker_messages_api retry flakiness 2026-03-15 13:57:50 -07:00
test_scheduler.py fix(scheduler): remove orphan entries from queue - causing memory leak. (#20866) 2026-02-10 22:34:52 -08:00
test_secret_detect_hook.py
test_spend_calculate_endpoint.py test fix 2025-09-01 17:04:47 -07:00
test_stream_chunk_builder.py Fix flaky test_stream_chunk_builder_openai_audio_output_usage 2026-03-09 17:18:00 -07:00
test_streaming.py test: cleanup tests 2026-03-30 16:20:01 -07:00
test_supabase_integration.py
test_team_config.py
test_text_completion.py [LLM Translation] Fix Realtime API endpoint for no intent (#13476) 2025-08-14 16:24:14 -07:00
test_timeout.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_together_ai.py
test_tpm_rpm_routing_v2.py test: test 2026-03-28 19:17:38 -07:00
test_traceloop.py
test_ui_sso_helper_utils.py
test_unit_test_caching.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_update_spend.py fix(tests): skip remaining real prisma DB tests in CI and related test suites 2026-02-20 13:25:42 -03:00
test_validate_environment.py
test_wandb.py
user_cost.json
vertex_ai.jsonl
vertex_batch_completions.jsonl
vertex_key.json test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
whitelisted_bedrock_models.txt Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00