e9fb29061a
* Include model name + configured TPM/RPM in priority rate-limit 429 errors (#27215) * Include model name + configured TPM/RPM in priority rate-limit 429 errors The current 429 message ('Priority-based rate limit exceeded. Priority: prod, Rate limit type: tokens, Remaining: -664145, Model saturation: 86.3%') doesn't tell the operator which model was hit or what the configured limit is, so they can't tell whether the priority allocation needs tuning or the model TPM is just too small. Add Model, Model TPM, and Model RPM to both the priority-based 429 and the sibling Model-capacity 429 in dynamic_rate_limiter_v3._check_rate_limits. Pure error-message change — no behavior or schema impact. * test: assert priority 429 includes model name + configured TPM/RPM Adds a regression test for the new fields in the priority-based 429 detail ('Model:', 'Model TPM:', 'Model RPM:'). Verified locally that the test fails against the unpatched dynamic_rate_limiter_v3.py and passes after the patch. --------- Co-authored-by: shin-watcher <ext-agent-shin@berri.ai> * Update litellm/proxy/hooks/dynamic_rate_limiter_v3.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/hooks/dynamic_rate_limiter_v3.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: shin-watcher <ext-agent-shin@berri.ai> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| litellm | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| litellm-proxy-extras | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _flush_vcr_cache.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| eval_swe_bench.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| README.MD | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_team.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.