c42740a4b9
* refactor: comment out circuit breaker causes incorrect rate limiting in high traffic * fix(base_routing_strategy.py): don't reset value if redis val is lower than current in-memory value Fixes issue where redis might be trailing in-memory value * fix(parallel_request_limiter_v2.py): if in-memory higher than redis, don't reset value; add previous slot keys to redis increment to correctly 'get' them * fix(parallel_request_limiter_v3.py): v3 implementation of parallel request limiter does not use background redis syncing - increments redis in call simplify rate limiting logic, to improve accuracy * fix: fix ruff errors * fix(parallel_request_limiter_v3.py): don't decrement limit on post call success - causes double decrements * fix(parallel_request_limiter_v3.py): working accurate multi-instance logic ensured just 100 requests allowed on 100 users, 10 ramp up, 100 rpm limit key, 2 instances * fix(parallel_request_limiter_v3.py): working accurate rate limiting with time window resets allows rate limiting to work across multiple windows * test: add unit tests for v3 rate limiter * fix(parallel_request_limiter_v3.py): return window value into in-memory cache allows in-memory cache checks to be used correctly * refactor(parallel_request_limiter_v3.py): refactor rate limiting to work for multiple window/counter key pairs enables using for user/team/model rate limiting * feat(parallel_request_limiter_v3.py): working rate limiting, across key/user/team/end-user * fix(parallel_request_limiter_v3.py): add model specific rate limiting * fix(parallel_request_limiter_v3.py): ignore if no rate limits set skip unecessary rate limit checks - if no limits set * fix(parallel_request_limiter_v3.py): initial commit bringing token rate limits back * fix(parallel_request_limiter_v3.py): increment by value in list + update assertions to handle tokens + max parallel requests * test(parallel_request_limiter_v3.py): more testing * fix(parallel_request_limiter.py): working in-memory cache limiter * fix(redis_cache.py): ignore linting error - use safe hasattr * fix(parallel_request_limiter_v3.py): fix linting error * refactor: remove redundant parallel_Request_limiter_v2.py old / inaccurate implementation * test: update tests * style: cleanup * test: update test * docs(config_settings.md): document new env var * test(test_base_routing_strategy.py): update test |
||
|---|---|---|
| .. | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| code_coverage_tests | ||
| documentation_tests | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| litellm_utils_tests | ||
| litellm-proxy-extras | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| windows_tests | ||
| __init__.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| README.MD | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_passthrough_endpoints.py | ||
| test_ratelimit.py | ||
| test_spend_logs.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_team.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/litellm
This folder can only run mock tests.