Krish Dholakia 4be0ec8e35 GA Multi-instance rate limiting v2 Requirements + New - specify token rate limit type - output / input / total (#11646) ... * feat(parallel_request_limiter_v3.py): allows admin to enforce token rate limit based on just output tokens Useful when trying to rate limit for primarily self hosted model use-cases * test(test_parallel_request_limiter_v3.py): add unit test for token rate limit type * feat(parallel_request_limiter_v3.py): return remaining token limits in header * feat: return rate limit headers in response * feat(parallel_request_limiter_v3.py): working rate limit response headers * feat(parallel_request_limiter_v3.py): fix rate limit tracking for tpm when rpm also set * feat(parallel_request_limiter_v3.py): show headers for key/user/team * feat(parallel_request_limiter_v3.py): decrement max parallel request limiter on failure event * feat(parallel_request_limiter_v3.py): add in-memory cache implementation of parallel request rate limiter allows rate limiter to work even without redis cache setup Work for GA of parallel request limiter v3 * refactor(proxy/hooks/__init__.py): replace with new parallel request handler * test: update testing * fix: fix ruff check * fix: revert ga of multi instance rate limiting - needs more work to pass testing		2025-06-11 22:05:13 -07:00
..
anthropic_endpoints
auth	Show remaining users on UI (#11568)	2025-06-09 22:04:45 -07:00
client
common_utils	Update enduser spend and budget reset date based on budget duration (#8460)	2025-06-08 08:39:14 -07:00
db	build(VLLM-Passthrough-with-loadbalancing-support-(enables-using-model-list-for-VLLM-/classify-endpoint)): Closes #11205	2025-05-31 09:00:04 -07:00
experimental/mcp_server
guardrails	build(VLLM-Passthrough-with-loadbalancing-support-(enables-using-model-list-for-VLLM-/classify-endpoint)): Closes #11205	2025-05-31 09:00:04 -07:00
health_endpoints
hooks	GA Multi-instance rate limiting v2 Requirements + New - specify token rate limit type - output / input / total (#11646)	2025-06-11 22:05:13 -07:00
image_endpoints	Feat: add add azure endpoint for image endpoints (#11482)	2025-06-06 10:38:37 -07:00
management_endpoints	fix(internal_user_endpoints.py): support user with + in email on us… (#11601)	2025-06-10 22:13:10 -07:00
middleware
openai_files_endpoint
pass_through_endpoints	Fix: Respect user_header_name property for budget selection and user identification (#11419)	2025-06-06 14:21:02 -07:00
spend_tracking
test_configs
types_utils
ui_crud_endpoints	Refactor settings response models in proxy_setting_endpoints.py	2025-06-04 15:08:05 -06:00
test_caching_routes.py
test_common_request_processing.py	[Feat] DD Trace - Add instrumentation for streaming chunks (#11338)	2025-06-02 16:48:39 -07:00
test_custom_proxy.py	Custom Root Path Improvements: don't require reserving /litellm route (#11460)	2025-06-05 16:36:47 -07:00
test_litellm_pre_call_utils.py
test_proxy_cli.py	[Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout (#11594)	2025-06-10 13:30:19 -07:00
test_proxy_server.py
test_proxy_types.py	Litellm dev 06 06 2025 p1 (#11496)	2025-06-07 09:12:16 -07:00
test_proxy_utils.py	UI / SSO - Update proxy admin id role in DB + Handle SSO redirects with custom root path (#11384)	2025-06-03 21:16:55 -07:00
test_route_llm_request.py
test_spend_log_cleanup.py	[Feat] Make batch size for maximum retention in spend logs a controllable parameter (#11459)	2025-06-05 17:11:51 -07:00
test_team_member_update.py