litellm/docs/my-website/docs
ishaan-berri 0afffe4366
feat: multiple concurrent budget windows per API key and team (#24883) (#25109)
* feat: multiple concurrent budget windows per API key and team (#24883)

* feat(proxy): add BudgetLimitEntry type and wire budget_limits into key/team models

* feat(schema): add budget_limits Json column to VerificationToken and TeamTable

* feat(migrations): add migration for budget_limits column on keys and teams

* feat(keys): initialize budget_limits windows with reset_at on key create/update

* feat(teams): initialize budget_limits windows with reset_at on team create/update

* feat(auth): add _virtual_key_multi_budget_check and _team_multi_budget_check

* feat(auth): call multi-budget checks from common_checks for keys and teams

* feat(proxy): increment per-window Redis spend counters after each request

* feat(budget): reset individual budget windows on schedule via reset_budget_job

* feat(ui): add hourly option to BudgetDurationDropdown

* feat(ui): add budget_limits field to KeyResponse type

* feat(ui): add Budget Windows editor to key edit view

* feat(ui): add Budget Windows editor to create key form

* fix(proxy): strip budget_limits=None before Prisma upsert to fix login 500

Prisma rejects nullable JSON fields (Json? without @default) when passed as
Python None — it needs the field omitted entirely so the DB stores NULL via
the column's nullable constraint. This was breaking /v2/login because the UI
session key creation path hit the upsert with budget_limits=None.

* ui(key-edit): use antd InputNumber+Button for budget windows, add reset hints

* ui(create-key): use antd InputNumber+Button for budget windows, add reset hints

* docs(users): add multiple budget windows section with API + dashboard walkthrough

* fix: BudgetExceededError returns HTTP 429 instead of 400

- Add status_code=429 to BudgetExceededError class
- auth_exception_handler hardcoded code=400 → code=429

* fix: no-op else branch in multi-budget auth checks causes KeyError

- BudgetLimitEntry objects must be coerced via model_dump() not left as-is
- Move _virtual_key_multi_budget_check into common_checks (was asymmetric
  with _team_multi_budget_check which already lived there)

* fix: len() on JSON string returns char count not window count

Guard with isinstance check + json.loads() before iterating per-window
Redis counters in increment_spend_counters

* fix: silent except:pass hides Redis reset failures in reset_budget_windows

Log Redis counter reset failures as warnings so they are observable

* test: add unit tests for multi-budget window enforcement

5 tests covering: no budget_limits passes, under budget passes,
over hourly window raises 429, over monthly window raises 429,
BudgetLimitEntry objects coerced without KeyError

* fix: key per-window counters stable across reorders (duration key, not index)

* fix: team+key per-window spend increments use duration key, not index

* fix: budget window reset uses duration key; log failures instead of swallowing

* refactor: extract BudgetWindowsEditor to shared component

* refactor: key_edit_view imports BudgetWindowsEditor from shared component

* refactor: create_key_button imports BudgetWindowsEditor from shared component

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>

* fix(reset_budget_job): extract _reset_expired_window helper to fix PLR0915 too many statements

* feat(skills): Skills Registry & Hub — register skills, browse in AI Hub, public skill hub (#25118)

* feat(skills): add domain and namespace fields to plugin types

* feat(skills): store and return domain/namespace inside manifest_json

* feat(skills): add /public/skill_hub endpoint for unauthenticated access

* feat(skills): whitelist /public/skill_hub from auth requirements

* feat(skills): add domain, namespace to Plugin and RegisterPluginRequest types

* feat(skills): smart URL parser — paste github URL, auto-detect source type and name

* feat(skills): replace enable toggle with Public badge, make rows clickable

* feat(skills): add skill detail view with Overview and How to Use tabs

* feat(skills): add MakeSkillPublicForm modal for publishing skills to the hub

* feat(skills): rename panel to Skills, wire in skill detail view on row click

* feat(skills): add skill hub table columns — name, description, domain, source, status

* feat(skills): add SkillHubDashboard with stats row, domain dropdown filter, and table

* feat(skills): add Skill Hub tab to AI Hub with Select Skills to Make Public button

* feat(skills): move Skills to top-level nav item directly under MCP Servers

* feat(skills): add skillHubPublicCall and NEXT_PUBLIC_BASE_URL support

* feat(skills): add Skill Hub tab to public AI Hub page

* feat(skills): add skills page routing in main app router

* feat(skills): add /skills page route

* chore: update package-lock after npm install

* docs(skills): add Skills Gateway doc page with mermaid architecture diagram

* docs(skills): add Skills Gateway to sidebar under Agent & MCP Gateway

* docs(skills): add loom walkthrough video to Skills Gateway doc

* chore: fixes

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
2026-04-06 14:02:04 -07:00
..
adding_provider feat(guardrails): team-based guardrail registration and approval workflow (#22459) 2026-03-02 22:06:49 -08:00
anthropic_unified Merge branch 'upstream/main' into HEAD 2026-03-13 22:56:08 -03:00
caching fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
completion Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
contribute_integration
contributing merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
debugging fix: cleanup 2026-03-25 09:20:25 -07:00
embedding merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
extras fix: prompt registry 2026-02-18 00:34:54 +05:30
guides docs: sidebar updates, letta resources links, Google GenAI SDK, cost tracking order 2026-03-18 04:28:06 +05:30
integrations docs: Learn page updates, card links, integrations, sidebar changes 2026-03-18 17:19:27 +05:30
langchain
learn fix: fix azure audio test 2026-03-28 18:14:01 -07:00
observability docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
pass_through Development environment setup (#22432) 2026-02-28 14:50:06 -08:00
projects fix: prompt registry 2026-02-18 00:34:54 +05:30
provider_registration merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
providers Litellm ishaan march30 (#24887) (#25151) 2026-04-04 14:44:07 -07:00
proxy feat: multiple concurrent budget windows per API key and team (#24883) (#25109) 2026-04-06 14:02:04 -07:00
search feat(search): add Serper (serper.dev) as search provider (#23112) 2026-03-09 08:40:37 -07:00
secret_managers docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
troubleshoot docs: pip venv upgrade workflow (#23290) 2026-03-10 13:53:54 -07:00
tutorials Merge pull request #25032 from BerriAI/litellm_docs-default-team-params 2026-04-03 16:07:46 -07:00
vector_stores
a2a_agent_headers.md docs(agents): add A2A agent authentication headers guide 2026-03-05 15:11:18 +05:30
a2a_agent_permissions.md
a2a_cost_tracking.md fix: prompt registry 2026-02-18 00:34:54 +05:30
a2a_invoking_agents.md fix: prompt registry 2026-02-18 00:34:54 +05:30
a2a_iteration_budgets.md Agents - add max budget + tpm/rpm limiting per agent AND per agent session (#22849) 2026-03-07 19:12:42 -08:00
a2a.md Agents - add max budget + tpm/rpm limiting per agent AND per agent session (#22849) 2026-03-07 19:12:42 -08:00
aiohttp_benchmarks.md
anthropic_count_tokens.md Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
apply_guardrail.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
assistants.md
audio_transcription.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
batches.md fix: prompt registry 2026-02-18 00:34:54 +05:30
bedrock_converse.md
bedrock_invoke.md
benchmarks.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
budget_manager.md
contact.md
container_files.md fix: prompt registry 2026-02-18 00:34:54 +05:30
containers.md
contributing.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
count_tokens.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
data_retention.md fix: prompt registry 2026-02-18 00:34:54 +05:30
data_security.md fix: cleanup 2026-03-25 09:20:25 -07:00
default_code_snippet.md
enterprise.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
evals_api.md fix: prompt registry 2026-02-18 00:34:54 +05:30
exception_mapping.md
files_endpoints.md feat(anthropic): add Files API support for SDK 2026-03-11 12:45:19 -03:00
fine_tuning.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
generateContent.md fix: Metadata / Trace ID Missing in S3 Streaming Callbacks 2026-02-25 14:16:42 +05:30
image_edits.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
image_generation.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
image_variations.md
index.md Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
interactions.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
load_test_advanced.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
load_test_rpm.md
load_test_sdk.md
load_test.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_aws_sigv4.md Litellm ishaan march30 (#24887) (#25151) 2026-04-04 14:44:07 -07:00
mcp_control.md [Chore] update mcp documentation for header forwarding 2026-03-04 13:07:25 +05:30
mcp_cost.md
mcp_guardrail.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
mcp_oauth.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_openapi.md feat(mcp): allow admins to override tool name and description per MCP server (#22828) 2026-03-04 17:58:05 -08:00
mcp_public_internet.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_semantic_filter.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_toolsets.md Litellm ishaan march23 - MCP Toolsets + GCP Caching fix (#25146) (#25155) 2026-04-04 16:23:21 -07:00
mcp_troubleshoot.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_usage.md
mcp_zero_trust.md docs(mcp_zero_trust): add MCP zero trust auth guide (#23918) 2026-03-17 17:45:16 -07:00
mcp.md Litellm ishaan march30 (#24887) (#25151) 2026-04-04 14:44:07 -07:00
migration_policy.md
migration.md fix: cleanup 2026-03-25 09:20:25 -07:00
moderation.md
ocr.md Enable local file support for OCR (#22133) 2026-02-27 10:50:02 -08:00
oidc.md Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
old_guardrails.md
projects.md
prompt_management.md Add support for prompt management for responses 2026-03-18 15:48:31 +05:30
proxy_api.md
proxy_auth.md fix: prompt registry 2026-02-18 00:34:54 +05:30
proxy_server.md docs: remove phone numbers from readme and docs 2026-03-25 12:40:40 -07:00
rag_ingest.md fix: prompt registry 2026-02-18 00:34:54 +05:30
rag_query.md fix: prompt registry 2026-02-18 00:34:54 +05:30
realtime.md Announce April Townhall (#25021) 2026-04-02 14:10:49 -07:00
reasoning_content.md docs(reasoning_content.md): update guide 2026-03-23 17:23:14 -07:00
rerank.md feat: Add IBM watsonx.ai rerank support (#21303) 2026-02-16 20:12:16 -08:00
response_api_compact.md fix: prompt registry 2026-02-18 00:34:54 +05:30
response_api.md Merge pull request #24110 from BerriAI/Sameerlite/model-level-affinity 2026-03-20 18:05:46 +05:30
router_architecture.md
routing.md Merge pull request #24611 from Sameerlite/Sameerlite/order-fallback2 2026-03-27 20:15:30 +05:30
rules.md
scheduler.md
sdk_custom_pricing.md
secret.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
set_keys.md
skills_gateway.md feat: multiple concurrent budget windows per API key and team (#24883) (#25109) 2026-04-06 14:02:04 -07:00
skills.md
text_completion.md
text_to_speech.md fix: prompt registry 2026-02-18 00:34:54 +05:30
traffic_mirroring.md fix: prompt registry 2026-02-18 00:34:54 +05:30
troubleshoot.md docs: remove phone numbers from readme and docs 2026-03-25 12:40:40 -07:00
vector_store_files.md
vertex_batch_passthrough.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
videos.md Add new videos docs 2026-03-16 17:57:14 +05:30
wildcard_routing.md