* feat(parallel_request_limiter_v2.py): add sliding window logic
allows rate limiting to work across minutes
* fix(parallel_request_limiter_v2.py): decrement usage on rate limit error
* fix(base_routing_strategy.py): fix merge from redis - preserve values in in-memory cache during gap b/w push to redis and read from redis
* fix(base_routing_strategy.py): catch the delta change during redis sync
ensures values are kept in sync
* fix(parallel_request_limiter_v2.py): update tpm tracking to use slot key logic
* fix: fix linting error
* test: update testing
* test: update tests
* test: skip on rate limit or internal server errors
* test: use pytest fixture instead
* test: bump mistral model
* fix(utils.py): support non default params for audio transcription
allows passing provider specific params straight through on transcription calls
* fix(gpt_transformation.py): fix o_series model routing
call _transform_request on async event
* refactor: refactor tests
* test(test_azure_chat_o_series_transformation.py): add unit test for azure o series error
* test: update test
* test: update json
* fix: fix mutiple keyword error
* fix(helicone.py): add helicone api base support
Fixes https://github.com/BerriAI/litellm/issues/10825
* test: add unit test for cache hit response on embedding calls
* fix(caching_handler.py): fix handling cache hit on embedding when input is string
Fixes LIT-197
* docs(helicone_integration.md): document new helicone api base param
* fix: init commit for object permissions
* fix: init commit for object permissions
* fix: add vector_store_id to permissions
* fix vector store selector
* feat:add vector store permission mgmt
* feat: ui add allowed vector stores dropdown
* feat: add new vector store object permissions
* testing: key mgmt
* fix: stor vector store permissions on team
* ui select vector store for teams
* ui add vector store settings for orgs
* feat: allow setting org vector store permissions
* test: adding team permissions for vector stores
* feat: MCP Servers with CRUD operations (#10699)
* feat: mcp CRUD operations with authn/authz
* feat: mcp server UI
* mcp server page with overview, mcp tools, and settings page
* Adding MCP Server flow
* prisma generate before test
* UI callbacks add/remove with api server refetch
* test fix: poetry run prisma
* feat: mcp server db and config connection
* fix: MCPTool filter on description when not present
* feat: mcp on UI and integrated with list tools
* feat: Update mcp server endpoint
* tests: Unit and integration tests for mcp management endpoints
* fix: docs and ensuring global_mcp_manage up to date
* ui: remove the mcp tools view
* fix: ruff lint
* fix: unit -> integration test area
* fix(ui): remove left nav menu of previous tools
---------
Co-authored-by: wagnerjt <wagnerjt@github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* fix: sync DB MCP tools with in memory
* fix: sync DB MCP tools with in memory
* fix: stop using prisma.models
* fix: code qa check
* fix: import MCP
* fix: code QA checks
* fix: code QA checks
* fixes - only list tools for the specific MCP server
* fix: only list MCP tools for selected server
* fix linting error
---------
Co-authored-by: Tyler Wagner <wagnerjt@users.noreply.github.com>
Co-authored-by: wagnerjt <wagnerjt@github.com>
* added support for custom scope in get_azure_ad_token_provider
* if AZURE_FEDERATED_TOKEN_FILE not set, use azure_token_provider to retrive token with the oidc audiances as scope
* fix bug where oidc audience that contains "/" won't be extract correctly
* added tests for get_secret with oidc
* moved tests to litellm tests folder
* tes file naming aligned with source code
* renamed test_main because it caused issue in the test in github workflow
* updated docs
* moved docs to the end of file
* fix aws region in example config
* renamed test file
* added support for custom scope in get_azure_ad_token_provider
* if AZURE_FEDERATED_TOKEN_FILE not set, use azure_token_provider to retrive token with the oidc audiances as scope
* fix bug where oidc audience that contains "/" won't be extract correctly
* added tests for get_secret with oidc
* moved tests to litellm tests folder
* tes file naming aligned with source code
* renamed test_main because it caused issue in the test in github workflow
* updated docs
* moved docs to the end of file
* fix aws region in example config
* renamed test file
* fix merge conflict resolution error
* added support for custom scope in get_azure_ad_token_provider
* if AZURE_FEDERATED_TOKEN_FILE not set, use azure_token_provider to retrive token with the oidc audiances as scope
* fix bug where oidc audience that contains "/" won't be extract correctly
* added tests for get_secret with oidc
* moved tests to litellm tests folder
* tes file naming aligned with source code
* renamed test_main because it caused issue in the test in github workflow
* updated docs
* moved docs to the end of file
* fix aws region in example config
* renamed test file
* added tests for get_secret with oidc
* moved tests to litellm tests folder
* tes file naming aligned with source code
* renamed test_main because it caused issue in the test in github workflow
* updated docs
* moved docs to the end of file
* renamed test file
* fix merge conflict resolution error
* fix(anthropic/chat/handler.py): Fixes https://github.com/BerriAI/litellm/issues/10328
Adopts changes from https://github.com/BerriAI/litellm/pull/10329
* fix(vertex_and_google_ai_studio.py): don't set 'include thoughts' if thinking budget = 0
VertexAI raises errors
* fix(vertex_llm_base.py): new function for deciding the api base, handles 'global' api base
Fixes https://github.com/BerriAI/litellm/issues/11190
* fix(vertex_ai/partner_models): fix instrumentation for custom api base check
* refactor(vertex_ai/partner): refactor function to keep below 50 LOC
* fix(vertex_ai/gemini): remove parallel tool calls error for >1 tool - just ignore (prevent call from failing)
* fix: fix linting error
* added Pangea as a guardrail vendor
* Adding output recipe, cleaning up some imports
* Add Pangea guardrails tests
* Add docs and sidebar
* Move to use async_precall_hook instead of moderation hook
* Update to "new" format (accept mode)
Add response for redaction, support transforming request / response
based off the original type of call (/v1/completions,
/v1/chat/completions)
* Fix tests
* Fix unused imports
* Fix .md
---------
Co-authored-by: michael weinberger <michael.lee.weinberger@gmail.com>
* fix(vertex_and_google_ai_studio_gemini.py): handle both camel case and underscores in the tool for vertex ai code execution
support vertex ai code execution
* docs(vertex.md): add code execution example to vertex ai
* fix(vertex_ai/common_utils.py): when anyof in field, just select anyof - don't include other k,v pairs - vertex throws error
Fixes https://github.com/BerriAI/litellm/issues/11164
* fix(common_utils.py): add title field inside anyof - to retain some description
Addresses https://github.com/BerriAI/litellm/issues/11164#issuecomment-2914728385
* feat(codestral/completion): return litellm latency overhead for codestral
enables easier debugging of latency issues
* fix(types/utils.py): support _response_ms on hidden params model dump
Fixes issue where 'x-litellm-overhead-duration-ms' wasn't being returned on text c
ompletion calls
* fix(types/utils.py): add '__contains__' support for chatcompletiondeltatool call
Fixes https://github.com/BerriAI/litellm/issues/7099
* fix: fix linting error
* fix: fix linting error
* fix(utils.py): prevent leaking sensitive keys to langfuse
Fixes https://github.com/BerriAI/litellm/issues/11150
* test(langfuse/): unit test preventing future bedrock key leaks
Fixes https://github.com/BerriAI/litellm/issues/11150
* test(test_langfuse_e2e_test.py): add unit test for vertex - make sure no key leaks occur
* ci(test-litellm.yml): add pytest retry to github workflow
* fix(proxy_server.py): support forwarding `/sso/key/generate` to the server root path url
Fixes https://github.com/BerriAI/litellm/issues/10761
* fix(proxy_server.py): don't rewrite absolute path (PROXY_BASE_URL) with relative path (SERVER_ROOT_PATH)
This causes issues when using a custom path with sso, when doing redirects
* fix(utils.py): ignore token - will mistakenly redact 'max_tokens' as well
* fix: AiohttpResponseStream transport
* fix: use AiohttpResponseStream transport by default
* fix: AiohttpResponseStream transport
* fixes: mapping aiohttp exceptions
* fixes: aiohttp rollout
* fixes: add support ssl_verify for aiohttp
* fixes: add support ssl_verify for aiohttp
* fixes: remove duplicates
* feat: add video support for bedrock converse api (#11043)
* fixes: bedrock add video support
* fixes: bedrock add video support
---------
Co-authored-by: yytdfc <fuchen@foxmail.com>
* feat: Make gemini accept the openai parameter parallel_tool_calls
When mapping, allow the parameter: True because that is the
intrinsic behavior of Gemini. Allow False, but reject if there
are multiple tools because there's no actual equivalent in Gemini.
fixes#9686
ref: issues/9686
* chore: cleanup and move test_vertex.py down to tests/litellm
as suggested in https://github.com/BerriAI/litellm/pull/11125#discussion_r2105905871
* fix(auth_checks.py): set ttl on in-memory team objects
ensures objects expire
* fix(in_memory_cache.py): allow ttl override if existing ttl expired (no 'get_cache' ever called)
* fix: revert user redis changes
* fix(in_memory_cache.py): don't reset ttl, if already set for object
Allows objects to expire and db checks to occur
* test: add unit test for change
* fix(main.py): use processed non-default-params as standard input params for langfuse
Fixes https://github.com/BerriAI/litellm/issues/11072
Fixes https://github.com/BerriAI/litellm/issues/11096
* fix(main.py): rename variable to be more accurate
* test(test_langfuse_e2e_test.py): add router unit test for langfuse e2e testing
Prevent https://github.com/BerriAI/litellm/issues/11072 from happening again
* build: update lock
* fix(utils.py): refactor optional params function
make it easier to get the standardized non default params
* fix(utils.py): improve process non default params function
* fix(main.py): include provider specific params in processed non default params used in logging
ensures user can see any provider specific params on langfuse
ensures user can see any provider specific params on langfus e
