Commit Graph

22511 Commits

Author SHA1 Message Date
Ishaan Jaff
52ef96261f
[UI] Add Deepgram provider to supported providers list and mappings (#11634)
* Add Deepgram provider to supported providers list and mappings

* add logo

* Add deepgram to model cost map

* ui - require api key for deepgram

* fix logo path

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-11 12:12:12 -07:00
Ishaan Jaff
aeaf2d185a
fix inference endpoints (#11630) (#11631)
Co-authored-by: célina <hanouticelina@gmail.com>
2025-06-11 10:45:54 -07:00
Krrish Dholakia
ec52600f98 test: handle fireworks ai instability 2025-06-11 10:09:28 -07:00
Krrish Dholakia
e4ac1cdef2 build(model_prices_and_context_window.json): fix o3-pro mode to 'responses' 2025-06-11 09:08:58 -07:00
tanjiro
c002e4cd4e
admin can reset password for user (#11618) 2025-06-11 08:55:36 -07:00
Krrish Dholakia
0d766808e9 bump: version 1.72.4 → 1.72.5 2025-06-11 08:50:13 -07:00
Ganesh J.
ee81ca9ce0
Fix: passes api_base, api_key, litellm_params_dict to custom_llm embedding methods (#11450)
* fix: passing litellm_params_dict instead of litellm_params which is a list of strings

* updated custom llm embedding params and litellm_params_dict

---------

Co-authored-by: Ganesh Jagadeesan <cjagadeesan@dataminr.com>
2025-06-11 08:29:09 -07:00
Krish Dholakia
39de3610be
fix(internal_user_endpoints.py): support user with + in email on us… (#11601)
* fix(internal_user_endpoints.py): support user with `+` in email on user info

ensures user is correctly parsed from input

* fix(factory.py): support vertex function call args as None

handles empty string in args for vertex gemini calls

* docs(langfuse_integration.md): pin langfuse sdk version on docs

* fix(vertex_ai/): return empty dict, instead of none when empty string given

* refactor: reduce function size

* fix: fix linting errors

* fix: revert check

* fix(internal_user_endpoints.py): fix check

* test: update tests

* test: update tests
2025-06-10 22:13:10 -07:00
Krish Dholakia
3b7f1d563d
Add VertexAI Anthropic passthrough - cost calculation, token tracking (#11611)
* fix(vertex_ai/anthropic): extract vertex credentials from correct dict

Fixes passing credentials for vertex anthropic via `/v1/messages`

* fix(vertex_ai/anthropic): working e2e vertex ai anthropic messages via `/v1/messages`

Ensures vertex anthropic models can be called via /v1/messages

* fix(vertex_passthrough_logging_handler.py): fix vertex anthropic logging to emit the model name on spend logs

enables observability of passthrough route

* fix(vertex_passthrough_logging_handler.py): don't redefine variable

* fix(vertex_passthrough_logging_handler.py): log cost + token usage for anthropic passthrough requests on vertex ai
2025-06-10 21:26:46 -07:00
Krish Dholakia
01ce21667c
Fix calling vertex anthropic via /v1/messages + log anthropic model name on vertex passthrough
* fix(vertex_ai/anthropic): extract vertex credentials from correct dict

Fixes passing credentials for vertex anthropic via `/v1/messages`

* fix(vertex_ai/anthropic): working e2e vertex ai anthropic messages via `/v1/messages`

Ensures vertex anthropic models can be called via /v1/messages

* fix(vertex_passthrough_logging_handler.py): fix vertex anthropic logging to emit the model name on spend logs

enables observability of passthrough route

* fix(vertex_passthrough_logging_handler.py): don't redefine variable
2025-06-10 21:08:20 -07:00
Ishaan Jaff
6f2d28c8af fix naming 2025-06-10 17:27:46 -07:00
Ishaan Jaff
3ecda03748 bump: version 1.72.3 → 1.72.4 2025-06-10 17:27:04 -07:00
Ishaan Jaff
30a8a80a65 ui new build 2025-06-10 17:26:48 -07:00
Ishaan Jaff
c9ade3a3a2
[UI] Polish New MCP Server Add Form (#11604)
* fixes MCP new form

* refactor existing logic

* docs add MCP on ui
2025-06-10 17:20:17 -07:00
Krish Dholakia
56f481a47e
Add new o3 models pricing (#11606)
* build(model_prices_and_context_window.json): add o3-pro pricing

* build(model_prices_and_context_window.json): add updated o3 model pricing

* build(model_prices_and_context_window.json): add new o3-pro model version
2025-06-10 16:33:11 -07:00
Ishaan Jaff
4dc9626fd5
[Feat] New LLM API Endpoint - Add List input items for Responses API (#11602)
* (feat) add list_input_items

* add alist_input_items to router

* add GET input_items for responses API

* test_basic_openai_list_input_items_endpoint

* TestTransformListInputItemsRequest

* test_ensure_initialize_azure_sdk_client_always_used
2025-06-10 15:47:16 -07:00
Ishaan Jaff
9058354ced
[Feat] UI - Add controls for MCP Permission Management (#11598)
* init commit to render MCP servers

* allow adding MCP servers to keys

* fix allow adding MCP permissions per team

* ui mcp permissions

* updates based on review
2025-06-10 15:46:43 -07:00
Ishaan Jaff
2d0ea74cf4
[Bug Fix] No module named 'diskcache' (#11600)
* (build) show clear error when disk cache does not exist

* docs disk cache

* add caching to pyproject
2025-06-10 14:54:11 -07:00
Ishaan Jaff
55cd5f096c
[Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout (#11594)
* Add keepalive timeout option for uvicorn server configuration

* docs Keepalive Timeout

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-10 13:30:19 -07:00
dhs-shine
01dd518201
(fix:exception_mapping_utils.py) fix sglang rate limit error issue (#11575) 2025-06-10 13:06:57 -07:00
Cole McIntosh
74bf90164c
Merge pull request #11588 from colesmcintosh/add-magistral-sm-md
Add support for new Mistral Magistral models (magistral-medium-2506 and magistral-small-2506)
2025-06-10 13:55:13 -06:00
Krrish Dholakia
fb78822dc3 fix(proxy/_types.py): add budget reset at to litellm budget full table 2025-06-10 09:51:17 -07:00
Yan
8f62b1068a
Update README.md (#11586)
It will not work if followed exactly like in readme, because .env will contain just one parameter. Fixing that
2025-06-10 09:32:11 -07:00
Krrish Dholakia
798a5c0a20 refactor: remove bad tests - mixes docker testing with code testing 2025-06-10 09:28:16 -07:00
Konstantin Lapine
62e6cb315b
Pangea/kl/udpate readme (#11570)
* chore(pangea-guardrail): Fix typo in debug message.

* docs(pangea-guardrail): Fix YAML example in pangea.md (README)."

* docs(pangea-guardrail): Update pangea.md (README).

* chore(pangea-guardrail): Format with Black.
2025-06-10 08:29:12 -07:00
Krrish Dholakia
d365f2fa08 fix(proxy/_types.py): fix doc checks 2025-06-10 08:24:53 -07:00
Cole McIntosh
3919b64209 Add new Mistral models to pricing and context window JSON: add 'mistral/magistral-medium-2506' and 'mistral/magistral-small-2506' with token limits and cost details 2025-06-10 08:38:24 -06:00
Krish Dholakia
25c0d39307
Add VertexAI claude-opus-4 + Assign users to orgs on creation (#11572)
* build(model_prices_and_context_window.json): add 'claude-opus-4' on vertexai (no @)

* build(model_prices_and_context_window.json): add claude sonnet 4 without 2

*@

* feat(internal_user_endpoints.py): assign user to orgs on user creation

allows user to be a member of orgs on creation - work to enable default orgs on UI

* fix(internal_user_endpoints.py): fix http_request
2025-06-09 23:24:06 -07:00
Krish Dholakia
c569056ea8
Show remaining users on UI (#11568)
* docs(deploy.md): move docker recommendation to `main-stable`

* feat(enterprise/internal_user_endpoints.py): expose endpoint for checking available premium users

* feat(usage_indictor.tsx): add new element to help track remaining premium users

* feat(usage_indicator.tsx): show premium user remaining usage

allows users with user caps to know how much is left

* fix(vertex_and_google_ai_studio_gemini.py): bubble up stream is not finished, even if stop reason is given

prevents early completion of stream

Closes https://github.com/BerriAI/litellm/issues/11549

* fix(streaming_handler.py): respect is_finished = False in hidden params

internal logic for preventing ending stream early

* fix(litellm_license.py): add function to check if user is over limit

* fix(internal_user_endpoints.py): add function to check if user is over limit

* refactor: move test

* docs(customer_endpoints.py): document new param
2025-06-09 22:04:45 -07:00
Low Jian Sheng
ac30462270
Fix gemini tool call indexes (#11558)
* fix index in gemini tool calls

* pass in the right index

* remove print

* add unit test for _transform_parts
2025-06-09 19:07:36 -07:00
Ishaan Jaff
c6d0878160
[Feat] Add Lasso Guardrail to LiteLLM (#11565)
* Feature/lasso guardrail (#9002)

* first version of lasso guardrail in litellm

* update to the new Lasso API

* change  prod api_base and kill the request when lasso detect issue.

* change test for now api, local test pass

* add async tests

* all tests pass

* add docs for the new lasso guardrail

* Remove support for modes other than pre_call in Lasso guardrail

* code structure and naming

* only pre_call docs

* fix lint errors

* move test to the new location  follows the same directory structure as litellm/.

* add lasso guard

* docs lasso docs

* add lasso guardrail

* fix lasso guardrail

---------

Co-authored-by: oroxenberg <oro@lasso.security>
2025-06-09 18:47:26 -07:00
Ishaan Jaff
9241fca2f5
Fix: Adds support for choosing the default region based on where the model is available (#11566)
* fix: vtx default region for global only models

* track gemini-2.5-pro-preview-05-06

* fix is_global_only_vertex_model

* test_is_global_only_vertex_model

* test_get_vertex_region_global_only_model

* fix json format

* fix get_supported_regions
2025-06-09 18:29:44 -07:00
Krrish Dholakia
230dd70604 docs(data_security.md): data_security.md
update to indicate litellm does have soc 2 type2
2025-06-09 17:53:11 -07:00
Marc Abramowitz
3bd36238dc
Simplify management_cli.md CLI docs (#10799)
Offer just 1 easy option for installing with `uv tool` so people can get
started quickly
2025-06-09 17:38:06 -07:00
Marc Abramowitz
ad9b79259f
Make all commands show server URL (#10801)
* Make all commands show server URL

so users know which server they're accessing

* Write msg to stderr & only if it's a tty
2025-06-09 17:37:40 -07:00
Ishaan Jaff
cd8ec4556f
[Feat] Add reasoning_effort support for perplexity models (#11562)
* fix: add reasoning_effort for pplx

* docs pplx reasoning

* [tests] add mock tests for pplx reasoning (#11564)

* Add tests for Perplexity reasoning models and effort parameter

* tests perplexity reasoning effort

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* test pplx reasoning effort

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-09 17:07:31 -07:00
Ishaan Jaff
9b87fde633
[Bug fix]: aiohttp fixes for transfer encoding error on aiohttp transport (#11561)
* fix: fixes for transfer encoding error on aiohttp transport

* Update tests/test_litellm/llms/custom_httpx/test_aiohttp_transport.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-09 16:14:34 -07:00
Cole McIntosh
322aefb97d
Update documentation for configuring web search options in config.yaml (#11537)
* Update web_search.md to include new supported providers and models, enhance web search options, and improve documentation for using web search with various AI models.

* Update LiteLLM version in web_search.md to reflect the latest stable release.

* Fix formatting in web_search.md for model declaration consistency.

* docs(web_search.md): add configuration options for web search in config.yaml

This update introduces sections for setting default and custom web search options in the proxy config file, including examples for different models and context sizes. A note clarifies that users can override these settings in API requests.
2025-06-09 15:19:57 -07:00
fengbohello
400acd4297
docs: fix database_url config description (#11547) 2025-06-09 15:18:37 -07:00
Cole McIntosh
abe4c8fe4c
feat: add gpt-4o-audio-preview model configuration to model_prices_and_context_window.json (#11560) 2025-06-09 14:56:36 -07:00
Cole McIntosh
7f44d1a2f8
Merge pull request #11538 from colesmcintosh/feat/add-cursor-to-gitignore
feat: add .cursor to .gitignore
2025-06-08 15:50:09 -06:00
Cole McIntosh
f1e3609296 feat: add .cursor to .gitignore 2025-06-08 14:35:50 -06:00
Laurien
0c50f8bcc9
Update enduser spend and budget reset date based on budget duration (#8460) 2025-06-08 08:39:14 -07:00
Krrish Dholakia
9f15907697 bump: version 1.72.2 → 1.72.3 2025-06-07 20:50:33 -07:00
Krish Dholakia
8dd8615a54
Ensure consistent 'created' across all chunks + set tool call id for ollama streaming calls (#11528)
* fix(streaming_handler.py): maintain same 'created' across all chunks

Fixes https://github.com/BerriAI/litellm/issues/11437

* test: add unit test to ensure created is always the same across all chunks

* fix(types/utils.py): set a tool call id, if missing in delta tool call

Ensures stream chunk builder can reconstruct tool calls correctly

Fixes https://github.com/BerriAI/litellm/issues/11262

* fix(responses/transformation.py): support passing mcp server tool call to anthropic

allows switching between openai and anthropic for mcp tool calling

* fix(ollama/chat/transformation.py): set tool call id's when missing
2025-06-07 20:50:07 -07:00
Krish Dholakia
2654d3b0b1
Support env var vertex credentials for passthrough + ignore space id on watsonx deployment (throws Json validation errors) (#11527)
* fix(router.py): support vertex credentials set in env var for passthrough

Closes https://github.com/BerriAI/litellm/issues/11245

* fix(watsonx/common_utils.py): do not pass space_id on watsonx deployment requests - raises Json validation error

Fixes https://github.com/BerriAI/litellm/issues/10941

* test: update unit test
2025-06-07 20:31:05 -07:00
Krrish Dholakia
7acaa08ad4 refactor(parallel_request_limiter_v3.py): remove redundant code - make it easier for users to understand what's happening 2025-06-07 19:52:36 -07:00
Krrish Dholakia
35bc2d52a3 docs(index.md): document rate limiting improvements 2025-06-07 19:51:26 -07:00
Krrish Dholakia
ff5a52fe3f docs(index.md): reference anthropic mcp docs on release notes 2025-06-07 19:27:54 -07:00
Krrish Dholakia
483f835c65 docs(anthropic.md): document anthropic mcp tool calling support 2025-06-07 19:21:14 -07:00