Commit Graph

22747 Commits

Author SHA1 Message Date
Thakee Nathees
34bfaf5e71
Support for Ollama Structured output json_schema (#11880) 2025-06-20 23:19:53 -07:00
Erv Walter
aaa41d1e24
Update Azure o3 pricing to match OpenAI pricing ($2/$8 per 1M tokens) (#11937)
* Initial plan for issue

* Update Azure o3 pricing to match OpenAI pricing ($2/$8 per 1M tokens)

Co-authored-by: ervwalter <768790+ervwalter@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ervwalter <768790+ervwalter@users.noreply.github.com>
2025-06-20 23:17:46 -07:00
Krrish Dholakia
1e6d43e761 Squashed commit of the following:
commit 440bc027251d8180174d762d83d271d0f7b68cc5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 23:04:11 2025 -0700

    fix: fix check

commit 89a7451cb9ee26ff9f642335714dcc6f449d1fc2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 22:42:30 2025 -0700

    fix: fix test

commit 1322e3b3497e5d334fdcaa18f0cf7a98ea758df4
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 20:52:40 2025 -0700

    style: add more tooltips

commit 172738b98b7864aabcacf3334a394098b300283f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 20:51:09 2025 -0700

    feat(team_member_view.tsx): add a tooltip

commit 895eb28deb9127985e30b5e859e5bca8530951c9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:46:49 2025 -0700

    fix(teams.tsx): support setting team member budget on create

commit 003cc54a6dd0f65030c4f39a8487adc771b62e11
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:40:49 2025 -0700

    fix(team_member_view.tsx): style improvements

commit a627a044f21df788f80d92a4081212072be91632
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:40:01 2025 -0700

    fix(team_member_view.tsx): handle scientific notation in string

commit c5a3b7bd8419f6394e1b490849555d02d473baed
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:34:25 2025 -0700

    feat(team_membership_view.tsx): show team member spend + max budget on UI

commit e986d12ad5b07c676f4cac5e16745939d7473dee
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:28:06 2025 -0700

    feat(team_member_view.tsx): show team member spend + budget on team info

commit 8e398607b25f8a8f0bab41964810b5dd27c5e3f2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:18:16 2025 -0700

    feat(team_info.tsx): show team member budget on team info

commit 1f56886b5913dafefc0c00fbe741c0c9c01144a6
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:15:30 2025 -0700

    feat(team_endpoints.py): get team budget table on team info

    allows user to see max budget set for team members

commit 0a4320bbfa406c24ad32a420f82152da7bdd7323
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:10:06 2025 -0700

    feat(team_endpoints.py): return team member budget on team info

    allows ui to display this to admin / team member

commit 6a4e29f87b333ae9977e8f878960e63becd89150
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 17:57:20 2025 -0700

    fix(team_endpoints.py): support updating team budget on UI

commit 53f0fff34032977433dfe6935ce0a684a4141fd8
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 17:38:17 2025 -0700

    feat(proxy/_types.py): return team member spend

    update pydantic object to include spend

    Allows showing spend of team member within team on UI

commit ef2a1a43ecf7fecfb904042cbf47b3d56246edcb
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 16:31:42 2025 -0700

    feat(team_endpoints.py): support 'team_member_budget' param on `/team/update`

    enables budget working across all team members

commit 512999f1249b00a02a30f049a0cfa36e829ff989
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 16:20:04 2025 -0700

    test: add unit tests for default team member budget

commit 90fa3f61a2d63e12b9f3e1da9775f5c8b7294b5f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 15:37:51 2025 -0700

    feat(team_endpoints.py): support using default team member budget id, if set

    allows all team members to use the same budget id

commit acef5324b1a0935a482c71060f610c3d8823e8c3
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 15:22:30 2025 -0700

    feat(team_endpoints.py): support `team_member_budget` param on `/team/new`

    Allow creating 1 budget for all users within team (makes it easier to increase/reduce budget if needed for all team members)

commit 2e867ac70fbd8768e7c27cf3b078e6dc10e566b9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 13:45:06 2025 -0700

    fix(ui_sso.py): ensure user is added to team, if set via default internal settings

    allows users signed up via SSO to be added to default team
2025-06-20 23:11:53 -07:00
Ishaan Jaff
7504d6ecae ui new build 2025-06-20 18:37:27 -07:00
Ishaan Jaff
dc2c1122bf
[Feat] UI QA: Pass through endpoints (#11939)
* use ID for pass through management

* use id for pass through

* fix columns

* fix PassThroughInfoView

* cleanup

* working edit and delete pass through

* fix rendering id for pt row

* fixes for pt info view

* working delete pass through

* fix use NumericalInput

* fix alignment

* qa - creating pt

* show route preview

* fix show just 1 msg

* test_create_pass_through_endpoint

* fix ui linting
2025-06-20 18:35:38 -07:00
Ishaan Jaff
99d851544a
[Feat] Add Azure Codex Models on LiteLLM + new /v1 preview Azure OpenAI API (#11934)
* fix get_complete_url

* fixes _is_azure_v1_api_version

* test_azure_responses_api_preview_api_version

* TestAzureResponsesAPIConfig

* add azure/codex-mini

* fix azure/codex-mini

* Update litellm/llms/azure/responses/transformation.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix linting

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-20 18:08:44 -07:00
Ishaan Jaff
75298af605
[Bug Fix] Cost tracking and logging via the /v1/messages API are not working when using Claude Code (#11928)
* add test_anthropic_messages_litellm_router_streaming_with_logging to base tests

* move test

* fixes for base ant tests

* working bedrock ant logging

* use BaseAnthropicMessagesStreamingIterator

* use common iterator for messages streaming

* TestAnthropicDirectAPI

* test_anthropic_claude3_transformation.py

* fix code QA checks

* fix logging for anthropic messages in SLP

* fix TestAnthropicOpenAIAPI

* remove hard coded usage for adapter

* test_anthropic_messages_litellm_router_streaming_with_logging
2025-06-20 18:08:35 -07:00
Ishaan Jaff
931b2e4875
[Bug Fix] Fix model_group tracked for /v1/messages and /moderations (#11933)
* fixes _get_router_metadata_variable_name

* fixes _update_kwargs_before_fallbacks

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* test_moderations_api_logging

* fix _pass_through_moderation_endpoint_factory
2025-06-20 14:51:50 -07:00
frank
0d486120bc
add ciphers in command and pass to hypercorn for proxy (#11916)
Signed-off-by: frankzye1 <frankzye@qq.com>
2025-06-20 14:45:48 -07:00
John Tong
c36d0f667b
Feat/add delete callback (#11654)
* feat(delete callback): add api and ui for deleting logging callbacks

* hotfix(delete callback): hotfix lint error

* move test case to test_litellm folder

* fix test case to use mock prisma client
2025-06-20 13:50:37 -07:00
Ishaan Jaff
2a698062e3
[Feat] UI Allow testing /v1/messages on the Test Key Page (#11930)
* Add ANTHROPIC_MESSAGES to chat UI

* add anthropic-ai/sdk

* add anthropic messages to UI

* add makeAnthropicMessagesRequest

* fixes for endpoint selector
2025-06-20 13:31:24 -07:00
Ishaan Jaff
cfee865f16 fix test MCP 2025-06-20 12:09:45 -07:00
Ishaan Jaff
19f13c842f
add gemini-2.5-pro (#11927) 2025-06-20 11:29:06 -07:00
Krish Dholakia
cf83b541e5
Volcengine - thinking param support + Azure - handle more gpt custom naming patterns (#11914)
* fix(volcengine.py): add thinking param support

Closes https://github.com/BerriAI/litellm/issues/11879

* fix(gpt_transformation.py): handle azure custom names - e.g. `gpt-4-1`

Closes https://github.com/BerriAI/litellm/issues/11834
2025-06-20 09:40:33 -07:00
Cole McIntosh
7abece4ad8
Merge pull request #11899 from colesmcintosh/add-health-check-success-modal
Add success modal for health check responses
2025-06-20 08:33:39 -06:00
Cole McIntosh
0ca70bf92e Merge remote-tracking branch 'origin/main' into add-health-check-success-modal 2025-06-20 08:05:36 -06:00
Krrish Dholakia
957df8d38c bump: version 1.72.8 → 1.72.9 2025-06-19 22:35:01 -07:00
Krish Dholakia
308e82d885
LiteLLM SDK <-> Proxy improvement (don't transform message client-side) + Bedrock - handle qs:.. in base64 file data + Tag Management - support adding public model names (#11908)
* fix(factory.py): handle qs:.. in mime type

Fixes https://github.com/BerriAI/litellm/issues/11839

* feat(litellm_proxy/): don't transform messages client-side

leave litellm proxy messages untouched - allow proxy to handle transformation

 prevents double transformation

* feat(tag_management_endpoints.py): support adding models to tag by adding model_name

Closes https://github.com/BerriAI/litellm/issues/11884

* test(test_tag_management_endpoints.py): add unit tests for adding new model by public model name

* test: update test
2025-06-19 22:34:18 -07:00
Krish Dholakia
0e6f1c089d
fix(proxy_server.py): fix loading ui on custom root path (#11912)
Closes https://github.com/BerriAI/litellm/issues/11865
2025-06-19 22:29:01 -07:00
Krish Dholakia
40cc61c8f3
build(model_prices_and_context_window.json): mark all gemini-2.5 mode… (#11907)
* build(model_prices_and_context_window.json): mark all gemini-2.5 models as supporting pdf input

Closes https://github.com/BerriAI/litellm/issues/11881

* fix(anthropic_transformation.py): set custom llm provider custom property

Fixes https://github.com/BerriAI/litellm/issues/11861

* test: add unit test for checking supports_reasoning

* test: add test for vertex ai flow

* feat(bedrock/anthropic): ensure thinking param correctly passed for bedrock/invoke
2025-06-19 21:07:25 -07:00
Ishaan Jaff
a1abb43319 bump: version 1.72.7 → 1.72.8 2025-06-19 20:44:57 -07:00
Ishaan Jaff
72f73cb56e ui new build 2025-06-19 20:39:28 -07:00
Ishaan Jaff
f6728e6e13
[UI] QA Items for adding pass through endpoints (#11909)
* ui - fix 1

* fixes

* fix path prefix

* fix path
2025-06-19 20:37:53 -07:00
Nathan Brake
1c4fdb4a8f
Enhance Mistral API: Add support for parallel tool calls (#11770)
* Enhance Mistral API: Add support for parallel tool calls and refine name handling in tool messages. Plus, introduce a new test for parallel tool calls in the Mistral model.

* tests

* make mypy happy

* Refine name handling in Mistral chat transformation: clarify conditions for removing the 'name' field based on message role and content.
2025-06-19 20:12:39 -07:00
Victor Krylov
1d58fc5429
Add deployment annotations (#11849)
* Add deployment annotations

* Correct the indent and simplify if 0 annotations
2025-06-19 20:11:31 -07:00
Rinor Maloku
35b791dc49
bumps the anthropic package (#11851) 2025-06-19 20:11:04 -07:00
lgruen-vcgs
e1c77e70c2
Add AWS Bedrock profiles for the APAC region (#11883)
Likely fixes #6905, #9228, and addresses https://github.com/BerriAI/litellm/issues/11057#issuecomment-2903257768.
2025-06-19 20:10:04 -07:00
Ishaan Jaff
d4b34549bc
[Fix] Networking - allow using CA Bundles (#11906)
* fix _get_ssl_context

* fixes for using HTTP handler
2025-06-19 20:09:08 -07:00
Ishaan Jaff
b90d3ca8d0
[Feat] MCP - Allow connecting to MCP with authentication headers + Allow clients to specify MCP headers (#11890) (#11891)
* initial mcp auth with special header (#11890)

Co-authored-by: wagnerjt <wagnerjt@github.com>

* add mcp auth header

* fixes MCP client for litellm proxy

* fixes loc of MCP types

* fixes use MCP client for auth to MCPs

* fix organization

* fix mcp auth header

* add MCP auth header to litellm auth

* fixes for MCP auth

* Add MCP auth to list tools

* fix MCP call tool

* fixes for MCP auth header

* tests for MCP transport

* TestMCPClientUnitTests

* docs MCP auth

* fix types

* docs fix

* fix MCP auth import

* fix code qa check

* test fix mcp auth token check

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>
2025-06-19 20:07:08 -07:00
Pascal Lim
ad2e2302e2
feat: add workload identity federation between GCP and AWS (#10210) 2025-06-19 18:31:58 -07:00
Ishaan Jaff
8cb381b519 ci/cd new build 2025-06-19 17:27:45 -07:00
Ishaan Jaff
0fe8bf2fc2 fix gemini-2.5-flash-lite-preview-06-17 2025-06-19 17:07:34 -07:00
Ishaan Jaff
4796e9c5ba ui - fix linting issue 2025-06-19 16:43:01 -07:00
fatih akyon
5b6ba871a5
[Bug Fix] add missing flash-2.5-flash-lite for gemini provider, fix gemini-2.5-flash pricing (#11901) 2025-06-19 16:38:17 -07:00
Krrish Dholakia
3540984965 test: fix tests 2025-06-19 16:35:31 -07:00
Cole McIntosh
677ef3cdde
Fix health check UI table design - Remove table-in-table design from HealthCheckComponent - Change wrapper from card styling to simple div with mb-6 spacing - Remove padding around table container for cleaner layout - Add proper icon-based actions in health check table - Use PlayIcon for new checks and RefreshIcon for re-running checks - Add loading animation with dots during health checks - Include proper tooltips for action buttons (#11897) 2025-06-19 16:07:43 -07:00
Cole McIntosh
d2e54944a3
Fix clickable model ID in health check table - Add setSelectedModelId prop to make blue model ID buttons clickable - Fix checkbox selection logic to use model names consistently - Add stopPropagation to prevent unwanted sort triggers on checkbox clicks - Now clicking Model ID opens model details, and select all works properly (#11898) 2025-06-19 16:07:23 -07:00
Ishaan Jaff
87c2be93c7
[Feat] V2 Add Pass through endpoints on UI (#11905)
* add pass through ui

* fix accordion for route path

* working route path renderer

* fix use sections

* clean up add pass through form

* docs fix add pass through routing

* clean up route preview

* add route preview
2025-06-19 16:05:29 -07:00
Krrish Dholakia
753ec7298a fix(internal_user_endpoints.py): fix check 2025-06-19 15:53:28 -07:00
Krish Dholakia
c6d7d5bc98
Revert "move buttons to the left (#11771)" (#11904)
This reverts commit bc9f111ee8.
2025-06-19 15:36:47 -07:00
Krrish Dholakia
b05742e814 fix(prometheus.py): fix metrics 2025-06-19 15:30:06 -07:00
Krrish Dholakia
649636b26b build(model_prices_and_context_window.json): ensure tpm/rpm limits are int
Closes https://github.com/BerriAI/litellm/issues/11882
2025-06-19 14:58:16 -07:00
Cole McIntosh
f892439671 Add success modal for health check responses - Add green info icon next to healthy status badges - Click icon to view full health check response including headers - Show rate limit headers and other response metadata - Store successResponse in health status state - Add showSuccessModal functionality to view JSON response details - Restore ability to inspect successful health check responses 2025-06-19 15:33:26 -06:00
Ishaan Jaff
cda759c8e7
[Bug Fix]: Fix gemini - web search error with responses API (#11894)
* feat - add websearch tools to responses to chat transform

* test_basic_google_ai_studio_responses_api_with_tools

* fix web search to responses api

* linting fixes
2025-06-19 14:11:35 -07:00
Ishaan Jaff
29bf89cf9c
fix(vertex_ai): Handle missing tokenCount in promptTokensDetails (#11… (#11896)
* fix(vertex_ai): Handle missing tokenCount in promptTokensDetails (#11581)

This PR is a Solution to the Error converting to a valid response block='tokenCount'. File an issue if litellm error - https://github.com/BerriAI/litellm/issues

It's happening because vertex_ai is not sometimes sending the token count for the audio modality.

* test_vertex_ai_usage_metadata_missing_token_count

---------

Co-authored-by: Nishith Jain <167524748+KingNish24@users.noreply.github.com>
2025-06-19 13:54:02 -07:00
Ishaan Jaff
08b2b4f5f5
[Feat] Enable Tool Calling for meta_llama (#11895)
* Enable Tool Calling for `meta_llama` (#11825)

* feat: enable tools and function_call features

* fix: ignore pydantic warnings for StreamingChoices from llama-api

* docs: add tool calling examples

* docs: change default models to Maverick

* docs: fix output of tool use

* test_map_openai_params

---------

Co-authored-by: Young Han <110819238+seyeong-han@users.noreply.github.com>
2025-06-19 13:44:22 -07:00
Ishaan Jaff
e1764af890 fix meta_llama/Llama-3.3-8B-Instruct 2025-06-19 13:44:05 -07:00
Ishaan Jaff
c975355d47
[Feat] UI - Add Allowed MCPs to Creating/Editing Organizations (#11893)
* Add MCP Org controls to UI

* Add allowed MCPs for Orgs
2025-06-19 12:50:09 -07:00
Ishaan Jaff
b30bd155d3 fix pip 2025-06-19 12:29:21 -07:00
Ishaan Jaff
e90473c40e v1.72.6-stable 2025-06-19 12:21:16 -07:00