Commit Graph

30881 Commits

Author SHA1 Message Date
Ishaan Jaff
07a17d6d6b
[Feat] Proxy CLI - dont store existing key in the URL, store it in the state param (#15290)
* Feat: CLI Auth fixes for UI SSO

* fix auth.py

* fix test ui sso.py
2025-10-07 12:36:17 -07:00
Tim Elfrink
c5eb22381d fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation
Fixes #15263

This PR fixes the cost calculation for Bedrock Anthropic models with prompt caching.

**Root Cause:**
PR #9838 incorrectly removed adding `cacheWriteInputTokens` to `prompt_tokens`
for Bedrock, based on the assumption that it would cause double counting (similar
to an Anthropic API issue). However, Bedrock's token structure is different:

- **Bedrock API**: `inputTokens`, `cacheReadInputTokens`, and `cacheWriteInputTokens`
  are ALL separate values that should be summed for total input tokens
- **Anthropic API**: Same structure - all three token types are separate

The fix in #9838 was later reverted for Anthropic (correctly re-adding
`cache_creation_input_tokens` to `prompt_tokens`), but Bedrock was never fixed.

**Changes:**
1. Re-add `cacheWriteInputTokens` to `input_tokens` in Bedrock transformation
2. Update test assertions to reflect correct behavior
3. Add regression test for prompt caching cost calculation
4. Fix typo in Anthropic transformation where `cache_creation_tokens` was
   incorrectly set to `cache_read_input_tokens`

**Testing:**
- All existing Bedrock transformation tests pass
- New test validates correct cost calculation with prompt caching
- Verified costs are non-negative and accurate
2025-10-07 20:28:46 +02:00
Sameer Kankute
51971f4750
Add gpt-realtime-mini support (#15283) 2025-10-07 11:27:04 -07:00
Sameer Kankute
e4892735f0
fix gemini cli by actually streaming the response (#15264)
* fix gemini cli by actually streaming the response

* fix cost tracking

* fix test
2025-10-07 11:26:39 -07:00
Sameer Kankute
73f96712f5
fix the reasoningresponse id (#15265) 2025-10-07 11:24:29 -07:00
Krrish Dholakia
421d38c94a build(ui/): build new ui 2025-10-07 10:48:19 -07:00
Krish Dholakia
f044eb80de
Merge pull request #15285 from BerriAI/litellm_infinity_new_provider_ui
feature: adds Infinity as a provider in the UI
2025-10-07 10:46:05 -07:00
Achintya Rajan
e2f21beb7f added Infinity as a provider in the UI 2025-10-07 10:21:18 -07:00
Sameer Kankute
c0d0424eb8
Added streaming support for response api streaming image generation (#15269) 2025-10-07 08:15:57 -07:00
xprilion
3e52079509 Add information about personal entities error 2025-10-07 20:10:38 +05:30
Krrish Dholakia
60289aa73e feat(scim_v2.py): if group.id doesn't exist, use external id 2025-10-07 07:12:45 -07:00
xprilion
888391127f Add W&B Inference documentation 2025-10-07 19:18:45 +05:30
Krrish Dholakia
faeb7484db docs(vertex.md): fix doc 2025-10-06 20:45:58 -07:00
Achintya Rajan
7ed4715a9b eliminates loading screen flash 2025-10-06 20:42:59 -07:00
Achintya Rajan
cc94fb87b5 added base URL helpers 2025-10-06 20:32:47 -07:00
Krish Dholakia
12cbac74b1
Merge pull request #15210 from uc4w6c/feat/add_global_cross_region
feat: add Global Cross-Region Inference
2025-10-06 20:21:18 -07:00
Achintya Rajan
051c9a7c61 Sidebar2 uses FF now 2025-10-06 20:17:19 -07:00
Krish Dholakia
077b5e105f
Merge pull request #15240 from BerriAI/litellm_dev_10_06_2025_p1
Azure - passthrough support with router models
2025-10-06 20:06:43 -07:00
Krish Dholakia
94a34dd53a
Merge branch 'main' into litellm_dev_10_06_2025_p1 2025-10-06 20:06:30 -07:00
Krish Dholakia
8b357c24c2
Merge pull request #15226 from otaviofbrito/chore/vertex-ai-context-caching
Chore/vertex ai context caching
2025-10-06 20:05:24 -07:00
Krish Dholakia
6b4415684e
Merge pull request #15253 from BerriAI/litellm_dev_10_06_2025_p2
fix(azure/responses): remove invalid status param from azure call + MCP - support setting CA_BUNDLE_PATH
2025-10-06 20:01:39 -07:00
Ishaan Jaff
d1d411ca2c
feat: add litellm_add_gpt_image1_mini_models (#15259) 2025-10-06 19:58:55 -07:00
Teddy Amkie
e5ca61d026
feat: add stop parameter to non-supported params for GPT-5 (#15244)
- Add 'stop' parameter to the non_supported_params list in OpenAIGPT5Config
- This ensures the stop parameter is automatically dropped when calling GPT-5 models
- Resolves issue where stop parameter was not being stripped for GPT-5
2025-10-06 19:57:21 -07:00
Sameer Kankute
8d7f39798c
Removed stop param from unsupported azure models (#15229)
* Removed stop param from unsupported model

* Use better handling for stop method

* Use better handling for stop method
2025-10-06 19:56:18 -07:00
Krish Dholakia
6e538033ed
Merge pull request #15249 from BerriAI/litellm_router_settings_update_fix
fix: Router settings do not update despite success message
2025-10-06 19:53:08 -07:00
Krish Dholakia
6bb34e5168
Merge pull request #15252 from BerriAI/litellm_view_key_pagination_calls_fix
(perf) fix: Replaces bloated key list calls with lean key aliases endpoint
2025-10-06 19:51:26 -07:00
Krish Dholakia
3f6717bfd8
Merge pull request #15250 from BerriAI/litellm_next_turbopack_for_dev
Adds turbopack to the npm run dev command in UI to build faster during development
2025-10-06 19:50:41 -07:00
Teddy Amkie
1ad4ef267d
Add GPT-5 Pro model configuration and documentation (#15258)
- Added gpt-5-pro entry to model_prices_and_context_window.json with complete specifications
- Mode: responses (Responses API only)
- Pricing: $15.00 input / $120.00 output (Standard), $7.50 input / $60.00 output (Batch)
- Context: 400,000 input tokens, 272,000 output tokens
- Endpoints: /v1/batch, /v1/responses only
- Features: Web Search, Function Calling, Vision, PDF Input, Prompt Caching, Reasoning
- No streaming support, no Code Interpreter, no Computer Use

- Added gpt-5-pro to supported models list in docs/my-website/src/pages/completion/supported.md
- Added gpt-5-pro to OpenAI provider documentation in docs/my-website/docs/providers/openai.md
- Added comprehensive GPT-5 Pro Special Notes section with usage examples and limitations
2025-10-06 19:43:40 -07:00
Achintya Rajan
eed19fb4ab slight improvement for Sidebar2 2025-10-06 19:36:40 -07:00
Ishaan Jaff
1d468a59f3
[MCP Gateway] Fine-grained Database Object Storage Control (#15255)
* add supported_db_objects

* add _should_load_db_object

* add docs on storing MCP objects in DB

* test_should_load_db_object_with_supported_db_objects

* type fix
2025-10-06 19:31:44 -07:00
Achintya Rajan
6529ed4d62 hidden dashboard routing, dev and build env files 2025-10-06 19:21:34 -07:00
Ishaan Jaff
930f88078e
[MCP Gateway] Backend - Allow storing allowed tools by team/key (#15243)
* add mcp_tool_permissions column to LiteLLM_ObjectPermissionTable

* add mcp_tool_permissions

* add LiteLLM_ObjectPermissionTable

* test key/team mgmt

* fix code QA check

* fix check_tool_permission_for_key_team

* test_key_tool_permission_allows_permitted_tool

* test_list_tools_filters_by_key_team_permissions

* fix _get_tools_from_mcp_servers

* fix logger.debug

* fix check_tool_permission_for_key_team

* fix: filter_tools_by_key_team_permissions

* fix _get_team_object_permission

* fix _set_object_permission

* fix handle_update_object_permission_common

* fix add allowed tools to keys

* test add key/team level tool permissions for MCPs
2025-10-06 18:49:32 -07:00
Krrish Dholakia
7f88a3f9c6 fix(mcp/): add ssl certificate settings for mcp clients
respect ca bundle path for mcp calls
2025-10-06 18:36:05 -07:00
Alexsander Hamir
9aa2dc2e8e
[Refactor] Utils: extract inner function from client (#15234)
* fix: remove func definition from inside client

It makes the function bigger and harder to understand, I left just the wrappers.

* fix: test_arouter_test_team_model failure

- Added fallback to the model_name to index functionality.
2025-10-06 18:30:01 -07:00
Achintya Rajan
264f1cded1
Merge branch 'main' into litellm_view_key_pagination_calls_fix 2025-10-06 18:10:57 -07:00
Krrish Dholakia
5336fcc000 fix(azure/responses): always remove status
unsupported parameter
2025-10-06 18:08:57 -07:00
=
ee00490c24 Update package.json 2025-10-06 18:05:48 -07:00
Achintya Rajan
19f0c498e5 Update general_settings.tsx 2025-10-06 17:52:13 -07:00
Krrish Dholakia
fbf7ca17ba docs(security_encryption_faq.md): answers common questions about data at rest vs. in transit 2025-10-06 17:34:10 -07:00
Ishaan Jaff
4c9bd3e0de
[MCP Gateway] UI - Select allowed tools for Key, Teams (#15241)
* fix: MCPServerSelectorProps

* Tool Permissions

* Teams: tool permissions

* Add MCP team level permissions

* fix MCP tool permissions

* allow selecting MCP tools for Keys, teams

* fix: add mcp tool level permissions by key, team

* MCP Tool permissions test
2025-10-06 14:37:43 -07:00
Achintya Rajan
b1f3553ae4 Update networking.tsx 2025-10-06 14:28:12 -07:00
Achintya Rajan
00f61b214c more window guards 2025-10-06 14:17:02 -07:00
Achintya Rajan
cee408c0c0 diagnosing next build issue 2025-10-06 14:03:53 -07:00
Krrish Dholakia
6f671d2bea feat(llm_passthrough_endpoints.py): working streaming on azure passthrough via router 2025-10-06 13:56:30 -07:00
Krrish Dholakia
d7aab1d5de test: add unit tests 2025-10-06 13:53:36 -07:00
Krrish Dholakia
6177b23ebf feat(azure/passthrough): working azure passthrough with router models for openai calls
Fixes issue where azure passthrough could not work with workload identity + litellm model name
2025-10-06 13:44:10 -07:00
Krrish Dholakia
7a61526b7b feat(azure/): partially working passthrough flow via router
allows azure passthrough to work for litellm models
2025-10-06 13:35:00 -07:00
Achintya Rajan
f4d9e4869a renamed (console), added use clients 2025-10-06 13:33:38 -07:00
Ishaan Jaffer
a56069b722 UI: new build 2025-10-06 13:16:52 -07:00
Sameer Kankute
cd25782359
fix passing headers for gemini (#15231) 2025-10-06 12:59:19 -07:00