Commit Graph

30881 Commits

Author SHA1 Message Date
Ishaan Jaff
628cd13755
[Feat] UI - Allow scheduling key rotations when creating virtual keys (#14960)
* fix design of key reset interval

* fix: LiteLLM_VerificationToken

* fix key manager

* add key_rotation_at

* ui fix

* fix ui view

* set key_rotation_at  on creation

* add key_rotation_at

* fix info

* fix: _set_key_rotation_fields

* fix KeyRotationManager

* fix key edit view

* test_update_key_fn_auto_rotate_enable

* fix KeyRotationManager
2025-09-26 16:24:40 -07:00
Maximgitman
f2d7c1c14d Fix Anthropic streaming IDs 2025-09-26 18:31:10 -04:00
Krrish Dholakia
342e80d48d feat: initial commit for v2 oauth flow 2025-09-26 15:25:42 -07:00
Sameer Kankute
076cb4654e
fix mypy errors from bitbucket integration (#14959) 2025-09-26 14:08:17 -07:00
Ishaan Jaffer
56b3258c40 Revert "fix design of key reset interval"
This reverts commit 75146f04428b45796ee4e7dab83ef958f77daee6.
2025-09-26 13:38:38 -07:00
Ishaan Jaffer
3ea45ec03a fix design of key reset interval 2025-09-26 13:38:38 -07:00
Sameerlite
ce0b815959 fix test 2025-09-27 02:08:09 +05:30
Mubashir Osmani
b039374c71
Azure Managed Batches - api key error (#14932)
* added qwen models and gpt-5-codex

* fix flaky test

* fix failing test

* Added retries to prisma client state

* fix: prisma client state retries in pods

* Revert "fix failing test"

This reverts commit dbec4988a2627257fd05b905e216225664517f32.

* Revert "fix flaky test"

This reverts commit b0ac2f2dc35ca433af0c82f3cda770d6981caff4.

* Revert "added qwen models and gpt-5-codex"

This reverts commit 9a8a8f2d47ab4dc8aecb0cd9a6a4f82ed81bb056.

* Revert "fix: prisma client state retries in pods"

This reverts commit 04e58e5ca1a489916e3b49e9b674f5c6713fd7cd.

* fix lint

* Revert "fix lint"

This reverts commit 5303d52a5e3bee7e131dcabd098e94f0613a7bb9.

* fixed lint

* fix: azure api key managed batch files
2025-09-26 13:35:35 -07:00
Sameerlite
92cb34eb25 fix mypy errors 2025-09-27 02:02:24 +05:30
Alexsander Hamir
2973ff8be9
fix: remove slow string operation (#14955)
* fix: remove slow string operation

* fix: behavior change

* fix: behavior change

* fix: avoid default call
2025-09-26 13:25:59 -07:00
Sameer Kankute
94edbd1b35
Merge pull request #14957 from BerriAI/main
merge main
2025-09-27 01:54:44 +05:30
Sameer Kankute
3dac7e28fc
Potential fix for code scanning alert no. 3413: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-09-27 01:18:34 +05:30
Sameerlite
61a450f2e2 fix lint 2025-09-27 01:16:09 +05:30
Sameerlite
66cf281331 fix lint 2025-09-27 01:03:21 +05:30
Sameerlite
67e7ad5aa9 Add vertex live api passthrough with cost tracking 2025-09-27 00:55:47 +05:30
Ishaan Jaffer
35930d7ccb fix key rotation settings 2025-09-26 12:21:57 -07:00
Ishaan Jaffer
1009a38ff4 Add key expiry and auto-rotation settings
- Create KeyLifecycleSettings component grouping expiry and auto-rotation
- Add dedicated 'Key Expiry & Auto-Rotation' accordion section
- Fix auto-rotation toggle visibility with proper layout
- Move expire key field from Optional Settings to new section
- Remove old AutoRotationSettings component
- Integrate auto-rotation metadata in form submission
2025-09-26 12:02:14 -07:00
Ishaan Jaffer
e2e2ae78a4 test fix 2025-09-26 11:48:28 -07:00
Ishaan Jaffer
4a75d042fb fix test 2025-09-26 11:45:52 -07:00
Ishaan Jaff
ea8d0bb7d5 fix: re-add scheduled rotations 2025-09-26 11:40:46 -07:00
Nicolas Herment
50f625433d
Revert incorrect changes to sonnet-4 max output tokens (#14933) 2025-09-26 11:19:52 -07:00
Ishaan Jaffer
539d10f9cb test fix 2025-09-26 11:06:26 -07:00
Ishaan Jaff
d04c6d4eea
[Feat] Add new anthropic web fetch tool support (#14951)
* add web_fetch tool for ANTHROPIC_HOSTED_TOOLS

* add ANTHROPIC_BETA_HEADER_VALUES, ANTHROPIC_HOSTED_TOOLS

* feat: add web fetch tool anthropic

* test_anthropic_tool_use

* docs web fetch

* docs fix

* docs fix
2025-09-26 11:04:11 -07:00
Olivier Cornelis
6fe8c33448
docs: add documentation for additional cost-related keys in custom pricing (#14949)
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-09-26 10:32:30 -07:00
Ishaan Jaff
360befa216
[Feat] Add support for Gemini 2.5 Flash and Flash-lite preview models (09-2025 release) (#14948)
* add gemini-2.5-flash-preview-09-2025

* docs add gemini-2.5-flash-preview-09-2025 model family
2025-09-26 09:51:38 -07:00
Alex Shoop
914844122c
Fix: revert fastuuid optional dependency, always use fastuuid in .__uid helper (#14941)
* always use fastuuid

* rm from proxy extras since its now default

* poetry lock
2025-09-26 09:14:20 -07:00
Daniel Klein
de795a4531 Fix inconsistent token configs for gpt-5 models 2025-09-26 09:43:26 -04:00
Toy-97
6c95bd926f
update: DeepInfra model data refresh [2025-09-26]
Added models:
deepinfra/deepseek-ai/DeepSeek-V3.1-Terminus

Removed models:
deepinfra/zai-org/GLM-4.5-Air

Modified models:
deepinfra/NousResearch/Hermes-3-Llama-3.1-70B:
   - input_cost_per_token: 1.2e-07 → 3e-07

deepinfra/Qwen/Qwen3-32B:
   - output_cost_per_token: 3e-07 → 2.8e-07

deepinfra/Qwen/Qwen3-Next-80B-A3B-Instruct:
   - max_tokens: 4096 → 262144
   - max_output_tokens: 4096 → 262144
   - max_input_tokens: 4096 → 262144

deepinfra/Qwen/Qwen3-Next-80B-A3B-Thinking:
   - max_tokens: 4096 → 262144
   - max_output_tokens: 4096 → 262144
   - max_input_tokens: 4096 → 262144

deepinfra/Qwen/Qwen3-235B-A22B-Instruct-2507:
   - input_cost_per_token: 1.3e-07 → 9e-08

deepinfra/meta-llama/Meta-Llama-3.1-70B-Instruct:
   - input_cost_per_token: 2.3e-07 → 4e-07

deepinfra/google/gemini-2.5-flash:
   - output_cost_per_token: 1.75e-06 → 2.5e-06
   - input_cost_per_token: 2.1e-07 → 3e-07

deepinfra/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo:
   - output_cost_per_token: 2e-08 → 3e-08
   - input_cost_per_token: 1.5e-08 → 2e-08

deepinfra/meta-llama/Llama-3.2-3B-Instruct:
   - output_cost_per_token: 2.4e-08 → 2e-08
   - input_cost_per_token: 1.2e-08 → 2e-08

deepinfra/Sao10K/L3-8B-Lunaris-v1-Turbo:
   - input_cost_per_token: 2e-08 → 4e-08

deepinfra/openai/gpt-oss-120b:
   - input_cost_per_token: 9e-08 → 5e-08

deepinfra/google/gemini-2.5-pro:
   - output_cost_per_token: 7e-06 → 1e-05
   - input_cost_per_token: 8.75e-07 → 1.25e-06

deepinfra/NousResearch/Hermes-3-Llama-3.1-405B:
   - output_cost_per_token: 8e-07 → 1e-06
   - input_cost_per_token: 7e-07 → 1e-06

deepinfra/Qwen/Qwen3-235B-A22B:
   - output_cost_per_token: 6e-07 → 5.4e-07
   - input_cost_per_token: 1.3e-07 → 1.8e-07

deepinfra/nvidia/Llama-3.1-Nemotron-70B-Instruct:
   - output_cost_per_token: 3e-07 → 6e-07
   - input_cost_per_token: 1.2e-07 → 6e-07

deepinfra/meta-llama/Llama-3.3-70B-Instruct-Turbo:
   - output_cost_per_token: 1.2e-07 → 3.9e-07
   - input_cost_per_token: 3.8e-08 → 1.3e-07

deepinfra/deepseek-ai/DeepSeek-V3-0324:
   - input_cost_per_token: 2.8e-07 → 2.5e-07
   - cache_read_input_token_cost: 2.24e-07 → None

deepinfra/mistralai/Mistral-Small-3.2-24B-Instruct-2506:
   - output_cost_per_token: 1e-07 → 2e-07
   - input_cost_per_token: 5e-08 → 7.5e-08

deepinfra/Qwen/Qwen3-235B-A22B-Thinking-2507:
   - output_cost_per_token: 6e-07 → 2.9e-06
   - input_cost_per_token: 1.3e-07 → 3e-07

deepinfra/zai-org/GLM-4.5:
   - output_cost_per_token: 2e-06 → 1.6e-06
   - input_cost_per_token: 5.5e-07 → 4e-07

deepinfra/mistralai/Mixtral-8x7B-Instruct-v0.1:
   - output_cost_per_token: 2.4e-07 → 4e-07
   - input_cost_per_token: 8e-08 → 4e-07

deepinfra/openai/gpt-oss-20b:
   - output_cost_per_token: 1.6e-07 → 1.5e-07

deepinfra/google/gemma-3-27b-it:
   - output_cost_per_token: 1.7e-07 → 1.6e-07
2025-09-26 20:11:26 +08:00
Matéo Evan Muller
b12fa388ad
Fix vLLM provider rerank endpoint from /v1/rerank to /rerank 2025-09-26 13:13:17 +02:00
Krish Dholakia
2f3155c2ee
Merge pull request #14858 from oytunkutrup1/litellm_fix_gpt3.5_price_fix
GPT-3.5-Turbo price updated.
2025-09-25 23:47:43 -07:00
Krish Dholakia
6a1be4722e
Merge pull request #14879 from huangyafei/update_price
Add gpt-5 and gpt-5-codex to OpenRouter cost map
2025-09-25 23:46:35 -07:00
Krish Dholakia
79ebb2c95e
Merge pull request #14888 from mrFranklin/feat/improve-opik
feat: improve opik integration code
2025-09-25 23:40:41 -07:00
Krish Dholakia
f2f75bf911
Merge pull request #14893 from vertexcover-io/fix/openai-image-edit-support-images
🐛 Fix a bug where openai image edit siltently ignores multiple images
2025-09-25 23:37:52 -07:00
Ritesh Kadmawala
af0cb7c277 🐛 Fix a bug where openai image edit siltently ignores multiple images 2025-09-26 10:56:06 +05:30
Mubashir Osmani
625ed3f8cf
fix: prisma client state retries (#14925)
* added qwen models and gpt-5-codex

* fix flaky test

* fix failing test

* Added retries to prisma client state

* fix: prisma client state retries in pods

* Revert "fix failing test"

This reverts commit dbec4988a2627257fd05b905e216225664517f32.

* Revert "fix flaky test"

This reverts commit b0ac2f2dc35ca433af0c82f3cda770d6981caff4.

* Revert "added qwen models and gpt-5-codex"

This reverts commit 9a8a8f2d47ab4dc8aecb0cd9a6a4f82ed81bb056.

* Revert "fix: prisma client state retries in pods"

This reverts commit 04e58e5ca1a489916e3b49e9b674f5c6713fd7cd.

* fix lint

* Revert "fix lint"

This reverts commit 5303d52a5e3bee7e131dcabd098e94f0613a7bb9.

* fixed lint
2025-09-25 21:54:00 -07:00
Vishnu Patibandla
fb9e2c93a0
add noma guardrail provider to ui (#14415)
* add noma guardrail provider to ui

* resolved linting issues

* fix return value
2025-09-25 21:53:15 -07:00
Teddy Amkie
0e8f60fe67
Remove aiohttp_ prefix from config (#14920)
* Fix: Correct model name in load test advanced docs

Co-authored-by: teddy <teddy@berri.ai>

* Fix: Update load testing provider to openai/fake

Co-authored-by: teddy <teddy@berri.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-25 16:09:41 -07:00
Ishaan Jaffer
dcfe6dd3d9 linting fix 2025-09-25 16:07:02 -07:00
Sameer Kankute
1681bf7175
(Feat) Add BitBucket Integration for Prompt Management (#14882)
* Add bitbucket integration for prompt management

* remove not needed file

* remove not needed file

* Add the correct readme

* Add the correct readme

* fix test for bitbucket

* fix test for bitbucket
2025-09-25 15:51:00 -07:00
Teddy Amkie
dcbccd1fea
Corrected docs updates sept 2025 (#14916)
* docs: Corrected documentation updates from Sept 2025

This PR contains the actual intended documentation changes, properly synced with main:

 Real changes applied:
- Added AWS authentication link to bedrock guardrails documentation
- Updated Vertex AI with Gemini API alternative configuration
- Added async_post_call_success_hook code snippet to custom callback docs
- Added SSO free for up to 5 users information to enterprise and custom_sso docs
- Added SSO free information block to security.md
- Added cancel response API usage and curl example to response_api.md
- Added image for modifying default user budget via admin UI
- Re-ordered sidebars in documentation

 Sync issues resolved:
- Kept all upstream changes that were added to main after branch diverged
- Preserved Provider-Specific Metadata Parameters section that was added upstream
- Maintained proper curl parameter formatting (-d instead of -D)

This corrects the sync issues from the original PR #14769.

* docs: Restore missing files from original PR

Added back ~16 missing documentation files that were part of the original PR:

 Restored files:
- docs/my-website/docs/completion/usage.md
- docs/my-website/docs/fine_tuning.md
- docs/my-website/docs/getting_started.md
- docs/my-website/docs/image_edits.md
- docs/my-website/docs/image_generation.md
- docs/my-website/docs/index.md
- docs/my-website/docs/moderation.md
- docs/my-website/docs/observability/callbacks.md
- docs/my-website/docs/providers/bedrock.md
- docs/my-website/docs/proxy/caching.md
- docs/my-website/docs/proxy/config_settings.md
- docs/my-website/docs/proxy/db_deadlocks.md
- docs/my-website/docs/proxy/load_balancing.md
- docs/my-website/docs/proxy_api.md
- docs/my-website/docs/rerank.md

 Fixed context-caching issue:
- Restored provider_specific_params.md to main version (preserving Provider-Specific Metadata Parameters section)
- Your original PR didn't intend to modify this file - it was just a sync issue

Now includes all ~26 documentation files from the original PR #14769.

* docs: Remove files that were deleted in original PR

- Removed docs/my-website/docs/providers/azure_ai_img_edit.md (was deleted in original PR)
- sdk/headers.md was already not present

Now matches the complete intended changes from original PR #14769.

* docs: Restore azure_ai_img_edit.md from main

- Restored docs/my-website/docs/providers/azure_ai_img_edit.md from main branch
- This file should not have been deleted as it was a newer commit
- SDK headers file doesn't exist in main (was reverted) and wasn't part of your original changes

Fixes the file restoration issues.

* docs: Fix vertex.md - preserve context caching from newer commit

- Restored vertex.md to main version to preserve context caching content (lines 817-887)
- Added back only your intended change: alternative gemini config example
- Context caching content from newer commit is now preserved

Fixes the vertex.md sync issue where newer content was incorrectly deleted.

* docs: Fix providers/bedrock.md - restore deleted content from newer commit

- Restored providers/bedrock.md to main version
- Preserves 'Usage - Request Metadata' section that was added in newer commit
- Your actual intended change was to proxy/guardrails/bedrock.md (authentication tip) which is preserved
- Now only has additions, no subtractions as intended

Fixes the bedrock.md sync issue.

* docs: Restore missing IAM policy section in bedrock.md

Added back your intended IAM policy documentation that was lost when restoring main version:

 Added IAM AssumeRole Policy section:
- Explains requirement for sts:AssumeRole permission
- Shows error message example when permission missing
- Provides complete IAM policy JSON example
- Links to AWS AssumeRole documentation
- Clarifies trust policy requirements

Now bedrock.md has both:
- All newer content preserved (Request Metadata section)
- Your intended IAM policy addition restored

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-25 15:49:19 -07:00
Ishaan Jaff
2dd38420a7
[Feat] - Cost Tracking - show input, output, tool call cost breakdown in StandardLoggingPayload (#14921)
* add new CostBreakdown typed dict

* fix CostBreakdown type

* fix fix _store_cost_breakdown_in_logging_obj

* fix CostBreakdown

* test_cost_breakdown_in_standard_logging_payload
2025-09-25 15:48:22 -07:00
Alexsander Hamir
eaa04cd8ce
fix: use fastuuid helper (#14903)
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
2025-09-25 15:47:01 -07:00
Ishaan Jaff
ab5fb704c2
[Feat] Logging - datadog callback Log message content w/o sending to datadog (#14909)
* add DatadogInitParams

* fix _get_datadog_params

* test_datadog_message_redaction
2025-09-25 15:46:22 -07:00
Teddy Amkie
be44735de4
Update docs to reflect session management for all models (#14914)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-25 15:45:13 -07:00
Ishaan Jaff
6916bf15de
Remove 'Useful Links' tab from admin settings UI (#14918)
- Removed 'Useful Links' tab from TabList in admin panel
- Removed corresponding TabPanel containing UsefulLinksManagement component
- Removed unused import for UsefulLinksManagement component
- Fixes issue where useful links section was displaying blank content

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-25 15:15:46 -07:00
Krish Dholakia
958db9df8c
Merge pull request #14899 from arsh72/fix/presidio-custom-entities-union-type
Fix: Support custom entity types in Presidio guardrail with Union[PiiEntityType, str]
2025-09-25 11:39:11 -07:00
Sameerlite
9d4eb814d4 initial int live api 2025-09-25 22:40:54 +05:30
Arshdeep Singh
cb14bb341d fix: update PresidioAnalyzeRequest type to support Union entities 2025-09-25 13:00:05 -04:00
Arshdeep Singh
471f011ae5 fix: add return type annotation for full mypy compliance 2025-09-25 12:52:12 -04:00
Arshdeep Singh
a9f39396b2 fix: resolve mypy type checking errors for Union type 2025-09-25 12:47:15 -04:00