Commit Graph

22383 Commits

Author SHA1 Message Date
Tom Bocklisch
d7982bb0af
Use proper attribute for sagemaker request (#11362) 2025-06-04 22:47:43 -07:00
Jimmy Tsai
4019f79808
feat: add deepseek-r1 family model configuration to pricing JSON (#11394) 2025-06-04 22:39:06 -07:00
Ishaan Jaff
02a34d319a
bump to ddtrace==3.8.0 (#11426) 2025-06-04 22:18:07 -07:00
Ishaan Jaff
f0e0007eaf fix: gemini-2.0-flash-preview-image-generation test 2025-06-04 21:21:28 -07:00
Cole McIntosh
049e65a84e
Merge pull request #11417 from colesmcintosh/sso-config-ui 2025-06-04 20:51:59 -06:00
Ishaan Jaff
de306cfcb3
[Performance] Performance improvements for /v1/messages route (#11421)
* fix: perf anthropic /v1/messages

* fix: perf anthropic /v1/messages

* fix: linting checks

* fix: linting checks
2025-06-04 18:47:53 -07:00
raz-alon
fada9c79be
Add User ID validation to ensure it is not an email or phone number (#10102) 2025-06-04 18:38:02 -07:00
Cole McIntosh
c1a324c2fb Merge remote-tracking branch 'origin/main' into sso-config-ui 2025-06-04 18:42:43 -06:00
Cole McIntosh
65b28826a6 Add uiAuditLogsCall function 2025-06-04 18:19:53 -06:00
Krish Dholakia
9da32d9e14
Litellm audit log staging (#11418)
* Audit logs added (#11226)

* audit logs added

* audit logs populated

* adding json response

* collapsible json columns

* add created at column

* added changed field

* added premiumUser description

* added paginated filtered logs

* convert table names

* remove test file

* added new ui for audit logs

* only show the difference in before value and updated value

* fix: add lucide-react to package json

---------

Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
2025-06-04 14:34:17 -07:00
Cole McIntosh
3a946933ee Refactor settings response models in proxy_setting_endpoints.py
- Renamed SSOSettingsResponse to inherit from a new base class SettingsResponse for better structure.
- Introduced InternalUserSettingsResponse and DefaultTeamSettingsResponse models for internal user and default team settings.
- Updated endpoint responses to use field_schema instead of schema for consistency.
- Enhanced test cases to validate the new response structure and ensure proper functionality of SSO settings.
2025-06-04 15:08:05 -06:00
Cole McIntosh
3cc9460922 Add SSO settings response model in proxy_setting_endpoints.py
- Introduced SSOSettingsResponse model to encapsulate SSO configuration values and schema information.
- Updated the get_sso_settings endpoint to utilize the new response model, enhancing API clarity and usability.
2025-06-04 14:55:29 -06:00
Cole McIntosh
c41b14e27b Add clear SSO settings functionality in SSOModals component
- Introduced a confirmation modal for clearing SSO settings.
- Implemented handleClearSSO function to reset SSO settings and provide user feedback.
- Updated UI to include a 'Clear' button for SSO settings, enhancing user experience.
- Added state management for the confirmation modal visibility.
2025-06-04 14:39:26 -06:00
Cole McIntosh
72c7fd63bf Implement SSO configuration check in AdminPanel and update SSOModals to reflect SSO status
- Added logic to check SSO configuration and set state in AdminPanel.
- Introduced a new function to handle SSO configuration checks.
- Updated UI to conditionally render SSO button text based on configuration status.
- Passed SSO configuration status as a prop to SSOModals for better integration.
2025-06-04 14:34:18 -06:00
Cole McIntosh
58b1f78ff0 Remove console log from handleShowInstructions in AdminPanel component 2025-06-04 14:25:57 -06:00
Cole McIntosh
9c481e3ba8 Add SSO configuration endpoints and UI integration
- Introduced new SSOConfig class to manage SSO settings.
- Added endpoints for fetching and updating SSO settings in proxy_setting_endpoints.py.
- Created a new __init__.py file to expose the SSO router.
- Updated AdminPanel and SSOModals components to handle SSO settings retrieval and updates.
- Removed deprecated SSO update logic from AdminPanel.
- Enhanced error handling and logging for SSO operations.
2025-06-04 14:09:19 -06:00
Cole McIntosh
7d4a70bfe3 Enhance Admin Panel UI: Adjust button styles and layout for better accessibility and user experience 2025-06-04 13:32:43 -06:00
Lik Wei
a24077fb4e
fix: add enterprise feature gating to RegenerateKeyModal in KeyInfoView (#11400)
Pass premiumUser prop to RegenerateKeyModal to control access
Disable Regenerate Key button and show tooltip for non-premium users
Ensure consistent enterprise feature messaging across components
2025-06-04 12:05:56 -07:00
Krrish Dholakia
26891c23c5 docs: update docs 2025-06-04 11:42:53 -07:00
Walt Wells
f782cc7e4d
update GCSBucketBase to handle GSM project ID if passed (#11409) 2025-06-04 10:02:26 -07:00
Krish Dholakia
e74ff23b45
Litellm dev 06 03 2025 p3 (#11388)
* fix(vertex_ai/common_utils.py): Close https://github.com/BerriAI/litellm/issues/11383

* feat(anthropic/batches): transformation.py

new transformation config for anthropic batches

* feat(anthropic/batches): working token tracking for anthropic batch calls via `/anthropic` passthrough route

* fix(anthropic_passthrough_logging_handler.py): ruff check fixes
2025-06-03 22:35:51 -07:00
Krrish Dholakia
b21efd435a test(test_sso_sign_in.py): update test 2025-06-03 21:46:34 -07:00
Krish Dholakia
3bd128630d
Anthropic - pass file url's as Document content type + Gemini - cache token tracking on streaming calls (#11387)
* fix(anthropic/): fix regression when passing file url's to the 'file_id' parameter

add test and ensure anthropic file url's are correctly sent as 'document' blocks

* fix(vertex_and_google_ai_studio.py): Use same usage calculation function as non-streaming

Closes https://github.com/BerriAI/litellm/issues/10667

* test(test_vertex_and_google_ai_studio_gemini.py): update test
2025-06-03 21:36:49 -07:00
Krrish Dholakia
0273b72b68 build: bump poetry 2025-06-03 21:25:53 -07:00
Krish Dholakia
e0fa33f099
UI / SSO - Update proxy admin id role in DB + Handle SSO redirects with custom root path (#11384)
* fix(ui_sso.py): update user as proxy admin in db table, when checking for proxy_admin_id

Fixes issue where existing internal user, unable to make calls when set as proxy admin id

* fix(utils.py): fix custom base path
2025-06-03 21:16:55 -07:00
Ishaan Jaff
a1f3a1c5dc
[Feat] Performance - Don't create 1 task for every hanging request alert (#11385)
* feat: add async_get_oldest_n_keys in memory cache

* fix: add add_request_to_hanging_request_check

* test: alerting

* feat: v2 hanging request check

* fix: HangingRequestData

* fix: AlertingHangingRequestCheck

* fix: check_for_hanging_requests

* fix: use correct metadata location for hanging requests

* fix: formatting alert

* test hanging request check

* fix: add guard flags for background tasks alerting
2025-06-03 21:12:54 -07:00
Krrish Dholakia
affc4524e1 build: cascade budget changes 2025-06-03 21:02:19 -07:00
Krrish Dholakia
2fe0a2750b test: ensure aws region correctly set 2025-06-03 20:58:21 -07:00
Cole McIntosh
7bbd8262ed
Add Claude 4 Sonnet & Opus, DeepSeek R1, and fix Llama Vision model pricing configurations (#11339)
* fix: update model path for llama-v3p2-90b-vision-instruct in pricing configuration (missing fireworks_ai/ prefix)

* feat: add deepseek-r1-0528 model configuration to pricing JSON

* feat: add configurations for new Claude 4 model alias to pricing JSON

* undo prefix change

* fix: update supports_response_schema to false in pricing JSON for litellm_provider

* update supports_tool_choice and supports_response_schema

* Update model configuration to disable function calling and tool choice for multiple models in fireworks_ai. Adjusted supported parameters in FireworksAIConfig to conditionally include tools and tool_choice based on model compatibility.

* Refactor FireworksAIConfig to use supports_function_calling from utils

* Enhance FireworksAIConfig to conditionally support tool_choice based on model capabilities
2025-06-03 20:39:47 -07:00
Ishaan Jaff
c0c04b0e63
[Performance]: Add debugging endpoint to track active /asyncio-tasks (#11382)
* feat: add debug/asyncio-tasks

* Update litellm/proxy/common_utils/debug_utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-03 19:04:30 -07:00
Marty Sullivan
d247a390bd
add gemini-embeddings-001 model prices and context window (#11332)
* add gemini-embeddings-001 model prices and context window

* use scientific notation
2025-06-03 15:59:30 -07:00
hagan
0f449bf038
[Bug Fix] Create/Update team member api 500 errror (#10479)
* Fixes issue with team_endpoints on member budget update

* refactored location of budget membership fix

* added test for _upsert_budget_membership func
2025-06-03 15:57:33 -07:00
Ishaan Jaff
7c1e1790e3
fix: health liveliness check auth (#11378) 2025-06-03 15:16:12 -07:00
Ishaan Jaff
209362664f add Dockerfile.dev 2025-06-03 12:03:52 -07:00
Ishaan Jaff
99c91fe41f
[Feat]: Performance add DD profiler to monitor python profile of LiteLLM CPU% (#11375)
* feat: add DD profile

* fix: test_should_use_dd_profiler

* docs dd profiler

* docs DD profiler
2025-06-03 12:03:08 -07:00
Ishaan Jaff
41a2a62511 fix: bedrock kb test 2025-06-03 11:55:41 -07:00
Cole McIntosh
50d5ee9138
Fix linting error (#11374) 2025-06-03 11:32:39 -07:00
Cole McIntosh
621d609879
feat: add cerebras/qwen-3-32b model pricing and capabilities to model_prices_and_context_window.json (#11373) 2025-06-03 11:32:13 -07:00
Krrish Dholakia
d5842edf09 docs(vllm.md): add vllm - model list loadbalancing tutorial to docs 2025-06-03 09:38:33 -07:00
Krrish Dholakia
cff8c3971c fix: fix import 2025-06-03 09:22:37 -07:00
AnilAren
2486743904
Doc : Nvidia embedding models (#11352)
* fix: bedrock ai21 jamba models will work now

* Update supported_embedding.md

* Update supported_embedding.md
2025-06-03 09:17:07 -07:00
Krrish Dholakia
5ff6db5062 fix: fix merge conflicts 2025-06-03 08:26:33 -07:00
Krish Dholakia
4611b821ec
Support returning virtual key in custom auth + Handle provider-specific optional params for embedding calls (#11346)
* feat(custom_auth_auto.py): support returning a litellm virtual key from custom auth

allows admin to remap old keys to litellm virtual keys

* fix(utils.py): correctly handle optional params for openai sdk calls

Fixes https://github.com/BerriAI/litellm/issues/11126

* test: update test

* fix(utils.py): handle edge cases
2025-06-03 07:24:13 -07:00
Krrish Dholakia
3f87b8611b bump: version 1.72.0 → 1.72.1 2025-06-02 23:44:22 -07:00
Krish Dholakia
ccc085faee
Merge in - Gemini streaming - thinking content parsing - return in reasoning_content (#11298)
* fix(base_routing_strategy.py): compress increments to redis - reduces write ops

* fix(base_routing_strategy.py): make get and reset in memory keys atomic

* fix(base_routing_strategy.py): don't reset keys - causes discrepency on subsequent requests to instance

* fix(parallel_request_limiter.py): retrieve values of previous slots from cache

more accurate rate limiting with sliding window

* fix: fix test

* fix: fix linting error

* fix(gemini/): fix streaming handler for function calling

Closes https://github.com/BerriAI/litellm/pull/11294

* fix: fix linting error

* test: update test

* fix(vertex_and_google_ai_studio_gemini.py): return none on skipped chunk

* fix(streaming_handler.py): skip none chunks on async streaming
2025-06-02 23:14:38 -07:00
Ishaan Jaff
a366f9247a docs update s3 logger 2025-06-02 21:53:47 -07:00
Ishaan Jaff
a73cf185ac
fixes: expose flag to disable token counter (#11344)
* fixes: expose flag to disable token counter

* fix add disable_token_counter
2025-06-02 21:53:08 -07:00
Ishaan Jaff
3db272b6d2
[Perf] - Add Async + Batched S3 Logging (#11340)
* fix: add s3 v2 async

* fix: add s3 v2 async

* fix: add s3 v2 async

* test: s3 v2 logging

* fixes: s3 logging

* fixes: s3 logging use max upload batch size

* fixes: s3 logging tests

* fixes: s3 logging tests

* fixes: s3 logging tests
2025-06-02 21:52:34 -07:00
Krrish Dholakia
d408814978 build(ui/): new ui build 2025-06-02 17:55:14 -07:00
Krish Dholakia
00be76abf4
UI - Custom Server Root Path (Multiple Fixes) (#11337)
* fix(proxy_server.py): working swagger on custom base

removes the swagger monkey patch - this seems to render the swagger on custom base paths

* fix(ui/): working custom auth uptil login success event

* fix(ui/): working custom server root path for login

* fix(proxy_server.py): create typed dict for ui returned token

allows better documentation of expected params

* refactor(proxy_server.py): refactor all ui login endpoints to use same returned ui token object

* feat(ui_sso.py): add server root path to ui token

* feat(ui_sso.py): allows ui to call correct endpoint

* fix(networking.tsx): update proxy base url with custom root path

* fix(networking.tsx): handle updating proxy base url for non-local instances

* refactor: remove uneccessary references to proxybaseurl in ui code - reduce potential for errors

* fix: fix linting error

* fix(onboarding_link.tsx): fix onboarding link when custom server path is set

* feat(ui_discovery_endpoints.py): add new public .well-known/ route for litellm ui config

returns the server root path and proxy base url for constructing api calls

* feat(_types.py): add litellm well known config as public route

allows ui to query it

* fix(/_types.py): add .well-known config to as public route

* fix(page.tsx): create pattern for loading in ui config before making network requests

ensures requests are formatted correctly

* fix(page.tsx): call credential endpoint once ui config is loaded

* fix(page.tsx): route correctly to litellm dashboard from new user login

* fix(page.tsx): remove hardcoded `/litellm` for /sso/key/generate request

* fix(proxy_server.py): re-add moderations endpoint

* fix(proxy_server.py): mount __next__ at / and /litellm

allows it to work when proxy is mounted on root

* docs(contributing.md): remove /ui on ui doc - it will now run on root

* docs(custom_root_ui.md): add docs on custom root path
2025-06-02 17:48:03 -07:00