CyrusTC
5d326386fb
feat(bedrock): add serviceTier support for Converse API ( #17810 )
...
Add support for the Bedrock Converse API serviceTier parameter to allow
specifying processing tier (priority, default, or flex).
Changes:
- Add ServiceTierBlock type in litellm/types/llms/bedrock.py
- Add serviceTier to CommonRequestObject
- Add serviceTier to get_config_blocks() in AmazonConverseConfig
- Add comprehensive tests for serviceTier functionality
- Add documentation for serviceTier usage
This allows users to configure service tier via:
- litellm_params in proxy config
- optional_params in SDK calls
2025-12-11 08:16:32 -08:00
Cesar Garcia
97be0da0d2
fix(azure_ai): Remove unsupported params from Azure AI Anthropic requests ( #17822 )
...
* fix(azure_ai): Remove unsupported params from Azure AI Anthropic requests
Azure AI Anthropic endpoint rejects max_retries and stream_options parameters
with "Extra inputs are not permitted" error. These are LiteLLM-internal
parameters that should not be sent to the API.
Fixes 400 Bad Request error when using azure_ai/claude-sonnet-4-5 and other
Azure AI Anthropic models.
* test(azure_ai): Add test for unsupported params removal in Azure AI Anthropic
Verifies that max_retries, stream_options, and extra_body are properly
removed from the request before sending to Azure AI Anthropic endpoint.
2025-12-11 08:09:13 -08:00
Ashton Sidhu
a514313540
Add Hiddenlayer Guardrail Hooks ( #17728 )
...
* Core logic working, need to add tests
* Re add removed files
* Remove mistaken files
* one more file
* Add deployment params
* Add tests
* Remove unused imports
* Update docs from feedback
* Update guardrails
2025-12-11 07:43:26 -08:00
Sameer Kankute
3de46889bb
Merge pull request #17805 from BerriAI/litellm_fix_azure_anthropic_model_map
...
Fix llm provider for azure_ai in model map
2025-12-11 16:04:11 +05:30
Sameer Kankute
118a06ddfb
bump openai package to 2.9.0
2025-12-11 16:01:52 +05:30
Sameer Kankute
7b1cef86a7
Add support for target_storage param
2025-12-11 15:08:17 +05:30
Sameer Kankute
51065295ba
Fix llm provider for azure_ai in model map
2025-12-11 11:36:56 +05:30
yuneng-jiang
7ca2c2abfc
Adding tests
2025-12-10 21:37:38 -08:00
yuneng-jiang
6f6a8f782e
Fix UI settings
2025-12-10 21:31:35 -08:00
Sameer Kankute
8942053c8b
Merge pull request #17700 from BerriAI/litellm_batches_passthrough_cost_tracking
...
Add anthropic retrieve batches and retreive file content support
2025-12-11 10:31:54 +05:30
Sameer Kankute
1107febf05
Merge pull request #17803 from BerriAI/litellm_preserve_system_instructions
...
fix: Preserve systemInstructions for vertex ai generate content request
2025-12-11 10:31:08 +05:30
yuneng-jiang
084ed01518
proxy extras and migration
2025-12-10 20:53:13 -08:00
yuneng-jiang
3784b5597a
bump: version 0.4.12 → 0.4.13
2025-12-10 20:51:23 -08:00
yuneng-jiang
91056c1d7e
Merge remote-tracking branch 'origin' into litellm_daily_agent_table
2025-12-10 20:47:58 -08:00
yuneng-jiang
1c6de2b80d
Merge remote-tracking branch 'origin' into litellm_sendgrid
2025-12-10 20:39:25 -08:00
yuneng-jiang
cffd0ac350
sendgrid docs
2025-12-10 20:39:10 -08:00
yuneng-jiang
ca11264bbf
Fixing sendgrid integration
2025-12-10 20:31:33 -08:00
Sameer Kankute
9344d29a15
fix: Preserve systemInstructions for vertex ai generate content request
2025-12-11 09:52:32 +05:30
Sameer Kankute
2ea855d225
Merge pull request #17707 from raghav-stripe/raghav-fix-responsesapi-rl
...
fix: responses api not applying tpm rate limits on api keys
2025-12-11 08:57:16 +05:30
Shivam Rawat
9d7a255d55
made litellm proxy and sdk difference cleaner in overview ( #17790 )
2025-12-10 19:14:49 -08:00
Cesar Garcia
b2e3f56f69
feat(models): add Mistral Codestral 2508, Devstral 2512, and Labs Devstral Small 2512 ( #17801 )
...
Add newly released Mistral coding models:
- mistral/codestral-2508: 256K context, $0.30/$0.90 per M tokens
- mistral/devstral-2512: 256K context, $0.40/$2.00 per M tokens
- mistral/labs-devstral-small-2512: 256K context, $0.10/$0.30 per M tokens
2025-12-10 19:14:02 -08:00
Ishaan Jaff
4a7437ba5f
[Feat] Agent Gateway - allow adding langgraph, bedrock agent core agents ( #17802 )
...
* fix: langgraph bridge streaming
* add public/agents/fields
* test_a2a_completion_bridge_non_streaming
* TestA2AStreamingTransformation
* AgentCredentialFieldMetadata
* add new logo
* refactor add agent
* fix add dynamic fields
* feat allow adding langgraph agent
* add langgraph provider
* stash
* add AgentCreateInfo
* agent_create_fields
* fix fields
* test_a2a_completion_bridge_bedrock_agentcore
* test_a2a_completion_bridge_bedrock_agentcore
* add public endpoints
* fix a2a endpoints
* fix dynamic fields
2025-12-10 19:13:50 -08:00
Sameer Kankute
ad5d94a82f
Merge pull request #17771 from BerriAI/litellm_db_videos_issue
...
fix: respect videos content db creds
2025-12-11 08:27:43 +05:30
Sameer Kankute
44d57b695a
remove print statment
2025-12-11 08:22:22 +05:30
yuneng-jiang
439d42ba67
bump: version 0.1.24 → 0.1.25
2025-12-10 18:26:07 -08:00
Yuta Saito
4efa21ee7d
docs: clarify MCP tool support across providers
2025-12-11 10:39:24 +09:00
Yuta Saito
6393277bf4
fix: separate MCP handling out of main.py
2025-12-11 10:39:14 +09:00
Ishaan Jaff
059fedbed5
[Feat] Agent Gateway - Track agent_id in SpendLogs ( #17795 )
...
* add agent_id in metadata in spend logs
* add agent_id in SpendLogsPayload
* add agent_id in SpendLogsPayload
* add _set_agent_id_on_logging_obj
* add agent id tracking in SpendLogs
* add agent id in spend logs
* fix create_a2a_client
* test_asend_message_passes_agent_id_to_callback
* test_get_logging_payload_includes_agent_id_from_kwargs
* test_asend_message_streaming_triggers_callbacks
* fix asend_message_streaming
* asend_message_streaming
* A2AStreamingIterator
* _handle_stream_message
* test_asend_message_streaming_propagates_metadata
2025-12-10 16:09:56 -08:00
yuneng-jiang
8ed1dfcb6a
Agent Usage UI
2025-12-10 15:46:22 -08:00
Ishaan Jaff
5d456bcdc3
[Feat] UI SSO - allow fetching role from generic SSO provider (Keycloak) ( #17787 )
...
* fix ui SSO
* TestGenericResponseConvertorUserRole
* Assigning User Roles via SSO
2025-12-10 13:09:28 -08:00
Alexsander Hamir
439bb5bfe3
fix: suggest Gunicorn instead of uvicorn when using max_requests_before_restart ( #17788 )
2025-12-10 13:09:00 -08:00
Ishaan Jaff
7ad2a58dcd
[Feat] A2a Gateway - allow using bedrock agentcore, langgraph agents with A2a Gateway ( #17786 )
...
* init LANGGRAPH
* init LangGraphConfig
* init LangGraphConfig types
* init langgraph
* init getting api base and key
* init transform langgraph
* fix SSE issues
* test_langgraph_acompletion_non_streaming
* add LangGraph to docs
* docs: Setting Up a Local LangGraph Server
* fix langgraph SSE
* fix import uuid
* init A2A to LiteLLM Completion Bridge
* add send message for bridge
* test_a2a_completion_bridge_non_streaming
* add A2ACompletionBridgeTransformation
* add a2a send message support
* init a2a bridge
2025-12-10 13:08:43 -08:00
Ishaan Jaff
49b91c4a35
[Feat] A2a gateway - Add cost per token pricing ( #17780 )
...
* fix calculate_a2a_cost
* add cost_per_query
* add test_asend_message_uses_cost_per_query
* fix: _initialize_slack_alerting_jobs
* feat: add token tracking for agents invoke
* add A2ARequestUtils
* add _set_usage_on_logging_obj
* test_asend_message_token_tracking
* add _handle_a2a_response_logging
* test_asend_message_streaming_token_tracking
* add A2AStreamingIterator
* add cost calculator for agents
* test_asend_message_uses_input_output_cost_per_token
* docs gix
2025-12-10 13:08:15 -08:00
Alexsander Hamir
ccb6cd880a
[Perf] - Cut memory leak in half ( #17784 )
...
Stop passing references to large objects to fire-and-forget tasks.
2025-12-10 12:31:58 -08:00
Ishaan Jaff
5ee32167c0
[Feat] New Provider - add langgraph ( #17783 )
...
* init LANGGRAPH
* init LangGraphConfig
* init LangGraphConfig types
* init langgraph
* init getting api base and key
* init transform langgraph
* fix SSE issues
* test_langgraph_acompletion_non_streaming
* add LangGraph to docs
* docs: Setting Up a Local LangGraph Server
* fix langgraph SSE
* fix import uuid
2025-12-10 12:30:35 -08:00
yuneng-jiang
1cad479297
Daily Agent Usage Table WIP
2025-12-10 11:50:52 -08:00
Ishaan Jaff
115a14891d
[Feat] Agent Gateway - Add token counting non streaming + streaming ( #17779 )
...
* fix calculate_a2a_cost
* add cost_per_query
* add test_asend_message_uses_cost_per_query
* fix: _initialize_slack_alerting_jobs
* feat: add token tracking for agents invoke
* add A2ARequestUtils
* add _set_usage_on_logging_obj
* test_asend_message_token_tracking
* add _handle_a2a_response_logging
* test_asend_message_streaming_token_tracking
* add A2AStreamingIterator
2025-12-10 11:45:28 -08:00
Krish Dholakia
8bc5e2ca7f
Add /v1/messages/count_tokens endpoint documentation ( #17772 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-10 11:34:11 -08:00
Krish Dholakia
7a773b738b
Add docstring to add_new_model endpoint ( #17778 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-10 11:25:07 -08:00
Krrish Dholakia
4ea758767a
build: update poetry.lock
2025-12-10 11:09:28 -08:00
Ishaan Jaff
7a33579af6
[Feat] Agent Gateway - Add cost per query for agent invocations ( #17774 )
...
* fix calculate_a2a_cost
* add cost_per_query
* add test_asend_message_uses_cost_per_query
* fix: _initialize_slack_alerting_jobs
2025-12-10 10:43:52 -08:00
yuneng-jiang
3d7c6cb690
Merge remote-tracking branch 'origin' into litellm_sendgrid
2025-12-10 10:42:47 -08:00
saisurya237
f3ead2c153
add a new feature fix to expose the team alias when authenticating th… ( #17725 )
...
* CLI SSO: show team aliases in team selection
* temp poetry lock fix
* add poetry.lock to gitignore
* Revert proxy_server background job refactor
* Revert proxy_server background job refactor
* rever gitignore and poetry lock file
2025-12-10 10:10:28 -08:00
yuneng-jiang
ba554a86b9
Merge pull request #16843 from BerriAI/litellm_allow_custom_mount_paths
...
[Feature] Allow Root Path to Redirect when Docs not on Root Path
2025-12-10 09:52:30 -08:00
Sameer Kankute
15e5a8251e
fix: respect videos content db creds
2025-12-10 23:00:01 +05:30
Sameer Kankute
dea62e3f60
Merge pull request #17769 from BerriAI/litellm_test_fix
...
Fix nvdia and geminin tests
2025-12-10 22:29:04 +05:30
Sameer Kankute
ad45fc7cce
Merge pull request #17770 from BerriAI/litellm_deepseek_v3.2
...
Add DeepSeek-V3.2 model support
2025-12-10 22:28:50 +05:30
Sameer Kankute
4c78c1afc8
Merge pull request #17756 from BerriAI/litellm_add_gemini_computer_use
...
Add support for computer use for gemini
2025-12-10 22:28:37 +05:30
Sameer Kankute
05a82ae570
Add support for deepseek v3.2
2025-12-10 22:21:04 +05:30
Sameer Kankute
ec30032bc3
Fix nvdia and geminin tests
2025-12-10 22:05:11 +05:30
