* Fix duplicate imports in SAP embedding transformation
* fix: add missing prompt_spec parameter to HumanloopLogger.get_chat_completion_prompt
- Add prompt_spec: Optional[PromptSpec] = None parameter to match base class signature
- Import PromptSpec from litellm.types.prompts.init_prompts
- Pass prompt_spec to super().get_chat_completion_prompt() call
- Fixes mypy type error: Signature incompatible with supertype CustomLogger
* fix: add missing parameters to AnthropicCacheControlHook.async_get_chat_completion_prompt
- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Change litellm_logging_obj type from Any to LiteLLMLoggingObj using TYPE_CHECKING pattern
- Pass all parameters including prompt_spec to get_chat_completion_prompt call
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase
* fix: add missing parameters to DotpromptManager.async_get_chat_completion_prompt
- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Change litellm_logging_obj type from Any to LiteLLMLoggingObj using TYPE_CHECKING pattern
- Pass all parameters including ignore flags to PromptManagementBase.async_get_chat_completion_prompt
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase
* fix: document envs
* fix: add missing parameters to LangfusePromptManagement.async_get_chat_completion_prompt
- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Pass all parameters including prompt_spec and ignore flags to get_chat_completion_prompt
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase
* fix: add missing parameters to prompt management async methods (Category 1)
- vector_store_pre_call_hook: add ignore_prompt_manager_model, ignore_prompt_manager_optional_params, prompt_spec
- gitlab_prompt_manager: add ignore parameters, fix litellm_logging_obj type
- bitbucket_prompt_manager: add ignore parameters, fix litellm_logging_obj type
- proxy/custom_prompt_management: add prompt_spec parameter
- Fixes mypy type errors: Signature incompatible with supertype
* fix: fix arize_phoenix_prompt_manager and custom_prompt_management (Category 2)
- arize_phoenix_prompt_manager: add prompt_spec to all methods, fix prompt_id types, implement async_compile_prompt_helper
- custom_prompt_management: implement async_compile_prompt_helper abstract method
- Fixes mypy type errors: Signature incompatible with supertype and abstract method errors
* fix: fix obvious type errors (Category 3 - Quick Wins)
- langfuse: change 'callable' to 'Callable' type annotation
- presidio: add type narrowing check for Choices vs StreamingChoices
- StreamingChoices doesn't have .message attribute, only Choices does
- Add hasattr check before accessing choice.message
- Fixes mypy type errors: callable? not callable and union-attr errors
* fix: handle expires_after None in Azure files handler (Todo 14)
- Extract logic to _prepare_create_file_data helper method
- Remove expires_after from dict if None to match SDK's Omit pattern
- Add type ignore for FileExpiresAfter -> file_create_params.ExpiresAfter mismatch
- Fixes mypy error: Argument expires_after has incompatible type
* fix: change purpose parameter type to OpenAIFilesPurpose (Todo 18)
- Import OpenAIFilesPurpose in storage_backend_service.py
- Change upload_file_to_storage_backend purpose parameter from str to OpenAIFilesPurpose
- Change _create_file_object_with_storage_metadata purpose parameter from str to OpenAIFilesPurpose
- Fixes mypy error: Argument purpose has incompatible type str; expected Literal type
- Purpose is already validated in files_endpoints.py before reaching these functions
* fix: handle UploadFile | str type for expires_after form fields (Todo 19)
- Validate expires_after[anchor] and expires_after[seconds] are strings, not UploadFiles
- Validate anchor equals 'created_at' before using literal in TypedDict
- Use literal 'created_at' (not variable) in FileExpiresAfter to satisfy Literal type
- Add proper error handling for invalid anchor values and int conversion
- Fixes mypy errors: Incompatible types for anchor and seconds in FileExpiresAfter
* fix: add type narrowing for expires_after_seconds_str to fix mypy error
- Add assert statement after UploadFile validation to help mypy narrow type
- Use validated variable with explicit str type annotation
- Fixes: Argument of type 'UploadFile | str' cannot be assigned to int()
* fix: trigger async_success_handler for MCP tool calls to enable cost tracking and logging
- Set call_type to CallTypes.call_mcp_tool.value before calling async_success_handler
- Update mcp_tool_call_metadata with cost info when server is found
- Call async_success_handler to build standard_logging_object and trigger callbacks
- Fixes test_mcp_cost_tracking by ensuring standard_logging_payload is populated
* refactor: use positive isinstance check for safer type narrowing
- Replace assert with positive isinstance(..., str) check
- Matches codebase pattern (see pass_through_endpoints.py)
- Safer than assert: assertions can be disabled with -O flag
- Mypy properly narrows type after positive isinstance check
- More explicit and readable than assert statement
* fix: add missing REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE to ServiceTypes enum (Todo 17)
- Add REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE enum value following the pattern of other daily spend queues
- Add corresponding entry to DEFAULT_SERVICE_CONFIGS with GAUGE metrics
- Fixes mypy error: 'type[ServiceTypes]' has no attribute 'REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE'
- This enum value is already used in redis_update_buffer.py for agent spend tracking
Add support for the 'xhigh' reasoning effort level on all gpt-5.2 model
variants, not just gpt-5.2-pro. This enables deeper reasoning capabilities
for the base gpt-5.2 model.
Changes:
- Add is_model_gpt_5_2_model() method to detect gpt-5.2 variants
- Update xhigh validation to allow gpt-5.2 models
- Update documentation with gpt-5.2 reasoning_effort support
- Update tests to reflect new behavior
* feat(deepseek): add native support for thinking and reasoning_effort params
Add proper parameter mapping for DeepSeek thinking mode, allowing users
to use the unified LiteLLM interface instead of extra_body workarounds.
Supported formats:
- thinking={"type": "enabled"}
- thinking={"type": "enabled", "budget_tokens": X} (budget_tokens ignored)
- reasoning_effort="low|medium|high" (maps to thinking enabled)
DeepSeek only supports {"type": "enabled"} without budget_tokens,
so any budget_tokens are stripped and all reasoning_effort values
(except "none") map to enabled.
Reference: https://api-docs.deepseek.com/guides/thinking_mode
* docs(deepseek): add thinking and reasoning_effort parameter documentation
* fix: resolve mypy type errors in hiddenlayer guardrail and transformation
- Fix return type of apply_guardrail from str to GenericGuardrailAPIInputs
- Add None checks for logging_obj before accessing attributes
- Convert AllMessageValues to dict format for HiddenLayer API compatibility
- Fix payload type annotation in _call_hiddenlayer
- Ensure transformed_output always returns list[dict[str, Any]] in transformation.py
* fix: use litellm_call_id as trace_id fallback in langfuse logging
- Only use standard_logging_object.trace_id if explicitly set via litellm_session_id or litellm_trace_id params
- Fallback to litellm_call_id when no explicit trace_id is provided (matches test expectation)
- Return the trace_id we set instead of generation_client.trace_id for consistency
- Add warning if langfuse modifies the trace_id to help debug potential issues
Fixes test_logging_trace_id test failure where auto-generated UUID was used instead of litellm_call_id
* fix: document envs
* fix: handle None response in /spend/logs endpoint when no records found
- Return empty list [] instead of [None] when spend_log is None
- Prevents 500 errors when querying by request_id, api_key, or user_id with no matching records
- Fixes test_chat_completion_bad_model_with_spend_logs test failure
* fix: use standard_logging_object trace_id when available in langfuse logger
- Fix trace_id selection logic to use standard_logging_object.trace_id when available
- Previously only used standard_logging_object.trace_id if explicitly set via params
- Now uses standard_logging_object.trace_id whenever it's present, matching test expectations
- Falls back to litellm_call_id if no trace_id is found
- Fixes test_log_langfuse_v2_uses_standard_trace_id_when_available test failure
Add support for the Bedrock Converse API serviceTier parameter to allow
specifying processing tier (priority, default, or flex).
Changes:
- Add ServiceTierBlock type in litellm/types/llms/bedrock.py
- Add serviceTier to CommonRequestObject
- Add serviceTier to get_config_blocks() in AmazonConverseConfig
- Add comprehensive tests for serviceTier functionality
- Add documentation for serviceTier usage
This allows users to configure service tier via:
- litellm_params in proxy config
- optional_params in SDK calls
* docs(prompt_management.md): document how to onboard prompts to litellm
* feat(arize_phoenix_prompt_manager.py): support new prompt management integration
allows users to connect arize phoenix prompt manager to litellm
* fix(proxy/utils.py): remove prompt variables to avoid re-processing prompt
* docs(arize_phoenix_prompts.md): document new prompt management integration
The relative link was causing Docusaurus to incorrectly associate the
/supported_endpoints page with SDK Functions category instead of the
actual Supported Endpoints generated-index.
* Add community contribution guide for integration partners
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Update community docs to direct users to #integration-partners
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(voyage): add rerank API support
Add support for Voyage AI rerank models (rerank-2.5, rerank-2.5-lite,
rerank-2, rerank-2-lite) to the LiteLLM rerank API.
Changes:
- Add VoyageRerankConfig transformation class
- Register voyage provider in rerank_api/main.py
- Add voyage case in utils.py get_provider_rerank_config
- Add rerank-2.5 and rerank-2.5-lite models to pricing JSON
- Add unit tests for transformation logic
- Update documentation for voyage.md and rerank.md
Usage:
```python
from litellm import rerank
response = rerank(
model="voyage/rerank-2.5",
query="What is the capital of France?",
documents=["Paris is...", "London is..."],
top_n=3,
)
```
* refactor(voyage): simplify rerank transformation code
Remove verbose docstrings to align with other providers (jina_ai pattern).
No functional changes - 168 lines vs 169 for jina_ai.
* fix(voyage): remove incorrect input_cost_per_query from rerank models
Voyage AI charges per token, not per query. The input_cost_per_query
field was incorrectly set to the same value as input_cost_per_token
in the existing rerank-2 and rerank-2-lite models.
Removes input_cost_per_query from all Voyage rerank models:
- voyage/rerank-2
- voyage/rerank-2-lite
- voyage/rerank-2.5
- voyage/rerank-2.5-lite
Pricing source: https://docs.voyageai.com/docs/pricing
* attempt to implement the passthrough feature
* Formatting and small change
* Fix formatting
* feat: grayswan guardrail overwrite ModelResponse in passthrough mode
* fix missing exception error catching on certain
endpoints
* fix wrong call site
* fix: patch anthropic endpoint internal error on streaming obj
* fix grayswan testcase
* feat: update the violation response to more natural
* Formatting
* move passthrough exception definition to custom_guardrail.
* Enhancement: show whether the blocked at input or output
* update exception name
* fix a typo in testing unit.
---------
Co-authored-by: Xiaohan Fu <xiaohan@grayswan.ai>
* docs: update Getting Started page with accurate endpoints and fix exception handling
- Update endpoints list to include /responses, /audio, /batches
- Change "Consistent output" to be endpoint-agnostic
- Clarify Response Format title as "OpenAI Chat Completions Format"
- Fix exception handling example: use litellm exceptions instead of deprecated openai.error
- Add model prefix (anthropic/) to example
* docs: reorganize sidebar and improve SDK documentation structure
Sidebar changes:
- Reorder: Python SDK first, then AI Gateway (Proxy)
- Rename "LiteLLM - Getting Started" to "Getting Started"
- Restructure SDK section with Core Functions, Configuration subsections
- Move budget_manager to Guides
- Move sdk_custom_pricing and migration to Extras
- Remove duplicate embedding/async_embedding and embedding/moderation
Content changes:
- Add Response Format section to response_api.md
- Add async aembedding() section to supported_embedding.md
* docs: add deprecation notice for OpenAI Assistants API
OpenAI has deprecated the Assistants API, shutting down on August 26, 2026.
Added warning banner directing users to the Responses API.
* docs: expand Core Functions in SDK sidebar
Add more SDK functions to Core Functions category:
- text_completion()
- image_generation()
- transcription()
- speech()
- Link to "All Supported Endpoints" for complete list
* Rename Sidebar Item
* docs: revert Getting Started label to original
* Rename sidebar label from 'LiteLLM - Getting Started' to 'Getting Started'
* fix: correct type annotations for anthropic streaming handlers
- Fix return type of _handle_accumulated_json_chunk from Optional[GenericStreamingChunk] to Optional[ModelResponseStream]
- Fix return type of _parse_sse_data from Optional[GenericStreamingChunk] to Optional[ModelResponseStream]
- Add type annotation for output_items in background_streaming.py
These changes align type annotations with actual return values from chunk_parser() which returns ModelResponseStream.
* docs: add missing ONYX_API_KEY and ONYX_API_BASE to environment variables reference
- Add ONYX_API_BASE documentation entry
- Add ONYX_API_KEY documentation entry
- Fixes test_env_keys.py test failure
* Add Amazon Nova as a first party provider
* Added new provider folder under llms/ to outline the openai supported params
* Updated supported endpoints on the documnetation
* docs: add Microsoft GraphRAG to projects using LiteLLM
* docs: add arXiv paper link for GraphRAG
* docs: add GraphRAG to sidebar
* Update projects in sidebars.js
Reordered items in the projects list to include 'GraphRAG'.
* fix(responses): Add image generation support for Responses API
Fixes#16227
## Problem
When using Gemini 2.5 Flash Image with /responses endpoint, image generation
outputs were not being returned correctly. The response contained only text
with empty content instead of the generated images.
## Solution
1. Created new `OutputImageGenerationCall` type for image generation outputs
2. Modified `_extract_message_output_items()` to detect images in completion responses
3. Added `_extract_image_generation_output_items()` to transform images from
completion format (data URL) to responses format (pure base64)
4. Added `_extract_base64_from_data_url()` helper to extract base64 from data URLs
5. Updated `ResponsesAPIResponse.output` type to include `OutputImageGenerationCall`
## Changes
- litellm/types/responses/main.py: Added OutputImageGenerationCall type
- litellm/types/llms/openai.py: Updated ResponsesAPIResponse.output type
- litellm/responses/litellm_completion_transformation/transformation.py:
Added image detection and extraction logic
- tests/test_litellm/responses/litellm_completion_transformation/test_image_generation_output.py:
Added comprehensive unit tests (16 tests, all passing)
## Result
/responses endpoint now correctly returns:
```json
{
"output": [{
"type": "image_generation_call",
"id": "..._img_0",
"status": "completed",
"result": "iVBORw0KGgo..." // Pure base64, no data: prefix
}]
}
```
This matches OpenAI Responses API specification where image generation
outputs have type "image_generation_call" with base64 data in "result" field.
* docs(responses): Add image generation documentation and tests
- Add comprehensive image generation documentation to response_api.md
- Include examples for Gemini (no tools param) and OpenAI (with tools param)
- Document response format and base64 handling
- Add supported models table with provider-specific requirements
- Add unit tests for image generation output transformation
- Test base64 extraction from data URLs
- Test image generation output item creation
- Test status mapping and integration scenarios
- Verify proper transformation from completions to responses format
Related to #16227
* fix(responses): Correct status type for image generation output
- Add _map_finish_reason_to_image_generation_status() helper function
- Fix MyPy type error: OutputImageGenerationCall.status only accepts
['in_progress', 'completed', 'incomplete', 'failed'], not the full
ResponsesAPIStatus union which includes 'cancelled' and 'queued'
Fixes MyPy error in transformation.py:838
When Gemini image generation models return `text_tokens=0` with `image_tokens > 0`,
the cost calculator was assuming no token breakdown existed and treating all
completion tokens as text tokens, resulting in ~10x underestimation of costs.
Changes:
- Fix cost calculation logic to respect token breakdown when image/audio/reasoning
tokens are present, even if text_tokens=0
- Add `output_cost_per_image_token` pricing for gemini-3-pro-image-preview models
- Add test case reproducing the issue
- Add documentation explaining image token pricing
Fixes#17410
* fix: resolve code quality issues from ruff linter
- Fix duplicate imports in anthropic guardrail handler
- Remove duplicate AllAnthropicToolsValues import
- Remove duplicate ChatCompletionToolParam import
- Remove unused variable 'tools' in guardrail handler
- Replace print statement with proper logging in json_loader
- Use verbose_logger.warning() instead of print()
- Remove unused imports
- Remove _update_metadata_field from team_endpoints
- Remove unused ChatCompletionToolCallChunk imports from transformation
- Refactor update_team function to reduce complexity (PLR0915)
- Extract budget_duration handling into _set_budget_reset_at() helper
- Minimal refactoring to reduce function from 51 to 50 statements
All ruff linter errors resolved. Fixes F811, F841, T201, F401, and PLR0915 errors.
* docs: add missing environment variables to documentation
Add 8 missing environment variables to the environment variables reference section:
- AIOHTTP_CONNECTOR_LIMIT_PER_HOST: Connection limit per host for aiohttp connector
- AUDIO_SPEECH_CHUNK_SIZE: Chunk size for audio speech processing
- CYBERARK_SSL_VERIFY: Flag to enable/disable SSL certificate verification for CyberArk
- LITELLM_DD_AGENT_HOST: Hostname or IP of DataDog agent for LiteLLM-specific logging
- LITELLM_DD_AGENT_PORT: Port of DataDog agent for LiteLLM-specific log intake
- WANDB_API_KEY: API key for Weights & Biases (W&B) logging integration
- WANDB_HOST: Host URL for Weights & Biases (W&B) service
- WANDB_PROJECT_ID: Project ID for Weights & Biases (W&B) logging integration
Fixes test_env_keys.py test that was failing due to undocumented environment variables.
* fix(generic_guardrail_api.py): add 'structured_messages' support
allows guardrail provider to know if text is from system or user
* fix(generic_guardrail_api.md): document 'structured_messages' parameter
give api provider a way to distinguish between user and system messages
* feat(anthropic/): return openai chat completion format structured messages when calls made via `/v1/messages` on Anthropic
* feat(responses/guardrail_translation): support 'structured_messages' param for guardrails
structured openai chat completion spec messages, for guardrail checks when using /v1/responses api
allows guardrail checks to work consistently across APIs
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail
* fix: add more rigorous call type checks
* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint
ensures call id + trace id are emitted to guardrail api
* feat(anthropic/chat/guardrail_translation): support streaming guardrails
sample on every 5 chunks
* fix(openai/chat/guardrail_translation): support openai streaming guardrails
* fix: initial commit fixing output guardrails for responses api
* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api
* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming
* test: update tests
* test: update tests
* fix: support multiple kinds of input to the guardrail api
* feat(guardrail_translation/handler.py): support extracting tool calls from openai chat completions for guardrail api's
* feat(generic_guardrail_api.py): support extracting + returning modified tool calls on generic_guardrails_api
allows guardrail api to analyze tool call being sent to provider - to run any analysis on it
* fix(guardrails.py): support anthropic /v1/messages tool calls
* feat(responses_api/): extract tool calls for guardrail processing
* docs(generic_guardrail_api.md): document tools param support
* docs: generic_guardrail_api.md
improve documentation
Add Agent Lightning, Microsoft's open-source framework for training
AI agents with RL, APO, and SFT. Uses LiteLLM Proxy for LLM routing
and trace collection.
Both frameworks integrate with LiteLLM:
- Google ADK uses LiteLLM for model-agnostic agent building
- Harbor uses LiteLLM for agent evaluation across providers
* docs: update getting started page
- Add Core Functions table with link to full list
- Add Responses API section
- Add Async section with acompletion() example
- Add "Switch Providers with One Line" example
- Clarify Basic Usage supports multiple endpoints
- Update models to current versions (openai/gpt-4o, anthropic/claude-sonnet-4)
- Use provider/model format throughout
- Fix deprecated import: from openai.error -> from openai
- Keep original structure: community key, More details links, observability env vars
* Cleanup: Remove orphan docs pages and Docusaurus template files
- Remove orphan getting_started.md (not linked in sidebar)
- Remove Docusaurus template intro.md
- Remove tutorial-basics/ directory (Docusaurus template)
- Remove tutorial-extras/ directory (Docusaurus template)
* docs vertex tts
* place vertex ai types in file
* use VertexAITextToSpeechConfig
* use vertex_voice_dict
* refactor docs
* docs vertex ai chirp
* TestVertexAITextToSpeechConfig
* new provider vertex ai chirp3
* test_litellm_speech_vertex_ai_chirp
* add vertex_ai/chirp cost trackign
* docs: add Azure AI Foundry documentation for Claude models
Add documentation explaining how to use Claude models (Sonnet 4.5,
Haiku 4.5, Opus 4.1) deployed on Azure AI Foundry with LiteLLM.
Azure exposes Claude using Anthropic's native API, so users can use
the existing anthropic/ provider with their Azure endpoint.
Closes#17066
* docs: Add alternative method for Azure AI Foundry using anthropic/ provider
Document that users can use anthropic/ provider with Azure endpoint
as an alternative to the dedicated azure_ai/ provider.
* Support for Custom Vertex AI Models via PSC Endpoint with api_base
* Add docs related psc
* remove not needed files
* remove print statemnt
* fix mypy errors
* refactor(generic_guardrail_api.py): refactor to update to new guardrail api logic
* refactor: refactor llm api integrations to support passing in text as a list[str] instead of one at a time
* refactor: fix linting errors
* refactor: pass request type to guardrail api
allows request vs. response processing to occur
* feat: pass user api key dict information to the guardrail api
* fix: pass user api key dict information to the guardrail api
* feat: pass litellm call id + trace id, if present
* docs: update docs
* feat(generic_guardrail_api.py): new generic api for guardrails
Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM
* docs(generic_guardrail_api.md): document new generic guardrail api
* Fix: Improve PII detection and guardrail API integration
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* feat: correctly extract raw request from guardrail api
* docs(generic_guardrail_api.md): document this is a beta feature
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat: allow fetching OIDC user info
* test: use test_auth_builder_with_oidc_userinfo_enabled gets user info when enabled
* fix tool permission doc
* docs fix diagram
* store and fetch lasso-conversation id from cache
* include gateway/v# in the baseUrl to allow simpler version migrations in the future
* add tests for cached conversation ID
* docs: add OpenAI Agents SDK to projects
Add documentation for OpenAI Agents SDK which has an official
LiteLLM extension enabling 100+ LLM providers in multi-agent workflows.
* Update project items in sidebars.js
* feat(generic_api_callback.py): make generic api OSS + support multiple generic API's
Enables https://github.com/BerriAI/litellm/pull/17094#discussion_r2562832967
* feat(callback_utils.py): support custom generic api callbacks
* feat(generic_api_callback.py): support specifying which event types to run the generic api for
* fix(litellm_logging.py): log system prompt for anthropic messages
* feat(generic_api_callback.py): support generic api compatible api's - e.g. rubrik agent cloud
* docs(sidebars.js): document new OSS generic api
* docs(generic_api.md): document new OSS Generic API
* docs(custom_webhook_api.md): document custom webhook api integration tutorial
* docs(custom_webhook_api.md): cleanup
* docs(custom_webhook_api.md): document what get's logged to custom webhook api
* Refactor: Pass callback config to GenericAPILogger
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Fix: Handle empty messages list in logging payload
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Checkpoint before follow-up message
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* feat: Cache GenericAPILogger instances to improve performance
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
- Add model identifier to FLASH_IMAGE_PREVIEW_MODEL_IDENTIFIERS
- Add imageSize parameter support (1K, 2K, 4K) with GeminiImageSize type
- Add tests for imageSize parameter transformation
- Update documentation with new model
* Added tool search support for anthropic
* Add programtic tool calling support
* Add tool use input examples support
* Add anthropic effort param support
* Add anthropic effort param support
* Add blog for new features
* fix mypy and lint errors
* fix mypy and lint errors
* fix mypy and lint errors
* fix mypy and lint errors
* Add better handling
* Add better handling
* fix(gemini): exclude image models from automatic thinking_level parameter (#17013)
- gemini-3-pro-image-preview does not support thinking_level parameter
- Added check to skip adding thinkingConfig for models containing "image"
- Fixes BadRequestError: "Thinking level is not supported for this model"
- Only affects automatic default behavior, user can still pass reasoning_effort explicitly
Fixes#17013
* test: add tests for gemini-3 image models thinking_level exclusion
* update docs
* add claude opus 4.5
* Apply suggestion from @Chesars
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
---------
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
* add prompt security guardrails provider
* cosmetic
* small
* add file sanitization and update context window
* add pdf and OOXML files support
* add system prompt support
* add tests and documentation
* remove print
* fix PLR0915 Too many statements (96 > 50)
* cosmetic
* fix mypy error
* Fix failed tests due to naming conflict of responses directory with same-named pip package
* Fix mypy error: use 'aembedding' instead of 'embeddings' for async embedding call type
* Fix: Install enterprise package into Poetry virtualenv for tests
The GitHub Actions workflow was installing litellm-enterprise to system Python
using 'python -m pip install -e .', but tests run in Poetry's virtualenv using
'poetry run pytest'. This caused ImportError for enterprise package types.
Changed to 'poetry run pip install -e .' so the package is available in the
same virtualenv where pytest executes.
Fixes enterprise test collection errors in GitHub Actions CI.
* Move Prompt Security guardrail tests to tests/test_litellm/
Per reviewer feedback, move test_prompt_security_guardrails.py from
tests/guardrails_tests/ to tests/test_litellm/proxy/guardrails/ so
it will be executed by GitHub Actions workflow test-litellm.yml.
This ensures the Prompt Security integration tests run in CI.
---------
Co-authored-by: Ori Tabac <oritabac@prompt.security>
Co-authored-by: Vitaly Neyman <vitaly@prompt.security>
* Add doc for adding model pricing and context window
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Refactor model pricing documentation to include sample spec and examples
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* Litellm dev 11 22 2025 p1 (#16975)
* fix(model_armor.py): return response after applying changes
* fix: initial commit adding guardrail span logging to otel on post-call runs
sends it as a separate span right now, need to include in the same llm request/response span
* fix(opentelemetry.py): include guardrail in received request log + set input/ouput fields on parent otel span instead of nesting it
allows request/response to be seen easily on observability tools
* fix(model_armor.py): working model armor logging on post call events
* fix: fix exception message
* fix(opentelemetry.py): add backwards compatibility for litellm_request
allow users building on the spec change to use previous spec
* fix: prevent memory blowout in LoggingWorker
Tasks were being executed sequentially with each task awaited before
processing the next one. When the queue had 10k+ tasks, only one could
execute at a time. Since the request rate exceeded execution speed,
objects accumulated in memory (50k+), holding references to heavy
objects and causing memory blowout.
The new implementation uses a semaphore to allow up to 1000 concurrent
tasks while properly tracking and cleaning up each task, significantly
improving throughput and preventing queue buildup.
* fix: require semaphor before removing task from queue
* fix: make worker concurrency configurable
* fix: clean comments
* fix: clarify new env purpose
* fix: add missing lib
* make constants configurable instead of hardcoded
* add more aggressive cleaning when queue is full
* add helpers function for the aggressive cleaning functionality
* use envs instead of static constants
* import and document constants
* add unit test for new functionality
* fix default value on config_settings
* fix: remove unused variables and imports to resolve linter errors
- Remove unused time_since_last_clear variable in logging_worker.py
The variable was calculated but never used in _handle_queue_full()
method, causing F841 linter error.
- Remove unused TYPE_CHECKING import in mcp_server/server.py
The import was not used anywhere in the file, causing F401 linter error.
These changes improve code cleanliness and ensure the codebase passes
all linter checks without affecting functionality.
* add missing log expected by test_queue_full_handling
* fix: clean config_setting.md file
* fix: handle logging errors gracefully during shutdown in _flush_on_exit
During process shutdown, logging handlers may be closed while _flush_on_exit
tries to flush queued logging coroutines. This causes 'ValueError: I/O
operation on closed file' errors when coroutines attempt to log.
Changes:
- Add _safe_log helper method that wraps logging calls and suppresses
errors when logging handlers are closed (ValueError, OSError, AttributeError)
- Replace all verbose_logger calls in _flush_on_exit with _safe_log
- Remove logging from exception handler in coroutine execution loop
to prevent cascading errors during shutdown
This ensures graceful shutdown even when logging handlers are closed,
which is common during process termination.
* feat(anthropic/chat/transformations): for claude-4-5-sonnet and opus-4-1 support passing structured output to anthropic api
* docs: document new feature
* fix: fix output format
* fix: cleanup
* fix(transformation.py): conditionally pass in json tool call
* fix: support ARIZE_SPACE_ID instead of ARIZE_SPACE_KEY
* docs(arize_integration.md): cleanup arize docs
* feat(callback_info_helpers.tsx): allow setting arize space id via ui
* fix: fix linting error
* fix(opentelemetry.py): working arize phoenix root span tracing
* docs: Add mini-swe-agent to projects page
Add mini-swe-agent to the documentation projects page.
mini-swe-agent is a minimal AI coding agent that resolves >70% of
GitHub issues in SWE-bench, built on LiteLLM for model flexibility.
- Added projects/mini-swe-agent.md documentation
- Updated sidebars.js to include mini-swe-agent in projects list
* docs: Update Singularity to Apptainer in mini-swe-agent.md
* Add openai metadata filed in the request
* Add docs related to openai metadata
* Add utils
* test_completion_openai_metadata[True]
* Added support for though signature for gemini 3 in responses api (#16872)
* Added support for though signature for gemini 3
* Update docs with all supported endpoints and cost tracking
* Added config based routing support for batches and files
* fix lint errors
* Litellm anthropic image url support (#16868)
* Add image as url support to anthropic
* fix mypy errors
* fix tests
* Fix: Populate spend_logs_metadata in batch and files endpoints (#16921)
* Add spend-logs-metadata to the metadata
* Add tests for spend logs metadata in batches
* use better names
* Remove support for penalty param for gemini 3 (#16907)
* Remove support for penalty param
* remove halucinated model names
* fix mypy/test errors
* fix tests
* fix too many lines error
* fix too many lines error
* Add config for cicd test case
* Fix final tests
* fix batch tests
* fix batch tests
* attempt to implement the passthrough feature
* Formatting and small change
* Fix formatting
* Format test file
---------
Co-authored-by: Xiaohan Fu <xiaohan@grayswan.ai>
