Remove 'none' from gpt-5-mini's supported reasoning_effort values in the documentation table. gpt-5-mini does not support reasoning_effort="none", only minimal, low, medium, and high.
- Add OAuth M2M (Machine-to-Machine) authentication via DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET
- Add Databricks SDK auto-auth with automatic credential discovery
- Add sensitive data redaction for secure logging (tokens, API keys, secrets)
- Add custom user_agent parameter for partner attribution in Databricks telemetry
- Support user_agent in LiteLLM Proxy via config.yaml litellm_params
- Add 49 mocked unit tests for all new functionality
- Add 13 E2E tests for real-world validation (skipped in CI)
- Update documentation with new features and examples
Added concise PostgreSQL and Redis specifications based on benchmark results and industry standards for API gateway deployments. Includes tiered recommendations for different RPS workloads, configuration best practices, and scaling guidelines.
* Add monitor mode support to Lakera guardrail
- Add on_flagged parameter to LakeraV2GuardrailConfigModel (default: 'block')
- Support 'monitor' mode that logs violations without blocking requests
- Support 'block' mode (default) that raises HTTPException on violations
- Update async_pre_call_hook and async_moderation_hook to check on_flagged
- Update guardrail initializer to pass on_flagged from config
- Add documentation with monitor mode examples
This allows users to tune Lakera security policies by monitoring violations
without blocking legitimate requests, similar to Pillar's on_flagged_action.
* Add tests for Lakera guardrail monitor mode
- Test monitor mode allows flagged content through (pre_call hook)
- Test block mode raises HTTPException for violations (pre_call hook)
- Test monitor mode works with during_call (moderation_hook)
These tests verify the on_flagged parameter functionality for both
monitor and block modes across different guardrail hooks.
---------
Co-authored-by: Steve <steve.giguere@lakera.ai>
* feat(litellm_content_filter.py): add support for content filtering categories
make it easy for proxy admin to prevent messages about violence, self harm or illegal weapons going through litellm
* feat: initial commit adding bias detection
allows admin to block inappropriate content about sexual orientation, etc.
* refactor: simplify content_filter.py
use a more exhaustive set of keywords, instead of guessing at potential phrases user can use
* feat(content_filter.py): add new denied topics for in-built content filter guardrails
allow user to automatically block content relating to certain categories from being sent to the LLML
* refactor(content-filter): document new params to litellm content filter
* feat(ui/): litellm content filter - select content categories on ui
* docs: update documentation
* docs(litellm_content_filter.md): document new content filters
* feat: initial commit adding support for inappropriate images via litellm content filter
* feat(content_filter.py): support blocking images containing blocked content
prevent images which contain disallowed content from being sent to the llm api
* docs(litellm_content_filter.md): document new image capabilities of litellm_content_filter
* fix: fix expected error code
* feat(litellm_content_filter.py): add support for content filtering categories
make it easy for proxy admin to prevent messages about violence, self harm or illegal weapons going through litellm
* feat: initial commit adding bias detection
allows admin to block inappropriate content about sexual orientation, etc.
* refactor: simplify content_filter.py
use a more exhaustive set of keywords, instead of guessing at potential phrases user can use
* feat(content_filter.py): add new denied topics for in-built content filter guardrails
allow user to automatically block content relating to certain categories from being sent to the LLML
* refactor(content-filter): document new params to litellm content filter
* feat(ui/): litellm content filter - select content categories on ui
* docs: update documentation
* docs(litellm_content_filter.md): document new content filters
- Remove 20 deprecated/unavailable Groq models from registry
- Add groq/meta-llama/llama-guard-4-12b (new safety model)
- Add supports_vision to Llama 4 models (maverick, scout)
- Update Groq documentation with current model list
- Clean up test file references to deprecated models
Fixes#18043
- Add 'mask' action to SUPPORTED_ON_FLAGGED_ACTIONS
- Automatically sanitizes sensitive content using masked_session_messages
- Allows requests to proceed with masked content instead of blocking
- Add MCP call support
- Add pre_mcp_call and during_mcp_call to supported_event_hooks
- Verify mcp_call is supported in call_type Literal types
- Control exception details based on config
- Conditionally include scanners/evidence in exceptions based on
include_scanners and include_evidence settings
- Reduces payload size when detailed exception info isn't needed
- Add comprehensive test coverage
- Tests for masking functionality
- Tests for conditional exception details
- Tests for MCP call support
- Update documentation
- Add Mask section explaining masking functionality
- Clarify exception details control
All changes maintain backward compatibility.
* feat(custom_llm): add image_edit and aimage_edit support
Add support for image_edit and aimage_edit methods in CustomLLM class,
allowing users to implement custom image editing providers.
Changes:
- Add image_edit() and aimage_edit() methods to CustomLLM base class
- Add custom provider detection in litellm.image_edit() function
- Add tests for sync and async image_edit with custom handlers
* docs: add image_edit to CustomLLM documentation
- Add /v1/images/edits to supported routes
- Add Image Edit section with example
- Update Custom Handler Spec with image_edit methods
* docs: add OpenAI-compatible API limitations for Anthropic thinking
Document the fundamental incompatibility between Anthropic extended
thinking and OpenAI-compatible API clients. Explains:
- Why thinking_blocks must be resent (stateless vs stateful APIs)
- OpenAI vs Anthropic architecture differences
- Solutions for client developers
* Update docs
* fix: auto-drop thinking param when thinking_blocks missing
When modify_params=True, LiteLLM now automatically drops the 'thinking'
param if the last assistant message with tool_calls is missing
thinking_blocks. This prevents the Anthropic error:
"Expected thinking or redacted_thinking, but found tool_use"
This workaround addresses the OpenAI-Anthropic API incompatibility where
OpenAI-compatible clients don't preserve thinking_blocks.
Add direct Stability AI REST API support for image generation endpoints.
This enables using Stability's SD3, SD3.5, and Stable Image models via
LiteLLM's OpenAI-compatible interface.
Changes:
- Add STABILITY provider to LlmProviders enum
- Create StabilityImageGenerationConfig with multipart/form-data support
- Add OpenAI size to Stability aspect_ratio mapping
- Register provider in ProviderConfigManager
- Add 9 Stability models to model_prices_and_context_window.json
- Add documentation at docs/providers/stability.md
- Add 25 unit tests
Supported models:
- stability/sd3, sd3-large, sd3-large-turbo, sd3-medium
- stability/sd3.5-large, sd3.5-large-turbo, sd3.5-medium
- stability/stable-image-ultra, stable-image-core
* Fix duplicate imports in SAP embedding transformation
* fix: add missing prompt_spec parameter to HumanloopLogger.get_chat_completion_prompt
- Add prompt_spec: Optional[PromptSpec] = None parameter to match base class signature
- Import PromptSpec from litellm.types.prompts.init_prompts
- Pass prompt_spec to super().get_chat_completion_prompt() call
- Fixes mypy type error: Signature incompatible with supertype CustomLogger
* fix: add missing parameters to AnthropicCacheControlHook.async_get_chat_completion_prompt
- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Change litellm_logging_obj type from Any to LiteLLMLoggingObj using TYPE_CHECKING pattern
- Pass all parameters including prompt_spec to get_chat_completion_prompt call
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase
* fix: add missing parameters to DotpromptManager.async_get_chat_completion_prompt
- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Change litellm_logging_obj type from Any to LiteLLMLoggingObj using TYPE_CHECKING pattern
- Pass all parameters including ignore flags to PromptManagementBase.async_get_chat_completion_prompt
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase
* fix: document envs
* fix: add missing parameters to LangfusePromptManagement.async_get_chat_completion_prompt
- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Pass all parameters including prompt_spec and ignore flags to get_chat_completion_prompt
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase
* fix: add missing parameters to prompt management async methods (Category 1)
- vector_store_pre_call_hook: add ignore_prompt_manager_model, ignore_prompt_manager_optional_params, prompt_spec
- gitlab_prompt_manager: add ignore parameters, fix litellm_logging_obj type
- bitbucket_prompt_manager: add ignore parameters, fix litellm_logging_obj type
- proxy/custom_prompt_management: add prompt_spec parameter
- Fixes mypy type errors: Signature incompatible with supertype
* fix: fix arize_phoenix_prompt_manager and custom_prompt_management (Category 2)
- arize_phoenix_prompt_manager: add prompt_spec to all methods, fix prompt_id types, implement async_compile_prompt_helper
- custom_prompt_management: implement async_compile_prompt_helper abstract method
- Fixes mypy type errors: Signature incompatible with supertype and abstract method errors
* fix: fix obvious type errors (Category 3 - Quick Wins)
- langfuse: change 'callable' to 'Callable' type annotation
- presidio: add type narrowing check for Choices vs StreamingChoices
- StreamingChoices doesn't have .message attribute, only Choices does
- Add hasattr check before accessing choice.message
- Fixes mypy type errors: callable? not callable and union-attr errors
* fix: handle expires_after None in Azure files handler (Todo 14)
- Extract logic to _prepare_create_file_data helper method
- Remove expires_after from dict if None to match SDK's Omit pattern
- Add type ignore for FileExpiresAfter -> file_create_params.ExpiresAfter mismatch
- Fixes mypy error: Argument expires_after has incompatible type
* fix: change purpose parameter type to OpenAIFilesPurpose (Todo 18)
- Import OpenAIFilesPurpose in storage_backend_service.py
- Change upload_file_to_storage_backend purpose parameter from str to OpenAIFilesPurpose
- Change _create_file_object_with_storage_metadata purpose parameter from str to OpenAIFilesPurpose
- Fixes mypy error: Argument purpose has incompatible type str; expected Literal type
- Purpose is already validated in files_endpoints.py before reaching these functions
* fix: handle UploadFile | str type for expires_after form fields (Todo 19)
- Validate expires_after[anchor] and expires_after[seconds] are strings, not UploadFiles
- Validate anchor equals 'created_at' before using literal in TypedDict
- Use literal 'created_at' (not variable) in FileExpiresAfter to satisfy Literal type
- Add proper error handling for invalid anchor values and int conversion
- Fixes mypy errors: Incompatible types for anchor and seconds in FileExpiresAfter
* fix: add type narrowing for expires_after_seconds_str to fix mypy error
- Add assert statement after UploadFile validation to help mypy narrow type
- Use validated variable with explicit str type annotation
- Fixes: Argument of type 'UploadFile | str' cannot be assigned to int()
* fix: trigger async_success_handler for MCP tool calls to enable cost tracking and logging
- Set call_type to CallTypes.call_mcp_tool.value before calling async_success_handler
- Update mcp_tool_call_metadata with cost info when server is found
- Call async_success_handler to build standard_logging_object and trigger callbacks
- Fixes test_mcp_cost_tracking by ensuring standard_logging_payload is populated
* refactor: use positive isinstance check for safer type narrowing
- Replace assert with positive isinstance(..., str) check
- Matches codebase pattern (see pass_through_endpoints.py)
- Safer than assert: assertions can be disabled with -O flag
- Mypy properly narrows type after positive isinstance check
- More explicit and readable than assert statement
* fix: add missing REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE to ServiceTypes enum (Todo 17)
- Add REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE enum value following the pattern of other daily spend queues
- Add corresponding entry to DEFAULT_SERVICE_CONFIGS with GAUGE metrics
- Fixes mypy error: 'type[ServiceTypes]' has no attribute 'REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE'
- This enum value is already used in redis_update_buffer.py for agent spend tracking
Add support for the 'xhigh' reasoning effort level on all gpt-5.2 model
variants, not just gpt-5.2-pro. This enables deeper reasoning capabilities
for the base gpt-5.2 model.
Changes:
- Add is_model_gpt_5_2_model() method to detect gpt-5.2 variants
- Update xhigh validation to allow gpt-5.2 models
- Update documentation with gpt-5.2 reasoning_effort support
- Update tests to reflect new behavior
* feat(deepseek): add native support for thinking and reasoning_effort params
Add proper parameter mapping for DeepSeek thinking mode, allowing users
to use the unified LiteLLM interface instead of extra_body workarounds.
Supported formats:
- thinking={"type": "enabled"}
- thinking={"type": "enabled", "budget_tokens": X} (budget_tokens ignored)
- reasoning_effort="low|medium|high" (maps to thinking enabled)
DeepSeek only supports {"type": "enabled"} without budget_tokens,
so any budget_tokens are stripped and all reasoning_effort values
(except "none") map to enabled.
Reference: https://api-docs.deepseek.com/guides/thinking_mode
* docs(deepseek): add thinking and reasoning_effort parameter documentation
* fix: resolve mypy type errors in hiddenlayer guardrail and transformation
- Fix return type of apply_guardrail from str to GenericGuardrailAPIInputs
- Add None checks for logging_obj before accessing attributes
- Convert AllMessageValues to dict format for HiddenLayer API compatibility
- Fix payload type annotation in _call_hiddenlayer
- Ensure transformed_output always returns list[dict[str, Any]] in transformation.py
* fix: use litellm_call_id as trace_id fallback in langfuse logging
- Only use standard_logging_object.trace_id if explicitly set via litellm_session_id or litellm_trace_id params
- Fallback to litellm_call_id when no explicit trace_id is provided (matches test expectation)
- Return the trace_id we set instead of generation_client.trace_id for consistency
- Add warning if langfuse modifies the trace_id to help debug potential issues
Fixes test_logging_trace_id test failure where auto-generated UUID was used instead of litellm_call_id
* fix: document envs
* fix: handle None response in /spend/logs endpoint when no records found
- Return empty list [] instead of [None] when spend_log is None
- Prevents 500 errors when querying by request_id, api_key, or user_id with no matching records
- Fixes test_chat_completion_bad_model_with_spend_logs test failure
* fix: use standard_logging_object trace_id when available in langfuse logger
- Fix trace_id selection logic to use standard_logging_object.trace_id when available
- Previously only used standard_logging_object.trace_id if explicitly set via params
- Now uses standard_logging_object.trace_id whenever it's present, matching test expectations
- Falls back to litellm_call_id if no trace_id is found
- Fixes test_log_langfuse_v2_uses_standard_trace_id_when_available test failure
Add support for the Bedrock Converse API serviceTier parameter to allow
specifying processing tier (priority, default, or flex).
Changes:
- Add ServiceTierBlock type in litellm/types/llms/bedrock.py
- Add serviceTier to CommonRequestObject
- Add serviceTier to get_config_blocks() in AmazonConverseConfig
- Add comprehensive tests for serviceTier functionality
- Add documentation for serviceTier usage
This allows users to configure service tier via:
- litellm_params in proxy config
- optional_params in SDK calls
* docs(prompt_management.md): document how to onboard prompts to litellm
* feat(arize_phoenix_prompt_manager.py): support new prompt management integration
allows users to connect arize phoenix prompt manager to litellm
* fix(proxy/utils.py): remove prompt variables to avoid re-processing prompt
* docs(arize_phoenix_prompts.md): document new prompt management integration
The relative link was causing Docusaurus to incorrectly associate the
/supported_endpoints page with SDK Functions category instead of the
actual Supported Endpoints generated-index.
