Add cheap .get() guards in should_run_callback() to short-circuit
the expensive EnterpriseCallbackControls.is_callback_disabled_dynamically()
call. When neither litellm_disabled_callbacks nor x-litellm-disable-callbacks
header is set (the common case), the enterprise function is never entered,
reducing should_run_callback from ~485ms to ~93-165ms across 54k calls.
CallTypes(call_type) was constructing an enum from string on every call,
taking ~4.6µs/call (69.6% of function time). Replace with a frozenset
membership test for ~0.8µs/call (8.3x faster).
Guard verbose_logger.debug() f-strings with isEnabledFor(logging.DEBUG)
checks in the router and cost calculation hot paths. Python evaluates
f-string arguments before the logging framework checks the log level,
causing expensive formatting on every request even with debug logging
disabled.
Changes:
- Remove redundant litellm_params.copy() in _completion/_acompletion
- Guard 5 debug logs in router.py (+ remove 1 duplicate log)
- Guard 6 debug logs in cost_calculator.py and utils.py
- get_model_info(): formatted 50+ field dict every call
- _apply_cost_margin(): called list(dict.keys()) every request
Profiled improvement: completion_cost 769µs → 637µs/call (-17.2%)
* perf: Optimize get_litellm_params with sparse kwargs extraction
- Add _OPTIONAL_KWARGS_KEYS frozenset for O(1) lookups
- Replace 28 unconditional kwargs.get() calls with sparse extraction
- Only add kwargs keys that are actually present in the dict
- Simplify _get_base_model_from_litellm_call_metadata by removing redundant None checks
This reduces get_litellm_params() time by ~31% (743ms → 509ms across 6000 calls)
and Logging.__init__ total time by ~24% (1.61s → 1.23s).
* test: add unit tests for get_litellm_params sparse kwargs extraction
* perf: add early-exit guards in completion_cost for unused features
Skip function calls to get_cost_for_built_in_tools, _apply_cost_discount,
_apply_cost_margin, and _store_cost_breakdown_in_logging_obj when their
respective features are not configured. Reduces completion_cost() time
by ~20% (4.39s → 3.53s over 6K requests) for the common case where
built-in tools, discounts, margins, and logging object are not active.
* fix: always call get_cost_for_built_in_tools regardless of standard_built_in_tools_params
The function can detect web search usage from the usage object (e.g.
server_tool_use.web_search_requests, prompt_tokens_details.web_search_requests)
even when standard_built_in_tools_params is None, so guarding on it can
under-count cost for providers like Vertex AI and Anthropic.
Adds regression test for completion_cost with web search in usage but
no standard_built_in_tools_params.
* fix(tests): Mock async_container_create_handler for async router test
The test was mocking container_create_handler (sync), but
router.acreate_container uses _is_async=True which calls
async_container_create_handler. This caused the test to hit
the real OpenAI API.
Fixed by using AsyncMock on async_container_create_handler.
* fix(tests): Use uuid for unique model name in scientific notation test
The test was using a static "unique" model name which could cause
conflicts when running tests in parallel (-n 16 in CI). Using uuid
ensures truly unique names to prevent test pollution.
---------
Co-authored-by: Shin <shin@openclaw.ai>
* perf: Optimize get_standard_logging_metadata with set intersection
- Cache StandardLoggingMetadata.__annotations__.keys() as module-level frozenset
- Use set intersection to iterate only keys present in both metadata and supported keys
- Single lookup for user_api_key instead of 3 separate .get() calls
Results:
- get_standard_logging_metadata: 1.55s → 1.41s (9.2% faster)
* test: add unit tests for get_standard_logging_metadata non-string user_api_key handling
* fix(tests): Fix sendgrid email tests to properly mock httpx client
The tests were potentially hitting the real SendGrid API because the mock
was patching get_async_httpx_client() but the actual client could be cached
or the mock timing could be off.
Fix by directly replacing logger.async_httpx_client after instantiation,
which guarantees the mock is used regardless of caching or initialization
timing issues.
Changes:
- Replace mock_httpx_client fixture with simpler mock_async_client fixture
- Directly inject mock client into logger instance after creation
- Remove respx decorator (no longer needed with direct injection)
- Simplify test structure while maintaining same assertions
* fix(lint): remove unused imports from SendGrid test
Extract the allowed MCP servers resolution logic with IP filtering into
a dedicated helper function _resolve_allowed_mcp_servers_with_ip_filter.
This reduces the statement count in call_tool_rest_api from 51 to under
50, fixing the ruff PLR0915 (too many statements) lint error.
Co-authored-by: Shin <shin@openclaw.ai>
Replace text-embedding-004 with gemini-embedding-001.
The old model was deprecated and returns 404:
'models/text-embedding-004 is not found for API version v1beta'
Co-authored-by: Shin <shin@openclaw.ai>
- a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment
- caching/redis_cache.py: Add type ignore for async ping return type
- caching/redis_cluster_cache.py: Add type ignore for async ping return type
- llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text
- proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument
All changes add appropriate type: ignore comments to handle library typing inconsistencies.
* fix(mypy): resolve missing return statements and type casting issues
* fix(pangea): use elif to prevent UnboundLocalError and handle None messages
Address Greptile review feedback:
- Make branches mutually exclusive using elif to prevent input_messages from being overwritten
- Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API
---------
Co-authored-by: Shin <shin@openclaw.ai>
* fix: fix styling
* fix(custom_code_guardrail.py): add http support for custom code guardrails
allows users to call external guardrails on litellm with minimal code changes (no custom handlers)
Test guardrail integrations more easily
* feat(a2a/): add guardrails for agent interactions
allows the same guardrails for llm's to be applied to agents as well
* fix(a2a/): support passing guardrails to a2a from the UI
* style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails
* feat(mcp/): support custom code guardrails for mcp calls
allows custom code guardrails to work on mcp input
* feat(chatui.tsx): support guardrails on mcp tool calls on playground
Notes: General support for Opus 4.6 was added in #20506 however
it omitted the AU (australian) specific instance profile used
in Bedrock. This change only adds the the au id. It is copied
from the US model settings which is consistent with past
additions of this regional model profile.
Keycloak (and similar OIDC providers) include role claims in the JWT
access token but not in the UserInfo endpoint response. Previously,
roles were only extracted from UserInfo, causing all SSO users to
default to internal_user_view_only regardless of their actual role.
Changes:
- Extract user roles from JWT access token in process_sso_jwt_access_token()
when UserInfo doesn't provide them (tries role_mappings first, then
GENERIC_USER_ROLE_ATTRIBUTE)
- Handle list-type role values in get_litellm_user_role() since Keycloak
returns roles as arrays (e.g. ["proxy_admin"] instead of "proxy_admin")
- Add 9 new unit tests covering role extraction and list handling
- Update 3 existing tests for new JWT decode behavior
Closes#20407
- process_mcp_request() now falls back to OAuth2 passthrough when Authorization header contains a non-LiteLLM token (catches HTTPException and ProxyException 401/403)
- MCPClient._get_auth_headers() adds missing MCPAuth.oauth2 case
