0b0a9abd90
This fixes a critical issue where capitalized values in tool_permission guardrail
configurations (e.g., "Deny" instead of "deny") caused Pydantic validation errors
during proxy startup, leading to repeated initialization failures and latency issues.
Problem:
- Users could save guardrails with capitalized values through UI/API
- Data was written to database without validation (e.g., default_action: "Deny")
- On proxy startup, loading from DB triggered strict Pydantic validation
- ValidationError caused guardrail initialization to fail in a retry loop
- This resulted in startup delays and repeated error logging
Root Cause:
- Write path had no case normalization
- Read path enforced strict lowercase Literal validation
- Asymmetry between write and read caused latent data corruption
Solution:
Added field validators to normalize case before Pydantic validation:
1. ToolPermissionRule.decision ("allow"/"deny")
- Normalizes decision field in rules array
2. ToolPermissionGuardrailConfigModel.default_action ("allow"/"deny")
- Normalizes default fallback action
3. ToolPermissionGuardrailConfigModel.on_disallowed_action ("block"/"rewrite")
- Normalizes disallowed tool behavior
4. ToolPermissionGuardrail.__init__ normalization
- Defensive normalization for direct instantiation
- Ensures normalization regardless of code path
Impact:
- Prevents validation errors during guardrail initialization
- Eliminates startup retry loops and latency issues
- Handles existing database records with capitalized values
- Accepts case-insensitive input from all sources (UI, API, direct calls)
- Fully backward compatible with existing lowercase configurations
Testing:
- Added 3 comprehensive tests for case-insensitive handling
- All 27 existing tests still pass
- Tests verify normalization across all affected fields
Files Changed:
- litellm/types/proxy/guardrails/guardrail_hooks/tool_permission.py
Added @field_validator decorators for case normalization
- litellm/proxy/guardrails/guardrail_hooks/tool_permission.py
Added runtime normalization in __init__ method
- tests/test_litellm/proxy/guardrails/guardrail_hooks/test_tool_permission.py
Added case-insensitive validation tests
|
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| code_coverage_tests | ||
| documentation_tests | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| litellm | ||
| litellm_utils_tests | ||
| litellm-proxy-extras | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| README.MD | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_passthrough_endpoints.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_spend_logs.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_team.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.