* fixed issues with gcs cache to verify functionality
* restore changes
* Fix capitalization of 'S3 Bucket Cache'
---------
Co-authored-by: Nelson Alfonso <45660392+Dashing-Nelson@users.noreply.github.com>
* init guardrails
* init guardrails
* some fixes
* some fixes
* ruff
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* docs
Add support for Z.AI GLM-4.7, latest flagship model with enhanced reasoning capabilities.
Changes:
- Add zai/glm-4.7 to model pricing with /bin/bash.60/M input, .20/M output
- Add cached input pricing (/bin/bash.11/M) for GLM-4.7
- Add supports_reasoning flag to enable thinking parameter
- Update ZAIChatConfig to support thinking parameter for models with reasoning
- Update documentation with GLM-4.7 as latest flagship model
- Add cached input column to pricing table (GLM-4.7 only)
- Add tests for GLM-4.7 reasoning support and cost calculation
- Update all examples to use GLM-4.7
Model specifications:
- Context: 200K input, 128K output
- Supports: reasoning, function calling, tool choice, prompt caching
- Pricing: Same as GLM-4.6 with cache support
See: https://docs.z.ai/guides/llm/glm-4.7
Adds log_format parameter supporting json_array (default), ndjson, and single formats. NDJSON format enables webhook integrations like Sumo Logic to parse individual log records at ingest time. Defaults to json_array for backward compatibility.
* Allow get_nested_value dot notation to support escaping for Kubernetes JWT Support
* Add support for team and org alias fields, add docs, tests
* Fix lint issue with max statements in handle jwt logic
Got an error message:
{"error":{"message":"Invalid JSON payload: trailing comma is not allowed: line 8 column 8 (char 141)","type":"invalid_request_error","param":"request_body","code":"400"}}%
Remove 'none' from gpt-5-mini's supported reasoning_effort values in the documentation table. gpt-5-mini does not support reasoning_effort="none", only minimal, low, medium, and high.
- Add OAuth M2M (Machine-to-Machine) authentication via DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET
- Add Databricks SDK auto-auth with automatic credential discovery
- Add sensitive data redaction for secure logging (tokens, API keys, secrets)
- Add custom user_agent parameter for partner attribution in Databricks telemetry
- Support user_agent in LiteLLM Proxy via config.yaml litellm_params
- Add 49 mocked unit tests for all new functionality
- Add 13 E2E tests for real-world validation (skipped in CI)
- Update documentation with new features and examples
Added concise PostgreSQL and Redis specifications based on benchmark results and industry standards for API gateway deployments. Includes tiered recommendations for different RPS workloads, configuration best practices, and scaling guidelines.
* Add monitor mode support to Lakera guardrail
- Add on_flagged parameter to LakeraV2GuardrailConfigModel (default: 'block')
- Support 'monitor' mode that logs violations without blocking requests
- Support 'block' mode (default) that raises HTTPException on violations
- Update async_pre_call_hook and async_moderation_hook to check on_flagged
- Update guardrail initializer to pass on_flagged from config
- Add documentation with monitor mode examples
This allows users to tune Lakera security policies by monitoring violations
without blocking legitimate requests, similar to Pillar's on_flagged_action.
* Add tests for Lakera guardrail monitor mode
- Test monitor mode allows flagged content through (pre_call hook)
- Test block mode raises HTTPException for violations (pre_call hook)
- Test monitor mode works with during_call (moderation_hook)
These tests verify the on_flagged parameter functionality for both
monitor and block modes across different guardrail hooks.
---------
Co-authored-by: Steve <steve.giguere@lakera.ai>
* feat(litellm_content_filter.py): add support for content filtering categories
make it easy for proxy admin to prevent messages about violence, self harm or illegal weapons going through litellm
* feat: initial commit adding bias detection
allows admin to block inappropriate content about sexual orientation, etc.
* refactor: simplify content_filter.py
use a more exhaustive set of keywords, instead of guessing at potential phrases user can use
* feat(content_filter.py): add new denied topics for in-built content filter guardrails
allow user to automatically block content relating to certain categories from being sent to the LLML
* refactor(content-filter): document new params to litellm content filter
* feat(ui/): litellm content filter - select content categories on ui
* docs: update documentation
* docs(litellm_content_filter.md): document new content filters
* feat: initial commit adding support for inappropriate images via litellm content filter
* feat(content_filter.py): support blocking images containing blocked content
prevent images which contain disallowed content from being sent to the llm api
* docs(litellm_content_filter.md): document new image capabilities of litellm_content_filter
* fix: fix expected error code
* feat(litellm_content_filter.py): add support for content filtering categories
make it easy for proxy admin to prevent messages about violence, self harm or illegal weapons going through litellm
* feat: initial commit adding bias detection
allows admin to block inappropriate content about sexual orientation, etc.
* refactor: simplify content_filter.py
use a more exhaustive set of keywords, instead of guessing at potential phrases user can use
* feat(content_filter.py): add new denied topics for in-built content filter guardrails
allow user to automatically block content relating to certain categories from being sent to the LLML
* refactor(content-filter): document new params to litellm content filter
* feat(ui/): litellm content filter - select content categories on ui
* docs: update documentation
* docs(litellm_content_filter.md): document new content filters
