litellm/tests/test_litellm/integrations
Yassin Kortam 10bd7406e0
feat: add guardrail violation span attributes and fix missing spans on pre-call blocks (#28364)
- Fix missing guardrail child spans when a pre-call guardrail blocks the request before reaching the LLM provider; `async_post_call_failure_hook` now calls `_emit_guardrail_spans_from_request_data` to emit spans from `request_data["metadata"]` regardless of whether `_handle_failure` already fired
- Add `guardrail_status`, `guardrail_action`, and `guardrail_violation_categories` as queryable top-level OTEL span attributes so trace backends can filter/group by violation type without parsing the redacted `guardrail_response` blob
- Introduce `_emit_guardrail_spans_from_request_data` helper that constructs minimal kwargs from `request_data["metadata"]` and routes through `_create_guardrail_span`, sharing the same dedupe state to prevent double-emitting when both failure hooks fire
- Extend `BedrockGuardrail` with `_build_tracing_detail` and `_extract_violation_category_names` which flatten BLOCKED assessments into human-readable category labels (topic names, content-filter types, PII entity types, named regex names) before redaction, and surface Bedrock's raw `action` field via `tracing_detail`
- Security: violation category extraction deliberately omits `customWords.match` and unnamed regex `match` values because those fields carry the user-submitted content that triggered the rule; only operator-defined `name`/`type` labels are emitted
- Add `violation_categories` and `guardrail_action` fields to `StandardLoggingGuardrailInformation` and `GuardrailTracingDetail` TypedDicts to carry the pre-redaction metadata through the logging pipeline
- Add comprehensive test suite covering: guardrail span creation on failure, dedupe between `_handle_failure` and `async_post_call_failure_hook`, per-span status attributes for multi-guardrail sequences, Bedrock category extraction for all policy types, security leak prevention, and end-to-end `CustomGuardrail` violation path

Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
2026-05-21 15:49:42 -07:00
..
arize Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_oss_staging_04_25_2026 2026-05-02 06:04:39 +00:00
azure_storage
bitbucket fix: block path traversal SSRF in BitBucket, Arize Phoenix, and AssemblyAI clients (#26943) 2026-05-01 11:45:12 -07:00
cloudzero
compression_interception
datadog
dotprompt
focus
gcs_bucket fix cloud storage file guards 2026-05-01 15:34:11 -07:00
gcs_pubsub
gitlab
langfuse
levo
litellm_agent
open_telemetry/data
SlackAlerting
websearch_interception Emit native web_search_tool_result blocks for Anthropic clients (Claude Desktop / Cowork citations) (#27886) 2026-05-14 12:30:47 -07:00
rubrik_test_helpers.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_agentops.py
test_anthropic_cache_control_hook.py
test_athina.py
test_azure_sentinel.py Add Azure Sentinel audit log support (#27280) 2026-05-06 15:50:06 -07:00
test_braintrust_logging.py
test_braintrust_span_name.py
test_custom_guardrail_recursion.py
test_custom_guardrail.py Fix 3 OpenTelemetry tracing bugs in proxy integration (#27757) 2026-05-12 18:32:05 -07:00
test_custom_prompt_management.py
test_deepeval.py
test_langfuse_otel.py
test_langfuse.py
test_langsmith_init.py Remove unneeded metadata info from LangSmith 2026-05-01 15:42:12 -07:00
test_mlflow.py
test_openmeter.py
test_opentelemetry_dynamic_imports.py
test_opentelemetry.py fix: serialize guardrail_response to JSON in OTEL traces (#28362) 2026-05-20 17:34:36 -07:00
test_otel_guardrail_violation_spans.py feat: add guardrail violation span attributes and fix missing spans on pre-call blocks (#28364) 2026-05-21 15:49:42 -07:00
test_otel_team_attributes_matrix.py feat: propagate team_id and team_alias to all child OTEL spans (#28273) 2026-05-19 15:31:25 -07:00
test_prometheus_api_promql_escape.py fix(prometheus): quote api_key for PromQL string literal in spend lookup 2026-05-01 21:38:19 +00:00
test_prometheus_cache_metrics.py
test_prometheus_client_ip_user_agent.py
test_prometheus_custom_metadata_label_counts.py Fix Prometheus remaining metric zero values (#27348) 2026-05-06 17:22:20 -07:00
test_prometheus_end_user_cardinality.py perf: cap Prometheus end-user metric cardinality with TTL + LRU eviction (#27272) 2026-05-06 13:35:13 -07:00
test_prometheus_invalid_key_filtering.py
test_prometheus_labels.py
test_prometheus_metric_name_consistency.py
test_prometheus_missing_metrics.py
test_prometheus_none_metadata.py
test_prometheus_queue_guardrail_metrics.py
test_prometheus_remaining_tokens_router_fallback.py fix(prometheus): emit litellm_remaining_tokens_metric for Bedrock and Vertex (#27705) 2026-05-13 17:40:59 -07:00
test_prometheus_services.py
test_prometheus_spend_logs_metadata.py
test_prometheus_stream_label.py
test_prometheus_user_team_metrics.py feat(prometheus): add user_email and user_alias to user budget metrics (#28155) 2026-05-18 16:28:14 -07:00
test_prompt_manager_ssti.py fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers 2026-05-02 09:14:02 +00:00
test_responses_background_cost.py
test_rubrik.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_s3_v2.py decouple S3 audit-log config via s3_audit_callback_params 2026-05-05 13:23:32 -07:00
test_weave_otel.py