Commit Graph

30881 Commits

Author SHA1 Message Date
=
12fed102ba Update bedrock_guardrails.py 2025-09-18 17:25:31 -07:00
Ishaan Jaffer
0626affa11 fix: model cost map check 2025-09-18 17:25:28 -07:00
Krish Dholakia
c7af9d1b7e
Merge pull request #14696 from ARajan1084/bedrock-guardrail-view
NEW Amazon Bedrock Guardrail Info View in Logs
2025-09-18 17:22:17 -07:00
Alexsander Hamir
59409429d4
fix: reduced __inits__ overhead in 7% (#14689)
* fix: avoid redundant __init__ calls on hot path

Previously, imports on the request hot path caused __init__ to run
excessively for every request. This change ensures initialization
happens once, reducing cpu overhead.

* fix: remove redundant __init__ import

The current implementation no longer requires an import at the top of the function.

* fix: placed on core utils for future reuse

* test: add coverage & remove inline import

A general import-checking tool across all endpoints would be a large PR.
This commit focuses on a smaller, targeted fix for the discussed case.

* added import check to CI
2025-09-18 17:18:05 -07:00
Ishaan Jaff
4c983f985a
[Feat] Add Bedrock Twelve Labs embedding provider support (#14697)
* fix: add 12 labs to bedrock embedding

* fix: get_bedrock_embedding_provider

* test: test_text_embedding

* fix: 12 labs embedding transform

* fix: refactor 12 labs transform logic

* fix: test_e2e_bedrock_embedding

* fix: test_e2e_bedrock_embedding

* feat: add bedrock twelvelabs pricing

* DOCS: docs bedrock embedding

* DOCS: 12 labs bedrock overview

* fix: bedrock embeddings 12 labs
2025-09-18 17:16:45 -07:00
=
77e4008e7b lint fix 2025-09-18 17:13:55 -07:00
=
f80660acd9 vitests for GuardrailViewer components 2025-09-18 16:57:22 -07:00
=
2bbcf5a851 Update bedrock_guardrails.py 2025-09-18 16:05:30 -07:00
katsuhiro muto
ec61a7152a
Support for is_streamed_request widh datadog (#14673) 2025-09-18 15:55:16 -07:00
=
a3f0a3c05f Update test_bedrock_guardrails.py 2025-09-18 15:51:42 -07:00
=
911918474a removed duplicate code 2025-09-18 15:49:16 -07:00
Yuta Saito
7f8b1d0708 test: fix failing tests after conflict resolution 2025-09-19 07:48:37 +09:00
=
917c8bb43c Update test_bedrock_guardrails.py 2025-09-18 15:42:41 -07:00
=
a86b9a1808 check for AWS exceptions despite a 200 response 2025-09-18 15:42:36 -07:00
Mubashir Osmani
a7a6381926
fix: flaky passthrough tests (#14692)
* fix: flaky passthrough tests

* Revert "fix: flaky passthrough tests"

This reverts commit ffe692e017600a8853ab7c31f95485958ab74c5f.

* fix: serialize prisma objects
2025-09-18 15:35:14 -07:00
=
78cb8c71fd prelim changes for new Bedrock guardrail info view 2025-09-18 15:09:10 -07:00
Yuta Saito
c10d78aede fix: hide spec_version on MCP server views 2025-09-19 07:06:15 +09:00
Yuta Saito
654f1d3290 fix: stop including spec_version in MCP server registration inserts 2025-09-19 07:06:15 +09:00
Yuta Saito
6c291093e9 fix: remove adding Mcp-Protocol-Version header (#14069)
The Mcp-Protocol-Version header is already handled in the MCP Python SDK, so the explicit addition on LiteLLM Proxy was redundant.
2025-09-19 07:05:20 +09:00
Fabrício Ceschin
1ba40f0dab
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-18 17:03:00 -04:00
Ishaan Jaff
c6e37bb0bb
fix: BadRequestError cohere (#14691) 2025-09-18 12:32:54 -07:00
Fabrício Ceschin
56f7111a57
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-18 15:21:11 -04:00
=
c1963f7f02 removed wrapper which overwrote the guardrailResponse value incorrectly 2025-09-18 12:05:38 -07:00
Tim Elfrink
978cd80653 Update Bedrock documentation for Titan V2 encoding_format support
- Add encoding_format parameter to supported parameters table
- Document float and base64 encoding format options
- Add usage examples for both encoding formats
- Update parameter documentation for amazon.titan-embed-text-v2:0
2025-09-18 20:24:56 +02:00
Tim Elfrink
b100328435 Add test coverage for Bedrock Titan V2 encoding_format parameter
- Test encoding_format='float' parameter mapping and response handling
- Test encoding_format='base64' parameter mapping to binary format
- Verify parameter transformation and response processing
- Mock AWS API responses for both float and binary formats
- Ensure OpenAI compatibility with new encoding_format support
2025-09-18 20:24:41 +02:00
Tim Elfrink
72b492c761 Fix Bedrock Titan V2 encoding_format parameter support
- Add encoding_format to supported OpenAI parameters list
- Implement encoding_format to embeddingTypes parameter mapping
- Map 'float' to ['float'] and 'base64' to ['binary'] formats
- Handle response with proper fallback: binary > float > embedding field
- Support both float and binary response formats per AWS documentation

Fixes #14685 - UnsupportedParamsError when using encoding_format with Titan V2
2025-09-18 20:23:45 +02:00
Ishaan Jaffer
e733b619db fix: test_user_email_in_all_required_metrics 2025-09-18 11:23:13 -07:00
Sameer Kankute
36bedc69ff
Add TwelveLabs marengo model (#14674) 2025-09-18 11:21:35 -07:00
Tim Elfrink
0e096d7883 Update Bedrock Titan V2 type definitions for encoding format support
- Add embeddingTypes parameter to AmazonTitanV2EmbeddingRequest
- Add embeddingsByType response field for binary format support
- Update type hints for enhanced embedding response handling
2025-09-18 20:17:23 +02:00
Sameer Kankute
d213a2e066
correct the gaurdcontent name (#14684)
* correct the gaurdcontent name

* correct the gaurdcontent name

* fix model required error in test

* Add correct model
2025-09-18 11:00:19 -07:00
Ishaan Jaffer
e0bbc3f46b fix: test_cohere_request_body_with_allowed_params 2025-09-18 10:56:45 -07:00
Ishaan Jaffer
8296bfb866 fix: test metrics 2025-09-18 10:25:22 -07:00
Ishaan Jaffer
c6afa904bb fix: test_completion_with_no_model 2025-09-18 10:17:09 -07:00
Ishaan Jaffer
1e1d174733 fix: test_completion_with_no_model 2025-09-18 10:13:32 -07:00
Mritunjay Kumar Sharma
8b0e3c14ff
chore(docker): remove redundant Admin UI build step
The Admin UI is already built before packaging, so the second invocation of docker/build_admin_ui.sh after PyJWT adjustments was unnecessary. Removing it speeds up the builder stage, reduces cache invalidation, and doesn’t change the resulting wheel or runtime image.
2025-09-18 11:09:20 -05:00
Ishaan Jaffer
706b9214c0 fix: test_init_kwargs_for_pass_through_endpoint_basic 2025-09-18 07:59:05 -07:00
Ishaan Jaffer
3018c66867 fix: test_awesome_otel_with_message_logging_off 2025-09-18 07:49:39 -07:00
Ishaan Jaffer
09d63a00e2 fix: cohere generate api is deprecated 2025-09-18 07:40:28 -07:00
Ishaan Jaffer
da95e208da fix: BedrockCountTokensHandler 2025-09-18 07:32:33 -07:00
Ishaan Jaffer
7b5dd5c7d1 fix: cohere generate api is deprecated 2025-09-18 07:29:11 -07:00
Ishaan Jaff
759bb6cff5
fix: cohere generate api is deprecated (#14676) 2025-09-18 07:27:51 -07:00
Tim Elfrink
c9daf3bc8e update tests 2025-09-18 15:52:59 +02:00
Fabrício Ceschin
cb3bed1d96
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-18 09:38:52 -04:00
Adam Tao
9b0eaf0989 test: add a testcase for cached_tokens
Signed-off-by: Adam Tao <tcx4c70@gmail.com>
2025-09-18 18:40:11 +08:00
Adam Tao
f11083e992 fix: Fix cost calculation for responses
Signed-off-by: Adam Tao <tcx4c70@gmail.com>
2025-09-18 18:31:41 +08:00
Tim Elfrink
cb573f61c8 Document model_id parameter for Bedrock inference profiles with image generation
Add documentation for using AWS Bedrock Application Inference Profiles
with image generation APIs, specifically for Nova Canvas models.

The documentation includes:
- SDK examples showing model_id parameter usage
- Proxy configuration examples
- Follows the same pattern as chat completions inference profiles
2025-09-18 10:57:12 +02:00
Tim Elfrink
c30c775e63 Fix Nova Canvas model_id parameter handling for consistency
Follow the same pattern as chat completions and embeddings by extracting
model_id from optional_params into a variable, even though it's not used
in image generation. This maintains code consistency across Bedrock services.
2025-09-18 10:32:18 +02:00
Michael Tan
6721189b7c
feat: add Asia/Bangkok timezone support for budget resets
- Add Asia/Bangkok (UTC+7) to timezone_map in duration_parser.py
- Update documentation to include Bangkok in common timezone values
- Add test case to verify Bangkok timezone functionality
2025-09-18 14:56:12 +07:00
Tim Elfrink
27a37ce57e Use requestMetadata parameter name for consistency with other Bedrock params
- Change parameter from request_metadata to requestMetadata to match camelCase convention
- Consistent with guardrailConfig and performanceConfig naming pattern
- Update all references in transformation code and error messages
- Update tests and documentation to use correct parameter name
- Fix type checking for parameter validation
2025-09-18 09:21:45 +02:00
Tim Elfrink
352a0eedb5 refactor: Extract helper methods to fix linting issues
- Split _transform_request_helper into smaller focused methods
- Extract _prepare_request_params for parameter preparation logic
- Extract _process_tools_and_beta for tool processing logic
- Resolve PLR0915 (too many statements) linting issue
- Maintain all existing functionality and test compatibility
2025-09-18 09:21:45 +02:00