* fix(gemini): prevent negative text_tokens with explicit caching (#18750)
## Problem
When using Gemini with explicit caching (especially with images),
text_tokens would become negative (e.g., -3327) due to incorrectly
subtracting total cached_tokens from modality-specific text_tokens.
## Root Cause
The old code did:
```python
text_tokens = text_tokens - cached_tokens # 737 - 4064 = -3327
```
This was wrong because:
- cached_tokens includes ALL modalities (text + image + audio + video)
- text_tokens only contains text
- Subtracting total from specific caused negative values
## Solution
Parse cacheTokensDetails to get per-modality cached token breakdown:
```python
if "cacheTokensDetails" in usage_metadata:
cached_text_tokens = parse from cacheTokensDetails["TEXT"]
text_tokens = text_tokens - cached_text_tokens # Correct!
```
Now we subtract cached tokens per modality, preventing negatives.
## Changes
- Parse cacheTokensDetails field from Gemini response
- Calculate non-cached tokens per modality (text, image, audio)
- Remove incorrect global cached_tokens subtraction
- Add tests for explicit caching and implicit/no caching scenarios
## Testing
- Added test_gemini_cache_tokens_details_no_negative_values
- Added test_gemini_without_cache_tokens_details
- All existing Gemini caching tests pass
Fixes#18750
* feat: add cache_read_input_tokens to Usage object
Addresses reviewer feedback to include cached tokens at the top level
of the Usage object. This aligns with how Anthropic provider handles
cached tokens and ensures they are visible in the final usage response.
* fix: add cacheTokensDetails field to UsageMetadata TypedDict
Fixes mypy error where cacheTokensDetails was being accessed but not defined
in the UsageMetadata TypedDict type definition.
* fix(anthropic): prevent dropping thinking when any message has thinking_blocks
When Claude returns multiple assistant messages in a conversation, some may
have thinking_blocks while others may not (Claude's behavior varies). The
previous logic only checked the LAST assistant message with tool_calls,
dropping the thinking param if it had no thinking_blocks.
This caused errors when earlier messages still contained thinking_blocks:
"When thinking is disabled, an assistant message cannot contain thinking"
The fix adds a new check: only drop thinking if NO assistant messages
have thinking_blocks. If any message has thinking_blocks, we keep
thinking enabled.
Fixes#18926
* chore: re-trigger CI
* fix(generic-guardrail-api): fix SerializationIterator error on multimodal requests
When sending multimodal messages (with images) through the Generic Guardrail API,
the `model_dump()` call fails with "Object of type SerializationIterator is not
JSON serializable" error.
Root cause: The `ChatCompletionAssistantMessage` type defines `content` as an
`Iterable` (not just `List`), and Pydantic's `model_dump()` creates a
`SerializationIterator` for iterables which is not JSON serializable.
Fix: Use `model_dump(mode="json")` which properly converts all iterables to
lists and ensures all complex objects are JSON serializable.
* fix(guardrails): pass tools (function definitions) to guardrail inputs
The unified guardrail handler was not passing the `tools` parameter
(function definitions) from the request to the guardrail inputs.
This meant guardrails could not inspect or validate tool definitions.
Added extraction of `data.get("tools")` and inclusion in the
GenericGuardrailAPIInputs passed to `apply_guardrail()`.
* test(guardrails): add tests for tools passed to guardrail
Added tests verifying that tools (function definitions) are correctly
passed to guardrails in the unified guardrail handler:
- test_tools_passed_to_guardrail
- test_multiple_tools_passed_to_guardrail
- test_no_tools_in_request
- test_tools_and_tool_calls_both_passed
The Pydantic model was missing these timestamp fields that exist in the
Prisma schema. This caused the UI to display "Unknown date" for models
added via the dashboard, even though the data was being stored in the DB.
The fields were being read in get_model_info_with_id() using getattr(),
but since they weren't defined in the Pydantic model, getattr() always
returned None.
Fixes#15487
This fixes a breaking change where batch completion requests with comma-separated
model strings (e.g., 'gpt-3.5-turbo,fake-openai-endpoint') were failing validation.
The validate_model_access function now:
- Detects comma-separated model strings
- Validates each model individually
- Provides clear error messages for inaccessible models in batch requests
- Maintains backward compatibility for single model validation
Fixes test_batch_chat_completions test failure.
