litellm/docs
Kris Xia 56bf6001e9
Supports setting media_resolution and fps parameters on each video file, when using Gemini video understanding. (#19273)
* feat: add gemini video metadata and detail support

Implement support for video_metadata and enhanced detail parameter
for Gemini 3.0+ models:

- Add video_metadata field to ChatCompletionFileObjectFile type
  - Supports fps, start_offset, and end_offset parameters
  - Properly converts snake_case to camelCase for Gemini API
- Extend detail parameter to support medium and ultra_high levels
  - Maps to MEDIA_RESOLUTION_MEDIUM and MEDIA_RESOLUTION_ULTRA_HIGH
- Update _process_gemini_image to handle video metadata transformation
- Add version gating to only apply features for Gemini 3+ models
- Add comprehensive test coverage (6 new test cases)
  - Test detail parameter with file objects
  - Test video_metadata fields (fps, start_offset, end_offset)
  - Test combined detail + video_metadata usage
  - Test new detail levels (medium, ultra_high)
  - Test version gating (Gemini 1.5 vs 3.0)

Note: video_metadata is only supported for video files but error
handling is delegated to Vertex AI for other media types.

* refactor: rename _process_gemini_image to _process_gemini_media

The function handles multiple media types (images, audio, video, PDF),
not just images. Renamed to better reflect its actual purpose.

- Update function name in transformation.py
- Update all function calls and references
- Update test names and imports to match
- Improve docstring to clarify it handles all media types

* docs: add video metadata and media resolution control documentation

Add comprehensive documentation for Gemini 3+ video processing features:
- Document media resolution control (detail parameter) for images and videos
- Add video_metadata field documentation (fps, start_offset, end_offset)
- Include usage examples with tabs for basic, combined, and proxy scenarios
- Update both Gemini and Vertex AI provider documentation
- Clarify snake_case to camelCase field conversion for Gemini API

Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>

* refactor(gemini): extract metadata application into helper function

Extract duplicated Gemini 3+ media_resolution and video_metadata
application logic from _process_gemini_media into a dedicated
_apply_gemini_3_metadata helper function to improve code maintainability.

---------

Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
2026-01-20 11:36:55 -08:00
..
my-website Supports setting media_resolution and fps parameters on each video file, when using Gemini video understanding. (#19273) 2026-01-20 11:36:55 -08:00