ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
87f94172a9
litellm/tests/test_litellm
History
Cesar Garcia 87f94172a9
fix(responses): Add image generation support for Responses API (#16586) 
* fix(responses): Add image generation support for Responses API

Fixes #16227

## Problem
When using Gemini 2.5 Flash Image with /responses endpoint, image generation
outputs were not being returned correctly. The response contained only text
with empty content instead of the generated images.

## Solution
1. Created new `OutputImageGenerationCall` type for image generation outputs
2. Modified `_extract_message_output_items()` to detect images in completion responses
3. Added `_extract_image_generation_output_items()` to transform images from
   completion format (data URL) to responses format (pure base64)
4. Added `_extract_base64_from_data_url()` helper to extract base64 from data URLs
5. Updated `ResponsesAPIResponse.output` type to include `OutputImageGenerationCall`

## Changes
- litellm/types/responses/main.py: Added OutputImageGenerationCall type
- litellm/types/llms/openai.py: Updated ResponsesAPIResponse.output type
- litellm/responses/litellm_completion_transformation/transformation.py:
  Added image detection and extraction logic
- tests/test_litellm/responses/litellm_completion_transformation/test_image_generation_output.py:
  Added comprehensive unit tests (16 tests, all passing)

## Result
/responses endpoint now correctly returns:
```json
{
  "output": [{
    "type": "image_generation_call",
    "id": "..._img_0",
    "status": "completed",
    "result": "iVBORw0KGgo..."  // Pure base64, no data: prefix
  }]
}
```

This matches OpenAI Responses API specification where image generation
outputs have type "image_generation_call" with base64 data in "result" field.

* docs(responses): Add image generation documentation and tests

- Add comprehensive image generation documentation to response_api.md
  - Include examples for Gemini (no tools param) and OpenAI (with tools param)
  - Document response format and base64 handling
  - Add supported models table with provider-specific requirements

- Add unit tests for image generation output transformation
  - Test base64 extraction from data URLs
  - Test image generation output item creation
  - Test status mapping and integration scenarios
  - Verify proper transformation from completions to responses format

Related to #16227

* fix(responses): Correct status type for image generation output

- Add _map_finish_reason_to_image_generation_status() helper function
- Fix MyPy type error: OutputImageGenerationCall.status only accepts
  ['in_progress', 'completed', 'incomplete', 'failed'], not the full
  ResponsesAPIStatus union which includes 'cancelled' and 'queued'

Fixes MyPy error in transformation.py:838
2025-12-05 15:56:26 -08:00
..
caching fix(redis): handle float redis_version from AWS ElastiCache Valkey (#16207) 2025-11-04 19:20:00 -08:00
completion_extras Merge pull request #16844 from Chesars/fix/response-format-to-text-format-bridge-conversion 2025-11-26 08:51:09 +05:30
containers Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
enterprise [Bug Fix] - Fix litellm_enterprise ensure imported routes exist (#17337) 2025-12-01 19:14:12 -08:00
experimental_mcp_client refactor: drop MCPClient.connect and use run_with_session lifecycle (#16696) 2025-11-15 17:54:27 -08:00
google_genai [Oct Staging Branch] (#15460) 2025-10-17 17:52:25 -07:00
integrations Fix: MLflow streaming spans for Anthropic passthrough (#17288) 2025-12-05 14:59:36 -08:00
litellm_core_utils Fix: Gemini image_tokens incorrectly treated as text tokens in cost calculation (#17554) 2025-12-05 15:55:38 -08:00
llms Fixes #13652 - auth not working with ollama.com (#17191) 2025-12-05 15:52:54 -08:00
passthrough fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
proxy fix(presidio): handle empty content and error dict responses (#17489) 2025-12-05 15:45:19 -08:00
responses fix(responses): Add image generation support for Responses API (#16586) 2025-12-05 15:56:26 -08:00
router_strategy
router_utils 🆕 feat: support routing to only websearch supported deployments (#17500) 2025-12-04 14:18:20 -08:00
secret_managers [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297) 2025-11-05 17:20:26 -08:00
types Gemini models: capture image_tokens and support cost_per_output_image_token in costs calculations (#16912) 2025-11-21 19:59:24 -08:00
vector_stores Add vector store support for ragflow 2025-12-03 15:29:47 +05:30
__init__.py
conftest.py Perf speed up pytest (#15951) 2025-10-27 19:43:40 -07:00
log.txt
readme.md
test_acompletion_session_reuse_e2e.py [Fix] - Sessions not being shared (#15388) 2025-10-09 16:57:40 -07:00
test_add_deployment_no_master_key.py fix: remove strict master_key check in add_deployment (#16453) 2025-11-10 19:23:13 -08:00
test_aembedding_session_reuse_e2e.py [Fix] - shared session parsing and usage issue (#15440) 2025-10-10 19:26:53 -07:00
test_azure_video_router.py Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
test_claude_haiku_4_5_config.py fix haiku-4-5 bedrock configs (#16732) 2025-11-17 19:52:01 -08:00
test_constants.py tes numeric constants 2025-10-29 18:23:38 -07:00
test_container_router.py Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
test_cost_calculation_log_level.py
test_cost_calculator.py Fix audio transcription cost tracking (#16478) 2025-11-19 20:29:39 -08:00
test_exception_mapping_request_attribute.py Added test 2025-09-29 21:43:31 +05:30
test_filter_out_litellm_params.py [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855) 2025-10-23 11:56:30 -07:00
test_groq_streaming_encoding.py
test_logging.py
test_lowest_latency_zero_tokens.py fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 2025-09-29 15:49:46 -07:00
test_main.py fix(ocr): add missing ocr and aocr to CallTypes enum (#17435) 2025-12-03 21:28:13 -08:00
test_redis.py fix: Apply max_connections configuration to Redis async client (#15797) 2025-10-22 09:19:08 -07:00
test_responses_id_security.py test_encrypt_response_id_success 2025-11-14 17:28:15 -08:00
test_router_google_genai.py test(gemini): Add unit tests for Google GenAI adapter 2025-09-29 18:51:35 +08:00
test_router.py Add aws_bedrock_runtime_endpoint into Credential Types (#17053) 2025-11-24 20:48:51 -08:00
test_shared_session_integration.py
test_system_message_format_bug.py
test_utils.py Gemini models: capture image_tokens and support cost_per_output_image_token in costs calculations (#16912) 2025-11-21 19:59:24 -08:00
test_uuid_helper.py
test_video_generation.py fix(gemini-video): inherit BaseVideoConfig to enable async content response (#16875) 2025-11-21 16:01:21 -08:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py