litellm/tests
Sameer Kankute 0f9996a4d0
Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
..
audio_tests [Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style (#15810) 2025-10-22 14:41:11 -07:00
basic_proxy_startup_tests
batches_tests
code_coverage_tests [Feat] Add DataforSEO Search API (#15817) 2025-10-22 16:00:40 -07:00
documentation_tests
enterprise Implement Bedrock Guardrail apply_guardrail endpoint support (#15892) 2025-10-24 10:24:03 -07:00
guardrails_tests [Feat] Add Guardrails for /v1/messages and /v1/responses API (#15686) 2025-10-17 18:09:00 -07:00
image_gen_tests [Fix] Dall-e-2 for Image Edits API (#15604) 2025-10-16 13:24:24 -07:00
litellm/llms feat: Add imageConfig parameter for gemini-2.5-flash-image (#15530) 2025-10-21 17:08:30 -07:00
litellm_utils_tests test_models_by_provider 2025-10-23 09:10:41 -07:00
litellm-proxy-extras
llm_responses_api_testing [Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure (#15804) 2025-10-22 14:18:46 -07:00
llm_translation Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
load_tests
local_testing fix: replace deprecated gemini-1.5-pro-preview-0514 with gemini-2.5-flash-lite in function calling test (#15852) 2025-10-23 11:48:58 -07:00
logging_callback_tests Bedrock + MCP - working MCP calls to bedrock via Responses API + Log hidden params for OTEL calls (#15677) 2025-10-18 10:39:28 -07:00
mcp_tests Auth Header Fix for MCP Tool Call (#15736) 2025-10-21 13:58:03 -07:00
multi_instance_e2e_tests
ocr_tests [Feat] Add Cost Tracking for /ocr endpoints (#15678) 2025-10-17 15:54:10 -07:00
old_proxy_tests/tests test fix 2025-09-27 12:40:34 -07:00
openai_endpoints_tests feat: read from custom-llm-provider header (#15528) 2025-10-18 22:04:53 -07:00
otel_tests
pass_through_tests fix: always retain config models 2025-10-11 16:09:33 -07:00
pass_through_unit_tests Fix litellm_param based costing 2025-10-08 21:14:23 +05:30
proxy_admin_ui_tests
proxy_security_tests test fix 2025-10-04 10:57:02 -07:00
proxy_unit_tests [Fix] Forward anthropic-beta headers to Bedrock, VertexAI (#15700) 2025-10-18 16:26:32 -07:00
router_unit_tests [Feat] Add /search endpoint on LiteLLM Gateway (#15780) 2025-10-21 19:05:20 -07:00
scim_tests
search_tests [Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI (#15821) 2025-10-22 17:29:09 -07:00
spend_tracking_tests
store_model_in_db_tests
test_litellm Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
unified_google_tests fix: gooogle GenAI route tests 2025-10-04 10:18:25 -07:00
vector_store_tests (feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM (#15781) 2025-10-22 18:56:36 -07:00
windows_tests
__init__.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD [Feat] MCP Gateway Fine-grained Tools Addition (#15153) 2025-10-03 10:16:29 -07:00
test_budget_management.py
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_end_users.py
test_entrypoint.py
test_fallbacks.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py Add native Responses API support for litellm_proxy provider (#15347) 2025-10-08 18:31:26 -07:00
test_logging.conf
test_models.py test_add_model_run_health 2025-09-27 10:59:25 -07:00
test_openai_endpoints.py
test_organizations.py
test_passthrough_endpoints.py
test_ratelimit.py
test_resource_cleanup.py
test_spend_logs.py
test_team_logging.py
test_team_members.py
test_team.py
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.