litellm

History

Yuneng Jiang 3d67f00ede test(interactions): align openapi compliance with upstream rename outputs->steps Google updated their Interactions OpenAPI spec (https://ai.google.dev/static/api/interactions.openapi.json), removing the readOnly 'outputs' property from CreateModelInteractionParams in favor of 'steps' (a polymorphic transcript array). The compliance test fetches the live spec, so it began failing on every PR once the spec flipped over. Update the asserted output-field list to match. Note: this only re-aligns the spec-shape assertion. Our SDK response types (litellm/types/interactions/generated.py) still expose 'outputs' and need to be regenerated separately to add 'steps'/Step variants and decide on a back-compat path for callers reading .outputs.		2026-05-07 17:09:25 -07:00
..
agent_tests
audio_tests	[Test] Tests: Stop parametrizing API keys into pytest test IDs (#27249 )	2026-05-05 17:21:18 -07:00
basic_proxy_startup_tests
batches_tests	[Fix] Batches Tests: Remove VCR Auto-Marker	2026-05-07 15:39:46 -07:00
benchmarks
code_coverage_tests	feat(audio_transcription): add NVIDIA Riva STT provider (#27185 )	2026-05-05 17:17:51 -07:00
documentation_tests
enterprise	Fix Prometheus custom metadata label counts (#27268 ) (#27271 )	2026-05-05 20:04:56 -07:00
guardrails_tests	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
image_gen_tests	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
litellm	Add new chat model metadata (#27313 )	2026-05-06 15:15:21 -07:00
litellm_core_utils
litellm_utils_tests	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
litellm-proxy-extras
llm_responses_api_testing	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
llm_translation	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/sweet-mcclintock-2b3656	2026-05-07 13:46:53 -07:00
load_tests
local_testing	[Test] Tests: Stop parametrizing API keys into pytest test IDs (#27249 )	2026-05-05 17:21:18 -07:00
logging_callback_tests	Merge pull request #27222 from BerriAI/litellm_s3AuditParams	2026-05-07 12:49:03 -07:00
mcp_tests
multi_instance_e2e_tests
ocr_tests	[Fix] Tests: Reduce VCR cassette bloat and fix multipart caching	2026-05-07 11:54:19 -07:00
old_proxy_tests/tests
openai_endpoints_tests
otel_tests	[Fix] Tests: Pick chat-completion OTEL trace by content, not recency	2026-05-04 20:35:09 -07:00
pass_through_tests	chore(deps): refresh dependency locks	2026-05-04 11:36:18 -07:00
pass_through_unit_tests	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
proxy_admin_ui_tests	chore(deps): refresh dependency locks	2026-05-04 11:36:18 -07:00
proxy_e2e_anthropic_messages_tests
proxy_security_tests
proxy_unit_tests	Fix early proxy request size enforcement (#27311 )	2026-05-06 12:29:11 -07:00
router_unit_tests	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
scim_tests
search_tests	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
spend_tracking_tests
store_model_in_db_tests
test_litellm	test(interactions): align openapi compliance with upstream rename outputs->steps	2026-05-07 17:09:25 -07:00
unified_google_tests	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
vector_store_tests
windows_tests
__init__.py
_flush_vcr_cache.py	tests(vcr): isolate cassette redis to CASSETTE_REDIS_URL	2026-05-01 12:32:59 -07:00
_vcr_conftest_common.py	[Fix] Tests: Reduce VCR cassette bloat and fix multipart caching	2026-05-07 11:54:19 -07:00
_vcr_redis_persister.py	test: add 24hr Redis-backed VCR cache to additional test suites (#27159 )	2026-05-05 15:13:31 -07:00
eval_swe_bench.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py
test_openai_endpoints.py
test_organizations.py
test_otel_thread_leak.py
test_passthrough_endpoints.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py
test_team_logging.py
test_team_members.py
test_team.py
test_users.py

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.