litellm/tests/test_litellm
Mateo Wang ffd0e9fa7f
[internal copy of #27491] fix(realtime): Fix Realtime Audio Token Cost Tracking (#29722)
* Normalize Realtime usage dict keys before ResponseAPIUsage transform

* Test usage transform for Realtime versus tokens_details keys

* Avoid usage_input dict in-place

* Fix audio cost calculation

* fix(responses): forward output audio_tokens into completion usage details

Pass audio_tokens from output_tokens_details into CompletionTokensDetailsWrapper
so cost can use output_cost_per_audio_token. Support dict output details like
prompt path. Extend tests for Realtime and mixed completion audio.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix audio token usage formatting

* style: Black-format Realtime usage and completion usage merge

Resolve combine_usage_objects and responses/utils wrapping for CI black --check.
Restore model_fields comments above completion_tokens_details merge loop.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add test to cover combined usage objects

* Fix merge conflict with test cases

Removed unnecessary import statement and cleaned up assertions in test.

* fix(cost_calculator): remove dead None guard in completion_tokens_details combiner

---------

Co-authored-by: Liam McDonald <lmcdonald@godaddy.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-05 18:53:17 +05:30
..
a2a_protocol Fix : a2a bugs 030626 (#29566) 2026-06-03 11:14:15 -07:00
anthropic_interface/exceptions
caching fix(caching): replay openai/responses bridge cache hits as chat streams (#28158) 2026-05-18 16:27:06 -07:00
completion_extras Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
containers fix(containers): record ownership for service-account keys + fix Prisma Json serialization (#28990) 2026-05-27 21:00:07 -07:00
enterprise feat(enterprise): add RESEND_FROM_EMAIL for self-hosted Resend sends (#28830) 2026-05-30 10:09:05 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client
google_genai fix(gemini): normalize response_schema on native generateContent (#27775) 2026-05-12 23:26:34 -07:00
images
integrations [internal copy of #29511] feat(guardrails): add sensitive data routing to on-premise models (#29531) 2026-06-04 22:22:28 -07:00
interactions fix(interactions): never drop streamed text deltas; always emit terminal completion (#28394) 2026-05-20 16:41:40 -07:00
litellm_core_utils Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
llms fix(anthropic): route Claude Opus 4.8 through adaptive thinking (#29702) 2026-06-05 16:19:01 +05:30
ocr chore: reject bare str at file-input sinks to prevent local-file read (#27762) 2026-05-12 16:40:07 -07:00
passthrough
proxy fix(proxy): persist oauth2_flow on MCP server registration (#29690) 2026-06-05 18:52:52 +05:30
responses [internal copy of #27491] fix(realtime): Fix Realtime Audio Token Cost Tracking (#29722) 2026-06-05 18:53:17 +05:30
router_strategy fix(router): enforce deployment budgets for dynamically added models (#29273) 2026-05-29 19:43:14 -07:00
router_utils Gemini managed agents support (#28270) 2026-05-19 16:02:03 -07:00
secret_managers
test_router
types [internal copy of #29511] feat(guardrails): add sensitive data routing to on-premise models (#29531) 2026-06-04 22:22:28 -07:00
vector_stores
__init__.py
conftest.py
log.txt
readme.md
test__types.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py
test_anthropic_skills_transformation.py
test_azure_video_router.py
test_bedrock_anthropic_1hr_cache_pricing.py Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
test_bedrock_usgov_haiku_1hr_cache.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_bedrock_usgov_pricing.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_chat_ui_responses_session.py
test_check_licenses.py fix(check_licenses): read PEP 639 license-expression metadata (#28529) 2026-05-22 11:22:38 -07:00
test_claude_haiku_4_5_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_8_config.py fix(anthropic): route Claude Opus 4.8 through adaptive thinking (#29702) 2026-06-05 16:19:01 +05:30
test_claude_sonnet_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_completion_timeout_resolution.py
test_compression.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py [internal copy of #27491] fix(realtime): Fix Realtime Audio Token Cost Tracking (#29722) 2026-06-05 18:53:17 +05:30
test_count_tokens_public_api.py
test_dashscope_image_generation.py
test_deepseek_model_metadata.py
test_dockerfile_non_root.py
test_eager_tiktoken_load.py
test_exception_exports.py
test_exception_header_preservation.py
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_get_blog_posts.py
test_gpt_image_cost_calculator.py
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_lazy_imports.py
test_litellm_params_reserved_keys.py
test_logging.py
test_lowest_latency_zero_tokens.py
test_main_module_header.py
test_main.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_model_cost_aliases.py
test_model_param_helper.py
test_model_response_normalization.py
test_nested_drop_params.py
test_openai_embedding_encoding_format_default.py
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_redact_string_in_error_paths.py
test_redis.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_register_model_custom_pricing.py Litellm oss staging 2 (#28582) 2026-05-22 10:04:23 -07:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py
test_retrieve_batch_bedrock_dispatch.py feat(batch-job): bedrock batch model invocation job retrieval (#26834) 2026-05-11 13:22:26 -07:00
test_router_google_genai.py
test_router_model_cost_isolation.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_router_order_fallback.py
test_router_per_deployment_num_retries.py
test_router_redis_init.py
test_router_retry_non_retryable_errors.py
test_router_silent_experiment.py
test_router_weighted_failover.py feat: add weighted-routing failover (#27980) 2026-05-15 17:28:54 +00:00
test_router.py Litellm websocket improvements (#29563) 2026-06-03 11:48:35 -07:00
test_secret_redaction.py Litellm oss staging (#29492) 2026-06-02 08:48:10 -07:00
test_service_logger.py fix: missing span for guardrail passthrough (#29552) 2026-06-03 01:25:15 +00:00
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py Litellm OSS Staging (#29161) 2026-06-01 21:22:35 -07:00
test_stream_chunk_builder_annotations.py
test_streaming_connection_cleanup.py
test_system_message_format_bug.py
test_thinking_enabled.py Litellm oss staging 250526 (#28770) 2026-05-26 11:57:39 -07:00
test_utils_module_docstring.py
test_utils.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_uuid_helper.py
test_vcr_safe_body_matcher.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
test_video_generation.py fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (#29098) 2026-05-28 11:45:41 -07:00
test_xai_grok_4_3_model_metadata.py
test_xai_responses_auto_routing.py

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py