litellm/tests
Krish Dholakia 5ac61a7572
Add bedrock latency optimized inference support (#9623)
* fix(converse_transformation.py): add performanceConfig param support on bedrock

Closes https://github.com/BerriAI/litellm/issues/7606

* fix(converse_transformation.py): refactor to use more flexible single getter for params which are separate config blocks

* test(test_main.py): add e2e mock test for bedrock performance config

* build(model_prices_and_context_window.json): add versioned multimodal embedding

* refactor(multimodal_embeddings/): migrate to config pattern

* feat(vertex_ai/multimodalembeddings): calculate usage for multimodal embedding calls

Enables cost calculation for multimodal embeddings

* feat(vertex_ai/multimodalembeddings): get usage object for embedding calls

ensures accurate cost tracking for vertexai multimodal embedding calls

* fix(embedding_handler.py): remove unused imports

* fix: fix linting errors

* fix: handle response api usage calculation

* test(test_vertex_ai_multimodal_embedding_transformation.py): update tests

* test: mark flaky test

* feat(vertex_ai/multimodal_embeddings/transformation.py): support text+image+video input

* docs(vertex.md): document sending text + image to vertex multimodal embeddings

* test: remove incorrect file

* fix(multimodal_embeddings/transformation.py): fix linting error

* style: remove unused import
2025-03-29 00:23:09 -07:00
..
basic_proxy_startup_tests
batches_tests cleanup_azure_files 2025-02-15 15:32:42 -08:00
code_coverage_tests Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state 2025-03-28 13:11:19 -07:00
documentation_tests
image_gen_tests Nova Canvas complete image generation tasks (#9177) (#9525) 2025-03-26 11:28:20 -07:00
litellm Add bedrock latency optimized inference support (#9623) 2025-03-29 00:23:09 -07:00
litellm_utils_tests Support discovering gemini, anthropic, xai models by calling their /v1/model endpoint (#9530) 2025-03-27 22:50:48 -07:00
llm_responses_api_testing test_openai_o1_pro_response_api_streaming 2025-03-20 13:04:49 -07:00
llm_translation Add bedrock latency optimized inference support (#9623) 2025-03-29 00:23:09 -07:00
load_tests fix vertex embedding perf test 2025-03-26 10:25:50 -07:00
local_testing Add bedrock latency optimized inference support (#9623) 2025-03-29 00:23:09 -07:00
logging_callback_tests Merge pull request #9508 from BerriAI/litellm_fix_gcs_pub_sub 2025-03-24 18:22:43 -07:00
mcp_tests ci_cd_server_path 2025-03-21 19:06:29 -07:00
multi_instance_e2e_tests
old_proxy_tests/tests
openai_endpoints_tests test_bad_request_bad_param_error 2025-03-13 16:02:21 -07:00
otel_tests test local spend accuracy 2025-03-28 19:52:39 -07:00
pass_through_tests add litellm mcp endpoints 2025-03-20 21:12:56 -07:00
pass_through_unit_tests test: update tests 2025-03-22 12:56:42 -07:00
proxy_admin_ui_tests test: update tests 2025-03-22 12:56:42 -07:00
proxy_security_tests
proxy_unit_tests Merge branch 'main' into litellm_use_redis_for_updates 2025-03-28 20:12:29 -07:00
router_unit_tests test: update tests 2025-03-22 12:56:42 -07:00
store_model_in_db_tests test_chat_completion_bad_model_with_spend_logs 2025-02-28 20:19:43 -08:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD add bedrock llama vision support + cohere / infinity rerank - 'return_documents' support (#8684) 2025-02-20 21:23:54 -08:00
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_end_users.py test: run test earlier to catch error 2025-03-27 23:08:52 -07:00
test_entrypoint.py
test_fallbacks.py test: fix test 2025-03-10 22:00:50 -07:00
test_health.py
test_keys.py
test_logging.conf
test_models.py
test_openai_endpoints.py test string checked for model access control 2025-03-10 20:04:18 -07:00
test_organizations.py Add remaining org CRUD endpoints + support deleting orgs on UI (#8561) 2025-02-15 15:48:06 -08:00
test_passthrough_endpoints.py
test_ratelimit.py
test_spend_logs.py
test_team_logging.py
test_team_members.py
test_team.py fix(team_endpoints.py): ensure 404 raised when team not found (#9038) 2025-03-06 22:04:36 -08:00
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/litellm

This folder can only run mock tests.