litellm/tests
Krish Dholakia ba2d4d080f
feat(handle_jwt.py): map user to team when added via jwt auth (#11108)
* feat(handle_jwt.py): map user to team when added via jwt auth

makes it easy to ensure user belongs to team

* test: test_openai_image_edit_litellm_sdk

* use n 4 for mapped tests (#11109)

* Fix/background health check (#10887)

* fix: improve health check logic by deep copying model list on each iteration

* test: add async test for background health check reflecting model list changes

* fix: validate health check interval before executing background health check

* fix: specify type for health check results dictionary

* fix(user_api_key_auth.py): handle user custom auth set with no custom settings

* bump: version 0.1.21 → 0.2.0

* ci(config.yml): run enterprise and litellm tests separately

* fix: fix linting error

* docs: add missing docs

* [Feat] Add content policy violation error mapping for image editd (#11113)

* feat: add image edit mapping for content policy violations

* test fix

* Expose `/list` and `/info` endpoints for Audit Log events (#11102)

* feat(audit_logging_endpoints.py): expose list endpoint to show all audit logs

make it easier for user to retrieve individual endpoints

* feat(enterprise/): add audit logging endpoint

* feat(audit_logging_endpoints.py): expose new GET `/audit/{id}` endpoint

make it easier to retrieve view individual audit logs

* feat(key_management_event_hooks.py): correctly show the key of the user who initiated the change

* fix(key_management_event_hooks.py): add key rotations as an audit log event

'

* test(test_audit_logging_endpoints.py): add simple unit testing for audit log endpoint

* fix: testing fixes

* fix: fix ruff check

* [Feat] Use aiohttp transport by default - 97% lower median latency  (#11097)

* fix: add flag for disabling use_aiohttp_transport

* feat: add _create_async_transport

* feat: fixes for transport

* add httpx-aiohttp

* feat: fixes for transport

* refactor: fixes for transport

* build: fix deps

* fixes: test fixes

* fix: ensure aiohttp does not auto set content type

* test: test fixes

* feat: add LiteLLMAiohttpTransport

* fix: fixes for responses API handling

* test: fixes for responses API handling

* test: fixes for responses API handling

* feat: fixes for transport

* fix: base embedding handler

* test: test_async_http_handler_force_ipv4

* test: fix failing deepeval test

* fix: add YARL for bedrock urls

* fix: issues with transport

* fix: comment out linting issues

* test fix

* test: XAI is unstable

* test: fixes for using respx

* test: XAI fixes

* test: XAI fixes

* test: infinity testing fixes

* docs(config_settings.md): document param

* test: test_openai_image_edit_litellm_sdk

* test: remove deprecated test

* bump respx==0.22.0

* test: test_xai_message_name_filtering

* test: fix anthropic test after bumping httpx

* use n 4 for mapped tests (#11109)

* fix: use 1 session per event loop

* test: test_client_session_helper

* fix: linting error

* fix: resolving GET requests on httpx 0.28.1

* test fixes proxy unit tests

* fix: add ssl verify settings

* fix: proxy unit tests

* fix: refactor

* tests: basic unit tests for aiohttp transports

* tests: fixes xai

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>

* test: cleanup redundant test

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: JuHyun Bae <jhyun0408@nate.com>
2025-05-23 23:23:46 -07:00
..
basic_proxy_startup_tests
batches_tests Litellm managed file updates combined (#11040) 2025-05-22 17:20:41 -07:00
code_coverage_tests Gemini Multimodal Live API support (#10841) 2025-05-15 22:39:09 -07:00
documentation_tests
enterprise Expose /list and /info endpoints for Audit Log events (#11102) 2025-05-23 22:54:59 -07:00
guardrails_tests [Fix] List Guardrails - Show config.yaml guardrails on litellm ui (#10959) 2025-05-19 21:25:23 -07:00
image_gen_tests [Feat] Add content policy violation error mapping for image editd (#11113) 2025-05-23 21:57:43 -07:00
litellm feat(handle_jwt.py): map user to team when added via jwt auth (#11108) 2025-05-23 23:23:46 -07:00
litellm_utils_tests [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
litellm-proxy-extras Prisma Migrate - support setting custom migration dir (#10336) 2025-04-26 12:05:06 -07:00
llm_responses_api_testing [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
llm_translation [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
load_tests test: test_embedding_performance 2025-05-14 21:31:07 -07:00
local_testing [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
logging_callback_tests [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
mcp_tests test fixes 2025-03-29 18:34:58 -07:00
multi_instance_e2e_tests
old_proxy_tests/tests test: update tests to new deployment model (#10142) 2025-04-18 14:22:12 -07:00
openai_endpoints_tests # expect an error when getting the response again since 2025-04-25 09:42:35 -07:00
otel_tests [Feat] Prometheus - Track route on proxy_* metrics (#10992) 2025-05-20 22:55:55 -07:00
pass_through_tests test: add more debug logs 2025-04-29 15:40:18 -07:00
pass_through_unit_tests [Feat] - Add Support for Showing Passthrough endpoint Error Logs on LiteLLM UI (#10990) 2025-05-20 18:29:39 -07:00
proxy_admin_ui_tests fix(model_management_endpoints.py): allow team admin to update model … (#10539) 2025-05-03 19:34:35 -07:00
proxy_security_tests
proxy_unit_tests feat(handle_jwt.py): map user to team when added via jwt auth (#11108) 2025-05-23 23:23:46 -07:00
router_unit_tests complete unified batch id support - replace model in jsonl to be deployment model name (#10719) 2025-05-10 12:04:01 -07:00
scim_tests [Feat SSO] Add LiteLLM SCIM Integration for Team and User management (#10072) 2025-04-16 19:21:47 -07:00
spend_tracking_tests test_long_term_spend_accuracy_with_bursts 2025-03-31 21:09:29 -07:00
store_model_in_db_tests fix(router.py): handle edge case where user sets 'model_group' inside… (#10191) 2025-04-21 16:17:45 -07:00
windows_tests [Bug Fix] UnicodeDecodeError: 'charmap' on Windows during litellm import (#10542) 2025-05-03 21:31:05 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_end_users.py test: run test earlier to catch error 2025-03-27 23:08:52 -07:00
test_entrypoint.py
test_fallbacks.py
test_health.py
test_keys.py test: temporarily skip test due to change testing model change - need to update test for new model 2025-05-09 09:02:08 -07:00
test_logging.conf
test_models.py Litellm UI improvements 04 26 2025 p1 (#10346) 2025-04-26 18:24:49 -07:00
test_openai_endpoints.py
test_organizations.py UI - fix adding vertex models with reusable credentials + fix pagination on keys table + fix showing org budgets on table (#10528) 2025-05-03 08:16:53 -07:00
test_passthrough_endpoints.py
test_ratelimit.py
test_spend_logs.py
test_team_logging.py
test_team_members.py
test_team.py fix: test_team_update_sc_2 2025-05-09 11:59:13 -07:00
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/litellm

This folder can only run mock tests.