litellm/tests
Giulio Leone ea0464f41c fix: exclude gpt-5.2-chat from temperature passthrough (#22342)
* Add Prometheus child_exit cleanup for gunicorn workers

When a gunicorn worker exits (e.g. from max_requests recycling), its
per-process prometheus .db files remain on disk. For gauges using
livesum/liveall mode, this means the dead worker's last-known values
persist as if the process were still alive. Wire gunicorn's child_exit
hook to call mark_process_dead() so live-tracking gauges accurately
reflect only running workers.

* docs: update AssemblyAI docs with Universal-3 Pro, Speech Understanding, and LLM Gateway (#21130)

* docs: update AssemblyAI docs with Universal-3 Pro, Speech Understanding, and LLM Gateway provider config

* feat: add AssemblyAI LLM Gateway as OpenAI-compatible provider

* fix(mcp): update test mocks to use renamed filter_server_ids_by_ip_with_info

Tests were mocking the old method name `filter_server_ids_by_ip` but production
code at server.py:774 calls `filter_server_ids_by_ip_with_info` which returns
a (server_ids, blocked_count) tuple. The unmocked method on AsyncMock returned
a coroutine, causing "cannot unpack non-iterable coroutine object" errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): update realtime guardrail test assertions for voice violation behavior

Tests were asserting no response.create/conversation.item.create sent to
backend when guardrail blocks, but the implementation intentionally sends
these to have the LLM voice the guardrail violation message to the user.

Updated assertions to verify the correct guardrail flow:
- response.cancel is sent to stop any in-progress response
- conversation.item.create with violation message is injected
- response.create is sent to voice the violation
- original blocked content is NOT forwarded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(bedrock): restore parallel_tool_calls mapping in map_openai_params

The revert in 8565c70e53 removed the parallel_tool_calls handling from
map_openai_params, and the subsequent fix d0445e1e33 only re-added the
transform_request consumption but forgot to re-add the map_openai_params
producer that sets _parallel_tool_use_config. This meant parallel_tool_calls
was silently ignored for all Bedrock models.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): update Azure pass-through test to mock litellm.completion

Commit 99c62ca40e removed "azure" from _RESPONSES_API_PROVIDERS,
routing Azure models through litellm.completion instead of
litellm.responses. The test was not updated to match, causing it
to assert against the wrong mock.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add in_flight_requests metric to /health/backlog + prometheus (#22319)

* feat: add in_flight_requests metric to /health/backlog + prometheus

* refactor: clean class with static methods, add tests, fix sentinel pattern

* docs: add in_flight_requests to prometheus metrics and latency troubleshooting

* fix(db): add missing migration for LiteLLM_ClaudeCodePluginTable

PR #22271 added the LiteLLM_ClaudeCodePluginTable model to
schema.prisma but did not include a corresponding migration file,
causing test_aaaasschema_migration_check to fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update stale docstring to match guardrail voicing behavior

Addresses Greptile review feedback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [Feat] Agent RBAC Permission Fix - Ensure Internal Users cannot create agents (#22329)

* fix: enforce RBAC on agent endpoints — block non-admin create/update/delete

- Add /v1/agents/{agent_id} to agent_routes so internal users can
  access GET-by-ID (previously returned 403 due to missing route pattern)
- Add _check_agent_management_permission() guard to POST, PUT, PATCH,
  DELETE agent endpoints — only PROXY_ADMIN may mutate agents
- Add user_api_key_dict param to delete_agent so the role check works
- Add comprehensive unit tests for RBAC enforcement across all roles

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: mock prisma_client in internal user get-agent-by-id test

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat(ui): hide agent create/delete controls for non-admin users

Match MCP servers pattern: wrap '+ Add New Agent' button in
isAdmin conditional so internal users see a read-only agents view.
Delete buttons in card and table were already gated.
Update empty-state copy for non-admin users.
Add 7 Vitest tests covering role-based visibility.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: exclude gpt-5.2-chat from temperature passthrough (#21911)

gpt-5.2-chat and gpt-5.2-chat-latest only support temperature=1
(like base gpt-5), not arbitrary values (like gpt-5.2).
Update is_model_gpt_5_1_model() to exclude gpt-5.2-chat variants
so drop_params correctly drops unsupported temperature values.

Fixes #21911

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu>
Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com>
Co-authored-by: Dylan Duan <dylan.duan@assemblyai.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-02 19:21:21 +05:30
..
agent_tests [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
audio_tests
basic_proxy_startup_tests
batches_tests fix(tests): fix flaky test_create_vertex_fine_tune_jobs_mocked - handle background Datadog flush (#21838) 2026-02-21 14:44:01 -08:00
code_coverage_tests Revert "[Feature] Add /public/supported_endpoints endpoint" 2026-02-26 17:21:43 -08:00
documentation_tests fix failing tests 2026-02-21 15:48:26 -08:00
enterprise remove orphan comment from test file 2026-02-26 16:51:18 -08:00
guardrails_tests fix: bump litellm-proxy-extras to 0.4.50 and fix 3 failing tests (#22417) 2026-02-28 10:20:03 -08:00
image_gen_tests [Release - 02/10/2026] v1.81.10-nightly 2026-02-10 16:26:30 -08:00
litellm fix: escalate to heavy Prisma reconnect after consecutive lightweight failures (#22211) 2026-03-02 19:21:11 +05:30
litellm_core_utils
litellm_utils_tests Fix test_perform_health_check_filters_by_model_id 2026-02-26 10:43:05 +05:30
litellm-proxy-extras
llm_responses_api_testing Fix anthropic responses 2026-02-20 17:30:42 -08:00
llm_translation fix(types): normalize null top_logprobs to [] in ChatCompletionTokenLogprob (#22245) 2026-02-28 15:59:29 -08:00
load_tests
local_testing [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
logging_callback_tests Merge pull request #22180 from BerriAI/litellm_fix_vllm_test 2026-02-26 18:43:52 +05:30
mcp_tests Litellm stability fix v2 (#22452) 2026-02-28 15:29:45 -08:00
multi_instance_e2e_tests
ocr_tests
old_proxy_tests/tests
openai_endpoints_tests
otel_tests
pass_through_tests Litellm stability fix v2 (#22452) 2026-02-28 15:29:45 -08:00
pass_through_unit_tests [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
proxy_admin_ui_tests security: fix critical/high CVEs in OS-level libs and NPM transitive 2026-02-24 19:40:09 +05:30
proxy_e2e_anthropic_messages_tests Fix: litellm/tests/llm_responses_api_testing/test_anthropic_responses_api.py 2026-02-20 17:30:53 -08:00
proxy_security_tests
proxy_unit_tests [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
router_unit_tests [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
scim_tests
search_tests [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
spend_tracking_tests
store_model_in_db_tests
test_litellm fix: exclude gpt-5.2-chat from temperature passthrough (#22342) 2026-03-02 19:21:21 +05:30
unified_google_tests
vector_store_tests Add support for vertex ai for rag/ingest 2026-02-13 20:11:10 +05:30
windows_tests
__init__.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py
test_openai_endpoints.py
test_organizations.py
test_otel_thread_leak.py
test_passthrough_endpoints.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py
test_team_logging.py
test_team_members.py
test_team.py fix(test): skip 'projects' field in team update assertion (#21777) 2026-02-21 10:24:53 -08:00
test_users.py fix(tests): update deprecated Anthropic model in test_user_model_access (#21826) 2026-02-21 14:18:24 -08:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.