litellm/tests/test_litellm/proxy/db
Yuneng Jiang fbcdacc446
[Fix] Proxy: reconnect Prisma DB without blocking the event loop
When the DB becomes unreachable the reconnect path calls
`prisma.disconnect()`, which ultimately invokes prisma-client-py's
synchronous `subprocess.Popen.wait()` on the query engine subprocess.
That call does not yield to asyncio, so the event loop freezes for
however long the Rust engine takes to shut down (30-120+ seconds in
production when the engine is stuck on TCP close). During the freeze
`/health/liveliness` becomes unresponsive, and in Kubernetes the
liveness probe fails and the pod is SIGKILL'd.

Replace `disconnect()` in the reconnect paths with a direct, non-blocking
kill of the engine subprocess (SIGTERM -> 0.5s asyncio-yielding sleep ->
SIGKILL) followed by a fresh Prisma client and a new `connect()`. Both
`recreate_prisma_client` and the formerly-separate "direct reconnect"
path go through the same kill-then-recreate flow.

Also validate `_get_engine_pid` returns an int (defensive; prevents a
MagicMock leak under unit-test mocking).

Tests that encoded the old blocking behavior are updated or removed;
the deleted `test_lightweight_reconnect_skips_kill_on_successful_disconnect`
invariant ("don't kill on successful disconnect") was part of the bug.
2026-04-21 23:29:17 -07:00
..
db_transaction_queue style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
mcp_server build(VLLM-Passthrough-with-loadbalancing-support-(enables-using-model-list-for-VLLM-/classify-endpoint)): Closes #11205 2025-05-31 09:00:04 -07:00
test_check_migration.py build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
test_create_views.py [Fix] should_create_missing_views returns False for reltuples=0 (falsy zero bug) 2026-04-18 11:00:09 -07:00
test_db_spend_update_writer.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_handler.py fix(proxy): restore broad is_database_connection_error; add is_database_transport_error for reconnect (#21796) 2026-02-21 12:04:00 -08:00
test_prisma_client.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prisma_self_heal.py [Fix] Proxy: reconnect Prisma DB without blocking the event loop 2026-04-21 23:29:17 -07:00
test_rds_iam_token_expiry.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_tool_registry_writer.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00