From ea0464f41c6d65789d3d784c0d9ae381f22f26aa Mon Sep 17 00:00:00 2001 From: Giulio Leone Date: Sat, 28 Feb 2026 08:58:28 +0100 Subject: [PATCH] fix: exclude gpt-5.2-chat from temperature passthrough (#22342) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add Prometheus child_exit cleanup for gunicorn workers When a gunicorn worker exits (e.g. from max_requests recycling), its per-process prometheus .db files remain on disk. For gauges using livesum/liveall mode, this means the dead worker's last-known values persist as if the process were still alive. Wire gunicorn's child_exit hook to call mark_process_dead() so live-tracking gauges accurately reflect only running workers. * docs: update AssemblyAI docs with Universal-3 Pro, Speech Understanding, and LLM Gateway (#21130) * docs: update AssemblyAI docs with Universal-3 Pro, Speech Understanding, and LLM Gateway provider config * feat: add AssemblyAI LLM Gateway as OpenAI-compatible provider * fix(mcp): update test mocks to use renamed filter_server_ids_by_ip_with_info Tests were mocking the old method name `filter_server_ids_by_ip` but production code at server.py:774 calls `filter_server_ids_by_ip_with_info` which returns a (server_ids, blocked_count) tuple. The unmocked method on AsyncMock returned a coroutine, causing "cannot unpack non-iterable coroutine object" errors. Co-Authored-By: Claude Opus 4.6 * fix(test): update realtime guardrail test assertions for voice violation behavior Tests were asserting no response.create/conversation.item.create sent to backend when guardrail blocks, but the implementation intentionally sends these to have the LLM voice the guardrail violation message to the user. Updated assertions to verify the correct guardrail flow: - response.cancel is sent to stop any in-progress response - conversation.item.create with violation message is injected - response.create is sent to voice the violation - original blocked content is NOT forwarded Co-Authored-By: Claude Opus 4.6 * fix(bedrock): restore parallel_tool_calls mapping in map_openai_params The revert in 8565c70e53 removed the parallel_tool_calls handling from map_openai_params, and the subsequent fix d0445e1e33 only re-added the transform_request consumption but forgot to re-add the map_openai_params producer that sets _parallel_tool_use_config. This meant parallel_tool_calls was silently ignored for all Bedrock models. Co-Authored-By: Claude Opus 4.6 * fix(test): update Azure pass-through test to mock litellm.completion Commit 99c62ca40e removed "azure" from _RESPONSES_API_PROVIDERS, routing Azure models through litellm.completion instead of litellm.responses. The test was not updated to match, causing it to assert against the wrong mock. Co-Authored-By: Claude Opus 4.6 * feat: add in_flight_requests metric to /health/backlog + prometheus (#22319) * feat: add in_flight_requests metric to /health/backlog + prometheus * refactor: clean class with static methods, add tests, fix sentinel pattern * docs: add in_flight_requests to prometheus metrics and latency troubleshooting * fix(db): add missing migration for LiteLLM_ClaudeCodePluginTable PR #22271 added the LiteLLM_ClaudeCodePluginTable model to schema.prisma but did not include a corresponding migration file, causing test_aaaasschema_migration_check to fail. Co-Authored-By: Claude Opus 4.6 * fix: update stale docstring to match guardrail voicing behavior Addresses Greptile review feedback. Co-Authored-By: Claude Opus 4.6 * [Feat] Agent RBAC Permission Fix - Ensure Internal Users cannot create agents (#22329) * fix: enforce RBAC on agent endpoints — block non-admin create/update/delete - Add /v1/agents/{agent_id} to agent_routes so internal users can access GET-by-ID (previously returned 403 due to missing route pattern) - Add _check_agent_management_permission() guard to POST, PUT, PATCH, DELETE agent endpoints — only PROXY_ADMIN may mutate agents - Add user_api_key_dict param to delete_agent so the role check works - Add comprehensive unit tests for RBAC enforcement across all roles Co-authored-by: Ishaan Jaff * fix: mock prisma_client in internal user get-agent-by-id test Co-authored-by: Ishaan Jaff * feat(ui): hide agent create/delete controls for non-admin users Match MCP servers pattern: wrap '+ Add New Agent' button in isAdmin conditional so internal users see a read-only agents view. Delete buttons in card and table were already gated. Update empty-state copy for non-admin users. Add 7 Vitest tests covering role-based visibility. Co-authored-by: Ishaan Jaff --------- Co-authored-by: Cursor Agent Co-authored-by: Ishaan Jaff * fix: exclude gpt-5.2-chat from temperature passthrough (#21911) gpt-5.2-chat and gpt-5.2-chat-latest only support temperature=1 (like base gpt-5), not arbitrary values (like gpt-5.2). Update is_model_gpt_5_1_model() to exclude gpt-5.2-chat variants so drop_params correctly drops unsupported temperature values. Fixes #21911 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Ryan Crabbe Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com> Co-authored-by: Dylan Duan Co-authored-by: Julio Quinteros Pro Co-authored-by: Claude Opus 4.6 Co-authored-by: Ishaan Jaff Co-authored-by: Cursor Agent Co-authored-by: Ishaan Jaff Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../llms/openai/chat/gpt_5_transformation.py | 9 +++-- .../in_flight_requests_middleware.py | 12 +++---- .../llms/openai/test_gpt5_transformation.py | 36 +++++++++++++++++-- 3 files changed, 47 insertions(+), 10 deletions(-) diff --git a/litellm/llms/openai/chat/gpt_5_transformation.py b/litellm/llms/openai/chat/gpt_5_transformation.py index 05c003c8b7..e491770a24 100644 --- a/litellm/llms/openai/chat/gpt_5_transformation.py +++ b/litellm/llms/openai/chat/gpt_5_transformation.py @@ -40,11 +40,16 @@ class OpenAIGPT5Config(OpenAIGPTConfig): gpt-5.1/5.2 support temperature when reasoning_effort="none", unlike base gpt-5 which only supports temperature=1. Excludes - pro variants which keep stricter knobs. + pro variants which keep stricter knobs and gpt-5.2-chat variants + which only support temperature=1. """ model_name = model.split("/")[-1] is_gpt_5_1 = model_name.startswith("gpt-5.1") - is_gpt_5_2 = model_name.startswith("gpt-5.2") and "pro" not in model_name + is_gpt_5_2 = ( + model_name.startswith("gpt-5.2") + and "pro" not in model_name + and not model_name.startswith("gpt-5.2-chat") + ) return is_gpt_5_1 or is_gpt_5_2 @classmethod diff --git a/litellm/proxy/middleware/in_flight_requests_middleware.py b/litellm/proxy/middleware/in_flight_requests_middleware.py index e5e405fb07..d615640d87 100644 --- a/litellm/proxy/middleware/in_flight_requests_middleware.py +++ b/litellm/proxy/middleware/in_flight_requests_middleware.py @@ -6,7 +6,7 @@ Prometheus gauge `litellm_in_flight_requests`. """ import os -from typing import Any, Optional +from typing import Optional from starlette.types import ASGIApp, Receive, Scope, Send @@ -27,7 +27,7 @@ class InFlightRequestsMiddleware: """ _in_flight: int = 0 - _gauge: Optional[Any] = None + _gauge: Optional[object] = None _gauge_init_attempted: bool = False def __init__(self, app: ASGIApp) -> None: @@ -41,13 +41,13 @@ class InFlightRequestsMiddleware: InFlightRequestsMiddleware._in_flight += 1 gauge = InFlightRequestsMiddleware._get_gauge() if gauge is not None: - gauge.inc() # type: ignore[attr-defined] + gauge.inc() # type: ignore[union-attr] try: await self.app(scope, receive, send) finally: InFlightRequestsMiddleware._in_flight -= 1 if gauge is not None: - gauge.dec() # type: ignore[attr-defined] + gauge.dec() # type: ignore[union-attr] @staticmethod def get_count() -> int: @@ -55,14 +55,14 @@ class InFlightRequestsMiddleware: return InFlightRequestsMiddleware._in_flight @staticmethod - def _get_gauge() -> Optional[Any]: + def _get_gauge() -> Optional[object]: if InFlightRequestsMiddleware._gauge_init_attempted: return InFlightRequestsMiddleware._gauge InFlightRequestsMiddleware._gauge_init_attempted = True try: from prometheus_client import Gauge - kwargs: dict[str, Any] = {} + kwargs = {} if "PROMETHEUS_MULTIPROC_DIR" in os.environ: # livesum aggregates across all worker processes in the scrape response kwargs["multiprocess_mode"] = "livesum" diff --git a/tests/test_litellm/llms/openai/test_gpt5_transformation.py b/tests/test_litellm/llms/openai/test_gpt5_transformation.py index 386f264a4d..4ccde67409 100644 --- a/tests/test_litellm/llms/openai/test_gpt5_transformation.py +++ b/tests/test_litellm/llms/openai/test_gpt5_transformation.py @@ -267,7 +267,8 @@ def test_gpt5_1_model_detection(gpt5_config: OpenAIGPT5Config): assert gpt5_config.is_model_gpt_5_1_model("gpt-5.1-chat") assert gpt5_config.is_model_gpt_5_1_model("gpt-5.2") assert gpt5_config.is_model_gpt_5_1_model("gpt-5.2-2025-12-11") - assert gpt5_config.is_model_gpt_5_1_model("gpt-5.2-chat-latest") + assert not gpt5_config.is_model_gpt_5_1_model("gpt-5.2-chat") + assert not gpt5_config.is_model_gpt_5_1_model("gpt-5.2-chat-latest") assert not gpt5_config.is_model_gpt_5_1_model("gpt-5.2-pro") assert not gpt5_config.is_model_gpt_5_1_model("gpt-5") assert not gpt5_config.is_model_gpt_5_1_model("gpt-5-mini") @@ -395,7 +396,38 @@ def test_gpt5_temperature_still_restricted(config: OpenAIConfig): assert params["temperature"] == 1.0 -def test_gpt5_2_pro_allows_reasoning_effort_xhigh(config: OpenAIConfig): +def test_gpt5_2_chat_temperature_restricted(config: OpenAIConfig): + """Test that gpt-5.2-chat only supports temperature=1, like base gpt-5. + + Regression test for https://github.com/BerriAI/litellm/issues/21911 + """ + # gpt-5.2-chat should reject non-1 temperature when drop_params=False + for model in ["gpt-5.2-chat", "gpt-5.2-chat-latest"]: + with pytest.raises(litellm.utils.UnsupportedParamsError): + config.map_openai_params( + non_default_params={"temperature": 0.7}, + optional_params={}, + model=model, + drop_params=False, + ) + + # temperature=1 should still work + params = config.map_openai_params( + non_default_params={"temperature": 1.0}, + optional_params={}, + model=model, + drop_params=False, + ) + assert params["temperature"] == 1.0 + + # drop_params=True should silently drop non-1 temperature + params = config.map_openai_params( + non_default_params={"temperature": 0.5}, + optional_params={}, + model=model, + drop_params=True, + ) + assert "temperature" not in params params = config.map_openai_params( non_default_params={"reasoning_effort": "xhigh"}, optional_params={},