ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: exclude gpt-5.2-chat from temperature passthrough (#22342)

Browse Source 
* Add Prometheus child_exit cleanup for gunicorn workers

When a gunicorn worker exits (e.g. from max_requests recycling), its
per-process prometheus .db files remain on disk. For gauges using
livesum/liveall mode, this means the dead worker's last-known values
persist as if the process were still alive. Wire gunicorn's child_exit
hook to call mark_process_dead() so live-tracking gauges accurately
reflect only running workers.

* docs: update AssemblyAI docs with Universal-3 Pro, Speech Understanding, and LLM Gateway (#21130)

* docs: update AssemblyAI docs with Universal-3 Pro, Speech Understanding, and LLM Gateway provider config

* feat: add AssemblyAI LLM Gateway as OpenAI-compatible provider

* fix(mcp): update test mocks to use renamed filter_server_ids_by_ip_with_info

Tests were mocking the old method name `filter_server_ids_by_ip` but production
code at server.py:774 calls `filter_server_ids_by_ip_with_info` which returns
a (server_ids, blocked_count) tuple. The unmocked method on AsyncMock returned
a coroutine, causing "cannot unpack non-iterable coroutine object" errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): update realtime guardrail test assertions for voice violation behavior

Tests were asserting no response.create/conversation.item.create sent to
backend when guardrail blocks, but the implementation intentionally sends
these to have the LLM voice the guardrail violation message to the user.

Updated assertions to verify the correct guardrail flow:
- response.cancel is sent to stop any in-progress response
- conversation.item.create with violation message is injected
- response.create is sent to voice the violation
- original blocked content is NOT forwarded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(bedrock): restore parallel_tool_calls mapping in map_openai_params

The revert in 8565c70e53 removed the parallel_tool_calls handling from
map_openai_params, and the subsequent fix d0445e1e33 only re-added the
transform_request consumption but forgot to re-add the map_openai_params
producer that sets _parallel_tool_use_config. This meant parallel_tool_calls
was silently ignored for all Bedrock models.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): update Azure pass-through test to mock litellm.completion

Commit 99c62ca40e removed "azure" from _RESPONSES_API_PROVIDERS,
routing Azure models through litellm.completion instead of
litellm.responses. The test was not updated to match, causing it
to assert against the wrong mock.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add in_flight_requests metric to /health/backlog + prometheus (#22319)

* feat: add in_flight_requests metric to /health/backlog + prometheus

* refactor: clean class with static methods, add tests, fix sentinel pattern

* docs: add in_flight_requests to prometheus metrics and latency troubleshooting

* fix(db): add missing migration for LiteLLM_ClaudeCodePluginTable

PR #22271 added the LiteLLM_ClaudeCodePluginTable model to
schema.prisma but did not include a corresponding migration file,
causing test_aaaasschema_migration_check to fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update stale docstring to match guardrail voicing behavior

Addresses Greptile review feedback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [Feat] Agent RBAC Permission Fix - Ensure Internal Users cannot create agents (#22329)

* fix: enforce RBAC on agent endpoints — block non-admin create/update/delete

- Add /v1/agents/{agent_id} to agent_routes so internal users can
  access GET-by-ID (previously returned 403 due to missing route pattern)
- Add _check_agent_management_permission() guard to POST, PUT, PATCH,
  DELETE agent endpoints — only PROXY_ADMIN may mutate agents
- Add user_api_key_dict param to delete_agent so the role check works
- Add comprehensive unit tests for RBAC enforcement across all roles

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: mock prisma_client in internal user get-agent-by-id test

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat(ui): hide agent create/delete controls for non-admin users

Match MCP servers pattern: wrap '+ Add New Agent' button in
isAdmin conditional so internal users see a read-only agents view.
Delete buttons in card and table were already gated.
Update empty-state copy for non-admin users.
Add 7 Vitest tests covering role-based visibility.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: exclude gpt-5.2-chat from temperature passthrough (#21911)

gpt-5.2-chat and gpt-5.2-chat-latest only support temperature=1
(like base gpt-5), not arbitrary values (like gpt-5.2).
Update is_model_gpt_5_1_model() to exclude gpt-5.2-chat variants
so drop_params correctly drops unsupported temperature values.

Fixes #21911

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu>
Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com>
Co-authored-by: Dylan Duan <dylan.duan@assemblyai.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Giulio Leone 2026-02-28 08:58:28 +01:00 committed by Sameer Kankute
parent 61042f0aec
commit ea0464f41c
3 changed files with 47 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
litellm/llms/openai/chat/gpt_5_transformation.py
View File

@ -40,11 +40,16 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
gpt-5.1/5.2 support temperature when reasoning_effort="none",
unlike base gpt-5 which only supports temperature=1. Excludes
pro variants which keep stricter knobs.
pro variants which keep stricter knobs and gpt-5.2-chat variants
which only support temperature=1.
"""
model_name = model.split("/")[-1]
is_gpt_5_1 = model_name.startswith("gpt-5.1")
is_gpt_5_2 = model_name.startswith("gpt-5.2") and "pro" not in model_name
is_gpt_5_2 = (
model_name.startswith("gpt-5.2")
and "pro" not in model_name
and not model_name.startswith("gpt-5.2-chat")
)
return is_gpt_5_1 or is_gpt_5_2
@classmethod

12
litellm/proxy/middleware/in_flight_requests_middleware.py
View File

@ -6,7 +6,7 @@ Prometheus gauge `litellm_in_flight_requests`.
"""
import os
from typing import Any, Optional
from typing import Optional
from starlette.types import ASGIApp, Receive, Scope, Send
@ -27,7 +27,7 @@ class InFlightRequestsMiddleware:
"""
_in_flight: int = 0
_gauge: Optional[Any] = None
_gauge: Optional[object] = None
_gauge_init_attempted: bool = False
def __init__(self, app: ASGIApp) -> None:
@ -41,13 +41,13 @@ class InFlightRequestsMiddleware:
InFlightRequestsMiddleware._in_flight += 1
gauge = InFlightRequestsMiddleware._get_gauge()
if gauge is not None:
gauge.inc() # type: ignore[attr-defined]
gauge.inc() # type: ignore[union-attr]
try:
await self.app(scope, receive, send)
finally:
InFlightRequestsMiddleware._in_flight -= 1
if gauge is not None:
gauge.dec() # type: ignore[attr-defined]
gauge.dec() # type: ignore[union-attr]
@staticmethod
def get_count() -> int:
@ -55,14 +55,14 @@ class InFlightRequestsMiddleware:
return InFlightRequestsMiddleware._in_flight
@staticmethod
def _get_gauge() -> Optional[Any]:
def _get_gauge() -> Optional[object]:
if InFlightRequestsMiddleware._gauge_init_attempted:
return InFlightRequestsMiddleware._gauge
InFlightRequestsMiddleware._gauge_init_attempted = True
try:
from prometheus_client import Gauge
kwargs: dict[str, Any] = {}
kwargs = {}
if "PROMETHEUS_MULTIPROC_DIR" in os.environ:
# livesum aggregates across all worker processes in the scrape response
kwargs["multiprocess_mode"] = "livesum"

36
tests/test_litellm/llms/openai/test_gpt5_transformation.py
View File

@ -267,7 +267,8 @@ def test_gpt5_1_model_detection(gpt5_config: OpenAIGPT5Config):
assert gpt5_config.is_model_gpt_5_1_model("gpt-5.1-chat")
assert gpt5_config.is_model_gpt_5_1_model("gpt-5.2")
assert gpt5_config.is_model_gpt_5_1_model("gpt-5.2-2025-12-11")
assert gpt5_config.is_model_gpt_5_1_model("gpt-5.2-chat-latest")
assert not gpt5_config.is_model_gpt_5_1_model("gpt-5.2-chat")
assert not gpt5_config.is_model_gpt_5_1_model("gpt-5.2-chat-latest")
assert not gpt5_config.is_model_gpt_5_1_model("gpt-5.2-pro")
assert not gpt5_config.is_model_gpt_5_1_model("gpt-5")
assert not gpt5_config.is_model_gpt_5_1_model("gpt-5-mini")
@ -395,7 +396,38 @@ def test_gpt5_temperature_still_restricted(config: OpenAIConfig):
assert params["temperature"] == 1.0
def test_gpt5_2_pro_allows_reasoning_effort_xhigh(config: OpenAIConfig):
def test_gpt5_2_chat_temperature_restricted(config: OpenAIConfig):
"""Test that gpt-5.2-chat only supports temperature=1, like base gpt-5.
Regression test for https://github.com/BerriAI/litellm/issues/21911
"""
# gpt-5.2-chat should reject non-1 temperature when drop_params=False
for model in ["gpt-5.2-chat", "gpt-5.2-chat-latest"]:
with pytest.raises(litellm.utils.UnsupportedParamsError):
config.map_openai_params(
non_default_params={"temperature": 0.7},
optional_params={},
model=model,
drop_params=False,
)
# temperature=1 should still work
params = config.map_openai_params(
non_default_params={"temperature": 1.0},
optional_params={},
model=model,
drop_params=False,
)
assert params["temperature"] == 1.0
# drop_params=True should silently drop non-1 temperature
params = config.map_openai_params(
non_default_params={"temperature": 0.5},
optional_params={},
model=model,
drop_params=True,
)
assert "temperature" not in params
params = config.map_openai_params(
non_default_params={"reasoning_effort": "xhigh"},
optional_params={},