litellm/litellm
michelligabriele c47f77a348
fix(agentcore): handle JSON responses from agents using sync return (#23165)
* fix(agentcore): handle JSON responses from agents using sync return

BedrockAgentCoreApp agents that use synchronous `return` (instead of
async `yield`) respond with Content-Type: application/json instead of
text/event-stream. The streaming parser only handles SSE format, silently
discarding the JSON body and returning empty content to the client.

This adds Content-Type detection in both sync and async streaming
wrappers — when application/json is received, the response is parsed
and converted to a single-chunk stream. Also extends _parse_json_response
with a fallback chain supporting multiple agent response schemas (standard
AgentCore, Strands framework, plain string, raw JSON fallback).

* fix(agentcore): add dict-type guard to _parse_json_response

Prevent AttributeError when json.loads() returns a non-dict
(e.g. JSON array or primitive) by adding an isinstance check
at the top of _parse_json_response. Non-dict values fall back
to raw JSON string content.

* fix(agentcore): handle malformed JSON and split streaming chunks

- Wrap json.loads() in try/except in both sync and async streaming
  wrappers so malformed JSON bodies raise a structured BedrockError
  instead of a raw JSONDecodeError
- Split the JSON-fallback streaming path into two chunks (content
  chunk with finish_reason=None, then stop sentinel with empty delta)
  to match the SSE path convention

* fix(agentcore): catch IO errors in streaming JSON path + async error test

- Broaden except clause to catch both json.JSONDecodeError and IO-level
  exceptions (httpx.ReadError, etc.) from response.read()/aread(), so
  all failures surface as structured BedrockError
- Add async malformed-JSON test to mirror the sync test coverage
2026-03-09 10:22:36 -07:00
..
a2a_protocol CircleCI test stability (#23055) 2026-03-07 15:19:39 -08:00
anthropic_interface
assistants
batch_completion
batches Agents - add max budget + tpm/rpm limiting per agent AND per agent session (#22849) 2026-03-07 19:12:42 -08:00
caching fix(caching): check REDIS_CLUSTER_NODES env var in Cache and Router class selection (#22790) 2026-03-06 17:31:30 -08:00
completion_extras
containers
endpoints/speech/speech_to_completion_bridge
evals
experimental_mcp_client
files Merge pull request #22765 from BerriAI/main 2026-03-04 17:40:42 +05:30
fine_tuning Agents - add max budget + tpm/rpm limiting per agent AND per agent session (#22849) 2026-03-07 19:12:42 -08:00
google_genai
images
integrations feat: allow multiple calls from tags 2026-03-07 11:24:18 +05:30
interactions
litellm_core_utils Fix test_anthropic_messages_openai_model_streaming_cost_injection & openrouter image gen 2026-03-09 11:29:04 +05:30
llms fix(agentcore): handle JSON responses from agents using sync return (#23165) 2026-03-09 10:22:36 -07:00
ocr
passthrough fix(passthrough): propagate Azure 429/5xx errors in async streaming instead of silent HTTP 200 (#22913) 2026-03-05 10:12:43 -08:00
proxy fix fkey violation on deleting user (#23115) 2026-03-09 08:53:11 -07:00
proxy_auth
rag
realtime_api
rerank_api
responses fix(lint): resolve PLR0915 too-many-statements in 4 files 2026-03-04 20:34:15 -03:00
router_strategy fix: complexity_router crashes on list-format message content (OpenAI multi-part messages) (#22761) 2026-03-04 16:18:49 -08:00
router_utils
search
secret_managers feat: Hashicorp Vault config override backend endpoints 2026-03-05 16:57:08 -08:00
skills
types feat(proxy): add Prisma DB pool and engine health metrics to Prometheus (#22655) 2026-03-09 08:49:46 -07:00
vector_store_files
vector_stores
videos
__init__.py feat: feature flag on validate key alias 2026-03-07 12:32:21 +05:30
_lazy_imports_registry.py Merge pull request #22866 from mubashir1osmani/feat/bedrock-mantle-provider-clean 2026-03-05 18:24:00 +05:30
_lazy_imports.py
_logging.py
_redis.py
_service_logger.py
_uuid.py
_version.py
anthropic_beta_headers_config.json
anthropic_beta_headers_manager.py
blog_posts.json
budget_manager.py
constants.py feat(spend-logs): add truncation note when error logs are truncated for DB storage (#22936) 2026-03-05 16:58:46 -08:00
cost_calculator.py feat(azure_ai): add router flat cost when response contains actual model 2026-03-06 18:18:06 +05:30
cost.json
exceptions.py
main.py Merge pull request #22866 from mubashir1osmani/feat/bedrock-mantle-provider-clean 2026-03-05 18:24:00 +05:30
model_prices_and_context_window_backup.json feat(search): add Serper (serper.dev) as search provider (#23112) 2026-03-09 08:40:37 -07:00
mypy.ini
policy_templates_backup.json
provider_endpoints_support_backup.json feat(search): add Serper (serper.dev) as search provider (#23112) 2026-03-09 08:40:37 -07:00
py.typed
router.py Agents - add max budget + tpm/rpm limiting per agent AND per agent session (#22849) 2026-03-07 19:12:42 -08:00
scheduler.py
timeout.py
utils.py feat(search): add Serper (serper.dev) as search provider (#23112) 2026-03-09 08:40:37 -07:00