ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: increase chunk_size to 8 KB for optimal latency

Browse Source
This commit is contained in:
AlexsanderHamir 2025-11-17 12:55:58 -08:00
parent 697cb09060
commit f1895265e6
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
litellm/proxy/proxy_server.py
View File

@ -5236,7 +5236,7 @@ async def moderations(
async def _audio_speech_chunk_generator(
_response: HttpxBinaryResponseContent,
) -> AsyncGenerator[bytes, None]:
_generator = await _response.aiter_bytes(chunk_size=4096)
_generator = await _response.aiter_bytes(chunk_size=8192)
async for chunk in _generator:
yield chunk