Sameer Kankute 4dc645fc33 feat(polling): check rate limits before creating polling ID ... Move pre-call checks (rate limits, guardrails, budget) to run BEFORE polling ID creation in the background streaming flow. This prevents the edge case where a rate-limited request receives a polling ID that immediately fails. Changes: - Add skip_pre_call_logic parameter to base_process_llm_request to allow skipping pre-call checks (avoiding double-counting of RPM/parallel requests) - Run common_processing_pre_call_logic before generating polling ID in the responses API endpoint. If rate limits/guardrails fail, return error immediately without creating a polling ID - Background streaming task passes skip_pre_call_logic=True to avoid re-running pre-call checks that were already done before polling ID creation - Add tests verifying skip_pre_call_logic parameter works correctly Fixes the edge case where polling_via_cache would return a polling ID for a request that immediately fails due to rate limiting.		2026-03-19 13:59:59 +05:30
..
__init__.py	refactor: extract should_use_polling_for_request to polling_handler module	2025-12-05 09:02:15 -08:00
background_streaming.py	feat(polling): check rate limits before creating polling ID	2026-03-19 13:59:59 +05:30
polling_handler.py	style: run black formatter on entire codebase	2026-03-11 17:07:57 -03:00