- Fix s3_path configuration in cold storage logging to use actual logger instance path
- Add proper null/empty response validation in session handler to prevent processing invalid responses
- Create comprehensive provider documentation with usage examples
- Cover basic completion, streaming, async, and function calling
- Document AWS Marketplace subscription and API key setup process
- Include proxy configuration and advanced parameter examples
- Add error handling examples and model information
- Update website sidebar to include CompactifAI in provider list
- Update README.md with CompactifAI provider reference
- Add COMPACTIFAI to LlmProviders enum for type safety
- Register CompactifAIChatConfig in ProviderConfigManager
- Import CompactifAIChatConfig in main __init__.py
- Add 'compactifai/' model prefix detection in get_llm_provider()
- Wire CompactifAI completion handler in main.py routing logic
- Support COMPACTIFAI_API_KEY environment variable
- Enable base_llm_http_handler for OpenAI-compatible requests
- Maintain consistency with existing provider integration patterns
- Create CompactifAIChatConfig extending OpenAIGPTConfig for compatibility
- Handle authentication via COMPACTIFAI_API_KEY environment variable
- Set default API base to https://api.compactif.ai/v1
- Support OpenAI-compatible request/response transformation
- Implement JSON mode handling for tool calls
- Add proper model name prefixing with 'compactifai/' provider
- Leverage existing OpenAI infrastructure for minimal code complexity
- Test basic and streaming completions with proper mocking
- Cover authentication, parameter handling, and error scenarios
- Test provider detection and async functionality
- Verify request headers and response transformation
- Follow LiteLLM testing patterns with respx/httpx mocking
- Ensure full compatibility with OpenAI-style responses
* Use _PROXY_MaxParallelRequestsHandler_v3 by default (#14352)
(cherry picked from commit f3fa45cf8fbd5f5cce2f45a7312776d5005fb08e)
(cherry picked from commit 5b680bb4a3)
* Use random api_key for parallel requests test
* Fix off-by-one error in parallel request rate limit
The rate limiter was incorrectly rejecting requests when the limit was met, but not exceeded. The check in `is_cache_list_over_limit` was `int(counter_value) + 1 > current_limit`, which caused the first request to be rejected if the limit was 1.
This commit removes the `+ 1`, changing the logic to `int(counter_value) > current_limit`. The check now correctly allows requests up to the specified parallel limit.
* Test actual parallel requests
* Ensure rate limiting works correctly for multiple users
* Add sequential rate-limit test
* Revert random key usage
