litellm/tests/test_litellm/caching
michelligabriele 8dc0c97958
fix(caching): check REDIS_CLUSTER_NODES env var in Cache and Router class selection (#22790)
When Redis Cluster is configured via the REDIS_CLUSTER_NODES environment
variable, Cache.__init__() and Router._create_redis_cache() ignored the
env var and always created RedisCache instead of RedisClusterCache. This
caused the v3 rate limiter's cluster detection (_is_redis_cluster()) to
return False, skipping hash-slot key grouping. The resulting CROSSLOT
errors were silently caught, falling back to per-instance in-memory
counting — breaking RPM/TPM enforcement across multiple proxy instances.

Add REDIS_CLUSTER_NODES env var detection to both Cache.__init__() and
Router._create_redis_cache(), matching the existing pattern in
_redis.py:215-220. When the env var is set and no explicit startup_nodes
parameter is provided, parse it and create RedisClusterCache.

Fixes #22748
Related to #20836
2026-03-06 17:31:30 -08:00
..
test_azure_blob_cache.py
test_caching_handler.py
test_dual_cache.py
test_gcs_cache.py
test_in_memory_cache.py
test_llm_caching_handler.py fix: don't close HTTP/SDK clients on LLMClientCache eviction (#22925) 2026-03-05 12:00:38 -08:00
test_llm_client_cache_e2e.py fix: don't close HTTP/SDK clients on LLMClientCache eviction (#22925) 2026-03-05 12:00:38 -08:00
test_qdrant_semantic_cache.py feat(semantic-cache): support configurable vector dimensions for Qdrant (#21649) 2026-02-21 00:51:15 -08:00
test_redis_cache.py fix(proxy): add LPOP pipeline error checking and fix org spend ServiceType 2026-02-24 14:22:57 -08:00
test_redis_cluster_cache.py fix(caching): check REDIS_CLUSTER_NODES env var in Cache and Router class selection (#22790) 2026-03-06 17:31:30 -08:00
test_redis_connection_pool.py Revert "Add LLMClientCache regression tests for httpx client eviction safety" 2026-02-27 12:43:03 -08:00
test_redis_semantic_cache.py
test_s3_cache.py