History

wangsoft 42c883d64f fix redis event loop closed at first call (#16913 )		2025-11-21 19:15:56 -08:00
..
__init__.py	Add GCS bucket caching support (#13122 )	2025-08-04 16:09:33 -07:00
_internal_lru_cache.py	(litellm SDK perf improvements) - handle cases when unable to lookup model in model cost map (#7750 )	2025-01-13 19:58:46 -08:00
azure_blob_cache.py	add azure blob cache support (#12587 )	2025-07-15 11:47:38 -07:00
base_cache.py	[Feat] UI - Allow setting cache settings on UI (#16143 )	2025-10-31 17:43:59 -07:00
caching_handler.py	Update model logging format for custom LLM provider (#16485 )	2025-11-11 19:13:42 -08:00
caching.py	init asyc implementation	2025-08-21 10:22:54 +02:00
disk_cache.py	[Bug Fix] No module named 'diskcache' (#11600 )	2025-06-10 14:54:11 -07:00
dual_cache.py	[Perf] Alexsander fixes round 2 - Oct 18th (#15695 )	2025-10-18 11:12:00 -07:00
gcs_cache.py	Add GCS bucket caching support (#13122 )	2025-08-04 16:09:33 -07:00
in_memory_cache.py	[Performance] Reduce complexity of `InMemoryCache.evict_cache` from O(n*log(n)) to O(log(n)) (#15000 )	2025-09-30 16:49:35 -07:00
llm_caching_handler.py	Fix pytest event loop warning (#10512 )	2025-05-02 19:43:18 -07:00
qdrant_semantic_cache.py	fix: use fastuuid helper (#14903 )	2025-09-25 15:47:01 -07:00
Readme.md	add azure blob cache support (#12587 )	2025-07-15 11:47:38 -07:00
redis_cache.py	fix redis event loop closed at first call (#16913 )	2025-11-21 19:15:56 -08:00
redis_cluster_cache.py	fix: Resolve MyPy type checking errors and CI linting (#16277 )	2025-11-05 18:36:06 -08:00
redis_semantic_cache.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00
s3_cache.py	clean imports	2025-08-25 11:09:06 +02:00

Readme.md

Caching on LiteLLM

LiteLLM supports multiple caching mechanisms. This allows users to choose the most suitable caching solution for their use case.

The following caching mechanisms are supported:

RedisCache
RedisSemanticCache
QdrantSemanticCache
InMemoryCache
DiskCache
S3Cache
AzureBlobCache
DualCache (updates both Redis and an in-memory cache simultaneously)

Folder Structure

litellm/caching/
├── base_cache.py
├── caching.py
├── caching_handler.py
├── disk_cache.py
├── dual_cache.py
├── in_memory_cache.py
├── qdrant_semantic_cache.py
├── redis_cache.py
├── redis_semantic_cache.py
├── s3_cache.py

Readme.md

Caching on LiteLLM

Folder Structure

Documentation