litellm/litellm
Alexsander Hamir cdb46f919d
fix: cache SSL contexts to prevent excessive memory allocation (#16955)
Previously, get_ssl_configuration() created a new SSL context on every
call, even when the configuration was identical. This caused continuous
memory allocation from ssl.create_default_context(), especially during:
- Proxy server startup
- Background health checks
- HTTP client creation

Solution:
- Added _ssl_context_cache to cache SSL contexts by configuration
  parameters (cafile, ssl_security_level, ssl_ecdh_curve)
- Refactored SSL context creation into _create_ssl_context() helper
- Modified get_ssl_configuration() to reuse cached contexts when
  configuration matches

This significantly reduces memory allocation while maintaining backward
compatibility. SSL contexts are now reused instead of being recreated
repeatedly, eliminating the memory leak observed in memray profiling.

Fixes memory allocation issue where create_default_context was allocating
6.282MB+ continuously even without any requests.
2025-11-21 19:11:54 -08:00
..
anthropic_interface
assistants
batch_completion
batches [Infra] CI/CD Fixes (#16937) 2025-11-21 13:58:19 -08:00
caching Update model logging format for custom LLM provider (#16485) 2025-11-11 19:13:42 -08:00
completion_extras Fix optional param mapping (#16852) 2025-11-19 19:10:04 -08:00
containers Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
endpoints/speech/speech_to_completion_bridge
experimental_mcp_client [Feat] mcp resources support (#16800) 2025-11-20 14:53:44 -08:00
files feat(managed_files.py): support /delete for files + feat(managed_batches): support /cancel for batches (#16387) 2025-11-18 17:36:26 -08:00
fine_tuning
google_genai
images fix: forward OpenAI organization for image generation (#16607) 2025-11-13 21:51:27 -08:00
integrations fix: arize phoenix logging (#16301) 2025-11-21 18:46:18 -08:00
litellm_core_utils [Fix] Exclude litellm_credential_name from Sensitive Data Masker (Updated) (#16958) 2025-11-21 19:09:48 -08:00
llms fix: cache SSL contexts to prevent excessive memory allocation (#16955) 2025-11-21 19:11:54 -08:00
ocr [Feat] /ocr - Add VertexAI OCR provider support + cost tracking (#16216) 2025-11-03 15:56:49 -08:00
passthrough
proxy [Fix] Exclude litellm_credential_name from Sensitive Data Masker (Updated) (#16958) 2025-11-21 19:09:48 -08:00
realtime_api [Fix] OpenAI Realtime API integration fails due to websockets.exceptions.PayloadTooBig error (#15751) 2025-10-20 15:54:14 -07:00
rerank_api
responses [Infra] CI/CD Fixes (#16937) 2025-11-21 13:58:19 -08:00
router_strategy fix: add None check for litellm_params (#16754) 2025-11-17 19:44:29 -08:00
router_utils fix: cache cooldown key (#16954) 2025-11-21 19:11:17 -08:00
search [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855) 2025-10-23 11:56:30 -07:00
secret_managers [Feat] Adds IAM role assumption support for AWS Secret Manager (#16887) 2025-11-20 12:38:48 -08:00
types [Feat] New LLM Provider - Docker Model Runner (#16948) 2025-11-21 16:09:32 -08:00
vector_store_files Vector store files Stable Release (#16643) 2025-11-15 13:00:33 -08:00
vector_stores Fix vector store create issue (#16804) 2025-11-19 16:53:20 -08:00
videos Add Vertex and Gemini Videos API with Cost Tracking + UI support (#16323) 2025-11-08 16:03:51 -08:00
__init__.py [Feat] New LLM Provider - Docker Model Runner (#16948) 2025-11-21 16:09:32 -08:00
_logging.py
_redis.py fix: Apply max_connections configuration to Redis async client (#15797) 2025-10-22 09:19:08 -07:00
_service_logger.py
_uuid.py
_version.py
budget_manager.py
constants.py [Feat] New LLM Provider - Docker Model Runner (#16948) 2025-11-21 16:09:32 -08:00
cost_calculator.py Fix audio transcription cost tracking (#16478) 2025-11-19 20:29:39 -08:00
cost.json
exceptions.py Adds support for returning Azure Content Policy error information when exceptions from Azure OpenAI occur (#16231) 2025-11-08 16:04:36 -08:00
main.py fix(perf): add missing shared_sessions to audio/transcriptions (#16858) 2025-11-19 19:08:39 -08:00
model_prices_and_context_window_backup.json feat(bedrock): Add Claude 4.5 to US Gov Cloud (#16957) 2025-11-21 19:06:26 -08:00
mypy.ini
py.typed
router.py [Infra] CI/CD Fixes (#16937) 2025-11-21 13:58:19 -08:00
scheduler.py
timeout.py
utils.py [Feat] New LLM Provider - Docker Model Runner (#16948) 2025-11-21 16:09:32 -08:00