litellm/tests/test_litellm/proxy
Cole McIntosh 5f34ceea1a
Implement health check backend API and storage functionality (#11678)
* feat: Add health check functionality and endpoints

- Introduced methods for saving health check results to the database, including validation and cleaning of data.
- Added new health check endpoints to retrieve health check history and latest health statuses for models.
- Updated model prices and context window configuration for new Azure transcription models.

* test: Add unit tests for health check functionality

- Introduced tests for PrismaClient health check methods, including saving results and retrieving health check history.
- Added tests for the _save_health_check_to_db function to ensure proper handling of healthy and unhealthy endpoints.
- Implemented mock objects to simulate database interactions and validate method behaviors.

* Refactor health endpoint model ID handling and improve logging

- Updated health endpoint to use `get_deployment` for retrieving model names based on model IDs, enhancing error handling for missing models.
- Changed health check result saving to the database to be non-blocking by using `asyncio.create_task`.
- Cleaned up code for better readability and maintainability.

* Refactor utility functions in proxy module for improved readability and error handling

- Removed unused imports and simplified exception handling in `_get_redoc_url` and `_get_docs_url` functions to manage circular imports.
- Cleaned up logging statements for consistency and clarity.
- Streamlined error message formatting in `handle_exception_on_proxy` function.

* Enhance type hinting and default values in ProxyUpdateSpend class for improved clarity and robustness

- Added type hints for `_end_user_list_transactions` to specify it as a dictionary mapping end user IDs to spend amounts.
- Updated default values for optional fields in `SpendLogsPayload` to ensure they are initialized properly, enhancing error handling.
- Refactored `_premium_user_check` function to improve model validation logic and error handling.

* Fix disable_spend_updates method to handle None return value gracefully

- Updated the disable_spend_updates method to return False if the environment variable DISABLE_SPEND_UPDATES is not set or is None, improving robustness in configuration handling.

* Refactor join_paths function in utils.py for improved path handling

- Enhanced the join_paths function to better manage leading and trailing slashes, ensuring correct path concatenation.
- Added logic to handle cases where either base_path or route is empty, improving robustness and usability.

* Enhance health check functionality and improve error handling

- Introduced a new method `_save_health_check_to_db` for saving health check results to the database, utilizing safe JSON functions for data integrity.
- Refactored existing health check methods to streamline the process and improve error logging.
- Updated email sending logic to ensure secure connections and better error handling.
- Improved spend update logic with batch processing and retry mechanisms for database operations.
- Added utility functions for projected spend calculations and enhanced validation for team configurations.

* Add health check methods for database interaction

- Introduced `save_health_check_result` method to save health check results with detailed logging and validation.
- Added `get_health_check_history` method for retrieving health check records with optional filtering.
- Implemented `get_all_latest_health_checks` method to fetch the latest health checks for each model.
- Enhanced error handling and logging for all new methods to improve reliability and traceability.

* Refactor health check result saving to use typed arguments

- Updated the `_save_health_check_to_db` function to call `save_health_check_result` with explicitly typed arguments instead of a dictionary spread, enhancing code clarity and type safety.
- Removed unused method bindings in the mock Prisma client tests to streamline the test setup.

* Remove unused `_save_health_check_to_db` function from utils.py to streamline code and improve maintainability.

* Implement response time validation and details cleaning in health check result saving

- Added `_validate_response_time` method to ensure response time values are valid and handle exceptions gracefully.
- Introduced `_clean_details` method to validate and clean details JSON, improving data integrity.
- Refactored `save_health_check_result` to utilize these new methods for optional fields, enhancing code clarity and maintainability.
- Updated tests to bind new methods to the mock Prisma client for comprehensive testing.

* Add health check utility functions and refactor existing endpoints

- Introduced `_convert_health_check_to_dict` to standardize health check record conversion to dictionary format for JSON responses.
- Added `_check_prisma_client` helper function to streamline database availability checks and improve error handling.
- Refactored health check endpoints to utilize the new utility functions, enhancing code clarity and maintainability.

* Refactor health check tests for improved clarity and coverage

- Simplified the mock PrismaClient setup by consolidating method bindings.
- Updated health check result saving tests to use parameterized scenarios for better coverage.
- Added tests for health check history retrieval and graceful handling when no database client is provided.
- Removed redundant mock functions to streamline the test suite.

* Implement helper function for health check and database saving

- Added `_perform_health_check_and_save` to encapsulate health check execution and optional database saving.
- Refactored health endpoint logic to utilize the new helper function, improving code clarity and reducing redundancy.
- Enhanced error handling and streamlined the process of saving health check results to the database.
2025-06-14 14:27:11 -07:00
..
_experimental/mcp_server/auth [Feat] Add Authentication + Permission Management for MCP List, Call Tool Ops (#11682) 2025-06-13 10:13:51 -07:00
anthropic_endpoints
auth Show remaining users on UI (#11568) 2025-06-09 22:04:45 -07:00
client
common_utils Update enduser spend and budget reset date based on budget duration (#8460) 2025-06-08 08:39:14 -07:00
db build(VLLM-Passthrough-with-loadbalancing-support-(enables-using-model-list-for-VLLM-/classify-endpoint)): Closes #11205 2025-05-31 09:00:04 -07:00
experimental/mcp_server
guardrails build(VLLM-Passthrough-with-loadbalancing-support-(enables-using-model-list-for-VLLM-/classify-endpoint)): Closes #11205 2025-05-31 09:00:04 -07:00
health_endpoints
hooks GA Multi-instance rate limiting v2 Requirements + New - specify token rate limit type - output / input / total (#11646) 2025-06-11 22:05:13 -07:00
image_endpoints Feat: add add azure endpoint for image endpoints (#11482) 2025-06-06 10:38:37 -07:00
management_endpoints fix(internal_user_endpoints.py): support user with + in email on us… (#11601) 2025-06-10 22:13:10 -07:00
middleware
openai_files_endpoint
pass_through_endpoints fix(vertex_ai): support global location in vertex ai passthrough (#11661) 2025-06-13 09:51:04 -07:00
spend_tracking
test_configs
types_utils
ui_crud_endpoints Refactor settings response models in proxy_setting_endpoints.py 2025-06-04 15:08:05 -06:00
test_caching_routes.py
test_common_request_processing.py [Feat] DD Trace - Add instrumentation for streaming chunks (#11338) 2025-06-02 16:48:39 -07:00
test_custom_proxy.py Custom Root Path Improvements: don't require reserving /litellm route (#11460) 2025-06-05 16:36:47 -07:00
test_health_check_functions.py Implement health check backend API and storage functionality (#11678) 2025-06-14 14:27:11 -07:00
test_litellm_pre_call_utils.py
test_proxy_cli.py [Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout (#11594) 2025-06-10 13:30:19 -07:00
test_proxy_server.py
test_proxy_types.py Litellm dev 06 06 2025 p1 (#11496) 2025-06-07 09:12:16 -07:00
test_proxy_utils.py UI / SSO - Update proxy admin id role in DB + Handle SSO redirects with custom root path (#11384) 2025-06-03 21:16:55 -07:00
test_route_llm_request.py
test_spend_log_cleanup.py [Feat] Make batch size for maximum retention in spend logs a controllable parameter (#11459) 2025-06-05 17:11:51 -07:00
test_team_member_update.py