1391e41916
* fix(vertex_ai): improve passthrough endpoint url parsing and construction (#17402) * test(proxy): add test for vertex passthrough load balancing Add a test that verifies _base_vertex_proxy_route uses get_available_deployment for proper load balancing instead of get_model_list. This ensures the correct deployment is selected from the router and vertex credentials are properly fetched. Also refactor the implementation to: - Use get_available_deployment instead of get_model_list - Add error handling for deployment retrieval - Improve code structure with try-except block * feat(proxy): add pass-through deployment filtering methods Add dedicated methods to filter and select deployments for pass-through endpoints: - Implement get_available_deployment_for_pass_through() to ensure only deployments with use_in_pass_through=True are considered - Implement async_get_available_deployment_for_pass_through() for async operations - Add _filter_pass_through_deployments() helper method to filter by use_in_pass_through flag - Update vertex pass-through route to use the new dedicated method This ensures pass-through endpoints respect the use_in_pass_through configuration and apply proper load balancing strategy only to configured deployments. Add comprehensive tests to verify filtering and load balancing behavior. |
||
|---|---|---|
| .. | ||
| llm_provider_handlers | ||
| test_llm_pass_through_endpoints.py | ||
| test_pass_through_endpoints.py | ||
| test_passthrough_endpoints_common_utils.py | ||
| test_passthrough_guardrails_field_targeting.py | ||
| test_passthrough_guardrails.py | ||
| test_vertex_ai_batch_passthrough.py | ||
| test_vertex_passthrough_load_balancing.py | ||