Alexsander Hamir
9338727960
feat(proxy): support absolute RPM/TPM in priority_reservation (#15813)
* feat(proxy): support absolute RPM/TPM in priority_reservation
Allow priority reservations as absolute values instead of percentages:
- Float: {'prod': 0.75} (75%, existing)
- RPM: {'prod': {'type': 'rpm', 'value': 750}}
- TPM: {'prod': {'type': 'tpm', 'value': 750000}}
Added _convert_to_percent() that converts absolute values to percentages
based on model capacity. Fully backward compatible.
* feat(types): convert priority_reservation Dict to TypedDict
Add PriorityReservationDict TypedDict to replace generic Dict type in priority_reservation configuration.
Changes:
- Add PriorityReservationDict to litellm/types/utils.py
- Update convert_priority_to_percent() signature in rate_limiter_utils.py
- Update litellm.priority_reservation type annotation in __init__.py
Improves IDE autocomplete and type checking for priority reservation configs.
* docs: update dynamic rate limiter priority reservation docs