docs how to use custom header for litellm proxy

This commit is contained in:
Ishaan Jaff 2024-07-16 16:54:13 -07:00 committed by Krrish Dholakia
parent 6c872fea1d
commit f52102e2e8

View File

@ -347,6 +347,70 @@ curl --location 'http://localhost:4000/key/generate' \
"max_budget": 0,}'
```
## Advanced - Pass LiteLLM Key in custom header
Use this to make LiteLLM proxy look for the virtual key in a custom header instead of the default `"Authorization"` header
**Step 1** Define `litellm_key_header_name` name on litellm config.yaml
```yaml
model_list:
- model_name: fake-openai-endpoint
litellm_params:
model: openai/fake
api_key: fake-key
api_base: https://exampleopenaiendpoint-production.up.railway.app/
general_settings:
master_key: sk-1234
litellm_key_header_name: "X-Litellm-Key" # 👈 Key Change
```
**Step 2** Test it
In this request, litellm will use the Virtual key in the `X-Litellm-Key` header
<Tabs>
<TabItem value="curl" label="curl">
```shell
curl http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "X-Litellm-Key: Bearer sk-1234" \
-H "Authorization: Bearer bad-key" \
-d '{
"model": "fake-openai-endpoint",
"messages": [
{"role": "user", "content": "Hello, Claude gm!"}
]
}'
```
**Expected Response**
Expect to see a successfull response from the litellm proxy since the key passed in `X-Litellm-Key` is valid
```shell
{"id":"chatcmpl-f9b2b79a7c30477ab93cd0e717d1773e","choices":[{"finish_reason":"stop","index":0,"message":{"content":"\n\nHello there, how may I assist you today?","role":"assistant","tool_calls":null,"function_call":null}}],"created":1677652288,"model":"gpt-3.5-turbo-0125","object":"chat.completion","system_fingerprint":"fp_44709d6fcb","usage":{"completion_tokens":12,"prompt_tokens":9,"total_tokens":21}
```
</TabItem>
<TabItem value="python" label="OpenAI Python SDK">
```python
client = openai.OpenAI(
api_key="not-used",
base_url="https://api-gateway-url.com/llmservc/api/litellmp",
default_headers={
"Authorization": f"Bearer {API_GATEWAY_TOKEN}", # (optional) For your API Gateway
"X-Litellm-Key": f"Bearer sk-1234" # For LiteLLM Proxy
}
)
```
</TabItem>
</Tabs>
## Advanced - Custom Auth
You can now override the default api key auth.