(docs) simple proxy

2023-11-09 10:46:15 -08:00 · 2023-11-09 10:46:15 -08:00 · fbebb28970
commit fbebb28970
parent 86ef2a02f7
1 changed files with 13 additions and 0 deletions
--- a/docs/my-website/docs/simple_proxy.md
+++ b/docs/my-website/docs/simple_proxy.md
@ -342,6 +342,19 @@ print(result)
 </Tabs>

 ## Proxy Configs
+The Config allows you to set a `model_list` and define litellm specific params
+Supported params in the config
+```yaml
+model_list:
+  - model_name: zephyr-alpha
+    litellm_params: # params for litellm.completion() - https://docs.litellm.ai/docs/completion/input#input---request-body
+      model: huggingface/HuggingFaceH4/zephyr-7b-alpha
+      api_base: http://0.0.0.0:8001
+  - model_name: zephyr-beta
+    litellm_params:
+      model: huggingface/HuggingFaceH4/zephyr-7b-beta
+      api_base: https://<my-hosted-endpoint>
+```


 ### Set Custom Prompt Templates