dcbccd1fea
* docs: Corrected documentation updates from Sept 2025 This PR contains the actual intended documentation changes, properly synced with main: ✅ Real changes applied: - Added AWS authentication link to bedrock guardrails documentation - Updated Vertex AI with Gemini API alternative configuration - Added async_post_call_success_hook code snippet to custom callback docs - Added SSO free for up to 5 users information to enterprise and custom_sso docs - Added SSO free information block to security.md - Added cancel response API usage and curl example to response_api.md - Added image for modifying default user budget via admin UI - Re-ordered sidebars in documentation ❌ Sync issues resolved: - Kept all upstream changes that were added to main after branch diverged - Preserved Provider-Specific Metadata Parameters section that was added upstream - Maintained proper curl parameter formatting (-d instead of -D) This corrects the sync issues from the original PR #14769. * docs: Restore missing files from original PR Added back ~16 missing documentation files that were part of the original PR: ✅ Restored files: - docs/my-website/docs/completion/usage.md - docs/my-website/docs/fine_tuning.md - docs/my-website/docs/getting_started.md - docs/my-website/docs/image_edits.md - docs/my-website/docs/image_generation.md - docs/my-website/docs/index.md - docs/my-website/docs/moderation.md - docs/my-website/docs/observability/callbacks.md - docs/my-website/docs/providers/bedrock.md - docs/my-website/docs/proxy/caching.md - docs/my-website/docs/proxy/config_settings.md - docs/my-website/docs/proxy/db_deadlocks.md - docs/my-website/docs/proxy/load_balancing.md - docs/my-website/docs/proxy_api.md - docs/my-website/docs/rerank.md ✅ Fixed context-caching issue: - Restored provider_specific_params.md to main version (preserving Provider-Specific Metadata Parameters section) - Your original PR didn't intend to modify this file - it was just a sync issue Now includes all ~26 documentation files from the original PR #14769. * docs: Remove files that were deleted in original PR - Removed docs/my-website/docs/providers/azure_ai_img_edit.md (was deleted in original PR) - sdk/headers.md was already not present Now matches the complete intended changes from original PR #14769. * docs: Restore azure_ai_img_edit.md from main - Restored docs/my-website/docs/providers/azure_ai_img_edit.md from main branch - This file should not have been deleted as it was a newer commit - SDK headers file doesn't exist in main (was reverted) and wasn't part of your original changes Fixes the file restoration issues. * docs: Fix vertex.md - preserve context caching from newer commit - Restored vertex.md to main version to preserve context caching content (lines 817-887) - Added back only your intended change: alternative gemini config example - Context caching content from newer commit is now preserved Fixes the vertex.md sync issue where newer content was incorrectly deleted. * docs: Fix providers/bedrock.md - restore deleted content from newer commit - Restored providers/bedrock.md to main version - Preserves 'Usage - Request Metadata' section that was added in newer commit - Your actual intended change was to proxy/guardrails/bedrock.md (authentication tip) which is preserved - Now only has additions, no subtractions as intended Fixes the bedrock.md sync issue. * docs: Restore missing IAM policy section in bedrock.md Added back your intended IAM policy documentation that was lost when restoring main version: ✅ Added IAM AssumeRole Policy section: - Explains requirement for sts:AssumeRole permission - Shows error message example when permission missing - Provides complete IAM policy JSON example - Links to AWS AssumeRole documentation - Clarifies trust policy requirements Now bedrock.md has both: - All newer content preserved (Request Metadata section) - Your intended IAM policy addition restored --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>
86 lines
2.8 KiB
Markdown
86 lines
2.8 KiB
Markdown
# 🔑 LiteLLM Keys (Access Claude-2, Llama2-70b, etc.)
|
|
|
|
Use this if you're trying to add support for new LLMs and need access for testing. We provide a free $10 community-key for testing all providers on LiteLLM:
|
|
|
|
## usage (community-key)
|
|
|
|
```python
|
|
import os
|
|
from litellm import completion
|
|
|
|
## set ENV variables
|
|
os.environ["OPENAI_API_KEY"] = "your-api-key"
|
|
os.environ["COHERE_API_KEY"] = "your-api-key"
|
|
|
|
messages = [{ "content": "Hello, how are you?","role": "user"}]
|
|
|
|
# openai call
|
|
response = completion(model="gpt-3.5-turbo", messages=messages)
|
|
|
|
# cohere call
|
|
response = completion("command-nightly", messages)
|
|
```
|
|
|
|
**Need a dedicated key?**
|
|
Email us @ krrish@berri.ai
|
|
|
|
## Supported Models for LiteLLM Key
|
|
These are the models that currently work with the "sk-litellm-.." keys.
|
|
|
|
For a complete list of models/providers that you can call with LiteLLM, [check out our provider list](./providers/) or check out [models.litellm.ai](https://models.litellm.ai/)
|
|
|
|
* OpenAI models - [OpenAI docs](./providers/openai.md)
|
|
* gpt-4
|
|
* gpt-3.5-turbo
|
|
* gpt-3.5-turbo-16k
|
|
* Llama2 models - [TogetherAI docs](./providers/togetherai.md)
|
|
* togethercomputer/llama-2-70b-chat
|
|
* togethercomputer/llama-2-70b
|
|
* togethercomputer/LLaMA-2-7B-32K
|
|
* togethercomputer/Llama-2-7B-32K-Instruct
|
|
* togethercomputer/llama-2-7b
|
|
* togethercomputer/CodeLlama-34b
|
|
* WizardLM/WizardCoder-Python-34B-V1.0
|
|
* NousResearch/Nous-Hermes-Llama2-13b
|
|
* Falcon models - [TogetherAI docs](./providers/togetherai.md)
|
|
* togethercomputer/falcon-40b-instruct
|
|
* togethercomputer/falcon-7b-instruct
|
|
* Jurassic/AI21 models - [AI21 docs](./providers/ai21.md)
|
|
* j2-ultra
|
|
* j2-mid
|
|
* j2-light
|
|
* NLP Cloud models - [NLPCloud docs](./providers/nlp_cloud.md)
|
|
* dolpin
|
|
* chatdolphin
|
|
* Anthropic models - [Anthropic docs](./providers/anthropic.md)
|
|
* claude-2
|
|
* claude-instant-v1
|
|
|
|
|
|
## For OpenInterpreter
|
|
This was initially built for the Open Interpreter community. If you're trying to use this feature in there, here's how you can do it:
|
|
**Note**: You will need to clone and modify the Github repo, until [this PR is merged.](https://github.com/KillianLucas/open-interpreter/pull/288)
|
|
|
|
```
|
|
git clone https://github.com/krrishdholakia/open-interpreter-litellm-fork
|
|
```
|
|
To run it do:
|
|
```
|
|
poetry build
|
|
|
|
# call gpt-4 - always add 'litellm_proxy/' in front of the model name
|
|
poetry run interpreter --model litellm_proxy/gpt-4
|
|
|
|
# call llama-70b - always add 'litellm_proxy/' in front of the model name
|
|
poetry run interpreter --model litellm_proxy/togethercomputer/llama-2-70b-chat
|
|
|
|
# call claude-2 - always add 'litellm_proxy/' in front of the model name
|
|
poetry run interpreter --model litellm_proxy/claude-2
|
|
```
|
|
|
|
And that's it!
|
|
|
|
Now you can call any model you like!
|
|
|
|
|
|
Want us to add more models? [Let us know!](https://github.com/BerriAI/litellm/issues/new/choose) |