7.4 KiB
Embedding Models
Quick Start
from litellm import embedding
import os
os.environ['OPENAI_API_KEY'] = ""
response = embedding(model='text-embedding-ada-002', input=["good morning from litellm"])
Input Params for litellm.embedding()
Required Fields
-
model: string - ID of the model to use.model='text-embedding-ada-002' -
input: array - Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays. The input must not exceed the max input tokens for the model (8192 tokens for text-embedding-ada-002), cannot be an empty string, and any array must be 2048 dimensions or less.
input=["good morning from litellm"]
Optional LiteLLM Fields
-
user: string (optional) A unique identifier representing your end-user, -
timeout: integer - The maximum time, in seconds, to wait for the API to respond. Defaults to 600 seconds (10 minutes). -
api_base: string (optional) - The api endpoint you want to call the model with -
api_version: string (optional) - (Azure-specific) the api version for the call -
api_key: string (optional) - The API key to authenticate and authorize requests. If not provided, the default API key is used. -
api_type: string (optional) - The type of API to use.
Output from litellm.embedding()
{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [
-0.0022326677571982145,
0.010749882087111473,
...
...
...
]
}
],
"model": "text-embedding-ada-002-v2",
"usage": {
"prompt_tokens": 10,
"total_tokens": 10
}
}
OpenAI Embedding Models
Usage
from litellm import embedding
import os
os.environ['OPENAI_API_KEY'] = ""
response = embedding('text-embedding-ada-002', input=["good morning from litellm"])
| Model Name | Function Call | Required OS Variables |
|---|---|---|
| text-embedding-ada-002 | embedding('text-embedding-ada-002', input) |
os.environ['OPENAI_API_KEY'] |
Azure OpenAI Embedding Models
API keys
This can be set as env variables or passed as params to litellm.embedding()
import os
os.environ['AZURE_API_KEY'] =
os.environ['AZURE_API_BASE'] =
os.environ['AZURE_API_VERSION'] =
Usage
from litellm import embedding
response = embedding(
model="azure/<your deployment name>",
input=["good morning from litellm"],
api_key=api_key,
api_base=api_base,
api_version=api_version,
)
print(response)
| Model Name | Function Call |
|---|---|
| text-embedding-ada-002 | embedding(model="azure/<your deployment name>", input=input) |
h/t to Mikko for this integration
OpenAI Compatible Embedding Models
Use this for calling /embedding endpoints on OpenAI Compatible Servers, example https://github.com/xorbitsai/inference
Note add openai/ prefix to model so litellm knows to route to OpenAI
Usage
from litellm import embedding
response = embedding(
model = "openai/<your-llm-name>", # add `openai/` prefix to model so litellm knows to route to OpenAI
api_base="http://0.0.0.0:8000/" # set API Base of your Custom OpenAI Endpoint
input=["good morning from litellm"]
)
Bedrock Embedding
API keys
This can be set as env variables or passed as params to litellm.embedding()
import os
os.environ["AWS_ACCESS_KEY_ID"] = "" # Access key
os.environ["AWS_SECRET_ACCESS_KEY"] = "" # Secret access key
os.environ["AWS_REGION_NAME"] = "" # us-east-1, us-east-2, us-west-1, us-west-2
Usage
from litellm import embedding
response = embedding(
model="amazon.titan-embed-text-v1",
input=["good morning from litellm"],
)
print(response)
| Model Name | Function Call |
|---|---|
| Titan Embeddings - G1 | embedding(model="amazon.titan-embed-text-v1", input=input) |
| Cohere Embeddings - English | embedding(model="cohere.embed-english-v3", input=input) |
| Cohere Embeddings - Multilingual | embedding(model="cohere.embed-multilingual-v3", input=input) |
Cohere Embedding Models
https://docs.cohere.com/reference/embed
Usage
from litellm import embedding
os.environ["COHERE_API_KEY"] = "cohere key"
# cohere call
response = embedding(
model="embed-english-v3.0",
input=["good morning from litellm", "this is another item"],
input_type="search_document" # optional param for v3 llms
)
| Model Name | Function Call |
|---|---|
| embed-english-v3.0 | embedding(model="embed-english-v3.0", input=["good morning from litellm", "this is another item"]) |
| embed-english-light-v3.0 | embedding(model="embed-english-light-v3.0", input=["good morning from litellm", "this is another item"]) |
| embed-multilingual-v3.0 | embedding(model="embed-multilingual-v3.0", input=["good morning from litellm", "this is another item"]) |
| embed-multilingual-light-v3.0 | embedding(model="embed-multilingual-light-v3.0", input=["good morning from litellm", "this is another item"]) |
| embed-english-v2.0 | embedding(model="embed-english-v2.0", input=["good morning from litellm", "this is another item"]) |
| embed-english-light-v2.0 | embedding(model="embed-english-light-v2.0", input=["good morning from litellm", "this is another item"]) |
| embed-multilingual-v2.0 | embedding(model="embed-multilingual-v2.0", input=["good morning from litellm", "this is another item"]) |
HuggingFace Embedding Models
LiteLLM supports all Feature-Extraction Embedding models: https://huggingface.co/models?pipeline_tag=feature-extraction
Usage
from litellm import embedding
import os
os.environ['HUGGINGFACE_API_KEY'] = ""
response = embedding(
model='huggingface/microsoft/codebert-base',
input=["good morning from litellm"]
)
Usage - Custom API Base
from litellm import embedding
import os
os.environ['HUGGINGFACE_API_KEY'] = ""
response = embedding(
model='huggingface/microsoft/codebert-base',
input=["good morning from litellm"],
api_base = "https://p69xlsj6rpno5drq.us-east-1.aws.endpoints.huggingface.cloud"
)
| Model Name | Function Call | Required OS Variables |
|---|---|---|
| microsoft/codebert-base | embedding('huggingface/microsoft/codebert-base', input=input) |
os.environ['HUGGINGFACE_API_KEY'] |
| BAAI/bge-large-zh | embedding('huggingface/BAAI/bge-large-zh', input=input) |
os.environ['HUGGINGFACE_API_KEY'] |
| any-hf-embedding-model | embedding('huggingface/hf-embedding-model', input=input) |
os.environ['HUGGINGFACE_API_KEY'] |