LiteLLM Client: 1-Click Deploy LLMs + Debug Logs

LiteLLM offers a UI to:

1-Click Deploy LLMs - the client stores your api keys + model configurations
Debug your Call Logs

Getting Started

Make a litellm.completion() call 👉 get your debugging dashboard

Example Code: Regular litellm.completion() call:

from litellm import completion
messages = [{ "content": "Hello, how are you?" ,"role": "user"}]
response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}])

Completion() Output with dashboard

All completion() calls print with a link to your session dashboard

Example Output from litellm completion

Here's your LiteLLM Dashboard 👉 https://admin.litellm.ai/88911906-d786-44f2-87c7-9720e6031b45
<OpenAIObject chat.completion id=chatcmpl-7r6LtlUXYYu0QayfhS3S0OzroiCel at 0x7fb307375030> JSON: {
  "id": "chatcmpl-7r6LtlUXYYu0QayfhS3S0OzroiCel",
  "object": "chat.completion",
  "created": 1692890157,
  "model": "gpt-3.5-turbo-0613",
..............

Once created, your dashboard is viewable at - admin.litellm.ai/<your_email> 👋 Tell us if you need better privacy controls See our live dashboard 👉 admin.litellm.ai

Code Setup

import litellm

## Setup for activating / using the litellm dashboard
litellm.email = "test_email@test.com"

LiteLLM Dashboard - 1-Click Deploy LLMs

LiteLLM allows you to add a new model using the liteLLM Dashboard

Navigate to the 'Add New LLM' Section <Image img={require('../../img/add_model.png')} alt="Dashboard" />

Select Provider
Select your LLM
Add your LLM Key

LiteLLM Dashboard - Debug Logs

All your completion() and embedding() call logs are available on admin.litellm.ai/<your-token> See your Logs below

Using your new LLM - Completion() with the LiteLLM Dashboard

from litellm import embedding, completion
# keys set in admin.litellm.ai/<your_email> or .env OPENAI_API_KEY
messages = [{ "content": "Hello, how are you?" ,"role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}])

2.6 KiB Raw Blame History