From fcb19af842810c4c87f56c72bb658ed1d800f300 Mon Sep 17 00:00:00 2001 From: Vince Lwt Date: Mon, 21 Aug 2023 11:59:02 +0200 Subject: [PATCH] add llmonitor to docs --- docs/my-website/docs/index.md | 28 +++++++++----- .../docs/observability/callbacks.md | 20 +++++----- .../observability/llmonitor_integration.md | 38 +++++++++++++++++++ 3 files changed, 68 insertions(+), 18 deletions(-) create mode 100644 docs/my-website/docs/observability/llmonitor_integration.md diff --git a/docs/my-website/docs/index.md b/docs/my-website/docs/index.md index 4ca4703356..8fbce1e29d 100644 --- a/docs/my-website/docs/index.md +++ b/docs/my-website/docs/index.md @@ -1,23 +1,30 @@ # 🚅 litellm -a light 100 line package to simplify calling OpenAI, Azure, Cohere, Anthropic APIs + +a light 100 line package to simplify calling OpenAI, Azure, Cohere, Anthropic APIs ###### litellm manages: -* Calling all LLM APIs using the OpenAI format - `completion(model, messages)` -* Consistent output for all LLM APIs, text responses will always be available at `['choices'][0]['message']['content']` -* Consistent Exceptions for all LLM APIs, we map RateLimit, Context Window, and Authentication Error exceptions across all providers to their OpenAI equivalents. [see Code](https://github.com/BerriAI/litellm/blob/ba1079ff6698ef238c5c7f771dd2b698ec76f8d9/litellm/utils.py#L250) + +- Calling all LLM APIs using the OpenAI format - `completion(model, messages)` +- Consistent output for all LLM APIs, text responses will always be available at `['choices'][0]['message']['content']` +- Consistent Exceptions for all LLM APIs, we map RateLimit, Context Window, and Authentication Error exceptions across all providers to their OpenAI equivalents. [see Code](https://github.com/BerriAI/litellm/blob/ba1079ff6698ef238c5c7f771dd2b698ec76f8d9/litellm/utils.py#L250) ###### observability: -* Logging - see exactly what the raw model request/response is by plugging in your own function `completion(.., logger_fn=your_logging_fn)` and/or print statements from the package `litellm.set_verbose=True` -* Callbacks - automatically send your data to Helicone, Sentry, Posthog, Slack - `litellm.success_callbacks`, `litellm.failure_callbacks` [see Callbacks](https://litellm.readthedocs.io/en/latest/advanced/) + +- Logging - see exactly what the raw model request/response is by plugging in your own function `completion(.., logger_fn=your_logging_fn)` and/or print statements from the package `litellm.set_verbose=True` +- Callbacks - automatically send your data to Helicone, LLMonitor, Sentry, Posthog, Slack - `litellm.success_callbacks`, `litellm.failure_callbacks` [see Callbacks](https://litellm.readthedocs.io/en/latest/advanced/) ## Quick Start + Go directly to code: [Getting Started Notebook](https://colab.research.google.com/drive/1gR3pY-JzDZahzpVdbGBtrNGDBmzUNJaJ?usp=sharing) + ### Installation + ``` pip install litellm ``` ### Usage + ```python from litellm import completion @@ -33,11 +40,14 @@ response = completion(model="gpt-3.5-turbo", messages=messages) # cohere call response = completion("command-nightly", messages) ``` + Need Help / Support : [see troubleshooting](https://litellm.readthedocs.io/en/latest/troubleshoot) -## Why did we build liteLLM +## Why did we build liteLLM + - **Need for simplicity**: Our code started to get extremely complicated managing & translating calls between Azure, OpenAI, Cohere ## Support -* [Meet with us 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) -* Contact us at ishaan@berri.ai / krrish@berri.ai + +- [Meet with us 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) +- Contact us at ishaan@berri.ai / krrish@berri.ai diff --git a/docs/my-website/docs/observability/callbacks.md b/docs/my-website/docs/observability/callbacks.md index 7ac67b30df..74641f147c 100644 --- a/docs/my-website/docs/observability/callbacks.md +++ b/docs/my-website/docs/observability/callbacks.md @@ -1,29 +1,31 @@ # Callbacks ## Use Callbacks to send Output Data to Posthog, Sentry etc -liteLLM provides `success_callbacks` and `failure_callbacks`, making it easy for you to send data to a particular provider depending on the status of your responses. -liteLLM supports: +liteLLM provides `success_callbacks` and `failure_callbacks`, making it easy for you to send data to a particular provider depending on the status of your responses. + +liteLLM supports: - [Helicone](https://docs.helicone.ai/introduction) -- [Sentry](https://docs.sentry.io/platforms/python/) +- [LLMonitor](https://llmonitor.com/docs) +- [Sentry](https://docs.sentry.io/platforms/python/) - [PostHog](https://posthog.com/docs/libraries/python) - [Slack](https://slack.dev/bolt-python/concepts) ### Quick Start + ```python from litellm import completion # set callbacks -litellm.success_callback=["posthog", "helicone"] -litellm.failure_callback=["sentry"] +litellm.success_callback=["posthog", "helicone", "llmonitor"] +litellm.failure_callback=["sentry", "llmonitor"] ## set env variables os.environ['SENTRY_API_URL'], os.environ['SENTRY_API_TRACE_RATE']= "" os.environ['POSTHOG_API_KEY'], os.environ['POSTHOG_API_URL'] = "api-key", "api-url" -os.environ["HELICONE_API_KEY"] = "" +os.environ["HELICONE_API_KEY"] = "" +os.environ["LLMONITOR_APP_ID"] = "" -response = completion(model="gpt-3.5-turbo", messages=messages) +response = completion(model="gpt-3.5-turbo", messages=messages) ``` - - diff --git a/docs/my-website/docs/observability/llmonitor_integration.md b/docs/my-website/docs/observability/llmonitor_integration.md new file mode 100644 index 0000000000..f263050440 --- /dev/null +++ b/docs/my-website/docs/observability/llmonitor_integration.md @@ -0,0 +1,38 @@ +# LLMonitor Tutorial + +[LLMonitor](https://llmonitor.com/) is an open source observability platform that provides cost tracking, user tracking and powerful agent tracing. + +## Use LLMonitor to log requests across all LLM Providers (OpenAI, Azure, Anthropic, Cohere, Replicate, PaLM) + +liteLLM provides `success_callbacks` and `failure_callbacks`, making it easy for you to send data to a particular provider depending on the status of your responses. + +### Using Callbacks + +Use just 2 lines of code, to instantly log your responses **across all providers** with llmonitor: + +``` +litellm.success_callback=["llmonitor"] +litellm.error_callback=["llmonitor"] +``` + +Complete code + +```python +from litellm import completion + +## set env variables +os.environ["LLMONITOR_APP_ID"] = "your-llmonitor-app-id" +# Optional: os.environ["LLMONITOR_API_URL"] = "self-hosting-url" + +os.environ["OPENAI_API_KEY"], os.environ["COHERE_API_KEY"] = "", "" + +# set callbacks +litellm.success_callback=["llmonitor"] +litellm.error_callback=["llmonitor"] + +#openai call +response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}]) + +#cohere call +response = completion(model="command-nightly", messages=[{"role": "user", "content": "Hi 👋 - i'm cohere"}]) +```