Commit Graph

468 Commits

Author SHA1 Message Date
Krish Dholakia
00be76abf4
UI - Custom Server Root Path (Multiple Fixes) (#11337)
* fix(proxy_server.py): working swagger on custom base

removes the swagger monkey patch - this seems to render the swagger on custom base paths

* fix(ui/): working custom auth uptil login success event

* fix(ui/): working custom server root path for login

* fix(proxy_server.py): create typed dict for ui returned token

allows better documentation of expected params

* refactor(proxy_server.py): refactor all ui login endpoints to use same returned ui token object

* feat(ui_sso.py): add server root path to ui token

* feat(ui_sso.py): allows ui to call correct endpoint

* fix(networking.tsx): update proxy base url with custom root path

* fix(networking.tsx): handle updating proxy base url for non-local instances

* refactor: remove uneccessary references to proxybaseurl in ui code - reduce potential for errors

* fix: fix linting error

* fix(onboarding_link.tsx): fix onboarding link when custom server path is set

* feat(ui_discovery_endpoints.py): add new public .well-known/ route for litellm ui config

returns the server root path and proxy base url for constructing api calls

* feat(_types.py): add litellm well known config as public route

allows ui to query it

* fix(/_types.py): add .well-known config to as public route

* fix(page.tsx): create pattern for loading in ui config before making network requests

ensures requests are formatted correctly

* fix(page.tsx): call credential endpoint once ui config is loaded

* fix(page.tsx): route correctly to litellm dashboard from new user login

* fix(page.tsx): remove hardcoded `/litellm` for /sso/key/generate request

* fix(proxy_server.py): re-add moderations endpoint

* fix(proxy_server.py): mount __next__ at / and /litellm

allows it to work when proxy is mounted on root

* docs(contributing.md): remove /ui on ui doc - it will now run on root

* docs(custom_root_ui.md): add docs on custom root path
2025-06-02 17:48:03 -07:00
Krish Dholakia
83becdbc11
Litellm doc fixes 05 31 2025 (#11305)
* docs: cleanup

* docs: add anthropic file tutorial

* docs: add to sidebar
2025-06-01 00:53:56 -07:00
Ishaan Jaff
5e6f6ddc52
[Feat]: Add Bedrock InvokeAgents as a /chat/completions route on LiteLLM (#11239)
* feat: init structure for bedrock AGENTs

* feat: add basic  routing for bedrock AGENTs

* feat: add basic transforms for bedrock AGENTs

* fix: url for bedrock agent runtime

* fix: working agents request

* feat: working agents non-streaming request

* feat: bedrock agents

* feat: add streaming for bedrock agents

* feat: add cost tracking for bedrock agents

* docs litellm with bedrock agents

* fix: linting errors

* test: invoke agents tests
2025-05-29 16:48:55 -07:00
Akim Tsvigun
acaa80294c
Integration with Nebius AI Studio added (#11143)
* integration with Nebius AI Studio added

* Merged with main

* Reviewer's comments resolved

* spelling error fixed

* accidental change reverted
2025-05-27 11:05:22 -07:00
Ishaan Jaff
1637413677 docs image edits 2025-05-24 19:43:48 -07:00
Krrish Dholakia
c725cbbde6 docs: add managed finetuning docs 2025-05-24 16:14:35 -07:00
Mayank
8da898c55e
[feature] ConfidentAI logging enabled for proxy and sdk (#10649)
* async success implemented

* fail async event

* sync events added

* docs added

* docs added

* test added

* style

* test

* .

* lock file genrated due to tenacity change

* mypy errors

* resolved comments

* resolved comments

* resolved comments

* resolved comments

* style

* style

* resolved comments
2025-05-23 11:40:48 -07:00
DarinVerheijke
54a0766ff9
Add provider Featherless AI (access to 4200+ models) (#10596)
* add featherless provider

* change docs

* clean lint

* fix provider logic

* Add fixes per review

* update parameters

* update error msg

* remove featherless

* add tests

* add mock test e2e

* update to openai mock test
2025-05-17 19:38:46 -07:00
Ishaan Jaff
d367eb3460
[Docs] Pii masking with LiteLLM (#10924)
* doc pii masking

* remove old doc

* docs add deployment options

* docs guardrail pii masking

* docs pii masking / tracing

* add images of tracing guards litellm ui

* docs presidio pii
2025-05-17 18:35:33 -07:00
Ishaan Jaff
cdf6026fdd
docs apply guard (#10923) 2025-05-17 18:34:35 -07:00
Krish Dholakia
b4fc703d3b
LiteLLM Stable release notes (#10919)
* docs(index/v1.70.1-stable): style improvements

* style: add style improvements to docs

* docs: cleanup docs

* docs: more style improvements

* docs: style improvements

* docs(gemini/realtime): add docs on realtime api via Google AI Studio

* docs: add openai example to anthropic web search docs

* docs: add missing doc links

* docs: doc cleanup

* docs: add more doc links

* fix: cleanup

* docs: add docker information

* docs: update doc links

* docs: add demo instance details to doc

s
2025-05-17 17:54:25 -07:00
Jugal D. Bhatt
b342aa9253
add docs for spend logs (#10913)
* add docs for spend logs

* change docs

* change docs

* change docs and change default run loops.

There was a calc error previously, it was not a million when run 100 times, but 100k, now default run would delete 500k records in 50s (running 500 loops of function before exiting)

* add docs changes to ui logs to metion deletion
2025-05-17 16:00:28 -07:00
Krish Dholakia
d37cc63250
Add new model provider Novita AI (#7582) (#9527)
* Add new model provider Novita AI (#7582)

* feat: add new model provider Novita AI

* feat: use deepseek r1 model for examples in Novita AI docs

* fix: fix tests

* fix: fix tests for novita

* fix: fix novita transformation

* ci: fix ci yaml

* fix: fix novita transformation and test (#10056)

---------

Co-authored-by: Jason <ggbbddjm@gmail.com>
2025-05-12 21:49:30 -07:00
Ishaan Jaff
2cc4a87861
[Docs] Using litellm with Google ADK (#10777)
* docs litellm ADK usage

* docs litellm google adk

* docs litellm ADK

* docs litellm with ADK usage examples

* docs litellm proxy with ADK

* cookbook litellm ADK
2025-05-12 16:41:49 -07:00
Krrish Dholakia
990d6f5c93 docs(managed_batches.md): add docs on unified file id with batches 2025-05-10 11:28:24 -07:00
Marc Abramowitz
3787aa27b0
Add management client docs (#10703)
* Add management client docs

* Docs enhancements

* Add "proxy/management_cli" to sidebar

* Small formatting changes

* Add overview table

* Wrap long line

* Add "Quick Start" section

* Document uv tool upgrade and uninstall

* Add blank lines to eliminate yellow squigglies in VS Code

* For each section link to section on swagger used

* Wrap some long command examples

* Break long http request command on to multiple lines
2025-05-09 18:59:24 -07:00
GPTLocalhost (Word Add-in)
86f19d2fae
Add GPTLocalhost to "docs/my-website/docs/projects" (#10687)
* Create GPTLocalhost.md

* Update sidebars.js
2025-05-09 07:44:48 -07:00
Tom Matthews
ff279faf35
Add support for Nscale (EU-Sovereign) Provider (#10638)
* Add support for nscale provider

* Add image generation support and fix unit tests

* Add docs for nscale

* Fix unit test import issues

* Minor doc improvement

* Remove redundant null tokens from model cost map

* Address PR review comments for doc updates

* Revert changes to large text
2025-05-08 08:50:15 -07:00
Ishaan Jaff
f52593486c
[Contributor PR] Support Llama-api as an LLM provider (#10451) (#10538)
* Support Llama-api as an LLM provider (#10451)

* init: support llama-api as a llm provider

* docs: fix endpoint url

* fix: rename meta dir to meta-llama

* docs: add meta-llama info

* fix: mv LlamaAPIConfig under chat directory

* feat: add LlamaAPIConfig in ProviderConfigManager

* fix: provider_config from ProviderConfigManager

* feat: add supports_tool_choice param

* fix: remove optional_params using model_info

* fix: rename meta-llama to meta_llama

* init: test for meta_llama

* fix: model names

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* fix file naming convention

* fix file naming convention for meta_llama

* docs meta llama api litellm

---------

Co-authored-by: Young Han <110819238+seyeong-han@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-05-03 16:29:03 -07:00
Krish Dholakia
7ce687ef39
Litellm stable release notes 05 03 2025 (#10536)
* build(release_cycle.md): document bar for minor vs. patch updates

* docs(index.md): initial changelog doc

* docs(index.md): update llama docs

* docs(index.md): add docs for llm api endpoints + spend tracking/budget improvements

* docs: more doc cleanup

* docs(index.md): more doc cleanup

* docs(index.md): final doc cleanup
2025-05-03 12:47:11 -07:00
Ishaan Jaff
42ea608b4b docs add section on bedrock provider to show how to use KBs 2025-05-03 11:55:16 -07:00
Ishaan Jaff
de7870cb54
Add llamafile as a provider (#10203) (#10482)
* Update docs for OpenAI compatible providers, add Llamafile docs, include Llamafile in the sidebar

* Add Llamafile as an LlmProviders enum

* Add llamafile as a OpenAI compatible provider (in the list of compatible providers)

* Add Llamafile chat config and tests

* Wire up Llamafile

Co-authored-by: Peter Wilson <peter@mozilla.ai>
2025-05-01 18:36:55 -07:00
Ishaan Jaff
f30871ef13
[Feat] Add support for using Bedrock Knowledge Bases with LiteLLM /chat/completions requests (#10413)
* add make_bedrock_kb_retrieve_request

* working bedrock KB hook

* working bedrock KB hook

* test_openai_with_knowledge_base_mock_openai

* fix linting

* fix BedrockKnowledgeBaseHook

* docs using bedrock kb with litellm

* docs kb with litellm

* fix bedrock kb test

* DynamicPromptManagementParamLiteral

* fix _should_run_prompt_management_hooks_without_prompt_id

* test_init_custom_logger_compatible_class_as_callback
2025-04-29 17:29:02 -07:00
Ishaan Jaff
f7e75fa276
[Docs] UI Session Logs (#10334)
* add ui logs session doc

* docs add instructions on how to do session management

* docs session management

* docs session management

* docs responses api session management

* docs ui logs
2025-04-26 08:26:23 -07:00
Dwij
b2955a2bdd
Add AgentOps Integration to LiteLLM (#9685)
* feat(sidebars): add new item for agentops integration in Logging & Observability category

* Update agentops_integration.md to enhance title formatting and remove redundant section

* Enhance AgentOps integration in documentation and codebase by removing LiteLLMCallbackHandler references, adding environment variable configurations, and updating logging initialization for AgentOps support.

* Update AgentOps integration documentation to include instructions for obtaining API keys and clarify environment variable setup.

* Add unit tests for AgentOps integration and improve error handling in token fetching

* Add unit tests for AgentOps configuration and token fetching functionality

* Corrected agentops test directory

* Linting fix

* chore: add OpenTelemetry dependencies to pyproject.toml

* chore: update OpenTelemetry dependencies and add new packages in pyproject.toml and poetry.lock
2025-04-22 10:29:01 -07:00
Krrish Dholakia
1d9b58688b docs(sidebars.js): place scim doc in correct place 2025-04-20 09:20:10 -07:00
Ishaan Jaff
1be36be72e
Litellm docs SCIM (#10174)
* docs scim

* docs SCIM stash

* docs litellm SCIM

* docs fix

* docs scim with LiteLLM
2025-04-19 18:29:09 -07:00
Krish Dholakia
bbfcb1ac7e
Litellm release notes 04 19 2025 (#10169)
* docs(index.md): initial draft release notes

* docs: note all pending docs

* build(model_prices_and_context_window.json): add o3, gpt-4.1, o4-mini pricing

* docs(vllm.md): update vllm doc to show file message type support

* docs(mistral.md): add mistral passthrough route doc

* docs(gemini.md): add gemini thinking to docs

* docs(vertex.md): add thinking/reasoning content for gemini models to docs

* docs(index.md): more links

* docs(index.md): add more links, images

* docs(index.md): cleanup highlights
2025-04-19 17:26:30 -07:00
Ishaan Jaff
3d5022bd79
[Feat] Support for all litellm providers on Responses API (works with Codex) - Anthropic, Bedrock API, VertexAI, Ollama (#10132)
* transform request

* basic handler for LiteLLMCompletionTransformationHandler

* complete transform litellm to responses api

* fixes to test

* fix stream=True

* fix streaming iterator

* fixes for transformation

* fixes for anthropic codex support

* fix pass response_api_optional_params

* test anthropic responses api tools

* update responses types

* working codex with litellm

* add session handler

* fixes streaming iterator

* fix handler

* add litellm codex example

* fix code quality

* test fix

* docs litellm codex

* litellm codexdoc

* docs openai codex with litellm

* docs litellm openai codex

* litellm codex

* linting fixes for transforming responses API

* fix import error

* fix responses api test

* add sync iterator support for responses api
2025-04-18 19:53:59 -07:00
Ishaan Jaff
dcc43e797a
[Docs] Auto prompt caching (#10044)
* docs prompt cache controls

* doc fix auto prompt caching
2025-04-15 22:29:47 -07:00
Marc Klingen
db857c74d4
chore: ordering of logging & observability docs (#9994) 2025-04-14 16:49:04 -07:00
Ishaan Jaff
c86e678809
[Docs] v1.66.0-stable fixes (#9953)
* add categories for spend tracking improvements

* xai reasoning usage

* docs tag management

* docs tag based routing

* [Beta] Routing based

* docs tag based routing

* docs tag routing

* docs enterprise web search
2025-04-12 16:57:25 -07:00
Krish Dholakia
25d4cf1c1d
Litellm managed files docs (#9948)
* docs(files_endpoints.md): add doc on litellm managed files

* refactor: separate litellm managed file docs from `/files` docs

clearer

* docs(litellm_managed_files.md): add architecture diagram explaining managed files
2025-04-12 13:02:33 -07:00
Ishaan Jaff
98e34cbf5d
[Docs] Tutorial using MSFT auto team assignment with LiteLLM (#9898)
* add default_team_params as a config.yaml setting

* create_litellm_team_from_sso_group

* test_default_team_params

* test_create_team_without_default_params

* docs default team settings

* docs msft entra id tutorial

* commit litellm docs msft group assignment

* litellm MSFT sso

* member, team assignment on litellm

* docs msft auto assignment

* bug fix default team setting

* docs litellm default team settings

* test_default_team_params
2025-04-10 20:07:55 -07:00
Krrish Dholakia
86bfb8cd66 Revert "docs: initial commit adding api playground to docs"
This reverts commit 9d68008152904157eac61116b413d886702309ce.
2025-04-09 17:50:13 -07:00
Krrish Dholakia
5ca93a1950 docs: initial commit adding api playground to docs
makes it easy to see how litellm transforms your request
2025-04-09 17:50:13 -07:00
Krish Dholakia
792ee079c2
Litellm 04 05 2025 release notes (#9785)
* docs: update docs

* docs: additional cleanup

* docs(index.md): add initial links

* docs: more doc updates

* docs(index.md): add more links

* docs(files.md): add gemini files API to docs

* docs(index.md): add more docs

* docs: more docs

* docs: update docs
2025-04-06 09:03:51 -07:00
Ishaan Jaff
b48b8366c2 docs new deadlock fixing architecture 2025-04-02 13:24:53 -07:00
Tomer Bin
0690f7a3cb
Virtual key based policies in Aim Guardrails (#9499)
* report key alias to aim

* send litellm version to aim

* Update docs

* blacken

* add docs

* Add info part about virtual keys specific guards

* sort guardrails alphabetically

* fix ruff
2025-04-01 21:57:23 -07:00
Ishaan Jaff
2238a5585b doc fix mcp 2025-03-29 20:35:53 -07:00
Ishaan Jaff
08a4ba1b7e Merge branch 'main' into litellm_exp_mcp_server 2025-03-24 19:03:56 -07:00
Krish Dholakia
d0e022cfac
Merge pull request #9475 from BerriAI/litellm_dev_03_22_2025_release_note
Litellm dev 03 22 2025 release note
2025-03-22 20:12:43 -07:00
Krrish Dholakia
d92c5ad71c docs(index.md): completed documentation for new stable release 2025-03-22 20:11:53 -07:00
Ishaan Jaff
a10b3164be docs web search 2025-03-22 19:02:39 -07:00
Ishaan Jaff
f21a0c2da7 Merge branch 'main' into litellm_exp_mcp_server 2025-03-22 18:51:25 -07:00
Krrish Dholakia
f68e27a0c8 docs(image_handling.md): architecture doc on image handling on the proxy 2025-03-21 14:19:05 -07:00
Ishaan Jaff
145d2d8dd6 doc litellm MCP client 2025-03-21 09:26:03 -07:00
Ishaan Jaff
8089c5e3c0 litellm MCP 2025-03-20 21:35:06 -07:00
Ishaan Jaff
768dc79df9
Merge pull request #9332 from SrilakshmiC/patch-1
docs: Include Phoenix Page into sidebar under observability
2025-03-20 13:35:29 -07:00
Ishaan Jaff
973ad94550 docs prompt management 2025-03-19 14:37:32 -07:00