Commit Graph

30881 Commits

Author SHA1 Message Date
Thom Lane
5f47df4627
Update instructor.md (#10549)
Simplified examples
2025-05-05 21:15:53 -07:00
Kavinkumar
57c5e7ea3f
Update gemini-2.5-pro-exp-03-25 max_tokens to 65,535 (#10548)
Fixes:
Error generating conversation title: litellm.BadRequestError: VertexAIException BadRequestError - {
  "error": {
    "code": 400,
    "message": "Unable to submit request because it has a maxOutputTokens value of 65536 but the supported range is from 1 (inclusive) to 65536 (exclusive). Update the value and try again.",
    "status": "INVALID_ARGUMENT"
  }
}
2025-05-05 21:15:06 -07:00
Krrish Dholakia
9ae239b3e1 docs(index.md): add graphic 2025-05-05 20:49:47 -07:00
Ishaan Jaff
cd48ffb6c4
[Bug fix] JSON logs - Ensure only 1 log is emitted (previously duplicate json logs were getting emitted) (#10580)
* fix logging with JSON logs

* test: test_initialize_loggers_with_handler_sets_propagate_false
2025-05-05 20:25:47 -07:00
Ishaan Jaff
ddc24da2a7 Revert "fix logging with JSON logs"
This reverts commit 1109a6ea6e.
2025-05-05 18:13:39 -07:00
Ishaan Jaff
1109a6ea6e fix logging with JSON logs 2025-05-05 18:13:15 -07:00
Ishaan Jaff
489f1a6c25
[Feat] v2 Custom Logger API Endpoints (#10575)
* fixes for generic api logger

* tests for generic api logger

* test_generic_api_callback_multiple_logs

* allow health checking generic api endpoints

* docs generic api endpoint for logging

* allow setting headers for generic api  callback

* fix for test_init_custom_logger_compatible_class_as_callback

* fix linting
2025-05-05 16:57:55 -07:00
Young Han
b1e28a3bc4
[Docs] Change llama-api link for litellm (#10556)
* docs: change llama-api link for litellm

* docs: init supported models table
2025-05-05 12:37:37 -07:00
Zoltan K
91dcc50768
Github: Increase timeout of litellm tests (#10568) 2025-05-05 12:37:04 -07:00
Krrish Dholakia
914ab00805 docs(index.md): add key highlights to docs 2025-05-03 22:04:22 -07:00
Ishaan Jaff
bccab23a6a add vector stores to release note 2025-05-03 21:51:03 -07:00
Krrish Dholakia
3f09da21e3 docs(meta_llama.md): clarify meta llama works on responses api 2025-05-03 21:41:00 -07:00
Ishaan Jaff
9801a3a416 ui new build 2025-05-03 21:33:48 -07:00
Krish Dholakia
522e334fbd
fix(converse_transformation.py): handle meta llama tool call response (#10541)
* fix(converse_transformation.py): handle meta llama tool call response

Fixes issue where bedrock meta llama would return tool call response as content str

* test(test_converse_transformation.py): add unit testing for new function

* fix: fix linting error

* fix: fix linting error
2025-05-03 21:31:14 -07:00
Ishaan Jaff
7c4fae477a
[Bug Fix] UnicodeDecodeError: 'charmap' on Windows during litellm import (#10542)
* test using litellm on windows

* fix using litellm on windows

* test using litellm on windows

* fix change on ci/cd

* fix test on ci/cd
2025-05-03 21:31:05 -07:00
Ishaan Jaff
8aa57ff0c0 add meta llma api to release notes 2025-05-03 21:00:22 -07:00
Ishaan Jaff
3f45200802 test_openai_responses_api_web_search_cost_tracking 2025-05-03 20:22:42 -07:00
Krish Dholakia
880c2a736b
fix(model_management_endpoints.py): allow team admin to update model … (#10539)
* fix(model_management_endpoints.py): allow team admin to update model via `/model/{model_id}/update` route

Fixes ui regression where team admin could not modify their own models

* fix(provider_specific_fields.tsx): style fix

* fix(table.tsx): allow expanding multiple rows

* fix(organization_endpoints.py): more robust check if user can give org model access

handle when user has models=["all-proxy-models"]

* fix(organization_endpoints.py): enable proxy admin with 'all-proxy-model' access to create new org with specific models

Fixes LIT-135

* fix: fix linting error

* fix: fix ui linting error

* fix(index.tsx): fix linting errors
2025-05-03 19:34:35 -07:00
Krrish Dholakia
42a91bae6b test: comment out new ui test - needs more investigation 2025-05-03 19:32:36 -07:00
Krrish Dholakia
7581a0e3e0 test: reduce timeout 2025-05-03 19:16:54 -07:00
Krrish Dholakia
3ad108cb63 test: add timeout on keys page 2025-05-03 19:08:04 -07:00
Ishaan Jaff
8faa7dbc39 fixes for web search tests 2025-05-03 18:10:30 -07:00
Ishaan Jaff
f52593486c
[Contributor PR] Support Llama-api as an LLM provider (#10451) (#10538)
* Support Llama-api as an LLM provider (#10451)

* init: support llama-api as a llm provider

* docs: fix endpoint url

* fix: rename meta dir to meta-llama

* docs: add meta-llama info

* fix: mv LlamaAPIConfig under chat directory

* feat: add LlamaAPIConfig in ProviderConfigManager

* fix: provider_config from ProviderConfigManager

* feat: add supports_tool_choice param

* fix: remove optional_params using model_info

* fix: rename meta-llama to meta_llama

* init: test for meta_llama

* fix: model names

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* fix file naming convention

* fix file naming convention for meta_llama

* docs meta llama api litellm

---------

Co-authored-by: Young Han <110819238+seyeong-han@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-05-03 16:29:03 -07:00
Ishaan Jaff
299f8f18a0 text fix redis cache 2025-05-03 16:28:43 -07:00
Ishaan Jaff
85d843ab4d text fix vertex deprecated a model 2025-05-03 16:22:24 -07:00
Ishaan Jaff
fb9e5dbd6e
add perplexity/sonar-deep-research (#10537) 2025-05-03 15:16:57 -07:00
Krish Dholakia
7ce687ef39
Litellm stable release notes 05 03 2025 (#10536)
* build(release_cycle.md): document bar for minor vs. patch updates

* docs(index.md): initial changelog doc

* docs(index.md): update llama docs

* docs(index.md): add docs for llm api endpoints + spend tracking/budget improvements

* docs: more doc cleanup

* docs(index.md): more doc cleanup

* docs(index.md): final doc cleanup
2025-05-03 12:47:11 -07:00
Ishaan Jaff
a9ee95e0cf fix linting error 2025-05-03 12:09:59 -07:00
Ishaan Jaff
14646bdbc6 ui new build 2025-05-03 12:08:27 -07:00
Ishaan Jaff
ee8aa23c19 docs VectorStoreRegistry 2025-05-03 12:05:49 -07:00
Ishaan Jaff
42ea608b4b docs add section on bedrock provider to show how to use KBs 2025-05-03 11:55:16 -07:00
Ishaan Jaff
a37e4cb0b8
fix typo (#10535) 2025-05-03 11:39:40 -07:00
Ishaan Jaff
071c8a2e26
[Docs] Using LiteLLM with vector stores / knowledge bases (#10534)
* docs vector store registry

* img for kbs

* docs kb with litellm

* fix code qa
2025-05-03 11:28:33 -07:00
Krrish Dholakia
0e91f31908 build: update version + new ui build 2025-05-03 10:53:38 -07:00
Krish Dholakia
552b7e4013
Add customer + model per key level multi-instance tpm/rpm limiting (#10518)
* fix(redis_cache.py): handle multiple event loops

* fix(parallel_request_limiter_v2.py): add customer tpm limiting

* fix(parallel_request_limiter.py): add customer rpm limiting

* fix(parallel_request_limiter_v2.py): add model per key + customer tpm/rpm limiting

* fix(parallel_request_limiter_v2.py): make error more informative

* fix: fix ruff error

* fix: generate new poetry lock
2025-05-03 10:28:55 -07:00
user202729
2138170861
Update contributing-to-documentation documentation to include pnpm guide (#10523) 2025-05-03 10:22:50 -07:00
Takashi Iwamoto
1969e351fe
Add Amazon Nova Premier (#10520) 2025-05-03 10:14:39 -07:00
Ishaan Jaff
931c0c760c fix test_anthropic_messages_litellm_router_streaming_with_logging 2025-05-03 09:34:32 -07:00
Ishaan Jaff
9cf2a95640 bump litellm proxy extras 2025-05-03 09:30:10 -07:00
tanjiro
f858a7d670
fix edit credentials modal (#10519) 2025-05-03 08:53:25 -07:00
Krish Dholakia
7273bb442a
UI - allow reassigning team to other org (#10527)
* feat(team_info.tsx): allow user to reassign team to another org

* style(team_info.tsx): fix org id styling

* feat(team_endpoints.py): add validation check before migrating team to another org

ensure model access, budgets and membership is respected

* fix(team_endpoints.py): update model migration validation to check if org has 'all-proxy-models' access

* fix(organization_view.tsx): show teams belonging to org

* feat(team_endpoints.py): handle wildcard model check on org migration

* fix(team_endpoints.py): nest router check

* test: update testing - use model with higher quota

* build: update poetry lock
2025-05-03 08:44:43 -07:00
Ishaan Jaff
988e20aa36
[QA] Bedrock Vector Stores Integration - Allow using with registry + in OpenAI API spec with tools (#10516)
* refactor KB implementation to use central registry

* allow passing tools when making KB calls

* test fixes

* linting fix

* fix kb tests

* QA for KB stored in DB

* fix, use litellm_credential_name when adding KB on litellm UI

* QA list endpoint vector stores

* allow using UI creds with KBs
2025-05-03 08:30:38 -07:00
Krish Dholakia
6d5f396b59
Playwright test for team admin (#10366) (#10470)
* Playwright test for team admin (#10366)

* Create team_admin.spec.ts

* add todo

* modify test and timeout (#10525)

---------

Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
2025-05-03 08:20:00 -07:00
Krish Dholakia
b412d70232
UI - fix adding vertex models with reusable credentials + fix pagination on keys table + fix showing org budgets on table (#10528)
* feat(provider_specific_fields.tsx): Use correct form instance

Fixes https://github.com/BerriAI/litellm/issues/10115

* Fix broken pagination by correctly passing page and pageSize to keyListCall (#10498)

* [Organization] Include litellm_budget_table in /organization/list response (#10488)

* join litellm_budget_table in /organization/list endpoint

* update test

---------

Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
2025-05-03 08:16:53 -07:00
Marc Abramowitz
2f3f076079
Fix pytest event loop warning (#10512)
Fixes: GH-9641

This is a Cursor-generated fix for the following warning from pytest:

```
litellm/caching/llm_caching_handler.py:17
  /Users/abramowi/Code/OpenSource/litellm/litellm/caching/llm_caching_handler.py:17: DeprecationWarning: There is no current event loop
    event_loop = asyncio.get_event_loop()
```

Cursor prompt:

  Fix this pytest warning
  ```
  litellm/caching/llm_caching_handler.py:17
    /Users/abramowi/Code/OpenSource/litellm/litellm/caching/llm_caching_handler.py:17: DeprecationWarning: There is no current event loop
      event_loop = asyncio.get_event_loop()
  ```

Fixes https://github.com/BerriAI/litellm/issues/9641
2025-05-02 19:43:18 -07:00
Ishaan Jaff
e013d7e529
[Feat] Show Vector Store / KB Request on LiteLLM Logs Page (#10514)
* ensure vector store results are logged in SLP

* fix tests

* fix tests with vector_store_request_metadata

* fix linting

* track duration of vector store, only log content when user opts into it

* working vector store viewer

* fix custom llm provider - Vector Store Requests

* fix vector store viewer

* fix logging redacted vector searches

* testing for storing KB queries in DB
2025-05-02 16:14:09 -07:00
Elijas Dapšauskas
0db5c991d4
fix: PydanticDeprecatedSince20: Support for class-based config is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. (#9372) 2025-05-02 16:10:57 -07:00
Ishaan Jaff
a816d64bcf
Don't depend on uvloop on windows (#10060) (#10483)
Co-authored-by: Dave Cunningham <sparkprime@gmail.com>
2025-05-02 16:09:27 -07:00
Ishaan Jaff
28cb7cc0ed
[Feat] KnowledgeBase/Vector Store - Log StandardLoggingVectorStoreRequest for requests made when a vector store is used (#10509)
* ensure vector store results are logged in SLP

* fix tests

* fix tests with vector_store_request_metadata

* fix linting
2025-05-02 13:43:20 -07:00
Krrish Dholakia
2791b1be1a test: bump limit on flaky load test 2025-05-02 12:10:22 -07:00