Commit Graph

21977 Commits

Author SHA1 Message Date
Ishaan Jaff
f2dc261efc
[Feat] UI: Add support for Deleting Guardrail on UI (#10869)
* show id on guardrails UI

* feat: show guardrail_id on UI

* guardrailToDelete

* ui - add DELETE guardrail

* Update ui/litellm-dashboard/src/components/guardrails.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update ui/litellm-dashboard/src/components/guardrails.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix ui linting

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-15 15:38:18 -07:00
Ishaan Jaff
263f447795
[CI/CD] Test: fix mapped tests (#10870)
* test: fix mapped tests

* test: fix mapped tests

* test: fix mapped tests

* test: fix mapped tests
2025-05-15 15:36:15 -07:00
Ishaan Jaff
3d37cc68f6
[Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail (#10867)
* refactor: add types for mask pii request/response

* feat: add mask pii hook for guardrails

* feat: rename to apply_guardrail

* test: test_presidio_apply_guardrail

* fix: code QA

* fix: linting error

* feat: allow non-admins to access apply_guardrail routes
2025-05-15 14:52:24 -07:00
Ishaan Jaff
86a9b9902e
refactor: fix send_management_endpoint_alert (#10863) 2025-05-15 10:35:27 -07:00
Ishaan Jaff
faed9860c0
[Refactor] Move enterprise_routes within litellm_enterprise (#10860)
* fix: move enterprise routes to litellm_enterprise

* refactor: move enterprise routes to litellm_enterprise

* fix: litellm_enterprise routes

* fix test litellm on github workflow
2025-05-15 10:34:26 -07:00
Ishaan Jaff
42e6e664b2
[Refactor] Make Pagerduty a free feature (#10857)
* refactor: make pagerduty free

* refactor: make pagerduty free

* fix: pagerduty loc

* fix: linting error
2025-05-15 10:12:06 -07:00
dependabot[bot]
1e08816489
build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website (#10855)
Bumps [undici](https://github.com/nodejs/undici) from 6.21.1 to 6.21.3.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v6.21.1...v6.21.3)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.21.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-15 08:33:12 -07:00
Jugal D. Bhatt
16280d9fa9
added fix to resolve delete API (#10845)
* added fix to resolve delete API

* add get requests to ignore model and add test cases
2025-05-15 08:31:37 -07:00
Anibal Angulo
0434ca1781
Updated bedrock invoke transform for mistral to support pixtral models (#10439) 2025-05-14 23:08:37 -07:00
user202729
044f5f973b
Add documentation on in-memory and disk caching (#10522) 2025-05-14 23:07:13 -07:00
Marc Abramowitz
322b67833b
Add models import command (#10581)
* Add --only-models-matching-regex option

to `models import` which only processes models where
`litelllm_params.model` matches the regex

* Add test_models_import_only_models_matching_regex

* Print each model we're importing

* Add --only-access-groups-matching-regex option

to `models import` which only processes models where at least one item
in `model_info.access_groups` matches the regex. Add a unit test.

* Add `models import` examples to README.md

Add `models import` examples to proxy/client/cli/README.md

* ruff format litellm/proxy/client/cli/commands/models.py

* Make `models import` display tabular output

* models import refactoring

* Fix failing tests in test_models_commands.py

* Refactor import_models to make it shorter and more readable

* Extract from `import_models` a function called `get_model_list_from_yaml_file`

* Fix mypy error

* Add more specific typing

for better understandability and Intellisense

* More import_models refactoring

* More refactoring

* More refactoring

* Write unit tests for format_iso_datetime_str

* Add more unit tests

* ruff format tests/litellm/proxy/client/cli/test_models_commands.py

* ruff format litellm/proxy/client/cli/commands/models.py

* Make test_format_timestamp use UTC time
2025-05-14 23:05:07 -07:00
Camille Farineau
b88e56ebde
Fix/issue 10113 embeddings use non default tokenizer (#10629)
* fix(embeddings): use non default tokenizer when passing list of lists of tokens (int)

* feat(embeddings): allow for passthrough of list of lists of tokens to hosted_vllm models

* Revert "fix(embeddings): use non default tokenizer when passing list of lists of tokens (int)"

This reverts commit a48acd95f860c4fc85853e20668eabffff07cae7.

* refactor(embeddings): use a list to verify if provider accept as input a list of tokens

* fix(embeddings): verify the model name before validating if provider accept a arrays of tokens as input

When passing a list of tokens as input, verify the provider of the model by going through the list of models (`llm_model_list`). First, it check for model name then get the provider and verify if it accept or not arrays of tokens. If yes, then pass, else decode.
Previously, it was verifying provider and model name at the same time resulting in decoding even if the current model checked was not the target one (looping onto `llm_model_list`)

* test(embedding): add unit test to bypass decode for some providers with input as array of tokens

Ref: https://github.com/BerriAI/litellm/issues/10113
2025-05-14 22:55:49 -07:00
Pac
41374bfa46
fix: bedrock-sonnet-caching-cost (#10752) 2025-05-14 22:50:45 -07:00
Iguchi Tomokatsu
d8a5bd9e0f
AttributeError: 'Message' object has no attribute 'audio' (#10814) 2025-05-14 22:45:14 -07:00
Krish Dholakia
5146b2903f
Anthropic unified web search + tool cost tracking support (#10846)
* fix(duration_parser.py): support `mo` unit

* test(test_key_management_endpoints.py): add test confirming generate_key_helper_fn uses predictable budgets

Closes https://github.com/BerriAI/litellm/issues/10800

* fix(anthropic/chat/transformation.py): add tool use cost tracking

* fix(anthropic/): refactor how hosted tool usage tracking is done

keep it separate from prompt / completion token details

* fix(anthropic/): add web search tool cost tracking

accurate cost tracking

* feat(anthropic/chat/transformation.py): map openai 'web_search_options' param to anthropic hosted tool

Allows calling anthropic web search in same format as openai

* feat(anthropic/chat/transformation.py): support unified anthropic 'web_search_options' param

Allows calling anthropic's web search tool in the openai format

* feat(anthropic/chat/transformation.py): map openai 'search_context_size' to anthropic 'max_uses' param

Translate search effort across both providers

* fix: mark web_search_options param as supported by openai + azure

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting error

* fix: check if usage hasattr

* fix: pass web search options param
2025-05-14 22:41:12 -07:00
Dima-Mediator
11740ce144
Capture Gemini reasoning tokens usage in streaming mode - fix (#10789) 2025-05-14 22:23:54 -07:00
Ishaan Jaff
75e4d27885 test: test_embedding_performance 2025-05-14 21:31:07 -07:00
Jugal D. Bhatt
ca4390a166
fix failing test typo (#10843) 2025-05-14 20:46:52 -07:00
Ishaan Jaff
a5a4453c77 test: fix groq/gemma2-9b-it 2025-05-14 19:24:20 -07:00
Ishaan Jaff
24cae5aca1 groq/llama3-70b-8192 2025-05-14 19:12:33 -07:00
Ishaan Jaff
80c4b981ec fix groq/llama3-8b-8192 2025-05-14 19:02:20 -07:00
Ishaan Jaff
4a0d711bed ui: fix render error on test key page 2025-05-14 18:57:51 -07:00
Ishaan Jaff
2ebd750289 test: test supports tool I 2025-05-14 18:49:27 -07:00
Ishaan Jaff
85601a69e5 fix groq/llama3-70b-8192 2025-05-14 18:39:25 -07:00
Ishaan Jaff
fc8d28f7df docs - DEFAULT_MAX_RECURSE_DEPTH_SENSITIVE_DATA_MASKER 2025-05-14 18:29:31 -07:00
Ishaan Jaff
9e06c42f72 test: fix litellm_mapped_tests 2025-05-14 18:29:01 -07:00
Ishaan Jaff
abe7b71e6a Revert "Revert "[Fix] Allow overriding all constants using a .env variable (#10803)""
This reverts commit cf4b2ebda4.
2025-05-14 18:27:43 -07:00
Jugal D. Bhatt
a754a25828
Feat add retention config (#10815)
* add function to check config flag

* added unit tests

* convert to seconds support

* added in settings.md

* Updated config_settings.md

* remove extra point

* change config var

* resolve conflict
2025-05-14 18:16:25 -07:00
Ishaan Jaff
235ae79037 ui new build 2025-05-14 18:01:48 -07:00
Ishaan Jaff
91f32cd227 ui linting fixes 2025-05-14 18:00:50 -07:00
Ishaan Jaff
be18923250 fix code qa check for OTEL env vars 2025-05-14 17:56:27 -07:00
Ishaan Jaff
19b2d70732 bump litellm_enterprise-0.1.3.tar.gz 2025-05-14 17:55:03 -07:00
Ishaan Jaff
a30c574f9a bump litellm_proxy_extras-0.1.21.tar.gz 2025-05-14 17:53:34 -07:00
Ishaan Jaff
526e6aaaca
patch: validate_team_org_change (#10835)
* fix: validate_team_org_change

* test: test_validate_team_org_change_same_org_id
2025-05-14 17:50:41 -07:00
Ishaan Jaff
2a994d7016
[Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook (#10840)
* feat: add GuardrailProviderSpecificParams

* feat: add add apply_guardrail helper for presidio

* ui cleanup

* fixes pii config on ui

* fixes for adding presidio pii

* refactor: InitializeGuardrails

* feat: init guardrails from DB

* allow running guardrails from test key pg

* fix: running a request with guardrails on UI

* fix: types/guardrails.py

* test: test_presidio_pre_call_hook_with_different_call_types

* test: test_initialize_presidio_guardrail

* test: fix custom guardrail tests
2025-05-14 17:41:33 -07:00
Damien Pontifex
63a4287bff
Supper OTLP standard environment variables for configuration (#10813)
- Environment variables outlined at https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/
- Default to OTLP standard, but fallback to existing so as to not break backwards compatibility
2025-05-14 17:40:28 -07:00
Ishaan Jaff
9ac47cd05a
[UI] Fixes for bulk invite users - add validation for bad inputs (#10836)
* fixes for bulk invite users

* add validation for bulk import
2025-05-14 17:36:30 -07:00
Ishaan Jaff
652b5693cc build: add guardrails table 2025-05-14 14:23:07 -07:00
Ishaan Jaff
b9c3a3835c
[Feat UI] Add Guardrails on LiteLLM UI (#10834)
* feat: add DB Add, Edit, Delete for Guardrails

* feat: endpoints for guardrail management

* add guardrail info helpers

* add presidio logo

* add createGuardrailCall

* basic add guardrails on UI

* fix cleanup form

* feat: working add guardrails

* ui view guardrails

* allow viweing list of guardrails

* fix: show created_at and updated_at for guardrails on UI

* cleanup guardrail view

* use 1 file for guardrail table

* docs adding guardrails

* fix: expose endpoint to show allowed modes, entities, actions

* fix allow adding entity configs for PII

* fixes guardrails

* fix adding guardrails on UI

* feat: add PII_ENTITY_CATEGORIES_MAP

* use categories for pii

* use step based form

* fixes pii configs

* fix pii config

* refactor pii config on ui

* clear form on submit

* add ability to create guardrails

* fix linting error
2025-05-14 14:21:35 -07:00
Ishaan Jaff
ee1557afcd
[Feat] Add endpoints for adding, deleting, editing guardrails in DB (#10833)
* feat: add DB Add, Edit, Delete for Guardrails

* feat: endpoints for guardrail management

* add guardrail info helpers

* add presidio logo

* add createGuardrailCall
2025-05-14 14:19:51 -07:00
naliotopier
007524a972
Update Groq models on model_prices_and_context_window.json (#10077)
* Update Groq models on model_prices_and_context_window.json

Add support for 6 new models; add deprecations dates to 12 model; and update context-windows/max-tokens for 3 models

* Round model costs to 8 decimal places
2025-05-14 13:08:03 -07:00
tanjiro
1fc5320e9f
Team id + Status filter on logs (#10831)
* added team_id filter

* improve hook

* pagination fixed

* fix pagination for team id filter

* minor

* added status filter

* fixed pagination
2025-05-14 13:06:42 -07:00
Jugal D. Bhatt
468b881ec5
add support to parse metadata (#10832)
* add support to parse metadata

* add safe loads

* remove depth checks
2025-05-14 12:03:49 -07:00
Ishaan Jaff
d558c223b3 bump resource_class: xlarge 2025-05-14 08:31:53 -07:00
Ishaan Jaff
98bed9b698 update resource_class for litellm_mapped_tests 2025-05-14 07:43:56 -07:00
Krish Dholakia
340a0453d3
Migrate cohere to /v2/embed - adds support for output_dimensions param (#10809)
* feat(cohere/embed): v2 embed api support

adds output_dimensions param support

* fix(cohere/embed): migrate to v2 embedding

Adds output dimension support

* fix: maintain /v1/embedding compatibility for bedrock cohere

Bedrock cohere is still using /v1 endpoints

* fix: fix linting error

* fix: fix passing extra headers

* test: update tests
2025-05-13 23:06:56 -07:00
Krish Dholakia
a421316e56
fix(litellm_logging.py): log custom headers in requester metadata (#10818)
* fix(litellm_logging.py): log custom headers in requester metadata

allows passing along custom headers from client to logging integration - e.g. `x-correlation-id`

* refactor: move enterprise code out of OSS package

work towards simplified CE version of docker image

* test: update test

* fix: fix linting error
2025-05-13 23:04:37 -07:00
Ishaan Jaff
cf4b2ebda4 Revert "[Fix] Allow overriding all constants using a .env variable (#10803)"
This reverts commit de901bbf249b5db6060b2a18cd9e271544222132.
2025-05-13 21:27:44 -07:00
Ishaan Jaff
741ea6e149 ci/cd: remove n -4 change 2025-05-13 21:21:37 -07:00
Ishaan Jaff
6b6794a926 fix mapped tests 2025-05-13 21:17:41 -07:00