Commit Graph

30881 Commits

Author SHA1 Message Date
Юрій
b8445436cd
Adjust model pricing metadata (#8147)
* added cache_read_input_token_cost

* added input price for audio whisper

* fixed prices for google gemini-1.0-pro-vision

---------

Co-authored-by: yurii.izo <yurii.izo@justanswer.com>
2025-01-31 21:12:05 -08:00
Ishaan Jaff
ab939d881f bump: version 1.59.12 → 1.60.0 2025-01-31 21:07:42 -08:00
Ishaan Jaff
a74ecf5dbc new release 2025-01-31 21:07:42 -08:00
Yigit Konur
ed53b82e5d
Adding Azure OpenAI o3-mini costs & specs (#8166) 2025-01-31 21:05:18 -08:00
Ishaan Jaff
9ff27809b2
(Feat) add bedrock/deepseek custom import models (#8132)
* add support for using llama spec with bedrock

* fix get_bedrock_invoke_provider

* add support for using bedrock provider in mappings

* working request

* test_bedrock_custom_deepseek

* test_bedrock_custom_deepseek

* fix _get_model_id_for_llama_like_model

* test_bedrock_custom_deepseek

* doc DeepSeek-R1-Distill-Llama-70B

* test_bedrock_custom_deepseek
2025-01-31 18:40:44 -08:00
Ishaan Jaff
29a8a613a7
(UI) Allow adding custom pricing when adding new model (#8165)
* ui custom pricing

* fix validateJSON

* ui allow entering custom pricing

* ui fix add model form

* working edit model modal

* working edit custom pricing

* ui add custom pricing on ui
2025-01-31 17:54:29 -08:00
Ishaan Jaff
2cf0daa31c
(Fixes) OpenAI Streaming Token Counting + Fixes usage track when litellm.turn_off_message_logging=True (#8156)
* working streaming usage tracking

* fix test_async_chat_openai_stream_options

* fix await asyncio.sleep(1)

* test_async_chat_azure

* fix s3 logging

* fix get_stream_options

* fix get_stream_options

* fix streaming handler

* test_stream_token_counting_with_redaction

* fix codeql concern
2025-01-31 15:06:37 -08:00
Krish Dholakia
9f0f2b3f01
build: fix model cost map (#8153) 2025-01-31 13:30:06 -08:00
Ventz Petkov
9678bc3140
FEATURE: OpenAI o3-mini (#8151)
Co-authored-by: Ventz Petkov <ventz.petkov@cerbo.io>
2025-01-31 13:08:02 -08:00
Krrish Dholakia
4b6fc4bba3 docs: fix dead links 2025-01-31 10:09:49 -08:00
Krish Dholakia
a699000a4b
New stable release - release notes (#8148)
* docs(v1.59.8-stable): add release note

* docs(index.md): cleanup new stable release, release notes
2025-01-31 10:02:59 -08:00
Krrish Dholakia
b8b927f23b bump: version 1.59.11 → 1.59.12 2025-01-30 23:07:33 -08:00
Krish Dholakia
de261e2120
Doc updates + management endpoint fixes (#8138)
* Litellm dev 01 29 2025 p4 (#8107)

* fix(key_management_endpoints.py): always get db team

Fixes https://github.com/BerriAI/litellm/issues/7983

* test(test_key_management.py): add unit test enforcing check_db_only is always true on key generate checks

* test: fix test

* test: skip gemini thinking

* Litellm dev 01 29 2025 p3 (#8106)

* fix(__init__.py): reduces size of __init__.py and reduces scope for errors by using correct param

* refactor(__init__.py): refactor init by cleaning up redundant params

* refactor(__init__.py): move more constants into constants.py

cleanup root

* refactor(__init__.py): more cleanup

* feat(__init__.py): expose new 'disable_hf_tokenizer_download' param

enables hf model usage in offline env

* docs(config_settings.md): document new disable_hf_tokenizer_download param

* fix: fix linting error

* fix: fix unsafe comparison

* test: fix test

* docs(public_teams.md): add doc showing how to expose public teams for users to join

* docs: add beta disclaimer on public teams

* test: update tests
2025-01-30 22:56:41 -08:00
Krish Dholakia
69a6da4727
Litellm dev 01 30 2025 p2 (#8134)
* feat(lowest_tpm_rpm_v2.py): fix redis cache check to use >= instead of >

makes it consistent

* test(test_custom_guardrails.py): add more unit testing on default on guardrails

ensure it runs if user sent guardrail list is empty

* docs(quick_start.md): clarify default on guardrails run even if user guardrails list contains other guardrails

* refactor(litellm_logging.py): refactor no-log to helper util

allows for more consistent behavior

* feat(litellm_logging.py): add event hook to verbose logs

* fix(litellm_logging.py): add unit testing to ensure `litellm.disable_no_log_param` is respected

* docs(logging.md): document how to disable 'no-log' param

* test: fix test to handle feb

* test: cleanup old bedrock model

* fix: fix router check
2025-01-30 22:18:53 -08:00
Krish Dholakia
41407f7be1
Doc updates - add key rotations to docs (#8136)
* docs(virtual_keys.md): add key rotations to virtual keys doc

* docs(enterprise.md): add key rotations to enterprise docs
2025-01-30 22:17:00 -08:00
Ishaan Jaff
da3ddd2282 fix test_generate_and_update_key 2025-01-30 21:34:33 -08:00
Ishaan Jaff
d2f087c93e bump: version 1.59.10 → 1.59.11 2025-01-30 21:19:58 -08:00
Ishaan Jaff
dad6c8f09c ui new build 2025-01-30 21:19:39 -08:00
miraclebakelaser
527e283c51
fix: propagating json_mode to acompletion (#8133) 2025-01-30 21:17:26 -08:00
Ishaan Jaff
4005a51db2
(UI) fix adding Vertex Models (#8129)
* fix handleSubmit

* update handleAddModelSubmit

* add jest testing for ui

* add step for running ui unit tests

* add validate json step to add model

* ui jest testing fixes

* update package lock

* ci/cd run again

* fix antd import

* run jest tests first

* fix antd install

* fix ui unit tests

* fix unit test ui
2025-01-30 21:11:08 -08:00
Ishaan Jaff
8a235e7d38
(Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added (#8112)
* LoggingCallbackManager

* add logging_callback_manager

* use logging_callback_manager

* add add_litellm_failure_callback

* use add_litellm_callback

* use add_litellm_async_success_callback

* add_litellm_async_failure_callback

* linting fix

* fix logging callback manager

* test_duplicate_multiple_loggers_test

* use _reset_all_callbacks

* fix testing with dup callbacks

* test_basic_image_generation

* reset callbacks for tests

* fix check for _add_custom_logger_to_list

* fix test_amazing_sync_embedding

* fix _get_custom_logger_key

* fix batches testing

* fix _reset_all_callbacks

* fix _check_callback_list_size

* add callback_manager_test

* fix test gemini-2.0-flash-thinking-exp-01-21
2025-01-30 19:35:50 -08:00
Ishaan Jaff
3eac1634fa
[Fix Ui ]: Taha-hexcode (#8082) (#8122)
* [Bug] UI: Newly created key does not display on the View Key Page (#8039)

- Fixed issue where all keys appeared blank for admin users.
- Implemented filtering of data via team settings to ensure all keys are displayed correctly.

* Fix:
- Updated the validator to allow model editing when `keyTeam.team_alias === "Default Team"`.
- Ensured other teams still follow the original validation rules.

Co-authored-by: Taha Ali <123803932+tahaali-dev@users.noreply.github.com>
2025-01-30 15:00:13 -08:00
Ishaan Jaff
89d0d893fd fix test gemini-2.0-flash-thinking-exp-01-21 2025-01-30 14:05:59 -08:00
Ishaan Jaff
c501ec9b6d
(UI) Fix SpendLogs page - truncate bedrock models + show end_user (#8118)
* ui spend logs table truncate bedrock page

* ui - show user / internal user fields
2025-01-30 13:59:13 -08:00
miraclebakelaser
217776ed23
add groq/deepseek-r1-distill-llama-70b (#8078) 2025-01-30 11:13:00 -08:00
Marco Aleixo
aa9591cb41
adding new model (#8089) 2025-01-30 11:09:28 -08:00
Ishaan Jaff
395e22ea72 fix code qa check 2025-01-30 07:13:18 -08:00
Krish Dholakia
ba8ba9eddb
feat(databricks/chat/transformation.py): add tools and 'tool_choice' param support (#8076)
* feat(databricks/chat/transformation.py): add tools and 'tool_choice' param support

Closes https://github.com/BerriAI/litellm/issues/7788

* refactor: cleanup redundant file

* test: mark flaky test

* test: mark all parallel request tests as flaky
2025-01-29 21:09:07 -08:00
Krrish Dholakia
9fa44a4fbe docs(bedrock.md): update docs to show how to use converse like route for internal proxy usage
Resolves https://github.com/BerriAI/litellm/issues/8085
2025-01-29 21:00:45 -08:00
Ishaan Jaff
12a84897cf run ci/cd again 2025-01-29 20:55:49 -08:00
Krish Dholakia
dad24f2b52
Litellm dev 01 29 2025 p2 (#8102)
* docs: cleanup doc

* feat(bedrock/): initial commit adding bedrock/converse_like/<model> route support

allows routing to a converse like endpoint

Resolves https://github.com/BerriAI/litellm/issues/8085

* feat(bedrock/chat/converse_transformation.py): make converse config base config compatible

enables new 'converse_like' route

* feat(converse_transformation.py): enables using the proxy with converse like api endpoint

Resolves https://github.com/BerriAI/litellm/issues/8085
2025-01-29 20:53:37 -08:00
Krish Dholakia
a57fad1e29
Litellm dev 01 29 2025 p1 (#8097)
* Added deprecation date to model_prices_and_context_window.json (#8061)

* build(model_prices_and_context_window.json): add xai model pricing

---------

Co-authored-by: Юрій <yurikizio@gmail.com>
2025-01-29 19:50:40 -08:00
Ishaan Jaff
d10bd67d33
ui chat ui fixes (#8105) 2025-01-29 18:53:49 -08:00
Ishaan Jaff
31e967cbbd test_generate_and_update_key 2025-01-29 18:48:34 -08:00
Ishaan Jaff
233d82b3b6 bump: version 1.59.9 → 1.59.10 2025-01-29 18:28:39 -08:00
Ishaan Jaff
6c2a87552a ui new build 2025-01-29 18:02:30 -08:00
Ishaan Jaff
bce8c74810 ui new build 2025-01-29 18:01:55 -08:00
Ishaan Jaff
33470e15b4 ci/cd run again 2025-01-29 17:57:32 -08:00
Ishaan Jaff
79c54256a0
(UI) Allow using a model / credentials for pass through routes (#8099)
* test_add_vertex_pass_through_deployment

* VertexPassThroughRouter

* fix use_in_pass_through

* VertexPassThroughRouter

* fix vertex_credentials

* allow using _initialize_deployment_for_pass_through

* test_add_vertex_pass_through_deployment

* _set_default_vertex_config

* fix verbose_proxy_logger

* fix use_in_pass_through

* fix _get_token_and_url

* test_get_vertex_location_from_url

* test_get_vertex_credentials_none

* run pt unit testing again

* fix add_vertex_credentials

* test_adding_deployments.py

* rename file

* revert to og add model flow

* move placement

* ui somewhat working wildcard

* separate out code for slecting model name

* fix ConditionalPublicModelName

* ui fix confitional render public model name

* fix litellm model name page

* fix advanced settings page page

* ui fix litellm model name flow

* adding pas through models ui

* fix passing litellm params

* ui fix advanced settings

* ui fix order of provider info
2025-01-29 17:54:20 -08:00
Ishaan Jaff
b6d61ec22b
(Feat) pass through vertex - allow using credentials defined on litellm router for vertex pass through (#8100)
* test_add_vertex_pass_through_deployment

* VertexPassThroughRouter

* fix use_in_pass_through

* VertexPassThroughRouter

* fix vertex_credentials

* allow using _initialize_deployment_for_pass_through

* test_add_vertex_pass_through_deployment

* _set_default_vertex_config

* fix verbose_proxy_logger

* fix use_in_pass_through

* fix _get_token_and_url

* test_get_vertex_location_from_url

* test_get_vertex_credentials_none

* run pt unit testing again

* fix add_vertex_credentials

* test_adding_deployments.py

* rename file
2025-01-29 17:54:02 -08:00
Ishaan Jaff
892581ffc3
(UI) - View Logs Page - Refinement (#8087)
* working refetch interval

* ui show provider logo in SpendLogs Table

* fix padding

* improve time range filter

* ui fix diff minutes

* fix refresh button placement
2025-01-29 08:46:20 -08:00
Ishaan Jaff
46b44f3a7f ci/cd run again 2025-01-28 22:18:03 -08:00
Ishaan Jaff
5a7dc11432 test fix test_async_create_batch use only openai for testing, hitting azure limits 2025-01-28 22:17:49 -08:00
Ishaan Jaff
2141a2e3f1 ui new build 2025-01-28 22:15:08 -08:00
Ishaan Jaff
64a3d08e1a
(UI) Fixes for Adding model page - keep existing page as default, have 2nd tab for wildcard models (#8073)
* use basic / advanced setup

* fix providerKey

* ui tab wildcard models

* ui fix entering api keys
2025-01-28 22:13:04 -08:00
Ishaan Jaff
b812286534
(fix) - proxy reliability, ensure duplicate callbacks are not added to proxy (#8067)
* refactor _add_callbacks_from_db_config

* fix check for _custom_logger_exists_in_litellm_callbacks

* move loc of test utils

* run ci/cd again

* test_add_custom_logger_callback_to_specific_event_with_duplicates_callbacks

* fix _custom_logger_class_exists_in_success_callbacks

* unit testing for test_add_callbacks_from_db_config

* test_custom_logger_exists_in_callbacks_individual_functions

* fix config.yml

* fix test test_stream_chunk_builder_openai_audio_output_usage - use direct dict comparison
2025-01-28 21:01:56 -08:00
Ishaan Jaff
ae7b042bc2
(beta ui - spend logs view fixes & Improvements 1) (#8062)
* ui 1 - show correct msg on no logs

* fix dup country col

* backend - allow filtering by team_id and api_key

* fix ui_view_spend_logs

* ui update query params

* working team id and key hash filters

* fix filter ref - don't hold on them as they are

* fix _model_custom_llm_provider_matches_wildcard_pattern

* fix test test_stream_chunk_builder_openai_audio_output_usage - use direct dict comparison
2025-01-28 20:34:22 -08:00
Ishaan Jaff
311997ee40 ui new build 2025-01-28 18:25:45 -08:00
Ishaan Jaff
727046c5aa ui fix selecting a provider 2025-01-28 18:23:57 -08:00
Krish Dholakia
d9eb8f42ff
Litellm dev 01 27 2025 p3 (#8047)
* docs(reliability.md): add doc on disabling fallbacks per request

* feat(litellm_pre_call_utils.py): support reading request timeout from request headers - new `x-litellm-timeout` param

Allows setting dynamic model timeouts from vercel's AI sdk

* test(test_proxy_server.py): add simple unit test for reading request timeout

* test(test_fallbacks.py): add e2e test to confirm timeout passed in request headers is correctly read

* feat(main.py): support passing metadata to openai in preview

Resolves https://github.com/BerriAI/litellm/issues/6022#issuecomment-2616119371

* fix(main.py): fix passing openai metadata

* docs(request_headers.md): document new request headers

* build: Merge branch 'main' into litellm_dev_01_27_2025_p3

* test: loosen test
2025-01-28 18:01:27 -08:00