Edward Samuel Pasaribu
36f160b582
Update openrouter/openai/gpt-5-mini pricing
2025-08-12 18:54:06 +08:00
Edward Samuel Pasaribu
d898f9e0dd
Add openrouter gpt-5 family models pricing
2025-08-12 18:51:56 +08:00
TensorNull
a3bf332697
Merge pull request #3 from cometapi-dev/dev
...
docs: Add CometAPI documentation with authentication, usage examples, and error handling
2025-08-12 17:35:48 +08:00
TensorNull
fa81c20df6
fix: Remove outdated models from the model list in the CometAPI document
2025-08-12 17:28:10 +08:00
TensorNull
8b602f9507
[Feat] - Add CometAPI documentation with authentication, usage examples, and error handling
2025-08-12 17:18:33 +08:00
NULL
4d2b5076a0
Merge branch 'BerriAI:main' into dev
2025-08-12 16:51:25 +08:00
NULL
12aba1352c
Merge branch 'BerriAI:main' into main
2025-08-12 16:51:07 +08:00
Krrish Dholakia
79e262d12b
feat(common_utils.py): make default azure openai responses api use /openai/v1/responses logic
...
Fixes https://github.com/BerriAI/litellm/issues/13527#issuecomment-3177882103
2025-08-11 23:40:05 -07:00
Krrish Dholakia
e1fd49ce91
build(model_prices_and_context_window.json): fix claude-sonnet-4 on openrouter
...
Fixes https://github.com/BerriAI/litellm/issues/13520
2025-08-11 23:26:06 -07:00
Krrish Dholakia
2aacf64db1
test: add unit tests
2025-08-11 22:58:58 -07:00
Krrish Dholakia
72c7e82ef8
fix(azure/common_utils.py): generify api version logic
2025-08-11 22:54:44 -07:00
Krrish Dholakia
ca642d32e1
fix(azure/common_utils.py): add default api version for openai responses api calls
2025-08-11 22:53:45 -07:00
Ishaan Jaff
008ea864a7
[Feat] - Add key/team logging for Langfuse OTEL Logger ( #13512 )
...
* feat - add key/team logging for LF
* test_construct_dynamic_otel_headers_with_langfuse_keys
* update LangfuseOtelLogger
* test_construct_dynamic_otel_headers_with_langfuse_keys
* cleanup
* OpenTelemetryConfig fixes
* fix code qa checks
* TestLangfuseOtelIntegration
2025-08-11 22:06:25 -07:00
Krrish Dholakia
bc9d0484e4
fix(cooldown_cache.py): mask error string to avoid leaking sensitive prompt data
...
Fixes https://github.com/BerriAI/litellm/issues/13329
2025-08-11 18:37:57 -07:00
Krrish Dholakia
dab3acdd26
fix(handle_error.py): add unit tests
2025-08-11 18:25:30 -07:00
Krrish Dholakia
56241e937a
fix(handle_error.py): don't return backend exception to user - can contain prompt leakage
...
Fixes https://github.com/BerriAI/litellm/issues/13329
2025-08-11 18:19:47 -07:00
tanjiro
d5135bba31
Allow unsetting TPM and RPM - Teams Settings ( #13430 )
...
* modified tpm rpm for 0 and null values
* show unlimited if tpm/rpm 0 or null
* bring back unlimited
2025-08-11 18:11:43 -07:00
NULL
f3dcae2528
feat: add CometAPI provider support with chat completions and streaming ( #13458 )
...
* feat: add CometAPI support with config, error handling and tests
* fix: specify type for extra_body in CometAPIConfig
---------
Signed-off-by: NULL <129579691+TensorNull@users.noreply.github.com>
2025-08-11 18:06:37 -07:00
breno-aumo
7484a19edf
Fix OCI streaming ( #13437 )
...
* add oci models to model_prices_and_context_window.json
* remove unsupported and unavailable oci models from docs
* fix: split data chunks when using async streaming
* add oci models to model_prices_and_context_window.json
* fix: split data chunks when using async streaming
* revert model.json modifications
2025-08-11 18:04:58 -07:00
Ishaan Jaff
49ca4dc792
Sending tags to LiteLLM Proxy ( #13517 )
2025-08-11 17:58:50 -07:00
Krrish Dholakia
21e067e47f
fix: fix check
2025-08-11 17:56:19 -07:00
Krrish Dholakia
92ebf5b918
fix(router.py): fix print statement
2025-08-11 17:46:14 -07:00
Jugal D. Bhatt
5efe59e6e2
[Draft] [LLM Translation] Add model id check ( #13507 )
...
* Add model id check
* add ruff fix
2025-08-11 15:41:08 -07:00
Ishaan Jaff
9f78287000
[Bug Fix]: Azure OpenAI GPT-5 max_tokens + reasoning param support ( #13510 )
...
* add AzureOpenAIGPT5Config
* add AzureOpenAIGPT5Config
* add AzureOpenAIGPT5Config
* add AzureOpenAIGPT5Config
* test_azure_gpt5_supports_reasoning_effort
* test_azure_gpt5_reasoning
* test_azure_gpt5_reasoning
* ruff check fixes
* docs azure gpt5
2025-08-11 15:40:53 -07:00
Timothy Lowrimore
cc3ec33fe9
removes unused imports that are causing linting failures
2025-08-11 09:52:20 -06:00
Timothy Lowrimore
95d9e30448
Merge branch 'main' into heroku-llms
2025-08-11 09:45:52 -06:00
Timothy Lowrimore
51b52534fb
fixes misconfigured OCI models by setting supports_tool_choice: true
2025-08-11 09:40:40 -06:00
Cole McIntosh
8197fd74d5
Revert "merge from upstream"
...
This reverts commit e6ca91869a
2025-08-11 07:54:47 -06:00
Cole McIntosh
e6ca91869a
merge from upstream
2025-08-11 07:38:16 -06:00
Cole McIntosh
d5d7e00d34
Enhance chunk parsing for Ollama streaming responses
...
Updated the chunk_parser method to return a ModelResponseStream when handling 'thinking' field content, allowing UIs to render reasoning information. Adjusted tests to verify the new behavior, ensuring that reasoning content is correctly returned in the response.
2025-08-11 07:02:25 -06:00
Cole McIntosh
7a1c6efa0a
Merge branch 'BerriAI:main' into fix/ollama-gpt-oss-thinking-field
2025-08-11 06:54:51 -06:00
Ishaan Jaff
1cd827874f
[Bug Fix] - Allow using reasoning_effort for gpt-5 model family and reasoning for Responses API ( #13475 )
...
* test_openai_gpt5_reasoning
* test_openai_gpt5_reasoning_effort_parameter
* add OpenAIGPT5ResponsesAPIConfig
* test_openai_gpt5_reasoning_effort_parameter
* fixes
2025-08-10 09:55:36 -07:00
Krrish Dholakia
bd8a0ae0d0
docs: fix order
2025-08-10 09:42:55 -07:00
Krrish Dholakia
1dbac75675
docs(index.md): update release with deployment information
2025-08-10 09:31:28 -07:00
Krish Dholakia
0aeb4f1653
fix(health_check_helpers.py): set max tokens for wildcard call to 10, fixes calling gpt-5-nano via wildcard on openai ( #13482 )
...
gpt-5-nano raises errors for max_tokens=1
2025-08-10 09:23:36 -07:00
Krish Dholakia
184687157e
Litellm model cost map fixes ( #13480 )
...
* build(model_prices_and_context_window.json): fix max token values
* build(model_prices_and_context_window.json): fix max token values
* build(model_prices_and_context_window.json): fix azure gpt-5-chat pricing
2025-08-10 07:38:35 -07:00
Krish Dholakia
c742c76288
Litellm release notes 08 10 2025 ( #13479 )
...
* docs(index.md): initial doc
* build(index.md): initial notes
* docs(index.md): add llm translation tickets
* docs(index.md): document new model support
* docs(index.md): document all pricing changes
* docs(index.md): add llm api endpoints
* docs(index.md): add doc on mcp gateway
* docs(index.md): add all remaining rc notes
* docs(index.md): cleanup
2025-08-10 07:32:11 -07:00
Krrish Dholakia
ece2c9c65d
bump: version 1.75.4 → 1.75.5
2025-08-09 16:31:51 -07:00
Krrish Dholakia
0eedf7c447
build: update local model cost map
2025-08-09 16:31:41 -07:00
Krish Dholakia
9f6f96d76c
Litellm dev 08 07 2025 p1 ( #13418 )
...
* fix(router.py): support base model for model group usage
allows model group info to show accurate cost information for azure models
* fix(router.py): fix changes
* test: add unit tests
* build(pyproject.toml): bump openai version requirements
support custom tool from responses api
Closes https://github.com/BerriAI/litellm/issues/13391
* docs(responses_api.md): add verbosity + free-form function calling parameters
* docs(responses_api.md): add cfg + minimal reasoning to docs
Closes https://github.com/BerriAI/litellm/issues/13391
* docs(responses_api.md): add proxy examples to docs
* refactor: fix ruff error
2025-08-09 16:30:04 -07:00
Sannan Nasir
0e53b1feab
Add digitalocean provider ( #12169 )
...
* Add digitalocean provider
* Add digitalocean provider
* Revert "Add digitalocean provider"
This reverts commit 96dda40f45b3d12ea03e861d060ec81460b7759e.
* changes
* fixes
* Update transformation
* refactoring
* rename provider to Gradient AI
* fixes
* Incorporte review comments
* revert changes
* fix typo
* revert change
* incorporated review comments
* Revert "Incorporte review comments"
This reverts commit 37bd51bd54ef4fd52ccc12866e47f8de9476d597.
* changes
* Revert "Revert "Incorporte review comments"
This reverts commit 37bd51bd54ef4fd52ccc12866e47f8de9476d597."
This reverts commit 68c8a198ee0d6441c3a52f6c6a49c9c95a4cb0a8.
* changes
* fixes
* Update provider_specific_fields.tsx
2025-08-09 16:26:33 -07:00
Ishaan Jaff
f60a9cf908
[Bug]: Fix JWTs access not working with model groups ( #13474 )
...
* fix can_team_access_model
* test_find_team_with_model_access_model_group
2025-08-09 16:14:51 -07:00
Jugal D. Bhatt
95fbe59c46
Add local storage auth ( #13473 )
2025-08-09 16:13:56 -07:00
Jugal D. Bhatt
67833590d6
[Proxy changes] Litellm add model price reload schedule for multi-pod ( #13470 )
...
* added mcp guardrails doc in mcp.md
* add button to reload models
* Added button changes
* added button for scheduling reload
* add multi pod support to reloading the model price json
* fix ruff
2025-08-09 16:12:13 -07:00
Krish Dholakia
1c8761111f
Router - reduce p99 latency w/ redis enabled by 50% + OTEL - track pre_call hook latency ( #13362 )
...
* feat(proxy/utils.py): track pre-call hooks in OTEL
some pre call hooks can cause latency in high traffic - make sure this is tracked
* fix(router.py): move redis call on deployment_callback_on_success to pipeline operation
reduces p99 latency by half when redis is enabled
* fix(parallel_request_limiter_v3.py): only run check if any item has rate limits set
Prevents unnecessary latency added by rate limit checks
* test: add unit tests
* Latency Improvements: only track tpm/rpm usage when set on deployment+ LLM Caching - use an in-memory cache to reduce redis calls + OTEL - track time spent on LLM caching (#13472 )
* fix(router.py): only track usage for deployments with tpm/rpm set
ensures additional latency avoided for non-tpm/rpm models
* fix(caching_handler.py): log time spent on request get cache to OTEL
enables easy debugging of call latency
* fix(caching_handler.py): use dual cache object for in-memory caching + trace redis call within caching handler
* fix(caching_handler.py): working in-memory cache for redis calls
ensures dual cache works when redis cache setup for llm calls
makes calls quicker by only checking redis when in-memory cache missed for llm api call
* test: remove redundant test
* test: add unit tests
2025-08-09 16:09:51 -07:00
Ishaan Jaff
60306d34a0
[Bug Fix] Allow using Swagger for /chat/completions ( #13469 )
...
* fix get_openapi_schema
* fixes for ProxyChatCompletionRequest
* TestSwaggerChatCompletions
* fix working request body
* fix - add "messages"
* fix messages
* TestSwaggerChatCompletions
* test_messages_field_has_example
* ruff check fix
2025-08-09 15:35:45 -07:00
Jugal D. Bhatt
1270df08a4
[Proxy + UI] Litellm add reload model api and button ( #13464 )
...
* added mcp guardrails doc in mcp.md
* add button to reload models
* Added button changes
* remove the model_reload
2025-08-09 13:52:56 -07:00
Jugal D. Bhatt
10a1fe21c5
[LLM Translation] Litellm azure o series drop params ( #13353 )
...
* added route check
* fix ruff
* Added support for dropping o_series params
* Added ruff fix
* fix tests
2025-08-09 13:52:45 -07:00
Ishaan Jaff
6184e898b7
Generate unique IDs for litellm_call_id and function_id using UUID ( #13468 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-08-09 12:59:09 -07:00
Ishaan Jaff
eb4bd26f24
[Bug Fix] - Get Routes ( #13466 )
...
* fixes get_routes_for_mounted_app
* fix - use _safe_get_endpoint_name
* fix code QA check
* test_get_routes_for_mounted_app_with_static_files
* test fixes
2025-08-09 12:52:23 -07:00
