ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
30,881 Commits 9 Branches 7 Tags 935 MiB
12bc66aa5b
Commit Graph

30881 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Krish Dholakia
68caebe119
Merge pull request #15580 from aniketmaurya/patch-1 
Add type hint to function_to_dict and fix typo
 2025-10-16 07:30:29 -07:00
Krish Dholakia
e95a42d0b8
Merge pull request #15583 from TeddyAmkie/fix/entraid-app-roles-jwt-claim-clean 
Fix/entraid app roles jwt claim clean
 2025-10-16 07:29:49 -07:00
Krish Dholakia
af8cf6861d
Merge pull request #15591 from TensorNull/fix/cometapi 
fix(cometapi): improve CometAPI provider support (embeddings, image generation, docs)
 2025-10-16 07:26:59 -07:00
Ariel Fogel
b075cf4a6c PLR-2400: support no persistence in litellm proxy 2025-10-16 17:22:58 +03:00
Krish Dholakia
e314d946e2
Merge pull request #15598 from shinharaguchi/feature/add_jp_haiku4_5_price 
Add JP Cross-Region Inference jp.anthropic.claude-haiku-4-5-20251001
 2025-10-16 07:12:58 -07:00
shinharaguchi
359a095319 Add jp.anthropic.claude-haiku-4-5-20251001 price 2025-10-16 21:52:44 +09:00
TensorNull
074b9954a0 docs(readme): Add CometAPI to supported providers table 2025-10-16 14:18:55 +08:00
TensorNull
e3566faf15 fix(cookbook): Remove the CometAPI key used for testing. 2025-10-16 14:14:58 +08:00
TensorNull
55a6dd3a8b feat(cometapi): Add CometAPI provider support (embeddings, image generation, docs) 
- Add CometAPI embedding and image generation transformations and configs
- Add image cost calculator and export/init files
- Register provider in constants, utils, main (embedding path) and sidebars
- Add CometAPI docs page and cookbook notebook (Colab) for usage examples
 2025-10-16 13:08:14 +08:00
berri-teddy
1b55803c76 test: reduce EntraID app roles tests to essential scenarios 
- Keep only 2 focused tests: single role extraction and no roles claim
- Remove complex fixtures and edge case tests
- Maintain clean, maintainable test coverage
 2025-10-15 18:15:04 -07:00
berri-teddy
6b166e046d fix: correct EntraID app roles JWT claim extraction 
- Fix get_app_roles_from_id_token to use 'app_roles' claim instead of 'roles'
- Add comprehensive unit tests for EntraID app roles functionality
- Prevent regressions in Microsoft EntraID SSO authentication

Resolves issue where EntraID app roles were not being extracted correctly
from JWT tokens, causing authentication failures for users with assigned
app roles in Microsoft EntraID.
 2025-10-15 18:06:03 -07:00
Alexsander Hamir
a17f3a7aa8
Merge pull request #15578 from BerriAI/litellm_remove_list_lookup 
perf(router): optimize model lookups with O(1) data structures
 2025-10-15 17:59:55 -07:00
AlexsanderHamir
56838e2388 test: ensure model_names is a O(1) datastructure 2025-10-15 17:52:44 -07:00
Andreas
f1fbae8409
feat(pricing): Add Gemini 2.5 Flash Image (Nano Banana) in GA (#15557) 
* feat(pricing): Add Gemini 2.5 Flash Image (Nano Banana) in GA

* change mode to image_generation

---------

Co-authored-by: Andreas Feldl <andreas.feldl@netlight.com>
 2025-10-15 17:50:15 -07:00
Ishaan Jaffer
fd83b292a6 test_bedrock_anthropic_prompt_caching 2025-10-15 17:46:44 -07:00
Ishaan Jaffer
8ab720409c bump v 2025-10-15 17:44:29 -07:00
Ishaan Jaff
354e62bde4
[Feat] Add anthropic.claude-haiku-4-5-20251001-v1:0 on Bedrock, VertexAI (#15581) 
* add claude 4-5

* add anthropic.claude-haiku-4-5-20251001-v1:0
 2025-10-15 17:41:09 -07:00
Aniket Maurya
495e6c0a2b
Add type hint to function_to_dict and fix typo 
Updated type hint for function_to_dict and fixed typo in docstring.
 2025-10-16 01:36:32 +01:00
Mubashir Osmani
ad7f4411a3
added claude-haiku-4.5 (#15579) 2025-10-15 17:28:49 -07:00
Alexsander Hamir
4e84937efc
Merge pull request #15575 from BerriAI/litellm_remove_constly_string_operation 
perf(router): optimize string concatenation in hash generation
 2025-10-15 17:26:32 -07:00
Alexsander Hamir
170519057e
Merge pull request #15576 from BerriAI/litellm_remove_deepcopy 
perf(router): use shallow copy instead of deepcopy for model aliases
 2025-10-15 17:26:02 -07:00
Ishaan Jaff
bc26845ec4
[Feat] Native /ocr endpoint support (#15573) 
* [Feat] Add native litellm.ocr() functions (#15567)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* Potential fix for code scanning alert no. 3521: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* [Feat] Add /ocr route on LiteLLM AI Gateway - Adds support for native mistral ocr calling (#15571)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* [Feat] Add Azure AI Mistral OCR Integration  (#15572)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* add azure ai to get_provider_ocr_config

* add AzureAIOCRConfig

* TestAzureAIOCR

* TestAzureAIOCR

* test fixes for azure ai ocr

* fix async OCR transform for Azure

* fix transform_ocr_request

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-10-15 17:20:01 -07:00
AlexsanderHamir
6e4da1da65 perf(router): optimize model lookups with O(1) data structures 
Replace O(n) scans with index map lookups and convert model_names to set. Standardize timing calls.
 2025-10-15 16:43:08 -07:00
AlexsanderHamir
294fb94b18 perf(router): use shallow copy instead of deepcopy for model aliases 
Replace copy.deepcopy() with dict.copy() in _get_all_deployments when
creating model aliases.

Safe because:
1. Only modifies top-level 'model_name' field (isolated by shallow copy)
2. Nested dicts (litellm_params, model_info) are never modified after return
3. When model_alias=None, returns original dict with NO copy, proving
   callers expect nested structures to be read-only
4. Key insight: Code that needs to modify nested structures does deepcopy FIRST.
   This proves the contract is: \"treat returned deployments as read-only for
   nested fields.\" (see line 6894: copy.deepcopy before modifying litellm_params)

Performance: ~10-100x faster than deepcopy on nested dict structures.
Tests: All 114 router unit tests pass.
 2025-10-15 16:25:37 -07:00
AlexsanderHamir
97ed3d01ec test(router): update error message assertion after string concat optimization 
Update test_generate_model_id_with_deployment_model_name to accept the new
error message format that results from the list+join optimization.

The function still correctly rejects None values with a TypeError, but the
error message changed from 'unsupported operand type(s) for +=' to
'expected str instance, NoneType found' due to the implementation change
from string concatenation to list joining.
 2025-10-15 15:58:08 -07:00
AlexsanderHamir
c98a30a4b6 perf(router): optimize string concatenation in hash generation 
Replace string concatenation in loop with list append + join pattern.
This improves time complexity from O(n²) to O(n) and avoids creating
many temporary string objects during hash ID generation.
 2025-10-15 15:44:09 -07:00
AlexsanderHamir
36c9066372 perf(router): optimize model lookups with O(1) index maps and standardize timing 
- Use model_id_to_deployment_index_map and model_name_to_deployment_indices for O(1) lookups in get_model_info, get_deployment_by_model_group_name, and get_model_ids
 2025-10-15 15:12:40 -07:00
Felipe Gare
a87a3d0f95 adding support parameters to model_prices json 2025-10-15 16:06:44 -03:00
Krrish Dholakia
74bfbeb439 fix(proxy/utils.py): support configuring db schema for non-iam auth db url 2025-10-15 08:43:29 -07:00
Krrish Dholakia
0526334d9c docs: fix url 2025-10-15 08:33:28 -07:00
Ishaan Jaff
5b7e2fd778
[Feat] UI - Allow setting Provider Discounts on UI (#15550) 
* fix cost_discount_config

* add CostBreakdown

* fix: set_cost_breakdown

* test_cost_discount_vertex_ai

* docs fix

* docs fix discounts

* docs fix

* docs custom pricing

* docs fix

* fixes for getting cost breakdown in response headers

* test - response headers wth discount

* - feat: allow setting cost tracking settings

* add cost tracking endpoints

* feat add cost_tracking_settings_router

* add validation when setting cost settings

* add CostTrackingSettings

* add page

* refactor

* use simple table

* add ProviderDiscountTable

* fix ui

* save

* fix: discount settings

* fix edits

* add settings

* add HelpLink

* add help link

* docs HelpLink

* docs fix

* fix how it works

* fix layout

* test TestCostTrackingSettings

* fix Dict
 2025-10-14 20:08:18 -07:00
Ishaan Jaff
a6c57cb5bd
[Feat] Cost Tracking - specify a global vendor discount for costs. (#15546) 
* fix cost_discount_config

* add CostBreakdown

* fix: set_cost_breakdown

* test_cost_discount_vertex_ai

* docs fix

* docs fix discounts

* docs fix

* docs custom pricing

* docs fix

* fixes for getting cost breakdown in response headers

* test - response headers wth discount
 2025-10-14 20:07:04 -07:00
AlexsanderHamir
d094a334d9 add: unit test 2025-10-14 16:38:04 -07:00
Krish Dholakia
ac021161c4
Merge pull request #15540 from FelipeRodriguesGare/fix/gpt-5-codex-params 
change gpt-5-codex support in model_price json
 2025-10-14 16:27:41 -07:00
Krish Dholakia
276c00a7f6
Merge pull request #15544 from davizucon/main 
small fix code snippet  custom_prompt_management.md
 2025-10-14 16:23:32 -07:00
Eddie Richter
900a57f5b1 Adding new models to the lemonade provider 2025-10-14 16:56:36 -06:00
Ishaan Jaff
a10425d1ce
[Feat] Allow Team Admins to export a report of the team spending (#15542) 
* v0 for export

* v0 for Export

* add types for EntityUsageExportModalProps

* add folder struct

* add summry selector

* add selector for export

* add utils for entity usage export

* refactored buttons

* fixes name

* fix alignment

* test: EntityUsageExportModal

* fix lint
 2025-10-14 15:07:59 -07:00
Ishaan Jaff
65163c7ccb [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) 
* fix: add google_routes to llm_api_routes

* test: test_virtual_key_llm_api_routes_allows_google_routes
 2025-10-14 13:57:39 -07:00
AlexsanderHamir
5bffa58ee0 feat(ssl): add configurable ECDH curve for TLS performance 
Add `ssl_ecdh_curve` setting to configure TLS key exchange curve.
Allows disabling PQC on OpenSSL 3.x for better performance.

Configurable via SDK (litellm.ssl_ecdh_curve), YAML (litellm_settings),
or env var (SSL_ECDH_CURVE). Common curves: X25519, prime256v1, secp384r1.
 2025-10-14 13:57:39 -07:00
Krrish Dholakia
b0d963cc0c docs(index.md): bump rc 2025-10-14 13:57:39 -07:00
AlexsanderHamir
fcb85f8856 perf(router): optimize timing functions in completion hot path 
Replace time.time() with more appropriate timing functions for better
performance and reliability:

- Use time.perf_counter() for duration measurements in acompletion(),
  _acompletion(), and async_get_available_deployment()
- Use time.monotonic() for timeout calculations in scheduler methods
  (schedule_acompletion and _schedule_factory)

Benefits:
- 30-40% faster timing calls (~300ns savings per call)
- time.monotonic() provides reliable timeouts unaffected by system
  clock changes (NTP adjustments, DST, manual time changes)
- time.perf_counter() offers highest resolution for performance metrics
- Follows Python best practices for timing operations
 2025-10-14 13:57:39 -07:00
Krrish Dholakia
cb29e33cad docs: fix doc 2025-10-14 13:57:39 -07:00
Dhruv Yadav
b57406e53e add tests for openrouter cost tracking 2025-10-14 13:57:39 -07:00
Dhruv Yadav
09662b5081 direct cost calculation from openrouter 2025-10-14 13:57:39 -07:00
AlexsanderHamir
2e57d19a55 update benchmarks 2025-10-14 13:57:39 -07:00
AlexsanderHamir
6df051e87b docs: update benchmark results with improved infrastructure 
- Update to 2 instance baseline (1035 RPS @ 200ms median)
- Add 4 instance results (1170 RPS @ 100ms median)
- Update machine specs to 4 CPU / 8GB RAM
- Update Locust settings to 1000 users
 2025-10-14 13:57:39 -07:00
kowyo
92bd238b87 fix comment 2025-10-14 13:57:39 -07:00
kowyo
6896c60b5b fix: only use think level for gpt-oss model 2025-10-14 13:57:39 -07:00
Kowyo
9557a37d37 fix: update 'think' parameter assignment to use provided value in transformation.py 2025-10-14 13:57:39 -07:00
Kowyo
e30b716ba8 fix: add 'think' parameter handling in ollama_chat.py 2025-10-14 13:57:39 -07:00