* Add v1 cut of container api
* fix lint errors
* Add proxy support to container apis & logging support (#16049)
* Add proxy support to container apis
* Add logging support
* Add cost tracking support for containers and documentation
* Add new constant documentation
* Add container cost in model map
* fix failing azure tests
* Update tests based on model map changes
* fix model map tests
* fix model map tests
* Container modeshould be container
* Container tests fix
* Merge branch 'main' into litellm_sameer_oct_staging_2
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
* Fix vercel_ai_gateway entry for glm-4.6
* Fix: add unit test for vercel_ai_gateway/zai/glm-4.6
* Fix:
Add fix for unit test on vercel_ai_gateway/zai/glm-4.6
* Remove print statements in test_vercel_ai_gateway_glm46_cost_math
* Add gemini/gemini-embedding-001 pricing entry for Google GenAI API
Add pricing and configuration entry for "gemini/gemini-embedding-001"
model when called via Google GenAI API (google-genai provider). This
mirrors the existing Vertex AI entry ("gemini-embedding-001") but uses
the 'gemini' provider instead.
Co-Authored-By: Claude <noreply@anthropic.com>
* Update tpm and rpm for gemini/gemini-embedding-001
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix(opentelemetry.py): fix issue where headers were not being split correctly
* feat(bedrock/image): Support bedrock titan image generation
Closes https://github.com/BerriAI/litellm/issues/361
* build(model_prices_and_context_window.json): track titan image gen pricing
enables cost tracking per request
* feat(amazon_titan_transformation.py): support titan image generation cost tracking
* docs: document new model
* docs: update docs to indicate cost tracking + refactor rerank into separate doc
* fix: fix mypy linting error
* fix: fix type ignore
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm
new vector store provider
* feat(vector_store/): use vector store id for vertex ai search api
* fix: transformation.py
cleanup
* fix: implement abstract function
* fix: fix linting error
* fix: main.py
fix check
* feat: initial commit with working passthrough support for vertex ai search api through litellm
* feat(llm_passthrough_endpoints.py): fix passing correct project on datastore passthrough
* feat(vertex_ai/): support passthrough call for vertex ai search vector store
* docs(vertex_ai_search_datastore.md): document new vertex ai passthrough endpoint
* docs(sidebars.js): document new endpoint
* feat: initial commit adding logging for vertex ai passthrough api
allows vertex ai vector search api to work with cost calculation
* feat(vertex_ai/): search vector store cost tracking
* fix(vertex_passthrough_logging_handler.py): log the cost
* fix: improve logged response
* fix(vertex_passthrough_logging_handler.py): logging
* feat(litellm_logging): main.py
add cost tracking for vertex ai search api via unified api
* refactor: fix ruff checks
* fix(llm_passthrough_endpoints.py): fix linting
* Addd v2/chat support for cohere
* fix streaming
* Use v2_transformation for logging passthrough:
* Use v2_transformation for logging passthrough:
* Add test for checking if document and citation_options is getting passed
* Update the cohere model
* Add cost tracking for vertex ai passthrough batch jobs
* Add full passthrough support
* refactor code according to the comments
* Add passthrough handler
* remove invalid params
* Updated documentation
* Updated documentation
* Updated documentation
* Correct the import
* Add openai videos generation and retrieval support
* add retrieval endpoint
* Add docs
* Add imports
* remove orjson
* remove double import
* fix openai videos format
* remove mock code
* remove not required comments
* Add tests
* Add tests
* Add other video endpoints
* Fix cost calculation and transformation
* Fixed mypy tests
* remove not used imports
* fix documentation for get batch req (#15742)
* Add grounding info to responses API (#15737)
* Add grounding info to responses API
* fix lint errors
* Use typed objects for annotations
* Use typed objects for annotations
* fix mypy error
* Litellm fix json serialize alreting 2 (#15741)
* fix json serializable error for alerts
* Add test
* fix mypt errors
* fix mypt errors
* Add Qwen3 imported model support for AWS Bedrock (#15783)
* Add qwen imported model support
* fix mypy errors
* fix empty user message error (#15784)
* fix typed dict for list
* Add azure supported videos endpoint
* fix mapped tests
* add azure sora models to model map
* Add OpenAI video generation and content retrieval support (#15745)
* Add openai videos generation and retrieval support
* add retrieval endpoint
* Add docs
* Add imports
* remove orjson
* remove double import
* fix openai videos format
* remove mock code
* remove not required comments
* Add tests
* Add tests
* Add other video endpoints
* Fix cost calculation and transformation
* Fixed mypy tests
* remove not used imports
* fix typed dict for list
* fix mypy errors
* move directory
* make v2 chat default
* Fix mypy tests
* Fix mypy tests
* Fix mypy tests
* Fix mypy tests
* Revert "Add Azure Video Generation Support with Sora Integration"
* refactor videos repo
* add test
* Add azure openai videos support
* Add azure openai videos support
* Add router endpoint support for videos
* fix mypy error
* add azure models
* fix mapped test
* fix mypy error
* Add proxy router test
* Add proxy router test
* remove deprecated model name from tests
* fix import error
* fix import error
* Add gaurdrail integration in videos endpoint
* Add logging support for videos endpoint
* Add final documentation supporting videos integration
* fix model name and document input
* Update literals to avoid mypy errors
* Remove unused imports and print statements
* revert guardrail support for video generation and video remix
* revert guardrail support for video generation and video remix
* Fix failing mapped and llm translation tests
* Implement fix for thinking_blocks and converse API calls
This fixes Claude's models via the Converse API, which should also fix
Claude Code.
* Add thinking literal
* Fix mypy issues
* Type fix for redacted thinking
* Add voyage model integration in sagemaker
* Add config file logic
* Use already exiting voyage transformation
* refactor code as per comments
* fix merge error
* refactor code as per comments
* refactor code as per comments
* UI new build
* [Fix] router - regression when adding/removing models (#15451)
* fix(router): update model_name_to_deployment_indices on deployment removal
When a deployment is deleted, the model_name_to_deployment_indices map
was not being updated, causing stale index references. This could lead
to incorrect routing behavior when deployments with the same model_name
were dynamically removed.
Changes:
- Update _update_deployment_indices_after_removal to maintain
model_name_to_deployment_indices mapping
- Remove deleted indices and decrement indices greater than removed index
- Clean up empty entries when no deployments remain for a model name
- Update test to verify proper index shifting and cleanup behavior
* fix(router): remove redundant index building during initialization
Remove duplicate index building operations that were causing unnecessary
work during router initialization:
1. Removed redundant `_build_model_id_to_deployment_index_map` call in
__init__ - `set_model_list` already builds all indices from scratch
2. Removed redundant `_build_model_name_index` call at end of
`set_model_list` - the index is already built incrementally via
`_create_deployment` -> `_add_model_to_list_and_index_map`
Both indices (model_id_to_deployment_index_map and
model_name_to_deployment_indices) are properly maintained as lookup
indexes through existing helper methods. This change eliminates O(N)
duplicate work during initialization without any behavioral changes.
The indices continue to be correctly synchronized with model_list on
all operations (add/remove/upsert).
* fix(prometheus): Fix Prometheus metric collection in a multi-workers environment (#14929)
Co-authored-by: sotazhang <sotazhang@tencent.com>
* Add tiered pricing and cost calculation for xai
* Use generic cost calculator
* Resolve conflicts in generated HTML files
* Remove penalty params as supported params for gemini preview model (#15503)
* fix conversion of thinking block
* add application level encryption in SQS (#15512)
* docs: fix doc
* docs(index.md): bump rc
* [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500)
* fix: add google_routes to llm_api_routes
* test: test_virtual_key_llm_api_routes_allows_google_routes
* build: bump version
* bump: version 1.78.0 → 1.78.1
* add application level encryption in SQS
* add application level encryption in SQS
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
* [Feat] Bedrock Knowledgebase - return search_response when using /chat/completions API with LiteLLM (#15509)
* docs: fix doc
* docs(index.md): bump rc
* [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500)
* fix: add google_routes to llm_api_routes
* test: test_virtual_key_llm_api_routes_allows_google_routes
* add AnthropicCitation
* fix async_post_call_success_deployment_hook
* fix add vector_store_custom_logger to global callbacks
* test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call
* async_post_call_success_deployment_hook
* add async_post_call_streaming_deployment_hook
* async def test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call_streaming(setup_vector_store_registry):
* fix _call_post_streaming_deployment_hook
* fix async_post_call_streaming_deployment_hook
* test update
* docs: Accessing Search Results
* docs KB
* fix chatUI
* fix searchResults
* fix onSearchResults
* fix kb
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* [Feat] Add dynamic rate limits on LiteLLM Gateway (#15518)
* docs: fix doc
* docs(index.md): bump rc
* [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500)
* fix: add google_routes to llm_api_routes
* test: test_virtual_key_llm_api_routes_allows_google_routes
* build: bump version
* bump: version 1.78.0 → 1.78.1
* fix: KeyRequestBase
* fix rpm_limit_type
* fix dynamic rate limits
* fix use dynamic limits here
* fix _should_enforce_rate_limit
* fix _should_enforce_rate_limit
* fix counter
* test_dynamic_rate_limiting_v3
* use _create_rate_limit_descriptors
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* Add google rerank endpoint
* Add docs
* fix mypy error
* fix mypy and lint errors
* Add haiku 4.5 integration
* Add haiku 4.5 integration for other regions as well
* Handle citation field correctly
* Fix filtering headers for signature calcs
* Add haiku 4.5 integration (#15650)
---------
Co-authored-by: Leslie Cheng <leslie.cheng5@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: Lucas <10226902+LoadingZhang@users.noreply.github.com>
Co-authored-by: sotazhang <sotazhang@tencent.com>
Co-authored-by: Deepanshu Lulla <deepanshu.lulla@gmail.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
