Commit Graph

23619 Commits

Author SHA1 Message Date
Ishaan Jaff
078096247b
[Bug Fix] The model gemini-2.5-flash with the merge_reasoning_content_in_choices parameter does not work (#13066)
* _optional_combine_thinking_block_in_choices

* test_optional_combine_thinking_block_with_none_content
2025-07-28 17:42:28 -07:00
Krrish Dholakia
16af2d9a50 test: skip dbrx claude 3-7 sonnet test - rate limit errors 2025-07-28 17:34:42 -07:00
Jugal D. Bhatt
ba95541571
[MCP gateway] add url namespacing docs (#13063)
* added the url docs

* Added url change
2025-07-28 17:28:55 -07:00
Jugal D. Bhatt
33c012a49d
fix scrolling issue (#13015) 2025-07-28 17:28:10 -07:00
Alex Strick van Linschoten
75ae43e667
feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration (#12956)
* feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration

This commit brings the langfuse_otel integration to feature parity with the vanilla Langfuse integration by adding support for all metadata fields.

Changes:
- Extended LangfuseSpanAttributes enum with all supported metadata fields:
  - Generation-level: generation_name, generation_id, parent_observation_id, version, mask_input/output
  - Trace-level: trace_user_id, session_id, tags, trace_name, trace_id, trace_metadata, trace_version, trace_release, existing_trace_id, update_trace_keys
  - Debug: debug_langfuse

- Implemented metadata extraction and mapping in langfuse_otel.py:
  - Added _extract_langfuse_metadata() helper to extract metadata from kwargs
  - Support for header-based metadata (langfuse_* headers) via proxy
  - Enhanced _set_langfuse_specific_attributes() to map all metadata to OTEL attributes
  - JSON serialization for complex types (lists, dicts) for OTEL compatibility

- Updated documentation:
  - Added 'Metadata Support' section explaining all fields are now supported
  - Provided usage example showing how to pass metadata
  - Clarified that traces are viewed in Langfuse UI (not generic OTEL backends)
  - Added opentelemetry-exporter-otlp to required dependencies

This allows users to pass metadata like:
metadata={
    'generation_name': 'my-generation',
    'trace_id': 'trace-123',
    'session_id': 'session-456',
    'tags': ['prod', 'v1'],
    'trace_metadata': {'user_type': 'premium'}
}

All metadata is exported as OpenTelemetry span attributes with 'langfuse.*' prefix for easy filtering and analysis in the Langfuse UI.

* Fix ruff linting error

* test(langfuse-otel): Fix failing test and add comprehensive metadata tests

- Fix test_set_langfuse_environment_attribute to use positional arguments
  instead of keyword arguments when asserting safe_set_attribute calls
- Add test_extract_langfuse_metadata_basic to verify metadata extraction
  from litellm_params
- Add test_extract_langfuse_metadata_with_header_enrichment to test
  integration with header-based metadata using a stubbed LangFuseLogger
- Add test_set_langfuse_specific_attributes_full_mapping to comprehensively
  test all metadata field mappings and JSON serialization of complex types

These tests ensure full coverage of the langfuse_otel metadata features
added in commit ab1dbe355 and fix the CI test failure.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-07-28 16:53:36 -07:00
Richard Tweed
3abf7cc871
chore: Improve docs for cost tracking (#12976) 2025-07-28 16:49:14 -07:00
direcision
31d8edb1bf
fix: correct CompletionRequest messages type to match OpenAI API spec (#12980)
* fix: correct CompletionRequest messages type to match OpenAI API spec

- Changed messages field type from List[str] to List[ChatCompletionMessageParam]
- This ensures proper OpenAI API compatibility where messages should be objects with role and content fields
- Fixes type inconsistency in completion request handling

* feat(tests): Add comprehensive tests for CompletionRequest model

- Add test_completion.py for litellm.types.completion module
- Test ChatCompletionMessageParam type validation
- Test tool message format compatibility
- Test function message format (deprecated)
- Test multimodal content (text + image)
- Test default empty messages list
- Test all optional parameters
- Validate OpenAI ChatCompletion API message format compatibility
2025-07-28 16:47:20 -07:00
stellasec
47f6984dce
Properly parse json options for key generation in the UI (#12989) 2025-07-28 16:43:51 -07:00
Emmanuel Ferdman
9459d72096
Remove duplicate test case verifying field filtering logic (#13023)
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-07-28 16:40:06 -07:00
Ishaan Jaff
deb1efa8ce
[Bug Fix] Pass through logging handler VertexAI - ensure multimodal embedding responses are logged (#13050)
* fix _is_multimodal_embedding_response

* test_vertex_passthrough_handler_multimodal_embedding_response
2025-07-28 16:30:19 -07:00
Krrish Dholakia
2d697ca476 build: bump pip 2025-07-28 16:10:19 -07:00
Ishaan Jaff
d5c61cc526
docs - openweb show how to include reasoning content (#13060) 2025-07-28 15:25:47 -07:00
Christoph Koehler
8f7d896e26
fix: add X-Initiator header for GitHub Copilot to reduce premium requests (#13016)
- Implement X-Initiator header logic in GithubCopilotConfig.validate_environment()
- Set header to "agent" when messages contain agent or tool roles, "user" otherwise
- Reduces unnecessary premium Copilot API usage for non-user calls

Fixes #12859
2025-07-28 09:55:24 -07:00
Krrish Dholakia
2da6d457ef test: update logic to fix for unit tests 2025-07-28 07:38:58 -07:00
Krrish Dholakia
8d49dd66cb build: update ui build 2025-07-27 11:50:56 -07:00
Krrish Dholakia
39b7ce5388 refactor(create_key_button.tsx): cleanup read only option (confusing) 2025-07-27 11:49:56 -07:00
Krrish Dholakia
3359464624 fix(proxy_settings_endpoints.py): fix clearing SSO settings 2025-07-27 11:13:40 -07:00
Krrish Dholakia
1b7f7ba643 build: build new ui 2025-07-27 11:08:27 -07:00
Krish Dholakia
94aa6b0b5d
Bulk User Edit - additional improvements - edit all users + set 'no-default-models' on all users (#12925)
* feat(bulk_user_update/): support updating all users on proxy

* fix(bulk_edit_user.tsx): persist user settings when 'add to team' clicked

* fix(team_endpoints.py): bulk add all proxy users to team

supports flow from UI to add all existing users to a team

* fix: minor fixes

* feat(user_edit_view.tsx): support setting no default model on user edit

allows preventing users from calling models outside team scope

* fix(user_edit_view.tsx): prevent triggering submit when 'cancel' is clicked

* refactor(internal_user_endpoints.py): refactor to reduce function size
2025-07-27 10:12:30 -07:00
Krish Dholakia
1737cf4257
VertexAI - camelcase optional params for image generation + Anthropic - streaming, always ensure assistant role set on only first chunk (#12889)
* fix(vertex_ai/image_generation): transform `_` param to camelcase

Fixes https://github.com/BerriAI/litellm/issues/12690

* test(test_vertex_image_generation.py): add unit tests

* fix(streaming_handler.py): assert only 1 assistant chunk in stream

Fixes https://github.com/BerriAI/litellm/issues/12616

* fix(streaming_handler.py): fix check
2025-07-27 10:09:43 -07:00
Krrish Dholakia
ff0b40a22b test: fix test 2025-07-27 09:52:22 -07:00
Krish Dholakia
e8a374fc47
Litellm release notes 07 27 2025 p1 (#13027)
* docs(index.md): initial commit for v1.74.9-stable release note

* docs(index.md): add more cost tracking models

* docs(index.md): add new llm api endpoints + mcp gateway features

* docs: add logging/guardrail improvements

* docs(index.md): complete initial draft

* build(model_prices_and_context_window.json): fix or pricing

* build(model_prices_and_context_window.json): fix or pricing
2025-07-27 09:46:25 -07:00
Ishaan Jaff
2c38dc0de7 test_router_auto_router 2025-07-26 13:33:53 -07:00
Jugal D. Bhatt
b921e22594
[MCP Gateway] Litellm mcp multi header propagation (#13003)
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* add multi server header support

* add and fix tests

* fix tests

* fix tests

* add a common util

* ruff fix

* fix ruff

* fix tests

* fix migration

* mypy fix

* change server py
2025-07-26 11:45:14 -07:00
Krish Dholakia
eed0cf2ee9
UI SSO - fix reset env var when ui_access_mode is updated (#13011)
* fix(ui_sso.py): fix form action on login when sso is enabled

* fix: multiple fixes - fix resetting env var in proxy config + add key to exception message on key decryption

fixes issue where env vars would be reset

* refactor(proxy_server.py): cleanup redundant decryption line

* fix(proxy_setting_endpoints.py): show saved ui access mode

allows admin to know what they'd previously stored in db
2025-07-26 11:42:41 -07:00
Krish Dholakia
4b50566d6d
[FEAT] Model-Guardrails: Add on UI (#13006)
* feat(proxy_server.py): working guardrails on streaming output

ensures guardrail actually raises an error if flagged during streaming output

* test: add unit tests

* feat(advanced_settings.tsx): add guardrails option as ui component on model add

enables setting guardrails on model add

* feat(add_model_tab.tsx): fix add model form

* feat(model_info_view.tsx): support adding guardrails on model update

* fix(add_model_tab.tsx/): working health check when guardrails selected

* fix(proxy_server.py): fix yield
2025-07-26 11:42:03 -07:00
Jugal D. Bhatt
586b1ec8b4
[Vector Store] make vector store permission management OSS (#12990)
* add vector store on ui behind enterprise in vector store

* remove enterprise
2025-07-26 11:13:49 -07:00
Matthias Dittrich
9211158e73
Fixup ollama model listing (again) (#13008) 2025-07-26 11:07:49 -07:00
Jugal D. Bhatt
10595ea077
[MCP Gateway] add Litellm mcp alias for prefixing (#12994)
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* fix tests

* add a common util

* ruff fix

* fix migration
2025-07-25 17:57:52 -07:00
Jugal D. Bhatt
c7774ba495
add fix for redaction (#13005) 2025-07-25 17:48:48 -07:00
Emerson Gomes
dc8c14b504
Fix issue writing db (#13001) 2025-07-25 17:27:13 -07:00
Krish Dholakia
d6423d913d
build: update pip package (#12998) 2025-07-25 16:23:53 -07:00
Krrish Dholakia
5ad116ae89 feat(proxy_server.py): support checking full str on streaming guardrails post call hook
ensures streaming guardrails are actually useful
2025-07-25 15:49:50 -07:00
Krrish Dholakia
e5d68e5222 feat(proxy/utils.py): support model level guardrails on stream event
enables guardrails to work with streaming
2025-07-25 15:37:27 -07:00
Jugal D. Bhatt
d1b63566ac
remove features from enterprise (#12988) 2025-07-25 10:55:28 -07:00
Jugal D. Bhatt
c66b288bcf
[LLM Translation] fix query params for realtime api intent (#12838)
* fix query params for realtime api intent

* fix my py

* Add typed dict

* remove typed dict

* fix comments

* add test

* add test

* added proxt log revert

* add real time q params
2025-07-25 10:09:53 -07:00
Jugal D. Bhatt
c2833e693e
clean and verify key before inserting (#12840)
* clean and verify key

* change checking logic

* Add unit test
2025-07-25 10:09:28 -07:00
Jugal D. Bhatt
b4da29c83e
[LLM Translation] Add bytedance/ui-tars-1.5-7b on openrouter (#12882)
* add bytedance model

* add source
2025-07-25 10:09:14 -07:00
Krish Dholakia
23d562b1a2
build(github/manual_pypi_publish.yml): manual workflow to publish pip package - used for pushing dev releases (#12985)
* build(github/manual_pypi_publish.yml): manual workflow to publish pip package - used for pushing dev releases

* ci: remove redundant file
2025-07-25 09:26:47 -07:00
Ishaan Jaff
0f015a2f68 docs fix link 2025-07-25 09:08:18 -07:00
Ishaan Jaff
8f86355361 fix ci/cd testing 2025-07-25 09:05:27 -07:00
Ishaan Jaff
1a15bd4885 docs auto router 2025-07-25 09:04:27 -07:00
Ishaan Jaff
f2e9834dc6 docs - auto router on litellm proxy 2025-07-25 08:58:54 -07:00
Ishaan Jaff
7bb48ffaf9 docs auto routing 2025-07-25 08:50:16 -07:00
Ishaan Jaff
50574c0c3e docs - auto routing 2025-07-25 07:31:39 -07:00
Ishaan Jaff
20450bfe94 fix mapped test 2025-07-25 07:24:42 -07:00
Ishaan Jaff
ecfafd1243 docs update 2025-07-25 07:19:32 -07:00
Ishaan Jaff
7745f61fb1 docs update 2025-07-25 07:19:20 -07:00
Ishaan Jaff
9c76ccd92c docs Health Check Server 2025-07-25 07:17:26 -07:00
Krish Dholakia
6d3ce76e1c
Guardrails - support model-level guardrails (#12968)
* fix(custom_guardrail.py): initial logic for model level guardrails

* feat(custom_guardrail.py): working pre call guardrails

* fix(custom_guardrails.py): check if custom guardrails set before running event hook

* test(test_custom_guardrail.py): add unit tests for async pre call deployment hook on custom guardrail

* feat(custom_guardrail.py): add post call processing support for guardrails

allows model based guardrails to run on the post call event for that model only

* fix(utils.py): only run if call type is in enum

* test: update unit tests to work
2025-07-24 22:38:03 -07:00