ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
30,881 Commits 9 Branches 7 Tags 935 MiB
12bc66aa5b
Commit Graph

30881 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Colin Lin
0bd144103d [stripe] simplify opus test 2025-12-05 10:56:09 -05:00
Colin Lin
3046b9f163 [stripe] opus budget thinking 2025-12-05 10:56:02 -05:00
Sameer Kankute
3d6b7f0d3d Add background health checks to db 2025-12-05 14:27:37 +05:30
yuneng-jiang
b9b5d638c8
Merge pull request #17524 from BerriAI/litellm_team_user_settings_fix 
[Fix] Select in Edit Membership Modal
 2025-12-04 23:13:13 -08:00
yuneng-jiang
37bfe65bdd Adding screenshot to debug 2025-12-04 23:05:00 -08:00
Sameer Kankute
c8fbcc7f1c add tutorial as well 2025-12-05 12:32:23 +05:30
yuneng-jiang
50283a00a3 e2e fix 2025-12-04 22:51:52 -08:00
Sameer Kankute
acc0b5fe27
Merge pull request #17362 from BerriAI/litellm_vertex-bge-cherrypick 
[Feat] VertexAI - Add BGE Embeddings support
 2025-12-05 11:53:42 +05:30
Sameer Kankute
99fd96687f Fix vector store configuration synchronization failure 2025-12-05 11:46:14 +05:30
Krish Dholakia
b3a3081e8e
Guardrails API - new structured_messages param (#17518) 
* fix(generic_guardrail_api.py): add 'structured_messages' support

allows guardrail provider to know if text is from system or user

* fix(generic_guardrail_api.md): document 'structured_messages' parameter

give api provider a way to distinguish between user and system messages

* feat(anthropic/): return openai chat completion format structured messages when calls made via `/v1/messages` on Anthropic

* feat(responses/guardrail_translation): support 'structured_messages' param for guardrails

structured openai chat completion spec messages, for guardrail checks when using /v1/responses api

allows guardrail checks to work consistently across APIs
 2025-12-04 22:08:00 -08:00
Krish Dholakia
51cc102c30
fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514) 
* fix(unified_guardrail.py): support during_call event type for unified guardrails

allows guardrails overriding apply_guardrails to work 'during_call'

* feat(generic_guardrail_api.py): support new 'tool_calls' field for generic guardrail api

returns the tool calls emitted by the LLM API to the user

* fix(generic_guardrail_api.py): working anthropic /v1/messages tool call response

send llm tool calls to guardrail api when called via `/v1/messages` API

* fix(responses/): run generic_guardrail_api on responses api tool call responses

* fix: fix tests

* test: fix tests

* fix: fix tests
 2025-12-04 22:06:13 -08:00
Cesar Garcia
316f7671a9
fix(gemini): handle partial JSON chunks after first valid chunk (#17496) 
* fix(gemini): allow JSON accumulation on any chunk, not just first

* test(gemini): add tests for partial JSON chunk handling
 2025-12-04 22:01:59 -08:00
yuneng-jiang
8da265b901 Fix select in edit membership 2025-12-04 21:59:43 -08:00
Kristian Brünn
63fae79493
fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage (#17504) 
* fix: optimize SpendLogs queries to use timestamp filtering (#17487)

* use timestamptz & enhance test
 2025-12-04 21:52:57 -08:00
Krish Dholakia
8776336c3c
Enable detailed debugging for reference (#17508) 
* Deprecate set_verbose in favor of LITELLM_LOG

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Update debugging documentation links

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-12-04 21:51:56 -08:00
Devaj Mody
48b5100c18
fix(guardrails): mask all matching keywords in content filter (#17521) 
Fixes #17517

  - Fixed bug where only the first matching blocked keyword was masked
  - Now iterates through ALL blocked keywords and masks each one
  - Added 3 regression tests for multiple keyword masking
 2025-12-04 21:50:15 -08:00
yuneng-jiang
a96677c299 Fix /get/config/callbacks callback variables 2025-12-04 21:48:49 -08:00
Sameer Kankute
392e5059b0 Add steps to add litellm proxy in cursor 2025-12-05 10:02:42 +05:30
Sameer Kankute
01ee46b493 Add steps to add litellm proxy in cursor 2025-12-05 10:01:48 +05:30
Sameer Kankute
4d83a48b59 Add steps to add litellm proxy in cursor 2025-12-05 09:39:58 +05:30
Sameer Kankute
a6006e698c Add support for cursor BYOK with its own configuration 2025-12-05 09:34:49 +05:30
Dominic Feliton
6af693de36
(fix): empty response + vllm streaming (#17516) 
* Fix empty response + vllm streaming

* Add unit test
 2025-12-04 19:24:28 -08:00
YutaSaito
7106509ef0
chore: add stack trace to mcp error message (#17269) 2025-12-04 19:19:00 -08:00
Ishaan Jaffer
4f3b843efe docs openai 2025-12-04 18:32:23 -08:00
Xianzong Xie
52d784b763 fix: correct mock setup for delete_polling test 
- Use Mock instead of AsyncMock for init_async_client (sync method)

Committed-By-Agent: cursor
 2025-12-04 18:00:28 -08:00
Ishaan Jaff
b2e8d3fd42
[Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider (#17448) 
* feat: Add JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Add simple JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Implement JSON-based provider config and migrate PublicAI

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* docs fix

* undo change

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-12-04 17:59:25 -08:00
Xianzong Xie
03ee5c4489 test: add comprehensive tests for polling via cache feature 
- Add TestPollingConditionChecks: tests for all condition combinations
- Add TestStreamingEventParsing: tests for OpenAI streaming event handling
- Add TestEdgeCases: tests for empty model, multiple slashes, edge cases

Total test count increased significantly for better coverage.

Committed-By-Agent: cursor
 2025-12-04 17:55:38 -08:00
Xianzong Xie
56cbdde64d remove file 2025-12-04 17:53:51 -08:00
Xianzong Xie
a8a38778a3 fix: resolve provider from router for polling_via_cache 
- Fix bug where model names without slash (e.g., 'gpt-5') couldn't
  match providers in polling_via_cache list
- Look up model in llm_router.model_name_to_deployment_indices
- Check ALL deployments for matching provider (supports load balancing)
- Check custom_llm_provider first, then extract from model string
- Add comprehensive tests for provider resolution logic

Committed-By-Agent: cursor
 2025-12-04 17:47:30 -08:00
Ishaan Jaff
fadfbb13d3
[Docs] A2a - Permission management (#17515) 
* docs add a2a gateway + mcp gateway

* docs a2a permissions

* docs a2a permission

* docs

* docs a2a

* docs a2a

* add new img

* docs agent permissions
 2025-12-04 17:29:47 -08:00
yuneng-jiang
de8f0a3409 Ensure fresh data to prevent race condition in scim v2 2025-12-04 17:04:59 -08:00
Ishaan Jaffer
dfe3124335 UI new build 2025-12-04 16:34:59 -08:00
Ishaan Jaffer
5cc1a5031b fix invoke_agent_a2a 2025-12-04 16:32:15 -08:00
Ishaan Jaff
575e769bff
[Feat] UI - Agent Gateway - set allowed agents by key, team (#17511) 
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route

* add AgentPermissionsProps

* add agents on team/key create

* add agent selector on team/key

* add agent selector on key edit /info

* add AgentPermissions

* docs list + invoke agents
 2025-12-04 16:31:17 -08:00
Ishaan Jaff
8f592088ed
[Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend (#17502) 
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route
 2025-12-04 16:31:00 -08:00
yuneng-jiang
5a2e89a49e Customer Usage UI 2025-12-04 15:13:17 -08:00
Ishaan Jaff
e3116da653
feat: Add /global/spend/tags to admin viewer routes (#17501) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-12-04 14:43:33 -08:00
Ishaan Jaffer
c6e26a20b1 refactor invoke 2025-12-04 14:21:01 -08:00
Raghav Jhavar
72eb4c3a1c
🆕 feat: support routing to only websearch supported deployments (#17500) 
* support routing to only websearch supported deployments

* add docs
 2025-12-04 14:18:20 -08:00
Xianzong Xie
748bb6d5f5 test: add tests for all ResponsesAPIResponse fields 
- Add test_update_state_with_all_responses_api_fields to verify all fields
- Add test_update_state_preserves_existing_fields to verify partial updates

Committed-By-Agent: cursor
 2025-12-04 14:15:06 -08:00
Anas AbdelR
2abcc77944
fix: resolve ruff lint errors (#17490) 
Fixed 20 of 22 lint errors:

- batches/batch_utils.py: Added missing model_name parameter to functions
- integrations/custom_guardrail.py: Removed unused Tuple import
- llms/custom_httpx/http_handler.py: Removed unused AIOHTTP_NEEDS_CLEANUP_CLOSED import
- llms/anthropic/chat/guardrail_translation/handler.py: Prefixed unused variable with underscore
- llms/openai/responses/guardrail_translation/handler.py: Removed duplicate BaseModel import, prefixed unused variable
- llms/pass_through/guardrail_translation/handler.py: Prefixed unused variables with underscore
- proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py: Removed unused List and Tuple imports
- proxy/hooks/parallel_request_limiter_v3.py: Removed unused import

Remaining 2 errors are PLR0915 (too many statements) which require refactoring.
 2025-12-04 14:12:57 -08:00
Xianzong Xie
9a0a37fffa feat: extract all ResponsesAPIResponse fields from response.completed 
- Add support for all ResponsesAPIResponse fields in update_state
- Extract model, instructions, temperature, top_p, max_output_tokens,
  previous_response_id, text, truncation, parallel_tool_calls, user,
  store, and incomplete_details from response.completed event
- Pass all fields to final update_state call

Committed-By-Agent: cursor
 2025-12-04 14:11:13 -08:00
Yannay Hammer
2627f0d519
Fix aim security guardrail tests (#17499) 2025-12-04 13:29:20 -08:00
yuneng-jiang
b12ccb1a7a Publish proxy extras 2025-12-04 13:11:20 -08:00
yuneng-jiang
1834377950 Adding migration 2025-12-04 12:57:06 -08:00
yuneng-jiang
5439f03bfc bump: version 0.4.9 → 0.4.10 2025-12-04 12:56:43 -08:00
yuneng-jiang
2e65c464ad Adding tests 2025-12-04 12:36:15 -08:00
yuneng-jiang
562afb208d v0 customer usage, pending tests + extras version bump 2025-12-04 12:30:08 -08:00
Krrish Dholakia
5aeba81538 docs(multi_tenant_architecture.md): add new architecture doc 2025-12-04 11:13:50 -08:00
Ishaan Jaff
8ee8cad4f1
feat - new model amazon.titan-image-generator-v2:0 (#17488) 2025-12-04 09:26:55 -08:00