Commit Graph

90 Commits

Author SHA1 Message Date
Sameer Kankute
ab377f396e
Merge pull request #23718 from BerriAI/litellm_fix_vertex_ai_batch
Fix: Vertex ai Batch Output File Download Fails with 500
2026-03-16 19:05:49 +05:30
Sameer Kankute
22b333cae6 Fix downloading vertex ai files 2026-03-16 12:08:06 +05:30
yuneng-jiang
4fc0975d22 Fix flaky e2e batch test: set batch_processed=True on completion in retrieve_batch
The retrieve_batch endpoint sets batch status to "complete" but never set
batch_processed=True, permanently blocking file deletion. CheckBatchCost
(the safety net) also excluded completed batches from its primary query,
so batch_processed was never set by either path.

Three fixes:
1. update_batch_in_database sets batch_processed=True when status reaches
   "complete", with old-schema fallback retry
2. CheckBatchCost primary query no longer excludes complete/completed
   (batch_processed=False filter prevents reprocessing)
3. retrieve_batch early-return now includes "complete" (DB-normalized
   spelling) to avoid unnecessary provider re-polls

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 18:18:32 -07:00
Ishaan Jaff
1b96064600
fix(proxy): prevent OOM/Prisma connection loss from unbounded managed-object poll (#23472)
* fix(proxy): cap managed-object poll size + expire stale rows + kill-switch flag to prevent OOM/Prisma connection loss

* fix(constants): simplify PROXY_BATCH_POLLING_ENABLED readability

* docs+test: document new polling env vars, add pagination+stale-cleanup tests

* fix: exclude stale_expired from batch poll queries; fix update_many assertions in tests

* fix: scope stale cleanup to file_purpose, fix file_object mocks, add CheckBatchCost tests

* fix: avoid duplicate cost logging in fallback path; guard integer constants against zero/negative values

* fix: cache _has_batch_processed_column; guard cleanup from aborting poll; narrow fallback except

* fix: add complete/completed to primary query not_in; fix vacuous test assertion

- Primary find_many was missing "complete" and "completed" in its not_in
  filter, creating asymmetry with the fallback query. A job whose status
  was set to "complete" but whose batch_processed flag update failed would
  be silently re-fetched and re-processed every cycle, emitting duplicate
  cost logs.

- test_fallback_completion_update_omits_batch_processed patched
  _is_base64_encoded_unified_file_id to return None, causing an immediate
  continue — so update() was never called and the assertion looped over an
  empty list (vacuously true). Rewrote the test to mock the full
  completion pipeline, verify update() is called exactly once, and assert
  batch_processed is absent from the update data.

- Added symmetric test (primary path) proving batch_processed IS included
  when the column exists.

Made-with: Cursor
2026-03-13 11:01:40 -07:00
Ryan Crabbe
c4db53a98a Address review feedback: remove dead code, add error handling, strengthen test assertions
- Remove unused `completed_jobs` list (dead code after per-job update refactor)
- Wrap DB update in try/except to prevent one failed update from aborting remaining jobs
- Add test assertions verifying batch_processed, status, and file_object are written to DB
2026-03-06 09:25:50 -08:00
Ryan Crabbe
3d55f7f6ab Fix batch list showing stale "validating" status after completion
CheckBatchCost poller updated the status column but not the file_object
JSON column. The list_batches endpoint reads status from file_object,
so batches appeared stuck in "validating" even after Azure reported
them as completed. Now update file_object alongside status in the
per-job DB write.
2026-03-06 08:54:21 -08:00
yuneng-jiang
ba7a6d9bfd
Merge pull request #22476 from BerriAI/litellm_audit_pagination_fix
[Fix] UI - Audit Logs: Server-side pagination, filtering, and drawer view
2026-03-03 16:52:33 -08:00
yuneng-jiang
657a60ea5b fix(audit): AND semantics for combined JSON filters; remove unused allTeams prop
- Fix object_team_id + object_key_hash combining incorrectly as OR — each
  filter now adds an AND clause wrapping an internal OR over before_value
  and updated_values, so both conditions must be satisfied simultaneously
- Rename helper to _build_json_field_or_condition to reflect its purpose
- Remove allTeams from AuditLogsProps and its call site in index.tsx

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:24:55 -08:00
yuneng-jiang
705ef64ffc fix(ui): Audit logs - server-side pagination, filtering, and drawer view
- Replace client-side full-fetch loop with single react-query call using
  keepPreviousData; remove 5-second polling
- All filters (object ID, action, table, changed_by, team ID, key hash)
  now passed as query params to the backend
- Add object_team_id and object_key_hash params to /audit endpoint using
  Prisma JSON path filtering (PostgreSQL) to search inside before_value
  and updated_values JSON columns
- Migrate table from custom TanStack DataTable to AntD Table with
  server-side pagination
- Replace inline row expansion with a right-side AntD Drawer showing
  metadata and before/after diff
- Refactor uiAuditLogsCall to accept a structured options object

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:19:07 -08:00
Ephrim Stanley
35f6fd4223 Managed batches fixes for Gemini/Vertex 2026-02-28 20:21:04 -05:00
Ephrim Stanley
27a33565e7 State management fixes for CheckBatchCost - Address greptile comments 2026-02-23 07:52:58 -05:00
Ephrim Stanley
7b5dc3fb9c State management fixes for CheckBatchCost 2026-02-23 07:16:25 -05:00
Sameer Kankute
03f5717456 Fixes based on greptile reviews 2026-02-18 12:19:11 +05:30
Sameer Kankute
9f5580fddd Fixes based on greptile reviews 2026-02-18 11:55:06 +05:30
Sameer Kankute
8f80b1085e Add File deletion criteria with batch references 2026-02-18 11:39:32 +05:30
Sameer Kankute
791cef6d99 fix test_chat_completion 2026-02-17 20:26:28 +05:30
Ephrim Stanley
a3762e7d49 Addressed greptile comments to extract common helpers and return 404 2026-02-16 07:58:04 -05:00
Ephrim Stanley
7d794b567c fix: thread deployment model_info through batch cost calculation
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
2026-02-15 14:53:30 -05:00
Ephrim Stanley
a5626768a3 Add comments 2026-02-14 19:46:56 -05:00
Ephrim Stanley
4d87cb8fe3 Fix deleted managed files returning 403 instead of 404 2026-02-14 19:12:03 -05:00
Ephrim Stanley
e59c8d22af fix: afile_retrieve returns unified ID for batch output files 2026-02-14 10:07:12 -05:00
Ephrim Stanley
5433ae7e8c Fix: bypass managed files access check in batch polling by calling afile_content directly 2026-02-14 00:30:35 -05:00
Ephrim Stanley
358180eb2d Fix: pass deployment credentials to afile_retrieve in managed_files post-call hook 2026-02-14 00:16:34 -05:00
Sameer Kankute
aaf5938864
Merge pull request #21089 from BerriAI/litellm_vector_store_endpoints
Add target_model_names for vector store endpoints
2026-02-13 22:18:29 +05:30
Sameer Kankute
4ad0ecd9eb Fix mypy issues 2026-02-13 22:02:27 +05:30
Shivam Rawat
1321cd276b removed /models and v1/models from llm api routes (#20988) 2026-02-13 18:32:41 +05:30
Sameer Kankute
fa48166b10 Add _PROXY_LiteLLMManagedVectorStores class 2026-02-13 09:57:17 +05:30
Sameer Kankute
ce4bebbedf Changed asyncio.create_task() to await for storing batch objects 2026-02-10 12:42:39 +05:30
Sameer Kankute
9bdb163269 Add error file ids as managed files 2026-02-10 12:06:40 +05:30
Sameer Kankute
8b3213ce5c Add mapping for responses tools in file ids 2026-02-04 13:12:45 +05:30
Sameer Kankute
410e54648c Fix: Managed Batches: Inconsistent State Management for list and cancel batches 2026-02-03 14:47:28 +05:30
shin-bot-litellm
5434b66b9c
litellm_fix(mypy): fix remaining type errors (#20164)
- route_llm_request.py: add acancel_batch and afile_delete to route_type Literal
- router.py: add SearchToolInfoTypedDict and search_tool_info to SearchToolTypedDict
- gemini/files/transformation.py: fix validate_environment signature to match base class
- responses transformation.py: fix Dict type annotations to use int instead of Optional[int]
- vector_stores/endpoints.py: add team_id and user_id to LiteLLM_ManagedVectorStoresTable constructor

Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
2026-01-31 10:25:23 -08:00
Sameer Kankute
70684ca86f Fix File access permissions for .retreive and .delete 2026-01-29 11:19:24 +05:30
Sameer Kankute
833cf6a2cf Fix: Batch cancellation ownership bug 2026-01-29 10:54:42 +05:30
Sameer Kankute
42a0d576f3
Merge pull request #19910 from BerriAI/main
merge 01 27
2026-01-28 08:30:47 +05:30
michelligabriele
fc7a9b4cb0
fix(enterprise): correct error message for DISABLE_ADMIN_ENDPOINTS (#19861)
The error message for DISABLE_ADMIN_ENDPOINTS incorrectly said
"DISABLING LLM API ENDPOINTS is an Enterprise feature" instead of
"DISABLING ADMIN ENDPOINTS is an Enterprise feature".

This was a copy-paste bug from the is_llm_api_route_disabled() function.

Added regression tests to verify both error messages are correct.
2026-01-27 09:34:30 -08:00
Ephrim Stanley
88280d9cca Fix batch creation to return the input file's expires_at attribute 2026-01-26 12:02:35 -05:00
Ephrim Stanley
caa2c57619 Fix /batches to return encoded ids (from managed objects table) 2026-01-26 10:36:43 -05:00
Matthias Humt
9adc19deab
Normalize OpenAI SDK BaseModel choices/messages to avoid Pydantic serializer warnings (#18972)
* Normalize BaseModel choices + suppress serializer warnings

* Fix ModelResponse normalization and test deps
2026-01-14 03:40:11 +05:30
houdataali
cbb72045a3
fix(ui): use non-streaming method for endpoint v1/a2a/message/send in… (#19025)
* Add end to end integration tests for batches

* Add end to end integration tests for batches

* Add end to end integration tests for batches

* Fix linter errors: remove unused imports and variables

* Add end to end integration tests for batches

* Add end to end integration tests for batches

* Add end to end integration tests for batches

* Add end to end integration tests for batches

* chore: document temporary grype ignore for CVE-2019-1010022

* chore: add config option

* chore: add ALLOWED_CVES

* refetch after key create

* test: remove flaky azure oidc embedding test

* fixing build

* bump: version 1.80.15 → 1.80.16

* [Fix] MSFT SSO - allow setting custom MSFT Base URLs (#18977)

* fix TestCustomMicrosoftSSO

* init CustomMicrosoftSSO

* use CustomMicrosoftSSO

* docs fix

* docs fix

* [Feat] UI Feedback Form - why LiteLLM  (#18999)

* init survey prompt

* init survey modal

* init Survey Modal

* POST feedback hook

* survey Modal

* add other

* in product survey fixes

* fix survey prompt

* fix survey

* fix build

* ui new build

* [Feat] MSFT SSO - allow overriding env var attribute names  (#18998)

* add MSFT SSO constants

* fix MSFT SSO env vars

* test_microsoft_sso_handler_openid_from_response_with_custom_attributes

* Add pricing of azure_ai/claude-opus-4-5

* test: temporarily disable flaky responses_id_security tests

* fix(ui): use non-streaming method for endpoint v1/a2a/message/send in A2A playground

'

---------

Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com>
Co-authored-by: Yuta Saito <uc4w6c@bma.biglobe.ne.jp>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
2026-01-14 03:29:10 +05:30
Sameer Kankute
7d0f41f437 Add cost tracking for responses api in background mode 2025-12-19 13:35:48 +05:30
Sameer Kankute
755f024087 remove print statments 2025-12-16 21:37:09 +05:30
Sameer Kankute
3222b4e3a8 Add batch output file in managed table 2025-12-16 15:44:30 +05:30
Sameer Kankute
7b1cef86a7 Add support for target_storage param 2025-12-11 15:08:17 +05:30
Sameer Kankute
99fd96687f Fix vector store configuration synchronization failure 2025-12-05 11:46:14 +05:30
Ishaan Jaff
4370f6fb74
[Feat] Agent Gateway - Allow invoking agents through AI Gateway (#17440)
* init litellm A2a client

* simpler a2a client interface

* test a2a

* move a2a invoking tests

* test fix

* ensure a2a send message is tracked n logs

* rename tags

* add streaming handlng

* add a2a invocation
2025-12-03 18:54:55 -08:00
Ishaan Jaff
1cdfb3da8f
[Bug Fix] - Fix litellm_enterprise ensure imported routes exist (#17337)
* test_enterprise_routes.py

* test_enterprise_routes_all_imports_exist
2025-12-01 19:14:12 -08:00
Ishaan Jaff
38ddd50628
[Bug fix] Vector Store List Endpoint Returns 404 (#17229)
* fix vector store management

* fix: add vector_store_management_router

* TestVectorStoreManagementEndpointsExist

* fix pass_through
2025-11-27 12:51:51 -08:00
Krish Dholakia
70e1e83102
feat(managed_files.py): support /delete for files + feat(managed_batches): support /cancel for batches (#16387)
* feat(managed_files.py): initial commit fixing managed file delete on litellm

* fix(managed_files.py): fix file delete

* feat(batches_endpoints/endpoints.py): fix cancelling a batch

ensures managed batches works

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
2025-11-18 17:36:26 -08:00
Sameer Kankute
7cebc151b0
Add managed files support for responses API (#16733)
* Fix responses api with managed files

* fix litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py mypy

* fix litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py mypy

* fix mypy errors
2025-11-17 18:41:26 -08:00