* fix: support Anthropic tool_use and tool_result in token counter
* refactor(token_counter): add dynamic field inference for Anthropic content blocks
* test: Add additional tests
* make format
* Fix lint error
* Fix mypy narrow type lint errors
Implements support for reasoning_effort="none" parameter for Gemini models,
providing significant cost savings (up to 96% cheaper) by disabling thinking
budget while maintaining response quality.
Changes:
- Added "supports_reasoning": true to gemini-2.0-flash-thinking-exp-01-21 in model config
- Implemented mapping for reasoning_effort="none" to thinkingConfig {thinkingBudget: 0, includeThoughts: false}
- Added unit test to verify the mapping works correctly
Performance impact:
- Without reasoning_effort: ~313 tokens
- With reasoning_effort="none": ~12 tokens (96% cheaper)
Closes#16420
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Related to PR #16210 which fixed automatic summary field addition
Changes:
- Document reasoning_effort string vs dict formats
- Add summary field options (auto, detailed, concise)
- Add table of supported reasoning_effort values by GPT-5 model
- Clarify model-specific support and limitations
- Note that summary field requires org verification
The previous implementation automatically added summary field causing
400 errors for unverified orgs. Now users can opt-in by passing
reasoning_effort as dict with explicit summary field.
Fixes#16533
Before this fix, non-ASCII characters (Japanese, Spanish, Chinese, etc.)
in function call arguments were being escaped as Unicode sequences.
Example:
- Before: "やあ" → "\u3084\u3042"
- After: "やあ" → "やあ" (preserved)
Changes:
- Add ensure_ascii=False to json.dumps() in _transform_parts()
- Add test for Japanese and Spanish Unicode character preservation
This is not a breaking change as both formats are equivalent in JSON.
The fix improves readability and aligns with OpenAI's behavior.
* fix: Improve Azure auth parameter handling for None values
Previously, litellm_params.get() with default fallbacks could ignore
environment variables when the param existed but was None. Now explicitly
checks for None values before falling back to environment variables.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix lint
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix(passthrough): use VLLM passthrough config for hosted VLLM provider instead of raising an error
* test(passthrough): add tests for hosted VLLM passthrough configuration and routing
* fix account linking for CLA
Fixes#16486
## Problem
Callbacks configured via litellm.success_callback (e.g., PostHog, LangSmith)
were not being invoked for litellm.acompletion() in short-lived scripts.
The callbacks worked correctly for synchronous completions but async
completions would queue callbacks that were lost when the script exited
before GLOBAL_LOGGING_WORKER could process them.
Root cause: asyncio.run() closes the event loop immediately after the
async function completes, preventing the background worker from processing
queued callbacks.
## Solution
Implemented a two-level atexit handler approach:
1. GLOBAL_LOGGING_WORKER atexit handler (logging_worker.py):
- Processes remaining callbacks from queue before exit
- Creates new event loop to run pending coroutines synchronously
- Applies time and iteration limits to prevent blocking shutdown
2. Integration-specific atexit handlers (posthog.py as example):
- Flushes internal queue to external service
- Uses synchronous HTTP client for reliable delivery
- Each integration needs its own handler due to varying sync APIs
## Changes
- litellm/litellm_core_utils/logging_worker.py:
- Added _flush_on_exit() method
- Registered atexit handler in __init__
- Processes up to MAX_ITERATIONS_TO_CLEAR_QUEUE events
- Time-limited to MAX_TIME_TO_CLEAR_QUEUE seconds
- litellm/integrations/posthog.py:
- Added _flush_on_exit() method
- Registered atexit handler in __init__
- Groups events by credentials for batch sending
- Uses sync_client for reliable HTTP delivery
- tests/logging_callback_tests/test_posthog.py:
- Added test_async_callback_atexit_handler_exists()
- Added test_posthog_atexit_flushes_internal_queue()
- Added test_sync_callback_not_affected_by_atexit()
## Testing
- All existing tests pass
- Manual end-to-end testing confirms fix:
- Async events now arrive in PostHog
- Sync events continue working (no regression)
- Unit tests verify atexit handlers registered and functional
## Impact
- Fixes async callback delivery for ALL integrations using GLOBAL_LOGGING_WORKER
- No breaking changes - only adds missing functionality
- Sync path unchanged - no performance impact
* Update quick_start.md
changed -D to -d
* Update users.md
Changed in a number of locations:
"budget_duration": 10s,
to
"budget_duration": "10s",
* Update users.md
Changed all 10s to 30s to keep in line with the example
* Use end user budget instead of key budget when creating new team
* Fixed implementation to use user's max budget from the UserTable instead of EndUserTable
- Remove enterprise-only label from 'View Available Guardrails' section
- The /guardrails/list endpoint appears to be available in OSS version
- Makes documentation more accurate for OSS users
