litellm/test_script.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

20 lines
619 B
Python
Raw Normal View History

Xai, VertexAI, Google AI Studio - live web search support in OpenAI format (#11251) * build(model_prices_and_context_window.json): fix 'supports_web_search' flag - openai only supports it on 2 models - gpt-4o-search-preview and gpt-4o-mini-search-preview * feat(xai/chat): add xai web search options param support * test: add max tokens to test xai output very verbose * build(xai/): add web search support for all xai models * build(model_prices_and_cost.json): add gemini-2.0 supports web search * feat(gemini/): map openai 'web_search_options' to google's 'googlesearch' tool * build(model_prices_and_context_window.json): add supports_web_search for vertex_ai/gemini-2 models * fix: fix circular reference error * fix(convert_dict_to_response.py): handle scenario where xai returns finish reason as 'stop' for tool calls * fix: reduce function size * fix: import session handling * Revert "fix: import session handling" This reverts commit deb257dc107fc72bdee932953ee803023b73c838. * fix: linting pin mypy * [Feat]: Guardrails - Add streaming for bedrock post guard (#11247) * feat: add streaming for bedrock post guard * fix: bedrock guardrails * fix: add clear comments * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: clean up bedrock guardrails --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [Fix] Responses API - Session management (#11254) * fix: import session handling * fix: imports for session handler * tests: tests for session handler * Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * bump: bump litellm enterprise * fixes: test_create_user_default_budget * fix(xai/): filter 'strict' on tool call * test: update test for new error string * fix(utils.py): default to None if not set in model cost map ensures consistent usage of 'supports_[x]' flags * fix(fireworks_ai/): support fireworks ai document inlining on pdf's sent via openai 'file' message type * test: update test * test: name filter_value_from_dict * fix(fireworks_ai/): handle cache control flag in messages * fix(xai/chat): fix check --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-01 05:26:16 +08:00
import json
mistral_model_cost_map = json.load(open("model_prices_and_context_window.json"))
Xai, VertexAI, Google AI Studio - live web search support in OpenAI format (#11251) * build(model_prices_and_context_window.json): fix 'supports_web_search' flag - openai only supports it on 2 models - gpt-4o-search-preview and gpt-4o-mini-search-preview * feat(xai/chat): add xai web search options param support * test: add max tokens to test xai output very verbose * build(xai/): add web search support for all xai models * build(model_prices_and_cost.json): add gemini-2.0 supports web search * feat(gemini/): map openai 'web_search_options' to google's 'googlesearch' tool * build(model_prices_and_context_window.json): add supports_web_search for vertex_ai/gemini-2 models * fix: fix circular reference error * fix(convert_dict_to_response.py): handle scenario where xai returns finish reason as 'stop' for tool calls * fix: reduce function size * fix: import session handling * Revert "fix: import session handling" This reverts commit deb257dc107fc72bdee932953ee803023b73c838. * fix: linting pin mypy * [Feat]: Guardrails - Add streaming for bedrock post guard (#11247) * feat: add streaming for bedrock post guard * fix: bedrock guardrails * fix: add clear comments * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: clean up bedrock guardrails --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [Fix] Responses API - Session management (#11254) * fix: import session handling * fix: imports for session handler * tests: tests for session handler * Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * bump: bump litellm enterprise * fixes: test_create_user_default_budget * fix(xai/): filter 'strict' on tool call * test: update test for new error string * fix(utils.py): default to None if not set in model cost map ensures consistent usage of 'supports_[x]' flags * fix(fireworks_ai/): support fireworks ai document inlining on pdf's sent via openai 'file' message type * test: update test * test: name filter_value_from_dict * fix(fireworks_ai/): handle cache control flag in messages * fix(xai/chat): fix check --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-01 05:26:16 +08:00
for model, model_info in mistral_model_cost_map.items():
if (
(model_info.get("litellm_provider") == "mistral")
and model_info.get("mode") == "chat"
and ("codestral-mamba" not in model)
):
Xai, VertexAI, Google AI Studio - live web search support in OpenAI format (#11251) * build(model_prices_and_context_window.json): fix 'supports_web_search' flag - openai only supports it on 2 models - gpt-4o-search-preview and gpt-4o-mini-search-preview * feat(xai/chat): add xai web search options param support * test: add max tokens to test xai output very verbose * build(xai/): add web search support for all xai models * build(model_prices_and_cost.json): add gemini-2.0 supports web search * feat(gemini/): map openai 'web_search_options' to google's 'googlesearch' tool * build(model_prices_and_context_window.json): add supports_web_search for vertex_ai/gemini-2 models * fix: fix circular reference error * fix(convert_dict_to_response.py): handle scenario where xai returns finish reason as 'stop' for tool calls * fix: reduce function size * fix: import session handling * Revert "fix: import session handling" This reverts commit deb257dc107fc72bdee932953ee803023b73c838. * fix: linting pin mypy * [Feat]: Guardrails - Add streaming for bedrock post guard (#11247) * feat: add streaming for bedrock post guard * fix: bedrock guardrails * fix: add clear comments * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: clean up bedrock guardrails --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [Fix] Responses API - Session management (#11254) * fix: import session handling * fix: imports for session handler * tests: tests for session handler * Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * bump: bump litellm enterprise * fixes: test_create_user_default_budget * fix(xai/): filter 'strict' on tool call * test: update test for new error string * fix(utils.py): default to None if not set in model cost map ensures consistent usage of 'supports_[x]' flags * fix(fireworks_ai/): support fireworks ai document inlining on pdf's sent via openai 'file' message type * test: update test * test: name filter_value_from_dict * fix(fireworks_ai/): handle cache control flag in messages * fix(xai/chat): fix check --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-01 05:26:16 +08:00
"""
Update all mistral models to supports_response_schema
Xai, VertexAI, Google AI Studio - live web search support in OpenAI format (#11251) * build(model_prices_and_context_window.json): fix 'supports_web_search' flag - openai only supports it on 2 models - gpt-4o-search-preview and gpt-4o-mini-search-preview * feat(xai/chat): add xai web search options param support * test: add max tokens to test xai output very verbose * build(xai/): add web search support for all xai models * build(model_prices_and_cost.json): add gemini-2.0 supports web search * feat(gemini/): map openai 'web_search_options' to google's 'googlesearch' tool * build(model_prices_and_context_window.json): add supports_web_search for vertex_ai/gemini-2 models * fix: fix circular reference error * fix(convert_dict_to_response.py): handle scenario where xai returns finish reason as 'stop' for tool calls * fix: reduce function size * fix: import session handling * Revert "fix: import session handling" This reverts commit deb257dc107fc72bdee932953ee803023b73c838. * fix: linting pin mypy * [Feat]: Guardrails - Add streaming for bedrock post guard (#11247) * feat: add streaming for bedrock post guard * fix: bedrock guardrails * fix: add clear comments * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: clean up bedrock guardrails --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [Fix] Responses API - Session management (#11254) * fix: import session handling * fix: imports for session handler * tests: tests for session handler * Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * bump: bump litellm enterprise * fixes: test_create_user_default_budget * fix(xai/): filter 'strict' on tool call * test: update test for new error string * fix(utils.py): default to None if not set in model cost map ensures consistent usage of 'supports_[x]' flags * fix(fireworks_ai/): support fireworks ai document inlining on pdf's sent via openai 'file' message type * test: update test * test: name filter_value_from_dict * fix(fireworks_ai/): handle cache control flag in messages * fix(xai/chat): fix check --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-01 05:26:16 +08:00
"""
model_info["supports_response_schema"] = True
print(f"Updated {model} to support response schema")
Xai, VertexAI, Google AI Studio - live web search support in OpenAI format (#11251) * build(model_prices_and_context_window.json): fix 'supports_web_search' flag - openai only supports it on 2 models - gpt-4o-search-preview and gpt-4o-mini-search-preview * feat(xai/chat): add xai web search options param support * test: add max tokens to test xai output very verbose * build(xai/): add web search support for all xai models * build(model_prices_and_cost.json): add gemini-2.0 supports web search * feat(gemini/): map openai 'web_search_options' to google's 'googlesearch' tool * build(model_prices_and_context_window.json): add supports_web_search for vertex_ai/gemini-2 models * fix: fix circular reference error * fix(convert_dict_to_response.py): handle scenario where xai returns finish reason as 'stop' for tool calls * fix: reduce function size * fix: import session handling * Revert "fix: import session handling" This reverts commit deb257dc107fc72bdee932953ee803023b73c838. * fix: linting pin mypy * [Feat]: Guardrails - Add streaming for bedrock post guard (#11247) * feat: add streaming for bedrock post guard * fix: bedrock guardrails * fix: add clear comments * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: clean up bedrock guardrails --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [Fix] Responses API - Session management (#11254) * fix: import session handling * fix: imports for session handler * tests: tests for session handler * Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * bump: bump litellm enterprise * fixes: test_create_user_default_budget * fix(xai/): filter 'strict' on tool call * test: update test for new error string * fix(utils.py): default to None if not set in model cost map ensures consistent usage of 'supports_[x]' flags * fix(fireworks_ai/): support fireworks ai document inlining on pdf's sent via openai 'file' message type * test: update test * test: name filter_value_from_dict * fix(fireworks_ai/): handle cache control flag in messages * fix(xai/chat): fix check --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-01 05:26:16 +08:00
json.dump(
mistral_model_cost_map, open("model_prices_and_context_window.json", "w"), indent=4
)