Joe Cheng
33f4411f17
Fix tool call coalescing
...
The previous code seemed to assume that the tool call index property
started at 0, but Anthropic sometimes returns them starting at 1.
This was causing an extra null-ish tool call to be materialized.
2024-08-02 13:05:23 -07:00
Joe Cheng
90dd60fa71
fix(main.py): Handle bedrock tool calling in stream_chunk_builder
...
Fixes #5022 .
The streaming chunks from Anthropic seem to violate an assumption
that is implicit in the stream_chunk_builder implementation: that
only tool_calls OR function_calls OR content will appear in a
streamed response. The repro in #5022 shows that you can get
content followed by tool calls.
These changes properly handle these combinations by building
separate lists of each type of chunk (note that in theory a chunk
could appear in multiple lists, e.g. both delta.tool_calls and
delta.content being present on one chunk).
2024-08-02 12:41:13 -07:00
Ishaan Jaff
7f1d32a75b
use regular ci/cd pipeline
2024-08-02 12:13:15 -07:00
Ishaan Jaff
fc8a87efec
ci/cd run again
2024-08-02 11:46:45 -07:00
Ishaan Jaff
ffa7818339
fix config.yaml
2024-08-02 11:41:53 -07:00
Ishaan Jaff
459964decf
fix config
2024-08-02 11:39:54 -07:00
Ishaan Jaff
53d57509ab
bump: version 1.42.10 → 1.42.11
2024-08-02 11:29:56 -07:00
Ishaan Jaff
d42549d6e4
temp testing ci/cd
2024-08-02 11:29:41 -07:00
Ishaan Jaff
b3196df0b8
queue stable release testing after new GH release
2024-08-02 10:57:56 -07:00
Krrish Dholakia
0ce6c038ee
fix(anthropic.py): fix linting error
2024-08-02 10:37:53 -07:00
Krish Dholakia
70c45767c2
Merge pull request #5020 from BerriAI/litellm_openai_raw_response_headers
...
refactor(openai/azure.py): move to returning openai/azure response headers by default
2024-08-02 10:36:14 -07:00
Krrish Dholakia
fe2aa706e8
refactor(openai/azure.py): move to returning openai/azure response headers by default
...
Allows token tracking to work more reliably across multiple azure/openai deployments
2024-08-02 09:42:08 -07:00
Krrish Dholakia
0a30ba9674
fix(types/utils.py): support passing prompt cache usage stats in usage object
...
Passes deepseek prompt caching values through to end user
2024-08-02 09:30:50 -07:00
Krrish Dholakia
cd073d5ad3
test: handle anthropic rate limit error
2024-08-02 08:57:09 -07:00
Haadi Rakhangi
5439e72a6b
Merge branch 'BerriAI:main' into main
2024-08-02 21:08:48 +05:30
Haadi Rakhangi
851db5ecea
qdrant semantic caching added
2024-08-02 21:07:19 +05:30
Ishaan Jaff
48419ca8bc
add step to ghcr deploy
2024-08-02 08:06:09 -07:00
Krrish Dholakia
fe7e68adc8
fix(utils.py): fix codestral streaming
2024-08-02 07:38:06 -07:00
Ishaan Jaff
75fba18c9f
fix langfuse hardcoded public key
2024-08-02 07:21:02 -07:00
Krrish Dholakia
4c2ef8ea64
fix(bedrock_httpx.py): fix ai21 streaming
2024-08-01 22:03:24 -07:00
Krrish Dholakia
57e3044974
bump: version 1.42.9 → 1.42.10
2024-08-01 21:29:36 -07:00
Krish Dholakia
13337bca57
Merge pull request #5001 from BerriAI/litellm_fix_streaming_usage_calc
...
fix(utils.py): Add streaming token usage in hidden params
2024-08-01 21:29:10 -07:00
Krish Dholakia
25ac9c2d75
Merge branch 'main' into litellm_fix_streaming_usage_calc
2024-08-01 21:29:04 -07:00
Krish Dholakia
6bdc443157
Merge pull request #4996 from BerriAI/litellm_response_cost_logging
...
fix(litellm_logging.py): Fix azure base model cost calc in response headers
2024-08-01 21:28:34 -07:00
Krish Dholakia
e7f83602d8
Merge branch 'main' into litellm_response_cost_logging
2024-08-01 21:28:22 -07:00
Krrish Dholakia
d096c6c0ee
fix(langfuse.py): cleanup
2024-08-01 21:24:41 -07:00
Krish Dholakia
d8778380d8
Merge pull request #5004 from BerriAI/litellm_codestral_fim_support
...
feat(vertex_ai_partner.py): add vertex ai codestral FIM support
2024-08-01 21:24:12 -07:00
Ishaan Jaff
f5b8ecbcc3
docs spend tracking enteprrise
2024-08-01 20:52:23 -07:00
Krrish Dholakia
4355e923e5
fix(langfuse.py): cleanup
2024-08-01 20:46:29 -07:00
Ishaan Jaff
7f93fa01e9
ci/cd run again
2024-08-01 19:55:12 -07:00
Ishaan Jaff
85ced920a0
fix test traceloop.py
2024-08-01 19:49:53 -07:00
Krrish Dholakia
f506eb341b
feat(litellm_logging.py): log exception response headers to langfuse
2024-08-01 18:07:47 -07:00
Ishaan Jaff
17d007d6da
Update prompt_injection.md
2024-08-01 18:07:38 -07:00
Ishaan Jaff
c09d95d3d3
ci/cd - anyscale discontinued their API endoints - skip test
2024-08-01 18:07:38 -07:00
Ishaan Jaff
69d58b08eb
fix model prices formatting
2024-08-01 18:07:38 -07:00
David Manouchehri
6bf2166f93
Add new model for gemini-1.5-pro-exp-0801.
2024-08-01 18:07:38 -07:00
Ishaan Jaff
3d1cdd5374
bump: version 1.42.8 → 1.42.9
2024-08-01 18:07:38 -07:00
Ishaan Jaff
673a991da2
fix test_traceparent_not_added_by_default
2024-08-01 18:07:38 -07:00
Ishaan Jaff
2029e0607c
docs dbrx
2024-08-01 18:07:38 -07:00
Ishaan Jaff
f89a0427df
docs add new dbrx models
2024-08-01 18:07:38 -07:00
Ishaan Jaff
42bf2bb551
add correct context window
2024-08-01 18:07:38 -07:00
Ishaan Jaff
ff27e5813b
add new dbrx models
2024-08-01 18:07:38 -07:00
Ishaan Jaff
ee9b9623ab
docs gcs buckets
2024-08-01 18:07:38 -07:00
lowjiansheng
f02325bf76
update helm chart
2024-08-01 18:07:38 -07:00
Ishaan Jaff
026e380b92
enforce premium user cheks on gcs bucket
2024-08-01 18:07:38 -07:00
Ishaan Jaff
950f803035
docs logging to GCS
2024-08-01 18:07:38 -07:00
Ishaan Jaff
7f3dd3072c
docs setting service accounts
2024-08-01 18:07:38 -07:00
Ishaan Jaff
6c899ced28
init gcs using gcs_bucket
2024-08-01 18:07:38 -07:00
Ishaan Jaff
1595dc4dc6
docs using gcs
2024-08-01 18:07:38 -07:00
Ishaan Jaff
781bfd1517
fix type errors
2024-08-01 18:07:38 -07:00
