Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Include a function_invoke_attempt index with Streaming CMC #10009

Merged
merged 8 commits into from
Dec 19, 2024

Conversation

moonbox3
Copy link
Contributor

@moonbox3 moonbox3 commented Dec 18, 2024

Motivation and Context

During auto function calling, we're yielding all messages back without any indication as to which invocation index they are related to. This information could be helpful to the caller to understand in which order message chunks were received during the auto function invocation loop.

Depending upon the behavior of auto function calling, the request_index iterates up to the maximum_auto_invoke_attempts. The caller doesn't know today which function auto invoke attempt they're currently on -- so simply handing all yielded messages can be confusing. In a new PR, we will handle adding the request_index (perhaps with a different name) to make it easier to know which streaming message chunks to concatenate, which should help reduce the confusion down the line.

Description

This PR adds:

  • The function_invoke_attempt attribute to the StreamingChatMessageContent class. This can help callers/users track which streaming chat message chunks belong to which auto function invocation attempt.
  • A new keyword argument was added to the _inner_get_streaming_chat_message_contents to allow the function_invoke_attempt int to be passed through to the StreamingChatMessageContent creation in each AI Service. This additive keyword argument should not break existing.
  • Updates unit tests
  • Creates four new samples to showcase auto function calling: streaming auto invoke / manual invoke (print tool calls), and non-streaming auto invoke / manual invoke (print tool calls). These samples allow one to specify the AI service that supports function calling, as listed in the samples.
  • Closes Python: add request index to streaming chat message content to track invocation count #10006

Contribution Checklist

@moonbox3 moonbox3 self-assigned this Dec 18, 2024
@moonbox3 moonbox3 requested a review from a team as a code owner December 18, 2024 10:45
@markwallace-microsoft markwallace-microsoft added the python Pull requests for the Python Semantic Kernel label Dec 18, 2024
@markwallace-microsoft
Copy link
Member

markwallace-microsoft commented Dec 18, 2024

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
semantic_kernel/connectors/ai
   chat_completion_client_base.py127298%406, 416
semantic_kernel/connectors/ai/anthropic/services
   anthropic_chat_completion.py163895%157, 163, 176, 182, 186, 243–245, 378
semantic_kernel/connectors/ai/azure_ai_inference/services
   azure_ai_inference_chat_completion.py105694%110–113, 122, 144, 168
semantic_kernel/connectors/ai/bedrock/services
   bedrock_chat_completion.py1361490%117, 139, 164, 168–171, 229, 247–266, 325
semantic_kernel/connectors/ai/google/google_ai/services
   google_ai_chat_completion.py119497%126, 153, 179, 181
semantic_kernel/connectors/ai/google/vertex_ai/services
   vertex_ai_chat_completion.py119497%121, 148, 174, 176
semantic_kernel/connectors/ai/mistral_ai/services
   mistral_ai_chat_completion.py1223869%119–122, 132, 147–150, 165, 181–185, 200–208, 225–233, 246–259, 265, 274–278, 323–326
semantic_kernel/connectors/ai/ollama/services
   ollama_chat_completion.py1383475%116, 141, 145–146, 156, 169, 186, 206–207, 211, 224–234, 245–247, 258–267, 279, 289–290, 312, 323–324, 350, 359–367
semantic_kernel/connectors/ai/onnx/services
   onnx_gen_ai_chat_completion.py72692%69–70, 100, 126, 174, 180
semantic_kernel/connectors/ai/open_ai/services
   open_ai_chat_completion_base.py127794%71, 81, 102, 122, 143, 179, 291
semantic_kernel/contents
   streaming_chat_message_content.py70199%225
TOTAL16797185189% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
2968 4 💤 0 ❌ 0 🔥 1m 5s ⏱️

Copy link
Member

@eavanvalkenburg eavanvalkenburg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some big questions for you :D

@TaoChenOSU
Copy link
Contributor

Create an item to track this issue for .Net: #10017

@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Dec 19, 2024
@moonbox3 moonbox3 removed this pull request from the merge queue due to a manual request Dec 19, 2024
@moonbox3 moonbox3 enabled auto-merge December 19, 2024 07:40
Copy link
Member

@eavanvalkenburg eavanvalkenburg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of nits on the samples, feel free to ignore

@moonbox3 moonbox3 added this pull request to the merge queue Dec 19, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 19, 2024
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Dec 19, 2024
Merged via the queue into microsoft:main with commit 36701a2 Dec 19, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python Pull requests for the Python Semantic Kernel
Projects
Status: Sprint: Done
Development

Successfully merging this pull request may close these issues.

Python: add request index to streaming chat message content to track invocation count
4 participants