Vercel AI SDK Streaming Response Stuck in Loop

[troycampano]

Description

When attempting to use the Vercel AI SDK for the first time, with a very specific prompt, the response seems to get stuck in a loop. When using the normal REST blocking response, the program runs and never finishes. When using the streamed response, you can see it appears to be stuck in a loop on this particular person, Kurt.

I've included videos of this running from the command line as well a React app using an express backend (easier to visual the response that is coming back from the stream). If I change the system prompt, I get a different answer and don't run into this issue. The issue appears to be specific to this name.

You can see the details in the code, but this is using the gpt-4o-mini model from Azure OpenAI.

Screen.Recording.2024-12-18.at.10.36.45.AM.mov

Screen.Recording.2024-12-18.at.10.31.59.AM.mov

The code below can be used to reproduce the issue, assuming you have access to the same Azure OpenAI model:

Code example

import { createAzure } from '@ai-sdk/azure';
import { generateText,streamText } from 'ai'

let apiKey = process.env.OPENAI_API_KEY ? process.env.OPENAI_API_KEY : ''

const azure = createAzure({
    resourceName: 'openai-sandbox', // Azure resource name
    apiKey: apiKey,
    apiVersion: '2024-08-01-preview',

});

async function runProgramStream() {
     
    const { textStream } = streamText({
        model: azure('gpt-4o-mini'),
        //system: 'You are a helpful AI assistant.',
        prompt: 'Who is the founder of Intranet Software Unlimited whose first name started with "Kurt"?',
      });

      console.log(textStream)
      
      for await (const textPart of textStream) {
        console.log(textPart);
      } 
 }

runProgramStream()

AI provider

Azure OpenAI

Additional context

No response