[JS] Forced function calling does not work for the Vertex AI plugin

[couchrishi]

Describe the bug
This is related to this FR #361. While we now have functionCallingConfig with support. for "ANY" mode, it seems to work only for Google Gemini, not Vertex AI Gemini models.

This allows the developer to choose whether the LLM output should only contain function calls, or if the LLM should decide whether to return text or a function call.

To Reproduce
Steps to reproduce the behavior. Code samples.

Step-0: Initialize Genkit with VertexAI plugin
// Initialize Genkit
const ai = genkit({
plugins: [
vertexAI({
projectId: ‘*****’,
location: 'us-central1',
}),
],
});

Step-1 : Get the current session details of the user from Firestore (or) Json session store
const session = await getSession(sessionId, network);

Step-2: Load the current sessions and initialize it with the LLM primitives along with forced function calling
const session = await getSession(sessionId, network);

const chat = session.chat({
  model: gemini15Pro,
  system: SYSTEM_PROMPT,
  tools: [approveTransfer, rejectTransfer], // Tools for LLM
  config: {
    functionCallingConfig: {
      mode: "ANY"
    }
  }

});

Step-3 Generate an LLM response & also call tool for every single query

const chat = session.chat({
  model: gemini15Pro,
  system: SYSTEM_PROMPT,
  tools: [approveTransfer, rejectTransfer], // Tools for LLM
  config: {
    functionCallingConfig: {
      mode: "ANY"
    }
  }

});

Expected behavior
The LLM is expected to invoke the tool for every request. This doesn't happen for Vertex AI Gemini. It does call the tool every time for Google Gemini.

Runtime (please complete the following information):

• OS: Linux, MacOS [e.g. Linux, MacOS]

** Node version
v20.18.0

[pavelgj]

I've been testing this, trying to reproduce the issue.
I've verified that the Genkit is passing the functionCallingConfig to the Vertex SDK correctly, but it looks like there's a bug in the Vertex AI SDK. I send a fix there... I'll update here when they push the fix.