ChatVertexAI integration

Integrate with the ChatVertexAI chat model using LangChain JavaScript.

Google Vertex is a service that exposes all foundation models available in Google Cloud, like gemini-2.5-pro, gemini-2.5-flash, etc. It also provides some non-Google models such as Anthropic’s Claude.

This will help you getting started with ChatVertexAI chat models. For detailed documentation of all ChatVertexAI features and configurations head to the API reference.

This library will be deprecated

This library will be replaced by the ChatGoogle library. New implementations should use the ChatGoogle library instead and existing implementations should consider migrating.

Overview

Integration details

Class	Package	Serializable	PY support	Downloads	Version
ChatVertexAI	@langchain/google-vertexai	✅	✅

Model features

See the links in the table headers below for guides on how to use specific features.

Tool calling	Structured output	Image input	Audio input	Video input	Token-level streaming	Token usage	Logprobs
✅	✅	✅	✅	✅	✅	✅	✅

Note that while logprobs are supported, Gemini has fairly restricted usage of them.

Setup

LangChain.js supports two different authentication methods based on whether you’re running in a Node.js environment or a web environment. It also supports the authentication method used by Vertex AI Express Mode using either package.

To access ChatVertexAI models you’ll need to setup Google VertexAI in your Google Cloud Platform (GCP) account, save the credentials file, and install the @langchain/google-vertexai integration package. On Node.js, that package uses @langchain/google-gauth for authentication (you do not need to install it separately).

Credentials

Head to your GCP account and generate a credentials file. Once you’ve done this set the GOOGLE_APPLICATION_CREDENTIALS environment variable:

export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/credentials.json"

Alternatively, on your local machine you can run gcloud auth application-default login to use Application Default Credentials.

If running in a web environment, install the @langchain/google-vertexai-web package (which uses @langchain/google-webauth for authentication). Set service account JSON in GOOGLE_WEB_CREDENTIALS:

export GOOGLE_WEB_CREDENTIALS='{"type":"service_account","project_id":"YOUR_PROJECT-12345",...}'

GOOGLE_VERTEX_AI_WEB_CREDENTIALS is also supported but deprecated.

If you are using Vertex AI Express Mode, you can install either the @langchain/google-vertexai or @langchain/google-vertexai-web package. You can then go to the Express Mode API Key page and set your API Key in the GOOGLE_API_KEY environment variable:

export GOOGLE_API_KEY="api_key_value"

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGSMITH_TRACING="true"
# export LANGSMITH_API_KEY="your-api-key"

Installation

The LangChain ChatVertexAI integration lives in the @langchain/google-vertexai package:

npm

npm install @langchain/google-vertexai @langchain/core

yarn add @langchain/google-vertexai @langchain/core

pnpm add @langchain/google-vertexai @langchain/core

Or if using in a web environment like a Vercel Edge function:

npm

npm install @langchain/google-vertexai-web @langchain/core

yarn add @langchain/google-vertexai-web @langchain/core

pnpm add @langchain/google-vertexai-web @langchain/core

Instantiation

Now we can instantiate our model object and generate chat completions:

import { ChatVertexAI } from "@langchain/google-vertexai"
// Uncomment the following line if you're running in a web environment:
// import { ChatVertexAI } from "@langchain/google-vertexai-web"

const llm = new ChatVertexAI({
    model: "gemini-2.5-flash",
    temperature: 0,
    maxRetries: 2,
    // For web, authOptions.credentials
    // authOptions: { ... }
    // other params...
})

Invocation

const aiMsg = await llm.invoke([
    [
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ],
    ["human", "I love programming."],
])
aiMsg

AIMessageChunk {
  "content": "J'adore programmer. \n",
  "additional_kwargs": {},
  "response_metadata": {},
  "tool_calls": [],
  "tool_call_chunks": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 20,
    "output_tokens": 7,
    "total_tokens": 27
  }
}

console.log(aiMsg.content)

J'adore programmer.

Tool calling with Google search retrieval

It is possible to call the model with a Google search tool which you can use to ground content generation with real-world information and reduce hallucinations.

Grounding is currently not supported by gemini-2.0-flash-exp.

You can choose to either ground using Google Search or by using a custom data store. Here are examples of both:

Google search retrieval

Grounding example that uses Google Search:

import { ChatVertexAI } from "@langchain/google-vertexai"

const searchRetrievalTool = {
  googleSearchRetrieval: {
    dynamicRetrievalConfig: {
      mode: "MODE_DYNAMIC", // Use Dynamic Retrieval
      dynamicThreshold: 0.7, // Default for Dynamic Retrieval threshold
    },
  },
};

const searchRetrievalModel = new ChatVertexAI({
  model: "gemini-2.5-pro",
  temperature: 0,
  maxRetries: 0,
}).bindTools([searchRetrievalTool]);

const searchRetrievalResult = await searchRetrievalModel.invoke("Who won the 2024 NBA Finals?");

console.log(searchRetrievalResult.content);

The Boston Celtics won the 2024 NBA Finals, defeating the Dallas Mavericks 4-1 in the series to claim their 18th NBA championship. This victory marked their first title since 2008 and established them as the team with the most NBA championships, surpassing the Los Angeles Lakers' 17 titles.

Google search retrieval with data store

First, set up your data store (this is a schema of an example data store):

ID	Date	Team 1	Score	Team 2
3001	2023-09-07	Argentina	1 - 0	Ecuador
3002	2023-09-12	Venezuela	1 - 0	Paraguay
3003	2023-09-12	Chile	0 - 0	Colombia
3004	2023-09-12	Peru	0 - 1	Brazil
3005	2024-10-15	Argentina	6 - 0	Bolivia

Then, use this data store in the example provided below:

(Note that you have to use your own variables for projectId and datastoreId)

import { ChatVertexAI } from "@langchain/google-vertexai";

const projectId = "YOUR_PROJECT_ID";
const datastoreId = "YOUR_DATASTORE_ID";

const searchRetrievalToolWithDataset = {
  retrieval: {
    vertexAiSearch: {
      datastore: `projects/${projectId}/locations/global/collections/default_collection/dataStores/${datastoreId}`,
    },
    disableAttribution: false,
  },
};

const searchRetrievalModelWithDataset = new ChatVertexAI({
  model: "gemini-2.5-pro",
  temperature: 0,
  maxRetries: 0,
}).bindTools([searchRetrievalToolWithDataset]);

const searchRetrievalModelResult = await searchRetrievalModelWithDataset.invoke(
  "What is the score of Argentina vs Bolivia football game?"
);

console.log(searchRetrievalModelResult.content);

Argentina won against Bolivia with a score of 6-0 on October 15, 2024.

You should now get results that are grounded in the data from your provided data store.

Context caching

Vertex AI offers context caching functionality, which helps optimize costs by storing and reusing long blocks of message content across multiple API requests. This is particularly useful when you have lengthy conversation histories or message segments that appear frequently in your interactions.

To use this feature, first create a context cache by following this official guide.

Once you’ve created a cache, you can pass its id in as a runtime param as follows:

import { ChatVertexAI } from "@langchain/google-vertexai";

const modelWithCachedContent = new ChatVertexAI({
  model: "gemini-2.5-pro-002",
  location: "us-east5",
});

await modelWithCachedContent.invoke("What is in the content?", {
  cachedContent:
    "projects/PROJECT_NUMBER/locations/LOCATION/cachedContents/CACHE_ID",
});

You can also bind this field directly onto the model instance:

const modelWithBoundCachedContent = new ChatVertexAI({
  model: "gemini-2.5-pro-002",
  location: "us-east5",
}).bind({
  cachedContent:
    "projects/PROJECT_NUMBER/locations/LOCATION/cachedContents/CACHE_ID",
});

Note that not all models currently support context caching.

---

API reference

For detailed documentation of all ChatVertexAI features and configurations head to the API reference.

---

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.