Skip to main content

Google Vertex AI

Langchain.js supports two different authentication methods based on whether you're running in a Node.js environment or a web environment.

Setup​

Node.js​

To call Vertex AI models in Node, you'll need to install Google's official auth client as a peer dependency.

You should make sure the Vertex AI API is enabled for the relevant project and that you've authenticated to Google Cloud using one of these methods:

  • You are logged into an account (using gcloud auth application-default login) permitted to that project.
  • You are running on a machine using a service account that is permitted to the project.
  • You have downloaded the credentials for a service account that is permitted to the project and set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of this file.
npm install google-auth-library

Web​

To call Vertex AI models in web environments (like Edge functions), you'll need to install the web-auth-library pacakge as a peer dependency:

npm install web-auth-library

Then, you'll need to add your service account credentials directly as a GOOGLE_VERTEX_AI_WEB_CREDENTIALS environment variable:

GOOGLE_VERTEX_AI_WEB_CREDENTIALS={"type":"service_account","project_id":"YOUR_PROJECT-12345",...}

You can also pass your credentials directly in code like this:

import { GoogleVertexAI } from "langchain/llms/googlevertexai/web";

const model = new GoogleVertexAI({
  authOptions: {
    credentials: {"type":"service_account","project_id":"YOUR_PROJECT-12345",...},
  },
});

Usage​

Several models are available and can be specified by the model attribute in the constructor. These include:

  • text-bison (default)
  • text-bison-32k
  • code-gecko
  • code-bison
import { GoogleVertexAI } from "langchain/llms/googlevertexai";
// Or, if using the web entrypoint:
// import { GoogleVertexAI } from "langchain/llms/googlevertexai/web";

/*
 * Before running this, you should make sure you have created a
 * Google Cloud Project that is permitted to the Vertex AI API.
 *
 * You will also need permission to access this project / API.
 * Typically, this is done in one of three ways:
 * - You are logged into an account permitted to that project.
 * - You are running this on a machine using a service account permitted to
 *   the project.
 * - The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set to the
 *   path of a credentials file for a service account permitted to the project.
 */
const model = new GoogleVertexAI({
  temperature: 0.7,
});
const res = await model.call(
  "What would be a good company name for a company that makes colorful socks?"
);
console.log({ res });

API Reference:

Google also has separate models for their "Codey" code generation models.

The "code-gecko" model is useful for code completion:

import { GoogleVertexAI } from "langchain/llms/googlevertexai";

/*
 * Before running this, you should make sure you have created a
 * Google Cloud Project that is permitted to the Vertex AI API.
 *
 * You will also need permission to access this project / API.
 * Typically, this is done in one of three ways:
 * - You are logged into an account permitted to that project.
 * - You are running this on a machine using a service account permitted to
 *   the project.
 * - The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set to the
 *   path of a credentials file for a service account permitted to the project.
 */

const model = new GoogleVertexAI({
  model: "code-gecko",
});
const res = await model.call("for (let co=0;");
console.log({ res });

API Reference:

While the "code-bison" model is better at larger code generation based on a text prompt:

import { GoogleVertexAI } from "langchain/llms/googlevertexai";

/*
 * Before running this, you should make sure you have created a
 * Google Cloud Project that is permitted to the Vertex AI API.
 *
 * You will also need permission to access this project / API.
 * Typically, this is done in one of three ways:
 * - You are logged into an account permitted to that project.
 * - You are running this on a machine using a service account permitted to
 *   the project.
 * - The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set to the
 *   path of a credentials file for a service account permitted to the project.
 */

const model = new GoogleVertexAI({
  model: "code-bison",
  maxOutputTokens: 2048,
});
const res = await model.call("A Javascript function that counts from 1 to 10.");
console.log({ res });

API Reference:

Streaming​

Streaming in multiple chunks is supported for faster responses:

import { GoogleVertexAI } from "langchain/llms/googlevertexai";

const model = new GoogleVertexAI({
  temperature: 0.7,
});
const stream = await model.stream(
  "What would be a good company name for a company that makes colorful socks?"
);

for await (const chunk of stream) {
  console.log("\n---------\nChunk:\n---------\n", chunk);
}

/*
  ---------
  Chunk:
  ---------
    1. Toe-tally Awesome Socks
  2. The Sock Drawer
  3. Happy Feet
  4. 

  ---------
  Chunk:
  ---------
  Sock It to Me
  5. Crazy Color Socks
  6. Wild and Wacky Socks
  7. Fu

  ---------
  Chunk:
  ---------
  nky Feet
  8. Mismatched Socks
  9. Rainbow Socks
  10. Sole Mates

  ---------
  Chunk:
  ---------
  

*/

API Reference: