CountTokens API

curl

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \

http://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:countTokens \
-d '{
  "contents": [{
    ...
  }],
  "system_instruction": {
  "role": "...",
  "parts": [{
      ...
    }],
  "tools": [{
      "function_declarations": [{
        ...
      }]
    }],
  }
}'

Python

gemini_model = GenerativeModel(MODEL_ID)
model_response = gemini_model.count_tokens([...])

REST

To get the token count and the number of billable characters for a prompt by using the Vertex AI API, send a POST request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

• LOCATION: The region to process the request. Available options include the following:

  Click to expand available regions

  • us-central1
  • us-west4
  • northamerica-northeast1
  • us-east4
  • us-west1
  • asia-northeast3
  • asia-southeast1
  • asia-northeast1
• PROJECT_ID: Your project ID.
• MODEL_ID: The model ID of the multimodal model that you want to use.
• ROLE: The role in a conversation associated with the content. Specifying a role is required even in singleturn use cases. Acceptable values include the following:
  • USER: Specifies content that's sent by you.
• TEXT: The text instructions to include in the prompt.
• NAME: The name of the function to call.
• DESCRIPTION: Description and purpose of the function.

HTTP method and URL:

POST http://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens

Request JSON body:

{
  "contents": [{
    "role": "ROLE",
    "parts": [{
      "text": "TEXT"
    }]
  }],
  "system_instruction": {
    "role": "ROLE",
    "parts": [{
      "text": "TEXT"
    }]
  }
  "tools": [{
    "function_declarations": [
      {
        "name": "NAME",
        "description": "DESCRIPTION",
        "parameters": {
          "type": "OBJECT",
          "properties": {
            "location": {
              "type": "TYPE",
              "description": "DESCRIPTION"
            }
          },
          "required": [
            "location"
          ]
        }
      }
    ]
  }]
}

To send your request, choose one of these options:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "http://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "http://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens" | Select-Object -Expand Content

You should receive a JSON response similar to the following.

{ "totalTokens": 43 }

Python

To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Python API reference documentation.

import vertexai
from vertexai.generative_models import GenerativeModel

# TODO(developer): Update and un-comment below line
# project_id = "PROJECT_ID"

vertexai.init(project=project_id, location="us-central1")

model = GenerativeModel(model_name="gemini-1.0-pro-002")

prompt = "Why is the sky blue?"

# Prompt tokens count
response = model.count_tokens(prompt)
print(f"Prompt Token Count: {response.total_tokens}")
print(f"Prompt Character Count: {response.total_billable_characters}")

# Send text to Gemini
response = model.generate_content(prompt)

# Response tokens count
usage_metadata = response.usage_metadata
print(f"Prompt Token Count: {usage_metadata.prompt_token_count}")
print(f"Candidates Token Count: {usage_metadata.candidates_token_count}")
print(f"Total Token Count: {usage_metadata.total_token_count}")

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Node.js API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

const {VertexAI} = require('@google-cloud/vertexai');

/**
 * TODO(developer): Update these variables before running the sample.
 */
async function countTokens(
  projectId = 'PROJECT_ID',
  location = 'us-central1',
  model = 'gemini-1.0-pro-002'
) {
  // Initialize Vertex with your Cloud project and location
  const vertexAI = new VertexAI({project: projectId, location: location});

  // Instantiate the model
  const generativeModel = vertexAI.getGenerativeModel({
    model: model,
  });

  const req = {
    contents: [{role: 'user', parts: [{text: 'How are you doing today?'}]}],
  };

  const countTokensResp = await generativeModel.countTokens(req);
  console.log('count tokens response: ', countTokensResp);
}

REST

To get the token count and the number of billable characters for a prompt by using the Vertex AI API, send a POST request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

• LOCATION: The region to process the request. Available options include the following:

  Click to expand available regions

  • us-central1
  • us-west4
  • northamerica-northeast1
  • us-east4
  • us-west1
  • asia-northeast3
  • asia-southeast1
  • asia-northeast1
• PROJECT_ID: Your project ID.
• MODEL_ID: The model ID of the multimodal model that you want to use.
• ROLE: The role in a conversation associated with the content. Specifying a role is required even in singleturn use cases. Acceptable values include the following:
  • USER: Specifies content that's sent by you.
• TEXT: The text instructions to include in the prompt.
• FILE_URI: The Cloud Storage URI of the image or video to include in the prompt. The bucket that stores the file must be in the same Google Cloud project that's sending the request. You must also specify MIMETYPE.
• MIME_TYPE: The media type of the image, PDF, or video specified in the data or fileUri fields. Acceptable values include the following:

  Click to expand MIME types

  • application/pdf
  • audio/mpeg
  • audio/mp3
  • audio/wav
  • image/png
  • image/jpeg
  • text/plain
  • video/mov
  • video/mpeg
  • video/mp4
  • video/mpg
  • video/avi
  • video/wmv
  • video/mpegps
  • video/flv

HTTP method and URL:

POST http://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens

Request JSON body:

{
  "contents": [{
    "role": "ROLE",
    "parts": [
      {
        "file_data": {
          "file_uri": "FILE_URI",
          "mime_type": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT
      }
    ]
  }]
}

To send your request, choose one of these options:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "http://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "http://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens" | Select-Object -Expand Content

You should receive a JSON response similar to the following.

{ "totalTokens": 43 }

Python

Before trying this sample, follow the Python setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Python API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

import vertexai
from vertexai.generative_models import GenerativeModel, Part

# TODO(developer): Update and un-comment below lines
# project_id = "PROJECT_ID"

vertexai.init(project=project_id, location="us-central1")

model = GenerativeModel(model_name="gemini-1.5-flash-preview-0514")

contents = [
    Part.from_uri(
        "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
        mime_type="video/mp4",
    ),
    "Provide a description of the video.",
]

# Prompt tokens count
response = model.count_tokens(contents)
print(f"Prompt Token Count: {response.total_tokens}")
print(f"Prompt Character Count: {response.total_billable_characters}")

# Send text to Gemini
response = model.generate_content(contents)
usage_metadata = response.usage_metadata

# Response tokens count
print(f"Prompt Token Count: {usage_metadata.prompt_token_count}")
print(f"Candidates Token Count: {usage_metadata.candidates_token_count}")
print(f"Total Token Count: {usage_metadata.total_token_count}")

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Node.js API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

const {VertexAI} = require('@google-cloud/vertexai');

/**
 * TODO(developer): Update these variables before running the sample.
 */
async function countTokens(
  projectId = 'PROJECT_ID',
  location = 'us-central1',
  model = 'gemini-1.5-pro-preview-0409'
) {
  // Initialize Vertex with your Cloud project and location
  const vertexAI = new VertexAI({project: projectId, location: location});

  // Instantiate the model
  const generativeModel = vertexAI.getGenerativeModel({
    model: model,
  });

  const req = {
    contents: [
      {
        role: 'user',
        parts: [
          {
            file_data: {
              file_uri:
                'gs://cloud-samples-data/generative-ai/video/pixel8.mp4',
              mime_type: 'video/mp4',
            },
          },
          {text: 'Provide a description of the video.'},
        ],
      },
    ],
  };

  const countTokensResp = await generativeModel.countTokens(req);
  console.log('Prompt Token Count:', countTokensResp.totalTokens);
  console.log(
    'Prompt Character Count:',
    countTokensResp.totalBillableCharacters
  );

  // Sent text to Gemini
  const result = await generativeModel.generateContent(req);
  const usageMetadata = result.response.usageMetadata;

  console.log('Prompt Token Count:', usageMetadata.promptTokenCount);
  console.log('Candidates Token Count:', usageMetadata.candidatesTokenCount);
  console.log('Total Token Count:', usageMetadata.totalTokenCount);
}