Set up Vertex AI TensorBoard

The following are required to setup Vertex AI TensorBoard:

1. Create a service account with required permissions.
2. Create a Cloud Storage bucket to store Vertex AI TensorBoard logs.
3. Create a Vertex AI TensorBoard instance.

Create a service account with required permissions

The Vertex AI TensorBoard integration with custom training requires attaching a service account.

1. Create a service account:

   gcloud --project=PROJECT_ID iam service-accounts create USER_SA_NAME
   

   Replace the following:

   • PROJECT_ID: the ID of the project in which you are creating a service account

   • USER_SA_NAME: a unique name for the service account you're creating

2. The new service account is used by the Vertex AI Training Service to access Google Cloud services and resources. Use the following commands to grant these roles if needed:

   SA_EMAIL="USER_SA_NAME@PROJECT_ID.iam.gserviceaccount.com"
   
   gcloud projects add-iam-policy-binding PROJECT_ID \
      --member="serviceAccount:${SA_EMAIL}" \
      --role="roles/storage.admin"
   
   gcloud projects add-iam-policy-binding PROJECT_ID \
      --member="serviceAccount:${SA_EMAIL}" \
      --role="roles/aiplatform.user"
   

Create a Cloud Storage bucket to store Vertex AI TensorBoard logs

A Cloud Storage bucket is required to store the Vertex AI TensorBoard logs your training script generates. The bucket must be regional that is, not multi-region or dual-region, and the following resources must be in same region:

• Cloud Storage bucket
• Vertex AI training job
• Vertex AI TensorBoard instance

You can use an existing bucket instead of following the bucket creation step described here. When using an existing bucket, the REGION of the bucket has to be in the same region your Vertex AI TensorBoard instance was created in.

LOCATION=LOCATION_ID
GCS_BUCKET_NAME="PROJECT_ID-tensorboard-logs-${LOCATION_ID}"
gsutil mb -l ${LOCATION_ID} "gs://${GCS_BUCKET_NAME}"

Replace LOCATION_ID with the region that your Vertex AI TensorBoard instance was created in, for example us-central1.

Create a Vertex AI TensorBoard instance

A Vertex AI TensorBoard instance, which is a regionalized resource storing your Vertex AI TensorBoard experiments, must be present before experiments can be visualized. There are two options. You can either use a default instance, or manually create one. You can create multiple instances within a project and region, however most users only need a single instance.

Use the default Vertex AI TensorBoard instance

A default TensorBoard instance is automatically created when initializing a Vertex AI experiment. This backing TensorBoard is associated with the Vertex AI experiment and is used with all subsequent Vertex AI Experiments runs. This is the easiest way to get started with Vertex AI TensorBoard and should meet most users needs.

def create_experiment_default_tensorboard_sample(
    experiment_name: str,
    experiment_description: str,
    project: str,
    location: str,
):
    aiplatform.init(
        experiment=experiment_name,
        experiment_description=experiment_description,
        project=project,
        location=location,
    )

Manually create a Vertex AI TensorBoard instance

You can manually create a Vertex AI TensorBoard. This is useful for users more comfortable with the Google Cloud console, users that need a CMEK enabled TensorBoard (see CMEK), or users who want to use multiple TensorBoards. This instance can then be specified directly when initializing a Vertex AI experiment, starting an Experiment Run, or configuring the training code.

def create_tensorboard_sample(
    project: str,
    location: str,
    display_name: Optional[str] = None,
):
    aiplatform.init(project=project, location=location)

    tensorboard = aiplatform.Tensorboard.create(
        display_name=display_name,
        project=project,
        location=location,
    )

    aiplatform.init(
        project=project,
        location=location,
        experiment_tensorboard=tensorboard
    )

    return tensorboard

resource "google_vertex_ai_tensorboard" "default" {
  display_name = "vertex-ai-tensorboard-sample-name"
  region       = "us-central1"
}

Delete a TensorBoard instance

Deleting a TensorBoard instance deletes that TensorBoard and all associated TensorBoard experiments and TensorBoard runs. The Vertex AI Experiments the instance is associated with isn't deleted.

To delete a Vertex AI Experiments and it's associated Vertex AI TensorBoard experiments, see Delete an experiment.

def delete_tensorboard_instance_sample(
    tensorboard_resource_name: str,
    project: str,
    location: str,
):
    aiplatform.init(project=project, location=location)

    tensorboard = aiplatform.Tensorboard(
        tensorboard_name=tensorboard_resource_name
    )

    tensorboard.delete()

Relevant terms

These terms, "TensorBoard resource name", and "TensorBoard instance ID" are referenced in numerous samples.

TensorBoard resource name

The TensorBoard Resource name is used to fully identify the Vertex AI TensorBoard instance. The format is as follows:

projects/PROJECT_ID_OR_NUMBER/locations/REGION/tensorboards/TENSORBOARD_INSTANCE_ID

The TensorBoard resource name is printed in the log messages when created using gcloud CLI or Vertex AI SDK, or can be created by providing the appropriate values for the placeholders.

TensorBoard instance ID

The TensorBoard instance ID is a generated ID value associated with a TensorBoard instance. To find the TENSORBOARD_INSTANCE_ID, go to the Experiments page Vertex AI section of the Google Cloud console, and select the TensorBoard Instances tab.