REST Resource: projects.locations.modelMonitors

Resource: ModelMonitor

Vertex AI Model Monitoring service serves as a central hub for the analysis and visualization of data quality and performance related to models. ModelMonitor stands as a top level resource for overseeing your model monitoring tasks.

JSON representation
{
  "name": string,
  "displayName": string,
  "modelMonitoringTarget": {
    object (ModelMonitoringTarget)
  },
  "trainingDataset": {
    object (ModelMonitoringInput)
  },
  "notificationSpec": {
    object (ModelMonitoringNotificationSpec)
  },
  "outputSpec": {
    object (ModelMonitoringOutputSpec)
  },
  "explanationSpec": {
    object (ExplanationSpec)
  },
  "modelMonitoringSchema": {
    object (ModelMonitoringSchema)
  },
  "createTime": string,
  "updateTime": string,

  // Union field default_objective can be only one of the following:
  "tabularObjective": {
    object (TabularObjective)
  }
  // End of list of possible types for union field default_objective.
}
Fields
name

string

Immutable. Resource name of the ModelMonitor. Format: projects/{project}/locations/{location}/modelMonitors/{modelMonitor}.

displayName

string

The display name of the ModelMonitor. The name can be up to 128 characters long and can consist of any UTF-8.

modelMonitoringTarget

object (ModelMonitoringTarget)

The entity that is subject to analysis. Currently only models in Vertex AI Model Registry are supported. If you want to analyze the model which is outside the Vertex AI, you could register a model in Vertex AI Model Registry using just a display name.

trainingDataset

object (ModelMonitoringInput)

Optional training dataset used to train the model. It can serve as a reference dataset to identify changes in production.

notificationSpec

object (ModelMonitoringNotificationSpec)

Optional default notification spec, it can be overridden in the ModelMonitoringJob notification spec.

outputSpec

object (ModelMonitoringOutputSpec)

Optional default monitoring metrics/logs export spec, it can be overridden in the ModelMonitoringJob output spec. If not specified, a default Google Cloud Storage bucket will be created under your project.

explanationSpec

object (ExplanationSpec)

Optional model explanation spec. It is used for feature attribution monitoring.

modelMonitoringSchema

object (ModelMonitoringSchema)

Monitoring Schema is to specify the model's features, prediction outputs and ground truth properties. It is used to extract pertinent data from the dataset and to process features based on their properties. Make sure that the schema aligns with your dataset, if it does not, we will be unable to extract data from the dataset. It is required for most models, but optional for Vertex AI AutoML Tables unless the schem information is not available.

createTime

string (Timestamp format)

Output only. timestamp when this ModelMonitor was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

updateTime

string (Timestamp format)

Output only. timestamp when this ModelMonitor was updated most recently.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

Union field default_objective. Optional default monitoring objective, it can be overridden in the ModelMonitoringJob objective spec. default_objective can be only one of the following:
tabularObjective

object (TabularObjective)

Optional default tabular model monitoring objective.

ModelMonitoringTarget

The monitoring target refers to the entity that is subject to analysis. e.g. Vertex AI Model version.

JSON representation
{

  // Union field source can be only one of the following:
  "vertexModel": {
    object (VertexModelSource)
  }
  // End of list of possible types for union field source.
}
Fields

Union field source.

source can be only one of the following:

vertexModel

object (VertexModelSource)

Model in Vertex AI Model Registry.

VertexModelSource

Model in Vertex AI Model Registry.

JSON representation
{
  "model": string,
  "modelVersionId": string
}
Fields
model

string

Model resource name. Format: projects/{project}/locations/{location}/models/{model}.

modelVersionId

string

Model version id.

ModelMonitoringSchema

The Model Monitoring Schema definition.

JSON representation
{
  "featureFields": [
    {
      object (FieldSchema)
    }
  ],
  "predictionFields": [
    {
      object (FieldSchema)
    }
  ],
  "groundTruthFields": [
    {
      object (FieldSchema)
    }
  ]
}
Fields
featureFields[]

object (FieldSchema)

feature names of the model. Vertex AI will try to match the features from your dataset as follows: * For 'csv' files, the header names are required, and we will extract the corresponding feature values when the header names align with the feature names. * For 'jsonl' files, we will extract the corresponding feature values if the key names match the feature names. Note: Nested features are not supported, so please ensure your features are flattened. Ensure the feature values are scalar or an array of scalars. * For 'bigquery' dataset, we will extract the corresponding feature values if the column names match the feature names. Note: The column type can be a scalar or an array of scalars. STRUCT or JSON types are not supported. You may use SQL queries to select or aggregate the relevant features from your original table. However, ensure that the 'schema' of the query results meets our requirements. * For the Vertex AI Endpoint Request Response Logging table or Vertex AI Batch Prediction Job results. If the instanceType is an array, ensure that the sequence in featureFields matches the order of features in the prediction instance. We will match the feature with the array in the order specified in [featureFields].

predictionFields[]

object (FieldSchema)

Prediction output names of the model. The requirements are the same as the featureFields. For AutoML Tables, the prediction output name presented in schema will be: predicted_{targetColumn}, the targetColumn is the one you specified when you train the model. For Prediction output drift analysis: * AutoML Classification, the distribution of the argmax label will be analyzed. * AutoML Regression, the distribution of the value will be analyzed.

groundTruthFields[]

object (FieldSchema)

Target /ground truth names of the model.

FieldSchema

Schema field definition.

JSON representation
{
  "name": string,
  "dataType": string,
  "repeated": boolean
}
Fields
name

string

Field name.

dataType

string

Supported data types are: float integer boolean string categorical

repeated

boolean

Describes if the schema field is an array of given data type.

Methods

create

Creates a ModelMonitor.

delete

Deletes a ModelMonitor.

get

Gets a ModelMonitor.

list

Lists ModelMonitors in a Location.

patch

Updates a ModelMonitor.

searchModelMonitoringAlerts

Returns the Model Monitoring alerts.

searchModelMonitoringStats

Searches Model Monitoring Stats generated within a given time window.