Try Gemini 1.5 models, our newest multimodal models in Vertex AI, and see what you can build with a 1M token context window. Try Gemini 1.5 models, our newest multimodal models in Vertex AI, and see what you can build with a 1M token context window.

Package google.cloud.aiplatform.v1.schema.predict.prediction

Index

ClassificationPredictionResult (message)
ImageObjectDetectionPredictionResult (message)
ImageSegmentationPredictionResult (message)
TabularClassificationPredictionResult (message)
TabularRegressionPredictionResult (message)
TextExtractionPredictionResult (message)
TextSentimentPredictionResult (message)
TftFeatureImportance (message)
TimeSeriesForecastingPredictionResult (message)
VideoActionRecognitionPredictionResult (message)
VideoClassificationPredictionResult (message)
VideoObjectTrackingPredictionResult (message)
VideoObjectTrackingPredictionResult.Frame (message)

ClassificationPredictionResult

Prediction output format for Image and Text Classification.

Fields

Fields
`ids[]`	`int64` The resource IDs of the AnnotationSpecs that had been identified.
`display_names[]`	`string` The display names of the AnnotationSpecs that had been identified, order matches the IDs.
`confidences[]`	`float` The Model's confidences in correctness of the predicted IDs, higher value means higher confidence. Order matches the Ids.

ids[]

int64

The resource IDs of the AnnotationSpecs that had been identified.

display_names[]

string

The display names of the AnnotationSpecs that had been identified, order matches the IDs.

confidences[]

float

The Model's confidences in correctness of the predicted IDs, higher value means higher confidence. Order matches the Ids.

ImageObjectDetectionPredictionResult

Prediction output format for Image Object Detection.

Fields
`ids[]`	`int64` The resource IDs of the AnnotationSpecs that had been identified, ordered by the confidence score descendingly.
`display_names[]`	`string` The display names of the AnnotationSpecs that had been identified, order matches the IDs.
`confidences[]`	`float` The Model's confidences in correctness of the predicted IDs, higher value means higher confidence. Order matches the Ids.
`bboxes[]`	`ListValue` Bounding boxes, i.e. the rectangles over the image, that pinpoint the found AnnotationSpecs. Given in order that matches the IDs. Each bounding box is an array of 4 numbers `xMin`, `xMax`, `yMin`, and `yMax`, which represent the extremal coordinates of the box. They are relative to the image size, and the point 0,0 is in the top left of the image.

ImageSegmentationPredictionResult

Prediction output format for Image Segmentation.

Fields

Fields
`category_mask`	`string` A PNG image where each pixel in the mask represents the category in which the pixel in the original image was predicted to belong to. The size of this image will be the same as the original image. The mapping between the AnntoationSpec and the color can be found in model's metadata. The model will choose the most likely category and if none of the categories reach the confidence threshold, the pixel will be marked as background.
`confidence_mask`	`string` A one channel image which is encoded as an 8bit lossless PNG. The size of the image will be the same as the original image. For a specific pixel, darker color means less confidence in correctness of the cateogry in the categoryMask for the corresponding pixel. Black means no confidence and white means complete confidence.

category_mask

string

A PNG image where each pixel in the mask represents the category in which the pixel in the original image was predicted to belong to. The size of this image will be the same as the original image. The mapping between the AnntoationSpec and the color can be found in model's metadata. The model will choose the most likely category and if none of the categories reach the confidence threshold, the pixel will be marked as background.

confidence_mask

string

A one channel image which is encoded as an 8bit lossless PNG. The size of the image will be the same as the original image. For a specific pixel, darker color means less confidence in correctness of the cateogry in the categoryMask for the corresponding pixel. Black means no confidence and white means complete confidence.

TabularClassificationPredictionResult

Prediction output format for Tabular Classification.

Fields

Fields
`classes[]`	`string` The name of the classes being classified, contains all possible values of the target column.
`scores[]`	`float` The model's confidence in each class being correct, higher value means higher confidence. The N-th score corresponds to the N-th class in classes.

classes[]

string

The name of the classes being classified, contains all possible values of the target column.

scores[]

float

The model's confidence in each class being correct, higher value means higher confidence. The N-th score corresponds to the N-th class in classes.

TabularRegressionPredictionResult

Prediction output format for Tabular Regression.

Fields
`value`	`float` The regression value.
`lower_bound`	`float` The lower bound of the prediction interval.
`upper_bound`	`float` The upper bound of the prediction interval.
`quantile_values[]`	`float` Quantile values.
`quantile_predictions[]`	`float` Quantile predictions, in 1-1 correspondence with quantile_values.

TextExtractionPredictionResult

Prediction output format for Text Extraction.

Fields
`ids[]`	`int64` The resource IDs of the AnnotationSpecs that had been identified, ordered by the confidence score descendingly.
`display_names[]`	`string` The display names of the AnnotationSpecs that had been identified, order matches the IDs.
`text_segment_start_offsets[]`	`int64` The start offsets, inclusive, of the text segment in which the AnnotationSpec has been identified. Expressed as a zero-based number of characters as measured from the start of the text snippet.
`text_segment_end_offsets[]`	`int64` The end offsets, inclusive, of the text segment in which the AnnotationSpec has been identified. Expressed as a zero-based number of characters as measured from the start of the text snippet.
`confidences[]`	`float` The Model's confidences in correctness of the predicted IDs, higher value means higher confidence. Order matches the Ids.

TextSentimentPredictionResult

Prediction output format for Text Sentiment

Fields

Fields
`sentiment`	`int32` The integer sentiment labels between 0 (inclusive) and sentimentMax label (inclusive), while 0 maps to the least positive sentiment and sentimentMax maps to the most positive one. The higher the score is, the more positive the sentiment in the text snippet is. Note: sentimentMax is an integer value between 1 (inclusive) and 10 (inclusive).

sentiment

int32

The integer sentiment labels between 0 (inclusive) and sentimentMax label (inclusive), while 0 maps to the least positive sentiment and sentimentMax maps to the most positive one. The higher the score is, the more positive the sentiment in the text snippet is. Note: sentimentMax is an integer value between 1 (inclusive) and 10 (inclusive).

TftFeatureImportance

Fields
`context_weights[]`	`float` TFT feature importance values. Each pair for {context/horizon/attribute} should have the same shape since the weight corresponds to the column names.
`context_columns[]`	`string`
`horizon_weights[]`	`float`
`horizon_columns[]`	`string`
`attribute_weights[]`	`float`
`attribute_columns[]`	`string`

TimeSeriesForecastingPredictionResult

Prediction output format for Time Series Forecasting.

Fields
`value`	`float` The regression value.
`quantile_values[]`	`float` Quantile values.
`quantile_predictions[]`	`float` Quantile predictions, in 1-1 correspondence with quantile_values.
`tft_feature_importance`	`TftFeatureImportance` Only use these if TFt is enabled.

VideoActionRecognitionPredictionResult

Prediction output format for Video Action Recognition.

Fields
`id`	`string` The resource ID of the AnnotationSpec that had been identified.
`display_name`	`string` The display name of the AnnotationSpec that had been identified.
`time_segment_start`	`Duration` The beginning, inclusive, of the video's time segment in which the AnnotationSpec has been identified. Expressed as a number of seconds as measured from the start of the video, with fractions up to a microsecond precision, and with "s" appended at the end.
`time_segment_end`	`Duration` The end, exclusive, of the video's time segment in which the AnnotationSpec has been identified. Expressed as a number of seconds as measured from the start of the video, with fractions up to a microsecond precision, and with "s" appended at the end.
`confidence`	`FloatValue` The Model's confidence in correction of this prediction, higher value means higher confidence.

VideoClassificationPredictionResult

Prediction output format for Video Classification.

Fields
`id`	`string` The resource ID of the AnnotationSpec that had been identified.
`display_name`	`string` The display name of the AnnotationSpec that had been identified.
`type`	`string` The type of the prediction. The requested types can be configured via parameters. This will be one of - segment-classification - shot-classification - one-sec-interval-classification
`time_segment_start`	`Duration` The beginning, inclusive, of the video's time segment in which the AnnotationSpec has been identified. Expressed as a number of seconds as measured from the start of the video, with fractions up to a microsecond precision, and with "s" appended at the end. Note that for 'segment-classification' prediction type, this equals the original 'timeSegmentStart' from the input instance, for other types it is the start of a shot or a 1 second interval respectively.
`time_segment_end`	`Duration` The end, exclusive, of the video's time segment in which the AnnotationSpec has been identified. Expressed as a number of seconds as measured from the start of the video, with fractions up to a microsecond precision, and with "s" appended at the end. Note that for 'segment-classification' prediction type, this equals the original 'timeSegmentEnd' from the input instance, for other types it is the end of a shot or a 1 second interval respectively.
`confidence`	`FloatValue` The Model's confidence in correction of this prediction, higher value means higher confidence.

VideoObjectTrackingPredictionResult

Prediction output format for Video Object Tracking.

Fields
`id`	`string` The resource ID of the AnnotationSpec that had been identified.
`display_name`	`string` The display name of the AnnotationSpec that had been identified.
`time_segment_start`	`Duration` The beginning, inclusive, of the video's time segment in which the object instance has been detected. Expressed as a number of seconds as measured from the start of the video, with fractions up to a microsecond precision, and with "s" appended at the end.
`time_segment_end`	`Duration` The end, inclusive, of the video's time segment in which the object instance has been detected. Expressed as a number of seconds as measured from the start of the video, with fractions up to a microsecond precision, and with "s" appended at the end.
`confidence`	`FloatValue` The Model's confidence in correction of this prediction, higher value means higher confidence.
`frames[]`	`Frame` All of the frames of the video in which a single object instance has been detected. The bounding boxes in the frames identify the same object.

Frame

The fields xMin, xMax, yMin, and yMax refer to a bounding box, i.e. the rectangle over the video frame pinpointing the found AnnotationSpec. The coordinates are relative to the frame size, and the point 0,0 is in the top left of the frame.

Fields
`time_offset`	`Duration` A time (frame) of a video in which the object has been detected. Expressed as a number of seconds as measured from the start of the video, with fractions up to a microsecond precision, and with "s" appended at the end.
`x_min`	`FloatValue` The leftmost coordinate of the bounding box.
`x_max`	`FloatValue` The rightmost coordinate of the bounding box.
`y_min`	`FloatValue` The topmost coordinate of the bounding box.
`y_max`	`FloatValue` The bottommost coordinate of the bounding box.