aiy.vision.models

A collection of modules that perform ML inferences with specific types of image classification and object detection models.

Each of these modules has a corresponding sample app in src/examples/vision. Also see the instructions to run the models with the Vision Kit.

aiy.vision.models.dish_classification

API for Dish Classification.

aiy.vision.models.dish_classification.get_classes(result, top_k=None, threshold=0.0)

Converts dish classification model output to list of detected objects.

Parameters:

• result – output tensor from dish classification model.
• top_k – int; max number of objects to return.
• threshold – float; min probability of each returned object.

Returns:

string, probability: float) pairs ordered by probability from highest to lowest. The number of pairs is not greater than top_k. Each probability is greater than threshold. For example:

[(‘Ramen’, 0.981934)

(‘Yaka mein, 0.005497)]

Return type:

A list of (class_name

aiy.vision.models.dish_classification.model()

aiy.vision.models.dish_detection

API for Dish Detection.

class aiy.vision.models.dish_detection.Dish(sorted_scores, bounding_box)

Bases: tuple

bounding_box

Alias for field number 1

sorted_scores

Alias for field number 0

aiy.vision.models.dish_detection.get_dishes(result, top_k=3, threshold=0.1)

Returns list of Dish objects decoded from the inference result.

aiy.vision.models.dish_detection.model()

aiy.vision.models.face_detection

API for Face Detection.

class aiy.vision.models.face_detection.Face(face_score, joy_score, bounding_box)

Bases: tuple

bounding_box

Alias for field number 2

face_score

Alias for field number 0

joy_score

Alias for field number 1

aiy.vision.models.face_detection.get_faces(result)

Returns list of Face objects decoded from the inference result.

aiy.vision.models.face_detection.model()

aiy.vision.models.image_classification

API for Image Classification tasks.

aiy.vision.models.image_classification.get_classes(result, top_k=None, threshold=0.0)

Converts image classification model output to list of detected objects.

Parameters:

• result – output tensor from image classification model.
• top_k – int; max number of objects to return.
• threshold – float; min probability of each returned object.

Returns:

string, probability: float) pairs ordered by probability from highest to lowest. The number of pairs is not greater than top_k. Each probability is greater than threshold. For example:

[(‘Egyptian cat’, 0.767578)

(‘tiger cat, 0.163574) (‘lynx/catamount’, 0.039795)]

Return type:

A list of (class_name

aiy.vision.models.image_classification.get_classes_sparse(result)

Converts sparse image classification model output to list of detected objects.

Parameters:result – sparse output tensor from image classification model. Returns:string, probability: float) pairs ordered by probability from highest to lowest. For example:

[(‘Egyptian cat’, 0.767578)

(‘tiger cat, 0.163574)

Return type:A list of (class_name

aiy.vision.models.image_classification.model(model_type='image_classification_mobilenet')

aiy.vision.models.image_classification.sparse_configs(top_k=0, threshold=0.0, model_type='image_classification_mobilenet')

aiy.vision.models.inaturalist_classification

API for detecting plants, insects, and birds from the iNaturalist dataset.

class aiy.vision.models.inaturalist_classification.Model

Bases: aiy.vision.models.inaturalist_classification.Model

compute_graph()

aiy.vision.models.inaturalist_classification.get_classes(result, top_k=None, threshold=0.0)

aiy.vision.models.inaturalist_classification.get_classes_sparse(result)

aiy.vision.models.inaturalist_classification.model(model_type)

aiy.vision.models.inaturalist_classification.sparse_configs(model_type, top_k=None, threshold=0.0)

aiy.vision.models.object_detection

API for Object Detection tasks.

class aiy.vision.models.object_detection.Object(bounding_box, kind, score)

Bases: object

Object detection result.

BACKGROUND = 0

CAT = 2

DOG = 3

PERSON = 1

aiy.vision.models.object_detection.get_objects(result, threshold=0.3, offset=(0, 0))

aiy.vision.models.object_detection.get_objects_sparse(result, offset=(0, 0))

aiy.vision.models.object_detection.model()

aiy.vision.models.object_detection.sparse_configs(threshold=0.3)