Currently, we offer two separate ways to perform an inference on the Edge TPU: with the Edge TPU API or with the TensorFlow Lite API.
The Edge TPU API (the
edgetpu module) provides simple APIs that perform image classification and
object detection. It's build on top of the TensorFlow Lite C++ API and abstracts-away a lot of the
code required to handle input tensors and output tensors. The Edge TPU API also includes APIs to
perform on-device transfer-learning with either weight imprinting or backpropagation.
The alternative is to use the TensorFlow Lite API directly. This requires more code in your application to process the input and output tensors, but it gives you more opportunities to customize the code for different types of model architectures. For details, instead read how to run inference using the TensorFlow Lite API with Python or with C++.
The rest of this document is focused on how to use the Edge TPU API.
If you just want to see some code, check out the Examples page.
Install the library and examples
If you're using the Dev Board or SoM, the Edge TPU library is included in the Mendel system image (just be sure you've updated to the latest software).
If you're setting up an accessory device such as the USB Accelerator, you need to install the Edge TPU library on your host computer. On a Mac or Windows host computer, install the library with the instructions here. Otherwise, on Linux, you can install it with a Debian package:
sudo apt-get update sudo apt-get install python3-edgetpu
To get the example code, you can clone the github.com/google-coral/edgetpu repo. However, if you're on a Linux computer (including the Dev Board), we recommend you download the examples from this Debian package:
sudo apt-get install edgetpu-examples
The examples are saved at
Edge TPU API overview
Key APIs in the
edgetpu module that perform inferencing are the following:
ClassificationEngine: Performs image classification. Create an instance by specifying a model, and then pass an image (such as a JPEG) to
ClassifyWithImage()and it returns a list of labels and scores.
DetectionEngine: Performs object detection. Create an instance by specifying a model, and then pass an image (such as a JPEG) to
DetectWithImage()and it returns a list of
DetectionCandidateobjects, each of which contains a label, a score, and the coordinates of the object.
Both engines are subclasses of
you can use to perform different types of inferencing. They both also support all image formats
supported by Pillow (including JPEG, PNG, BMP), except the files
must be in RGB color space (no transparency).
Additionally, we've included two APIs that perform on-device transfer learning (for classification models only):
ImprintingEngine: This implements a transfer-learning technique called weight imprinting that does not require backward propagation. It allows you to teach the model new classifications with very small sample sizes (literally just a few images). For more information, read Retrain an image classification model on-device.
SoftmaxRegression: This offers an abbreviated version of traditional backpropagation—it updates only the fully-connected layer at the end of the graph with new weights. It requires large datasets for training, but is still very fast and may result in more accurate models when the dataset has high intra-class variance. For more information, read Retrain an image classification model on-device.
The Python library takes care of all the low-level Edge TPU configuration for you. Even if you connect multiple Edge TPUs, the Python library automatically delegates separate models to execute on separate Edge TPUs. For more information about using using multiple models, read Run multiple models with multiple Edge TPUs.
For example code using the Edge TPU Python API, see our Examples page.
Is this content helpful?