Edge TPU inferencing overview

All inferencing with the Edge TPU is based on the TensorFlow Lite APIs (Python or C/C++). So if you already know how to run an inference using TensorFlow Lite, then running your model on the Edge TPU requires only a few new lines of code.

Note: The Edge TPU is compatible with TensorFlow Lite models only. For details about how to create a model that's compatible with the Edge TPU, read TensorFlow models on the Edge TPU.

With a compatible model in-hand, you can perform inferencing on the Edge TPU using either Python or C/C++. In either case, you also have the option to use our Coral APIs, which provide convenience functions that wrap the TensorFlow APIs and other advanced features.

Each option and the software required is described below and illustrated in figure 1.

Figure 1. The three options for inferencing and the corresponding software dependencies
  1. Use the TensorFlow Lite Python API:

    This is the standard Python API for running TensorFlow Lite models. With just a few lines of code, you can make existing TensorFlow Lite code run on the Edge TPU.

    All you need is the TensorFlow Lite Python API and the Edge TPU Runtime (libedgetpu).

    Optional: You can also use our PyCoral APIs, which provide several convenience functions for pre-processing your tensor inputs and post-processing tensor outputs for common models, and additional features such as pipelining a model with multiple Edge TPUs and on-device transfer learning.

    To get started, read Run inference on the Edge TPU with Python.

  2. Use the C/C++ TensorFlow Lite APIs:

    This uses the standard C/C++ API for running TensorFlow Lite models. With just a few lines of code, you can make existing TensorFlow Lite code run on the Edge TPU.

    For this, you need the Edge TPU Runtime (libedgetpu)—linked statically or dynamically—and the compiled TensorFlow Lite C++ library.

    Optional: You can also use our libcoral APIs, which provide several convenience functions for pre-processing your tensor inputs and post-processing tensor outputs for common models, and additional features such as pipelining a model with multiple Edge TPUs.

    For details, read Run inference on the Edge TPU with C++.

You'll get all the libraries you need when you set up your Coral device (except for C++, which requires your own build configurations). But if you're not using one of our supported platforms, everything shown above is open sourced, so you can build these libraries yourself for your platform.