The setup guide for each Coral device shows you how to install the required software and run an inference on the Edge TPU. However, our pre-built software components are not compatible with all platform variants. So if you're trying to use one of our devices on a different platform, you'll need to build all these components yourself (either natively or in Docker/cross-compilation environments) using our library source code.
Building these components on your own might be necessary if your system matches some of the requirements for our pre-build components (such as the Python version) but not others (such as the Debian version). Other examples of when it makes sense to build on your own include if you want to use a specific TensorFlow version or run on a Linux system that's not Debian-based. We can't possibly build for every scenario, and in some situations it's easier for you to build the libraries and APIs yourself instead of trying to change your system to match the build environments that we used.
This document describes how to build everything you need to run the Edge TPU as well as how to integrate and test the components.
You need the following software components to run inference on the Edge TPU:
- Edge TPU Runtime (libedgetpu): A shared library (libedgetpu.so.1) required to communicate with the Edge TPU.
- TFLite Runtime: A Python library required to run inferences using the TensorFlow Lite API.
- Edge TPU Python API: A Python library that provides functionality beyond that of just TFLite inference, such as on-device training with backpropagation and weight imprinting.
- PCIe Driver (gasket-dkms): Required for PCIe devices only, this package uses DKMS to install the Edge TPU driver.
Normally, you can install the pre-built version of these by following our device setup guides or downloading them from our downloads page. But to build them for your own platform, you'll need to follow the steps in the following sections.
Before you begin
There are a few things you need to decide on and verify before beginning this process:
- A TensorFlow Commit: The Edge TPU runtime, TFLite runtime, and Edge TPU Python library must be built based on the same version of TensorFlow. Any arbitrary commit can work but we recommend using one of the TF Release Tags.
- Build natively or cross-compile: While native compilation will ensure that everything works as expected, cross-compilation can enable faster building on a workstation before deployment to devices. The directions for each will vary based on how you plan to build.
- For PCIe devices, verify your system supports MSI-X: While we can overcome many incompatibility issues by building, MSI-X is required by the Edge TPU silicon. Without it, there is no way to receive interrupts (and thus no way to know when operations complete).
Build the Edge TPU runtime (libedgetpu)
To install our pre-build Edge TPU runtime, you can run
sudo apt install
libedgetpu1-max in the case of maximum frequency for
USB). That package contains the shared library (libedgetpu.so.1).
To build it yourself, first clone our libedgetpu repo.
git clone https://github.com/google-coral/libedgetpu
Next, configure the bazel workspace to use the TensorFlow commit you have in
mind. For example, if you want to use TF v2.3.0 - as of this writing that tag
corresponds to commit
b36436b087bd8e8701ef51718179037cccdfc26e (determined from
TF Release Tags). So insert
that as the
TENSORFLOW_COMMIT value in the
A checksum is also required for the TF package that you'll download. As the instructions mention, you could curl the object and compute this. However, another approach is simply to leave it for now, let it fail, and wait for the correct value to be reported.
If you're building natively, just run the Makefile:
If you want to cross compile or use a known-good Docker container (which allows compilation of multiple CPU architectures), you can follow the libedgetpu README.
In any scenario, the output will be found in the
out folder, as shown here:
├── direct │ └── k8 │ ├── libedgetpu.so.1 -> libedgetpu.so.1.0 │ └── libedgetpu.so.1.0 └── throttled └── k8 ├── libedgetpu.so.1 -> libedgetpu.so.1.0 └── libedgetpu.so.1.0
The library files in
direct correspond to
libedgetpu1-max (max frequency)
and those in
throttled correspond to
libedgetpu1-std (reduced frequency).
The frequency behavior only applies to USB devices; for PCIe devices, you can
use either library and the frequency is managed with
dynamic frequency scaling.
To deploy the library, copy this into your
/usr/lib/. This exact location may
vary by system, but commonly you can move it as follows (for a x86_64 system):
sudo cp out/direct/k8/* /usr/lib/x86_64-linux-gnu/ sudo ldconfig
Build the TFLite runtime
To install our pre-built TFLite runtime, you can get it from Pip as per the Python TFLite quickstart.
To build it yourself, you first need to checkout the TensorFlow repo:
git clone https://github.com/tensorflow/tensorflow.git cd tensorflow git checkout <TF Commit ID you decided on above>
Remember that you must checkout the same TensorFlow version that you used for
TENSORFLOW_COMMIT value when
building the Edge TPU runtime.
The instructions that follow are the latest as of this publishing (and
simplified). For a more thorough procedure and the latest documentation, you
should see the
build instructions in the TensorFlow repo.
In that documentation, you'll see both
build_pip_package_with_bazel.sh. While both work, we recommend the Bazel
option (it doesn't require additional host dependencies) as follows.
To build natively (for TF versions > 2.2):
To build natively (for TF versions < 2.2):
To cross-compile, see the steps at the build instructions in the TensorFlow repo.
To deploy, simply use pip to install (for version 2.4.0 in this case):
python3 -m pip install \ tensorflow/lite/tools/pip_package/gen/tflite_pip/python3/dist/tflite_runtime-2.4.0-py3-none-any.whl
Build the Edge TPU Python API
To install our pre-built Edge TPU library, you can get it as a Debian package or with Pip as per our downloads page.
To build it yourself, you should follow the steps found in the edgetpu repo—also summarized below.
Note that while the wheel building steps are only required for getting Coral-specific functionality in Python, the repo is still worth looking at for a C++ design. The build system for C++ can help show how to use Bazel (and Docker) to build C++ applications.
The Edge TPU Python API has a swig-based native layer so an
architecture-specific build is required. Using
build_swig.sh provides support
for ARM64, ARMv7, and x86_64 for Python 3.5, 3.6, 3.7, and 3.8 (note that only
one wheel is generated supporting all of these).
Note: The following build steps require Docker.
Before you build the library, edit the
WORKSPACE file at the root of the repository
to specify the
TENSORFLOW_COMMIT, using the same value you used when
building the Edge TPU runtime.
Then build the library as follows:
scripts/build_swig.sh make wheel python3 -m pip install $(ls dist/*.whl)
Build the PCIe Driver (gasket-dkms)
This is required only for PCIe devices such as the Accelerator Module and Mini PCIe or M.2 Accelerator cards.
To get our pre-build PCie driver, you can get it from a Debian package (gasket-dkms), as per the getting started guide. This package requires the appropriate kernel headers for your device, in which case it will use DKMS to install the module into your kernel when the package is installed.
If you don't have a Debian system or are encountering issues with the DKMS install, you can add the source directly to your kernel tree. The source directory should be copied into your kernel. For example, here's how to add it to your kernel:
wget https://coral.googlesource.com/linux-imx/+archive/refs/heads/dkms/drivers/staging/gasket.tar.gz gasket.tar.gz mkdir <kernel location>/drivers/staging/gasket tar -xf gasket.tar.gz -C <kernel location>/drivers/staging/gasket
You also need to add the config
CONFIG_STAGING_APEX_DRIVER to your kernel
config to ensure this is built. For example (building into the kernel, set to
"m" for a module):
echo "CONFIG_STAGING_APEX_DRIVER=y" >> defconfig
Then build your kernel as you usually would and deploy it to your device.
After you'd deployed all the necessary components as shown above, you can test it by running this classification demo on your device:
mkdir coral && cd coral git clone https://github.com/google-coral/tflite.git cd tflite/python/examples/classification bash install_requirements.sh python3 classify_image.py \ --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \ --labels models/inat_bird_labels.txt \ --input images/parrot.jpg
You should see a result like this (note the speed will vary based on the interface used):
INFO: Initialized TensorFlow Lite runtime. ----INFERENCE TIME---- Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory. 11.8ms 3.0ms 2.8ms 2.9ms 2.9ms -------RESULTS-------- Ara macao (Scarlet Macaw): 0.76562
The following are some common errors you might encounter.
Failed to load delegate from libedgetpu.so.1
Failing to load the delegate is a very general error that can indicate a few issues such as the following.
The runtime isn't installed: To verify your runtime is installed you can use Python's ctypes, which is what loads the library.
python3 -c "from ctypes.util import find_library; print(find_library(\"edgetpu\"))"
The output should yield
libedgetpu.so.1. If it doesn't, the runtime isn't
installed (see Edge TPU Runtime (libedgetpu)).
The Edge TPU isn't available: This error will also manifest if the device can't be found.
lsusb -d 1a6e:089a
The output should show
1a6e:089a Global Unichip Corp. If not, the device isn't
available over USB. Check the connection and your
dmesg output to see if there
was a kernel error preventing communication.
ls /dev | grep apex
The output should yield (assuming only one TPU)
/dev/apex_0. If there is no
Apex device, check your
dmesg output and see PCIe errors.
No module named 'tflite_runtime'
This error indicates that the TFLite runtime isn't installed. Follow the instructions in TFLite Runtime for building and installing the TFLite runtime.
Unsupported data type in custom op handler: (EdgeTpuDelegateForCustomOp) failed to prepare
After running a TFLite model through the Edge TPU Compiler, everything that is mapped to the Edge TPU is placed in a single op (edgetpu-custom-op). This means that if the model fully runs on the Edge TPU, the entire model is in this operation and input/output:
In order to run properly, this edgetpu-custom-op must be properly handled. So this error indicates that the delegate has issues running with the current setup. There is a mismatched version between one of the following:
- Edge TPU Runtime
- TFLite Runtime
- (For C++) Edge TPU Runtime linked against when building
If building everything (as described in this document), the key is ensuring that each component was built using the same TensorFlow commit.
It is also possible that the model was compiled using a new Edge TPU Compiler but the runtimes are an older version (for example, using a TensorFlow commit that is for TensorFlow 1.15 but then using the latest Edge TPU compiler).
If you've confirmed that the built packages use the same TensorFlow commit, the next step is to try a model from coral.ai/models/. If these models don't work, you likely need to pick a more recent TensorFlow commit (keep in mind the compatibility layer can provide most 1.x API in TF 2.x). If the website models work but your model doesn't, please contact email@example.com to help evaluate the compilation issues.
Is this content helpful?