Get started with the M.2 or Mini PCIe Accelerator

To get started with either the Mini PCIe or M.2 Accelerator, all you need to do is connect the card to your system, and then install our PCIe driver, Edge TPU runtime, and the TensorFlow Lite runtime. This page walks you through the setup and shows you how to run an example model.

The setup and operation is the same for both form-factors.

Requirements

To get started, you need a Linux computer with the following specs:

  • x86-64 or ARM64 system architecture
  • An available Mini PCIe or M.2 key slot
  • Debian 6.0 Linux distribution or higher, or any derivative thereof (such as Ubuntu 10.0+)
  • Python 3.5 or higher

1. Install the PCIe driver

Before you install our driver, you first need to check whether you have a pre-built Apex driver installed. Some versions of the driver have a bug that prevents updates and will result in failure when calling upon the Edge TPU. So first follow these steps:

  1. Check your Linux kernel version with this command:

    uname -r

    If it prints 4.18 or lower, you should be okay and can skip to begin installing our PCIe driver.

  2. If your kernel version is 4.19 or higher, now check if you have a pre-build Apex driver installed:

    lsmod | grep apex
    

    If it prints nothing, then you're okay and continue to install our PCIe driver.

    If it does print an Apex module name, stop here and follow the workaround to disable Apex and Gasket.

Install our PCIe driver as follows:

  1. Make sure the host system where you'll connect the module is shut down.

  2. Connect the Coral PCIe/M.2 module according to your host system recommendations.

  3. Boot the system and make sure it has an internet connection. Then add our Debian package repository to your system and install our PCIe driver with these commands:

    echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
    
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
    sudo apt update
    sudo apt install gasket-dkms
  4. If the user account you'll be using does not have root permissions, you might need to add the following udev rule, and then verify that the "apex" group exists and that your user is added to it:

    sudo sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"
    
    sudo groupadd apex
    sudo adduser $USER apex
  5. Reboot the system.

  6. Verify that the accelerator module is detected:

    lspci -x | grep 089a
    

    You should see something like this:

    03:00.0 System peripheral: Device 1ac1:089a
    

    The 03 number and System peripheral name might be different, because those are host-system specific, but as long as you see a device listed with 089a then you're okay to proceed.

  7. Verify that the PCIe driver is loaded:

    ls /dev/apex_0
    

    You should simply see the name repeated back:

    /dev/apex_0
    

2. Install the Edge TPU runtime

The Edge TPU runtime is required to communicate with the Edge TPU. You can install it on your host computer from a command line as follows:

sudo apt-get install libedgetpu1-std
Note: Unlike the USB Accelerator, the Mini PCIe and M.2 Accelerator do not use different runtime packages for "default" and "maximum" clock frequencies. Instead, these devices operate at the maximum frequency by default and perform power throttling based on thermal limits.

3. Install the TensorFlow Lite library

There are several ways you can install TensorFlow's APIs, but to get started with Python, the easiest option is to install the tflite_runtime package. This package provides the bare minimum code required to run an inference with Python (primarily, the Interpreter API), thus saving you a lot of disk space.

To install it, follow the TensorFlow Lite Python quickstart, and then return to this page after you run the pip3 install command.

4. Run a model using the TensorFlow Lite API

Now you're ready to run an inference on the Edge TPU. Follow these steps to perform image classification with our example code and model:

  1. Download the example code from GitHub:

    mkdir coral && cd coral
    
    git clone https://github.com/google-coral/tflite.git
  2. Download the bird classifier model, labels file, and a bird photo:

    cd tflite/python/examples/classification
    
    bash install_requirements.sh
  3. Run the image classifier with the bird photo (shown in figure 1):

    python3 classify_image.py \
    --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
    --labels models/inat_bird_labels.txt \
    --input images/parrot.jpg
    
Figure 1. parrot.jpg

You should see results like this:

INFO: Initialized TensorFlow Lite runtime.
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
11.8ms
3.0ms
2.8ms
2.9ms
2.9ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.76562

Congrats! You just performed an inference on the Edge TPU using TensorFlow Lite.

To demonstrate varying inference speeds, the example repeats the same inference five times. It prints the time to perform each inference and the top classification (the label ID/name and the confidence score, from 0 to 1.0). Your inference speeds might differ based on your host system.

The classify_image.py example above uses the TensorFlow Lite Python API. To learn more about how it works, take a look at the classify_image.py source code and read about how to run inference with TensorFlow Lite.

As an alternative to using the TensorFlow Lite API (used above), you can use the Edge TPU Python API, which provides high-level APIs to perform inference with image classification and object detection models with just a few lines of code. For an example, try our other version of classify_image.py using the Edge TPU API.

You can also run inference using C++ and TensorFlow Lite.

Next steps

To run some other types of neural networks, check out our example projects, including examples that perform real-time object detection, pose estimation, keyphrase detection, on-device transfer learning, and more.

If you want to create your own model, see these pages:

Or to create your own model that's compatible with the Edge TPU, read TensorFlow Models on the Edge TPU.

The following section describes how the power throttling works and how to customize the trip points.

Operating frequency and thermal settings

The Mini PCIe/M.2 Accelerator does not include a thermal solution to dissipate heat from the system. In order to sustain maximum performance from the Edge TPU, it's important that you design your system so the Edge TPU operates well below the maximum operating temperature specified in the datasheet. If the Edge TPU gets too hot, it slowly reduces the operating frequency and may reset to avoid permanent damage.

The PCIe driver includes a power throttling mechanism (also known as dynamic frequency scaling) and an emergency shutdown mechanism, based on temperature readings from the Edge TPU. By default, this system checks the Edge TPU temperature every 5 seconds and responds as follows:

  • If the Edge TPU is below 85°C, continue at the "maximum" operating frequency.
  • If the Edge TPU reaches 85°C, reduce the operating frequency 50% (from "maximum" to "normal").
  • If the Edge TPU reaches 90°C, reduce the operating frequency another 50% (from "normal" to "low").
  • If the Edge TPU reaches 95°C, reduce the operating frequency yet another 50% (from "low" to "lowest").
  • If the Edge TPU reaches 100°C, reset the Edge TPU.

By reducing the operating frequency, the Edge TPU's inferencing speed becomes slower, but it also consumes less power and hopefully avoids reaching the hardware reset threshold. But other heat sources in your system can also be a factor.

As long as the chip does not reset and the Edge TPU temperature returns to lower levels, the system restores the operating frequency in the reverse manner—ultimately returning to the maximum operating frequency when the Edge TPU is below 85°C.

Read the Edge TPU temperature

To avoid throttling at all, you might want to periodically read the temperature of the Edge TPU and—if the temperature is nearing the threshold for throttling (85°C, by default)—activate a thermal solution (such as a fan) or load-balance your work across other Edge TPUs in the system.

To read the temperature of an Edge TPU, you can use the following sysfs node: /sys/devices/_pci0000:00/0000:00:01.0/000:01:00.0_/apex/apex_0/temp

The three directories in the middle (pci0000:00/0000:00:01.0/000:01:00.0/) are unique for each Coral PCIe module so will differ from the names shown here.

Customize the trip points

If necessary, you can also adjust the parameters that define the Edge TPU's throttling and reset trip points, effectively disabling all throttling and even disabling the emergency reset—although we recommend that you always keep the emergency reset enabled.

The temperature for each trip point is defined with a file accessible as either a kernel module parameter or a sysfs node:

  • The kernel module parameters are located in this path: /sys/module/apex/parameters/ These parameters are persistent and applied at boot time, and useful if you have multiple modules for which you want to apply the same settings.

  • The sysfs nodes for each module are located in paths such as this: /sys/devices/_pci0000:00/0000:00:01.0/000:01:00.0_/apex/apex_0/The three directories in the middle (pci0000:00/0000:00:01.0/000:01:00.0/) are unique for each Coral PCIe module. These sysfs nodes are created by the PCIe driver at boot time and allow you to set different settings for different modules.

Note: The kernel module parameters (at /sys/module/apex/parameters/) are applied at system boot time, whereas the individual sysfs nodes can be changed at runtime and take immediate effect. To persistently change the kernel module parameters so you don't need to reapply the configuration upon every boot, read how to specify kernel module parameters.

Whether you decide to use the kernel module parameters or the individual sysfs node parameters, the files that specify the trip points for frequency throttling and the hardware reset are named the same, as shown in table 1 and table 2.

Table 1. Parameters for Edge TPU throttling (dynamic frequency scaling)
Filename Description Default value Units
trip_point0_temp If the Edge TPU temperature reaches or exceeds this value, the system sets the operating frequency to "normal" (50% of maximum) 85000 Millicelcius
trip_point1_temp If the Edge TPU temperature reaches or exceeds this value, the system sets the operating frequency to "low" (25% of maximum) 90000 Millicelcius
trip_point2_temp If the Edge TPU temperature reaches or exceeds this value, the system sets the operating frequency to "lowest" (~13% of maximum) 95000 Millicelcius
temp_poll_interval The interval at which to read the temperature (setting to 0 disables polling) 5000 Milliseconds

Whatever values you set for the trip_point* parameters, they must evaluate as follows: trip_point0_temp <= trip_point1_temp <= trip_point2_temp If you set values that don't match this logic, then the driver reverts to use the default values shown in table 1.

Note: We suggest that temp_poll_interval be a value of several seconds because the temperature doesn't change instantly (in response to changes in operational load or changes to the operating frequency). Yet, it also doesn't need to be much larger than the default value because the overhead of switching the operating frequency is negligible, so it isn't necessary to implement hysteresis around your trip points.
Table 2. Parameters for Edge TPU emergency hardware reset
Filename Description Default value Units
hw_temp_warn2 If the Edge TPU temperature reaches or exceeds this value, the Edge TPU resets 100000 Millicelcius
hw_temp_warn2_en Disable/enable emergency hardware reset 1 Boolean (1 is enabled, 0 is disabled)

When the Edge TPU resets, it enters an idle state. Generally, you must then restart your system to resume work with the Edge TPU.

Warning: Disabling the emergency reset or increasing the trip point above 100°C (100000 millicelcius) removes thermal protection of the device, which can result in permanent damage to the Edge TPU and surrounding components, and can possibly cause fire and other serious damage, injury, or death.
Note: The hardware reset parameters are saved to Edge TPU registers and the reset mechanism is managed completely inside the Edge TPU chip. So even if the host system fails, the Edge TPU will safely reset if it becomes too hot.

Troubleshooting

Here are some solutions to possible problems.

HIB error

If you are running on ARM64 platform and receive error messages such as the following when you run an inference...

HIB Error. hib_error_status = 0000000000002200, hib_first_error_status = 0000000000000200

... You should be able to solve it if you modify your kernel command line arguments to include gasket.dma_bit_mask=32.

For information about how to modify your kernel command line arguments, refer to your respective platform documentation. For bootloaders based on U-Boot, you can usually modify the arguments either by modifying the bootargs U-Boot environment variable or by setting othbootargs environment variable as follows:

=> setenv othbootargs gasket.dma_bit_mask=32
=> printenv othbootargs
othbootargs=gasket.dma_bit_mask=32
=> saveenv

If you make the above change and then receive errors such as, DMA: Out of SW-IOMMU space, then you need to increase the swiotlb buffer size by adding another kernel command line argument: swiotlb=65536.

pcieport error

If you see a lot of errors such as the following:

pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0008(Transmitter ID)
pcieport 0000:00:01.0: device [10de:0fae] error status/mask=00003100/00002000
pcieport 0000:00:01.0: [ 8] RELAY_NUM Rollover
pcieport 0000:00:01.0: [12] Replay Timer Timeout
pcieport 0000:00:01.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0008(Requester ID)
pcieport 0000:00:01.0: device [10de:0fae] error status/mask=00004000/00000000

... You should be able to solve it if you modify your kernel command line arguments to include pcie_aspm=off.

For information about how to modify your kernel command line arguments, refer to your respective platform documentation. If your device includes U-Boot, see the previous HIB error for an example of how to modify the kernel commands. For certain other devices, you might instead add pcie_aspm=off to an APPEND line in your system /boot/extlinux/extlinux.conf file:

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} quiet pcie_aspm=off

Workaround to disable Apex and Gasket

The following procedure is necessary only if your system includes a pre-build driver for Apex devices (as per the first steps for installing the PCIe driver). Due to a bug, updating this driver with ours can fail, so you need to first disable the apex and gasket modules as follows:

  1. Create a new file at /etc/modprobe.d/blacklist-apex.conf and add these two lines:

    blacklist gasket
    blacklist apex
    
  2. Reboot the system.

  3. Verify that the apex and gasket modules did not load by running this:

    lsmod | grep apex
    

    It should print nothing.

  4. Now follow the rest of the steps to install the PCIe driver.

  5. Finally, delete /etc/modprobe.d/blacklist-apex.conf and reboot your system.