Camera APIs

The Dev Board Micro includes an on-board camera module with 324 x 324 px resolution. All camera control is handled through the CameraTask singleton. You can capture images one-at-a-time or receive a stream of images. The camera also offers hardware-based motion detection.

To get started, you must acquire the CameraTask object with GetSingleton(). Then power on the camera with SetPower() and specify the camera mode (trigger or streaming mode) with Enable().

If you enable trigger mode, the camera captures a single image and saves it to the camera’s memory when you call Trigger().

If you enable streaming mode, the camera continuously captures new images and saves them to an internal buffer.

In either mode, you must manually fetch the latest image by calling GetFrame(), which requires you to specify the image format you want with CameraFrameFormat. This object also specifies your own buffer where you want to save the processed image. You can specify multiple formats for each frame, which is useful if you want one format to use as input for your ML model and another format to use for display.

Photo capture example:

The following code enables the camera with trigger mode so it will capture an image when you press the User button on the Dev Board Micro. The image is saved internally in raw format until the app fetches it with GetFrame(), which is called by the GetCapturedImage() RPC function.

This example is available in coralmicro/examples/camera_triggered/, which also provides a Python client to fetch images over RPC.

namespace coralmicro {
namespace {
void GetCapturedImage(struct jsonrpc_request* request) {
  int width;
  int height;

  // The width and height are specified by the RPC client.
  if (!JsonRpcGetIntegerParam(request, "width", &width)) return;
  if (!JsonRpcGetIntegerParam(request, "height", &height)) return;

  auto format = CameraFormat::kRgb;
  std::vector<uint8_t> image(width * height * CameraFormatBpp(format));
  CameraFrameFormat fmt{format,
                        CameraFilterMethod::kBilinear,
                        CameraRotation::k270,
                        width,
                        height,
                        /*preserve_ratio=*/false,
                        /*buffer=*/image.data(),
                        /*white_balance=*/true};
  if (!CameraTask::GetSingleton()->GetFrame({fmt})) {
    jsonrpc_return_error(request, -1, "Failed to get image from camera.",
                         nullptr);
    return;
  }

  jsonrpc_return_success(request, "{%Q: %d, %Q: %d, %Q: %V}", "width", width,
                         "height", height, "base64_data", image.size(),
                         image.data());
}

[[noreturn]] void Main() {
  printf("Camera Triggered Example!\r\n");
  // Turn on Status LED to show the board is on.
  LedSet(Led::kStatus, true);

  // Starting Camera in triggered mode.
  CameraTask::GetSingleton()->SetPower(true);
  CameraTask::GetSingleton()->Enable(CameraMode::kTrigger);

  // Set up an RPC server that serves the latest image.
  jsonrpc_export("get_captured_image", GetCapturedImage);
  UseHttpServer(new JsonRpcHttpServer);

  // Register callback for the user button.
  printf("Press the user button to take a picture.\r\n");
  GpioConfigureInterrupt(
      Gpio::kUserButton, GpioInterruptMode::kIntModeFalling,
      [handle = xTaskGetCurrentTaskHandle()]() { xTaskResumeFromISR(handle); },
      /*debounce_interval_us=*/50 * 1e3);
  while (true) {
    vTaskSuspend(nullptr);
    CameraTask::GetSingleton()->Trigger();
    printf("Picture taken\r\n");
  }
}
}  // namespace
}  // namespace coralmicro

extern "C" void app_main(void* param) {
  (void)param;
  coralmicro::Main();
}

Motion detection example:

This example turns on the User LED whenever motion is detected, passing the callback function a timer that turns the LED back off after a 200ms delay.

namespace coralmicro {
namespace {
void Main() {
  printf("Camera Motion Detection example\r\n");
  // Turn on Status LED to show the board is on.
  LedSet(Led::kStatus, true);

  TimerHandle_t motion_detection_timer = xTimerCreate(
      "motion_detection_timer", pdMS_TO_TICKS(200), pdFALSE, nullptr,
      [](TimerHandle_t timer) { LedSet(Led::kUser, false); });
  CHECK(motion_detection_timer);

  // Enable Power, configure motion detection, and enable streaming.
  CameraTask::GetSingleton()->SetPower(true);

  CameraMotionDetectionConfig config{};
  CameraTask::GetSingleton()->GetMotionDetectionConfigDefault(config);
  config.cb = [](void* param) {
    auto timer = reinterpret_cast<TimerHandle_t>(param);
    CHECK(timer);
    printf("Motion detected\r\n");
    LedSet(Led::kUser, true);
    if (xTimerIsTimerActive(timer) == pdTRUE) {
      xTimerReset(timer, portMAX_DELAY);
    } else {
      xTimerStart(timer, portMAX_DELAY);
    }
  };
  config.cb_param = motion_detection_timer;

  CameraTask::GetSingleton()->SetMotionDetectionConfig(config);
  CameraTask::GetSingleton()->Enable(CameraMode::kStreaming);

  vTaskSuspend(nullptr);
}
}  // namespace
}  // namespace coralmicro

extern "C" void app_main(void* param) {
  (void)param;
  coralmicro::Main();
  vTaskSuspend(nullptr);
}

[camera.h source]

namespace coralmicro

struct CameraFrameFormat¶

#include <camera.h>

Specifies your image buffer location and any image processing you want to perform when fetching images with CameraTask::GetFrame().

Public Members

CameraFormat fmt¶

Image format such as RGB or raw.

CameraFilterMethod filter = CameraFilterMethod::kBilinear¶

Filter method such as bilinear (default) or nearest-neighbor.

CameraRotation rotation = CameraRotation::k270¶

Image rotation in 90-degree increments. Default is 270 degree which corresponds to the device held vertically with USB port facing down.

int width¶

Image width. (Native size is CameraTask::kWidth.)

int height¶

Image height. (Native size is CameraTask::kHeight.)

bool preserve_ratio¶

If using non-native width/height, set this true to maintain the native aspect ratio, false to crop the image.

uint8_t *buffer¶

Location to store the image.

bool white_balance = true¶

Set true to perform auto whitebalancing (default), false to disable it.

struct CameraMotionDetectionConfig¶

#include <camera.h>

Specifies the configuration for motion detection (performed in camera).

You can pre-fill this configuration with defaults by passing an instance to CameraTask::GetMotionDetectionConfigDefault(). Then specify your callback function (the cb variable) and pass this config to CameraTask::SetMotionDetectionConfig(). You must also enable camera streaming mode with CameraTask::Enable().

The default configuration detects any movement in the image frame, but you can specify a smaller detection region by defining a bounding box with the x0, y0, x1, and y1 variables.

Public Members

CameraMotionDetectionCallback cb¶

The callback function to call when the camera detects motion. The default config is nullptr.

void *cb_param¶

Optional parameters to pass to the callback function. The default config is nullptr.

bool enable¶

Set true to enable motion detection; false to disable it. The default config is true.

size_t x0¶

The detection zone’s left-most pixel (index position). The default config is 0.

size_t y0¶

The detection zone’s top-most pixel (index position). The default config is 0.

size_t x1¶

The detection zone’s right-most pixel (index position). The default config is CameraTask::kWidth - 1 (323).

size_t y1¶

The detection zone’s left-most pixel (index position). The default config is CameraTask::kHeight - 1 (323).

class CameraTask : public coralmicro::QueueTask<camera::Request, camera::Response, kCameraTaskName, configMINIMAL_STACK_SIZE * 10, kCameraTaskPriority, 4>¶

#include <camera.h>

Provides access to the Dev Board Micro camera.

You can access the shared camera object with CameraTask::GetSingleton().

Public Functions

void Init(lpi2c_rtos_handle_t *i2c_handle)¶

Initializes the camera.

Programs on the M7 do not need to call this because it is automatically called internally. M7 programs can immediately turn on the camera with SetPower().

Programs on the M4 must call this to intialize the camera before they can turn on the camera. For example:

CameraTask::GetSingleton()->Init(I2C5Handle());
CameraTask::GetSingleton()->SetPower(true);

Parameters

i2c_handle – The camera I2C handle: I2C5Handle().

bool Enable(CameraMode mode)¶

Enables the camera to begin capture. You must call SetPower() before this.

Parameters

mode – The operating mode (either kStreaming or kTrigger).

Returns

True if camera is enabled, false otherwise.

void Disable()¶

Sets the camera into a low-power state, using appoximately 200 μW (compared to approximately 4 mW when streaming). The camera configuration is sustained so it can quickly start again with Enable().

bool GetFrame(const std::vector<CameraFrameFormat> &fmts)¶

Gets one frame from the camera buffer and processes it into one or more formats.

Parameters

fmts – A list of image formats you want to receive.

Returns

True if image processing succeeds, false otherwise.

Note

This blocks until a new frame is available from the camera. However, if trigger mode, it returns false if the camera has not been trigged (via CameraTask::Trigger) since the last time CameraTask::GetFrame was called.

bool SetPower(bool enable)¶

Turns the camera power on and off. You must call this before Enable().

Parameters

enable – True to turn the camera on, false to turn it off.

Returns

True if the action was successful, false otherwise.

void SetTestPattern(CameraTestPattern pattern)¶

Enables a camera test pattern instead of using actual sensor data.

Parameters

pattern – The test pattern to use.

void Trigger()¶

Triggers image capture when the camera is enabled with CameraMode::kTrigger.

The raw image is held in the camera module memory and you must then fetch it with GetFrame().

void DiscardFrames(int count)¶

Purges the image sensor data one frame at a time.

This essentially captures images without saving any of the data, which allows the sensor to calibrate exposure and rid the sensor of any image artifacts that sometimes occur upon initialization.

Parameters

The – number of frames to capture and immediately discard. To allow auto exposure to calibrate, try discarding 100 frames before you begin using images with GetFrame().

void GetMotionDetectionConfigDefault(CameraMotionDetectionConfig &config)¶

Gets the default configuration for motion detection.

Parameters

config – The CameraMotionDetectionConfig struct to fill with default values.

void SetMotionDetectionConfig(const CameraMotionDetectionConfig &config)¶

Sets the configuration for hardware motion detection.

Note: You must enable camera streaming mode with CameraTask::Enable().

Parameters

config – CameraMotionDetectionConfig to apply to the camera.

Public Static Attributes

static constexpr size_t kWidth = 324¶

Native image pixel width.

static constexpr size_t kHeight = 324¶

Native image pixel height.

Public Static Functions

static inline CameraTask *GetSingleton()¶

Gets the CameraTask singleton.

You must use this to acquire the shared CameraTask object.

Functions

int CameraFormatBpp(CameraFormat fmt)¶

Gets the bytes-per-pixel (the number of color channels) used by the given image format.

Parameters

The – image format (from CameraFormat).

Returns

The number of bytes per pixel.

Enums

enum class CameraMode : uint8_t¶

The camera operating mode for CameraTask::Enable().

Values:

enumerator kStreaming¶

Streaming mode. The camera continuously captures and pushes raw images to an internal image buffer. You can then fetch images one at a time in your preferred format with CameraTask::GetFrame().

enumerator kTrigger¶

Trigger mode. The camera captures one image at a time when you call CameraTask::Trigger(). You can then fetch each image and process it into your preferred format with CameraTask::GetFrame().

enum class CameraTestPattern : uint8_t¶

Test patterns to use with CameraTask::SetTestPattern()

Values:

enumerator kNone¶

enumerator kColorBar¶

enumerator kWalkingOnes¶

enum class CameraFormat¶

Image format options, used with CameraFrameFormat.

Values:

enumerator kRgb¶

RGB image.

enumerator kY8¶

Y8 (grayscale) image.

enumerator kRaw¶

Raw bayer image.

enum class CameraFilterMethod¶

Image resampling method (when resizing the image).

Values:

enumerator kBilinear¶

enumerator kNearestNeighbor¶

enum class CameraRotation¶

Clockwise image rotations.

Values:

enumerator k0¶

The natural orientation for the camera module.

enumerator k90¶

Rotated 90-degrees clockwise. Upside down, relative to the board’s “Coral” label.

enumerator k180¶

Rotated 180-degrees clockwise.

enumerator k270¶

Rotated 270-degrees clockwise. Right-side up, relative to the board’s “Coral” label.