API reference

Data models

common.data_models

Data models used throughout the project.

Vector3 dataclass

A 3D vector.

IMUData dataclass

Inertial measurement unit data.

Attributes:

Name	Type	Description
accelerometer	Vector3 | None	Acceleration in m/s^2, or None if not enabled.
gyroscope	Vector3 | None	Angular velocity in rad/s, or None if not enabled.
magnetometer	Vector3 | None	Magnetic field in Tesla, or None if not enabled.

Metaframe dataclass

A synchronized set of frames captured from a device at one instant.

Attributes:

Name	Type	Description
timestamp	datetime	When this set of frames was captured.
rgb	ndarray | None	Color image, or None if not provide it.
left	ndarray | None	Left mono image, or None if not provided.
right	ndarray | None	Right mono image, or None if not provided.
depth	ndarray | None	Depth map, or None if not provided.
imu	IMUData | None	IMU data, or None if not provided.

A field is None when that stream is not produced by the camera (e.g. it was disabled, or the camera does not support it).

Camera abstraction

common.generic_camera

Hardware-agnostic camera abstraction.

Camera

Bases: ABC

Hardware-agnostic interface for a stereo depth camera.

Any camera backend (e.g. OAK-D) plugs into the rest of the project by implementing this interface, so swapping hardware only means writing a new implementation, not touching the application code.

Implementations are expected to be used as a context manager, e.g.::

with camera:
    camera.start()
    while camera.is_running():
        frames = camera.get_frames()
    camera.stop()

__exit__ must release every hardware resource acquired by __enter__/start, even when an exception was raised anywhere inside the with block.

__enter__() -> Camera abstractmethod

Acquire the camera and prepare it to start streaming.

__exit__(exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None) -> None abstractmethod

Release the camera, regardless of whether an error occurred.

start() -> None abstractmethod

Start streaming frames from the camera.

stop() -> None abstractmethod

Stop streaming frames from the camera.

is_running() -> bool abstractmethod

Return whether the camera is currently streaming.

get_frames() -> Metaframe abstractmethod

Block until the next synchronized set of frames is available.

OAK-D camera

hardware.oakd.camera

OAK-D camera implementation.

OakDCamera

Bases: Camera

Camera implementation for the OAK-D stereo camera.

__init__(enable_rgb: bool = True, enable_left: bool = True, enable_right: bool = True, enable_depth: bool = True, enable_imu: bool = True) -> None

Configure which streams the pipeline will produce.

Parameters:

Name	Type	Description	Default
enable_rgb	bool	Whether to build and stream the color camera.	True
enable_left	bool	Whether to build and stream the left mono camera.	True
enable_right	bool	Whether to build and stream the right mono camera.	True
enable_depth	bool	Whether to compute and stream the stereo depth map. Requires the left and right cameras, which are enabled automatically (with a warning) if not requested.	True
enable_imu	bool	Whether to enable the IMU.	True

__enter__() -> Self

Open the pipeline and build the requested nodes.

__exit__(exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None) -> None

Stop the pipeline and release the device.

start() -> None

Start the pipeline so frames begin flowing.

stop() -> None

Stop the pipeline.

is_running() -> bool

Return whether the pipeline is currently running.

Returns:

Type	Description
bool	True if the pipeline is running, False otherwise.

get_frames() -> Metaframe

Block until the next synchronized frame is available.

Returns:

Type	Description
Metaframe	A Metaframe from the sync node, timestamped at the
Metaframe	moment frames were synchronized, with a None value for every
Metaframe	stream that was not enabled.

Webcam

hardware.webcam.camera

Webcam camera implementation.

WebcamCamera

Bases: Camera

Camera implementation for a webcam exposed through OpenCV.

__init__(device_index: int = 0) -> None

Configure which webcam device to use.

Parameters:

Name	Type	Description	Default
device_index	int	OpenCV device index, usually 0 for the default webcam.	0

__enter__() -> Self

Open the webcam device.

__exit__(exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None) -> None

Stop streaming and release the webcam device.

start() -> None

Start reading frames from the webcam.

stop() -> None

Stop reading frames from the webcam.

is_running() -> bool

Return whether the webcam is currently streaming.

get_frames() -> Metaframe

Read the next webcam frame.

Returns:

Type	Description
Metaframe	A Metaframe containing the webcam's color frame in rgb.

Display tools

theia.display_streams

Stream and display the camera data.

display_camera_streams(camera: Camera, display_rgb: bool = True, display_left: bool = True, display_right: bool = True, display_depth: bool = True) -> None

Stream whichever views the given Camera implementation provides.

Parameters:

Name	Type	Description	Default
camera	Camera	Any Camera implementation to stream from.	required
display_rgb	bool	Whether to display the RGB view.	True
display_left	bool	Whether to display the left view.	True
display_right	bool	Whether to display the right view.	True
display_depth	bool	Whether to display the depth view.	True

CLI

theia.cli

Application entry point for Theia CLI commands.

display_streams(camera_type: str, webcam_index: int, no_rgb: bool, no_left: bool, no_right: bool, no_depth: bool) -> None

Display camera streams in a GUI window.

Parameters:

Name	Type	Description	Default
camera_type	str	Camera hardware to use.	required
webcam_index	int	OpenCV device index when using a webcam.	required
no_rgb	bool	If True, disable the RGB stream.	required
no_left	bool	If True, disable the left stereo stream.	required
no_right	bool	If True, disable the right stereo stream.	required
no_depth	bool	If True, disable the depth stream.	required