Illustrated C/C++ source sheets, symbol index, API relationship graph, compiler console and debug trace

LIVE AI / FLEXIBLE SENSING / FLIGHT CONTEXT

Sensoraia API

Live AI and sensing, connected to the flight.
See, listen, inspect and return supported results while airborne.

Loading...
Searching...
No Matches
imx500_camera.hpp
Go to the documentation of this file.
1/*
2 * Copyright 2026 OpenUAS
3 * SPDX-License-Identifier: Apache-2.0
4 *
5 * Licensed under the Apache License, Version 2.0 (the "License");
6 * you may not use this file except in compliance with the License.
7 * You may obtain a copy of the License at
8 *
9 * http://www.apache.org/licenses/LICENSE-2.0
10 *
11 * Unless required by applicable law or agreed to in writing, software
12 * distributed under the License is distributed on an "AS IS" BASIS,
13 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14 * See the License for the specific language governing permissions and
15 * limitations under the License.
16 */
17
18/** @file imx500_camera.hpp
19 * @brief Neutral IMX500 frame ownership, JPEG publication and worker protocol.
20 * @details Model interpretation belongs to the caller. Tensor views are borrowed
21 * from one retained camera request and may not outlive its FrameLease. */
22#ifndef SENSORAIA_IMX500_CAMERA_HPP
23#define SENSORAIA_IMX500_CAMERA_HPP
24
25#include <array>
26#include <cstdint>
27#include <memory>
28#include <libcamera/framebuffer.h>
29#include <libcamera/geometry.h>
30#include <libcamera/base/span.h>
31#include "exifite.h"
32
33namespace imx500 {
34
35namespace detail {
37/** @brief Map libcamera metadata to this worker's frame disposition.
38 * @param status Driver-reported frame status.
39 * @return Ready, startup, or failed.
40 * @details Startup frames are not usable captures, but unlike failures
41 * they are expected while the camera pipeline settles. */
42constexpr FrameStatus frame_status(libcamera::FrameMetadata::Status status)
43{
44 return status == libcamera::FrameMetadata::FrameSuccess ? FrameStatus::ready
45 : status == libcamera::FrameMetadata::FrameStartup ? FrameStatus::startup
47}
48}
49
50#ifdef SENSORAIA_LIBCAMERA_STD_SPAN
51template <typename Element, std::size_t Extent = std::dynamic_extent>
52using Span = std::span<Element, Extent>;
53#else
54using libcamera::Span;
55#endif
56
57/** @brief Physical IMX500 array width, used to reject an unexpected sensor geometry. */
58constexpr unsigned sensor_width = 4056;
59/** @brief Physical IMX500 array height; model input dimensions are separate policy. */
60constexpr unsigned sensor_height = 3040;
61
62/** @brief Check that capture dimensions meet the worker's raster policy.
63 * @param width Requested columns.
64 * @param height Requested rows.
65 * @return True for an even raster within the physical array and minimum dimensions.
66 * @details Check here before driver setup; this does not guarantee that the
67 * driver will accept a specific format or stream configuration. */
68constexpr bool valid_capture_dimensions(unsigned width, unsigned height)
69{
70 return width >= 16 && width <= sensor_width && height >= 2 && height <= sensor_height
71 && width % 2 == 0 && height % 2 == 0;
72}
73
74/** @brief Borrowed same-exposure evidence, without object or navigation semantics.
75 * @details Views avoid copying neural outputs into another queue. Keep the
76 * associated lease alive while reading any tensor or interpreting its crop. */
77struct FrameView {
78 Span<const float> tensor; /**< Borrowed output values from the retained request. */
79 Span<const uint8_t> tensor_info; /**< Matching driver shape metadata, not model semantics. */
80 libcamera::Rectangle crop{}; /**< Sensor-space crop used by consumers such as box projection. */
81 libcamera::Size image{}; /**< Configured JPEG dimensions, not the neural model input size. */
82 uint64_t sequence = 0; /**< Camera request sequence; deliberately distinct from an FC shot number. */
83 uint64_t sensor_timestamp_ns = 0; /**< Sensor timestamp in its camera clock domain; zero when absent. */
84 int64_t exposure_us = -1; /**< Reported exposure duration; negative means unavailable. */
85 int64_t frame_duration_us = -1; /**< Reported frame duration; negative means unavailable. */
86 uint64_t monotonic_us = 0;
87 uint64_t time_error_us = 100001;
88 bool timing_sampled = false;
89 bool crop_valid = false; /**< Distinguishes a reported crop from zero-initialized placeholder geometry. */
90};
91
92namespace detail {
93/** @brief Confirm that a frame carries no neural output.
94 * @param frame Borrowed view of the completed request.
95 * @return True only if both tensor data and shape metadata are absent.
96 * @details Check both views so partially exposed inference metadata is not
97 * mistaken for a deliberately photo-only request. */
98inline bool photo_only_frame(const FrameView &frame)
99{
100 return frame.tensor.empty() && frame.tensor_info.empty();
101}
102}
103
104/** @brief Stack-owned model-specific operations; no dynamic plugin allocation.
105 * @details validate qualifies readiness. write validates and publishes its own
106 * analysis file; false or an exception must leave no partial analysis file.
107 * Keep this contract small: capture mechanics should not learn detector boxes,
108 * LOOM feature semantics or route-selection policy. */
110 const char *sidecar_suffix; /**< Trusted application-owned suffix with a lifetime covering the call. */
111 bool (*validate)(const FrameView &frame); /**< Qualify model-specific readiness without retaining borrowed data. */
112 bool (*write)(const FrameView &frame, const char *filename); /**< Exclusive-create analysis publication; failure must clean up its own partial file. */
113 bool (*embed)(const FrameView &frame, exif_document *document) = nullptr;
114};
115
116/** @brief Exactly-once release of a borrowed frame, including exception exits.
117 * @details The capture owner must outlive its lease. Only one lease per Camera
118 * may exist. Move transfers ownership; copies and move assignment are disabled. */
120public:
121 /** @brief Nonthrowing release callback; a destructor must never unwind through it. */
122 using Release = void (*)(void *) noexcept;
123
124 /** @brief Associate borrowed evidence with exactly one release operation.
125 * @param view Evidence whose backing storage is retained by @p owner.
126 * @param owner Borrowed capture-owner context, alive until release completes.
127 * @param release Callback to recycle the held request, or NULL for an unowned view.
128 * @details Camera constructs leases before interpreting metadata so failures
129 * during extraction also release the request. */
130 FrameLease(FrameView view, void *owner, Release release) noexcept;
131 /** @brief Release once on every scope exit, including a processing exception. */
132 ~FrameLease();
133 /** @brief Transfer release responsibility and empty the moved-from view.
134 * @param other Lease to consume; it will no longer recycle the request. */
135 FrameLease(FrameLease &&other) noexcept;
136 /** @brief Copies are forbidden because two destructors cannot own one request. */
137 FrameLease(const FrameLease &) = delete;
138 /** @brief Copy assignment cannot duplicate capture ownership. */
139 FrameLease &operator=(const FrameLease &) = delete;
140 /** @brief Assignment is disabled; move construction is the supported transfer. */
142
143 /** @brief Read evidence only while this unmoved lease remains alive.
144 * @return Borrowed immutable view; its tensor spans have the lease's lifetime. */
145 const FrameView &view() const noexcept { return view_; }
146
147private:
148 friend class Camera;
149 FrameView view_; /**< Borrowed metadata, never a second image/tensor allocation. */
150 void *owner_; /**< Context whose lifetime must exceed this lease. */
151 Release release_; /**< Sole remaining release operation, NULL after a move. */
152};
153
154/** @brief One exclusive IMX500 owner with bounded reusable YUV420 buffers.
155 * @pre Control operations and lease destruction are serialized by their owner;
156 * the camera's callback thread is synchronized internally. */
157class Camera {
158public:
159 /** @brief Construct unopened camera state; construction does not acquire the sensor. */
160 Camera();
161 /** @brief Stop streaming before freeing requests/buffers and releasing the camera.
162 * @pre Every frame lease has already been destroyed. */
164 /** @brief Camera owners cannot be copied. */
165 Camera(const Camera &) = delete;
166 /** @brief Assignment cannot replace an acquired sensor owner. */
167 Camera &operator=(const Camera &) = delete;
168
169 /** @brief Acquire one eligible camera and configure bounded capture, uploading an RPK only when requested.
170 * @param model Readable RPK for IMX500 inference, or NULL for plain pixels.
171 * @param width Requested JPEG width, not model-input width.
172 * @param height Requested JPEG height, not model-input height.
173 * @param photo_only Select IMX500 image-only capture and leave frame duration to libcamera.
174 * @throws std::runtime_error for unavailable/ambiguous hardware or rejected configuration.
175 * @details Configuration adjustments that change our expected geometry, YUV420
176 * layout or buffer budget are rejected rather than silently changing downstream assumptions. */
177 void open(const char *model, unsigned width = 2028, unsigned height = 1520,
178 bool photo_only = false);
179 /** @brief Lease a fresh post-request exposure and its own tensor metadata.
180 * @return Move-only lease; destroy it before the next capture or camera shutdown.
181 * @throws std::runtime_error for missing startup, a still-live prior lease,
182 * queue failure, unavailable clock or frame timeout.
183 * @details Freshness uses the sensor/BOOTTIME domain, not JPEG download time.
184 * Model-specific validation remains the processor's responsibility. */
186 /** @brief Map a leased frame and publish its JPEG plus available evidence.
187 * @param frame Lease obtained from this camera, still alive and unmoved.
188 * @param filename New absolute JPEG destination; existing images are not overwritten.
189 * @param processor Application-owned sidecar policy for this same exposure.
190 * @return True after JPEG publication, false for rejected layout/path or write failure.
191 * @details Mapping is scoped separately from the lease: CPU DMA ownership ends
192 * before the request can be recycled. Missing analysis does not discard good pixels.
193 * @warning Success is not proof of a useful neural match, geolocation accuracy,
194 * complete multi-file crash atomicity or parent-directory power-loss durability. */
195 bool save(const FrameLease &frame, const char *filename, const FrameProcessor &processor);
196 /** @brief Grant scoped CPU pixel access without transferring DMA/frame ownership.
197 * @param frame Live lease acquired from this camera.
198 * @param visitor Callback consuming validated YUV420 planes and stride before mappings are released.
199 * @param context Borrowed callback state, not retained by the camera.
200 * @return Callback result, or false for invalid ownership/layout/visitor.
201 * @pre The visitor must not retain pixel pointers or release the lease during the call.
202 * @details Prepare a model-sized tensor here, then release the lease before expensive inference. */
203 bool with_pixels(const FrameLease &frame,
204 bool (*visitor)(const FrameView &, const std::array<const uint8_t *, 3> &,
205 unsigned, void *), void *context);
206
207private:
208 struct Impl; /**< Camera/request implementation hidden from model-specific callers. */
209 std::unique_ptr<Impl> impl_; /**< One owned capture state; libcamera has its own allocation policy. */
210};
211
212/** @brief Encode validated YUV420 planes directly, without full-frame RGB staging.
213 * @param path New output path; exclusive creation refuses overwrites.
214 * @param planes Accessible Y, Cb and Cr plane starts, immutable during encoding.
215 * @param width Positive even image width in pixels.
216 * @param height Positive even image height in pixels.
217 * @param stride Even luma byte stride, at least the width rounded up to 16; chroma uses half.
218 * @return True after encoding, flush, file sync and close; false removes partial output.
219 * @pre The caller has checked plane lengths and image/stride layout. This low-level
220 * encoder does not accept arbitrary unchecked external plane pointers. */
221bool encode_jpeg(const char *path, const std::array<const uint8_t *, 3> &planes,
222 unsigned width, unsigned height, unsigned stride);
223
224/** @brief Publish caller-validated YUV420 planes and same-exposure evidence.
225 * @param view Borrowed evidence belonging to the supplied pixels.
226 * @param planes Accessible YUV420 planes retained by the caller.
227 * @param stride Luma byte stride; chroma uses half this stride.
228 * @param filename New absolute JPEG destination.
229 * @param processor Application's trusted sidecar suffix and writer callbacks.
230 * @return True when the JPEG is published, regardless of missing analysis;
231 * false when publication fails. File/processor exceptions may also propagate.
232 * @pre Plane spans cover the declared image/stride and remain immutable for this
233 * call. Capture retains its frame lease while invoking this routine.
234 * @details Analysis failure retains a usable photograph. Publication failure
235 * removes successfully created sidecars from this operation, never an existing image.
236 * @warning JPEG and sidecars are not one filesystem transaction. Consumers must
237 * check file/exposure identity, and directory power-loss durability is not guaranteed. */
238bool publish_frame(const FrameView &frame, const std::array<const uint8_t *, 3> &planes,
239 unsigned stride, const char *filename, const FrameProcessor &processor);
240
241/** @brief Run the shared probe/capture/READY-SHOT-OK worker with supplied model policy.
242 * @param argc Argument count for a capture mode, excluding saved-photo matcher modes.
243 * @param argv Mode, RPK and mode-specific dimensions/destination tokens.
244 * @param processor Readiness and sidecar callbacks; never interpreted as navigation here.
245 * @param application Diagnostic prefix selected by the independent entry point.
246 * @return Zero for success, two for syntax/configuration misuse, one for runtime failure.
247 * @details The worker announces READY after a fresh frame and, for detection,
248 * model-specific validation. Every SHOT leases a new exposure without replay.
249 * Standard output is protocol-only while diagnostics go to standard error. */
250int run_worker(int argc, char **argv, const FrameProcessor &processor, const char *application);
251
252/** @brief Run photo-only probe, capture or serve without model loading or inference.
253 * @param argc Argument count for the photo-only command.
254 * @param argv Command and mode-specific dimensions or destination.
255 * @param application Diagnostic prefix.
256 * @return Zero for success, two for invalid arguments, one for runtime failure. */
257int run_photo_worker(int argc, char **argv, const char *application);
258
259} // namespace imx500
260#endif
int(* open)(const struct usbfs_device *, struct usbfs_handle *)
Definition chdkbridge.c:56
Camera & operator=(const Camera &)=delete
Assignment cannot replace an acquired sensor owner.
bool with_pixels(const FrameLease &frame, bool(*visitor)(const FrameView &, const std::array< const uint8_t *, 3 > &, unsigned, void *), void *context)
Grant scoped CPU pixel access without transferring DMA/frame ownership.
bool save(const FrameLease &frame, const char *filename, const FrameProcessor &processor)
Map a leased frame and publish its JPEG plus available evidence.
~Camera()
Stop streaming before freeing requests/buffers and releasing the camera.
Camera(const Camera &)=delete
Camera owners cannot be copied.
std::unique_ptr< Impl > impl_
Camera()
Construct unopened camera state; construction does not acquire the sensor.
FrameLease capture()
Lease a fresh post-request exposure and its own tensor metadata.
Exactly-once release of a borrowed frame, including exception exits.
FrameLease(const FrameLease &)=delete
Copies are forbidden because two destructors cannot own one request.
FrameLease(FrameView view, void *owner, Release release) noexcept
Associate borrowed evidence with exactly one release operation.
void(*)(void *) noexcept Release
Nonthrowing release callback; a destructor must never unwind through it.
FrameLease & operator=(FrameLease &&)=delete
Assignment is disabled; move construction is the supported transfer.
~FrameLease()
Release once on every scope exit, including a processing exception.
const FrameView & view() const noexcept
Read evidence only while this unmoved lease remains alive.
FrameLease & operator=(const FrameLease &)=delete
Copy assignment cannot duplicate capture ownership.
static bool ready
Bounded C11 EXIF metadata and JPEG streaming interfaces.
uint8_t frame[LWIR_MAX_FRAME_BYTES]
Definition lwir_mock.c:58
constexpr FrameStatus frame_status(libcamera::FrameMetadata::Status status)
Map libcamera metadata to this worker's frame disposition.
bool photo_only_frame(const FrameView &frame)
Confirm that a frame carries no neural output.
bool encode_jpeg(const char *path, const std::array< const uint8_t *, 3 > &planes, unsigned width, unsigned height, unsigned stride)
Encode validated YUV420 planes directly, without full-frame RGB staging.
int run_worker(int argc, char **argv, const FrameProcessor &processor, const char *application)
Run the shared probe/capture/READY-SHOT-OK worker with supplied model policy.
constexpr bool valid_capture_dimensions(unsigned width, unsigned height)
Check that capture dimensions meet the worker's raster policy.
bool publish_frame(const FrameView &view, const std::array< const uint8_t *, 3 > &planes, unsigned stride, const char *filename, const FrameProcessor &processor)
Publish caller-validated YUV420 planes and same-exposure evidence.
int run_photo_worker(int argc, char **argv, const char *application)
Run photo-only probe, capture or serve without model loading or inference.
constexpr unsigned sensor_height
Physical IMX500 array height; model input dimensions are separate policy.
constexpr unsigned sensor_width
Physical IMX500 array width, used to reject an unexpected sensor geometry.
Caller-owned bounded writer with no heap allocation.
Definition exifite.h:156
Capture resources and the callback-to-owner handoff, not model results.
Stack-owned model-specific operations; no dynamic plugin allocation.
bool(* validate)(const FrameView &frame)
bool(* embed)(const FrameView &frame, exif_document *document)
bool(* write)(const FrameView &frame, const char *filename)
Borrowed same-exposure evidence, without object or navigation semantics.
Span< const uint8_t > tensor_info
Span< const float > tensor
libcamera::Size image
libcamera::Rectangle crop
uint64_t sensor_timestamp_ns