Illustrated C/C++ source sheets, symbol index, API relationship graph, compiler console and debug trace

LIVE AI / FLEXIBLE SENSING / FLIGHT CONTEXT

Sensoraia API

Live AI and sensing, connected to the flight.
See, listen, inspect and return supported results while airborne.

Loading...
Searching...
No Matches
imx500_camera.cpp
Go to the documentation of this file.
1/*
2 * Copyright 2026 OpenUAS
3 * SPDX-License-Identifier: Apache-2.0
4 *
5 * Licensed under the Apache License, Version 2.0 (the "License");
6 * you may not use this file except in compliance with the License.
7 * You may obtain a copy of the License at
8 *
9 * http://www.apache.org/licenses/LICENSE-2.0
10 *
11 * Unless required by applicable law or agreed to in writing, software
12 * distributed under the License is distributed on an "AS IS" BASIS,
13 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14 * See the License for the specific language governing permissions and
15 * limitations under the License.
16 */
17
18/** @file imx500_camera.cpp
19 * @brief Shared IMX500 capture, frame ownership and JPEG publication mechanics.
20 * @details Camera ownership and model interpretation deliberately have separate
21 * lifetimes. Retaining one request keeps its pixels, crop, tensors and timestamps
22 * together without another image queue. Application callbacks decide what those
23 * tensors mean; this implementation only captures and publishes their evidence.
24 * The owner serializes camera operations and lease destruction; libcamera's
25 * completion callback uses a mutex only for the handoff, not for image processing. */
26#include "imx500_camera.hpp"
27#include "usher_frame_timing.h"
28
29#include <algorithm>
30#include <cerrno>
31#include <chrono>
32#include <condition_variable>
33#include <cstdio>
34#include <cstdlib>
35#include <cstring>
36#include <filesystem>
37#include <limits.h>
38#include <mutex>
39#include <setjmp.h>
40#include <stdexcept>
41#include <sys/ioctl.h>
42#include <sys/mman.h>
43#include <time.h>
44#include <unistd.h>
45#include <vector>
46#include <fcntl.h>
47#include <linux/dma-buf.h>
48#include <linux/videodev2.h>
49#include <jpeglib.h>
50#include <libcamera/control_ids.h>
51#include <libcamera/libcamera.h>
52#include <libcamera/property_ids.h>
53
54namespace imx500 {
55namespace {
56
57constexpr unsigned camera_buffer_count = 3; /**< Requested reusable frame count; avoids an extra image queue. */
58constexpr unsigned maximum_camera_buffer_count = 4; /**< Reject driver adjustments that exceed the buffer budget. */
59constexpr unsigned jpeg_luma_lines_per_mcu = 16; /**< YUV420 JPEG consumes one sixteen-line luma group. */
60constexpr unsigned jpeg_chroma_lines_per_mcu = 8; /**< Half-resolution chroma accompanies that luma group. */
61constexpr size_t maximum_mapped_plane_bytes = 32 * 1024 * 1024; /**< Bound unexpectedly large DMA plane mappings. */
62constexpr int64_t camera_frame_duration_us = 125000; /**< Existing 8 FPS cadence, not an adaptive model policy. */
63constexpr int camera_request_timeout_seconds = 5; /**< Bound a missing fresh-frame callback. */
64constexpr int tensor_startup_timeout_seconds = 70; /**< Allow first-use firmware upload and inference warm-up. */
65constexpr unsigned capture_width_alignment = 16; /**< Luma stride/capture alignment expected by the worker. */
66constexpr unsigned capture_height_alignment = 2; /**< YUV420 needs paired luma rows for chroma sampling. */
67constexpr uint32_t imx500_network_firmware_control = 0x00982901; /**< Driver control accepting an open RPK descriptor. */
68constexpr uint32_t imx500_inference_roi_control = 0x00982900; /**< Driver control selecting sensor-space inference ROI. */
69
70/** @brief Local C encoder recovery state; no camera lease is unwound by longjmp. */
71struct JpegError {
72 jpeg_error_mgr manager; /**< libjpeg callbacks initialized before compressor creation. */
73 jmp_buf jump; /**< Recovery stays inside encode_frame_jpeg(), where partial output is removed. */
74};
75
76/** @brief Redirect libjpeg's fatal callback to the encoder's local cleanup path.
77 * @param codec Active codec carrying our error manager.
78 * @details libjpeg uses a C callback, not C++ exceptions. Keep the jump target
79 * inside the encoder so it never skips a model processor or frame-lease destructor. */
80void jpeg_failure(j_common_ptr codec)
81{
82 longjmp(reinterpret_cast<JpegError *>(codec->err)->jump, 1);
83}
84
85/** @brief Find a correctly typed array without assuming a vendor control ID number.
86 * @param metadata Metadata belonging to the retained camera request.
87 * @param name Driver-advertised control name.
88 * @param type Required element type before interpreting its bytes.
89 * @return Borrowed matching value, or NULL for absent/mismatched metadata.
90 * @details Rejecting scalar and differently typed controls prevents a superficially
91 * present field from being reinterpreted as a neural-output array. */
92const libcamera::ControlValue *named_metadata(const libcamera::ControlList &metadata,
93 const char *name, libcamera::ControlType type)
94{
95 if (!metadata.idMap()) {
96 return nullptr;
97 }
98 for (const auto &entry : metadata) {
99 const auto identifier = metadata.idMap()->find(entry.first);
100 if (identifier != metadata.idMap()->end() && identifier->second->name() == name
101 && entry.second.type() == type && entry.second.isArray()) {
102 return &entry.second;
103 }
104 }
105 return nullptr;
106}
107
108/** @brief Balance CPU/DMA ownership for the completed frame's borrowed planes.
109 * @details Several planes may share one DMA-BUF descriptor. Map plane ranges
110 * separately, but start/end CPU synchronization only once per shared descriptor. */
111class MappedFrame {
112public:
113 /** @brief Map a successful completed buffer and validate its accessible plane ranges.
114 * @param buffer Borrowed frame buffer retained by the surrounding request lease.
115 * @throws std::runtime_error for invalid layout, page size, synchronization or mapping.
116 * @details Constructor failure releases only mappings/syncs already acquired;
117 * the outer lease remains responsible for returning the camera request. */
118 explicit MappedFrame(libcamera::FrameBuffer *buffer)
119 {
120 if (buffer->planes().empty() || buffer->planes().size() > mappings_.size()
121 || buffer->metadata().planes().size() != buffer->planes().size()
122 || buffer->metadata().status != libcamera::FrameMetadata::FrameSuccess) {
123 std::fprintf(stderr, "invalid camera buffer: planes=%zu metadata_planes=%zu status=%d\n",
124 buffer->planes().size(), buffer->metadata().planes().size(),
125 static_cast<int>(buffer->metadata().status));
126 throw std::runtime_error("invalid camera buffer");
127 }
128 try {
129 const long page = sysconf(_SC_PAGESIZE);
130 if (page <= 0) {
131 throw std::runtime_error("invalid page size");
132 }
133 for (size_t index = 0; index < buffer->planes().size(); ++index) {
134 const auto &plane = buffer->planes()[index];
135 auto &mapping = mappings_[index];
136 mapping.fd = plane.fd.get();
137 bool duplicate = false;
138 for (size_t previous = 0; previous < index; ++previous) {
139 duplicate = duplicate || mappings_[previous].fd == mapping.fd;
140 }
141 /* Plane count is not descriptor count: synchronizing a shared DMA-BUF
142 * once keeps START/END balanced even when Y, Cb and Cr share storage. */
143 if (!duplicate) {
144 dma_buf_sync sync{DMA_BUF_SYNC_START | DMA_BUF_SYNC_READ};
145 if (ioctl(mapping.fd, DMA_BUF_IOCTL_SYNC, &sync) != 0) {
146 throw std::runtime_error("DMA read sync failed");
147 }
148 mapping.synced = true;
149 }
150 const size_t offset = plane.offset % size_t(page);
151 /* mmap starts at a page boundary; expose only the plane's adjusted start.
152 * Allocation capacity is not evidence that all those bytes were produced. */
153 mapping.length = size_t(plane.length) + offset;
154 if (!plane.length || mapping.length > maximum_mapped_plane_bytes) {
155 throw std::runtime_error("camera plane too large");
156 }
157 mapping.base = mmap(nullptr, mapping.length, PROT_READ, MAP_SHARED,
158 mapping.fd, plane.offset - offset);
159 if (mapping.base == MAP_FAILED) {
160 throw std::runtime_error("camera buffer mmap failed");
161 }
162 data[index] = static_cast<const uint8_t *>(mapping.base) + offset;
163 lengths[index] = std::min(plane.length, buffer->metadata().planes()[index].bytesused);
164 }
165 } catch (...) {
166 release();
167 throw;
168 }
169 }
170
171 /** @brief End scoped CPU access before the camera request can be reused. */
172 ~MappedFrame() { release(); }
173 /** @brief Copies cannot share responsibility for DMA synchronization. */
174 MappedFrame(const MappedFrame &) = delete;
175 /** @brief Mapping ownership is never reassigned implicitly. */
176 MappedFrame &operator=(const MappedFrame &) = delete;
177
178 std::array<const uint8_t *, 3> data{}; /**< Borrowed Y/Cb/Cr addresses, valid only while mapped. */
179 std::array<size_t, 3> lengths{}; /**< Produced readable bytes, capped by each plane allocation. */
180
181private:
182 /** @brief Release mapped ranges and balance each successful CPU-access START.
183 * @details This also serves the constructor's failure path. Cleanup cannot
184 * throw; a failed end-sync is reported rather than masking an earlier exception. */
185 void release() noexcept
186 {
187 for (auto &mapping : mappings_) {
188 if (mapping.base != MAP_FAILED) {
189 munmap(mapping.base, mapping.length);
190 }
191 if (mapping.synced) {
192 dma_buf_sync sync{DMA_BUF_SYNC_END | DMA_BUF_SYNC_READ};
193 if (ioctl(mapping.fd, DMA_BUF_IOCTL_SYNC, &sync) != 0) {
194 std::perror("DMA read sync end");
195 }
196 }
197 }
198 }
199
200 /** @brief Ownership facts needed to unwind a partially mapped frame. */
201 struct Mapping {
202 void *base = MAP_FAILED; /**< mmap address before the plane's page-offset adjustment. */
203 size_t length = 0; /**< Mapped bytes, including the page-prefix adjustment. */
204 int fd = -1; /**< Borrowed DMA-BUF descriptor; this class never closes it. */
205 bool synced = false; /**< This record owns the matching END for a successful START. */
206 };
207 std::array<Mapping, 3> mappings_{}; /**< Fixed maximum plane count, independent of image size. */
208};
209
210/** @brief Reject output names that cannot fit bounded sidecars or the line protocol.
211 * @param filename Accessible NUL-terminated JPEG destination, or NULL.
212 * @return True for a bounded absolute .jpg path without CR/LF characters.
213 * @details Leave room for temporary/sidecar suffixes before allocating any
214 * path strings. Absolute names keep publication independent of child working directories. */
215bool valid_path(const char *filename)
216{
217 if (!filename) {
218 return false;
219 }
220 const size_t length = strnlen(filename, PATH_MAX);
221 return length >= 5 && length <= PATH_MAX - 32 && filename[0] == '/'
222 && std::strcmp(filename + length - 4, ".jpg") == 0
223 && std::strpbrk(filename, "\r\n") == nullptr;
224}
225
226bool valid_yuv420_layout(unsigned width, unsigned height, unsigned stride)
227{
228 return width && width <= sensor_width && height && height <= sensor_height
229 && width % 2 == 0 && height % 2 == 0 && stride % 2 == 0
230 && stride >= ((width + capture_width_alignment - 1) / capture_width_alignment)
231 * capture_width_alignment;
232}
233
234} // namespace
235
236/** @details The lease owns the right to recycle the request, not the DMA memory.
237 * Keeping that distinction avoids both full-frame copies and premature reuse. */
238FrameLease::FrameLease(FrameView view, void *owner, Release release) noexcept
239 : view_(view), owner_(owner), release_(release)
240{
241}
242
243/** @brief Return a held request exactly once, including during exception unwinding. */
245{
246 /* All processing exits use the same release operation, including exceptions.
247 * A moved-from lease has no callback left and therefore cannot recycle twice. */
248 if (release_) {
250 }
251}
252
253/** @details Moving transfers the recycling obligation; it does not extend the
254 * lifetime of the camera state or make an owned copy of the borrowed pixels. */
256 : view_(other.view_), owner_(other.owner_), release_(other.release_)
257{
258 /* Transfer the resource, not just its metadata; empty the old view as a useful
259 * guard against accidentally reading bytes through a consumed lease. */
260 other.owner_ = nullptr;
261 other.release_ = nullptr;
262 other.view_ = FrameView{};
263}
264
265/** @brief Encode one YUV420 frame with optional same-exposure EXIF metadata.
266 * @param path New output path; exclusive creation never replaces an existing file.
267 * @param planes Borrowed Y, Cb and Cr starts, readable throughout encoding.
268 * @param width Positive even image width in pixels.
269 * @param height Positive even image height in pixels.
270 * @param stride Even luma byte stride; chroma uses half this stride.
271 * @param view Capture evidence to embed, or NULL for a pixels-only JPEG.
272 * @param processor Optional application callback for extending the EXIF document.
273 * @return True after encoding, flushing, file sync and close; false on metadata
274 * rejection or I/O/codec failure. Partial files created here are removed.
275 * @pre The caller has validated dimensions, strides and accessible plane lengths.
276 * @details Metadata is prepared before opening the output or entering libjpeg's
277 * C error boundary, so a throwing embed callback leaves no partial JPEG behind. */
278static bool encode_frame_jpeg(const char *path, const std::array<const uint8_t *, 3> &planes,
279 unsigned width, unsigned height, unsigned stride,
280 const FrameView *view, const FrameProcessor *processor)
281{
282 unsigned char capture_exif[EXIF_MAX_PAYLOAD];
283 size_t capture_size = 0;
284 if (view) {
285 const struct usher_frame_timing timing = {
286 view->sequence, view->sensor_timestamp_ns, view->monotonic_us, view->time_error_us,
287 view->exposure_us, view->frame_duration_us
288 };
289 exif_document document;
290 if (!usher_frame_document(&timing, width, height, &document)
291 || (processor && processor->embed && !processor->embed(*view, &document))
292 || exif_serialize(&document, capture_exif, sizeof(capture_exif), &capture_size)) return false;
293 }
294 FILE *file = std::fopen(path, "wbx");
295 if (!file) {
296 return false;
297 }
298 jpeg_compress_struct codec{};
299 JpegError error{};
300 codec.err = jpeg_std_error(&error.manager);
301 error.manager.error_exit = jpeg_failure;
302 /* Recover inside the encoder: libjpeg's longjmp must not bypass the outer
303 * frame lease or DMA mapping destructors that keep the camera queue usable. */
304 if (setjmp(error.jump)) {
305 jpeg_destroy_compress(&codec);
306 std::fclose(file);
307 unlink(path);
308 return false;
309 }
310 jpeg_create_compress(&codec);
311 jpeg_stdio_dest(&codec, file);
312 codec.image_width = width;
313 codec.image_height = height;
314 codec.input_components = 3;
315 codec.in_color_space = JCS_YCbCr;
316 jpeg_set_defaults(&codec);
317 /* Keep YUV420 in its native planar representation. An RGB staging image would
318 * add full-frame memory and conversion work without improving capture identity. */
319 codec.raw_data_in = TRUE;
320 jpeg_set_quality(&codec, 90, TRUE);
321 if (processor && !processor->sidecar_suffix && !processor->write && !processor->embed)
322 codec.dct_method = JDCT_FLOAT;
323 jpeg_start_compress(&codec, TRUE);
324 if (capture_size) jpeg_write_marker(&codec, JPEG_APP0 + 1, capture_exif, static_cast<unsigned>(capture_size));
325 while (codec.next_scanline < height) {
326 JSAMPROW rows[3][jpeg_luma_lines_per_mcu]{};
327 JSAMPARRAY components[3]{rows[0], rows[1], rows[2]};
328 for (unsigned row = 0; row < jpeg_luma_lines_per_mcu; ++row) {
329 /* JPEG consumes whole MCU groups; repeat the final valid row at the edge
330 * rather than reading padding outside the caller-validated plane span. */
331 rows[0][row] = const_cast<JSAMPROW>(
332 planes[0] + std::min(codec.next_scanline + row, height - 1) * stride);
333 if (row < jpeg_chroma_lines_per_mcu) {
334 for (size_t component = 1; component < 3; ++component) {
335 rows[component][row] = const_cast<JSAMPROW>(
336 planes[component]
337 + std::min(codec.next_scanline / 2 + row, height / 2 - 1) * (stride / 2));
338 }
339 }
340 }
341 jpeg_write_raw_data(&codec, components, jpeg_luma_lines_per_mcu);
342 }
343 jpeg_finish_compress(&codec);
344 jpeg_destroy_compress(&codec);
345 /* Only a fully flushed and synced file may be published by the caller.
346 * This sync covers file contents, not the parent directory's durability. */
347 bool written = std::fflush(file) == 0 && fsync(fileno(file)) == 0;
348 if (std::fclose(file) != 0) {
349 written = false;
350 }
351 if (!written) {
352 unlink(path);
353 }
354 return written;
355}
356
357/** @details Use the same encoder without capture/application metadata so
358 * pixels-only callers retain identical YUV layout and output-cleanup behavior. */
359bool encode_jpeg(const char *path, const std::array<const uint8_t *, 3> &planes,
360 unsigned width, unsigned height, unsigned stride)
361{
362 if (!path || !path[0] || !planes[0] || !planes[1] || !planes[2]
363 || !valid_yuv420_layout(width, height, stride)) return false;
364 return encode_frame_jpeg(path, planes, width, height, stride, nullptr, nullptr);
365}
366
367/** @brief Capture resources and the callback-to-owner handoff, not model results.
368 * @details The mutex protects callback-visible readiness state. Control calls and
369 * lease release remain serialized by the owner, as required by the public API. */
371 libcamera::CameraManager manager; /**< Owns camera discovery/event processing. */
372 std::shared_ptr<libcamera::Camera> camera; /**< The sole selected sensor, acquired exclusively. */
373 std::unique_ptr<libcamera::CameraConfiguration> configuration; /**< Validated stream geometry/layout. */
374 std::unique_ptr<libcamera::FrameBufferAllocator> allocator; /**< Owns CMA-backed buffers, not CPU copies. */
375 std::vector<std::unique_ptr<libcamera::Request>> requests; /**< One reusable request per allocated buffer. */
376 std::array<int64_t, 2> frame_duration{camera_frame_duration_us, camera_frame_duration_us}; /**< Fixed cadence range. */
377 std::mutex mutex; /**< Protects callback readiness, failure and waiting-request state. */
378 std::condition_variable ready; /**< Wakes a waiter for a fresh frame or queue failure. */
379 libcamera::Request *completed = nullptr; /**< Callback-held request awaiting owner handoff. */
380 libcamera::Request *leased = nullptr; /**< Request held until its sole lease releases it. */
381 int64_t minimum_timestamp = 0; /**< Earliest eligible sensor timestamp for the current request. */
382 bool wanted = false; /**< A capture call is awaiting one post-request exposure. */
383 bool failed = false; /**< Queue failure prevents treating capture as healthy. */
384 bool acquired = false; /**< Cleanup owes camera release only after acquisition succeeds. */
385 bool started = false; /**< Cleanup owes stream stop only after startup succeeds. */
386
387 /** @brief Unwind partial/full startup in libcamera's resource-lifetime order. */
389 {
390 /* Stop callbacks before destroying their requests and DMA buffers. The
391 * success flags also make failed-open cleanup safe at each setup stage. */
392 if (started) {
393 camera->stop();
394 }
395 requests.clear();
396 allocator.reset();
397 configuration.reset();
398 if (acquired) {
399 camera->release();
400 }
401 camera.reset();
402 manager.stop();
403 }
404
405 /** @brief Return an unneeded or fully consumed request to the bounded camera queue.
406 * @param request Completed request with no remaining borrowed consumers.
407 * @details A requeue failure wakes the owner as a failure; it must not wait for
408 * a callback that can no longer arrive or silently reuse stale evidence. */
409 void recycle(libcamera::Request *request)
410 {
411 request->reuse(libcamera::Request::ReuseBuffers);
412 if (camera->queueRequest(request) != 0) {
413 std::lock_guard<std::mutex> lock(mutex);
414 failed = true;
415 ready.notify_one();
416 }
417 }
418
419 /** @brief Recycle the sole leased request without throwing through its destructor.
420 * @param owner Capture implementation that outlives the lease.
421 * @details Clear ownership before requeueing. Unexpected exceptions cannot be
422 * hidden as a successful release, so this nonthrowing boundary terminates. */
423 static void release_lease(void *owner) noexcept
424 {
425 auto &state = *static_cast<Impl *>(owner);
426 libcamera::Request *request = state.leased;
427 state.leased = nullptr;
428 try {
429 if (request) {
430 state.recycle(request);
431 }
432 } catch (...) {
433 std::terminate();
434 }
435 }
436
437 /** @brief Hold one fresh requested exposure; immediately recycle other callbacks.
438 * @param request Completed or cancelled libcamera request.
439 * @details The timestamp gate is set when capture() starts. A previously queued
440 * frame is not automatically the photograph requested by the current SHOT. */
441 void complete(libcamera::Request *request)
442 {
443 /* Cancellation is part of shutdown, not a fresh frame to hand off or requeue. */
444 if (request->status() == libcamera::Request::RequestCancelled) {
445 return;
446 }
447 const auto &buffers = request->buffers();
448 const auto buffer = buffers.find(configuration->at(0).stream());
449 const auto status = buffer != buffers.end() && buffer->second
450 ? detail::frame_status(buffer->second->metadata().status)
452 if (status == detail::FrameStatus::startup) {
454 return;
455 }
456 if (status == detail::FrameStatus::failed) {
457 std::lock_guard<std::mutex> lock(mutex);
458 failed = true;
459 wanted = false;
460 ready.notify_one();
461 return;
462 }
463 {
464 std::lock_guard<std::mutex> lock(mutex);
465 const auto timestamp = request->metadata().get(libcamera::controls::SensorTimestamp);
466 if (wanted && timestamp && *timestamp >= minimum_timestamp) {
468 wanted = false;
469 ready.notify_one();
470 return;
471 }
472 }
474 }
475
476 /** @brief Load an RPK or clear a retained network through the driver control.
477 * @param model Accessible readable model path, or NULL to select image-only capture.
478 * @throws std::runtime_error for ambiguous/missing devices or failed controls.
479 * @details The driver's INT32_MAX control value clears a retained volatile network.
480 * Full-array inference ROI is set only when loading a model. */
481 static void configure_model(const char *model)
482 {
483 std::string device;
484 for (const auto &entry : std::filesystem::directory_iterator("/sys/class/video4linux")) {
485 if (entry.path().filename().string().find("v4l-subdev") != 0) {
486 continue;
487 }
488 std::error_code error;
489 const auto driver = std::filesystem::canonical(entry.path() / "device/driver", error);
490 if (!error && driver.filename() == "imx500") {
491 if (!device.empty()) {
492 throw std::runtime_error("ambiguous IMX500 subdevice");
493 }
494 device = "/dev/" + entry.path().filename().string();
495 }
496 }
497 const int sensor = ::open(device.c_str(), O_RDWR | O_CLOEXEC);
498 if (sensor < 0) {
499 throw std::runtime_error("cannot open IMX500 subdevice");
500 }
501 v4l2_control firmware{};
502 firmware.id = imx500_network_firmware_control;
503 if (!model) {
504 firmware.value = INT32_MAX;
505 const int status = ioctl(sensor, VIDIOC_S_CTRL, &firmware);
506 close(sensor);
507 if (status != 0) throw std::runtime_error("IMX500 network unload failed");
508 return;
509 }
510 const int network = ::open(model, O_RDONLY | O_CLOEXEC);
511 if (network < 0) {
512 close(sensor);
513 throw std::runtime_error("cannot open network.rpk");
514 }
515 /* Pass an open descriptor, not model bytes copied into another userspace
516 * buffer; keep it open until the synchronous driver control completes. */
517 firmware.value = network;
518 int status = ioctl(sensor, VIDIOC_S_CTRL, &firmware);
519 std::array<uint32_t, 4> region{0, 0, sensor_width, sensor_height};
520 v4l2_ext_control roi{};
521 roi.id = imx500_inference_roi_control;
522 roi.size = sizeof(region);
523 roi.p_u32 = region.data();
524 v4l2_ext_controls controls{};
525 controls.count = 1;
526 controls.controls = &roi;
527 if (status == 0) {
528 status = ioctl(sensor, VIDIOC_S_EXT_CTRLS, &controls);
529 }
530 close(network);
531 close(sensor);
532 if (status != 0) {
533 throw std::runtime_error("IMX500 firmware/ROI control failed");
534 }
535 }
536};
537
538/** @brief Allocate unopened camera state; discovery and acquisition wait for open(). */
539Camera::Camera() : impl_(std::make_unique<Impl>()) {}
540/** @brief Stop callbacks, free requests/buffers, then release the camera.
541 * @pre All leases and pixel visitors have finished; they borrow this state. */
542Camera::~Camera() = default;
543
544/** @details Exactly one eligible camera is required so discovery order cannot
545 * silently choose a different sensor. Partial startup remains owned by Impl and
546 * is unwound when the Camera is destroyed. */
547void Camera::open(const char *model, unsigned width, unsigned height, bool photo_only)
548{
549 auto &state = *impl_;
550 if (state.acquired || state.started) {
551 throw std::runtime_error("camera already open");
552 }
553 if (photo_only && model) throw std::invalid_argument("photo-only capture cannot load a model");
554 if (state.manager.start() != 0) {
555 throw std::runtime_error("libcamera startup failed");
556 }
557 for (const auto &candidate : state.manager.cameras()) {
558 const auto name = candidate->properties().get(libcamera::properties::Model);
559 if ((!model && !photo_only) || (name && name->find("imx500") != std::string::npos)) {
560 if (state.camera) {
561 throw std::runtime_error("capture requires exactly one eligible camera");
562 }
563 state.camera = candidate;
564 }
565 }
566 if (!state.camera || state.camera->acquire() != 0) {
567 throw std::runtime_error("IMX500 unavailable or busy");
568 }
569 state.acquired = true;
570 /* The SDK profile selects this switch from its span API. In the supported
571 * newer SDK, PixelArrayArea also replaces PixelArraySize; compare dimensions
572 * in both cases because the firmware ROI below assumes the full IMX500 array. */
573#ifdef SENSORAIA_LIBCAMERA_STD_SPAN
574 const auto sensor = state.camera->properties().get(libcamera::properties::PixelArrayArea);
575 const bool expected_sensor = sensor && sensor->size() == libcamera::Size(sensor_width, sensor_height);
576#else
577 const auto sensor = state.camera->properties().get(libcamera::properties::PixelArraySize);
578 const bool expected_sensor = sensor && *sensor == libcamera::Size(sensor_width, sensor_height);
579#endif
580 if ((model || photo_only) && !expected_sensor) {
581 throw std::runtime_error("unexpected IMX500 geometry");
582 }
583 if (model || photo_only) Impl::configure_model(model);
584 state.configuration = state.camera->generateConfiguration({libcamera::StreamRole::StillCapture});
585 if (!state.configuration || state.configuration->empty()) {
586 throw std::runtime_error("no still stream");
587 }
588 auto &stream = state.configuration->at(0);
589 stream.size = {width, height};
590 stream.pixelFormat = libcamera::formats::YUV420;
591 stream.colorSpace = libcamera::ColorSpace::Sycc;
592 state.configuration->orientation = libcamera::Orientation::Rotate0;
593 stream.bufferCount = camera_buffer_count;
594 /* libcamera can adjust a configuration. Reject changes that invalidate plane
595 * arithmetic, downstream geometry or our intended CMA-buffer budget. */
596 if (state.configuration->validate() == libcamera::CameraConfiguration::Invalid
597 || state.configuration->orientation != libcamera::Orientation::Rotate0
598 || stream.size != libcamera::Size(width, height)
599 || stream.pixelFormat != libcamera::formats::YUV420
600 || stream.colorSpace != libcamera::ColorSpace::Sycc || stream.stride % 2
601 || stream.stride < ((width + capture_width_alignment - 1) / capture_width_alignment)
602 * capture_width_alignment
603 || stream.bufferCount == 0
604 || stream.bufferCount > maximum_camera_buffer_count
605 || state.camera->configure(state.configuration.get()) != 0) {
606 throw std::runtime_error("unsupported still configuration");
607 }
608 state.allocator = std::make_unique<libcamera::FrameBufferAllocator>(state.camera);
609 if (state.allocator->allocate(stream.stream()) < 0) {
610 throw std::runtime_error("camera buffer allocation failed");
611 }
612 for (const auto &buffer : state.allocator->buffers(stream.stream())) {
613 /* Reuse requests/buffers rather than allocating another full-resolution
614 * frame per trigger. The lease, not a copied image, preserves identity. */
615 auto request = state.camera->createRequest();
616 if (!request || request->addBuffer(stream.stream(), buffer.get()) != 0) {
617 throw std::runtime_error("camera request allocation failed");
618 }
619 state.requests.push_back(std::move(request));
620 }
621 if (state.requests.empty()) throw std::runtime_error("camera allocated no capture buffers");
622 state.camera->requestCompleted.connect(&state, &Impl::complete);
623 libcamera::ControlList controls(state.camera->controls());
624 if (!photo_only) {
625 controls.set(libcamera::controls::FrameDurationLimits,
626 Span<const int64_t, 2>(state.frame_duration));
627 }
628 /* Install the callback before starting, then queue every allocated request.
629 * Requests that are not wanted by capture() keep circulating without copies. */
630 if (state.camera->start(&controls) != 0) {
631 throw std::runtime_error("camera start failed");
632 }
633 state.started = true;
634 for (auto &request : state.requests) {
635 state.recycle(request.get());
636 }
637}
638
639/** @details A queued frame can predate a SHOT command. Compare sensor timestamps
640 * in the BOOTTIME domain instead of treating callback arrival as exposure time.
641 * Tensor validity and interpretation remain the application's responsibility. */
643{
644 auto &state = *impl_;
645 std::unique_lock<std::mutex> lock(state.mutex);
646 if (!state.started || state.leased) {
647 throw std::runtime_error("camera not started or previous frame still leased");
648 }
649 timespec now{};
650 if (clock_gettime(CLOCK_BOOTTIME, &now) != 0) {
651 throw std::runtime_error("capture clock failed");
652 }
653 state.minimum_timestamp = int64_t(now.tv_sec) * 1000000000 + now.tv_nsec;
654 /* Only the callback can satisfy this wait. It may retain a fresh request,
655 * while older completed frames return immediately to the capture queue. */
656 state.wanted = true;
657 if (!state.ready.wait_for(lock, std::chrono::seconds(camera_request_timeout_seconds),
658 [&state] { return state.completed || state.failed; })) {
659 state.wanted = false;
660 throw std::runtime_error("camera frame timeout");
661 }
662 if (state.failed) {
663 throw std::runtime_error("camera request or frame failed");
664 }
665 /* Move the request from callback handoff to lease ownership under the lock.
666 * Release the lock before metadata work so other completions can circulate. */
667 state.leased = state.completed;
668 state.completed = nullptr;
669 lock.unlock();
670 /* Establish cleanup before inspecting optional metadata: an exception during
671 * extraction must still return the request exactly once. */
673 const auto *request = state.leased;
674 FrameView &view = lease.view_;
675 /* Every borrowed tensor/crop/time comes from this same retained request.
676 * Missing controls remain explicit sentinels, not invented capture evidence. */
677 const auto *tensor = named_metadata(request->metadata(), "CnnOutputTensor", libcamera::ControlTypeFloat);
678 const auto *info = named_metadata(request->metadata(), "CnnOutputTensorInfo", libcamera::ControlTypeByte);
679 if (tensor) {
680 view.tensor = tensor->get<Span<const float>>();
681 }
682 if (info) {
683 view.tensor_info = info->get<Span<const uint8_t>>();
684 }
685 const auto crop = request->metadata().get(libcamera::controls::ScalerCrop);
686 if (crop) {
687 view.crop = *crop;
688 view.crop_valid = true;
689 }
690 view.image = state.configuration->at(0).size;
691 view.sequence = request->sequence();
692 const auto sensor = request->metadata().get(libcamera::controls::SensorTimestamp);
693 view.sensor_timestamp_ns = sensor && *sensor > 0 ? static_cast<uint64_t>(*sensor) : 0;
694 const auto exposure = request->metadata().get(libcamera::controls::ExposureTime);
695 const auto duration = request->metadata().get(libcamera::controls::FrameDuration);
696 view.exposure_us = exposure ? *exposure : -1;
697 view.frame_duration_us = duration ? *duration : -1;
698 /* Estimate observation timing once while this exposure is retained. JPEG
699 * encoding/publication can happen later and must not become the capture time. */
702 view.monotonic_us = timing.monotonic_us;
703 view.time_error_us = timing.time_error_us;
704 view.timing_sampled = true;
705 return lease;
706}
707
708/** @details Encode first, then attempt sidecar output, then link the complete JPEG
709 * into place. The JPEG is the final publication step, but JPEG and sidecar are
710 * not one atomic transaction; callbacks must remove their own partial outputs.
711 * @throws std::exception Filesystem or processor exceptions propagate after cleanup. */
712bool publish_frame(const FrameView &view, const std::array<const uint8_t *, 3> &planes,
713 unsigned stride, const char *filename, const FrameProcessor &processor)
714{
715 if (!valid_path(filename) || bool(processor.sidecar_suffix) != bool(processor.write)
716 || !planes[0] || !planes[1] || !planes[2]
717 || !valid_yuv420_layout(view.image.width, view.image.height, stride)) {
718 return false;
719 }
720 const std::string path(filename);
721 const std::string sidecar = processor.sidecar_suffix ? path.substr(0, path.size() - 4) + processor.sidecar_suffix : "";
722 const std::string temporary = path + ".capture.tmp";
723 /* Existence checks avoid ordinary collisions; exclusive creation/link are
724 * still the authority if another writer races after these checks. */
725 if (std::filesystem::exists(path) || (!sidecar.empty() && std::filesystem::exists(sidecar))) {
726 return false;
727 }
728 FrameView evidence = view;
729 /* Live capture already sampled timing. Other callers may supply only raw
730 * sensor metadata; fill the EXIF estimate without mutating their shared view. */
731 if (!evidence.timing_sampled) {
734 evidence.monotonic_us = timing.monotonic_us;
735 evidence.time_error_us = timing.time_error_us;
736 }
737 if (!encode_frame_jpeg(temporary.c_str(), planes, view.image.width, view.image.height, stride, &evidence, &processor)) {
738 return false;
739 }
740 bool analysis_written = false;
741 try {
742 /* A false analysis result means no usable sidecar, not unusable pixels.
743 * Timing is tied to the temporary JPEG inode retained by the final hard link. */
744 if (processor.write && !sidecar.empty()) analysis_written = processor.write(view, sidecar.c_str());
745 /* A hard link exposes only the completed inode and refuses an existing
746 * destination, unlike a replacing rename. Both names stay in one directory. */
747 const bool published = link(temporary.c_str(), path.c_str()) == 0;
748 unlink(temporary.c_str());
749 if (!published) {
750 /* Remove only successful artifacts created here. A failed callback owes
751 * its own cleanup and must not give us ownership of somebody else's file. */
752 if (analysis_written) {
753 unlink(sidecar.c_str());
754 }
755 }
756 return published;
757 } catch (...) {
758 /* The callback contract covers partial analysis output; our exception path
759 * owns only the JPEG temporary and sidecars whose creation was reported. */
760 unlink(temporary.c_str());
761 if (analysis_written) {
762 unlink(sidecar.c_str());
763 }
764 throw;
765 }
766}
767
768/** @details with_pixels() scopes CPU/DMA access around publication; the caller
769 * still owns the frame lease and decides when the request returns to the queue.
770 * @throws std::exception Mapping, filesystem or processor failures may propagate. */
771bool Camera::save(const FrameLease &frame, const char *filename, const FrameProcessor &processor)
772{
773 struct SavePolicy { const char *filename; const FrameProcessor *processor; } policy{filename, &processor};
774 return with_pixels(frame, [](const FrameView &view, const std::array<const uint8_t *, 3> &planes,
775 unsigned stride, void *context) {
776 const auto &save = *static_cast<SavePolicy *>(context);
777 return publish_frame(view, planes, stride, save.filename, *save.processor);
778 }, &policy);
779}
780
781/** @details Plane allocation sizes alone do not prove that a frame produced enough
782 * bytes. Check bytesused before exposing chroma, and unmap/end CPU access before
783 * returning or unwinding to the code that can recycle the request.
784 * @throws std::exception DMA/mapping failures and visitor exceptions may propagate. */
786 bool (*visitor)(const FrameView &, const std::array<const uint8_t *, 3> &,
787 unsigned, void *), void *context)
788{
789 auto &state = *impl_;
790 if (!visitor || frame.owner_ != impl_.get() || frame.release_ == nullptr || !state.leased) {
791 return false;
792 }
793 auto *request = state.leased;
794 const auto &stream = state.configuration->at(0);
795 auto *buffer = request->buffers().at(stream.stream());
796 MappedFrame mapped(buffer);
797 /* Some drivers use one contiguous YUV allocation, others three planes.
798 * Validate produced byte counts before deriving chroma starts or encoding. */
799 auto planes = mapped.data;
800 const size_t luma = size_t(stream.stride) * stream.size.height;
801 const size_t chroma = luma / 4;
802 if (buffer->planes().size() == 1 && mapped.lengths[0] >= luma + 2 * chroma) {
803 planes[1] = planes[0] + luma;
804 planes[2] = planes[1] + chroma;
805 } else if (buffer->planes().size() != 3 || mapped.lengths[0] < luma
806 || mapped.lengths[1] < chroma || mapped.lengths[2] < chroma) {
807 return false;
808 }
809 return visitor(frame.view(), planes, stream.stride, context);
810}
811
812/** @retval 0 Probe/capture succeeded, or the serving input ended cleanly.
813 * @retval 1 Runtime, output or protocol-stream I/O failure.
814 * @retval 2 Invalid arguments, dimensions, processor contract or command syntax.
815 * @details A per-SHOT save failure emits ERROR without retrying the exposure.
816 * An exception terminates the worker after reporting the reason on stderr. */
817int run_worker(int argc, char **argv, const FrameProcessor &processor, const char *application)
818{
819 if (argc < 3) {
820 return 2;
821 }
822 const bool photo_only = std::strcmp(argv[2], "--photo-only") == 0;
823 if (!photo_only && (!processor.validate || ((!processor.write || !processor.sidecar_suffix) && !processor.embed))) {
824 return 2;
825 }
826 const FrameProcessor photo_processor{nullptr, nullptr, nullptr};
827 const FrameProcessor &selected_processor = photo_only ? photo_processor : processor;
828 const char *timing_setting = std::getenv("USHER_AI_CAM_TIMING");
829 const bool report_timing = timing_setting && std::strcmp(timing_setting, "1") == 0;
830 const bool probe = std::strcmp(argv[1], "--probe") == 0;
831 const bool serve = std::strcmp(argv[1], "--serve") == 0;
832 const bool capture = std::strcmp(argv[1], "--capture") == 0;
833 if ((!probe && !serve && !capture) || (probe && argc != 3)
834 || (serve && argc != 5) || (capture && argc != 4)) {
835 return 2;
836 }
837 try {
838 unsigned width = 2028;
839 unsigned height = 1520;
840 if (serve) {
841 char *end_width = nullptr;
842 char *end_height = nullptr;
843 errno = 0;
844 const unsigned long parsed_width = std::strtoul(argv[3], &end_width, 10);
845 const unsigned long parsed_height = std::strtoul(argv[4], &end_height, 10);
846 if (errno || end_width == argv[3] || end_height == argv[4] || *end_width || *end_height
847 || parsed_width > sensor_width || parsed_height > sensor_height
848 || !valid_capture_dimensions(static_cast<unsigned>(parsed_width), static_cast<unsigned>(parsed_height))) {
849 return 2;
850 }
851 width = static_cast<unsigned>(parsed_width);
852 height = static_cast<unsigned>(parsed_height);
853 }
855 camera.open(photo_only ? nullptr : argv[2], width, height, photo_only);
856 /* RPK upload and neural warm-up can outlast normal frame acquisition.
857 * Discard startup leases until this application's tensor contract passes;
858 * an available camera frame alone is not enough to announce readiness. */
859 const auto deadline = std::chrono::steady_clock::now()
860 + std::chrono::seconds(tensor_startup_timeout_seconds);
861 bool ready = false;
862 while (std::chrono::steady_clock::now() < deadline) {
863 auto frame = camera.capture();
864 if (photo_only && !detail::photo_only_frame(frame.view()))
865 throw std::runtime_error("unexpected IMX500 inference metadata in photo-only mode");
866 ready = photo_only || selected_processor.validate(frame.view());
867 if (ready) {
868 break;
869 }
870 }
871 if (!ready) {
872 throw std::runtime_error("no fresh valid IMX500 tensor before startup deadline");
873 }
874 if (probe) {
875 std::puts(photo_only ? "camera frame received" : "IMX500 frame and output tensor received");
876 return 0;
877 }
878 if (capture) {
879 /* The readiness frame is not the requested photograph: take another
880 * post-call exposure instead of publishing a frame consumed by warm-up. */
881 auto frame = camera.capture();
882 if (photo_only && !detail::photo_only_frame(frame.view()))
883 throw std::runtime_error("unexpected IMX500 inference metadata in photo-only mode");
884 return camera.save(frame, argv[3], selected_processor) ? 0 : 1;
885 }
886 std::puts("READY");
887 /* READY means this processor accepted a fresh tensor contract. It never
888 * means that LOOM found a navigable match or that objects were detected. */
889 if (std::fflush(stdout) != 0) {
890 return 1;
891 }
892 char command[PATH_MAX + 16];
893 while (std::fgets(command, sizeof(command), stdin)) {
894 const size_t length = std::strlen(command);
895 /* Missing newline also catches an overlong/truncated command. Reject it
896 * rather than treating the remainder as another independent SHOT. */
897 if (length == 0 || command[length - 1] != '\n' || std::strncmp(command, "SHOT ", 5) != 0) {
898 return 2;
899 }
900 command[length - 1] = '\0';
901 /* Each command takes a new scoped lease. ERROR is reported for that
902 * operation; the protocol never automatically repeats its exposure. */
903 std::chrono::steady_clock::time_point capture_started{}, capture_completed{};
904 if (report_timing) capture_started = std::chrono::steady_clock::now();
905 auto frame = camera.capture();
906 if (photo_only && !detail::photo_only_frame(frame.view()))
907 throw std::runtime_error("unexpected IMX500 inference metadata in photo-only mode");
908 if (report_timing) capture_completed = std::chrono::steady_clock::now();
909 const bool saved = camera.save(frame, command + 5, selected_processor);
910 if (report_timing) {
911 const auto save_completed = std::chrono::steady_clock::now();
912 std::fprintf(stderr, "%s TIMING: capture_ms=%.3f save_ms=%.3f exposure_us=%lld frame_us=%lld\n",
913 application,
914 std::chrono::duration<double, std::milli>(capture_completed - capture_started).count(),
915 std::chrono::duration<double, std::milli>(save_completed - capture_completed).count(),
916 static_cast<long long>(frame.view().exposure_us),
917 static_cast<long long>(frame.view().frame_duration_us));
918 }
919 std::puts(saved ? "OK" : "ERROR");
920 if (std::fflush(stdout) != 0) {
921 return 1;
922 }
923 }
924 return std::ferror(stdin) ? 1 : 0;
925 } catch (const std::exception &error) {
926 std::fprintf(stderr, "%s: %s\n", application, error.what());
927 return 1;
928 }
929}
930
931int run_photo_worker(int argc, char **argv, const char *application)
932{
933 if (argc < 2 || argc > 4) return 2;
934 char photo_only[] = "--photo-only";
935 char *arguments[] = {argv[0], argv[1], photo_only,
936 argc > 2 ? argv[2] : nullptr,
937 argc > 3 ? argv[3] : nullptr, nullptr};
938 const FrameProcessor processor{nullptr, nullptr, nullptr};
939 return run_worker(argc + 1, arguments, processor, application);
940}
941
942} // namespace imx500
size_t size
Definition chdk_pipe.c:143
void(* close)(struct usbfs_handle *)
Definition chdkbridge.c:57
static bool capture(struct transport *transport, FILE *file, const struct capture_settings *settings)
Capture to the camera's SD card and download the resulting JPEG.
int(* open)(const struct usbfs_device *, struct usbfs_handle *)
Definition chdkbridge.c:56
One exclusive IMX500 owner with bounded reusable YUV420 buffers.
bool with_pixels(const FrameLease &frame, bool(*visitor)(const FrameView &, const std::array< const uint8_t *, 3 > &, unsigned, void *), void *context)
Grant scoped CPU pixel access without transferring DMA/frame ownership.
bool save(const FrameLease &frame, const char *filename, const FrameProcessor &processor)
Map a leased frame and publish its JPEG plus available evidence.
~Camera()
Stop streaming before freeing requests/buffers and releasing the camera.
void open(const char *model, unsigned width=2028, unsigned height=1520, bool photo_only=false)
Acquire one eligible camera and configure bounded capture, uploading an RPK only when requested.
std::unique_ptr< Impl > impl_
Camera()
Construct unopened camera state; construction does not acquire the sensor.
FrameLease capture()
Lease a fresh post-request exposure and its own tensor metadata.
Exactly-once release of a borrowed frame, including exception exits.
FrameLease(FrameView view, void *owner, Release release) noexcept
Associate borrowed evidence with exactly one release operation.
void(*)(void *) noexcept Release
Nonthrowing release callback; a destructor must never unwind through it.
~FrameLease()
Release once on every scope exit, including a processing exception.
const FrameView & view() const noexcept
Read evidence only while this unmoved lease remains alive.
union dc_shot_union request
static bool ready
static struct inference_context network
int exif_serialize(const struct exif_document *document, unsigned char *payload, size_t capacity, size_t *size)
Serialize a document into one complete EXIF APP1 payload.
Definition exifite.c:736
#define EXIF_MAX_PAYLOAD
APP1 payload ceiling: the JPEG length includes its own two bytes.
Definition exifite.h:51
Neutral IMX500 frame ownership, JPEG publication and worker protocol.
char evidence[8192]
Definition lwir_image.c:66
uint8_t row[LWIR_MAX_WIDTH *3U]
Definition lwir_image.c:63
static void jpeg_failure(j_common_ptr encoder)
libjpeg error_exit replacement: abandon the photo instead of exiting.
Definition lwir_image.c:86
struct jpeg_error_mgr error
Definition lwir_image.c:60
uint8_t frame[LWIR_MAX_FRAME_BYTES]
Definition lwir_mock.c:58
constexpr FrameStatus frame_status(libcamera::FrameMetadata::Status status)
Map libcamera metadata to this worker's frame disposition.
bool photo_only_frame(const FrameView &frame)
Confirm that a frame carries no neural output.
bool encode_jpeg(const char *path, const std::array< const uint8_t *, 3 > &planes, unsigned width, unsigned height, unsigned stride)
Encode validated YUV420 planes directly, without full-frame RGB staging.
int run_worker(int argc, char **argv, const FrameProcessor &processor, const char *application)
Run the shared probe/capture/READY-SHOT-OK worker with supplied model policy.
constexpr bool valid_capture_dimensions(unsigned width, unsigned height)
Check that capture dimensions meet the worker's raster policy.
static bool encode_frame_jpeg(const char *path, const std::array< const uint8_t *, 3 > &planes, unsigned width, unsigned height, unsigned stride, const FrameView *view, const FrameProcessor *processor)
Encode one YUV420 frame with optional same-exposure EXIF metadata.
bool publish_frame(const FrameView &view, const std::array< const uint8_t *, 3 > &planes, unsigned stride, const char *filename, const FrameProcessor &processor)
Publish caller-validated YUV420 planes and same-exposure evidence.
int run_photo_worker(int argc, char **argv, const char *application)
Run photo-only probe, capture or serve without model loading or inference.
constexpr unsigned sensor_height
Physical IMX500 array height; model input dimensions are separate policy.
constexpr unsigned sensor_width
Physical IMX500 array width, used to reject an unexpected sensor geometry.
#define PATH_MAX
Definition path_utils.h:39
static size_t count
Definition pose_log.c:79
static uint64_t synced
Definition pose_log.c:81
int fd
Definition serial.c:39
Caller-owned bounded writer with no heap allocation.
Definition exifite.h:156
Capture resources and the callback-to-owner handoff, not model results.
libcamera::Request * completed
static void configure_model(const char *model)
Load an RPK or clear a retained network through the driver control.
std::unique_ptr< libcamera::FrameBufferAllocator > allocator
std::array< int64_t, 2 > frame_duration
libcamera::Request * leased
void recycle(libcamera::Request *request)
Return an unneeded or fully consumed request to the bounded camera queue.
std::vector< std::unique_ptr< libcamera::Request > > requests
std::unique_ptr< libcamera::CameraConfiguration > configuration
std::condition_variable ready
libcamera::CameraManager manager
static void release_lease(void *owner) noexcept
Recycle the sole leased request without throwing through its destructor.
~Impl()
Unwind partial/full startup in libcamera's resource-lifetime order.
std::shared_ptr< libcamera::Camera > camera
void complete(libcamera::Request *request)
Hold one fresh requested exposure; immediately recycle other callbacks.
Stack-owned model-specific operations; no dynamic plugin allocation.
bool(* validate)(const FrameView &frame)
bool(* embed)(const FrameView &frame, exif_document *document)
bool(* write)(const FrameView &frame, const char *filename)
Borrowed same-exposure evidence, without object or navigation semantics.
Span< const uint8_t > tensor_info
Span< const float > tensor
libcamera::Size image
libcamera::Rectangle crop
uint64_t sensor_timestamp_ns
Timing evidence for one exposure, with microsecond uncertainty.
uint64_t monotonic_us
Estimated exposure/readout-window midpoint in MONOTONIC microseconds.
uint64_t time_error_us
Conservative alignment allowance, not calibrated accuracy.
static struct camera_backend camera
Definition usher.c:521
bool usher_frame_document(const struct usher_frame_timing *timing, unsigned width, unsigned height, struct exif_document *document)
Initialize capture EXIF for the JPEG encoded from this exposure.
struct usher_frame_timing usher_frame_timing_estimate(uint64_t sequence, uint64_t sensor_ns, int64_t exposure_us, int64_t duration_us)
Translate fresh sensor evidence into a cautious pose-alignment time.
C interface for conservative frame-time estimation and EXIF evidence.