Illustrated C/C++ source sheets, symbol index, API relationship graph, compiler console and debug trace

LIVE AI / FLEXIBLE SENSING / FLIGHT CONTEXT

Sensoraia API

Live AI and sensing, connected to the flight.
See, listen, inspect and return supported results while airborne.

Loading...
Searching...
No Matches
camera_lens.cpp
Go to the documentation of this file.
1/*
2 * Copyright 2026 OpenUAS
3 * SPDX-License-Identifier: Apache-2.0
4 *
5 * Licensed under the Apache License, Version 2.0 (the "License");
6 * you may not use this file except in compliance with the License.
7 * You may obtain a copy of the License at
8 *
9 * http://www.apache.org/licenses/LICENSE-2.0
10 *
11 * Unless required by applicable law or agreed to in writing, software
12 * distributed under the License is distributed on an "AS IS" BASIS,
13 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14 * See the License for the specific language governing permissions and
15 * limitations under the License.
16 */
17
18/**
19 * @file camera_lens.cpp
20 * @brief Bounded native lens and exposure correction for USHER JPEGs.
21 * @details Validates a versioned profile, reads the clean source JPEG and
22 * flight EXIF, renders in memory-bounded strips, and atomically publishes a
23 * derivative with provenance. Default profiles are illustrative, not measured
24 * calibration.
25 */
26
27// clang-format off
28#include <stddef.h>
29#include <stdio.h>
30#include <jpeglib.h>
31#include "jpeg_marker_guard.h"
32// clang-format on
33#include "exifite.h"
34#include "image_exif.h"
35extern "C" {
36#include "sha256fst.h"
37}
38
39#include <opencv2/core/persistence.hpp>
40#include <opencv2/calib3d.hpp>
41#include <opencv2/imgproc.hpp>
42
43#include <fcntl.h>
44#include <sys/file.h>
45#include <sys/stat.h>
46#include <unistd.h>
47
48#include <algorithm>
49#include <array>
50#include <atomic>
51#include <chrono>
52#include <cmath>
53#include <csetjmp>
54#include <cstdio>
55#include <cstdlib>
56#include <fstream>
57#include <iostream>
58#include <list>
59#include <map>
60#include <memory>
61#include <sstream>
62#include <stdexcept>
63#include <string>
64#include <vector>
65
66#include "version.h"
67#include "opencv_json.hpp"
68#include "opencv_parallel.hpp"
69
70#if defined(__GNUC__) && !defined(__clang__) && defined(__x86_64__)
71#define CAMERA_LENS_SIMD __attribute__((target_clones("avx2,fma,default")))
72#else
73#define CAMERA_LENS_SIMD
74#endif
75
76/** Internal camera/lens implementation, kept together so the projection path is
77 * navigable. */
80/** Angle conversion shared by pose and calibration calculations. */
81constexpr double pi = 3.14159265358979323846;
82/** Binary-size unit used by file and memory bounds. */
83constexpr size_t mib = 1024 * 1024;
84/** JPEG APP1 segment data maximum (the 16-bit marker length reserves two
85 * bytes). */
86constexpr size_t maximum_exif_payload_bytes = 65533;
87/** Reject pathological dimensions before allocating decoder or OpenCV buffers.
88 */
89constexpr unsigned int maximum_image_dimension = 4096;
90/** MIMIC fixture guard: 8 megapixels bounds the full-frame software path. */
91constexpr uint64_t maximum_mock_pixels = 8388608;
92/** Fixed cache residency avoids turning tile access order into unbounded RAM
93 * growth while keeping wide fisheye strip working sets (~35 tiles) resident. */
94constexpr size_t tile_cache_entries = 48;
95/** Fisheye/Brown inversion must round-trip within a tenth of one pixel. */
96constexpr double maximum_reprojection_error_pixels = 0.1;
97/** OpenCV's iterative Brown inversion has a hard work ceiling. */
98constexpr int lens_inverse_iterations = 30;
99/** Forward-model correction passes and target after OpenCV inversion. */
100constexpr int lens_polish_passes = 8;
101constexpr double lens_polish_tolerance_px = 0.001;
102/** First exact-inverse node spacing; halved until interpolation is verified. */
103constexpr int ray_grid_initial_step = 16;
104/** Maximum verified ray interpolation error, in output pixels. */
105constexpr double ray_interpolation_tolerance_px = 0.02;
106/** Maximum ground motion between neighbouring exposure samples, in pixels. */
107constexpr double temporal_sample_spacing_px = 0.5;
109 /** @brief Close the stream owned by a File handle.
110 * @param file Non-null stream handed off by the unique pointer.
111 * @details Scope cleanup cannot report close errors; output paths that need
112 * to detect write failures must check them before relinquishing ownership. */
113 void operator()(FILE* file) const { fclose(file); }
114};
115using File = std::unique_ptr<FILE, FileCloser>;
116
117/** @brief Route a failed precondition to the CLI's diagnostic handler.
118 * @param valid Condition to enforce.
119 * @param message Diagnostic copied into the exception.
120 * @throws std::runtime_error If @p valid is false.
121 * @details Use exceptions so the job's scoped resources unwind on failure. */
122void require(bool valid, const char* message) {
123 if (!valid) throw std::runtime_error(message);
124}
125
126/** @copydoc require(bool, const char*) */
127void require(bool valid, const std::string& message) {
128 if (!valid) throw std::runtime_error(message);
129}
130
131/** Validate one finite numeric profile/geometry field and return its value.
132 * @param value JSON field.
133 * @param name Field name used in diagnostics.
134 * @param minimum Inclusive lower bound.
135 * @param maximum Inclusive upper bound.
136 * @return Validated number. */
137double number(const Json& value, const std::string& name, double minimum,
138 double maximum) {
139 require(value.is_number(), name + " must be numeric");
140 const double result = value.get<double>();
141 require(std::isfinite(result) && result >= minimum && result <= maximum,
142 name + " outside permitted range");
143 return result;
144}
145
146/** Open only a bounded regular file; O_NONBLOCK prevents special-file stalls.
147 * @param path Input path.
148 * @param maximum Maximum permitted size in bytes.
149 * @return Owning stream, or throws a descriptive error. */
150File open_regular(const std::string& path, size_t maximum) {
151 const int descriptor = open(path.c_str(), O_RDONLY | O_NONBLOCK | O_CLOEXEC);
152 require(descriptor >= 0, "cannot open input: " + path);
153 struct stat status{};
154 if (fstat(descriptor, &status) != 0 || !S_ISREG(status.st_mode) ||
155 status.st_size < 0 || static_cast<size_t>(status.st_size) > maximum) {
157 throw std::runtime_error("input must be a bounded regular file: " + path);
158 }
159 File input(fdopen(descriptor, "rb"));
160 if (!input) {
162 throw std::runtime_error("cannot open input stream: " + path);
163 }
164 return input;
165}
166
167/** Fill profile defaults, then validate schema, known keys, types and ranges.
168 * @param path USHER_LENS_PROFILE_V1 JSON file.
169 * @return Validated profile with all optional defaults materialized. */
170Json read_profile(const char* path) {
171 File input = open_regular(path, 16384);
172 std::array<char, 16385> buffer{};
173 const size_t length = fread(buffer.data(), 1, buffer.size(), input.get());
174 require(length <= 16384 && !ferror(input.get()),
175 "cannot read bounded lens profile");
176 const Json values = Json::parse(buffer.begin(), buffer.begin() + length);
177 // Defaults keep the worker runnable for examples; they are not optical
178 // measurements.
179 Json profile = Json::object({{"schema", "USHER_LENS_PROFILE_V1"},
180 {"name", "custom"},
181 {"calibration", "illustrative_not_measured"},
182 {"model", "brown"},
183 {"horizontal_fov_deg", 66.3},
184 {"vertical_fov_deg", 52.3},
185 {"diagonal_fov_deg", 160.0},
186 {"distortion", Json::array({0.0, 0.0, 0.0, 0.0, 0.0})},
187 {"chromatic_aberration", 0.0005},
188 {"exposure_s", 0.002},
189 {"exposure_ev", 0.0},
190 {"f_number", 1.79},
191 {"focal_length_mm", nullptr},
192 {"shutter", "rolling"},
193 {"readout_s", 0.02},
194 {"readout_direction", "top_to_bottom"},
195 {"motion_scale", 1.0},
196 {"angular_rate_deg_s", Json::array({0.0, 0.0, 0.0})},
197 {"vibration_deg", 0.03},
198 {"vibration_hz", 80.0},
199 {"vibration_phase_deg", 0.0},
200 {"samples", 7},
201 {"height_override_m", nullptr},
202 {"max_tiles", 121},
203 {"jpeg_quality", 95},
204 {"threads", 4},
205 {"memory_budget_mb", 192},
206 // Sensor/ISP stage: zero disables each effect, so older
207 // profiles render exactly as before.
208 {"vignetting", 0.0},
209 {"optical_blur_px", 0.0},
210 {"sensor_full_well_e", 0.0},
211 {"sensor_read_noise_e", 0.0},
212 {"isp_sharpen", 0.0}});
213 profile.set("output_width", nullptr);
214 profile.set("output_height", nullptr);
215 require(values.is_object() &&
216 values.value("schema", "") == "USHER_LENS_PROFILE_V1",
217 "invalid profile schema");
218 for (const auto& entry : values.items()) {
219 require(profile.contains(entry.first),
220 "unknown lens parameter: " + entry.first);
221 profile.set(entry.first, entry.second);
222 }
223 for (const char* key : {"name", "calibration"}) {
224 const std::string text = profile[key].get<std::string>();
225 require(!text.empty() && text.size() <= 160,
226 std::string("invalid ") + key);
227 }
228 require(profile["model"] == "brown" || profile["model"] == "fisheye",
229 "invalid lens model");
230 require(profile["shutter"] == "global" || profile["shutter"] == "rolling",
231 "invalid shutter");
232 require(profile["readout_direction"] == "top_to_bottom" ||
233 profile["readout_direction"] == "bottom_to_top",
234 "invalid readout direction");
235 struct Range {
236 const char* key;
237 double minimum, maximum;
238 };
239 // These bounds keep projective math finite and cap exposure/distortion work
240 // to supported inputs.
241 for (const Range& range :
242 std::initializer_list<Range>{{"horizontal_fov_deg", 1, 170},
243 {"vertical_fov_deg", 1, 170},
244 {"diagonal_fov_deg", 1, 170},
245 {"chromatic_aberration", 0, 0.01},
246 {"exposure_s", 0.000001, 1},
247 {"exposure_ev", -8, 8},
248 {"f_number", 0.5, 64},
249 {"readout_s", 0, 0.2},
250 {"motion_scale", 0, 10},
251 {"vibration_deg", 0, 5},
252 {"vibration_hz", 0, 1000},
253 {"vibration_phase_deg", -360, 360},
254 {"vignetting", 0, 0.95},
255 {"optical_blur_px", 0, 5},
256 {"sensor_full_well_e", 0, 1000000},
257 {"sensor_read_noise_e", 0, 100},
258 {"isp_sharpen", 0, 3}}) {
259 profile.set(range.key,
260 number(profile[range.key], range.key, range.minimum, range.maximum));
261 }
262 // Integer limits bound tile fan-out, thread creation, and explicit
263 // working-set requests.
264 for (const Range& range :
265 std::initializer_list<Range>{{"samples", 1, 65},
266 {"max_tiles", 1, 225},
267 {"jpeg_quality", 1, 100},
268 {"threads", 1, 32},
269 {"memory_budget_mb", 96, 1024}}) {
270 number(profile[range.key], range.key, range.minimum, range.maximum);
271 require(profile[range.key].is_number_integer(),
272 std::string(range.key) + " must be an integer");
273 }
274 require(profile["samples"].get<int>() % 2 == 1, "samples must be odd");
275 require(profile["output_width"].is_null() == profile["output_height"].is_null(),
276 "output_width and output_height must be set together");
277 if (!profile["output_width"].is_null()) {
278 for (const char* key : {"output_width", "output_height"}) {
280 require(profile[key].is_number_integer(), std::string(key) + " must be an integer");
281 }
282 require(profile["output_width"].get<uint64_t>() * profile["output_height"].get<uint64_t>() <=
283 maximum_mock_pixels, "output exceeds pixel limit");
284 }
285 require(
286 profile["shutter"] != "rolling" || profile["readout_s"].get<double>() > 0,
287 "rolling readout must be positive");
288 const double full_well = profile["sensor_full_well_e"].get<double>();
289 require(full_well == 0 || full_well >= 10,
290 "sensor_full_well_e must be 0 (noise off) or at least 10");
291 require(full_well > 0 || profile["sensor_read_noise_e"].get<double>() == 0,
292 "sensor_read_noise_e requires sensor_full_well_e");
293 if (!profile["height_override_m"].is_null())
294 number(profile["height_override_m"], "height_override_m", 0.1, 10000);
295 if (!profile["focal_length_mm"].is_null())
296 number(profile["focal_length_mm"], "focal_length_mm", 0.1, 1000);
297 for (const char* key : {"distortion", "angular_rate_deg_s"}) {
298 const bool distortion = std::string(key) == "distortion";
299 const size_t count =
300 distortion ? (profile["model"] == "fisheye" ? 4 : 5) : 3;
301 const double limit = distortion ? 2 : 360;
302 require(profile[key].is_array() && profile[key].size() == count,
303 std::string("invalid ") + key + " coefficients");
304 for (size_t index = 0; index < profile[key].size(); ++index)
305 number(profile[key].at(index), key, -limit, limit);
306 }
307 return profile;
308}
309
310/** libjpeg error state with a recovery point so malformed input becomes an
311 * exception. */
312struct JpegError {
313 jpeg_error_mgr manager{};
314 jmp_buf recovery;
315 void (*emit)(j_common_ptr, int) = nullptr;
316};
317
318/** @brief Return a fatal libjpeg error to the active codec recovery point.
319 * @param state Codec state carrying the installed JpegError.
320 * @details The recovery branch converts the failure to an exception; this
321 * C callback itself never unwinds the C++ stack. */
322void jpeg_failure(j_common_ptr state) {
323 longjmp(reinterpret_cast<JpegError*>(state->err)->recovery, 1);
324}
325
326/** @brief Enforce JPEG marker bounds before normal message handling.
327 * @param state Codec state carrying the installed JpegError.
328 * @param level libjpeg diagnostic level.
329 * @details Forward accepted messages to the saved handler so warning
330 * accounting remains available to the decoder. */
331void jpeg_message(j_common_ptr state, int level) {
333 reinterpret_cast<JpegError*>(state->err)->emit(state, level);
334}
335
336/** @brief Own decoder state across libjpeg's nonlocal error return. */
337struct Decoder {
338 jpeg_decompress_struct state{};
340 bool created = false;
341 cv::Mat image;
342 /** @brief Destroy the decoder if creation was attempted.
343 * @details The creation flag is armed before the libjpeg call so partial
344 * initialization can be cleaned up on the recovery path. */
346 if (created) jpeg_destroy_decompress(&state);
347 }
348};
349
350/** @brief Encoder facts of a decoded JPEG, used to calibrate profiles. */
352 int width = 0, height = 0;
353 std::array<unsigned, 64> luma_quant{}; ///< Natural order, 0 when absent.
354 std::string subsampling = "other";
355};
356
357/** Decode one regular JPEG after applying source/tile dimensions and memory
358 * checks.
359 * @param path JPEG path.
360 * @param tile True for a cache tile (must be exactly 256x256); false for a
361 * source frame.
362 * @param budget Maximum permitted working bytes.
363 * @param header Optional destination for encoder facts used by --measure.
364 * @param decode_pixels False to validate header/dimensions/budget only (used
365 * when rendering from a map cache without pinhole source sampling).
366 * @return Contiguous RGB pixels, or empty when @p decode_pixels is false. */
367cv::Mat read_jpeg(const std::string& path, bool tile, size_t budget,
368 JpegHeader* header = nullptr, bool decode_pixels = true) {
369 File input = open_regular(path, (tile ? 4 : 64) * mib);
370 auto decoder = std::make_unique<Decoder>();
371 decoder->state.err = jpeg_std_error(&decoder->error.manager);
372 decoder->error.emit = decoder->error.manager.emit_message;
373 decoder->error.manager.emit_message = jpeg_message;
374 decoder->error.manager.error_exit = jpeg_failure;
375 if (setjmp(decoder->error.recovery))
376 throw std::runtime_error("invalid JPEG: " + path);
377 decoder->created = true;
378 jpeg_create_decompress(&decoder->state);
379 jpeg_stdio_src(&decoder->state, input.get());
380 require(jpeg_read_header(&decoder->state, TRUE) == JPEG_HEADER_OK,
381 "invalid JPEG header");
382 const size_t width = decoder->state.image_width,
383 height = decoder->state.image_height;
384 require(width >= 2 && height >= 2 && width <= maximum_image_dimension &&
385 height <= maximum_image_dimension &&
386 width * height <= maximum_mock_pixels &&
387 (!tile || (width == 256 && height == 256)),
388 "unsupported JPEG dimensions: " + path);
389 require(32 * mib + width * height * 12 <= budget,
390 "input exceeds memory budget before decode");
391 if (header) {
392 header->width = static_cast<int>(width);
393 header->height = static_cast<int>(height);
394 const JQUANT_TBL* table = decoder->state.quant_tbl_ptrs[0];
395 for (size_t index = 0; table && index < header->luma_quant.size(); ++index)
396 header->luma_quant[index] = table->quantval[index];
397 if (decoder->state.num_components == 3 &&
398 decoder->state.jpeg_color_space == JCS_YCbCr) {
399 const auto* components = decoder->state.comp_info;
400 if (components[1].h_samp_factor == 1 && components[1].v_samp_factor == 1 &&
401 components[2].h_samp_factor == 1 && components[2].v_samp_factor == 1) {
402 const int horizontal = components[0].h_samp_factor,
403 vertical = components[0].v_samp_factor;
404 header->subsampling = horizontal == 2 && vertical == 2 ? "4:2:0"
405 : horizontal == 2 && vertical == 1 ? "4:2:2"
406 : horizontal == 1 && vertical == 1 ? "4:4:4"
407 : "other";
408 }
409 }
410 }
411 if (!decode_pixels) {
412 require(decoder->error.manager.num_warnings == 0, "damaged JPEG: " + path);
413 return {};
414 }
415 decoder->state.out_color_space = JCS_RGB;
416 jpeg_start_decompress(&decoder->state);
417 decoder->image.create(static_cast<int>(height), static_cast<int>(width),
418 CV_8UC3);
419 while (decoder->state.output_scanline < height) {
420 JSAMPROW row = decoder->image.ptr(decoder->state.output_scanline);
421 require(jpeg_read_scanlines(&decoder->state, &row, 1) == 1,
422 "incomplete JPEG scanline");
423 }
424 jpeg_finish_decompress(&decoder->state);
425 require(decoder->error.manager.num_warnings == 0, "damaged JPEG: " + path);
426 return decoder->image;
427}
428
429/** @brief Own encoder state independently of the output stream. */
430struct Encoder {
431 jpeg_compress_struct state{};
433 bool created = false;
434 /** @brief Release the libjpeg compressor when armed.
435 * @details The caller separately owns the stream and decides whether to
436 * publish the temporary output. */
438 if (created) jpeg_destroy_compress(&state);
439 }
440};
441
442/** @brief Own an unpublished derivative and its temporary sibling path. */
444 std::string path;
446 /** @brief Open an exclusive temporary file beside the requested output.
447 * @param output Final derivative path.
448 * @throws std::runtime_error If temporary-file creation or stream setup fails.
449 * @details A sibling path permits rename-based publication on the same
450 * filesystem. Clean up the descriptor and file if wrapping it fails. */
451 explicit TemporaryOutput(const std::string& output) {
452 std::string pattern = output + ".tmp-XXXXXX";
453 const int descriptor = mkstemp(pattern.data());
454 require(descriptor >= 0, "cannot create temporary derivative");
455 path = pattern;
456 file.reset(fdopen(descriptor, "wb"));
457 if (!file) {
459 unlink(path.c_str());
460 throw std::runtime_error("cannot open temporary derivative");
461 }
462 }
463 /** @brief Remove an unpublished temporary pathname.
464 * @details A successful rename has already moved the file; cleanup targets
465 * only the temporary name, never the published derivative. */
467 if (!path.empty()) unlink(path.c_str());
468 }
469};
470
471/** Write a JPEG derivative with EXIF into a same-directory temporary then
472 * rename it.
473 * @param output Destination, which is never the clean input.
474 * @param image RGB derivative.
475 * @param quality Profile JPEG quality.
476 * @param metadata EXIF payload to retain and extend. */
477void write_jpeg(const std::string& output, const cv::Mat& image, int quality,
478 const exif_document* metadata) {
479 std::array<unsigned char, EXIF_MAX_PAYLOAD> payload{};
480 size_t length = 0;
481 require(exif_serialize(metadata, payload.data(), payload.size(), &length) == 0,
482 "EXIF record exceeds JPEG APP1 capacity");
483 TemporaryOutput temporary(output);
484 auto encoder = std::make_unique<Encoder>();
485 encoder->state.err = jpeg_std_error(&encoder->error.manager);
486 encoder->error.manager.error_exit = jpeg_failure;
487 if (setjmp(encoder->error.recovery))
488 throw std::runtime_error("JPEG encoding failed");
489 encoder->created = true;
490 jpeg_create_compress(&encoder->state);
491 jpeg_stdio_dest(&encoder->state, temporary.file.get());
492 encoder->state.image_width = image.cols;
493 encoder->state.image_height = image.rows;
494 encoder->state.input_components = 3;
495 encoder->state.in_color_space = JCS_RGB;
496 jpeg_set_defaults(&encoder->state);
497 jpeg_set_quality(&encoder->state, quality, TRUE);
498 jpeg_start_compress(&encoder->state, TRUE);
499 jpeg_write_marker(&encoder->state, JPEG_APP0 + 1, payload.data(), static_cast<unsigned>(length));
500 while (encoder->state.next_scanline < static_cast<unsigned>(image.rows)) {
501 JSAMPROW row =
502 const_cast<JSAMPROW>(image.ptr(encoder->state.next_scanline));
503 require(jpeg_write_scanlines(&encoder->state, &row, 1) == 1,
504 "cannot encode scanline");
505 }
506 jpeg_finish_compress(&encoder->state);
507 require(fflush(temporary.file.get()) == 0 &&
508 fsync(fileno(temporary.file.get())) == 0,
509 "cannot flush derivative");
510 require(fclose(temporary.file.release()) == 0, "cannot close derivative");
511 require(rename(temporary.path.c_str(), output.c_str()) == 0,
512 "cannot publish derivative");
513 temporary.path.clear();
514}
515
516/** Read EXIF text while preserving the eight-byte UserComment encoding prefix.
517 * @param metadata Parsed EXIF object.
518 * @param ifd EXIF directory.
519 * @param tag Tag identifier.
520 * @return Bounded text, or empty when absent. */
521std::string tag_text(const exif_document* metadata, exif_directory ifd, uint16_t tag) {
522 exif_value entry{};
523 if (!exif_document_get(metadata, ifd, tag, &entry)) return {};
525 (entry.format == EXIF_ASCII || entry.format == EXIF_UNDEFINED), "invalid EXIF text");
526 std::string result(reinterpret_cast<const char*>(entry.data), entry.size);
527 const size_t minimum =
528 tag == EXIF_USER_COMMENT ? std::min<size_t>(8, result.size()) : 0;
529 while (result.size() > minimum && result.back() == '\0') result.pop_back();
530 return result;
531}
532
533/** Replace or create an EXIF entry with zero-initialized storage.
534 * @param metadata EXIF object to update.
535 * @param ifd Destination directory.
536 * @param tag EXIF tag identifier.
537 * @param format EXIF storage format.
538 * @param components Number of values.
539 * @return Borrowed entry owned by @p metadata; throws if bounded allocation
540 * fails. */
541unsigned char* replace_tag(exif_document* metadata, exif_directory ifd, uint16_t tag,
542 exif_format format, size_t components) {
543 require(components <= UINT32_MAX, "invalid EXIF component count");
544 unsigned char* entry = exif_document_reserve(metadata, ifd, tag, format,
545 static_cast<uint32_t>(components));
546 require(entry != nullptr, "EXIF tag exceeds bounded storage");
547 return entry;
548}
549
550/** Replace an ASCII or UNDEFINED EXIF text tag and include its terminating
551 * byte.
552 * @param metadata EXIF object to update.
553 * @param ifd Destination directory.
554 * @param tag EXIF tag identifier.
555 * @param text Text bytes.
556 * @param undefined Use EXIF_FORMAT_UNDEFINED (needed for UserComment). */
557void set_text(exif_document* metadata, exif_directory ifd, uint16_t tag,
558 const std::string& text, bool undefined = false) {
559 unsigned char* entry = replace_tag(
560 metadata, ifd, tag, undefined ? EXIF_UNDEFINED : EXIF_ASCII,
561 text.size() + 1);
562 for (size_t index = 0; index < text.size(); ++index)
563 entry[index] = static_cast<unsigned char>(text[index]);
564}
565
566/** Store a floating measurement as a bounded EXIF rational.
567 * @param metadata EXIF object to update.
568 * @param tag EXIF measurement tag.
569 * @param value Measurement value.
570 * @param denominator Decimal scale used to preserve useful precision.
571 * @param signed_value Select signed rational format for exposure compensation.
572 */
573void set_rational(exif_document* metadata, uint16_t tag, double value,
574 unsigned denominator, bool signed_value = false) {
575 require(denominator > 0 && std::isfinite(value), "invalid EXIF rational");
576 const double scaled = std::round(value * denominator);
577 require(std::isfinite(scaled) &&
578 (signed_value ? scaled >= INT32_MIN && scaled <= INT32_MAX && denominator <= INT32_MAX
579 : scaled >= 0 && scaled <= UINT32_MAX), "EXIF rational out of range");
580 unsigned char* entry = replace_tag(
581 metadata, EXIF_CAMERA, tag,
582 signed_value ? EXIF_SRATIONAL : EXIF_RATIONAL, 1);
583 if (signed_value) {
584 exif_encode_long(entry, static_cast<uint32_t>(static_cast<int32_t>(scaled)));
585 } else {
586 exif_encode_long(entry, static_cast<uint32_t>(scaled));
587 }
588 exif_encode_long(entry + 4, denominator);
589}
590
591/** Validate USHER shot tags and extract the source pose and camera geometry.
592 * @param metadata EXIF from the clean input JPEG.
593 * @param[out] geometry MIMIC source geometry, or null for an explicit
594 * real-camera profile.
595 * @return Validated pose values in JSON form. */
596Json read_shot(const exif_document* metadata, Json& geometry) {
597 geometry =
599 const bool xyz_geometry = geometry.value("tile_scheme", "") == "XYZ";
600 // Older MIMIC EXIF used GOOGLE as a cache-layout token, not imagery provenance.
601 const bool legacy_cache_geometry = geometry.value("provider", "") == "GOOGLE";
602 require(geometry.value("schema", "") == "USHER_MOCK_MAP_V1" &&
603 (xyz_geometry || legacy_cache_geometry) &&
604 geometry.value("zoom", 0) == 18 &&
605 geometry.value("projection", "") == "pinhole_flat_ground",
606 "input must be an unmodified USHER MIMIC JPEG");
607 // Normalize older cache-key metadata before writing derived-image EXIF.
608 Json normalized_geometry = Json::object();
609 for (const auto& field : geometry.items()) {
610 if (field.first != "provider")
611 normalized_geometry.set(field.first, field.second);
612 }
613 normalized_geometry.set("tile_scheme", "XYZ");
614 geometry = std::move(normalized_geometry);
616 .find("USHER_LENS_V1") == std::string::npos,
617 "lens effects already applied");
618 exif_value orientation{};
619 uint32_t orientation_value = 0;
620 require(!exif_document_get(metadata, EXIF_PRIMARY, EXIF_ORIENTATION, &orientation) ||
621 (orientation.format == EXIF_SHORT && orientation.count == 1 &&
622 exif_value_unsigned(&orientation, 0, &orientation_value) && orientation_value == 1),
623 "non-upright EXIF orientation is unsupported");
624 std::istringstream description(
626 std::string field;
627 std::getline(description, field, ';');
628 require(field.find("Paparazzi USHER ") == 0, "missing USHER flight metadata");
629 std::map<std::string, std::string> fields;
630 while (std::getline(description, field, ';')) {
631 const size_t start = field.find_first_not_of(" \t\r\n");
632 const size_t end = field.find_last_not_of(" \t\r\n");
633 require(start != std::string::npos, "empty flight metadata field");
634 field = field.substr(start, end - start + 1);
635 const size_t separator = field.find('=');
636 require(separator != std::string::npos &&
637 fields
638 .emplace(field.substr(0, separator),
639 field.substr(separator + 1))
640 .second,
641 "malformed or duplicate flight metadata field");
642 }
644 for (const char* key :
645 {"lat_deg", "lon_deg", "roll_deg", "pitch_deg", "yaw_deg", "speed_m_s",
646 "course_deg", "groundalt_m"}) {
647 const std::string original = "original_" + std::string(key);
648 const std::string& text = fields.at(
649 (std::string(key) == "lat_deg" || std::string(key) == "lon_deg") &&
650 fields.count(original)
651 ? original
652 : key);
653 size_t consumed = 0;
654 const double value = std::stod(text, &consumed);
655 require(consumed == text.size(), "invalid flight number");
656 shot.set(key, number(value, key, -1e7, 1e7));
657 }
658 number(shot["lat_deg"], "latitude", -85.05112878, 85.05112878);
659 number(shot["lon_deg"], "longitude", -180, 180);
660 number(shot["speed_m_s"], "speed", 0, 1000);
661 number(shot["groundalt_m"], "AGL", 0.001, 10000);
662 for (const char* key : {"horizontal_fov_deg", "vertical_fov_deg"})
663 number(geometry.at(key), key, 0.001, 178.999);
664 return shot;
665}
666
667/** Apply the profile budget and additional conservative host-memory/thread
668 * limits. */
669struct Resources {
670 size_t budget;
672 /** @brief Resolve requested resources against current host limits.
673 * @param profile Validated memory budget and thread settings.
674 * @details Restrict small hosts to one thread and at most one third of
675 * total memory. Available-memory evidence may lower the budget further to
676 * leave headroom for other processes. */
677 explicit Resources(const Json& profile)
678 : budget(profile["memory_budget_mb"].get<size_t>() * mib),
679 threads(profile["threads"].get<int>()) {
680 std::ifstream input("/proc/meminfo");
681 std::string line;
682 while (std::getline(input, line)) {
683 std::istringstream fields(line);
684 std::string key;
685 size_t value;
686 if (!(fields >> key >> value)) continue;
687 if (key == "MemTotal:" && value <= 768 * 1024) {
688 budget = std::min(budget, value * 1024 / 3);
689 threads = 1;
690 } else if (key == "MemAvailable:") {
691 budget =
692 std::min(budget, value > 128 * 1024 ? value * 1024 - 128 * mib : 0);
693 }
694 }
695 }
696};
697
698/** Inclusive tile bounds; @c right and @c bottom identify actual covered tiles.
699 */
700struct Bounds {
701 int64_t left, top, right, bottom;
702 /** @brief Validate an inclusive tile rectangle and count its cells.
703 * @param limit Maximum permitted tile count.
704 * @return Number of tiles including both boundary rows and columns.
705 * @throws std::runtime_error If the extent, count, or Mercator row range is invalid.
706 * @details Bound each span before multiplying. Columns may cross the
707 * antimeridian and are wrapped at lookup; rows cannot wrap past the poles. */
708 size_t count(int limit) const {
709 require(right >= left && bottom >= top && right - left < limit &&
710 bottom - top < limit,
711 "lens tile rectangle exceeds max_tiles");
712 const size_t count = (right - left + 1) * (bottom - top + 1);
713 require(count <= static_cast<size_t>(limit),
714 "lens tile rectangle exceeds max_tiles");
715 require(top >= 0 && bottom < (1 << 18),
716 "view beyond Web Mercator coverage");
717 return count;
718 }
719};
720
721/** @brief 8-bit sRGB to linear-light lookup, built once. */
722const std::array<float, 256>& srgb_decoding() {
723 static const std::array<float, 256> table = [] {
724 std::array<float, 256> values{};
725 for (size_t index = 0; index < values.size(); ++index) {
726 const float encoded = static_cast<float>(index) / 255.0f;
727 values[index] = encoded <= 0.04045f
728 ? encoded / 12.92f
729 : std::pow((encoded + 0.055f) / 1.055f, 2.4f);
730 }
731 return values;
732 }();
733 return table;
734}
735
736/** Small LRU for source tiles; the fixed entry cap bounds decoded tile memory.
737 */
739 struct Entry {
740 int64_t column, row;
741 cv::Mat linear;
742 };
743 std::list<Entry> entries;
744 std::vector<std::pair<int64_t, int64_t>> verified;
745 std::string root;
746 size_t budget;
747
748 public:
749 /** @brief Configure a cache with no resident tiles.
750 * @param path Root of the local map cache.
751 * @param limit Working-byte budget passed to the JPEG decoder.
752 * @details Load lazily so views that do not overlap need not share an
753 * eagerly decoded mosaic. */
754 TileCache(const std::string& path, size_t limit)
755 : root(path), budget(limit) {}
756 /** @brief Ensure a tile in the frame's bounding box exists and is valid
757 * without re-decoding tiles already verified earlier in the same render. */
758 void verify(int64_t column, int64_t row) {
759 const int64_t wrapped = (column % (1 << 18) + (1 << 18)) % (1 << 18);
760 for (const auto& seen : verified)
761 if (seen.first == wrapped && seen.second == row) return;
762 tile(column, row);
763 }
764 /** @brief Fetch a linearized float32 RGB tile from the local zoom-18 cache.
765 * @param column XYZ column, which may be outside the world range before wrapping.
766 * @param row Validated XYZ row; not wrapped.
767 * @return Reference-counted 256x256 CV_32FC3 pixels; callers must treat them
768 * as read-only.
769 * @throws std::runtime_error If the required tile cannot be read or decoded.
770 * @details Normalize the column before lookup so opposite antimeridian
771 * representations share an entry. Decoding and sRGB-to-linear conversion
772 * happen once per unique tile rather than per strip/channel. */
773 cv::Mat tile(int64_t column, int64_t row) {
774 column = (column % (1 << 18) + (1 << 18)) % (1 << 18);
775 for (auto entry = entries.begin(); entry != entries.end(); ++entry) {
776 if (entry->column == column && entry->row == row) {
777 entries.splice(entries.begin(), entries, entry);
778 return entries.front().linear;
779 }
780 }
781 const cv::Mat encoded =
782 read_jpeg(root + "/GOOGLE/18/" + std::to_string(column) + "/" +
783 std::to_string(row) + ".jpeg",
784 true, budget);
785 cv::Mat linear(256, 256, CV_32FC3);
786 const float* decode = srgb_decoding().data();
787 const unsigned char* source = encoded.data;
788 float* destination = linear.ptr<float>();
789 for (int index = 0; index < 256 * 256 * 3; ++index)
790 destination[index] = decode[source[index]];
791 bool recorded = false;
792 for (const auto& seen : verified)
793 if (seen.first == column && seen.second == row) recorded = true;
794 if (!recorded) verified.emplace_back(column, row);
795 entries.push_front({column, row, linear});
796 if (entries.size() > tile_cache_entries) entries.pop_back();
797 return linear;
798 }
799};
800
801/** Camera intrinsics and Brown/fisheye distortion coefficients for ray
802 * generation. */
803struct Lens {
806 cv::Mat distortion;
808 /** @brief Build intrinsics for the selected distortion model.
809 * @param width Output columns in pixels.
810 * @param height Output rows in pixels.
811 * @param values Validated model, field of view, and distortion coefficients.
812 * @details Brown intrinsics use the two rectilinear fields of view. Fisheye
813 * intrinsics derive a shared focal length from the distorted diagonal ray. */
814 Lens(int width, int height, const Json& values)
815 : profile(values), fisheye(values["model"] == "fisheye") {
816 const double center_x = (width - 1) * 0.5, center_y = (height - 1) * 0.5;
817 std::vector<double> coefficients =
818 values["distortion"].get<std::vector<double>>();
819 distortion = cv::Mat(coefficients).clone();
820 if (fisheye) {
821 const double angle = values["diagonal_fov_deg"].get<double>() * pi / 360;
822 double polynomial = 1, power = angle * angle;
823 for (double coefficient : coefficients) {
824 polynomial += coefficient * power;
825 power *= angle * angle;
826 }
827 focal_x = focal_y = std::hypot(center_x, center_y) / (angle * polynomial);
828 } else {
829 focal_x = center_x /
830 std::tan(values["horizontal_fov_deg"].get<double>() * pi / 360);
831 focal_y = center_y /
832 std::tan(values["vertical_fov_deg"].get<double>() * pi / 360);
833 }
834 require(std::isfinite(focal_x) && focal_x > 0 && focal_y > 0,
835 "invalid lens focal scale");
836 }
837 /** @brief Camera matrix for one channel focal scale. */
838 cv::Matx33d intrinsic(int width, int height, double scale) const {
839 return cv::Matx33d(focal_x * scale, 0, (width - 1) * 0.5, 0,
840 focal_y * scale, (height - 1) * 0.5, 0, 0, 1);
841 }
842 /** Forward lens model: normalized rays to distorted pixel coordinates.
843 * @param rays N x 1 CV_32FC2 normalized camera rays.
844 * @param width Full image width.
845 * @param height Full image height.
846 * @param scale Per-channel focal scaling.
847 * @return N x 1 CV_32FC2 pixel coordinates. */
848 cv::Mat project(const cv::Mat& rays, int width, int height,
849 double scale) const {
850 const cv::Matx33d camera = intrinsic(width, height, scale);
851 cv::Mat result;
852 if (fisheye) {
853 cv::fisheye::distortPoints(rays, result, camera, distortion);
854 return result;
855 }
856 cv::Mat objects(rays.rows, 1, CV_32FC3);
857 for (int index = 0; index < rays.rows; ++index) {
858 const cv::Vec2f point = rays.at<cv::Vec2f>(index);
859 objects.at<cv::Vec3f>(index) = cv::Vec3f(point[0], point[1], 1);
860 }
861 cv::projectPoints(objects, cv::Vec3d(), cv::Vec3d(), camera, distortion,
862 result, cv::noArray());
863 return result;
864 }
865 /** Exactly invert distortion for arbitrary pixels, then forward-project to
866 * reject non-convergent rays.
867 * @param pixels N x 1 CV_32FC2 pixel coordinates.
868 * @param width Full image width.
869 * @param height Full image height.
870 * @param scale Per-channel focal scaling for chromatic-aberration simulation.
871 * @return N x 1 normalized camera rays. */
872 cv::Mat undistort(const cv::Mat& pixels, int width, int height,
873 double scale) const {
874 const cv::Matx33d intrinsic = this->intrinsic(width, height, scale);
875 cv::Mat normalized, projected;
876 auto project = [&](const cv::Mat& rays) {
877 return this->project(rays, width, height, scale);
878 };
879 if (fisheye) {
880 cv::fisheye::undistortPoints(pixels, normalized, intrinsic, distortion);
881 } else {
882 cv::undistortPoints(
883 pixels, normalized, intrinsic, distortion, cv::noArray(),
884 cv::noArray(),
885 cv::TermCriteria(cv::TermCriteria::COUNT | cv::TermCriteria::EPS,
887 }
888 projected = project(normalized);
889 // OpenCV's fixed-point inverse can stop several hundredths of a pixel
890 // short under strong distortion; Newton-polish it against the forward
891 // model with a numeric Jacobian, keeping only improving passes.
892 double residual = cv::checkRange(normalized) && cv::checkRange(projected)
893 ? cv::norm(projected, pixels, cv::NORM_INF)
894 : INFINITY;
895 constexpr float delta = 1e-4f;
896 for (int pass = 0; pass < lens_polish_passes &&
897 residual > lens_polish_tolerance_px &&
898 std::isfinite(residual);
899 ++pass) {
900 const cv::Mat shift_x = normalized + cv::Scalar(delta, 0),
901 shift_y = normalized + cv::Scalar(0, delta);
902 const cv::Mat moved_x = project(shift_x), moved_y = project(shift_y);
903 cv::Mat candidate = normalized.clone();
904 for (int index = 0; index < pixels.rows; ++index) {
905 const cv::Vec2f base = projected.at<cv::Vec2f>(index);
906 const cv::Vec2f column_x = (moved_x.at<cv::Vec2f>(index) - base) / delta;
907 const cv::Vec2f column_y = (moved_y.at<cv::Vec2f>(index) - base) / delta;
908 const cv::Vec2f error = pixels.at<cv::Vec2f>(index) - base;
909 const float determinant =
910 column_x[0] * column_y[1] - column_y[0] * column_x[1];
911 if (!(std::abs(determinant) > 1e-12f)) continue;
912 candidate.at<cv::Vec2f>(index) += cv::Vec2f(
913 (column_y[1] * error[0] - column_y[0] * error[1]) / determinant,
914 (column_x[0] * error[1] - column_x[1] * error[0]) / determinant);
915 }
916 const cv::Mat candidate_projected = project(candidate);
917 if (!cv::checkRange(candidate) || !cv::checkRange(candidate_projected))
918 break;
919 const double candidate_residual =
920 cv::norm(candidate_projected, pixels, cv::NORM_INF);
921 if (candidate_residual >= residual) break;
922 normalized = candidate;
923 projected = candidate_projected;
924 residual = candidate_residual;
925 }
926 require(cv::checkRange(normalized) && cv::checkRange(projected) &&
927 cv::norm(projected, pixels, cv::NORM_INF) <=
929 "lens inversion did not converge");
930 return normalized;
931 }
932 /** Normalized rays for one strip; see RayField for the accuracy contract.
933 * @param width Full image width.
934 * @param height Full image height.
935 * @param first First row represented by this strip.
936 * @param rows Strip row count.
937 * @param scale Per-channel focal scaling.
938 * @return rows x width CV_32FC2 normalized rays. */
939 cv::Mat rays(int width, int height, int first, int rows, double scale) const;
940};
941
942/** Full-frame ray field: exact inverse distortion on a coarse node grid,
943 * bilinear interpolation between nodes.
944 * @details Brown and fisheye inversion is iterative and dominated rendering
945 * time when evaluated per pixel and per colour. The distortion field is
946 * smooth, so the node step is halved until every interpolated cell-centre
947 * ray forward-projects within ray_interpolation_tolerance_px of its pixel.
948 * Step 1 is the exact per-pixel inverse. */
949class RayField {
950 std::vector<float> node_x, node_y;
951 cv::Mat nodes;
952 std::vector<int> cell_x;
953 std::vector<float> weight_x;
955
956 static std::vector<float> axis(int size, int step) {
957 std::vector<float> values;
958 for (int value = 0; value < size - 1; value += step) values.push_back(value);
959 values.push_back(static_cast<float>(size - 1));
960 return values;
961 }
962 static void locate(const std::vector<float>& axis, float value, int& cell,
963 float& weight) {
964 cell = static_cast<int>(
965 std::upper_bound(axis.begin(), axis.end() - 1, value) - axis.begin()) - 1;
966 cell = std::clamp(cell, 0, static_cast<int>(axis.size()) - 2);
967 weight = (value - axis[cell]) / (axis[cell + 1] - axis[cell]);
968 }
969 // Nodes are unit view directions: bounded and smooth up to 90 degrees,
970 // unlike tangent-plane rays, so wide fisheye corners interpolate well.
971 cv::Vec2f blend(int cell_row, float weight_row, int cell_column,
972 float weight_column) const {
973 const cv::Vec3f* top = nodes.ptr<cv::Vec3f>(cell_row);
974 const cv::Vec3f* bottom = nodes.ptr<cv::Vec3f>(cell_row + 1);
975 const cv::Vec3f upper = top[cell_column] * (1 - weight_column) +
976 top[cell_column + 1] * weight_column;
977 const cv::Vec3f lower = bottom[cell_column] * (1 - weight_column) +
978 bottom[cell_column + 1] * weight_column;
979 const cv::Vec3f direction = upper * (1 - weight_row) + lower * weight_row;
980 return cv::Vec2f(direction[0] / direction[2], direction[1] / direction[2]);
981 }
982
983 public:
984 int step = 1;
985 double error_px = 0;
986 /** @brief Build and verify the ray field for one channel focal scale.
987 * @throws std::runtime_error If exact inversion fails at any node/check. */
988 RayField(const Lens& lens, int width, int height, double scale,
989 size_t budget = static_cast<size_t>(-1))
990 : columns(width) {
991 for (step = ray_grid_initial_step;; step /= 2) {
992 node_x = axis(width, step);
993 node_y = axis(height, step);
994 // Include inverse-polishing temporaries before allocating a dense grid.
995 require(node_x.size() * node_y.size() <= budget / 256,
996 "ray grid exceeds memory budget");
997 cv::Mat pixels(static_cast<int>(node_x.size() * node_y.size()), 1, CV_32FC2);
998 for (size_t row = 0; row < node_y.size(); ++row)
999 for (size_t column = 0; column < node_x.size(); ++column)
1000 pixels.at<cv::Vec2f>(static_cast<int>(row * node_x.size() + column)) =
1001 cv::Vec2f(node_x[column], node_y[row]);
1002 const cv::Mat rays = lens.undistort(pixels, width, height, scale);
1003 nodes.create(static_cast<int>(node_y.size()),
1004 static_cast<int>(node_x.size()), CV_32FC3);
1005 for (int index = 0; index < rays.rows; ++index) {
1006 const cv::Vec2f ray = rays.at<cv::Vec2f>(index);
1007 nodes.at<cv::Vec3f>(index / nodes.cols, index % nodes.cols) =
1008 cv::Vec3f(ray[0], ray[1], 1) / std::sqrt(1 + ray.dot(ray));
1009 }
1010 error_px = 0;
1011 if (step == 1) break;
1012 const int cells_x = static_cast<int>(node_x.size()) - 1,
1013 cells_y = static_cast<int>(node_y.size()) - 1;
1014 cv::Mat centres(cells_x * cells_y, 1, CV_32FC2);
1015 for (int row = 0; row < cells_y; ++row)
1016 for (int column = 0; column < cells_x; ++column)
1017 centres.at<cv::Vec2f>(row * cells_x + column) =
1018 cv::Vec2f((node_x[column] + node_x[column + 1]) * 0.5f,
1019 (node_y[row] + node_y[row + 1]) * 0.5f);
1020 cv::Mat blended(centres.rows, 1, CV_32FC2);
1021 for (int row = 0; row < cells_y; ++row)
1022 for (int column = 0; column < cells_x; ++column)
1023 blended.at<cv::Vec2f>(row * cells_x + column) =
1024 blend(row, 0.5f, column, 0.5f);
1025 const cv::Mat reprojected = lens.project(blended, width, height, scale);
1026 error_px = cv::checkRange(reprojected)
1027 ? cv::norm(reprojected, centres, cv::NORM_INF)
1028 : INFINITY;
1030 }
1031 cell_x.resize(width);
1032 weight_x.resize(width);
1033 for (int column = 0; column < width; ++column)
1034 locate(node_x, static_cast<float>(column), cell_x[column], weight_x[column]);
1035 }
1036 /** @brief Resident bytes of the node grid and column lookup. */
1037 size_t bytes() const {
1038 return nodes.total() * nodes.elemSize() +
1039 cell_x.size() * sizeof(int) + weight_x.size() * sizeof(float) +
1040 (node_x.size() + node_y.size()) * sizeof(float);
1041 }
1042 /** @brief Interpolate rays at horizontal grid nodes for one full-frame row.
1043 * @details Along each cell [node_x[c], node_x[c + 1]], flat-ground
1044 * projection is a fractional-linear function with positive denominator and
1045 * therefore monotone, so row extrema occur at these node columns. */
1046 int node_columns() const { return nodes.cols; }
1047 void row_nodes_into(int full_row, float* rx, float* ry) const {
1048 int cell_row;
1049 float weight_row;
1050 locate(node_y, static_cast<float>(full_row), cell_row, weight_row);
1051 const cv::Vec3f* top = nodes.ptr<cv::Vec3f>(cell_row);
1052 const cv::Vec3f* bottom = nodes.ptr<cv::Vec3f>(cell_row + 1);
1053 for (int col = 0; col < nodes.cols; ++col) {
1054 const cv::Vec3f direction =
1055 top[col] * (1.0f - weight_row) + bottom[col] * weight_row;
1056 const float inv_z = 1.0f / direction[2];
1057 rx[col] = direction[0] * inv_z;
1058 ry[col] = direction[1] * inv_z;
1059 }
1060 }
1061 /** @brief Interpolate rays for one full-frame row into a caller buffer. */
1062 void row_into(int full_row, cv::Vec2f* destination) const {
1063 int cell_row;
1064 float weight_row;
1065 locate(node_y, static_cast<float>(full_row), cell_row, weight_row);
1066 for (int column = 0; column < columns; ++column)
1067 destination[column] =
1068 blend(cell_row, weight_row, cell_x[column], weight_x[column]);
1069 }
1070 /** @brief Interpolate rays for full-frame rows [first, first + rows). */
1071 cv::Mat strip(int first, int rows) const {
1072 cv::Mat result(rows, columns, CV_32FC2);
1073 cv::parallel_for_(cv::Range(0, rows), [&](const cv::Range& range) {
1074 for (int row = range.start; row < range.end; ++row)
1075 row_into(first + row, result.ptr<cv::Vec2f>(row));
1076 });
1077 return result;
1078 }
1079 /** @brief Interpolate one ray at a sub-pixel position. */
1080 cv::Vec2f at(float x, float y) const {
1081 int cell_row, cell_column;
1082 float weight_row, weight_column;
1083 locate(node_y, y, cell_row, weight_row);
1084 locate(node_x, x, cell_column, weight_column);
1085 return blend(cell_row, weight_row, cell_column, weight_column);
1086 }
1087};
1088
1089cv::Mat Lens::rays(int width, int height, int first, int rows,
1090 double scale) const {
1091 return RayField(*this, width, height, scale).strip(first, rows);
1092}
1093
1094/** Flight pose and profile motion/exposure settings used for ground projection.
1095 */
1096struct Pose {
1098 std::array<double, 3> angles, rates;
1100 /** @brief Resolve the shot geometry and time-dependent motion model.
1101 * @param shot Validated flight pose and ground motion.
1102 * @param profile Validated lens motion, height, and shutter settings.
1103 * @details Global shutter removes the row readout term. Reverse rolling
1104 * readout negates it, preserving a common midpoint time for both directions. */
1105 Pose(const Json& shot, const Json& profile) {
1106 altitude = profile["height_override_m"].is_null()
1107 ? shot["groundalt_m"].get<double>()
1108 : profile["height_override_m"].get<double>();
1109 speed =
1110 shot["speed_m_s"].get<double>() * profile["motion_scale"].get<double>();
1111 course = shot["course_deg"].get<double>() * pi / 180;
1112 angles = {shot["roll_deg"].get<double>(), shot["pitch_deg"].get<double>(),
1113 shot["yaw_deg"].get<double>()};
1114 rates = profile["angular_rate_deg_s"].get<std::array<double, 3>>();
1115 readout = profile["shutter"] == "rolling"
1116 ? profile["readout_s"].get<double>()
1117 : 0;
1118 if (profile["readout_direction"] == "bottom_to_top") readout = -readout;
1119 exposure = profile["exposure_s"].get<double>();
1120 vibration = profile["vibration_deg"].get<double>();
1121 frequency = profile["vibration_hz"].get<double>();
1122 phase = profile["vibration_phase_deg"].get<double>() * pi / 180;
1123 }
1124 /** @brief Check whether camera geometry varies across exposure samples.
1125 * @return True without translation, angular rate, or time-varying vibration.
1126 * @details Fixed attitude is permitted; only motion over the exposure
1127 * requires evaluating distinct sample times. */
1128 bool stationary() const {
1129 return speed == 0 && rates == std::array<double, 3>{0, 0, 0} &&
1130 (vibration == 0 || frequency == 0);
1131 }
1132 /** Intersect each ray with a flat ground plane at its row/exposure time.
1133 * @param rays Normalized strip rays.
1134 * @param first First full-image row in @p rays.
1135 * @param height Full image height used for rolling-shutter timing.
1136 * @param sample Exposure-time offset from the center of the shot.
1137 * @return East and north displacement matrices in meters. */
1138 std::pair<cv::Mat, cv::Mat> ground(const cv::Mat& rays, int first, int height,
1139 double sample) const {
1140 cv::Mat east(rays.size(), CV_32F), north(rays.size(), CV_32F);
1141 std::atomic<bool> horizon{false};
1142 cv::parallel_for_(cv::Range(0, rays.rows), [&](const cv::Range& range) {
1143 for (int row = range.start; row < range.end; ++row) {
1144 const float instant =
1145 static_cast<float>(
1146 (static_cast<double>(first + row) / (height - 1) - 0.5) *
1147 readout) +
1148 static_cast<float>(sample);
1149 const double cycle = 2 * pi * frequency * instant + phase;
1150 const double roll =
1151 (angles[0] + rates[0] * instant + vibration * std::sin(cycle)) * pi /
1152 180;
1153 const double pitch = (angles[1] + rates[1] * instant +
1154 vibration * std::sin(cycle + pi / 2)) *
1155 pi / 180;
1156 const double yaw = (angles[2] + rates[2] * instant) * pi / 180;
1157 const float cos_roll = std::cos(roll), sin_roll = std::sin(roll),
1158 cos_pitch = std::cos(pitch), sin_pitch = std::sin(pitch);
1159 const float cos_yaw = std::cos(yaw), sin_yaw = std::sin(yaw);
1160 const float north_forward = cos_yaw * cos_pitch,
1161 north_right =
1162 cos_yaw * sin_pitch * sin_roll - sin_yaw * cos_roll;
1163 const float north_down =
1164 cos_yaw * sin_pitch * cos_roll + sin_yaw * sin_roll;
1165 const float east_forward = sin_yaw * cos_pitch,
1166 east_right =
1167 sin_yaw * sin_pitch * sin_roll + cos_yaw * cos_roll;
1168 const float east_down =
1169 sin_yaw * sin_pitch * cos_roll - cos_yaw * sin_roll;
1170 const float down_right = cos_pitch * sin_roll,
1171 down_down = cos_pitch * cos_roll;
1172 const float translation_north = speed * instant * std::cos(course),
1173 translation_east = speed * instant * std::sin(course);
1174 for (int column = 0; column < rays.cols; ++column) {
1175 const cv::Vec2f point = rays.at<cv::Vec2f>(row, column);
1176 const float forward = -point[1], right = point[0];
1177 const float down =
1178 -sin_pitch * forward + down_right * right + down_down;
1179 if (!(down > 1e-4f)) {
1180 horizon = true;
1181 return;
1182 }
1183 const float scale = altitude / down;
1184 north.at<float>(row, column) =
1185 (north_forward * forward + north_right * right + north_down) *
1186 scale +
1187 translation_north;
1188 east.at<float>(row, column) =
1189 (east_forward * forward + east_right * right + east_down) * scale +
1190 translation_east;
1191 }
1192 }
1193 });
1194 require(!horizon, "lens/exposure rays reach the horizon");
1195 require(cv::checkRange(east) && cv::checkRange(north),
1196 "non-finite ground projection");
1197 return {east, north};
1198 }
1199};
1200
1201struct Rendered {
1202 cv::Mat image;
1204};
1205
1206/** Temporal exposure discretization chosen from the actual blur extent. */
1209 double blur_px;
1210};
1211
1212/** @brief Choose the fewest odd exposure samples that resolve motion blur.
1213 * @param field Reference-channel ray field.
1214 * @param pose Shot motion, vibration and shutter model.
1215 * @param width Output width.
1216 * @param height Output height.
1217 * @param maximum Profile sample ceiling (odd).
1218 * @return Sample count whose neighbouring-sample ground spacing is at most
1219 * temporal_sample_spacing_px at every probe, plus the largest blur path.
1220 * @details Probe the corners, edge centres and centre of the frame, follow
1221 * each ray's ground intersection across the exposure (including vibration
1222 * reversals) and convert path length to pixels with the local ground
1223 * sampling distance. Sub-pixel blur therefore costs few samples, while
1224 * long exposures keep every sample the profile allows. */
1226 int width, int height, int maximum) {
1227 if (pose.stationary()) return {1, 0};
1228 const int steps = std::max(
1229 32, static_cast<int>(std::ceil(
1230 pose.exposure * (pose.vibration > 0 ? pose.frequency : 0) * 8)));
1231 double blur = 0;
1232 for (float y : {0.0f, (height - 1) * 0.5f, height - 2.0f}) {
1233 for (float x : {0.0f, (width - 1) * 0.5f, width - 2.0f}) {
1234 cv::Mat probe(1, 3, CV_32FC2);
1235 probe.at<cv::Vec2f>(0, 0) = field.at(x, y);
1236 probe.at<cv::Vec2f>(0, 1) = field.at(x + 1, y);
1237 probe.at<cv::Vec2f>(0, 2) = field.at(x, y + 1);
1238 const int row = static_cast<int>(y);
1239 const auto centre = pose.ground(probe, row, height, 0);
1240 const cv::Mat& east = centre.first;
1241 const cv::Mat& north = centre.second;
1242 const double gsd = std::min(
1243 std::hypot(east.at<float>(0, 1) - east.at<float>(0, 0),
1244 north.at<float>(0, 1) - north.at<float>(0, 0)),
1245 std::hypot(east.at<float>(0, 2) - east.at<float>(0, 0),
1246 north.at<float>(0, 2) - north.at<float>(0, 0)));
1247 require(std::isfinite(gsd) && gsd > 0, "invalid ground sampling distance");
1248 const cv::Mat ray = probe.colRange(0, 1);
1249 auto previous = pose.ground(ray, row, height, -0.5 * pose.exposure);
1250 double path = 0;
1251 for (int step = 1; step <= steps; ++step) {
1252 auto next = pose.ground(ray, row, height,
1253 (static_cast<double>(step) / steps - 0.5) * pose.exposure);
1254 path += std::hypot(next.first.at<float>(0, 0) - previous.first.at<float>(0, 0),
1255 next.second.at<float>(0, 0) - previous.second.at<float>(0, 0));
1256 previous = std::move(next);
1257 }
1258 blur = std::max(blur, path / gsd);
1259 }
1260 }
1261 int samples = static_cast<int>(
1262 std::min<double>(maximum, std::ceil(blur / temporal_sample_spacing_px))) + 1;
1263 if (samples % 2 == 0) ++samples;
1264 return {std::clamp(samples, 1, maximum), blur};
1265}
1266
1267/** @brief Linear-light to 8-bit sRGB lookup, built once.
1268 * @details 65,536 linear levels keep every round-trip of an 8-bit sRGB code
1269 * exact while replacing a per-pixel pow() call. */
1270const std::array<unsigned char, 65536>& srgb_encoding() {
1271 static const std::array<unsigned char, 65536> table = [] {
1272 auto encode_one = [](size_t index) -> unsigned char {
1273 const double linear = index / 65535.0;
1274 const double encoded = linear <= 0.0031308
1275 ? 12.92 * linear
1276 : 1.055 * std::pow(linear, 1 / 2.4) - 0.055;
1277 return cv::saturate_cast<unsigned char>(encoded * 255);
1278 };
1279 std::array<unsigned char, 65536> values{};
1280 size_t start = 0;
1281 for (int code = 0; code < 255; ++code) {
1282 const double midpoint = (code + 0.5) / 255.0;
1283 const double approx_linear =
1284 midpoint <= 0.04045 ? midpoint / 12.92
1285 : std::pow((midpoint + 0.055) / 1.055, 2.4);
1286 size_t boundary = std::clamp<size_t>(
1287 static_cast<size_t>(approx_linear * 65535.0), start, values.size());
1288 while (boundary > start && encode_one(boundary - 1) > code) --boundary;
1289 while (boundary < values.size() && encode_one(boundary) <= code)
1290 ++boundary;
1291 for (size_t index = start; index < boundary; ++index)
1292 values[index] = static_cast<unsigned char>(code);
1293 start = boundary;
1294 }
1295 for (size_t index = start; index < values.size(); ++index)
1296 values[index] = 255;
1297 return values;
1298 }();
1299 return table;
1300}
1301
1302/** @brief Reflect-101 index clamp matching OpenCV's BORDER_REFLECT_101. */
1303inline int reflect_101(int index, int size) {
1304 if (size <= 1) return 0;
1305 while (index < 0 || index >= size)
1306 index = index < 0 ? -index : 2 * size - 2 - index;
1307 return index;
1308}
1309
1310/** @brief Parallel separable Gaussian blur on a contiguous CV_32FC3 frame. */
1311CAMERA_LENS_SIMD void blur_linear_rgb(cv::Mat& frame, double sigma) {
1312 const int radius = std::max(1, static_cast<int>(std::ceil(sigma * 3.0)));
1313 const int kernel_size = 2 * radius + 1;
1314 std::vector<float> weights(kernel_size);
1315 float weight_sum = 0;
1316 for (int index = -radius; index <= radius; ++index) {
1317 weights[index + radius] = std::exp(
1318 -0.5f * static_cast<float>(index * index) /
1319 static_cast<float>(sigma * sigma));
1320 weight_sum += weights[index + radius];
1321 }
1322 for (float& weight : weights) weight /= weight_sum;
1323 const int width = frame.cols, height = frame.rows;
1324 cv::Mat horizontal(height, width, CV_32FC3);
1325 const float* kernel = weights.data();
1326 const int left_limit = std::min(radius, width),
1327 right_start = std::max(left_limit, width - radius);
1328 cv::parallel_for_(cv::Range(0, height), [&](const cv::Range& range) {
1329 for (int row = range.start; row < range.end; ++row) {
1330 const float* source = frame.ptr<float>(row);
1331 float* destination = horizontal.ptr<float>(row);
1332 for (int column = 0; column < left_limit; ++column) {
1333 float red = 0, green = 0, blue = 0;
1334 for (int tap = -radius; tap <= radius; ++tap) {
1335 const int sample = reflect_101(column + tap, width) * 3;
1336 const float weight = kernel[tap + radius];
1337 red += source[sample] * weight;
1338 green += source[sample + 1] * weight;
1339 blue += source[sample + 2] * weight;
1340 }
1341 destination[column * 3] = red;
1342 destination[column * 3 + 1] = green;
1343 destination[column * 3 + 2] = blue;
1344 }
1345 for (int column = left_limit; column < right_start; ++column) {
1346 float red = 0, green = 0, blue = 0;
1347 const float* window = source + (column - radius) * 3;
1348 for (int tap = 0; tap < kernel_size; ++tap) {
1349 const float weight = kernel[tap];
1350 red += window[tap * 3] * weight;
1351 green += window[tap * 3 + 1] * weight;
1352 blue += window[tap * 3 + 2] * weight;
1353 }
1354 destination[column * 3] = red;
1355 destination[column * 3 + 1] = green;
1356 destination[column * 3 + 2] = blue;
1357 }
1358 for (int column = right_start; column < width; ++column) {
1359 float red = 0, green = 0, blue = 0;
1360 for (int tap = -radius; tap <= radius; ++tap) {
1361 const int sample = reflect_101(column + tap, width) * 3;
1362 const float weight = kernel[tap + radius];
1363 red += source[sample] * weight;
1364 green += source[sample + 1] * weight;
1365 blue += source[sample + 2] * weight;
1366 }
1367 destination[column * 3] = red;
1368 destination[column * 3 + 1] = green;
1369 destination[column * 3 + 2] = blue;
1370 }
1371 }
1372 });
1373 cv::parallel_for_(cv::Range(0, height), [&](const cv::Range& range) {
1374 for (int row = range.start; row < range.end; ++row) {
1375 float* destination = frame.ptr<float>(row);
1376 const int values = width * 3;
1377 for (int index = 0; index < values; ++index) destination[index] = 0;
1378 for (int tap = -radius; tap <= radius; ++tap) {
1379 const float weight = kernel[tap + radius];
1380 const float* source =
1381 horizontal.ptr<float>(reflect_101(row + tap, height));
1382 for (int index = 0; index < values; ++index)
1383 destination[index] += source[index] * weight;
1384 }
1385 }
1386 });
1387}
1388
1389/** @brief Stateless zero-mean, unit-variance Irwin-Hall normal sample.
1390 * @details Keyed directly by (row_seed, channel_index) so noise is strictly
1391 * deterministic, unbiased, and bit-identical across any thread count. */
1392inline float stateless_normal(uint64_t row_seed, int index) {
1393 uint64_t state =
1394 row_seed + static_cast<uint64_t>(index) * 0x9e3779b97f4a7c15ULL;
1395 state = (state ^ (state >> 30)) * 0xbf58476d1ce4e5b9ULL;
1396 state = (state ^ (state >> 27)) * 0x94d049bb133111ebULL;
1397 const uint64_t first = state ^ (state >> 31);
1398 uint64_t second = state * 0x2545f4914f6cdd1dULL;
1399 second ^= second >> 29;
1400 const uint32_t sum =
1401 static_cast<uint32_t>(first & 1023ULL) +
1402 static_cast<uint32_t>((first >> 10) & 1023ULL) +
1403 static_cast<uint32_t>((first >> 20) & 1023ULL) +
1404 static_cast<uint32_t>((first >> 30) & 1023ULL) +
1405 static_cast<uint32_t>((first >> 40) & 1023ULL) +
1406 static_cast<uint32_t>((first >> 50) & 1023ULL) +
1407 static_cast<uint32_t>(second & 1023ULL) +
1408 static_cast<uint32_t>((second >> 10) & 1023ULL) +
1409 static_cast<uint32_t>((second >> 20) & 1023ULL) +
1410 static_cast<uint32_t>((second >> 30) & 1023ULL) +
1411 static_cast<uint32_t>((second >> 40) & 1023ULL) +
1412 static_cast<uint32_t>((second >> 50) & 1023ULL);
1413 return (static_cast<int>(sum) - 6138) * (1.0f / 1024.0f);
1414}
1415
1416/** @brief Parallel shot/read noise + sRGB encoding for a row range. */
1417CAMERA_LENS_SIMD void encode_sensor_rows(const cv::Mat& linear, cv::Mat& output,
1418 int start_row, int end_row,
1419 float well, float read_variance,
1420 uint64_t seed,
1421 const unsigned char* encode) {
1422 const int values_per_row = linear.cols * 3;
1423 for (int row = start_row; row < end_row; ++row) {
1424 const float* values = linear.ptr<float>(row);
1425 unsigned char* destination = output.ptr(row);
1426 if (well > 0) {
1427 const uint64_t row_seed =
1428 seed ^ (static_cast<uint64_t>(row) * 0xd1342543de82ef95ULL);
1429 for (int index = 0; index < values_per_row; ++index) {
1430 const float electrons = std::max(values[index], 0.0f) * well;
1431 const float noisy =
1432 electrons >= well
1433 ? 1.0f
1434 : (electrons + stateless_normal(row_seed, index) *
1435 std::sqrt(electrons + read_variance)) /
1436 well;
1437 destination[index] = encode[static_cast<size_t>(
1438 std::clamp(noisy, 0.0f, 1.0f) * 65535.0f + 0.5f)];
1439 }
1440 } else {
1441 for (int index = 0; index < values_per_row; ++index)
1442 destination[index] = encode[static_cast<size_t>(
1443 std::clamp(values[index], 0.0f, 1.0f) * 65535.0f + 0.5f)];
1444 }
1445 }
1446}
1447
1448/** @brief Parallel luma-only unsharp mask (sigma = 1 px) preserving chroma.
1449 * @details Adding the luma high-pass delta equally to R, G, and B shifts Y
1450 * while leaving YCrCb chroma differences (R - Y, B - Y) unchanged, avoiding
1451 * full-frame colour-space splits and merges. */
1452CAMERA_LENS_SIMD void sharpen_luma_rgb(cv::Mat& output, double sharpen) {
1453 const int width = output.cols, height = output.rows;
1454 cv::Mat luma(height, width, CV_8U), horizontal(height, width, CV_16U);
1455 static constexpr int weights[7] = {5, 55, 248, 408, 248, 55, 5};
1456 const int left_limit = std::min(3, width),
1457 right_start = std::max(left_limit, width - 3);
1458 cv::parallel_for_(cv::Range(0, height), [&](const cv::Range& range) {
1459 for (int row = range.start; row < range.end; ++row) {
1460 const unsigned char* rgb = output.ptr(row);
1461 unsigned char* y = luma.ptr(row);
1462 for (int column = 0; column < width; ++column)
1463 y[column] = static_cast<unsigned char>(
1464 (77 * rgb[column * 3] + 150 * rgb[column * 3 + 1] +
1465 29 * rgb[column * 3 + 2] + 128) >>
1466 8);
1467 uint16_t* filtered = horizontal.ptr<uint16_t>(row);
1468 for (int column = 0; column < left_limit; ++column) {
1469 int sum = 0;
1470 for (int tap = -3; tap <= 3; ++tap)
1471 sum += y[reflect_101(column + tap, width)] * weights[tap + 3];
1472 filtered[column] = static_cast<uint16_t>((sum + 16) >> 5);
1473 }
1474 for (int column = left_limit; column < right_start; ++column) {
1475 const int sum = y[column - 3] * 5 + y[column - 2] * 55 +
1476 y[column - 1] * 248 + y[column] * 408 +
1477 y[column + 1] * 248 + y[column + 2] * 55 +
1478 y[column + 3] * 5;
1479 filtered[column] = static_cast<uint16_t>((sum + 16) >> 5);
1480 }
1481 for (int column = right_start; column < width; ++column) {
1482 int sum = 0;
1483 for (int tap = -3; tap <= 3; ++tap)
1484 sum += y[reflect_101(column + tap, width)] * weights[tap + 3];
1485 filtered[column] = static_cast<uint16_t>((sum + 16) >> 5);
1486 }
1487 }
1488 });
1489 const int amount_q8 = static_cast<int>(std::nearbyint(sharpen * 256.0));
1490 cv::parallel_for_(cv::Range(0, height), [&](const cv::Range& range) {
1491 for (int row = range.start; row < range.end; ++row) {
1492 const uint16_t* taps[7];
1493 for (int tap = -3; tap <= 3; ++tap)
1494 taps[tap + 3] = horizontal.ptr<uint16_t>(reflect_101(row + tap, height));
1495 const unsigned char* y = luma.ptr(row);
1496 unsigned char* rgb = output.ptr(row);
1497 for (int column = 0; column < width; ++column) {
1498 const int blurred_q5 =
1499 (taps[0][column] * 5 + taps[1][column] * 55 +
1500 taps[2][column] * 248 + taps[3][column] * 408 +
1501 taps[4][column] * 248 + taps[5][column] * 55 +
1502 taps[6][column] * 5 + 512) >>
1503 10;
1504 const int delta =
1505 (((static_cast<int>(y[column]) << 5) - blurred_q5) * amount_q8 +
1506 4096) >>
1507 13;
1508 rgb[column * 3] = cv::saturate_cast<unsigned char>(rgb[column * 3] + delta);
1509 rgb[column * 3 + 1] =
1510 cv::saturate_cast<unsigned char>(rgb[column * 3 + 1] + delta);
1511 rgb[column * 3 + 2] =
1512 cv::saturate_cast<unsigned char>(rgb[column * 3 + 2] + delta);
1513 }
1514 }
1515 });
1516}
1517
1518/** Sensor and ISP stage applied after optics and exposure integration.
1519 * @details Order follows a real capture: optical PSF on linear light, photon
1520 * shot and read noise on the integrated signal with full-well clipping,
1521 * sRGB encoding, then ISP luma sharpening before JPEG compression. The
1522 * parameters are effective values that reproduce the saved JPEG, so they
1523 * include the camera ISP's denoising; calibrate them with --measure. */
1526 uint64_t seed;
1527 /** @brief Read effect settings and derive a reproducible per-shot seed. */
1529 : blur(profile["optical_blur_px"].get<double>()),
1530 full_well(profile["sensor_full_well_e"].get<double>()),
1531 read_noise(profile["sensor_read_noise_e"].get<double>()),
1532 sharpen(profile["isp_sharpen"].get<double>()),
1533 seed(0) {
1534 Json noise_profile = profile;
1535 noise_profile.set("threads", nullptr);
1536 noise_profile.set("memory_budget_mb", nullptr);
1537 const std::string identity = noise_profile.dump_sorted() + shot.dump_sorted();
1538 std::array<char, SHA256_HEX_SIZE> hash{};
1539 sha256_hex(identity.data(), identity.size(), hash.data());
1540 seed = std::stoull(std::string(hash.data(), 16), nullptr, 16);
1541 }
1542 /** @brief Extra full-frame bytes per pixel used by enabled effects. */
1543 size_t bytes_per_pixel() const {
1544 return (blur > 0 ? 12 : 0) + (sharpen > 0 ? 3 : 0);
1545 }
1546 /** @brief Develop a linear RGB frame into the saved 8-bit RGB image. */
1547 cv::Mat develop(cv::Mat& linear) const {
1548 if (blur > 0) blur_linear_rgb(linear, blur);
1549 cv::Mat output(linear.size(), CV_8UC3);
1550 const unsigned char* encode = srgb_encoding().data();
1551 const float well = static_cast<float>(full_well),
1552 read_variance = static_cast<float>(read_noise * read_noise);
1553 cv::parallel_for_(cv::Range(0, linear.rows), [&](const cv::Range& range) {
1554 encode_sensor_rows(linear, output, range.start, range.end, well,
1555 read_variance, seed, encode);
1556 });
1558 return output;
1559 }
1560};
1561
1562/** Precomputed row-and-sample ground projection (east/north in metres). */
1569
1570/** Precomputed Clean-JPEG pinhole reprojection for cache-free renders. */
1578
1579/** @brief Bilinearly sample one channel from a 2D pointer grid of 256x256
1580 * linear float32 RGB tiles without assembling a strip mosaic. */
1581inline float sample_tile_channel(const float* const* tile_ptrs, int grid_width,
1582 int max_x, int max_y, float map_x, float map_y,
1583 int channel) {
1584 const int sx = std::clamp(static_cast<int>(std::floor(map_x * 32.0f + 0.5f)),
1585 0, max_x * 32);
1586 const int sy = std::clamp(static_cast<int>(std::floor(map_y * 32.0f + 0.5f)),
1587 0, max_y * 32);
1588 const int ix = sx >> 5, iy = sy >> 5;
1589 const float fx = static_cast<float>(sx & 31) * (1.0f / 32.0f),
1590 fy = static_cast<float>(sy & 31) * (1.0f / 32.0f);
1591 const int tx = ix >> 8, ty = iy >> 8, lx = ix & 255, ly = iy & 255;
1592 if (lx < 255 && ly < 255) {
1593 const float* top =
1594 tile_ptrs[ty * grid_width + tx] + (ly * 256 + lx) * 3 + channel;
1595 const float* bottom = top + 256 * 3;
1596 const float upper = top[0] + fx * (top[3] - top[0]);
1597 const float lower = bottom[0] + fx * (bottom[3] - bottom[0]);
1598 return upper + fy * (lower - upper);
1599 }
1600 const int ix1 = std::min(ix + 1, max_x), iy1 = std::min(iy + 1, max_y);
1601 const float v00 = tile_ptrs[(iy >> 8) * grid_width + (ix >> 8)]
1602 [((iy & 255) * 256 + (ix & 255)) * 3 + channel];
1603 const float v01 = tile_ptrs[(iy >> 8) * grid_width + (ix1 >> 8)]
1604 [((iy & 255) * 256 + (ix1 & 255)) * 3 + channel];
1605 const float v10 = tile_ptrs[(iy1 >> 8) * grid_width + (ix >> 8)]
1606 [((iy1 & 255) * 256 + (ix & 255)) * 3 + channel];
1607 const float v11 = tile_ptrs[(iy1 >> 8) * grid_width + (ix1 >> 8)]
1608 [((iy1 & 255) * 256 + (ix1 & 255)) * 3 + channel];
1609 const float upper = v00 + fx * (v01 - v00);
1610 const float lower = v10 + fx * (v11 - v10);
1611 return upper + fy * (lower - upper);
1612}
1613
1614/** @brief Bilinearly sample all 3 RGB channels at identical coordinates from a
1615 * 2D pointer grid of 256x256 linear float32 tiles. */
1616inline void sample_tile_rgb(const float* const* tile_ptrs, int grid_width,
1617 int max_x, int max_y, float map_x, float map_y,
1618 float& red, float& green, float& blue) {
1619 const int sx = std::clamp(static_cast<int>(std::floor(map_x * 32.0f + 0.5f)),
1620 0, max_x * 32);
1621 const int sy = std::clamp(static_cast<int>(std::floor(map_y * 32.0f + 0.5f)),
1622 0, max_y * 32);
1623 const int ix = sx >> 5, iy = sy >> 5;
1624 const float fx = static_cast<float>(sx & 31) * (1.0f / 32.0f),
1625 fy = static_cast<float>(sy & 31) * (1.0f / 32.0f);
1626 const int tx = ix >> 8, ty = iy >> 8, lx = ix & 255, ly = iy & 255;
1627 if (lx < 255 && ly < 255) {
1628 const float* top = tile_ptrs[ty * grid_width + tx] + (ly * 256 + lx) * 3;
1629 const float* bottom = top + 256 * 3;
1630 red += (top[0] + fx * (top[3] - top[0])) +
1631 fy * ((bottom[0] + fx * (bottom[3] - bottom[0])) -
1632 (top[0] + fx * (top[3] - top[0])));
1633 green += (top[1] + fx * (top[4] - top[1])) +
1634 fy * ((bottom[1] + fx * (bottom[4] - bottom[1])) -
1635 (top[1] + fx * (top[4] - top[1])));
1636 blue += (top[2] + fx * (top[5] - top[2])) +
1637 fy * ((bottom[2] + fx * (bottom[5] - bottom[2])) -
1638 (top[2] + fx * (top[5] - top[2])));
1639 return;
1640 }
1641 const int ix1 = std::min(ix + 1, max_x), iy1 = std::min(iy + 1, max_y);
1642 const float* p00 =
1643 tile_ptrs[(iy >> 8) * grid_width + (ix >> 8)] +
1644 ((iy & 255) * 256 + (ix & 255)) * 3;
1645 const float* p01 =
1646 tile_ptrs[(iy >> 8) * grid_width + (ix1 >> 8)] +
1647 ((iy & 255) * 256 + (ix1 & 255)) * 3;
1648 const float* p10 =
1649 tile_ptrs[(iy1 >> 8) * grid_width + (ix >> 8)] +
1650 ((iy1 & 255) * 256 + (ix & 255)) * 3;
1651 const float* p11 =
1652 tile_ptrs[(iy1 >> 8) * grid_width + (ix1 >> 8)] +
1653 ((iy1 & 255) * 256 + (ix1 & 255)) * 3;
1654 red += (p00[0] + fx * (p01[0] - p00[0])) +
1655 fy * ((p10[0] + fx * (p11[0] - p10[0])) -
1656 (p00[0] + fx * (p01[0] - p00[0])));
1657 green += (p00[1] + fx * (p01[1] - p00[1])) +
1658 fy * ((p10[1] + fx * (p11[1] - p10[1])) -
1659 (p00[1] + fx * (p01[1] - p00[1])));
1660 blue += (p00[2] + fx * (p01[2] - p00[2])) +
1661 fy * ((p10[2] + fx * (p11[2] - p10[2])) -
1662 (p00[2] + fx * (p01[2] - p00[2])));
1663}
1664
1665/** @brief Bilinearly sample a linear CV_32FC3 pinhole source image matching
1666 * OpenCV INTER_LINEAR + BORDER_CONSTANT(0). */
1667inline float sample_pinhole_channel(const cv::Mat& source, float map_x,
1668 float map_y, int channel) {
1669 const int sx = static_cast<int>(std::floor(map_x * 32.0f + 0.5f)),
1670 sy = static_cast<int>(std::floor(map_y * 32.0f + 0.5f));
1671 const int ix = sx >> 5, iy = sy >> 5;
1672 const float fx = static_cast<float>(sx & 31) * (1.0f / 32.0f),
1673 fy = static_cast<float>(sy & 31) * (1.0f / 32.0f);
1674 if (ix >= 0 && ix + 1 < source.cols && iy >= 0 && iy + 1 < source.rows) {
1675 const float* top = source.ptr<float>(iy) + ix * 3 + channel;
1676 const float* bottom = source.ptr<float>(iy + 1) + ix * 3 + channel;
1677 const float upper = top[0] + fx * (top[3] - top[0]);
1678 const float lower = bottom[0] + fx * (bottom[3] - bottom[0]);
1679 return upper + fy * (lower - upper);
1680 }
1681 auto fetch = [&](int x, int y) -> float {
1682 if (x < 0 || x >= source.cols || y < 0 || y >= source.rows) return 0.0f;
1683 return source.ptr<float>(y)[x * 3 + channel];
1684 };
1685 const float v00 = fetch(ix, iy),
1686 v01 = fx > 0.0f ? fetch(ix + 1, iy) : 0.0f,
1687 v10 = fy > 0.0f ? fetch(ix, iy + 1) : 0.0f,
1688 v11 = (fx > 0.0f && fy > 0.0f) ? fetch(ix + 1, iy + 1) : 0.0f;
1689 const float upper = v00 + fx * (v01 - v00);
1690 const float lower = v10 + fx * (v11 - v10);
1691 return upper + fy * (lower - upper);
1692}
1693
1694/** @brief SIMD-friendly strip ground-bounds and horizon check. */
1696 const std::vector<std::unique_ptr<RayField>>& fields, int channels,
1697 const RowSamplePose* poses, int samples, int first, int start_row,
1698 int end_row, double* min_e, double* max_e, double* min_n, double* max_n,
1699 std::atomic<bool>& horizon, std::atomic<bool>& non_finite) {
1700 int max_nodes = 0;
1701 for (int channel = 0; channel < channels; ++channel)
1702 max_nodes = std::max(max_nodes, fields[channel]->node_columns());
1703 std::vector<float> rx_row(max_nodes), ry_row(max_nodes);
1704 for (int r = start_row; r < end_row; ++r) {
1705 if (horizon.load(std::memory_order_relaxed) ||
1706 non_finite.load(std::memory_order_relaxed))
1707 return;
1708 const int full_row = first + r;
1709 const RowSamplePose* row_poses =
1710 poses + static_cast<size_t>(full_row) * samples;
1711 float re_min = INFINITY, re_max = -INFINITY, rn_min = INFINITY,
1712 rn_max = -INFINITY;
1713 for (int channel = 0; channel < channels; ++channel) {
1714 const int count = fields[channel]->node_columns();
1715 fields[channel]->row_nodes_into(full_row, rx_row.data(), ry_row.data());
1716 const float* rx = rx_row.data();
1717 const float* ry = ry_row.data();
1718 for (int sample = 0; sample < samples; ++sample) {
1719 const RowSamplePose& p = row_poses[sample];
1720 float min_down = INFINITY;
1721 for (int column = 0; column < count; ++column)
1722 min_down =
1723 std::min(min_down, p.down_x * rx[column] + p.down_y * ry[column] +
1724 p.down_z);
1725 if (!(min_down > 1e-4f)) {
1726 horizon.store(true, std::memory_order_relaxed);
1727 return;
1728 }
1729 for (int column = 0; column < count; ++column) {
1730 const float scale =
1731 1.0f / (p.down_x * rx[column] + p.down_y * ry[column] + p.down_z);
1732 const float east =
1733 (p.east_x * rx[column] + p.east_y * ry[column] + p.east_z) *
1734 scale +
1736 const float north =
1737 (p.north_x * rx[column] + p.north_y * ry[column] + p.north_z) *
1738 scale +
1740 re_min = east < re_min ? east : re_min;
1741 re_max = east > re_max ? east : re_max;
1742 rn_min = north < rn_min ? north : rn_min;
1743 rn_max = north > rn_max ? north : rn_max;
1744 }
1745 }
1746 }
1747 if (!std::isfinite(re_min) || !std::isfinite(re_max) ||
1748 !std::isfinite(rn_min) || !std::isfinite(rn_max)) {
1749 non_finite.store(true, std::memory_order_relaxed);
1750 return;
1751 }
1752 min_e[r] = re_min;
1753 max_e[r] = re_max;
1754 min_n[r] = rn_min;
1755 max_n[r] = rn_max;
1756 }
1757}
1758
1759/** @brief Fused parallel strip kernel for map-cache rendering. */
1761 const std::vector<std::unique_ptr<RayField>>& fields, int channels,
1762 const RowSamplePose* poses, int samples, int first, int start_row,
1763 int end_row, int width, float inv_map_scale, float base_offset_x,
1764 float base_offset_y, const float* const* tile_ptrs, int grid_width,
1765 int grid_height, float gain, float vignette_scale, cv::Mat& linear) {
1766 const int max_x = grid_width * 256 - 1, max_y = grid_height * 256 - 1;
1767 std::vector<cv::Vec2f> row_rays_0(width), row_rays_1, row_rays_2;
1768 if (channels == 3) {
1769 row_rays_1.resize(width);
1770 row_rays_2.resize(width);
1771 }
1772 for (int row = start_row; row < end_row; ++row) {
1773 const int full_row = first + row;
1774 const RowSamplePose* row_poses =
1775 poses + static_cast<size_t>(full_row) * samples;
1776 float* destination = linear.ptr<float>(full_row);
1777 fields[0]->row_into(full_row, row_rays_0.data());
1778 if (channels == 1) {
1779 for (int column = 0; column < width; ++column) {
1780 const float rx = row_rays_0[column][0], ry = row_rays_0[column][1];
1781 float red = 0, green = 0, blue = 0;
1782 for (int sample = 0; sample < samples; ++sample) {
1783 const RowSamplePose& p = row_poses[sample];
1784 const float scale =
1785 1.0f / (p.down_x * rx + p.down_y * ry + p.down_z);
1786 const float east =
1787 (p.east_x * rx + p.east_y * ry + p.east_z) * scale +
1789 const float north =
1790 (p.north_x * rx + p.north_y * ry + p.north_z) * scale +
1792 const float mx = base_offset_x + east * inv_map_scale;
1793 const float my = base_offset_y - north * inv_map_scale;
1794 sample_tile_rgb(tile_ptrs, grid_width, max_x, max_y, mx, my, red,
1795 green, blue);
1796 }
1797 const float cos2 = 1.0f / (1.0f + rx * rx + ry * ry);
1798 const float factor =
1799 gain * (1.0f - vignette_scale * (1.0f - cos2 * cos2));
1800 destination[column * 3] = red * factor;
1801 destination[column * 3 + 1] = green * factor;
1802 destination[column * 3 + 2] = blue * factor;
1803 }
1804 } else {
1805 fields[1]->row_into(full_row, row_rays_1.data());
1806 fields[2]->row_into(full_row, row_rays_2.data());
1807 const cv::Vec2f* channel_rays[3] = {row_rays_0.data(), row_rays_1.data(),
1808 row_rays_2.data()};
1809 for (int column = 0; column < width; ++column) {
1810 for (int channel = 0; channel < 3; ++channel) {
1811 const float rx = channel_rays[channel][column][0],
1812 ry = channel_rays[channel][column][1];
1813 float sum = 0;
1814 for (int sample = 0; sample < samples; ++sample) {
1815 const RowSamplePose& p = row_poses[sample];
1816 const float scale =
1817 1.0f / (p.down_x * rx + p.down_y * ry + p.down_z);
1818 const float east =
1819 (p.east_x * rx + p.east_y * ry + p.east_z) * scale +
1821 const float north =
1822 (p.north_x * rx + p.north_y * ry + p.north_z) * scale +
1824 const float mx = base_offset_x + east * inv_map_scale;
1825 const float my = base_offset_y - north * inv_map_scale;
1826 sum += sample_tile_channel(tile_ptrs, grid_width, max_x, max_y, mx,
1827 my, channel);
1828 }
1829 const float cos2 = 1.0f / (1.0f + rx * rx + ry * ry);
1830 destination[column * 3 + channel] =
1831 sum * (gain * (1.0f - vignette_scale * (1.0f - cos2 * cos2)));
1832 }
1833 }
1834 }
1835 }
1836}
1837
1838/** Render bounded image strips and return the assumptions/resource evidence
1839 * used.
1840 * @param input Unmodified clean RGB source (may be empty when @p cache_path is
1841 * set and @p source_size supplies the input dimensions).
1842 * @param shot Validated flight pose.
1843 * @param geometry Source-camera geometry.
1844 * @param profile Validated lens/effect parameters.
1845 * @param cache_path Optional offline tile-cache directory.
1846 * @param resources Active worker resource policy.
1847 * @param source_size Optional clean-input dimensions when @p input was
1848 * header-probed without scanline decompression.
1849 * @return Corrected pixels and quantitative provenance fields. */
1850Rendered render(const cv::Mat& input, const Json& shot, const Json& geometry,
1851 const Json& profile, const std::string& cache_path,
1852 const Resources& resources, cv::Size source_size = {}) {
1853 const int source_width = input.empty() ? source_size.width : input.cols;
1854 const int source_height = input.empty() ? source_size.height : input.rows;
1855 const int width = profile["output_width"].is_null()
1856 ? source_width
1857 : profile["output_width"].get<int>();
1858 const int height = profile["output_height"].is_null()
1859 ? source_height
1860 : profile["output_height"].get<int>();
1861 const Lens lens(width, height, profile);
1862 const Pose pose(shot, profile);
1863 const bool cache = !cache_path.empty();
1864 require(cache || !input.empty(), "missing decoded clean input");
1865 require(cache || profile["height_override_m"].is_null(),
1866 "height override requires a map cache");
1867 const double aberration = profile["chromatic_aberration"].get<double>();
1868 const int channels = aberration == 0 ? 1 : 3;
1869 const Development development(profile, shot);
1870 const size_t input_pixels =
1871 static_cast<size_t>(source_width) * static_cast<size_t>(source_height);
1872 const size_t frame_bytes =
1873 32 * mib + input_pixels * (cache ? (input.empty() ? 0 : 3) : 15) +
1874 static_cast<size_t>(width) * height *
1875 (3 + 12 + development.bytes_per_pixel());
1876 require(frame_bytes <= resources.budget,
1877 "lens working set exceeds memory budget");
1879 cv::setNumThreads(resources.threads);
1880 std::vector<std::unique_ptr<RayField>> fields(channels);
1881 size_t field_bytes = 0;
1882 if (channels == 1) {
1883 require(field_bytes < resources.budget - frame_bytes,
1884 "ray grid exceeds memory budget");
1885 fields[0] = std::make_unique<RayField>(lens, width, height, 1.0,
1886 resources.budget - frame_bytes);
1887 field_bytes = fields[0]->bytes();
1888 } else {
1889 const size_t per_field_budget = (resources.budget - frame_bytes) / 3;
1890 std::atomic<bool> failed{false};
1891 std::string failure;
1892 cv::parallel_for_(cv::Range(0, 3), [&](const cv::Range& range) {
1893 for (int channel = range.start; channel < range.end; ++channel) {
1894 try {
1895 const double scale =
1896 channel == 0 ? 1 + aberration : (channel == 2 ? 1 - aberration : 1);
1897 fields[channel] = std::make_unique<RayField>(
1898 lens, width, height, scale, per_field_budget);
1899 } catch (const std::exception& error) {
1900 if (!failed.exchange(true)) failure = error.what();
1901 }
1902 }
1903 });
1904 if (failed) throw std::runtime_error(failure);
1905 for (const auto& field : fields) field_bytes += field->bytes();
1906 }
1907 const RayField& reference = *fields[channels == 1 ? 0 : 1];
1908 const TemporalSampling sampling = temporal_sampling(
1909 reference, pose, width, height, profile["samples"].get<int>());
1910 const int samples = sampling.samples;
1911 const int strip_rows = std::min(32, std::max(1, 524288 / (width * samples)));
1912 const size_t pose_bytes =
1913 static_cast<size_t>(height) * samples * sizeof(RowSamplePose);
1914 const size_t base_bytes =
1915 frame_bytes + field_bytes + pose_bytes +
1916 static_cast<size_t>(resources.threads) * width * channels * sizeof(cv::Vec2f);
1917 require(base_bytes <= resources.budget,
1918 "lens working set exceeds memory budget");
1919 size_t peak_bytes = base_bytes;
1920
1921 // Precompute time-dependent camera rotation and ground translation once
1922 // across (row, sample); all chromatic-aberration channels share this table.
1923 std::vector<RowSamplePose> poses(static_cast<size_t>(height) * samples);
1924 const float altitude_f = static_cast<float>(pose.altitude);
1925 const float cos_course = static_cast<float>(std::cos(pose.course)),
1926 sin_course = static_cast<float>(std::sin(pose.course));
1927 for (int row = 0; row < height; ++row) {
1928 for (int sample = 0; sample < samples; ++sample) {
1929 const double sample_offset =
1930 ((sample + 0.5) / samples - 0.5) * pose.exposure;
1931 const float instant =
1932 static_cast<float>(
1933 (static_cast<double>(row) / (height - 1) - 0.5) * pose.readout) +
1934 static_cast<float>(sample_offset);
1935 const double cycle = 2 * pi * pose.frequency * instant + pose.phase;
1936 const double roll = (pose.angles[0] + pose.rates[0] * instant +
1937 pose.vibration * std::sin(cycle)) *
1938 pi / 180;
1939 const double pitch = (pose.angles[1] + pose.rates[1] * instant +
1940 pose.vibration * std::sin(cycle + pi / 2)) *
1941 pi / 180;
1942 const double yaw = (pose.angles[2] + pose.rates[2] * instant) * pi / 180;
1943 const float cr = std::cos(roll), sr = std::sin(roll),
1944 cp = std::cos(pitch), sp = std::sin(pitch),
1945 cy = std::cos(yaw), sy = std::sin(yaw);
1946 const float nf = cy * cp, nr = cy * sp * sr - sy * cr,
1947 nd = cy * sp * cr + sy * sr;
1948 const float ef = sy * cp, er = sy * sp * sr + cy * cr,
1949 ed = sy * sp * cr - cy * sr;
1950 RowSamplePose& entry = poses[static_cast<size_t>(row) * samples + sample];
1951 // forward = -ray_y, right = ray_x
1952 entry.east_x = er * altitude_f;
1953 entry.east_y = -ef * altitude_f;
1954 entry.east_z = ed * altitude_f;
1955 entry.north_x = nr * altitude_f;
1956 entry.north_y = -nf * altitude_f;
1957 entry.north_z = nd * altitude_f;
1958 entry.down_x = cp * sr;
1959 entry.down_y = sp;
1960 entry.down_z = cp * cr;
1961 entry.translation_north = static_cast<float>(pose.speed) * instant * cos_course;
1962 entry.translation_east = static_cast<float>(pose.speed) * instant * sin_course;
1963 }
1964 }
1965
1966 cv::Mat linear(height, width, CV_32FC3);
1967 // Residual relative illumination after ISP shading correction: cos^4
1968 // falloff shape, scaled so the image corner loses exactly `vignetting`.
1969 const cv::Vec2f corner_ray = reference.at(0, 0);
1970 const double corner_cos2 = 1 / (1 + corner_ray.dot(corner_ray));
1971 const double corner_falloff = 1 - corner_cos2 * corner_cos2;
1972 const float vignette_scale =
1973 corner_falloff > 1e-9
1974 ? static_cast<float>(profile["vignetting"].get<double>() / corner_falloff)
1975 : 0.0f;
1976 const float gain =
1977 static_cast<float>(std::pow(2.0, profile["exposure_ev"].get<double>()) / samples);
1978
1979 Bounds bounds{};
1980 bool have_bounds = false;
1981 if (cache) {
1982 const double latitude = shot["lat_deg"].get<double>() * pi / 180;
1983 const double world = 256.0 * (1 << 18);
1984 const double center_x = (shot["lon_deg"].get<double>() + 180) / 360 * world;
1985 const double center_y =
1986 (1 - std::asinh(std::tan(latitude)) / pi) * world / 2;
1987 const double map_scale = 2 * pi * 6378137 * std::cos(latitude) / world;
1988 const float inv_map_scale = static_cast<float>(1.0 / map_scale);
1989 const int64_t base_tile_x = static_cast<int64_t>(std::floor(center_x / 256.0));
1990 const int64_t base_tile_y = static_cast<int64_t>(std::floor(center_y / 256.0));
1991 const float local_center_x = static_cast<float>(center_x - base_tile_x * 256.0);
1992 const float local_center_y = static_cast<float>(center_y - base_tile_y * 256.0);
1993 const int max_tiles = profile["max_tiles"].get<int>();
1994 TileCache tiles(cache_path, resources.budget);
1995
1996 for (int first = 0; first < height; first += strip_rows) {
1997 const int rows = std::min(strip_rows, height - first);
1998 std::vector<double> min_e(rows, INFINITY), max_e(rows, -INFINITY),
1999 min_n(rows, INFINITY), max_n(rows, -INFINITY);
2000 std::atomic<bool> horizon{false}, non_finite{false};
2001 cv::parallel_for_(cv::Range(0, rows), [&](const cv::Range& range) {
2002 compute_strip_bounds_rows(fields, channels, poses.data(), samples,
2003 first, range.start, range.end, min_e.data(),
2004 max_e.data(), min_n.data(), max_n.data(),
2005 horizon, non_finite);
2006 });
2007 require(!horizon, "lens/exposure rays reach the horizon");
2008 require(!non_finite, "non-finite ground projection");
2009 double minimum_east = INFINITY, maximum_east = -INFINITY,
2010 minimum_north = INFINITY, maximum_north = -INFINITY;
2011 for (int r = 0; r < rows; ++r) {
2012 minimum_east = std::min(minimum_east, min_e[r]);
2013 maximum_east = std::max(maximum_east, max_e[r]);
2014 minimum_north = std::min(minimum_north, min_n[r]);
2015 maximum_north = std::max(maximum_north, max_n[r]);
2016 }
2017 const Bounds region = {
2018 static_cast<int64_t>(
2019 std::floor((center_x + minimum_east / map_scale) / 256)),
2020 static_cast<int64_t>(
2021 std::floor((center_y - maximum_north / map_scale) / 256)),
2022 static_cast<int64_t>(
2023 std::floor((center_x + maximum_east / map_scale + 1) / 256)),
2024 static_cast<int64_t>(
2025 std::floor((center_y - minimum_north / map_scale + 1) / 256))};
2026 bounds = have_bounds ? Bounds{std::min(bounds.left, region.left),
2027 std::min(bounds.top, region.top),
2028 std::max(bounds.right, region.right),
2029 std::max(bounds.bottom, region.bottom)}
2030 : region;
2031 have_bounds = true;
2032 const size_t total_tile_count = bounds.count(max_tiles);
2033 const size_t strip_tile_count = region.count(max_tiles);
2034 const size_t resident_tiles =
2035 std::max(strip_tile_count,
2036 std::min(total_tile_count, tile_cache_entries));
2037 const size_t required =
2038 base_bytes + resident_tiles * 256 * 256 * sizeof(cv::Vec3f) +
2039 strip_tile_count * (sizeof(cv::Mat) + sizeof(const float*));
2040 require(required <= resources.budget,
2041 "lens tile strip exceeds memory budget");
2042 peak_bytes = std::max(peak_bytes, required);
2043
2044 const int grid_width = static_cast<int>(region.right - region.left + 1);
2045 const int grid_height = static_cast<int>(region.bottom - region.top + 1);
2046 std::vector<cv::Mat> strip_tiles(
2047 static_cast<size_t>(grid_width) * grid_height);
2048 std::vector<const float*> tile_ptrs(strip_tiles.size());
2049 for (int ty = 0; ty < grid_height; ++ty) {
2050 for (int tx = 0; tx < grid_width; ++tx) {
2051 const size_t slot = static_cast<size_t>(ty) * grid_width + tx;
2052 strip_tiles[slot] = tiles.tile(region.left + tx, region.top + ty);
2053 tile_ptrs[slot] = strip_tiles[slot].ptr<float>();
2054 }
2055 }
2056 const float base_offset_x =
2057 local_center_x + static_cast<float>((base_tile_x - region.left) * 256);
2058 const float base_offset_y =
2059 local_center_y + static_cast<float>((base_tile_y - region.top) * 256);
2060 cv::parallel_for_(cv::Range(0, rows), [&](const cv::Range& range) {
2061 sample_cache_rows(fields, channels, poses.data(), samples, first,
2062 range.start, range.end, width, inv_map_scale,
2063 base_offset_x, base_offset_y, tile_ptrs.data(),
2064 grid_width, grid_height, gain, vignette_scale,
2065 linear);
2066 });
2067 }
2068 for (int64_t row = bounds.top; row <= bounds.bottom; ++row)
2069 for (int64_t column = bounds.left; column <= bounds.right; ++column)
2070 tiles.verify(column, row);
2071 } else {
2072 cv::Mat linear_source(input.rows, input.cols, CV_32FC3);
2073 const float* decode = srgb_decoding().data();
2074 cv::parallel_for_(cv::Range(0, input.rows), [&](const cv::Range& range) {
2075 for (int row = range.start; row < range.end; ++row) {
2076 const unsigned char* src = input.ptr(row);
2077 float* dst = linear_source.ptr<float>(row);
2078 for (int index = 0; index < input.cols * 3; ++index)
2079 dst[index] = decode[src[index]];
2080 }
2081 });
2082 const double roll = shot["roll_deg"].get<double>() * pi / 180,
2083 pitch = shot["pitch_deg"].get<double>() * pi / 180,
2084 yaw = shot["yaw_deg"].get<double>() * pi / 180,
2085 shot_altitude = shot["groundalt_m"].get<double>();
2086 const double cr = std::cos(roll), sr = std::sin(roll),
2087 cp = std::cos(pitch), sp = std::sin(pitch),
2088 cy = std::cos(yaw), sy = std::sin(yaw);
2089 const SourcePinhole pinhole{
2090 cy * cp,
2091 sy * cp,
2092 -sp * shot_altitude,
2093 cy * sp * sr - sy * cr,
2094 sy * sp * sr + cy * cr,
2095 cp * sr * shot_altitude,
2096 cy * sp * cr + sy * sr,
2097 sy * sp * cr - cy * sr,
2098 cp * cr * shot_altitude,
2099 (input.cols - 1) /
2100 (2 * std::tan(geometry["horizontal_fov_deg"].get<double>() * pi / 360)),
2101 (input.rows - 1) /
2102 (2 * std::tan(geometry["vertical_fov_deg"].get<double>() * pi / 360)),
2103 (input.cols - 1) * 0.5,
2104 (input.rows - 1) * 0.5,
2105 input.cols,
2106 input.rows};
2107
2108 for (int first = 0; first < height; first += strip_rows) {
2109 const int rows = std::min(strip_rows, height - first);
2110 std::atomic<int> error_code{0};
2111 cv::parallel_for_(cv::Range(0, rows), [&](const cv::Range& range) {
2112 std::vector<cv::Vec2f> row_rays_0(width), row_rays_1, row_rays_2;
2113 if (channels == 3) {
2114 row_rays_1.resize(width);
2115 row_rays_2.resize(width);
2116 }
2117 for (int r = range.start; r < range.end; ++r) {
2118 if (error_code.load(std::memory_order_relaxed) != 0) return;
2119 const int full_row = first + r;
2120 const RowSamplePose* row_poses =
2121 poses.data() + static_cast<size_t>(full_row) * samples;
2122 fields[0]->row_into(full_row, row_rays_0.data());
2123 if (channels == 3) {
2124 fields[1]->row_into(full_row, row_rays_1.data());
2125 fields[2]->row_into(full_row, row_rays_2.data());
2126 }
2127 const cv::Vec2f* channel_rays[3] = {
2128 row_rays_0.data(),
2129 channels == 3 ? row_rays_1.data() : row_rays_0.data(),
2130 channels == 3 ? row_rays_2.data() : row_rays_0.data()};
2131 float* destination = linear.ptr<float>(full_row);
2132 for (int column = 0; column < width; ++column) {
2133 for (int c = 0; c < channels; ++c) {
2134 const float rx = channel_rays[c][column][0],
2135 ry = channel_rays[c][column][1];
2136 float acc0 = 0, acc1 = 0, acc2 = 0;
2137 for (int sample = 0; sample < samples; ++sample) {
2138 const RowSamplePose& p = row_poses[sample];
2139 const float down_ray = p.down_x * rx + p.down_y * ry + p.down_z;
2140 if (!(down_ray > 1e-4f)) {
2141 int expected = 0;
2142 error_code.compare_exchange_strong(expected, 1);
2143 return;
2144 }
2145 const float scale = 1.0f / down_ray;
2146 const float east =
2147 (p.east_x * rx + p.east_y * ry + p.east_z) * scale +
2149 const float north =
2150 (p.north_x * rx + p.north_y * ry + p.north_z) * scale +
2152 if (!std::isfinite(east) || !std::isfinite(north)) {
2153 int expected = 0;
2154 error_code.compare_exchange_strong(expected, 2);
2155 return;
2156 }
2157 const double forward = pinhole.f_north * north +
2158 pinhole.f_east * east + pinhole.f_bias;
2159 const double right = pinhole.r_north * north +
2160 pinhole.r_east * east + pinhole.r_bias;
2161 const double down = pinhole.d_north * north +
2162 pinhole.d_east * east + pinhole.d_bias;
2163 if (!(down > 1e-4)) {
2164 int expected = 0;
2165 error_code.compare_exchange_strong(expected, 3);
2166 return;
2167 }
2168 const float mx = static_cast<float>(
2169 right / down * pinhole.focal_x + pinhole.center_x);
2170 const float my = static_cast<float>(
2171 -forward / down * pinhole.focal_y + pinhole.center_y);
2172 if (!(mx >= -0.001f && mx <= pinhole.cols - 1 + 0.001f &&
2173 my >= -0.001f && my <= pinhole.rows - 1 + 0.001f)) {
2174 int expected = 0;
2175 error_code.compare_exchange_strong(expected, 4);
2176 return;
2177 }
2178 if (channels == 1) {
2179 acc0 += sample_pinhole_channel(linear_source, mx, my, 0);
2180 acc1 += sample_pinhole_channel(linear_source, mx, my, 1);
2181 acc2 += sample_pinhole_channel(linear_source, mx, my, 2);
2182 } else {
2183 acc0 += sample_pinhole_channel(linear_source, mx, my, c);
2184 }
2185 }
2186 const float cos2 = 1.0f / (1.0f + rx * rx + ry * ry);
2187 const float factor =
2188 gain * (1.0f - vignette_scale * (1.0f - cos2 * cos2));
2189 if (channels == 1) {
2190 destination[column * 3] = acc0 * factor;
2191 destination[column * 3 + 1] = acc1 * factor;
2192 destination[column * 3 + 2] = acc2 * factor;
2193 } else {
2194 destination[column * 3 + c] = acc0 * factor;
2195 }
2196 }
2197 }
2198 }
2199 });
2200 const int code = error_code.load();
2201 require(code != 1, "lens/exposure rays reach the horizon");
2202 require(code != 2, "non-finite ground projection");
2203 require(code != 3, "view is behind source camera");
2204 require(code != 4, "view exceeds source image; supply --cache");
2205 }
2206 }
2207 const cv::Mat output = development.develop(linear);
2208 const double relative_course =
2209 pose.course - shot["yaw_deg"].get<double>() * pi / 180;
2210 const double blur = std::hypot(lens.focal_x * std::sin(relative_course),
2211 lens.focal_y * std::cos(relative_course)) *
2212 pose.speed * pose.exposure / pose.altitude;
2213 return {
2214 output,
2216 {{"used_agl_m", pose.altitude},
2217 {"shot_agl_m", shot["groundalt_m"]},
2218 {"renderer", "native_opencv_bounded_strips_v2"},
2219 {"effective_threads", resources.threads},
2220 {"effective_samples", samples},
2221 {"configured_max_samples", profile["samples"]},
2222 {"exposure_blur_px_max", sampling.blur_px},
2223 {"ray_grid_step_px", reference.step},
2224 {"ray_interpolation_error_px", reference.error_px},
2225 {"sensor_model", development.full_well > 0
2226 ? "gaussian_shot_and_read_noise"
2227 : "noise_free"},
2228 {"noise_seed", std::to_string(development.seed)},
2229 {"strip_rows", strip_rows},
2230 {"memory_budget_bytes", resources.budget},
2231 {"estimated_peak_bytes", peak_bytes},
2232 {"center_translation_blur_px_level_estimate", blur},
2233 {"source", cache ? "cache" : "pinhole_jpeg"},
2234 {"tile_bounds_unwrapped",
2235 cache ? Json::array(
2236 {bounds.left, bounds.top, bounds.right, bounds.bottom})
2237 : Json(nullptr)},
2238 {"effective_readout_s", std::abs(pose.readout)},
2239 {"motion_model",
2240 "constant_ground_velocity_and_configured_euler_rates"},
2241 {"radiometry", "srgb_linearized_relative_EV_not_sensor_raw"},
2242 {"vibration_model",
2243 "configured_sinusoidal_roll_pitch_not_measured"},
2244 {"time_reference", "shot_pose_at_center_row_mid_exposure"}})};
2245}
2246
2247/** Apply one profile and preserve the clean source while writing a tagged
2248 * derivative.
2249 * @param input Original clean JPEG.
2250 * @param output New derivative path.
2251 * @param profile Validated profile.
2252 * @param cache Optional tile-cache directory.
2253 * @return EXIF evidence plus run-time/resource measurements. */
2254Json emulate(const std::string& input, const std::string& output,
2255 const Json& profile, const std::string& cache) {
2256 struct stat original{}, destination{};
2257 require(stat(input.c_str(), &original) == 0, "missing clean input");
2258 require(stat(output.c_str(), &destination) != 0 ||
2259 original.st_dev != destination.st_dev ||
2260 original.st_ino != destination.st_ino,
2261 "output must differ from clean source");
2262 const auto started = std::chrono::steady_clock::now();
2264 JpegHeader source_header{};
2265 cv::Mat image = read_jpeg(input, false, resources.budget, &source_header,
2266 cache.empty());
2267 auto metadata = std::make_unique<exif_document>();
2268 std::array<unsigned char, EXIF_MAX_PAYLOAD> payload;
2269 size_t payload_size = 0;
2270 exif_view view{};
2271 require(exif_read_jpeg(input.c_str(), payload.data(), payload.size(),
2272 &payload_size) == 0 &&
2273 exif_parse(&view, payload.data(), payload_size) == 0 &&
2274 image_exif_import(metadata.get(), &view) == 0,
2275 "missing or invalid EXIF metadata");
2276 Json geometry;
2277 const Json shot = read_shot(metadata.get(), geometry);
2279 render(image, shot, geometry, profile, cache, resources,
2280 cv::Size(source_header.width, source_header.height));
2281 const std::string serialized = profile.dump_sorted();
2282 std::array<char, SHA256_HEX_SIZE> hash{};
2283 sha256_hex(serialized.data(), serialized.size(), hash.data());
2284 Json record = Json::object({{"schema", "USHER_LENS_V1"}, {"simulated", true},
2285 {"profile", profile}, {"profile_sha256", hash.data()},
2286 {"source_geometry", geometry}, {"opencv", CV_VERSION}});
2287 record.update(result.evidence);
2288 const std::string record_text = record.dump(-1, ' ', true);
2289 set_text(metadata.get(), EXIF_PRIMARY, EXIF_DOCUMENT_NAME, record_text);
2290 set_text(metadata.get(), EXIF_PRIMARY, EXIF_SOFTWARE,
2291 "Paparazzi USHER native simulated lens / OpenCV");
2292 set_text(metadata.get(), EXIF_CAMERA, EXIF_LENS_MODEL,
2293 "SIMULATED " + profile["name"].get<std::string>());
2294 set_rational(metadata.get(), EXIF_EXPOSURE_TIME, profile["exposure_s"].get<double>(),
2295 1000000);
2296 set_rational(metadata.get(), EXIF_FNUMBER, profile["f_number"].get<double>(), 100);
2297 set_rational(metadata.get(), EXIF_EXPOSURE_BIAS,
2298 profile["exposure_ev"].get<double>(), 1000, true);
2299 if (!profile["focal_length_mm"].is_null())
2300 set_rational(metadata.get(), EXIF_FOCAL_LENGTH,
2301 profile["focal_length_mm"].get<double>(), 1000);
2302 else {
2304 "cannot remove focal length");
2305 }
2306 for (const auto& dimension :
2307 {std::make_pair(EXIF_PIXEL_X_DIMENSION, result.image.cols),
2308 std::make_pair(EXIF_PIXEL_Y_DIMENSION, result.image.rows)}) {
2309 unsigned char* entry = replace_tag(metadata.get(), EXIF_CAMERA,
2310 dimension.first, EXIF_LONG, 1);
2311 exif_encode_long(entry, static_cast<uint32_t>(dimension.second));
2312 }
2313 std::string comment =
2314 tag_text(metadata.get(), EXIF_CAMERA, EXIF_USER_COMMENT);
2315 const std::string prefix("ASCII\0\0\0", 8);
2316 if (comment.empty()) comment = prefix;
2317 require(comment.compare(0, prefix.size(), prefix) == 0,
2318 "expected ASCII USHER UserComment");
2319 set_text(metadata.get(), EXIF_CAMERA, EXIF_USER_COMMENT,
2320 comment + "\nUSHER_LENS_V1\n" + record_text, true);
2321 write_jpeg(output, result.image, profile["jpeg_quality"].get<int>(), metadata.get());
2322 record.set("elapsed_s",
2323 std::chrono::duration<double>(std::chrono::steady_clock::now() - started)
2324 .count());
2325 std::ifstream status("/proc/self/status");
2326 std::string line;
2327 while (std::getline(status, line)) {
2328 std::istringstream fields(line);
2329 std::string key;
2330 size_t value;
2331 if (fields >> key >> value && key == "VmHWM:")
2332 record.set("process_peak_rss_kib", value);
2333 }
2334 record.set("output", output);
2335 return record;
2336}
2337} // namespace camera_lens_detail
2338
2339using namespace camera_lens_detail;
2340
2341/** @brief Estimate the IJG quality whose scaled luma table best matches.
2342 * @param quant Natural-order luma quantization table.
2343 * @return {quality, mean absolute table error}; error 0 is an exact IJG
2344 * table, larger values mean a vendor table and an approximate quality. */
2345std::pair<int, double> ijg_quality(const std::array<unsigned, 64>& quant) {
2346 static const std::array<int, 64> standard = {
2347 16, 11, 10, 16, 24, 40, 51, 61, 12, 12, 14, 19, 26, 58, 60, 55,
2348 14, 13, 16, 24, 40, 57, 69, 56, 14, 17, 22, 29, 51, 87, 80, 62,
2349 18, 22, 37, 56, 68, 109, 103, 77, 24, 35, 55, 64, 81, 104, 113, 92,
2350 49, 64, 78, 87, 103, 121, 120, 101, 72, 92, 95, 98, 112, 100, 103, 99};
2351 std::pair<int, double> best{0, INFINITY};
2352 for (int quality = 1; quality <= 100; ++quality) {
2353 const int scale = quality < 50 ? 5000 / quality : 200 - quality * 2;
2354 double error = 0;
2355 for (size_t index = 0; index < standard.size(); ++index)
2356 error += std::abs(
2357 std::clamp((standard[index] * scale + 50) / 100, 1, 255) -
2358 static_cast<int>(quant[index]));
2359 if (error / 64 < best.second) best = {quality, error / 64};
2360 }
2361 return best;
2362}
2363
2364/** @brief Measure image statistics that a lens profile should reproduce.
2365 * @param path JPEG saved by a real or simulated camera.
2366 * @return Encoder, exposure, noise, sharpness, shading and colour facts.
2367 * @details Compare medians of real flight photos with medians of simulated
2368 * photos of similar terrain; single-image values are scene dependent. */
2369Json measure(const std::string& path) {
2371 const cv::Mat image = read_jpeg(path, false, 1024 * mib, &header);
2372 cv::Mat luma8, luma, gradient_x, gradient_y, magnitude, residual, laplacian;
2373 cv::cvtColor(image, luma8, cv::COLOR_RGB2GRAY);
2374 luma8.convertTo(luma, CV_32F);
2375 // Immerkaer (1996) noise estimate restricted to the flattest 30 percent of
2376 // unsaturated pixels so that scene texture is not mistaken for noise.
2377 float laplacian_difference[9] = {1, -2, 1, -2, 4, -2, 1, -2, 1};
2378 const cv::Mat kernel(3, 3, CV_32F, laplacian_difference);
2379 cv::filter2D(luma, residual, CV_32F, kernel);
2380 cv::Sobel(luma, gradient_x, CV_32F, 1, 0);
2381 cv::Sobel(luma, gradient_y, CV_32F, 0, 1);
2382 cv::magnitude(gradient_x, gradient_y, magnitude);
2383 std::vector<float> flat;
2384 flat.reserve(luma.total());
2385 for (int row = 1; row + 1 < luma.rows; ++row)
2386 for (int column = 1; column + 1 < luma.cols; ++column) {
2387 const unsigned char value = luma8.at<unsigned char>(row, column);
2388 if (value > 2 && value < 253) flat.push_back(magnitude.at<float>(row, column));
2389 }
2390 require(!flat.empty(), "image has no unsaturated pixels: " + path);
2391 auto cut = flat.begin() + flat.size() * 3 / 10;
2392 std::nth_element(flat.begin(), cut, flat.end());
2393 const float threshold = *cut;
2394 double absolute = 0;
2395 size_t count = 0;
2396 for (int row = 1; row + 1 < luma.rows; ++row)
2397 for (int column = 1; column + 1 < luma.cols; ++column) {
2398 const unsigned char value = luma8.at<unsigned char>(row, column);
2399 if (value > 2 && value < 253 &&
2400 magnitude.at<float>(row, column) <= threshold) {
2401 absolute += std::abs(residual.at<float>(row, column));
2402 ++count;
2403 }
2404 }
2405 cv::Laplacian(luma, laplacian, CV_32F);
2406 cv::Scalar mean, deviation;
2407 cv::meanStdDev(laplacian, mean, deviation);
2408 const double sharpness = deviation[0] * deviation[0];
2409 int histogram[256] = {};
2410 for (size_t index = 0; index < luma8.total(); ++index)
2411 ++histogram[luma8.data[index]];
2412 auto percentile = [&](double fraction) {
2413 size_t total = 0;
2414 for (int value = 0; value < 256; ++value)
2415 if ((total += histogram[value]) >= fraction * luma8.total()) return value;
2416 return 255;
2417 };
2418 const int patch_w = std::max(1, image.cols / 10),
2419 patch_h = std::max(1, image.rows / 10);
2420 auto patch = [&](int x, int y) {
2421 return cv::mean(luma(cv::Rect(x, y, patch_w, patch_h)))[0];
2422 };
2423 const double corners =
2424 (patch(0, 0) + patch(image.cols - patch_w, 0) +
2425 patch(0, image.rows - patch_h) +
2426 patch(image.cols - patch_w, image.rows - patch_h)) / 4;
2427 const double centre =
2428 patch((image.cols - patch_w) / 2, (image.rows - patch_h) / 2);
2429 const cv::Scalar colour = cv::mean(image);
2430 const auto quality = ijg_quality(header.luma_quant);
2431 return Json::object(
2432 {{"file", path},
2433 {"width", image.cols},
2434 {"height", image.rows},
2435 {"jpeg_quality_estimate", quality.first},
2436 {"jpeg_table_error", quality.second},
2437 {"chroma_subsampling", header.subsampling},
2438 {"noise_sigma", std::sqrt(pi / 2) * absolute / (6.0 * count)},
2439 {"laplacian_variance", sharpness},
2440 {"luma_mean", cv::mean(luma)[0]},
2441 {"luma_p1", percentile(0.01)},
2442 {"luma_p99", percentile(0.99)},
2443 {"corner_centre_ratio", centre > 0 ? Json(corners / centre) : Json(nullptr)},
2444 {"red_green_ratio", colour[1] > 0 ? Json(colour[0] / colour[1]) : Json(nullptr)},
2445 {"blue_green_ratio", colour[1] > 0 ? Json(colour[2] / colour[1]) : Json(nullptr)}});
2446}
2447
2448/** @brief Measure files and append per-key medians of numeric results. */
2449Json measure_all(const std::vector<std::string>& paths) {
2450 Json files = Json::array(), median = Json::object();
2451 std::map<std::string, std::vector<double>> values;
2452 for (const auto& path : paths) {
2453 Json result = measure(path);
2454 for (const char* key :
2455 {"width", "height", "jpeg_quality_estimate", "noise_sigma",
2456 "laplacian_variance", "luma_mean", "luma_p1", "luma_p99",
2457 "corner_centre_ratio", "red_green_ratio", "blue_green_ratio"})
2458 if (!result[key].is_null()) values[key].push_back(result[key].get<double>());
2459 files.push_back(result);
2460 }
2461 for (auto& [key, list] : values) {
2462 std::sort(list.begin(), list.end());
2463 const size_t middle = list.size() / 2;
2464 median.set(key, list.size() % 2 ? list[middle]
2465 : (list[middle - 1] + list[middle]) / 2);
2466 }
2467 return Json::object({{"schema", "USHER_LENS_MEASURE_V1"},
2468 {"count", static_cast<int>(paths.size())},
2469 {"median", median},
2470 {"files", files}});
2471}
2472
2473/** @brief Validate a lens profile or generate a simulated camera derivative.
2474 * @param argc Argument count including the program name.
2475 * @param argv Command-line argument vector.
2476 * @return Zero for success, or one after reporting an exception.
2477 * @details Version and validation requests bypass rendering. Render jobs
2478 * take a nonblocking executable lock so separate jobs cannot multiply the
2479 * working set managed by the per-job memory budget. */
2480int main(int argc, char** argv) {
2481 try {
2482 if (argc == 2 && std::string(argv[1]) == "--version") {
2483 std::cout << CAMERA_LENS_BUILD_VERSION << '\n';
2484 return 0;
2485 }
2486 if (argc == 3 && std::string(argv[1]) == "--validate-profile") {
2487 std::cout << read_profile(argv[2]).dump() << '\n';
2488 return 0;
2489 }
2490 const usher_cv_parallel::ScopedBackend opencv_parallel_backend;
2491 if (argc >= 3 && std::string(argv[1]) == "--measure") {
2492 std::cout << measure_all(std::vector<std::string>(argv + 2, argv + argc))
2493 .dump(-1, ' ', true)
2494 << '\n';
2495 return 0;
2496 }
2497 File job_lock(fopen("/proc/self/exe", "rb"));
2498 require(job_lock && flock(fileno(job_lock.get()), LOCK_EX | LOCK_NB) == 0,
2499 "busy or unavailable native lens job lock");
2500 std::string profile, cache;
2501 std::vector<std::string> positional;
2502 for (int argument = 1; argument < argc; ++argument) {
2503 const std::string value(argv[argument]);
2504 if ((value == "--profile" || value == "--cache") && argument + 1 < argc) {
2505 (value == "--profile" ? profile : cache) = argv[++argument];
2506 } else {
2507 require(value.empty() || value[0] != '-',
2508 "unknown or incomplete lens option");
2509 positional.push_back(value);
2510 }
2511 }
2512 require(!profile.empty() && positional.size() == 2,
2513 "usage: camera_lens --version | --validate-profile FILE | "
2514 "--measure JPEG... | --profile FILE [--cache DIR] INPUT "
2515 "OUTPUT");
2516 std::cout << emulate(positional[0], positional[1],
2517 read_profile(profile.c_str()), cache)
2518 .dump(-1, ' ', true)
2519 << '\n';
2520 return 0;
2521 } catch (const std::exception& error) {
2522 std::cerr << "CAMERA_LENS: " << error.what() << '\n';
2523 return 1;
2524 }
2525}
#define CAMERA_LENS_SIMD
Json measure_all(const std::vector< std::string > &paths)
Measure files and append per-key medians of numeric results.
int main(int argc, char **argv)
Validate a lens profile or generate a simulated camera derivative.
Json measure(const std::string &path)
Measure image statistics that a lens profile should reproduce.
std::pair< int, double > ijg_quality(const std::array< unsigned, 64 > &quant)
Estimate the IJG quality whose scaled luma table best matches.
static double latitude
static struct capture_pose poses[512]
size_t size
Definition chdk_pipe.c:143
int descriptor
Definition chdk_pipe.c:142
void(* close)(struct usbfs_handle *)
Definition chdkbridge.c:57
int(* open)(const struct usbfs_device *, struct usbfs_handle *)
Definition chdkbridge.c:56
std::vector< float > node_y
void row_nodes_into(int full_row, float *rx, float *ry) const
static void locate(const std::vector< float > &axis, float value, int &cell, float &weight)
cv::Mat strip(int first, int rows) const
Interpolate rays for full-frame rows [first, first + rows).
RayField(const Lens &lens, int width, int height, double scale, size_t budget=static_cast< size_t >(-1))
Build and verify the ray field for one channel focal scale.
cv::Vec2f at(float x, float y) const
Interpolate one ray at a sub-pixel position.
int node_columns() const
Interpolate rays at horizontal grid nodes for one full-frame row.
static std::vector< float > axis(int size, int step)
cv::Vec2f blend(int cell_row, float weight_row, int cell_column, float weight_column) const
std::vector< float > weight_x
size_t bytes() const
Resident bytes of the node grid and column lookup.
void row_into(int full_row, cv::Vec2f *destination) const
Interpolate rays for one full-frame row into a caller buffer.
std::vector< float > node_x
void verify(int64_t column, int64_t row)
Ensure a tile in the frame's bounding box exists and is valid without re-decoding tiles already verif...
TileCache(const std::string &path, size_t limit)
Configure a cache with no resident tiles.
std::vector< std::pair< int64_t, int64_t > > verified
cv::Mat tile(int64_t column, int64_t row)
Fetch a linearized float32 RGB tile from the local zoom-18 cache.
C++ value interface with explicit conversions and OpenCV JSON I/O.
std::vector< std::pair< std::string, Value > > items() const
Snapshot object names and shared child handles in map order.
static Value parse(Iterator first, Iterator last)
Copy a byte range and delegate parsing to the string overload.
Type get() const
Convert explicitly while preserving the distinction between types.
Value at(const char *key) const
Require an object member instead of substituting null.
void push_back(const Value &value)
Clone an element, append it in order and select array kind.
void set(const std::string &key, const Value &value)
Clone a member value, replace any matching key and select object kind.
std::string dump_sorted() const
Produce compact map-ordered output without FileStorage formatting.
bool is_number() const
Test all three numeric categories without treating bool as numeric.
std::string dump(int indent=-1, char=' ', bool=false) const
Serialize with OpenCV and restore native JSON scalar tokens.
Type value(const char *key, Type fallback) const
Use a default only for absence, never to hide an invalid present value.
static Value array()
Make an empty array.
static Value object()
Make an empty object.
bool is_object() const
Inspect the object tag.
RAII guard installing the backend at startup and joining workers before main() exits.
static bool bounds(const struct coverage_state *state, const struct coverage_candidate *candidate, struct coverage_bounds *box)
Reject unusable polygons before converting coordinates or scanning cells.
static bool start(uint32_t session)
Admit a nonzero session and discard correlation from the previous one.
static int record(const union dc_shot_union *shot)
Retrieve a point and admit a new observation only when its credit is free.
static uint8_t input_pixels[(size_t) MAX_INPUT_PIXELS *3U]
Reused input, tile and final RGB workspaces; avoid per-image allocator churn.
static struct ear_sample samples[EAR_MAX_SAMPLES]
static bool started
static bool range(size_t total, size_t offset, size_t length)
Check a byte span without forming a potentially overflowing end offset.
Definition exifite.c:133
Bounded C11 EXIF metadata and JPEG streaming interfaces.
int exif_serialize(const struct exif_document *document, unsigned char *payload, size_t capacity, size_t *size)
Serialize a document into one complete EXIF APP1 payload.
Definition exifite.c:736
bool exif_value_unsigned(const struct exif_value *value, size_t index, uint32_t *result)
Decode one BYTE, SHORT or LONG component without an unaligned load.
Definition exifite.c:339
unsigned char * exif_document_reserve(struct exif_document *document, enum exif_directory directory, uint16_t tag, enum exif_format format, uint32_t count)
Create or replace a field with zeroed, bounded value storage.
Definition exifite.c:508
void exif_encode_long(unsigned char *data, uint32_t value)
Encode a 32-bit value in the writer's little-endian representation.
Definition exifite.c:106
int exif_parse(struct exif_view *view, const unsigned char *payload, size_t size)
Validate an EXIF APP1 payload and create a borrowed view.
Definition exifite.c:257
exif_format
TIFF field-type numbers used by Sensoraia's metadata.
Definition exifite.h:64
int exif_document_remove(struct exif_document *document, enum exif_directory directory, uint16_t tag)
Remove a tag and reclaim its packed value storage.
Definition exifite.c:473
exif_directory
Directory identities, not on-disk tag numbers or offsets.
Definition exifite.h:59
int exif_read_jpeg(const char *filename, unsigned char *payload, size_t capacity, size_t *size)
Read the single EXIF APP1 segment before the first JPEG scan.
Definition exifite.c:1022
bool exif_document_get(const struct exif_document *document, enum exif_directory directory, uint16_t tag, struct exif_value *value)
Borrow a field from the current writable document.
Definition exifite.c:447
@ EXIF_ASCII
Definition exifite.h:66
@ EXIF_SHORT
Definition exifite.h:67
@ EXIF_SRATIONAL
Definition exifite.h:72
@ EXIF_LONG
Definition exifite.h:68
@ EXIF_UNDEFINED
Definition exifite.h:70
@ EXIF_RATIONAL
Definition exifite.h:69
@ EXIF_CAMERA
Definition exifite.h:59
@ EXIF_PRIMARY
Definition exifite.h:59
@ EXIF_EXPOSURE_BIAS
Definition exifite.h:91
@ EXIF_FOCAL_LENGTH
Definition exifite.h:92
@ EXIF_DOCUMENT_NAME
Definition exifite.h:79
@ EXIF_EXPOSURE_TIME
Definition exifite.h:86
@ EXIF_PIXEL_Y_DIMENSION
Definition exifite.h:97
@ EXIF_LENS_MODEL
Definition exifite.h:99
@ EXIF_PIXEL_X_DIMENSION
Definition exifite.h:96
@ EXIF_IMAGE_DESCRIPTION
Definition exifite.h:80
@ EXIF_SOFTWARE
Definition exifite.h:85
@ EXIF_USER_COMMENT
Definition exifite.h:93
@ EXIF_ORIENTATION
Definition exifite.h:81
@ EXIF_FNUMBER
Definition exifite.h:87
Flight, timing, and thermal-analysis metadata writing for captured JPEGs.
static int image_exif_import(struct exif_document *document, const struct exif_view *view)
Import standard metadata plus the application's approved capture evidence.
Definition image_exif.h:58
Reject invalid Huffman selectors before the bundled codec indexes its tables.
static void usher_jpeg_validate_marker(j_common_ptr state)
Validate marker parameters from an error manager's emit_message callback.
struct jpeg_compress_struct encoder
Definition lwir_image.c:59
uint8_t row[LWIR_MAX_WIDTH *3U]
Definition lwir_image.c:63
struct jpeg_error_mgr error
Definition lwir_image.c:60
struct jpeg_decompress_struct decoder
Definition lwir_mock.c:55
uint8_t frame[LWIR_MAX_FRAME_BYTES]
Definition lwir_mock.c:58
static uint32_t histogram[65536]
Definition lwir_render.c:53
static struct usher_camera_profile profile
constexpr size_t mib
const std::array< unsigned char, 65536 > & srgb_encoding()
Linear-light to 8-bit sRGB lookup, built once.
Json read_shot(const exif_document *metadata, Json &geometry)
Json emulate(const std::string &input, const std::string &output, const Json &profile, const std::string &cache)
constexpr uint64_t maximum_mock_pixels
File open_regular(const std::string &path, size_t maximum)
float sample_pinhole_channel(const cv::Mat &source, float map_x, float map_y, int channel)
Bilinearly sample a linear CV_32FC3 pinhole source image matching OpenCV INTER_LINEAR + BORDER_CONSTA...
Json read_profile(const char *path)
CAMERA_LENS_SIMD void sharpen_luma_rgb(cv::Mat &output, double sharpen)
Parallel luma-only unsharp mask (sigma = 1 px) preserving chroma.
TemporalSampling temporal_sampling(const RayField &field, const Pose &pose, int width, int height, int maximum)
Choose the fewest odd exposure samples that resolve motion blur.
void sample_tile_rgb(const float *const *tile_ptrs, int grid_width, int max_x, int max_y, float map_x, float map_y, float &red, float &green, float &blue)
Bilinearly sample all 3 RGB channels at identical coordinates from a 2D pointer grid of 256x256 linea...
void jpeg_message(j_common_ptr state, int level)
Enforce JPEG marker bounds before normal message handling.
double number(const Json &value, const std::string &name, double minimum, double maximum)
void set_rational(exif_document *metadata, uint16_t tag, double value, unsigned denominator, bool signed_value=false)
void write_jpeg(const std::string &output, const cv::Mat &image, int quality, const exif_document *metadata)
void set_text(exif_document *metadata, exif_directory ifd, uint16_t tag, const std::string &text, bool undefined=false)
constexpr unsigned int maximum_image_dimension
constexpr int lens_polish_passes
constexpr double maximum_reprojection_error_pixels
constexpr int lens_inverse_iterations
CAMERA_LENS_SIMD void compute_strip_bounds_rows(const std::vector< std::unique_ptr< RayField > > &fields, int channels, const RowSamplePose *poses, int samples, int first, int start_row, int end_row, double *min_e, double *max_e, double *min_n, double *max_n, std::atomic< bool > &horizon, std::atomic< bool > &non_finite)
SIMD-friendly strip ground-bounds and horizon check.
std::string tag_text(const exif_document *metadata, exif_directory ifd, uint16_t tag)
float sample_tile_channel(const float *const *tile_ptrs, int grid_width, int max_x, int max_y, float map_x, float map_y, int channel)
Bilinearly sample one channel from a 2D pointer grid of 256x256 linear float32 RGB tiles without asse...
std::unique_ptr< FILE, FileCloser > File
CAMERA_LENS_SIMD void blur_linear_rgb(cv::Mat &frame, double sigma)
Parallel separable Gaussian blur on a contiguous CV_32FC3 frame.
int reflect_101(int index, int size)
Reflect-101 index clamp matching OpenCV's BORDER_REFLECT_101.
usher_cv_json::Value Json
constexpr double temporal_sample_spacing_px
constexpr size_t tile_cache_entries
constexpr double ray_interpolation_tolerance_px
void require(bool valid, const char *message)
Route a failed precondition to the CLI's diagnostic handler.
constexpr size_t maximum_exif_payload_bytes
const std::array< float, 256 > & srgb_decoding()
8-bit sRGB to linear-light lookup, built once.
constexpr int ray_grid_initial_step
Rendered render(const cv::Mat &input, const Json &shot, const Json &geometry, const Json &profile, const std::string &cache_path, const Resources &resources, cv::Size source_size={})
CAMERA_LENS_SIMD void encode_sensor_rows(const cv::Mat &linear, cv::Mat &output, int start_row, int end_row, float well, float read_variance, uint64_t seed, const unsigned char *encode)
Parallel shot/read noise + sRGB encoding for a row range.
void jpeg_failure(j_common_ptr state)
Return a fatal libjpeg error to the active codec recovery point.
float stateless_normal(uint64_t row_seed, int index)
Stateless zero-mean, unit-variance Irwin-Hall normal sample.
constexpr double lens_polish_tolerance_px
unsigned char * replace_tag(exif_document *metadata, exif_directory ifd, uint16_t tag, exif_format format, size_t components)
CAMERA_LENS_SIMD void sample_cache_rows(const std::vector< std::unique_ptr< RayField > > &fields, int channels, const RowSamplePose *poses, int samples, int first, int start_row, int end_row, int width, float inv_map_scale, float base_offset_x, float base_offset_y, const float *const *tile_ptrs, int grid_width, int grid_height, float gain, float vignette_scale, cv::Mat &linear)
Fused parallel strip kernel for map-cache rendering.
cv::Mat read_jpeg(const std::string &path, bool tile, size_t budget, JpegHeader *header=nullptr, bool decode_pixels=true)
constexpr double pi
BasicResult decode(imx500::Span< const float > tensor, imx500::Span< const uint8_t > info, libcamera::Rectangle crop, libcamera::Size image)
const std::shared_ptr< ThreadPoolBackend > & ensure_installed()
Install the process-wide Sensoraia OpenCV parallel backend once.
Header-only C++17 value adapter around OpenCV's JSON persistence layer.
Header-only C++17 cv::parallel::ParallelForAPI backend for Sensoraia.
static const char header[]
Definition pose_log.c:87
static size_t count
Definition pose_log.c:79
static FILE * output
Definition pose_log.c:77
void sha256_hex(const void *data, size_t length, char *output)
Hash one memory region to a lowercase hexadecimal string.
Definition sha256fst.c:272
Allocation-free, portable SHA-256 checksums for memory and streams.
Compile-time build identity strings for USHER executables.
#define CAMERA_LENS_BUILD_VERSION
Definition version.h:77
size_t count(int limit) const
Validate an inclusive tile rectangle and count its cells.
Own decoder state across libjpeg's nonlocal error return.
jpeg_decompress_struct state
~Decoder()
Destroy the decoder if creation was attempted.
size_t bytes_per_pixel() const
Extra full-frame bytes per pixel used by enabled effects.
cv::Mat develop(cv::Mat &linear) const
Develop a linear RGB frame into the saved 8-bit RGB image.
Development(const Json &profile, const Json &shot)
Read effect settings and derive a reproducible per-shot seed.
Own encoder state independently of the output stream.
~Encoder()
Release the libjpeg compressor when armed.
jpeg_compress_struct state
void operator()(FILE *file) const
Close the stream owned by a File handle.
void(* emit)(j_common_ptr, int)
Encoder facts of a decoded JPEG, used to calibrate profiles.
std::array< unsigned, 64 > luma_quant
Natural order, 0 when absent.
cv::Mat rays(int width, int height, int first, int rows, double scale) const
cv::Matx33d intrinsic(int width, int height, double scale) const
Camera matrix for one channel focal scale.
Lens(int width, int height, const Json &values)
Build intrinsics for the selected distortion model.
cv::Mat project(const cv::Mat &rays, int width, int height, double scale) const
cv::Mat undistort(const cv::Mat &pixels, int width, int height, double scale) const
bool stationary() const
Check whether camera geometry varies across exposure samples.
Pose(const Json &shot, const Json &profile)
Resolve the shot geometry and time-dependent motion model.
std::array< double, 3 > rates
std::array< double, 3 > angles
std::pair< cv::Mat, cv::Mat > ground(const cv::Mat &rays, int first, int height, double sample) const
Resources(const Json &profile)
Resolve requested resources against current host limits.
Own an unpublished derivative and its temporary sibling path.
~TemporaryOutput()
Remove an unpublished temporary pathname.
TemporaryOutput(const std::string &output)
Open an exclusive temporary file beside the requested output.
Caller-owned bounded writer with no heap allocation.
Definition exifite.h:156
unsigned char data[EXIF_MAX_PAYLOAD]
Definition exifite.h:162
Borrowed bytes plus enough type information for safe interpretation.
Definition exifite.h:119
const unsigned char * data
Definition exifite.h:120
size_t size
Definition exifite.h:121
uint32_t count
Definition exifite.h:122
uint16_t format
Definition exifite.h:123
Borrowed view of a validated APP1 payload.
Definition exifite.h:131
One timestamped geographic position for a replay annotation.
static std::unique_ptr< pangea_survey::MapResources > resources
Definition survey.cpp:50
static struct camera_backend camera
Definition usher.c:521
bool valid
Definition usher.c:537
struct usher_control_result result
Definition usher.c:377
struct captured_image image
Definition usher.c:371
union dc_shot_union shot
Definition usher.c:534