Illustrated C/C++ source sheets, symbol index, API relationship graph, compiler console and debug trace

LIVE AI / FLEXIBLE SENSING / FLIGHT CONTEXT

Sensoraia API

Live AI and sensing, connected to the flight.
See, listen, inspect and return supported results while airborne.

Loading...
Searching...
No Matches
lwir_detection.c
Go to the documentation of this file.
1/*
2 * Copyright 2026 OpenUAS
3 * SPDX-License-Identifier: Apache-2.0
4 *
5 * Licensed under the Apache License, Version 2.0 (the "License");
6 * you may not use this file except in compliance with the License.
7 * You may obtain a copy of the License at
8 *
9 * http://www.apache.org/licenses/LICENSE-2.0
10 *
11 * Unless required by applicable law or agreed to in writing, software
12 * distributed under the License is distributed on an "AS IS" BASIS,
13 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14 * See the License for the specific language governing permissions and
15 * limitations under the License.
16 */
17
18/** @file
19 * @brief Pose-free thermal components using caller-owned bounded scratch storage.
20 *
21 * @details Analysis builds only (`LWIR_ENABLE_ANALYSIS=1`). Operates on the exact
22 * native Kelvin*64 plane recovered from a saved LWIR JPEG. Algorithm outline:
23 * -# Robust frame statistics: exact median (background) and median absolute
24 * deviation (noise) via 65536-bin histograms, so no sorting buffer is needed.
25 * -# Global threshold combining an absolute floor with a robust rise over background.
26 * -# 8-connected flood fill (breadth-first) of above-threshold pixels.
27 * -# Per component: local background, contrast gate, excess-weighted subpixel
28 * centroid and shape features.
29 * -# Keep the top two generic and top two tray-like components; report trays if any.
30 *
31 * Cost trade-off: every component clears and scans the full 65536-bin histogram
32 * for its local median, so frames with many small components cost roughly
33 * 65536 operations per component in addition to the per-pixel work.
34 */
35#include "lwir_detection.h"
36#include <math.h>
37
38/* Margin in pixels around a component's box used to sample its local background. */
40
41/**
42 * @brief Decode one unsigned 16-bit little-endian value.
43 * @param bytes At least two readable bytes; no alignment requirement.
44 * @return The decoded value, independent of host endianness.
45 */
46static uint16_t read_u16_le(const uint8_t *bytes)
47{
48 return (uint16_t)bytes[0] | (uint16_t)((uint16_t)bytes[1] << 8);
49}
50
51/**
52 * @brief Whether a raw code is a real measurement rather than a sensor endpoint code.
53 * @details 0 and 65535 (about 751 C, beyond the Tiny1C range) mark invalid or
54 * missing samples, never real temperatures, so they must not shape statistics
55 * or become hotspots.
56 * @param raw Kelvin*64 code.
57 * @return True for codes 1..65534.
58 */
59static bool valid_code(uint16_t raw)
60{
61 return raw != 0 && raw != UINT16_MAX;
62}
63
64/**
65 * @brief Test a pixel's visited bit.
66 * @param workspace Scratch storage holding the bitset.
67 * @param pixel Row-major native index within the current frame.
68 * @return True when the pixel was already queued.
69 */
70static bool bit_is_set(const struct lwir_detection_workspace *workspace, size_t pixel)
71{
72 return (workspace->seen[pixel >> 3] & (uint8_t)(1U << (pixel & 7U))) != 0;
73}
74
75/**
76 * @brief Set a pixel's visited bit.
77 * @details A bitset (rather than a byte per pixel) keeps the workspace small.
78 * @param workspace Scratch storage holding the bitset.
79 * @param pixel Row-major native index within the current frame.
80 */
81static void mark_bit(struct lwir_detection_workspace *workspace, size_t pixel)
82{
83 workspace->seen[pixel >> 3] |= (uint8_t)(1U << (pixel & 7U));
84}
85
86/**
87 * @brief Zero all 65536 histogram bins.
88 * @param workspace Scratch storage holding the histogram.
89 */
91{
92 for (size_t index = 0; index <= UINT16_MAX; ++index) workspace->histogram[index] = 0;
93}
94
95/**
96 * @brief Exact median of the values counted in the histogram.
97 * @details Walks the cumulative count once to find the lower and upper middle
98 * ranks; for an even count the result is their mean, so it can end in .5.
99 * Exact for integer inputs, avoiding a sort and a pixel-sized copy.
100 * @param workspace Scratch storage whose histogram holds exactly @p count samples.
101 * @param count Number of samples; must be at least 1 ((count - 1) would wrap).
102 * @return Median value in histogram-bin units.
103 */
104static double histogram_median(const struct lwir_detection_workspace *workspace, uint64_t count)
105{
106 const uint64_t lower_index = (count - 1) / 2;
107 const uint64_t upper_index = count / 2;
108 uint64_t accumulated = 0;
109 uint16_t lower = 0, upper = 0;
110 bool lower_found = false;
111 for (size_t value = 0; value <= UINT16_MAX; ++value) {
112 accumulated += workspace->histogram[value];
113 if (!lower_found && accumulated > lower_index) { lower = (uint16_t)value; lower_found = true; }
114 if (accumulated > upper_index) { upper = (uint16_t)value; break; }
115 }
116 return ((double)lower + upper) / 2.0;
117}
118
119/**
120 * @brief Ranking predicate for retained candidates.
121 * @details Generic order: higher contrast first, then larger excess heat.
122 * Tray order: larger excess heat first (presumably favouring an extended hot
123 * tray over a small very hot point), then contrast. A full tie returns false, so the
124 * earlier-found component (row-major seed order) keeps its place.
125 * @param candidate Newly evaluated component.
126 * @param current Retained component it is compared against.
127 * @param tray_order Select the tray ordering instead of the generic one.
128 * @return True when @p candidate should rank strictly above @p current.
129 */
130static bool candidate_better(const struct lwir_hotspot *candidate,
131 const struct lwir_hotspot *current, bool tray_order)
132{
133 if (tray_order && candidate->excess_celsius_pixels != current->excess_celsius_pixels)
134 return candidate->excess_celsius_pixels > current->excess_celsius_pixels;
135 if (candidate->contrast_celsius != current->contrast_celsius)
136 return candidate->contrast_celsius > current->contrast_celsius;
137 return candidate->excess_celsius_pixels > current->excess_celsius_pixels;
138}
139
140/**
141 * @brief Insert a candidate into a sorted top-two list.
142 * @details Insertion sort over a fixed two-slot array: while fewer than two are
143 * held the list grows; afterwards a candidate must beat the current second place
144 * to enter, and the displaced second place is dropped.
145 * @param candidates Sorted list, best first.
146 * @param[in,out] count Valid entries (0..2); incremented while below 2.
147 * @param candidate Component to consider; copied by value.
148 * @param tray_order Ordering passed to candidate_better().
149 */
150static void retain_candidate(struct lwir_hotspot candidates[2], size_t *count,
151 const struct lwir_hotspot *candidate, bool tray_order)
152{
153 size_t position = *count;
154 if (position < 2) ++*count;
155 else if (!candidate_better(candidate, &candidates[1], tray_order)) return;
156 if (position > 1) position = 1;
157 while (position > 0 && candidate_better(candidate, &candidates[position - 1], tray_order)) {
158 if (position < 2) candidates[position] = candidates[position - 1];
159 --position;
160 }
161 candidates[position] = *candidate;
162}
163
164/**
165 * @brief Detect and rank hot connected components in a native temperature plane.
166 * @details See lwir_detection.h for the contract and the file overview for the
167 * algorithm. All arithmetic stays in raw Kelvin*64 codes until results are
168 * converted to Celsius (code / 64 - 273.15); temperature differences convert
169 * by dividing by 64 only.
170 * @return True for a valid plane (even with zero candidates); false for invalid arguments.
171 */
172bool lwir_detect_hotspots(const uint8_t *temperature, size_t bytes, unsigned width, unsigned height,
174{
175 if (!temperature || !workspace || !result || !width || !height || width > LWIR_MAX_WIDTH ||
176 height > LWIR_MAX_HEIGHT || bytes != (size_t)width * height * 2) return false;
177 const size_t pixels = (size_t)width * height;
178 /* Frame median over valid codes only: robust against hotspots occupying a minority of pixels. */
180 uint64_t valid_pixels = 0;
181 for (size_t pixel = 0; pixel < pixels; ++pixel) {
182 const uint16_t raw = read_u16_le(temperature + pixel * 2);
183 if (valid_code(raw)) { ++workspace->histogram[raw]; ++valid_pixels; }
184 }
185 /* A plane without any real measurement is valid input with nothing to detect. */
186 if (!valid_pixels) {
187 *result = (struct lwir_hotspot_set){.background_celsius = NAN, .threshold_celsius = NAN};
188 return true;
189 }
190 const double background_raw = histogram_median(workspace, valid_pixels);
191 /* Median absolute deviation, binned to the nearest whole code, as a noise
192 * estimate unaffected by the hot tail. Very large deviations share the top bin. */
194 for (size_t pixel = 0; pixel < pixels; ++pixel) {
195 const uint16_t raw = read_u16_le(temperature + pixel * 2);
196 if (!valid_code(raw)) continue;
197 const double deviation = fabs((double)raw - background_raw);
198 const unsigned bucket = (unsigned)(deviation + 0.5);
199 ++workspace->histogram[bucket <= UINT16_MAX ? bucket : UINT16_MAX];
200 }
201 const double noise_raw = histogram_median(workspace, valid_pixels);
202 /* Preserve the existing mission policy; this ownership change is not detector tuning. */
203 /* Threshold in raw codes, step by step:
204 * minimum = 60 C absolute floor, presumably so ordinary warm ground on a
205 * low-noise scene cannot trigger detections;
206 * rise = max(20 C, 6 * sigma), sigma = 1.4826 * MAD (Gaussian-consistent);
207 * result = max(minimum, median + rise).
208 * 20 C appears as 20 * 64 because a temperature difference needs no 273.15 offset. */
209 const double minimum_raw = (60.0 + 273.15) * 64.0;
210 const double rise_raw = fmax(20.0 * 64.0, 6.0 * 1.4826 * noise_raw);
211 const double threshold_raw = fmax(minimum_raw, background_raw + rise_raw);
212 for (size_t index = 0; index < (pixels + 7) / 8; ++index) workspace->seen[index] = 0;
213 struct lwir_hotspot generic[2] = {{0}}, trays[2] = {{0}};
214 size_t generic_count = 0, tray_count = 0;
215 size_t generic_top_count = 0, tray_top_count = 0;
216 for (size_t seed = 0; seed < pixels; ++seed) {
217 /* Each unvisited above-threshold pixel seeds one 8-connected component.
218 * Code 0 can never reach the >= 60 C threshold, so only 65535 needs exclusion. */
219 const uint16_t seed_raw = read_u16_le(temperature + seed * 2);
220 if (bit_is_set(workspace, seed) || seed_raw == UINT16_MAX || seed_raw < threshold_raw) continue;
221 /* The queue restarts at 0 per component; after the fill, queue[0..tail)
222 * lists exactly this component's pixels for the centroid pass below. */
223 size_t head = 0, tail = 1;
224 workspace->queue[0] = (uint32_t)seed;
225 mark_bit(workspace, seed);
226 struct lwir_hotspot candidate = {0};
227 candidate.min_x = candidate.max_x = candidate.peak_x = seed % width;
228 candidate.min_y = candidate.max_y = candidate.peak_y = seed / width;
229 uint16_t peak = read_u16_le(temperature + seed * 2);
230 uint64_t sum_raw = 0;
231 while (head < tail) {
232 const size_t pixel = workspace->queue[head++];
233 const size_t x = pixel % width, y = pixel / width;
234 const uint16_t raw = read_u16_le(temperature + pixel * 2);
235 ++candidate.pixels;
236 sum_raw += raw;
237 if (x < candidate.min_x) candidate.min_x = x;
238 if (x > candidate.max_x) candidate.max_x = x;
239 if (y < candidate.min_y) candidate.min_y = y;
240 if (y > candidate.max_y) candidate.max_y = y;
241 if (raw > peak) { peak = raw; candidate.peak_x = x; candidate.peak_y = y; }
242 const size_t min_y = y ? y - 1 : 0, max_y = y + 1 < height ? y + 1 : y;
243 const size_t min_x = x ? x - 1 : 0, max_x = x + 1 < width ? x + 1 : x;
244 for (size_t neighbor_y = min_y; neighbor_y <= max_y; ++neighbor_y) {
245 for (size_t neighbor_x = min_x; neighbor_x <= max_x; ++neighbor_x) {
246 const size_t neighbor = neighbor_y * width + neighbor_x;
247 const uint16_t neighbor_raw = read_u16_le(temperature + neighbor * 2);
248 if (!bit_is_set(workspace, neighbor) && neighbor_raw != UINT16_MAX && neighbor_raw >= threshold_raw) {
249 /* Defensive: marking makes overflow impossible, but never write past the queue. */
250 if (tail >= pixels) return false;
251 /* Mark before enqueueing: each pixel occupies at most one queue slot. */
252 mark_bit(workspace, neighbor);
253 workspace->queue[tail++] = (uint32_t)neighbor;
254 }
255 }
256 }
257 }
258 /* Local background: median of sub-threshold pixels in the box grown by
259 * lwir_hotspot_radius, clamped to the frame. Excluding all hot pixels
260 * (including neighbouring components) keeps the estimate from being
261 * pulled up by the object itself. */
262 const size_t box_min_x = candidate.min_x > lwir_hotspot_radius ? candidate.min_x - lwir_hotspot_radius : 0;
263 const size_t box_min_y = candidate.min_y > lwir_hotspot_radius ? candidate.min_y - lwir_hotspot_radius : 0;
264 const size_t box_max_x = candidate.max_x + lwir_hotspot_radius < width ? candidate.max_x + lwir_hotspot_radius : width - 1;
265 const size_t box_max_y = candidate.max_y + lwir_hotspot_radius < height ? candidate.max_y + lwir_hotspot_radius : height - 1;
267 uint64_t background_count = 0;
268 for (size_t y = box_min_y; y <= box_max_y; ++y) for (size_t x = box_min_x; x <= box_max_x; ++x) {
269 const uint16_t raw = read_u16_le(temperature + (y * width + x) * 2);
270 if (valid_code(raw) && raw < threshold_raw) { ++workspace->histogram[raw]; ++background_count; }
271 }
272 const double local_background_raw = background_count ? histogram_median(workspace, background_count) : background_raw;
273 /* Centroid weights are the excess over local background, clamped at zero:
274 * x_c = sum(x * w) / sum(w), w = max(0, raw - background). Hotter pixels pull
275 * the centre, giving subpixel precision; sum(w) doubles as the heat score. */
276 double weighted_x = 0, weighted_y = 0, excess_raw_pixels = 0;
277 for (size_t index = 0; index < tail; ++index) {
278 const size_t pixel = workspace->queue[index];
279 const double weight = fmax(0, (double)read_u16_le(temperature + pixel * 2) - local_background_raw);
280 weighted_x += (pixel % width) * weight;
281 weighted_y += (pixel / width) * weight;
282 excess_raw_pixels += weight;
283 }
284 const double peak_celsius = peak / 64.0 - 273.15;
285 const double local_celsius = local_background_raw / 64.0 - 273.15;
286 const double contrast = peak_celsius - local_celsius;
287 /* Reject components that are hot overall but barely stand out locally
288 * (e.g. a large warm surface); they are not counted in generic_count. */
289 if (contrast < 15.0 || excess_raw_pixels <= 0) continue;
290 candidate.center_x = weighted_x / excess_raw_pixels;
291 candidate.center_y = weighted_y / excess_raw_pixels;
292 candidate.peak_celsius = peak_celsius;
293 candidate.mean_celsius = ((double)sum_raw / candidate.pixels) / 64.0 - 273.15;
294 candidate.local_background_celsius = local_celsius;
295 candidate.contrast_celsius = contrast;
296 candidate.excess_celsius_pixels = excess_raw_pixels / 64.0;
297 candidate.touches_edge = candidate.min_x == 0 || candidate.min_y == 0 || candidate.max_x + 1 == width || candidate.max_y + 1 == height;
298 const double span_x = candidate.max_x - candidate.min_x + 1.0;
299 const double span_y = candidate.max_y - candidate.min_y + 1.0;
300 candidate.fill_ratio = candidate.pixels / (span_x * span_y);
301 candidate.aspect_ratio = fmax(span_x, span_y) / fmin(span_x, span_y);
302 /* Fire-tray heuristic: fully inside the frame, at least 2x2 and 4 pixels
303 * (so a single hot pixel cannot qualify), peak >= 100 C, contrast >= 40 C,
304 * reasonably filled box (>= 25 %) and not elongated (aspect <= 3). Limits
305 * apply to apparent, uncorrected temperatures. */
306 candidate.tray_like = !candidate.touches_edge && candidate.pixels >= 4 && span_x >= 2 && span_y >= 2
307 && candidate.peak_celsius >= 100 && candidate.contrast_celsius >= 40
308 && candidate.fill_ratio >= 0.25 && candidate.aspect_ratio <= 3.0;
309 ++generic_count;
310 retain_candidate(generic, &generic_top_count, &candidate, false);
311 if (candidate.tray_like) {
312 ++tray_count;
313 retain_candidate(trays, &tray_top_count, &candidate, true);
314 }
315 }
316 *result = (struct lwir_hotspot_set){0};
317 result->background_celsius = background_raw / 64.0 - 273.15;
318 result->threshold_celsius = threshold_raw / 64.0 - 273.15;
319 result->generic_count = generic_count;
320 result->tray_count = tray_count;
321 /* Any tray-like component switches the whole report to the tray ranking;
322 * otherwise the generic ranking is the fallback. */
323 result->tray_pass_selected = tray_count != 0;
324 const struct lwir_hotspot *selected = result->tray_pass_selected ? trays : generic;
325 result->count = result->tray_pass_selected ? tray_top_count : generic_top_count;
326 for (size_t index = 0; index < result->count; ++index) result->candidates[index] = selected[index];
327 return true;
328}
329
uint8_t bytes[CHDK_CONTROL_REPLY_SIZE]
Definition chdk_pipe.c:144
static struct image_mock_workspace workspace
@ lwir_hotspot_radius
static void mark_bit(struct lwir_detection_workspace *workspace, size_t pixel)
Set a pixel's visited bit.
static bool bit_is_set(const struct lwir_detection_workspace *workspace, size_t pixel)
Test a pixel's visited bit.
static bool candidate_better(const struct lwir_hotspot *candidate, const struct lwir_hotspot *current, bool tray_order)
Ranking predicate for retained candidates.
static uint16_t read_u16_le(const uint8_t *bytes)
Decode one unsigned 16-bit little-endian value.
static double histogram_median(const struct lwir_detection_workspace *workspace, uint64_t count)
Exact median of the values counted in the histogram.
bool lwir_detect_hotspots(const uint8_t *temperature, size_t bytes, unsigned width, unsigned height, struct lwir_detection_workspace *workspace, struct lwir_hotspot_set *result)
Detect and rank hot connected components in a native temperature plane.
static void histogram_clear(struct lwir_detection_workspace *workspace)
Zero all 65536 histogram bins.
static void retain_candidate(struct lwir_hotspot candidates[2], size_t *count, const struct lwir_hotspot *candidate, bool tray_order)
Insert a candidate into a sorted top-two list.
static bool valid_code(uint16_t raw)
Whether a raw code is a real measurement rather than a sensor endpoint code.
Optional thermal candidate detection, independent of pose and image files.
#define LWIR_MAX_WIDTH
Largest supported native width in pixels.
Definition lwir_frame.h:47
#define LWIR_MAX_HEIGHT
Largest supported native height in pixels.
Definition lwir_frame.h:49
static uint16_t temperature(const uint8_t *frame, size_t pixels, size_t pixel)
Read one little-endian thermal code.
Definition lwir_render.c:63
static size_t head
Definition pose_log.c:79
static size_t count
Definition pose_log.c:79
Caller-owned scratch storage; never reserved by an imagery-only build.
Bounded ranking, retaining the existing two-candidate mission policy.
A thermal component expressed only in native sensor coordinates.
size_t min_x
Inclusive native bounding box; likewise min_y, max_x and max_y.
size_t peak_x
Column of the first pixel (breadth-first order) holding the peak code; peak_y is its row.
double excess_celsius_pixels
Sum of per-pixel excess over local background, an integrated heat score.
double fill_ratio
Pixel count divided by bounding-box area.
double local_background_celsius
Median of sub-threshold pixels within a 5-pixel margin around the box.
double center_x
Subpixel centroid column weighted by excess over local background; center_y likewise.
double aspect_ratio
Longer bounding-box side divided by the shorter side (at least 1).
double peak_celsius
size_t pixels
8-connected pixel count at or above the frame threshold.
double contrast_celsius
Peak minus local background; components below 15 C are discarded.
bool touches_edge
Bounding box reaches a frame border, so the object may be truncated.
bool tray_like
Passed the compact, hot, high-contrast fire-tray heuristic.
double mean_celsius
Unweighted mean over component pixels.
struct usher_control_result result
Definition usher.c:377