From 51241479cfc8c8e7e529c14a6e43f18101914aa1 Mon Sep 17 00:00:00 2001 From: "Charles J. Cliffe" <247927+cjcliffe@users.noreply.github.com> Date: Thu, 6 Aug 2026 19:39:37 -0400 Subject: [PATCH] Implemented document re-structuring --- AGENT-LOG.md | 38 +++++ docs/PLAN.md | 3 +- docs/design/README.md | 3 +- docs/design/audio-subsystem.md | 4 +- docs/design/signal-flow.md | 4 +- docs/design/threading.md | 18 +-- docs/design/visual-data-pipeline.md | 143 ++++++++++++++++++ ...al-architecture.md => visual-rendering.md} | 139 +---------------- 8 files changed, 200 insertions(+), 152 deletions(-) create mode 100644 docs/design/visual-data-pipeline.md rename docs/design/{visual-architecture.md => visual-rendering.md} (79%) diff --git a/AGENT-LOG.md b/AGENT-LOG.md index e5b88ae..fb900ab 100644 --- a/AGENT-LOG.md +++ b/AGENT-LOG.md @@ -783,3 +783,41 @@ content). No changes were made to any design document. | File | Action | |------|--------| | `docs/plans/reorganize-design-docs.md` | Added "Source-to-Destination Mapping" table and inline split reconciliation notes; clarified SpinMutex end-state phrasing | + +## Session 61: Reorg Plan Implementation — Visual Architecture Split + +**Date:** 2026-08-06 +**Model:** opencode/mimo-v2.5-free + +- Implemented `docs/plans/reorganize-design-docs.md` — split `visual-architecture.md` into owner-per-module docs +- Created `visual-rendering.md` (rendering/UI half: canvas hierarchy, GLPanel, PrimaryGLContext, GLFont, ColorTheme, rendering flow, mouse interaction) +- Created `visual-data-pipeline.md` (processing half: pipeline topology, VisualProcessor, distribution modes, FFTDataDistributor, SpectrumVisualProcessor, FFTVisualDataThread, ScopeVisualProcessor) +- Deleted `visual-architecture.md` +- Trimmed `signal-flow.md` "Visual Processing Pipeline" to cross-links (topology diagram kept, internals → visual-data-pipeline.md, canvas pull → visual-rendering.md) +- Trimmed `threading.md` — reduced ReBuffer Pooling to pointer, VisualProcessor Pipeline to threading-model statement, SpectrumVisualProcessor busy_run to lock-inventory pointer, dropped per-canvas OnIdle/OnPaint enumeration from wxWidgets Integration +- Trimmed `audio-subsystem.md` Buffer Management to pointer to signal-flow.md +- Updated `docs/design/README.md` and `docs/PLAN.md` indexes with the two new files +- Ran grep verification: ReBuffer described fully only in signal-flow.md; VisualProcessor internals only in visual-data-pipeline.md; no dead links to visual-architecture.md in design docs + +### Files Created + +| File | Description | +|------|-------------| +| `docs/design/visual-rendering.md` | Rendering/UI half: canvas hierarchy, GLPanel system, PrimaryGLContext, GLFont, ColorTheme, rendering flow, mouse interaction | +| `docs/design/visual-data-pipeline.md` | Processing half: pipeline topology, VisualProcessor template, distribution modes, FFT/scope processors | + +### Files Modified + +| File | Action | +|------|--------| +| `docs/design/signal-flow.md` | Trimmed "Visual Processing Pipeline" to cross-links | +| `docs/design/threading.md` | Reduced ReBuffer Pooling, VisualProcessor Pipeline, SpectrumVisualProcessor busy_run, wxWidgets Integration sections | +| `docs/design/audio-subsystem.md` | Trimmed Buffer Management to pointer | +| `docs/design/README.md` | Replaced "Visual Architecture" with two new entries | +| `docs/PLAN.md` | Replaced "Visual Architecture" with two new entries | + +### Files Deleted + +| File | Reason | +|------|--------| +| `docs/design/visual-architecture.md` | Split into visual-rendering.md and visual-data-pipeline.md | diff --git a/docs/PLAN.md b/docs/PLAN.md index b341ef3..8749bdd 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -20,7 +20,8 @@ Design documents covering the system architecture are in [docs/design/](design/) | Document | Description | |----------|-------------| | [Audio Subsystem](design/audio-subsystem.md) | Controller/bound mixing, WAV recording, device management | -| [Visual Architecture](design/visual-architecture.md) | Canvas hierarchy, GLFont, ColorTheme, rendering pipeline | +| [Visual Rendering](design/visual-rendering.md) | Canvas hierarchy, GLFont, ColorTheme, rendering flow | +| [Visual Data Pipeline](design/visual-data-pipeline.md) | VisualProcessor, distribution modes, FFT/scope processing | | [Configuration System](design/configuration-system.md) | AppConfig/DeviceConfig, DataTree serialization, sessions | | [Bookmark System](design/bookmark-system.md) | BookmarkMgr, groups/ranges/recents, persistence | | [SDR Device Layer](design/sdr-device-layer.md) | SDREnumerator, SDRDeviceInfo, manual devices | diff --git a/docs/design/README.md b/docs/design/README.md index 77190a9..8c77f88 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -17,7 +17,8 @@ This directory contains architectural documentation for CubicSDR. These document | Document | Description | |----------|-------------| | [Audio Subsystem](audio-subsystem.md) | Controller/bound mixing pattern, WAV recording pipeline, device management, and real-time audio callback | -| [Visual Architecture](visual-architecture.md) | Canvas hierarchy, GLPanel system, GLFont rendering, ColorTheme system, and visual data processing pipeline | +| [Visual Rendering](visual-rendering.md) | Canvas hierarchy, GLPanel system, GLFont rendering, ColorTheme system, and rendering flow | +| [Visual Data Pipeline](visual-data-pipeline.md) | VisualProcessor template, distribution modes, FFT processing, and ScopeVisualProcessor | | [Configuration System](configuration-system.md) | AppConfig/DeviceConfig persistence, DataTree serialization, session management, and file locations | | [Bookmark System](bookmark-system.md) | BookmarkMgr data model, groups/ranges/recents, XML persistence, and default frequency bands | | [SDR Device Layer](sdr-device-layer.md) | SDREnumerator discovery, SDRDeviceInfo capabilities, manual devices, and SoapySDR module loading | diff --git a/docs/design/audio-subsystem.md b/docs/design/audio-subsystem.md index 0c9a61f..2d2ae8a 100644 --- a/docs/design/audio-subsystem.md +++ b/docs/design/audio-subsystem.md @@ -256,9 +256,7 @@ Design constraints: ## Buffer Management -`DemodulatorThread` uses `ReBuffer` (defined in `IOThread.h`) to pool audio buffers. - -The pool works by tracking `shared_ptr` use counts: when a buffer's use count drops to 1 (only referenced by the pool itself), it becomes available for reuse. When `getBuffer()` finds the first reusable buffer, it selects it and resets its age to 1; subsequent reusable buffers found in the same call have their age decremented. The oldest buffer at the back of the pool is garbage-collected when its age drops below `-REBUFFER_GC_LIMIT` (-100). New buffers are allocated only when no reusable buffer is available. +`DemodulatorThread` uses `ReBuffer` (defined in `IOThread.h`) to pool audio output buffers. For the full pool mechanics (reuse logic, GC thresholds, warning thresholds), see [signal-flow.md](signal-flow.md) "Buffer Management". ## Muting diff --git a/docs/design/signal-flow.md b/docs/design/signal-flow.md index c8fedcc..60eccd0 100644 --- a/docs/design/signal-flow.md +++ b/docs/design/signal-flow.md @@ -164,6 +164,4 @@ DemodulatorThread +--[audioVisOutputQueue]--> ScopeVisualProcessor --> ScopeCanvas (UI thread) ``` -`FFTVisualDataThread` contains an internal sub-pipeline: an `FFTDataDistributor` accumulates raw IQ samples into FFT-sized chunks, which are then processed by a `SpectrumVisualProcessor` to produce FFT output. The `FFTDataDistributor` rate-limits by `linesPerSecond`, so not every incoming IQ frame produces an output line. - -The UI thread is **pull-based on the consumer side** — canvases poll their input queues via `try_pop()`, though the exact trigger varies: WaterfallCanvas does this in its `OnIdle()` handler, while SpectrumCanvas and ScopeCanvas do it in `OnPaint()`. Meanwhile, producer threads **push** data into these queues via non-blocking `try_push()`, which can silently drop data when queues are full. This is intentional for visual data where occasional dropped frames are acceptable. +For the visual data processing pipeline internals (VisualProcessor, distributors, FFT processing, ScopeVisualProcessor), see [visual-data-pipeline.md](visual-data-pipeline.md). For the canvas pull model and rendering flow (OnIdle vs OnPaint), see [visual-rendering.md](visual-rendering.md). diff --git a/docs/design/threading.md b/docs/design/threading.md index 1285441..4b26211 100644 --- a/docs/design/threading.md +++ b/docs/design/threading.md @@ -137,27 +137,23 @@ Non-recursive spinlock using `std::atomic_flag` with acquire/release memory orde **File:** `src/IOThread.h` -`ReBuffer` is a buffer pool with reference-counted recycling, used to minimize allocation overhead in hot data paths (audio output, visualization). `getBuffer()` iterates all pooled `shared_ptr` entries — if `use_count() == 1`, the buffer is available for reuse. The first available buffer is selected (age reset to 1); other available buffers have their age decremented. GC only checks the last element in the deque: if its age drops below `-REBUFFER_GC_LIMIT` (i.e. below -100), it is popped. A `use_count() == 0` (dangling pointer) is treated as a bug and erased with a warning. Used by `DemodulatorThread` (audio output buffers) and `VisualDataReDistributor` (visualization buffers). +`ReBuffer` is a `shared_ptr`-based buffer pool for minimizing allocation overhead in hot data paths. The pool's `getBuffer()` method is protected by `SpinMutex`. For the full pool mechanics (GC thresholds, reuse logic, warning thresholds), see [signal-flow.md](signal-flow.md) "Buffer Management". ### VisualProcessor Pipeline **File:** `src/process/VisualProcessor.h` -`VisualProcessor` is a template base class for the visualization pipeline. Each processor has one input queue and N output queues. `process()` is called by the owning thread's main loop; `distribute()` pushes results to all attached outputs. Two concrete subclasses handle different distribution strategies: -- `VisualDataDistributor` — zero-copy shared pointer forwarding -- `VisualDataReDistributor` — deep-copy distribution via `ReBuffer` pooling - -Protected by `std::mutex busy_update` for queue list mutations. - -Two threading models exist for VisualProcessor subclasses: +`VisualProcessor` is a template base class for the visualization pipeline. Two threading models exist for VisualProcessor subclasses: - **Dedicated thread:** `SpectrumVisualProcessor` runs on `SpectrumVisualDataThread` (sleep-loop calling `sproc.run()` periodically). `FFTVisualDataThread` runs both `FFTDataDistributor` and `SpectrumVisualProcessor` on a single `IOThread`. - **UI thread:** `ScopeVisualProcessor` runs on the wxWidgets main thread via `AppFrame::handleScopeProcessor()` → `process()` (non-blocking `try_pop`). See Pattern 7. +For the full pipeline internals (VisualProcessor API, distribution modes, FFT processing, ScopeVisualProcessor processing), see [visual-data-pipeline.md](visual-data-pipeline.md). + ### SpectrumVisualProcessor busy_run **File:** `src/process/SpectrumVisualProcessor.h` -`spectrumVisualProcessor` uses `std::mutex busy_run` to serialize FFT computation against parameter changes. The mutex protects all internal state: FFT plan, buffers, averaging accumulators, resampler, frequency shifter, and configuration fields. All setter/getter methods (called from the UI thread) acquire this mutex. `process()` uses a two-phase locking pattern: a short-lived scoped lock checks and clears `fftSizeChanged` (then releases before calling `setup()` outside the lock), followed by an `input->pop()` also outside the lock, then re-acquires `busy_run` for the remainder of the FFT computation (lines 245 onward). This means UI parameter changes block until the current FFT completes, and vice versa, but input polling and setup are not held up by the computation mutex. Before any processing, `process()` checks `isOutputEmpty()` — if any output queue is full, the frame is dropped to apply back-pressure and prevent unbounded memory growth. +`spectrumVisualProcessor` uses `std::mutex busy_run` to serialize FFT computation against parameter changes. All setter/getter methods (called from the UI thread) acquire this mutex. For the two-phase locking algorithm narrative (`process()` internals), see [visual-data-pipeline.md](visual-data-pipeline.md) "SpectrumVisualProcessor — Full Per-Frame Pipeline". ### SDREnumerator One-Shot Spawning @@ -249,8 +245,8 @@ Windows and Linux use default thread priorities. The UI thread is **primarily pull-based**: 1. `AppFrame::OnIdle()` is called continuously by the wx event loop and handles device params, modem properties, and UI state -2. Each canvas registers its own `EVT_IDLE` handler independently (`AppFrame::OnIdle`, `WaterfallCanvas::OnIdle`, `SpectrumCanvas::OnIdle`, `ScopeCanvas::OnIdle`, `TuningCanvas::OnIdle`, `ModeSelectorCanvas::OnIdle`, `MeterCanvas::OnIdle`, `GainCanvas::OnIdle`) — `WaterfallCanvas::OnIdle` calls `processInputQueue()` (which internally `try_pop()`s); other canvases typically just call `Refresh()` and defer queue pops to `OnPaint()` -3. Visual data flows are pull-based: worker threads push to queues, UI canvases pull via `try_pop()` in `OnIdle()` (WaterfallCanvas) or `OnPaint()` (SpectrumCanvas, ScopeCanvas) +2. Each canvas registers its own `EVT_IDLE` handler independently — canvases poll their input queues via `try_pop()` in `OnIdle()` (WaterfallCanvas) or `OnPaint()` (SpectrumCanvas, ScopeCanvas). See [visual-rendering.md](visual-rendering.md) "OnIdle Processing" for per-canvas details. +3. Visual data flows are pull-based: worker threads push to queues, UI canvases pull via `try_pop()` 4. Shared state uses `std::atomic` variables (frequency, signal levels, mute state) 5. UI-initiated changes go through atomic variables and flags, not wx events 6. **Exception:** `SDRThread` triggers `refreshGainUI()` from the worker thread via `notifyMainUIOfDeviceChange()`, which rebuilds `GainCanvas` panels and triggers `Refresh()` — a cross-thread UI mutation outside the pull-based pattern diff --git a/docs/design/visual-data-pipeline.md b/docs/design/visual-data-pipeline.md new file mode 100644 index 0000000..c247021 --- /dev/null +++ b/docs/design/visual-data-pipeline.md @@ -0,0 +1,143 @@ +# Visual Data Pipeline + +This document describes CubicSDR's visual data processing pipeline: the VisualProcessor template, distribution modes, FFT/scope processing, and the thread safety model for visual data. For the rendering/UI layer (canvases, GLPanel, fonts, themes), see [visual-rendering.md](visual-rendering.md). + +## Pipeline Topology + +``` +SDRPostThread (produces IQ data into pipe queues owned by CubicSDR) + | + +--[pipeIQVisualData]--------> SpectrumVisualDataThread --> SpectrumCanvas + +--[pipeWaterfallIQVisualData]-> FFTVisualDataThread --> WaterfallCanvas + +--[pipeDemodIQVisualData]----> SpectrumVisualDataThread (demodVisualThread) --> DemodSpectrumCanvas + DemodWaterfallCanvas + +DemodulatorThread + | + +--[audioVisOutputQueue]------> ScopeVisualProcessor --> ScopeCanvas +``` + +`SpectrumVisualDataThread` (`src/process/SpectrumVisualDataThread.h`) is a dedicated IOThread that wraps a `SpectrumVisualProcessor` and runs it in its own thread. Two separate instances exist: one for the main spectrum (`spectrumVisualThread`) and one for the demod spectrum (`demodVisualThread`). `FFTVisualDataThread` wraps its own processing pipeline (containing an `FFTDataDistributor` and a `SpectrumVisualProcessor`) for waterfall data. + +For the queue wiring (which queues connect which threads, queue types, and capacity limits), see [signal-flow.md](signal-flow.md) "Queue Wiring". + +## VisualProcessor Template (`src/process/VisualProcessor.h`) + +Base template for all visual data processors. Implements a generic 1:N pipeline with thread-safe queue attachment: + +``` +InputQueue → process() → distribute() → OutputQueues[] +``` + +**Core API:** +- `setInput(queue)` — attach input queue (protected by `busy_update` mutex) +- `attachOutput(queue)` / `removeOutput(queue)` — manage output queue list +- `run()` — captures a local copy of input, calls `process()` if input is non-empty +- `process()` — pure virtual, implemented by subclasses +- `distribute(item, timeout, errorMessage)` — pushes output to all attached output queues (locked iteration) +- `flushQueues()` — flushes both input and all outputs +- `isInputEmpty()` / `isOutputEmpty()` / `isAnyOutputEmpty()` — query queue states. Note: `isOutputEmpty()` returns true only when **all** output queues have room for more data (i.e., all are not full). `isAnyOutputEmpty()` returns true when **any** single output has room. Processors use `isOutputEmpty()` to skip processing when any consumer is backed up (backpressure). + +**Synchronization:** `busy_update` (std::mutex) protects the `input` and `outputs` vectors. Processors call `isOutputEmpty()` before processing to avoid redundant work when consumers are backed up. + +## Distribution Modes + +Two distribution strategies handle different multicast patterns: + +**`VisualDataDistributor`** — Zero-copy shared dispatch. Pops each input item and pushes the same `shared_ptr` to all output queues. Stops pushing when all outputs are full (backpressure). Used when consumers only read the data. + +**`VisualDataReDistributor`** — Deep-copy dispatch via `ReBuffer` pool. Each output gets its own copy of the data, allocated from a pre-pooled buffer to avoid per-frame allocation. Used when consumers modify or consume the data independently. + +Both `VisualDataDistributor` and `VisualDataReDistributor` are defined inline in `VisualProcessor.h`. + +## FFTDataDistributor (`src/process/FFTDataDistributor.h`) + +Specialized rate-limited distributor for IQ-to-FFT batching. Inherits from `VisualProcessor` and implements: +- Rate limiting via `lineRateAccum` / `linesPerSecond` to control FFT execution pace +- Internal buffering with `bufferMax`, `bufferOffset`, `bufferedItems` to batch IQ packets into FFT-sized chunks +- Uses non-blocking `distribute()` push (unlike the blocking push in base `VisualProcessor`) +- This is the class used by `FFTVisualDataThread` (as `fftDistrib`), not `VisualDataDistributor` + +## SpectrumVisualProcessor — Full Per-Frame Pipeline + +The most complex processor. Converts raw IQ samples into display-ready spectrum points. + +**Input:** `DemodulatorThreadIQData` (IQ samples with sample rate, center frequency metadata) +**Output:** `SpectrumVisualData` (interleaved [x,y] spectrum points, floor/ceiling, metadata) + +**Processing sequence:** + +1. **Guard check** — Skip if any output queue is full (backpressure from consumers) or input is empty +2. **Pop IQ data** — Blocking pop with 50ms timeout (`HEARTBEAT_CHECK_PERIOD_MICROS`) +3. **View mode resampling** — If viewing a sub-band: + - Compute resample ratio from center frequency offset + - Frequency-shift using NCO (`nco_crcf_mix_block_up/down`) + - Resample to FFT input size (`msresamp_crcf_execute`) +4. **FFT execution** — `fft_execute(fftPlan)` (liquid-dsp FFT, internal size = `DEFAULT_FFT_SIZE * SPECTRUM_VZM` = 4096 points; user-facing display resolution = 2048 points) +5. **Magnitude computation** — `sqrt(real² + imag²)` with FFT shift (swap halves to center DC) +6. **Smoothing** — Double exponential moving average with NaN guards (note: `maa` is updated first, using the *previous* frame's `ma` value, then `ma` is updated from raw input): + - `if (fft_result_maa != fft_result_maa) fft_result_maa = fft_result` (NaN guard) + - `fft_result_maa += (fft_result_ma - fft_result_maa) * fft_average_rate` (uses old `ma`) + - `if (fft_result_ma != fft_result_ma) fft_result_ma = fft_result` (NaN guard) + - `fft_result_ma += (fft_result - fft_result_ma) * fft_average_rate` + - Initial `fft_average_rate = 0.65f` + + **Note:** `ScopeVisualProcessor` uses a different update order within each iteration — `ma` is updated first from raw input, then `maa` is updated using the *new* `ma`. In `SpectrumVisualProcessor`, the order is reversed: `maa` is updated first using the *old* `ma`, then `ma` is updated from raw input. +7. **Floor/ceiling tracking** — Slow-moving averages (0.05 rate) with NaN guards; peak hold if enabled +8. **Log-scale normalization** — Maps FFT bins to [0,1] spectrum points using floor/ceiling +9. **DC spike removal** — If `hideDC` enabled, interpolates over ±2kHz around DC +10. **Distribute** — Push `SpectrumVisualData` to all attached output queues + +**Key constants:** + +| Constant | Value | Purpose | +|----------|-------|---------| +| `HEARTBEAT_CHECK_PERIOD_MICROS` | 50,000 (50ms) | Input pop timeout | +| `DEFAULT_FFT_SIZE` | 2048 | User-facing FFT bin count | +| `SPECTRUM_VZM` | 2 | Internal FFT multiplier (actual FFT = `DEFAULT_FFT_SIZE * SPECTRUM_VZM` = 4096 points) | +| `PEAK_RESET_COUNT` | 30 | Frames before peak hold resets | +| `fft_average_rate` | 0.65f | Smoothing factor (higher = more responsive) | + +## FFTVisualDataThread — The Glue Thread + +A dedicated thread that bridges IQ data to the waterfall display: + +``` +pipeIQDataIn → fftDistrib → fftQueue → wproc → pipeFFTDataOut +``` + +The thread loop: +1. Sleep ~10ms between iterations +2. `fftDistrib.run()` — packages IQ data into FFT-ready batches (rate-limited by `FFT_DISTRIBUTOR_BUFFER_IN_SECONDS = 0.250s`) +3. `wproc.run()` — executes FFT processing in a tight loop until input is drained (one FFT per iteration) + +This thread bridges IQ data to the waterfall display by running the FFT distributor and processor in a tight loop. The FFT distributor batches incoming IQ packets into FFT-sized chunks, and the processor executes one FFT per batch. Note: while `FFTDataDistributor` supports multiple outputs, `FFTVisualDataThread` attaches only one (`fftQueue`). The multi-consumer distribution (to both waterfall and spectrum) happens at the `SDRPostThread` level, which attaches separate queues to each consumer. + +## ScopeVisualProcessor + +Processes demodulated audio for scope/spectrum display. + +**Input:** `AudioThreadInput` (audio samples with sample rate) +**Output:** `ScopeRenderData` (waveform points or FFT spectrum) + +**Display modes:** +- `SCOPE_MODE_Y` — Single-channel time waveform +- `SCOPE_MODE_2Y` — Dual-channel overlaid waveforms +- `SCOPE_MODE_XY` — Lissajous figure (phase display) +- Spectrum mode — FFT of demodulated audio (default 1024 points); controlled by a separate boolean flag (`renderData->spectrum`) rather than a fourth `ScopeMode` enum value + +Uses `try_pop` (non-blocking) instead of blocking pop, since audio data arrives at a fixed rate and stale data should be dropped. + +`ScopeVisualProcessor` runs on the wxWidgets main thread via `AppFrame::handleScopeProcessor()` (see [threading.md](threading.md) Pattern 7), not on a dedicated IOThread. + +## Thread Safety Summary + +The visual data pipeline uses two synchronization mechanisms for processor-internal state: + +| Mechanism | Protects | Used By | +|-----------|----------|---------| +| `busy_update` (std::mutex) | Input/output queue pointers | VisualProcessor base | +| `busy_run` (std::mutex) | All processor state (FFT buffers, settings) | SpectrumVisualProcessor | + +For the canonical lock inventory (including `SpinMutex`, `ThreadBlockingQueue`, `std::shared_ptr`, and all other synchronization mechanisms in the codebase), see [threading.md](threading.md) "Synchronization Mechanisms". + +For the `ReBuffer` pool mechanics (used by `VisualDataReDistributor` for deep-copy distribution), see [signal-flow.md](signal-flow.md) "Buffer Management". diff --git a/docs/design/visual-architecture.md b/docs/design/visual-rendering.md similarity index 79% rename from docs/design/visual-architecture.md rename to docs/design/visual-rendering.md index 34a586a..7e43628 100644 --- a/docs/design/visual-architecture.md +++ b/docs/design/visual-rendering.md @@ -1,24 +1,12 @@ -# Visual Architecture +# Visual Rendering -This document describes CubicSDR's OpenGL rendering system, canvas hierarchy, font rendering, color themes, and the visual data processing pipeline. +This document describes CubicSDR's OpenGL rendering system, canvas hierarchy, GL panel system, font rendering, color themes, and rendering flow. For the visual data processing pipeline (VisualProcessor, distributors, FFT processing), see [visual-data-pipeline.md](visual-data-pipeline.md). ## Overview -The visual system is built on wxWidgets' OpenGL integration (`wxGLCanvas`/`wxGLContext`) and uses a **pull-based** architecture: worker threads produce FFT/scope data into queues, and UI canvases poll those queues in their `OnIdle()` handlers. Worker threads never push data to the UI. +The visual system is built on wxWidgets' OpenGL integration (`wxGLCanvas`/`wxGLContext`) and uses a **pull-based** architecture: worker threads produce FFT/scope data into queues, and UI canvases poll those queues in their `OnIdle()` or `OnPaint()` handlers. Worker threads never push data to the UI. -``` -SDRPostThread (produces IQ data into pipe queues owned by CubicSDR) - | - +--[pipeIQVisualData]--------> SpectrumVisualDataThread --> SpectrumCanvas - +--[pipeWaterfallIQVisualData]-> FFTVisualDataThread --> WaterfallCanvas - +--[pipeDemodIQVisualData]----> SpectrumVisualDataThread (demodVisualThread) --> DemodSpectrumCanvas + DemodWaterfallCanvas - -DemodulatorThread - | - +--[audioVisOutputQueue]------> ScopeVisualProcessor --> ScopeCanvas -``` - -`SpectrumVisualDataThread` (`src/process/SpectrumVisualDataThread.h`) is a dedicated IOThread that wraps a `SpectrumVisualProcessor` and runs it in its own thread. Two separate instances exist: one for the main spectrum (`spectrumVisualThread`) and one for the demod spectrum (`demodVisualThread`). `FFTVisualDataThread` wraps its own processing pipeline (containing an `FFTDataDistributor` and a `SpectrumVisualProcessor`) for waterfall data. +The visual data processing pipeline — how raw IQ samples become display-ready spectrum/scope data, the VisualProcessor template, and the FFT/scope processing internals — is documented in [visual-data-pipeline.md](visual-data-pipeline.md). ## Canvas Class Hierarchy @@ -432,123 +420,6 @@ Global singleton `ThemeMgr::mgr` manages theme selection: - `themes` map holds all `ColorTheme*` instances - Current theme is persisted in `AppConfig` -## Visual Data Processing Pipeline - -### VisualProcessor Template (`src/process/VisualProcessor.h`) - -Base template for all visual data processors. Implements a generic 1:N pipeline with thread-safe queue attachment: - -``` -InputQueue → process() → distribute() → OutputQueues[] -``` - -**Core API:** -- `setInput(queue)` — attach input queue (protected by `busy_update` mutex) -- `attachOutput(queue)` / `removeOutput(queue)` — manage output queue list -- `run()` — captures a local copy of input, calls `process()` if input is non-empty -- `process()` — pure virtual, implemented by subclasses -- `distribute(item, timeout, errorMessage)` — pushes output to all attached output queues (locked iteration) -- `flushQueues()` — flushes both input and all outputs -- `isInputEmpty()` / `isOutputEmpty()` / `isAnyOutputEmpty()` — query queue states. Note: `isOutputEmpty()` returns true only when **all** output queues have room for more data (i.e., all are not full). `isAnyOutputEmpty()` returns true when **any** single output has room. Processors use `isOutputEmpty()` to skip processing when any consumer is backed up (backpressure). - -**Synchronization:** `busy_update` (std::mutex) protects the `input` and `outputs` vectors. Processors call `isOutputEmpty()` before processing to avoid redundant work when consumers are backed up. - -### Distribution Modes - -Two distribution strategies handle different multicast patterns: - -**`VisualDataDistributor`** — Zero-copy shared dispatch. Pops each input item and pushes the same `shared_ptr` to all output queues. Stops pushing when all outputs are full (backpressure). Used when consumers only read the data. - -**`VisualDataReDistributor`** — Deep-copy dispatch via `ReBuffer` pool. Each output gets its own copy of the data, allocated from a pre-pooled buffer to avoid per-frame allocation. Used when consumers modify or consume the data independently. - -Both `VisualDataDistributor` and `VisualDataReDistributor` are defined inline in `VisualProcessor.h`. - -**`FFTDataDistributor`** (`src/process/FFTDataDistributor.h`) — Specialized rate-limited distributor for IQ-to-FFT batching. Inherits from `VisualProcessor` and implements: -- Rate limiting via `lineRateAccum` / `linesPerSecond` to control FFT execution pace -- Internal buffering with `bufferMax`, `bufferOffset`, `bufferedItems` to batch IQ packets into FFT-sized chunks -- Uses non-blocking `distribute()` push (unlike the blocking push in base `VisualProcessor`) -- This is the class used by `FFTVisualDataThread` (as `fftDistrib`), not `VisualDataDistributor` - -### SpectrumVisualProcessor — Full Per-Frame Pipeline - -The most complex processor. Converts raw IQ samples into display-ready spectrum points. - -**Input:** `DemodulatorThreadIQData` (IQ samples with sample rate, center frequency metadata) -**Output:** `SpectrumVisualData` (interleaved [x,y] spectrum points, floor/ceiling, metadata) - -**Processing sequence:** - -1. **Guard check** — Skip if any output queue is full (backpressure from consumers) or input is empty -2. **Pop IQ data** — Blocking pop with 50ms timeout (`HEARTBEAT_CHECK_PERIOD_MICROS`) -3. **View mode resampling** — If viewing a sub-band: - - Compute resample ratio from center frequency offset - - Frequency-shift using NCO (`nco_crcf_mix_block_up/down`) - - Resample to FFT input size (`msresamp_crcf_execute`) -4. **FFT execution** — `fft_execute(fftPlan)` (liquid-dsp FFT, internal size = `DEFAULT_FFT_SIZE * SPECTRUM_VZM` = 4096 points; user-facing display resolution = 2048 points) -5. **Magnitude computation** — `sqrt(real² + imag²)` with FFT shift (swap halves to center DC) -6. **Smoothing** — Double exponential moving average with NaN guards (note: `maa` is updated first, using the *previous* frame's `ma` value, then `ma` is updated from raw input): - - `if (fft_result_maa != fft_result_maa) fft_result_maa = fft_result` (NaN guard) - - `fft_result_maa += (fft_result_ma - fft_result_maa) * fft_average_rate` (uses old `ma`) - - `if (fft_result_ma != fft_result_ma) fft_result_ma = fft_result` (NaN guard) - - `fft_result_ma += (fft_result - fft_result_ma) * fft_average_rate` - - Initial `fft_average_rate = 0.65f` - - **Note:** `ScopeVisualProcessor` uses a different update order within each iteration — `ma` is updated first from raw input, then `maa` is updated using the *new* `ma`. In `SpectrumVisualProcessor`, the order is reversed: `maa` is updated first using the *old* `ma`, then `ma` is updated from raw input. -7. **Floor/ceiling tracking** — Slow-moving averages (0.05 rate) with NaN guards; peak hold if enabled -8. **Log-scale normalization** — Maps FFT bins to [0,1] spectrum points using floor/ceiling -9. **DC spike removal** — If `hideDC` enabled, interpolates over ±2kHz around DC -10. **Distribute** — Push `SpectrumVisualData` to all attached output queues - -**Key constants:** - -| Constant | Value | Purpose | -|----------|-------|---------| -| `HEARTBEAT_CHECK_PERIOD_MICROS` | 50,000 (50ms) | Input pop timeout | -| `DEFAULT_FFT_SIZE` | 2048 | User-facing FFT bin count | -| `SPECTRUM_VZM` | 2 | Internal FFT multiplier (actual FFT = `DEFAULT_FFT_SIZE * SPECTRUM_VZM` = 4096 points) | -| `PEAK_RESET_COUNT` | 30 | Frames before peak hold resets | -| `fft_average_rate` | 0.65f | Smoothing factor (higher = more responsive) | - -### FFTVisualDataThread — The Glue Thread - -A dedicated thread that bridges IQ data to the waterfall display: - -``` -pipeIQDataIn → fftDistrib → fftQueue → wproc → pipeFFTDataOut -``` - -The thread loop: -1. Sleep ~10ms between iterations -2. `fftDistrib.run()` — packages IQ data into FFT-ready batches (rate-limited by `FFT_DISTRIBUTOR_BUFFER_IN_SECONDS = 0.250s`) -3. `wproc.run()` — executes FFT processing in a tight loop until input is drained (one FFT per iteration) - -This thread bridges IQ data to the waterfall display by running the FFT distributor and processor in a tight loop. The FFT distributor batches incoming IQ packets into FFT-sized chunks, and the processor executes one FFT per batch. Note: while `FFTDataDistributor` supports multiple outputs, `FFTVisualDataThread` attaches only one (`fftQueue`). The multi-consumer distribution (to both waterfall and spectrum) happens at the `SDRPostThread` level, which attaches separate queues to each consumer. - -### ScopeVisualProcessor - -Processes demodulated audio for scope/spectrum display. - -**Input:** `AudioThreadInput` (audio samples with sample rate) -**Output:** `ScopeRenderData` (waveform points or FFT spectrum) - -**Display modes:** -- `SCOPE_MODE_Y` — Single-channel time waveform -- `SCOPE_MODE_2Y` — Dual-channel overlaid waveforms -- `SCOPE_MODE_XY` — Lissajous figure (phase display) -- Spectrum mode — FFT of demodulated audio (default 1024 points); controlled by a separate boolean flag (`renderData->spectrum`) rather than a fourth `ScopeMode` enum value - -Uses `try_pop` (non-blocking) instead of blocking pop, since audio data arrives at a fixed rate and stale data should be dropped. - -### Thread Safety Summary - -| Mechanism | Protects | Used By | -|-----------|----------|---------| -| `busy_update` (std::mutex) | Input/output queue pointers | VisualProcessor base | -| `busy_run` (std::mutex) | All processor state (FFT buffers, settings) | SpectrumVisualProcessor | -| `SpinMutex` (atomic flag) | ReBuffer pool allocation | ReBuffer::getBuffer() | -| `ThreadBlockingQueue` | Inter-thread data transfer | All queue connections | -| `std::shared_ptr` | Data object lifetime across threads | All data objects | - ## Rendering Flow ### Per-Frame Rendering (WaterfallCanvas) @@ -665,6 +536,8 @@ WaterfallCanvas (consumer, UI thread) Display (~30-60 FPS depending on system) ``` +For details on the visual data processing pipeline internals (VisualProcessor, distributors, FFT processing, ScopeVisualProcessor processing logic), see [visual-data-pipeline.md](visual-data-pipeline.md). + ## Mouse Interaction ### MouseTracker (`src/util/MouseTracker.h`)