Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 5e05938457 | |||
| caccf91b7e | |||
| bd582c0d81 | |||
| e593c3fed9 |
@@ -1,350 +0,0 @@
|
|||||||
# Audio Logging Design
|
|
||||||
|
|
||||||
This document defines how the Hub writes the audio it collects from the
|
|
||||||
Nodes to the SD card. It is the contract that the future Node-collection task
|
|
||||||
(producer) and the existing `StorageTask` (consumer) are written against.
|
|
||||||
|
|
||||||
The deliverable for this design is:
|
|
||||||
|
|
||||||
* `src/storage/data_logger.h` - the public interface (skeleton).
|
|
||||||
* `src/storage/data_logger.cpp` - stub bodies + fully implemented WAV metadata
|
|
||||||
helpers. The chunk-pool/queue/file logic is marked `TODO` and is the
|
|
||||||
remaining work.
|
|
||||||
* The constants in `src/storage/storage_config.h` under the "Audio logging"
|
|
||||||
section.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Data model
|
|
||||||
|
|
||||||
| Parameter | Value |
|
|
||||||
|-----------------------|------------|
|
|
||||||
| Sample rate | 48 kHz |
|
|
||||||
| Bit depth | 16 bit (signed little-endian PCM) |
|
|
||||||
| Microphones per node | 4 |
|
|
||||||
| Max nodes | 10 |
|
|
||||||
| **Total channels** | **40** |
|
|
||||||
|
|
||||||
One microphone == one channel == one "track". A single file holds all of them
|
|
||||||
as an interleaved multichannel WAV.
|
|
||||||
|
|
||||||
## 2. Throughput budget
|
|
||||||
|
|
||||||
```
|
|
||||||
raw stream rate = 40 ch x 48 kHz x 2 B = 3,840,000 B/s (3.66 MiB/s)
|
|
||||||
```
|
|
||||||
|
|
||||||
* SDIO 4-bit @ 40 MHz sustains roughly 8-12 MB/s -> the card write has ample
|
|
||||||
headroom at less than half of its budget.
|
|
||||||
* WiFi (AP) carries the same 3.84 MB/s inbound as UDP payload, which the
|
|
||||||
design already budgets for.
|
|
||||||
* The 128 KB chunk pool represents **~34 ms** of audio. The write must be able
|
|
||||||
to absorb bursts longer than that only by dropping (Section 8).
|
|
||||||
|
|
||||||
## 3. File format: single multichannel WAV
|
|
||||||
|
|
||||||
The stream is a **RIFF/WAVE file with 40 interleaved channels**. The 44-byte
|
|
||||||
header is the only metadata; no per-chunk headers are written to the file.
|
|
||||||
|
|
||||||
Header layout (all little-endian):
|
|
||||||
|
|
||||||
| Offset | Size | Value |
|
|
||||||
|--------|------|--------------------------------------------|
|
|
||||||
| 0 | 4 | `"RIFF"` |
|
|
||||||
| 4 | 4 | chunk size = file size - 8 (patched) |
|
|
||||||
| 8 | 4 | `"WAVE"` |
|
|
||||||
| 12 | 4 | `"fmt "` |
|
|
||||||
| 16 | 4 | 16 (PCM fmt chunk size) |
|
|
||||||
| 20 | 2 | 1 (PCM) |
|
|
||||||
| 22 | 2 | 40 (channels) |
|
|
||||||
| 24 | 4 | 48000 (sample rate) |
|
|
||||||
| 28 | 4 | 3,840,000 (byte rate = rate x blockAlign) |
|
|
||||||
| 32 | 2 | 80 (block align = channels x 2) |
|
|
||||||
| 34 | 2 | 16 (bits per sample) |
|
|
||||||
| 36 | 4 | `"data"` |
|
|
||||||
| 40 | 4 | data size = file size - 44 (patched) |
|
|
||||||
|
|
||||||
`buildWavHeader()` and `finalizeWavHeader()` in `data_logger.h/cpp` produce
|
|
||||||
these bytes and the two 32-bit values patched at offsets 4 and 40.
|
|
||||||
|
|
||||||
### Header finalization
|
|
||||||
|
|
||||||
The sizes at offsets 4 and 40 are unknown while the file is being written.
|
|
||||||
They are patched when the file is closed (rotation or session end):
|
|
||||||
|
|
||||||
```
|
|
||||||
file.seek(4); file.write(riffSize, 4);
|
|
||||||
file.seek(40); file.write(dataSize, 4);
|
|
||||||
```
|
|
||||||
|
|
||||||
### Crash tolerance
|
|
||||||
|
|
||||||
If power is lost mid-file, the header still contains the placeholder sizes.
|
|
||||||
PCM decoders that read to EOF (Audacity, ffmpeg, Python `wave`) play the valid
|
|
||||||
audio regardless; the last partial sector may be zero-padded. This satisfies
|
|
||||||
the design.md requirement that a mid-recording power loss still yields usable
|
|
||||||
audio.
|
|
||||||
|
|
||||||
### Opening the file on a PC
|
|
||||||
|
|
||||||
* **Audacity** / **ffmpeg** / **Python `wave`**: handle 40 channels.
|
|
||||||
* VLC and stock Windows players generally will not render 40 channels even
|
|
||||||
though the file is structurally valid. Convert with ffmpeg first if needed.
|
|
||||||
|
|
||||||
## 4. Architecture: producer / consumer with a chunk pool
|
|
||||||
|
|
||||||
```
|
|
||||||
Node collection task (core 1) StorageTask (core 0)
|
|
||||||
assembles one round of 40 channels | loop:
|
|
||||||
interleaved into frame order v
|
|
||||||
+----------------------+ +----------------------+
|
|
||||||
| c = acquireChunk(5ms)| <-- freeQ ---- | pool: 8 x 16 KB |
|
|
||||||
| fill c->data[...] | | (MALLOC_CAP_DMA) |
|
|
||||||
| submitChunk(c) | ---- filledQ -->| nextChunk(1000ms) |
|
|
||||||
+----------------------+ | writeChunk(c) |
|
|
||||||
| releaseChunk(c) |
|
|
||||||
| rotateIfNeeded() |
|
|
||||||
| updateStats() |
|
|
||||||
+----------------------+
|
|
||||||
|
|
||||||
DataLogger owns both queues and the pool. Producers and the consumer never
|
|
||||||
touch the file or the card directly - only DataLogger does.
|
|
||||||
```
|
|
||||||
|
|
||||||
* **Pool**: 8 chunks x 16 KB = 128 KB, allocated once in `begin()` with
|
|
||||||
`heap_caps_malloc(MALLOC_CAP_DMA)`.
|
|
||||||
* **Queues**: `freeQ` holds pointers to empty chunks, `filledQ` holds pointers
|
|
||||||
to chunks waiting to be written. Both are FreeRTOS queues of
|
|
||||||
`AudioChunk*`.
|
|
||||||
* **Cores**: the producer runs on core 1 (network), the consumer on core 0
|
|
||||||
(the existing pinned `StorageTask`). SD card work never touches core 1.
|
|
||||||
|
|
||||||
## 5. The interface
|
|
||||||
|
|
||||||
Types (from `data_logger.h`):
|
|
||||||
|
|
||||||
```cpp
|
|
||||||
struct AudioChunk
|
|
||||||
{
|
|
||||||
uint8_t* data; // DMA-capable buffer owned by the pool
|
|
||||||
uint32_t capacity; // buffer size in bytes (STORAGE_LOG_CHUNK_SIZE)
|
|
||||||
uint32_t length; // valid bytes written by the producer
|
|
||||||
uint32_t sequence; // monotonic order, for diagnostics only
|
|
||||||
};
|
|
||||||
|
|
||||||
class DataLogger
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
bool begin(fs::FS& files); // allocate pool, create log dir
|
|
||||||
void end(); // finalize current file, free pool
|
|
||||||
|
|
||||||
bool openSession(); // create rec_<uptime>_<n>.wav
|
|
||||||
bool closeSession(); // finalize + close current file
|
|
||||||
bool rotateIfNeeded(); // called by the consumer
|
|
||||||
|
|
||||||
// Producer (core 1)
|
|
||||||
AudioChunk* acquireChunk(TickType_t timeout);
|
|
||||||
void submitChunk(AudioChunk* chunk);
|
|
||||||
|
|
||||||
// Consumer (core 0)
|
|
||||||
AudioChunk* nextChunk(TickType_t timeout);
|
|
||||||
bool writeChunk(AudioChunk* chunk);
|
|
||||||
void releaseChunk(AudioChunk* chunk);
|
|
||||||
|
|
||||||
// Stats
|
|
||||||
uint32_t bytesWritten() const;
|
|
||||||
uint32_t chunksWritten() const;
|
|
||||||
uint32_t droppedChunks() const;
|
|
||||||
uint32_t writeSpeedBps() const;
|
|
||||||
bool overflowing() const;
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
### Producer contract (Node collection task, not yet written)
|
|
||||||
|
|
||||||
```cpp
|
|
||||||
AudioChunk* c = logger.acquireChunk(pdMS_TO_TICKS(5));
|
|
||||||
if (c == nullptr) { /* collection is behind; drop this round */ return; }
|
|
||||||
|
|
||||||
// Fill c->data[0..c->length) with interleaved PCM, frame order:
|
|
||||||
// frame0: ch0..ch39, frame1: ch0..ch39, ...
|
|
||||||
// 40 samples of 2 bytes per frame = 80 bytes per frame.
|
|
||||||
c->length = interleavedBytes;
|
|
||||||
c->sequence = nextSeq++;
|
|
||||||
logger.submitChunk(c);
|
|
||||||
```
|
|
||||||
|
|
||||||
* Chunks **must be submitted in strict stream order** (single producer, FIFO
|
|
||||||
queue -> order is preserved automatically).
|
|
||||||
* A chunk is not required to end on a frame boundary, but `c->length` should
|
|
||||||
stay a multiple of 80 bytes so the file is always frame-aligned.
|
|
||||||
* `acquireChunk` can return a chunk even when the pool is momentarily empty;
|
|
||||||
see the drop-oldest rule in Section 8.
|
|
||||||
|
|
||||||
### Consumer contract (StorageTask on core 0, future)
|
|
||||||
|
|
||||||
```cpp
|
|
||||||
while (true)
|
|
||||||
{
|
|
||||||
AudioChunk* c = logger.nextChunk(pdMS_TO_TICKS(1000));
|
|
||||||
if (c != nullptr)
|
|
||||||
{
|
|
||||||
logger.writeChunk(c); // one file.write(c->data, c->length)
|
|
||||||
logger.releaseChunk(c); // returns the chunk to freeQ
|
|
||||||
}
|
|
||||||
logger.rotateIfNeeded();
|
|
||||||
logger.updateStats(storageState); // bps + drops -> dashboard
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
* The 1 s timeout lets the loop pulse for stats/capacity updates even when no
|
|
||||||
audio is flowing (this replaces the current 1 s `vTaskDelayUntil` heartbeat).
|
|
||||||
* `writeChunk` is the only place the card is touched. It appends the chunk's
|
|
||||||
PCM directly to the WAV data section in one `file.write`.
|
|
||||||
|
|
||||||
### Ownership rules
|
|
||||||
|
|
||||||
* The pool owns the buffers; producers and the consumer borrow them.
|
|
||||||
* A borrowed chunk is in exactly one place at a time: a producer (between
|
|
||||||
acquire and submit), a queue, or the consumer (between next and release).
|
|
||||||
* `releaseChunk` returns it to `freeQ`. `writeChunk` never frees.
|
|
||||||
|
|
||||||
## 6. Upstream interleaving contract
|
|
||||||
|
|
||||||
The WAV must be written frame-interleaved, but the Nodes deliver one UDP
|
|
||||||
packet per Node (4 contiguous mono channels). Frames cannot be interleaved
|
|
||||||
until all Nodes in a round have been collected. That re-ordering is the job of
|
|
||||||
the collection task, **not** the logger:
|
|
||||||
|
|
||||||
1. Collect all 10 Node dumps for round `n`.
|
|
||||||
2. A missing/offline Node contributes silence: zero-fill its 4 channels.
|
|
||||||
3. Re-order into frame order: `frame j = node0[mic0..3], node1[mic0..3], ...`
|
|
||||||
4. Feed the resulting bytes into chunks and submit them in order.
|
|
||||||
|
|
||||||
Consequences the collection task must honor:
|
|
||||||
|
|
||||||
* **Round size**: with the 128 KB pool, a round must stay <= ~96 KB
|
|
||||||
(~25 ms of audio) so the pool can hold more than one round. See the sizing
|
|
||||||
math in Section 7.
|
|
||||||
* **Silence for offline Nodes** comes from this zero-fill; the logger never
|
|
||||||
invents data.
|
|
||||||
|
|
||||||
## 7. Chunk pool and RAM budget
|
|
||||||
|
|
||||||
```
|
|
||||||
chunk size = STORAGE_LOG_CHUNK_SIZE = 16 KiB
|
|
||||||
pool depth = STORAGE_LOG_POOL_SIZE = 8
|
|
||||||
pool total = 128 KiB (MALLOC_CAP_DMA)
|
|
||||||
|
|
||||||
buffering = 128 KiB / 3.84 MB/s ~= 34 ms
|
|
||||||
round size = 48 kHz x 2 B x 40 ch x round_s
|
|
||||||
25 ms round -> 96 KB (~6 chunks) [fits the pool with slack]
|
|
||||||
50 ms round -> 192 KB [does NOT fit - must shrink]
|
|
||||||
```
|
|
||||||
|
|
||||||
The collection round duration is therefore bounded by the pool unless the pool
|
|
||||||
grows. Keep rounds at <= 25 ms, or raise `STORAGE_LOG_POOL_SIZE`/chunk size and
|
|
||||||
re-run the math.
|
|
||||||
|
|
||||||
The round's interleave buffer lives in the **collection task's own RAM** (up
|
|
||||||
to 96 KB), not in the logger pool; it is freed after the round is submitted.
|
|
||||||
|
|
||||||
## 8. Overflow: drop-oldest + warning
|
|
||||||
|
|
||||||
If the SD card cannot keep up, the pool drains and the producer has no chunk.
|
|
||||||
Policy (chosen): **drop the oldest buffered chunk until caught up** - never
|
|
||||||
block, never halt.
|
|
||||||
|
|
||||||
Mechanism inside `acquireChunk`: on timeout, the logger pops one chunk off the
|
|
||||||
back of `filledQ` (the oldest unwritten data), returns it to `freeQ`, hands it
|
|
||||||
to the producer, increments `droppedChunks`, and raises the overflow warning.
|
|
||||||
The stream keeps flowing with the newest data at the cost of a gap.
|
|
||||||
|
|
||||||
The warning is visible two ways:
|
|
||||||
|
|
||||||
* **Serial**: a rate-limited `[Logger] X chunks dropped, Y MB behind` line.
|
|
||||||
* **LED on GPIO `STORAGE_WARN_LED_GPIO`** (default 4, active-high): on while
|
|
||||||
`overflowing()`, off once the queue drains below a low-water mark again.
|
|
||||||
|
|
||||||
## 9. Rotation, naming, flush
|
|
||||||
|
|
||||||
* **Rotate by size**: when `bytesThisFile >= STORAGE_LOG_ROTATE_BYTES`
|
|
||||||
(default 1 GiB, ~4.6 min at 3.84 MB/s) the consumer calls `rotateIfNeeded()`:
|
|
||||||
finalize + close the current file, open the next.
|
|
||||||
* **Files**: created under `STORAGE_LOG_DIR` (`/sdcard/audio`), named
|
|
||||||
`rec_<uptimeSeconds>_<n>.wav`. `n` increments per rotation within a boot;
|
|
||||||
the uptime prefix keeps names unique across boots. If a name already exists,
|
|
||||||
skip forward until it does not (never overwrite).
|
|
||||||
* **Flush**: `file.flush()` (f_sync) every `STORAGE_LOG_FLUSH_BYTES`
|
|
||||||
(default 16 MiB, ~4.3 s) so an unclean power-off loses at most that window
|
|
||||||
and never corrupts earlier data.
|
|
||||||
* **Card full**: rotation cannot create a file -> report a fatal error, light
|
|
||||||
the warning LED solid, and halt (matches design.md's SD-failure stance).
|
|
||||||
|
|
||||||
## 10. DMA requirements
|
|
||||||
|
|
||||||
The SDIO path (production) uses the SDMMC controller's internal IDMA engine:
|
|
||||||
block data moves to the card without CPU cycles. Two rules make this work:
|
|
||||||
|
|
||||||
* Buffers must be in DMA-capable memory: allocate the pool with
|
|
||||||
`heap_caps_malloc(MALLOC_CAP_DMA)` (guarantees internal DRAM + alignment).
|
|
||||||
* Never hand the card a buffer that lives in PSRAM or a stack array.
|
|
||||||
|
|
||||||
The SPI path (`STORAGE_IFACE_SPI`, current bring-up) does **not** use DMA -
|
|
||||||
the Arduino SPI driver busy-waits the FIFO. It is bring-up only and cannot
|
|
||||||
sustain the 3.84 MB/s target; production logging must run on SDIO.
|
|
||||||
|
|
||||||
## 11. Metering -> dashboard
|
|
||||||
|
|
||||||
`writeChunk` accumulates `bytesWritten`/`chunksWritten`; `writeSpeedBps` is
|
|
||||||
derived from a sliding 1 s window of real writes. The consumer publishes
|
|
||||||
`writeSpeedBps` and `droppedChunks` into `StorageState` each loop, replacing
|
|
||||||
the boot-time `measureWriteSpeed()` benchmark (whose interval is already 0).
|
|
||||||
Add a `droppedChunks` field to `StorageSnapshot` and surface it on the
|
|
||||||
dashboard when the logging loop lands.
|
|
||||||
|
|
||||||
## 12. Future integration points (StorageTask)
|
|
||||||
|
|
||||||
When the consumer loop is implemented, `StorageTask::run()`:
|
|
||||||
|
|
||||||
1. After mount: `logger.begin(storage.fs())`, `logger.openSession()`.
|
|
||||||
2. Replace the 1 s heartbeat loop with the consumer loop in Section 5.
|
|
||||||
3. Stop calling `measureWriteSpeed()` (the logger provides real bps).
|
|
||||||
|
|
||||||
## 13. SPI -> SDIO migration checklist
|
|
||||||
|
|
||||||
Already documented in `src/storage/storage_config.h`. Restated for logging:
|
|
||||||
|
|
||||||
1. `STORAGE_IFACE` SPI -> SDMMC.
|
|
||||||
2. Wire SD to `STORAGE_SDMMC_*` pins (freely re-routable on classic ESP32).
|
|
||||||
3. Keep 4-bit mode (`STORAGE_SDMMC_MODE_1BIT == false`) - 1-bit halves the
|
|
||||||
throughput margin.
|
|
||||||
4. `MALLOC_CAP_DMA` buffers work unchanged; nothing else moves.
|
|
||||||
|
|
||||||
## 14. Open TODOs (to finish the feature)
|
|
||||||
|
|
||||||
- [ ] `data_logger.cpp`: chunk pool + queue creation in `begin()`.
|
|
||||||
- [ ] `data_logger.cpp`: `acquireChunk` drop-oldest path + overflow warning
|
|
||||||
(Serial + LED on `STORAGE_WARN_LED_GPIO`).
|
|
||||||
- [ ] `data_logger.cpp`: `writeChunk`/`rotateIfNeeded`/`closeSession` file
|
|
||||||
handling + header finalization + flush cadence.
|
|
||||||
- [ ] `data_logger.cpp`: `writeSpeedBps` sliding window.
|
|
||||||
- [ ] `StorageTask`: consumer loop (Section 5).
|
|
||||||
- [ ] `StorageState`/dashboard: `droppedChunks` field.
|
|
||||||
- [ ] Node collection task: round assembly + interleave + zero-fill
|
|
||||||
(Section 6).
|
|
||||||
|
|
||||||
## Appendix: constants
|
|
||||||
|
|
||||||
| Constant | Default | Meaning |
|
|
||||||
|-----------------------------------|--------------|---------------------------------|
|
|
||||||
| `STORAGE_AUDIO_SAMPLE_RATE_HZ` | 48000 | WAV sample rate |
|
|
||||||
| `STORAGE_AUDIO_CHANNELS` | 40 | = nodes x mics, WAV channels |
|
|
||||||
| `STORAGE_AUDIO_BITS` | 16 | WAV bit depth |
|
|
||||||
| `STORAGE_LOG_CHUNK_SIZE` | 16 * 1024 | pool chunk size (bytes) |
|
|
||||||
| `STORAGE_LOG_POOL_SIZE` | 8 | pool chunk count |
|
|
||||||
| `STORAGE_LOG_DIR` | "/sdcard/audio" | recording directory |
|
|
||||||
| `STORAGE_LOG_ROTATE_BYTES` | 1 GiB | rotate when a file reaches this |
|
|
||||||
| `STORAGE_LOG_FLUSH_BYTES` | 16 MiB | f_sync cadence |
|
|
||||||
| `STORAGE_WARN_LED_GPIO` | 4 | overflow warning LED |
|
|
||||||
| `STORAGE_WARN_LED_ACTIVE_HIGH` | true | LED polarity |
|
|
||||||
@@ -11,7 +11,13 @@ void DiagnosticsState::update()
|
|||||||
ESP.getMinFreeHeap();
|
ESP.getMinFreeHeap();
|
||||||
|
|
||||||
current.cpuFrequency =
|
current.cpuFrequency =
|
||||||
ESP.getCpuFreqMHz();
|
ESP.getCpuFreqMHz();
|
||||||
|
|
||||||
|
Serial.print("#update: ");
|
||||||
|
Serial.println(current.freeHeap);
|
||||||
|
Serial.println("Calling current from update");
|
||||||
|
getCurrent();
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -23,5 +29,9 @@ DiagnosticSample DiagnosticsState::getCurrent()
|
|||||||
sample.freeHeap = current.freeHeap;
|
sample.freeHeap = current.freeHeap;
|
||||||
sample.minimumFreeHeap = current.minimumFreeHeap;
|
sample.minimumFreeHeap = current.minimumFreeHeap;
|
||||||
sample.cpuFrequency = current.cpuFrequency;
|
sample.cpuFrequency = current.cpuFrequency;
|
||||||
|
|
||||||
|
Serial.print("#getCurrent: ");
|
||||||
|
Serial.println(sample.freeHeap);
|
||||||
|
Serial.println(sample.timestamp);
|
||||||
return sample;
|
return sample;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,67 +0,0 @@
|
|||||||
#include "storage_state.h"
|
|
||||||
|
|
||||||
|
|
||||||
void StorageState::begin()
|
|
||||||
{
|
|
||||||
current.mounted = false;
|
|
||||||
current.timestamp = 0;
|
|
||||||
current.totalMB = 0;
|
|
||||||
current.usedMB = 0;
|
|
||||||
current.freeMB = 0;
|
|
||||||
current.writeSpeedBps = 0;
|
|
||||||
current.cardType[0] = '\0';
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void StorageState::setMounted(bool mounted)
|
|
||||||
{
|
|
||||||
current.mounted = mounted;
|
|
||||||
current.timestamp = millis();
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void StorageState::setCardType(const char* type)
|
|
||||||
{
|
|
||||||
size_t i = 0;
|
|
||||||
for (i = 0; i < sizeof(current.cardType) - 1 && type[i] != '\0'; i++)
|
|
||||||
{
|
|
||||||
current.cardType[i] = type[i];
|
|
||||||
}
|
|
||||||
current.cardType[i] = '\0';
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void StorageState::setCapacity(uint64_t totalBytes, uint64_t usedBytes)
|
|
||||||
{
|
|
||||||
uint64_t freeBytes = (totalBytes > usedBytes) ? (totalBytes - usedBytes) : 0;
|
|
||||||
|
|
||||||
current.totalMB = (uint32_t)(totalBytes >> 20);
|
|
||||||
current.usedMB = (uint32_t)(usedBytes >> 20);
|
|
||||||
current.freeMB = (uint32_t)(freeBytes >> 20);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void StorageState::setWriteSpeedBps(uint32_t bytesPerSecond)
|
|
||||||
{
|
|
||||||
current.writeSpeedBps = bytesPerSecond;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
StorageSnapshot StorageState::getCurrent()
|
|
||||||
{
|
|
||||||
StorageSnapshot snapshot;
|
|
||||||
|
|
||||||
snapshot.mounted = current.mounted;
|
|
||||||
snapshot.timestamp = current.timestamp;
|
|
||||||
snapshot.totalMB = current.totalMB;
|
|
||||||
snapshot.usedMB = current.usedMB;
|
|
||||||
snapshot.freeMB = current.freeMB;
|
|
||||||
snapshot.writeSpeedBps = current.writeSpeedBps;
|
|
||||||
|
|
||||||
for (size_t i = 0; i < sizeof(snapshot.cardType); i++)
|
|
||||||
{
|
|
||||||
snapshot.cardType[i] = current.cardType[i];
|
|
||||||
}
|
|
||||||
|
|
||||||
return snapshot;
|
|
||||||
}
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
|
|
||||||
|
|
||||||
// Cross-task snapshot of the SD card, filled by the StorageTask (core 0) and
|
|
||||||
// read by the WebService (core 1) for the dashboard broadcast. Values are
|
|
||||||
// stored as 32-bit quantities (MB / Bps) so each field reads atomically.
|
|
||||||
struct StorageSnapshot
|
|
||||||
{
|
|
||||||
bool mounted;
|
|
||||||
uint32_t timestamp;
|
|
||||||
uint32_t totalMB;
|
|
||||||
uint32_t usedMB;
|
|
||||||
uint32_t freeMB;
|
|
||||||
uint32_t writeSpeedBps;
|
|
||||||
char cardType[16];
|
|
||||||
};
|
|
||||||
|
|
||||||
|
|
||||||
class StorageState
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
void begin();
|
|
||||||
|
|
||||||
void setMounted(bool mounted);
|
|
||||||
void setCardType(const char* type);
|
|
||||||
void setCapacity(uint64_t totalBytes, uint64_t usedBytes);
|
|
||||||
void setWriteSpeedBps(uint32_t bytesPerSecond);
|
|
||||||
|
|
||||||
StorageSnapshot getCurrent();
|
|
||||||
|
|
||||||
private:
|
|
||||||
volatile StorageSnapshot current;
|
|
||||||
};
|
|
||||||
+1
-10
@@ -5,30 +5,25 @@
|
|||||||
|
|
||||||
#include "core/dashboard_state.h"
|
#include "core/dashboard_state.h"
|
||||||
#include "core/diagnostics_state.h"
|
#include "core/diagnostics_state.h"
|
||||||
#include "core/storage_state.h"
|
|
||||||
|
|
||||||
#include "services/service_manager.h"
|
#include "services/service_manager.h"
|
||||||
#include "services/wifi_service.h"
|
#include "services/wifi_service.h"
|
||||||
#include "services/ota_service.h"
|
#include "services/ota_service.h"
|
||||||
#include "services/web_service.h"
|
#include "services/web_service.h"
|
||||||
|
|
||||||
#include "storage/sd_manager.h"
|
|
||||||
|
|
||||||
#include "tasks/system_task.h"
|
#include "tasks/system_task.h"
|
||||||
#include "tasks/diagnostics_task.h"
|
#include "tasks/diagnostics_task.h"
|
||||||
#include "tasks/storage_task.h"
|
|
||||||
|
|
||||||
|
|
||||||
DashboardState dashboardState;
|
DashboardState dashboardState;
|
||||||
DiagnosticsState diagnosticsState;
|
DiagnosticsState diagnosticsState;
|
||||||
StorageState storageState;
|
|
||||||
|
|
||||||
// Simple service scheduler nice for grouping tasks
|
// Simple service scheduler nice for grouping tasks
|
||||||
ServiceManager services;
|
ServiceManager services;
|
||||||
|
|
||||||
WiFiService wifi;
|
WiFiService wifi;
|
||||||
OTAService ota;
|
OTAService ota;
|
||||||
WebService web(dashboardState, diagnosticsState, storageState);
|
WebService web(dashboardState, diagnosticsState);
|
||||||
|
|
||||||
|
|
||||||
// Actual FreeRTOS tasks that are scheduled
|
// Actual FreeRTOS tasks that are scheduled
|
||||||
@@ -36,9 +31,6 @@ SystemTask systemTask(services);
|
|||||||
|
|
||||||
DiagnosticsTask diagnosticsTask(diagnosticsState);
|
DiagnosticsTask diagnosticsTask(diagnosticsState);
|
||||||
|
|
||||||
SDManager sdManager;
|
|
||||||
StorageTask storageTask(sdManager, storageState);
|
|
||||||
|
|
||||||
void setup()
|
void setup()
|
||||||
{
|
{
|
||||||
Serial.begin(115200);
|
Serial.begin(115200);
|
||||||
@@ -50,7 +42,6 @@ void setup()
|
|||||||
|
|
||||||
systemTask.start();
|
systemTask.start();
|
||||||
diagnosticsTask.start();
|
diagnosticsTask.start();
|
||||||
storageTask.start();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,10 @@
|
|||||||
#include "web_service.h"
|
#include "web_service.h"
|
||||||
|
|
||||||
WebService::WebService(DashboardState& state, DiagnosticsState& diag_state, StorageState& storage_state)
|
WebService::WebService(DashboardState& state, DiagnosticsState& diag_state)
|
||||||
:
|
:
|
||||||
Service("Web", 10),
|
Service("Web", 10),
|
||||||
dashboardState(state),
|
dashboardState(state),
|
||||||
diagnosticsState(diag_state),
|
diagnosticsState(diag_state),
|
||||||
storageState(storage_state),
|
|
||||||
server(80),
|
server(80),
|
||||||
webSocket(81)
|
webSocket(81)
|
||||||
{
|
{
|
||||||
@@ -65,11 +64,6 @@ h1 {
|
|||||||
font-size: 28px;
|
font-size: 28px;
|
||||||
margin-top: 20px;
|
margin-top: 20px;
|
||||||
}
|
}
|
||||||
|
|
||||||
#sd_status {
|
|
||||||
font-size: 24px;
|
|
||||||
font-weight: bold;
|
|
||||||
}
|
|
||||||
</style>
|
</style>
|
||||||
|
|
||||||
<script>
|
<script>
|
||||||
@@ -88,13 +82,11 @@ function connectWebSocket() {
|
|||||||
socket.onmessage = function(event) {
|
socket.onmessage = function(event) {
|
||||||
const data = JSON.parse(event.data);
|
const data = JSON.parse(event.data);
|
||||||
|
|
||||||
document.getElementById("uptime").innerHTML = "Uptime: " + data.system.uptime;
|
document.getElementById("uptime").innerHTML = data.system.uptime;
|
||||||
document.getElementById("firmware_version").innerHTML = "Firmware Version: " + data.system.version;
|
document.getElementById("firmware_version").innerHTML = data.system.version;
|
||||||
document.getElementById("free_heap").innerHTML = "Free Heap: " + data.diagnostics.free_heap;
|
document.getElementById("free_heap").innerHTML = data.diagnostics.free_heap;
|
||||||
document.getElementById("minimum_free_heap").innerHTML = "Minimum Free Heap: " + data.diagnostics.minimum_free_heap;
|
document.getElementById("minimum_free_heap").innerHTML = data.diagnostics.minimum_free_heap;
|
||||||
document.getElementById("cpu_frequency").innerHTML = "CPU Frequency: " + data.diagnostics.cpu_frequency + "MHz";
|
document.getElementById("cpu_frequency").innerHTML = data.diagnostics.cpu_frequency;
|
||||||
|
|
||||||
updateStorage(data.storage);
|
|
||||||
};
|
};
|
||||||
|
|
||||||
socket.onclose = function() {
|
socket.onclose = function() {
|
||||||
@@ -103,37 +95,6 @@ function connectWebSocket() {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
function formatBytesMB(mb) {
|
|
||||||
const value = Number(mb);
|
|
||||||
if (value >= 1024) {
|
|
||||||
return (value / 1024).toFixed(2) + " GB";
|
|
||||||
}
|
|
||||||
return value.toFixed(0) + " MB";
|
|
||||||
}
|
|
||||||
|
|
||||||
function updateStorage(storage) {
|
|
||||||
const sdStatus = document.getElementById("sd_status");
|
|
||||||
|
|
||||||
if (storage && storage.mounted === "true") {
|
|
||||||
sdStatus.innerHTML = "SD card found";
|
|
||||||
sdStatus.style.color = "#00ff99";
|
|
||||||
|
|
||||||
document.getElementById("sd_type").innerHTML = "Card Type: " + storage.card_type;
|
|
||||||
document.getElementById("sd_free").innerHTML = "Space left: " +
|
|
||||||
formatBytesMB(storage.free_mb) + " free of " + formatBytesMB(storage.total_mb) +
|
|
||||||
" (" + Math.round(storage.used_mb / storage.total_mb * 100) + "% used)";
|
|
||||||
document.getElementById("sd_speed").innerHTML = "Estimated write speed: " +
|
|
||||||
(storage.write_speed_bps / 1048576).toFixed(2) + " MB/s";
|
|
||||||
} else {
|
|
||||||
sdStatus.innerHTML = "SD not found";
|
|
||||||
sdStatus.style.color = "red";
|
|
||||||
|
|
||||||
document.getElementById("sd_type").innerHTML = "-";
|
|
||||||
document.getElementById("sd_free").innerHTML = "-";
|
|
||||||
document.getElementById("sd_speed").innerHTML = "-";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
window.onload = connectWebSocket;
|
window.onload = connectWebSocket;
|
||||||
|
|
||||||
</script>
|
</script>
|
||||||
@@ -143,27 +104,23 @@ window.onload = connectWebSocket;
|
|||||||
<body>
|
<body>
|
||||||
<div class="card">
|
<div class="card">
|
||||||
<h1>ESP32 Dashboard OTA</h1>
|
<h1>ESP32 Dashboard OTA</h1>
|
||||||
<div id="uptime">Device uptime: Loading...</div>
|
<p>Device uptime:</p>
|
||||||
|
<div id="uptime">Loading...</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="card">
|
<div class="card">
|
||||||
<h1>Hub Diagnostics:</h1>
|
<h1>Hub Diagnostics:</h1>
|
||||||
<div id=free_heap>Free heap: Loading...</div>
|
<p>Free heap:</p>
|
||||||
<div id=minimum_free_heap>Minimum free heap: Loading...</div>
|
<div id=free_heap>Loading...</div>
|
||||||
<div id=cpu_frequency>CPU Frequency: Loading...</div>
|
<p>Minimum free heap:</p>
|
||||||
</div>
|
<div id=minimum_free_heap>Loading...</div>
|
||||||
|
<p>CPU Frequency:</p>
|
||||||
<div class="card">
|
<div id=cpu_frequency>Loading...</div>
|
||||||
<h1>SD Storage:</h1>
|
|
||||||
<div id=sd_status>Status: Loading...</div>
|
|
||||||
<div id=sd_type>Card type: Loading...</div>
|
|
||||||
<div id=sd_free>Space left: Loading...</div>
|
|
||||||
<div id=sd_speed>Estimated write speed: Loading...</div>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
</body>
|
</body>
|
||||||
<footer>
|
<footer>
|
||||||
<div id="firmware_version">Firmware Version: Loading...</div>
|
Firmware Version: <div id="firmware_version">Loading...</div>
|
||||||
</footer>
|
</footer>
|
||||||
</html>
|
</html>
|
||||||
)rawliteral";
|
)rawliteral";
|
||||||
@@ -171,7 +128,7 @@ window.onload = connectWebSocket;
|
|||||||
void WebService::broadcastState()
|
void WebService::broadcastState()
|
||||||
{
|
{
|
||||||
DiagnosticSample diagnostics = diagnosticsState.getCurrent();
|
DiagnosticSample diagnostics = diagnosticsState.getCurrent();
|
||||||
StorageSnapshot storage = storageState.getCurrent();
|
Serial.println(diagnostics.freeHeap);
|
||||||
String json = "{";
|
String json = "{";
|
||||||
|
|
||||||
// Opening system tag:
|
// Opening system tag:
|
||||||
@@ -205,35 +162,6 @@ void WebService::broadcastState()
|
|||||||
|
|
||||||
json += "}"; // Clost diagnostic tag
|
json += "}"; // Clost diagnostic tag
|
||||||
|
|
||||||
// Opening storage tag:
|
|
||||||
json += ",\"storage\":{";
|
|
||||||
|
|
||||||
json += "\"mounted\":\"";
|
|
||||||
json += storage.mounted ? "true" : "false";
|
|
||||||
json += "\",";
|
|
||||||
|
|
||||||
json += "\"card_type\":\"";
|
|
||||||
json += storage.cardType;
|
|
||||||
json += "\",";
|
|
||||||
|
|
||||||
json += "\"total_mb\":\"";
|
|
||||||
json += storage.totalMB;
|
|
||||||
json += "\",";
|
|
||||||
|
|
||||||
json += "\"free_mb\":\"";
|
|
||||||
json += storage.freeMB;
|
|
||||||
json += "\",";
|
|
||||||
|
|
||||||
json += "\"used_mb\":\"";
|
|
||||||
json += storage.usedMB;
|
|
||||||
json += "\",";
|
|
||||||
|
|
||||||
json += "\"write_speed_bps\":\"";
|
|
||||||
json += storage.writeSpeedBps;
|
|
||||||
json += "\"";
|
|
||||||
|
|
||||||
json += "}"; // Close storage tag
|
|
||||||
|
|
||||||
|
|
||||||
// Final close bracket
|
// Final close bracket
|
||||||
json += "}";
|
json += "}";
|
||||||
|
|||||||
@@ -7,13 +7,12 @@
|
|||||||
|
|
||||||
#include "../core/dashboard_state.h"
|
#include "../core/dashboard_state.h"
|
||||||
#include "../core/diagnostics_state.h"
|
#include "../core/diagnostics_state.h"
|
||||||
#include "../core/storage_state.h"
|
|
||||||
|
|
||||||
class WebService : public Service {
|
class WebService : public Service {
|
||||||
|
|
||||||
public:
|
public:
|
||||||
|
|
||||||
WebService(DashboardState& state, DiagnosticsState& diag_state, StorageState& storage_state);
|
WebService(DashboardState& state, DiagnosticsState& diag_state);
|
||||||
|
|
||||||
void begin() override;
|
void begin() override;
|
||||||
void update() override;
|
void update() override;
|
||||||
@@ -25,7 +24,6 @@ private:
|
|||||||
|
|
||||||
DashboardState& dashboardState;
|
DashboardState& dashboardState;
|
||||||
DiagnosticsState& diagnosticsState;
|
DiagnosticsState& diagnosticsState;
|
||||||
StorageState& storageState;
|
|
||||||
|
|
||||||
void handleWebSocketMessage(uint8_t clientNum, uint8_t *payload, size_t length);
|
void handleWebSocketMessage(uint8_t clientNum, uint8_t *payload, size_t length);
|
||||||
void broadcastState();
|
void broadcastState();
|
||||||
|
|||||||
@@ -1,175 +0,0 @@
|
|||||||
#include "data_logger.h"
|
|
||||||
|
|
||||||
#include <string.h>
|
|
||||||
|
|
||||||
// ============================================================================
|
|
||||||
// Skeleton implementation. The WAV metadata helpers below are complete;
|
|
||||||
// everything else is stubbed with a TODO and the work is tracked in
|
|
||||||
// docs/audio_logging.md section 14.
|
|
||||||
// ============================================================================
|
|
||||||
|
|
||||||
|
|
||||||
// --- WAV metadata helpers (complete) -----------------------------------------
|
|
||||||
|
|
||||||
void buildWavHeader(WavHeader& header,
|
|
||||||
uint16_t numChannels,
|
|
||||||
uint32_t sampleRate,
|
|
||||||
uint16_t bitsPerSample,
|
|
||||||
uint32_t dataSize)
|
|
||||||
{
|
|
||||||
uint16_t blockAlign = numChannels * (bitsPerSample / 8);
|
|
||||||
|
|
||||||
memset(&header, 0, sizeof(header));
|
|
||||||
|
|
||||||
memcpy(header.riff, "RIFF", 4);
|
|
||||||
header.riffSize = sizeof(header) + dataSize - 8;
|
|
||||||
memcpy(header.wave, "WAVE", 4);
|
|
||||||
|
|
||||||
memcpy(header.fmt, "fmt ", 4);
|
|
||||||
header.fmtChunkSize = 16;
|
|
||||||
header.audioFormat = 1; // PCM
|
|
||||||
header.numChannels = numChannels;
|
|
||||||
header.sampleRate = sampleRate;
|
|
||||||
header.byteRate = sampleRate * blockAlign;
|
|
||||||
header.blockAlign = blockAlign;
|
|
||||||
header.bitsPerSample = bitsPerSample;
|
|
||||||
|
|
||||||
memcpy(header.data, "data", 4);
|
|
||||||
header.dataSize = dataSize;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void finalizeWavHeader(uint32_t fileSize,
|
|
||||||
uint32_t& riffSizeOut,
|
|
||||||
uint32_t& dataSizeOut)
|
|
||||||
{
|
|
||||||
riffSizeOut = fileSize - 8;
|
|
||||||
dataSizeOut = fileSize - sizeof(WavHeader);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
// --- DataLogger ---------------------------------------------------------------
|
|
||||||
|
|
||||||
DataLogger::DataLogger()
|
|
||||||
:
|
|
||||||
files(nullptr),
|
|
||||||
pool(nullptr),
|
|
||||||
freeQ(nullptr),
|
|
||||||
filledQ(nullptr),
|
|
||||||
sessionSeq(0),
|
|
||||||
bytesThisFile(0),
|
|
||||||
totalBytes(0),
|
|
||||||
totalChunks(0),
|
|
||||||
totalDropped(0),
|
|
||||||
lastDropPrintMs(0),
|
|
||||||
windowBytes(0),
|
|
||||||
windowStartMs(0),
|
|
||||||
bps(0),
|
|
||||||
active(false),
|
|
||||||
warning(false)
|
|
||||||
{
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
bool DataLogger::begin(fs::FS& files)
|
|
||||||
{
|
|
||||||
// TODO: allocate the chunk pool with heap_caps_malloc(MALLOC_CAP_DMA)
|
|
||||||
// (STORAGE_LOG_POOL_SIZE x STORAGE_LOG_CHUNK_SIZE), create freeQ with all
|
|
||||||
// chunks and filledQ empty (xQueueCreate), mkdir STORAGE_LOG_DIR.
|
|
||||||
this->files = &files;
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void DataLogger::end()
|
|
||||||
{
|
|
||||||
// TODO: closeSession(), free the pool (heap_caps_free), delete the queues.
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
bool DataLogger::openSession()
|
|
||||||
{
|
|
||||||
// TODO: build "rec_<uptimeSeconds>_<sessionSeq>.wav" under
|
|
||||||
// STORAGE_LOG_DIR (skip forward if the name exists), open FILE_WRITE,
|
|
||||||
// write the 44-byte header with buildWavHeader(dataSize=0).
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
bool DataLogger::closeSession()
|
|
||||||
{
|
|
||||||
// TODO: finalizeWavHeader(file.size(), ...) -> patch offsets 4 and 40,
|
|
||||||
// file.flush(), file.close().
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
bool DataLogger::rotateIfNeeded()
|
|
||||||
{
|
|
||||||
// TODO: if active && bytesThisFile >= STORAGE_LOG_ROTATE_BYTES:
|
|
||||||
// closeSession(); sessionSeq++; openSession().
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
AudioChunk* DataLogger::acquireChunk(TickType_t timeout)
|
|
||||||
{
|
|
||||||
// TODO: xQueueReceive(freeQ, &chunk, timeout). On timeout, call
|
|
||||||
// dropOldestChunk() and return its chunk so the producer keeps streaming.
|
|
||||||
return nullptr;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void DataLogger::submitChunk(AudioChunk* chunk)
|
|
||||||
{
|
|
||||||
// TODO: xQueueSend(filledQ, &chunk, ...). Must preserve stream order.
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
AudioChunk* DataLogger::nextChunk(TickType_t timeout)
|
|
||||||
{
|
|
||||||
// TODO: xQueueReceive(filledQ, &chunk, timeout).
|
|
||||||
return nullptr;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
bool DataLogger::writeChunk(AudioChunk* chunk)
|
|
||||||
{
|
|
||||||
// TODO: if (!active) return false;
|
|
||||||
// n = file.write(chunk->data, chunk->length);
|
|
||||||
// bytesThisFile += n; totalBytes += n; totalChunks++;
|
|
||||||
// feed the writeSpeedBps window (windowBytes/windowStartMs);
|
|
||||||
// file.flush() every STORAGE_LOG_FLUSH_BYTES;
|
|
||||||
// return n == chunk->length; (card failure -> fatal)
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void DataLogger::releaseChunk(AudioChunk* chunk)
|
|
||||||
{
|
|
||||||
// TODO: chunk->length = 0; xQueueSend(freeQ, &chunk, ...).
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
AudioChunk* DataLogger::dropOldestChunk()
|
|
||||||
{
|
|
||||||
// TODO: xQueueReceive from the BACK of filledQ without writing, count it,
|
|
||||||
// setOverflowWarning(true). Called by acquireChunk on timeout.
|
|
||||||
return nullptr;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void DataLogger::setOverflowWarning(bool overflowing)
|
|
||||||
{
|
|
||||||
// TODO: rate-limited Serial line with totalDropped + bytes behind;
|
|
||||||
// digitalWrite(STORAGE_WARN_LED_GPIO, ...) using
|
|
||||||
// STORAGE_WARN_LED_ACTIVE_HIGH.
|
|
||||||
(void)overflowing;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
uint32_t DataLogger::bytesWritten() const { return (uint32_t)totalBytes; }
|
|
||||||
uint32_t DataLogger::chunksWritten() const { return totalChunks; }
|
|
||||||
uint32_t DataLogger::droppedChunks() const { return totalDropped; }
|
|
||||||
uint32_t DataLogger::writeSpeedBps() const { return bps; }
|
|
||||||
bool DataLogger::overflowing() const { return warning; }
|
|
||||||
@@ -1,174 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
// ============================================================================
|
|
||||||
// Audio logging interface (see docs/audio_logging.md for the full design).
|
|
||||||
//
|
|
||||||
// THIS FILE IS A SKELETON. The chunk-pool / queue / file logic in
|
|
||||||
// data_logger.cpp is stubbed with TODOs; only the WAV metadata helpers are
|
|
||||||
// implemented. The interface below is the contract the future Node
|
|
||||||
// collection task (producer) and the StorageTask (consumer) are written
|
|
||||||
// against.
|
|
||||||
// ============================================================================
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
#include <FS.h>
|
|
||||||
|
|
||||||
#include <freertos/FreeRTOS.h>
|
|
||||||
#include <freertos/queue.h>
|
|
||||||
|
|
||||||
#include "storage_config.h"
|
|
||||||
|
|
||||||
|
|
||||||
// ----------------------------------------------------------------------------
|
|
||||||
// WAV (RIFF) metadata helpers - fully implemented in data_logger.cpp.
|
|
||||||
// ----------------------------------------------------------------------------
|
|
||||||
|
|
||||||
// 44-byte PCM WAVE header. Layout and field meaning are documented in
|
|
||||||
// docs/audio_logging.md section 3.
|
|
||||||
struct WavHeader
|
|
||||||
{
|
|
||||||
uint8_t riff[4]; // "RIFF"
|
|
||||||
uint32_t riffSize; // file size - 8
|
|
||||||
uint8_t wave[4]; // "WAVE"
|
|
||||||
uint8_t fmt[4]; // "fmt "
|
|
||||||
uint32_t fmtChunkSize; // 16 (PCM)
|
|
||||||
uint16_t audioFormat; // 1 (PCM)
|
|
||||||
uint16_t numChannels; // STORAGE_AUDIO_CHANNELS
|
|
||||||
uint32_t sampleRate; // STORAGE_AUDIO_SAMPLE_RATE_HZ
|
|
||||||
uint32_t byteRate; // sampleRate * blockAlign
|
|
||||||
uint16_t blockAlign; // numChannels * (bitsPerSample / 8)
|
|
||||||
uint16_t bitsPerSample; // STORAGE_AUDIO_BITS
|
|
||||||
uint8_t data[4]; // "data"
|
|
||||||
uint32_t dataSize; // file size - 44
|
|
||||||
} __attribute__((packed));
|
|
||||||
|
|
||||||
static_assert(sizeof(WavHeader) == 44, "WAV header must be exactly 44 bytes");
|
|
||||||
|
|
||||||
// Fills the header for a PCM stream of the given channels/rate/depth.
|
|
||||||
// dataSize is typically 0 at file open and corrected on finalize.
|
|
||||||
void buildWavHeader(WavHeader& header,
|
|
||||||
uint16_t numChannels,
|
|
||||||
uint32_t sampleRate,
|
|
||||||
uint16_t bitsPerSample,
|
|
||||||
uint32_t dataSize);
|
|
||||||
|
|
||||||
// Returns the two little-endian values to patch at offsets 4 and 40 when a
|
|
||||||
// file is closed/rotated: riffSize = fileSize - 8, dataSize = fileSize - 44.
|
|
||||||
void finalizeWavHeader(uint32_t fileSize,
|
|
||||||
uint32_t& riffSizeOut,
|
|
||||||
uint32_t& dataSizeOut);
|
|
||||||
|
|
||||||
|
|
||||||
// ----------------------------------------------------------------------------
|
|
||||||
// Chunk pool
|
|
||||||
// ----------------------------------------------------------------------------
|
|
||||||
|
|
||||||
// One slot of the write pool. Producers borrow a chunk, fill `data[0..length)`
|
|
||||||
// with interleaved PCM (40 channels, frame order, 80 bytes/frame), and submit
|
|
||||||
// it. Chunks must be submitted in strict stream order.
|
|
||||||
struct AudioChunk
|
|
||||||
{
|
|
||||||
uint8_t* data; // DMA-capable buffer owned by the pool
|
|
||||||
uint32_t capacity; // STORAGE_LOG_CHUNK_SIZE
|
|
||||||
uint32_t length; // valid bytes, multiple of the WAV frame size
|
|
||||||
uint32_t sequence; // monotonic order (diagnostics only)
|
|
||||||
};
|
|
||||||
|
|
||||||
|
|
||||||
// ----------------------------------------------------------------------------
|
|
||||||
// DataLogger
|
|
||||||
// ----------------------------------------------------------------------------
|
|
||||||
|
|
||||||
// Producer/consumer bridge between the Node collection task (core 1) and the
|
|
||||||
// StorageTask (core 0). Owns the chunk pool, the two FreeRTOS queues, and the
|
|
||||||
// open WAV file. Only the consumer touches the card.
|
|
||||||
class DataLogger
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
DataLogger();
|
|
||||||
|
|
||||||
// Lifecycle --------------------------------------------------------------
|
|
||||||
|
|
||||||
// Allocates the pool (MALLOC_CAP_DMA), creates the queues, creates
|
|
||||||
// STORAGE_LOG_DIR. Does not open a session file.
|
|
||||||
bool begin(fs::FS& files);
|
|
||||||
|
|
||||||
// Finalizes and closes the current session file, frees the pool/queues.
|
|
||||||
void end();
|
|
||||||
|
|
||||||
// Session control --------------------------------------------------------
|
|
||||||
|
|
||||||
// Creates the next rec_<uptimeSeconds>_<n>.wav under STORAGE_LOG_DIR,
|
|
||||||
// writes the 44-byte WAV header. false if the card is full or unwritable.
|
|
||||||
bool openSession();
|
|
||||||
|
|
||||||
// Finalizes (patches RIFF/data sizes) and closes the current file.
|
|
||||||
bool closeSession();
|
|
||||||
|
|
||||||
// Closes/opens when bytesThisFile >= STORAGE_LOG_ROTATE_BYTES. Called by
|
|
||||||
// the consumer on every loop; must be a no-op when idle.
|
|
||||||
bool rotateIfNeeded();
|
|
||||||
|
|
||||||
// Producer API (Node collection task, core 1) ----------------------------
|
|
||||||
|
|
||||||
// Pops a free chunk, or on timeout drops the oldest queued chunk (Section
|
|
||||||
// 8) and returns it so the producer can keep streaming. nullptr only if
|
|
||||||
// there is nothing to drop (empty pool + empty queue).
|
|
||||||
AudioChunk* acquireChunk(TickType_t timeout);
|
|
||||||
|
|
||||||
// Returns a filled chunk to the write queue. Must be called in order.
|
|
||||||
void submitChunk(AudioChunk* chunk);
|
|
||||||
|
|
||||||
// Consumer API (StorageTask, core 0) --------------------------------------
|
|
||||||
|
|
||||||
// Blocks up to `timeout` for the next filled chunk. nullptr on timeout
|
|
||||||
// (lets the consumer pulse for stats even when idle).
|
|
||||||
AudioChunk* nextChunk(TickType_t timeout);
|
|
||||||
|
|
||||||
// Appends chunk->data[0..length) to the WAV data section with one
|
|
||||||
// file.write(), updates the byte/chunk counters and the bps window.
|
|
||||||
// false on card failure (fatal per design.md).
|
|
||||||
bool writeChunk(AudioChunk* chunk);
|
|
||||||
|
|
||||||
// Returns a chunk to the free pool after it has been written or dropped.
|
|
||||||
void releaseChunk(AudioChunk* chunk);
|
|
||||||
|
|
||||||
// Stats -------------------------------------------------------------------
|
|
||||||
|
|
||||||
uint32_t bytesWritten() const;
|
|
||||||
uint32_t chunksWritten() const;
|
|
||||||
uint32_t droppedChunks() const;
|
|
||||||
uint32_t writeSpeedBps() const;
|
|
||||||
bool overflowing() const;
|
|
||||||
|
|
||||||
private:
|
|
||||||
// Pops the oldest unwritten chunk off filledQ without writing it,
|
|
||||||
// increments droppedChunks, raises the warning. Returns it to the caller
|
|
||||||
// (used by acquireChunk on timeout).
|
|
||||||
AudioChunk* dropOldestChunk();
|
|
||||||
|
|
||||||
// Serial (rate-limited) + LED (STORAGE_WARN_LED_GPIO) overflow indication.
|
|
||||||
void setOverflowWarning(bool overflowing);
|
|
||||||
|
|
||||||
fs::FS* files; // SD backend, injected by begin()
|
|
||||||
AudioChunk* pool; // STORAGE_LOG_POOL_SIZE chunks
|
|
||||||
QueueHandle_t freeQ; // empty chunks
|
|
||||||
QueueHandle_t filledQ; // chunks waiting to be written
|
|
||||||
File file; // open session file
|
|
||||||
char filePath[96];
|
|
||||||
uint32_t sessionSeq; // rotation counter within this boot
|
|
||||||
|
|
||||||
uint64_t bytesThisFile;
|
|
||||||
uint64_t totalBytes;
|
|
||||||
uint32_t totalChunks;
|
|
||||||
uint32_t totalDropped;
|
|
||||||
uint32_t lastDropPrintMs;
|
|
||||||
|
|
||||||
// bps metering window (writeChunk is the only writer)
|
|
||||||
uint32_t windowBytes;
|
|
||||||
uint32_t windowStartMs;
|
|
||||||
uint32_t bps;
|
|
||||||
|
|
||||||
bool active;
|
|
||||||
bool warning;
|
|
||||||
};
|
|
||||||
@@ -1,443 +0,0 @@
|
|||||||
#include "sd_manager.h"
|
|
||||||
|
|
||||||
#include "storage_config.h"
|
|
||||||
|
|
||||||
#include <driver/gpio.h>
|
|
||||||
|
|
||||||
// Both backends are compiled into the image so that the SDIO path is
|
|
||||||
// compile-checked even while the system still runs SPI for bring-up.
|
|
||||||
#include <SD.h>
|
|
||||||
#include <SD_MMC.h>
|
|
||||||
|
|
||||||
namespace
|
|
||||||
{
|
|
||||||
|
|
||||||
class SpiBackend : public StorageBackend
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
bool begin() override
|
|
||||||
{
|
|
||||||
// Explicit SPI instance so the pin wiring is driven from
|
|
||||||
// storage_config.h rather than the board defaults. The second
|
|
||||||
// spi.begin() call made inside SDFS::begin() is a no-op because the
|
|
||||||
// bus is already started with these pins.
|
|
||||||
static SPIClass spi;
|
|
||||||
|
|
||||||
spi.begin(
|
|
||||||
STORAGE_SPI_SCK,
|
|
||||||
STORAGE_SPI_MISO,
|
|
||||||
STORAGE_SPI_MOSI,
|
|
||||||
STORAGE_SPI_CS
|
|
||||||
);
|
|
||||||
|
|
||||||
// Cheap breakout modules often omit the pull-up resistors the SD
|
|
||||||
// spec expects on the idle-high lines. The ESP32 SPI HAL clears the
|
|
||||||
// internal pull-ups when it attaches a pin (esp32-hal-spi.c), so a
|
|
||||||
// floating MISO/CS means the card never answers CMD0 during init
|
|
||||||
// ("Card Failed! cmd: 0x00"). Re-enable the pull-ups directly so we
|
|
||||||
// do not disturb the pin's peripheral function.
|
|
||||||
gpio_pullup_en((gpio_num_t)STORAGE_SPI_MISO);
|
|
||||||
gpio_pullup_en((gpio_num_t)STORAGE_SPI_CS);
|
|
||||||
|
|
||||||
// Give the card a moment to stabilize after power-on before the init
|
|
||||||
// handshake starts (the framework sends 74+ dummy clocks, but some
|
|
||||||
// cards need a little more settling time on a fresh mount attempt).
|
|
||||||
delay(20);
|
|
||||||
|
|
||||||
return SD.begin(
|
|
||||||
STORAGE_SPI_CS,
|
|
||||||
spi,
|
|
||||||
STORAGE_SPI_FREQ,
|
|
||||||
STORAGE_MOUNT_POINT,
|
|
||||||
STORAGE_MAX_OPEN_FILES,
|
|
||||||
false // format_if_empty: never auto-format
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
void end() override
|
|
||||||
{
|
|
||||||
SD.end();
|
|
||||||
}
|
|
||||||
|
|
||||||
fs::FS& fs() override
|
|
||||||
{
|
|
||||||
return SD;
|
|
||||||
}
|
|
||||||
|
|
||||||
StorageCardType cardType() override
|
|
||||||
{
|
|
||||||
return mapType(SD.cardType());
|
|
||||||
}
|
|
||||||
|
|
||||||
uint64_t totalBytes() override
|
|
||||||
{
|
|
||||||
return SD.totalBytes();
|
|
||||||
}
|
|
||||||
|
|
||||||
uint64_t usedBytes() override
|
|
||||||
{
|
|
||||||
return SD.usedBytes();
|
|
||||||
}
|
|
||||||
|
|
||||||
private:
|
|
||||||
static StorageCardType mapType(sdcard_type_t type)
|
|
||||||
{
|
|
||||||
switch (type)
|
|
||||||
{
|
|
||||||
case CARD_MMC: return StorageCardType::MMC;
|
|
||||||
case CARD_SD: return StorageCardType::SD;
|
|
||||||
case CARD_SDHC: return StorageCardType::SDHC;
|
|
||||||
case CARD_NONE: return StorageCardType::None;
|
|
||||||
default: return StorageCardType::Unknown;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
class SdmmcBackend : public StorageBackend
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
bool begin() override
|
|
||||||
{
|
|
||||||
// The plain esp32dev variant does not pre-wire the SDMMC pins, so
|
|
||||||
// they must always be set explicitly. The classic ESP32 routes the
|
|
||||||
// SDMMC peripheral through the GPIO matrix, so the pins defined in
|
|
||||||
// storage_config.h are fully re-routable.
|
|
||||||
if (!SD_MMC.setPins(
|
|
||||||
STORAGE_SDMMC_CLK,
|
|
||||||
STORAGE_SDMMC_CMD,
|
|
||||||
STORAGE_SDMMC_D0,
|
|
||||||
STORAGE_SDMMC_D1,
|
|
||||||
STORAGE_SDMMC_D2,
|
|
||||||
STORAGE_SDMMC_D3))
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
return SD_MMC.begin(
|
|
||||||
STORAGE_MOUNT_POINT,
|
|
||||||
STORAGE_SDMMC_MODE_1BIT, // false == 4-bit bus
|
|
||||||
STORAGE_SDMMC_FORMAT_IF_FAILED, // never auto-format
|
|
||||||
STORAGE_SDMMC_FREQ_HZ, // 40 MHz == SDMMC_FREQ_HIGHSPEED
|
|
||||||
STORAGE_MAX_OPEN_FILES
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
void end() override
|
|
||||||
{
|
|
||||||
SD_MMC.end();
|
|
||||||
}
|
|
||||||
|
|
||||||
fs::FS& fs() override
|
|
||||||
{
|
|
||||||
return SD_MMC;
|
|
||||||
}
|
|
||||||
|
|
||||||
StorageCardType cardType() override
|
|
||||||
{
|
|
||||||
return mapType(SD_MMC.cardType());
|
|
||||||
}
|
|
||||||
|
|
||||||
uint64_t totalBytes() override
|
|
||||||
{
|
|
||||||
return SD_MMC.totalBytes();
|
|
||||||
}
|
|
||||||
|
|
||||||
uint64_t usedBytes() override
|
|
||||||
{
|
|
||||||
return SD_MMC.usedBytes();
|
|
||||||
}
|
|
||||||
|
|
||||||
private:
|
|
||||||
static StorageCardType mapType(sdcard_type_t type)
|
|
||||||
{
|
|
||||||
switch (type)
|
|
||||||
{
|
|
||||||
case CARD_MMC: return StorageCardType::MMC;
|
|
||||||
case CARD_SD: return StorageCardType::SD;
|
|
||||||
case CARD_SDHC: return StorageCardType::SDHC;
|
|
||||||
case CARD_NONE: return StorageCardType::None;
|
|
||||||
default: return StorageCardType::Unknown;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
} // namespace
|
|
||||||
|
|
||||||
|
|
||||||
SDManager::SDManager()
|
|
||||||
:
|
|
||||||
backend(nullptr),
|
|
||||||
mounted(false)
|
|
||||||
{
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
bool SDManager::begin()
|
|
||||||
{
|
|
||||||
if (mounted)
|
|
||||||
{
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
#if STORAGE_IFACE == STORAGE_IFACE_SDMMC
|
|
||||||
backend = new SdmmcBackend();
|
|
||||||
#else
|
|
||||||
backend = new SpiBackend();
|
|
||||||
#endif
|
|
||||||
|
|
||||||
if (backend == nullptr)
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
mounted = backend->begin();
|
|
||||||
|
|
||||||
if (!mounted)
|
|
||||||
{
|
|
||||||
Serial.println("[Storage] SD card mount FAILED");
|
|
||||||
delete backend;
|
|
||||||
backend = nullptr;
|
|
||||||
}
|
|
||||||
|
|
||||||
return mounted;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void SDManager::end()
|
|
||||||
{
|
|
||||||
if (backend != nullptr)
|
|
||||||
{
|
|
||||||
backend->end();
|
|
||||||
delete backend;
|
|
||||||
backend = nullptr;
|
|
||||||
}
|
|
||||||
|
|
||||||
mounted = false;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
bool SDManager::isMounted() const
|
|
||||||
{
|
|
||||||
return mounted;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
fs::FS& SDManager::fs()
|
|
||||||
{
|
|
||||||
return backend->fs();
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
StorageCardType SDManager::cardType() const
|
|
||||||
{
|
|
||||||
if (mounted && backend != nullptr)
|
|
||||||
{
|
|
||||||
return backend->cardType();
|
|
||||||
}
|
|
||||||
return StorageCardType::None;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const char* SDManager::cardTypeName() const
|
|
||||||
{
|
|
||||||
return cardTypeName(cardType());
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
uint64_t SDManager::totalBytes() const
|
|
||||||
{
|
|
||||||
if (mounted && backend != nullptr)
|
|
||||||
{
|
|
||||||
return backend->totalBytes();
|
|
||||||
}
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
uint64_t SDManager::usedBytes() const
|
|
||||||
{
|
|
||||||
if (mounted && backend != nullptr)
|
|
||||||
{
|
|
||||||
return backend->usedBytes();
|
|
||||||
}
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
uint64_t SDManager::freeBytes() const
|
|
||||||
{
|
|
||||||
uint64_t total = totalBytes();
|
|
||||||
uint64_t used = usedBytes();
|
|
||||||
return (total > used) ? (total - used) : 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
uint32_t SDManager::measureWriteSpeed()
|
|
||||||
{
|
|
||||||
if (!mounted || backend == nullptr)
|
|
||||||
{
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
fs::FS& files = backend->fs();
|
|
||||||
|
|
||||||
const char* scratchPath = STORAGE_SPEED_MEASURE_PATH;
|
|
||||||
const size_t bufferSize = 16 * 1024;
|
|
||||||
|
|
||||||
// Remove any leftover from a previous crashed run.
|
|
||||||
files.remove(scratchPath);
|
|
||||||
|
|
||||||
uint8_t* buffer = (uint8_t*)malloc(bufferSize);
|
|
||||||
if (buffer == nullptr)
|
|
||||||
{
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
memset(buffer, 0xA5, bufferSize);
|
|
||||||
|
|
||||||
File file = files.open(scratchPath, FILE_WRITE);
|
|
||||||
if (!file)
|
|
||||||
{
|
|
||||||
free(buffer);
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
uint32_t remaining = STORAGE_SPEED_MEASURE_BYTES;
|
|
||||||
uint32_t startUs = micros();
|
|
||||||
|
|
||||||
while (remaining > 0)
|
|
||||||
{
|
|
||||||
size_t toWrite = (remaining < bufferSize) ? remaining : bufferSize;
|
|
||||||
size_t written = file.write(buffer, toWrite);
|
|
||||||
|
|
||||||
if (written == 0)
|
|
||||||
{
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
|
|
||||||
remaining -= written;
|
|
||||||
}
|
|
||||||
|
|
||||||
file.close();
|
|
||||||
|
|
||||||
uint32_t elapsedUs = micros() - startUs;
|
|
||||||
Serial.print("Elapsed microseconds: ");
|
|
||||||
Serial.println(elapsedUs);
|
|
||||||
|
|
||||||
files.remove(scratchPath);
|
|
||||||
free(buffer);
|
|
||||||
|
|
||||||
if (remaining != 0 || elapsedUs == 0)
|
|
||||||
{
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
uint64_t writtenBytes = (uint64_t)STORAGE_SPEED_MEASURE_BYTES - remaining;
|
|
||||||
return (uint32_t)((writtenBytes * 1000000ULL) / elapsedUs);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const char* SDManager::cardTypeName(StorageCardType type)
|
|
||||||
{
|
|
||||||
switch (type)
|
|
||||||
{
|
|
||||||
case StorageCardType::MMC: return "MMC";
|
|
||||||
case StorageCardType::SD: return "SD";
|
|
||||||
case StorageCardType::SDHC: return "SDHC";
|
|
||||||
case StorageCardType::Unknown: return "Unknown";
|
|
||||||
case StorageCardType::None:
|
|
||||||
default: return "None";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void SDManager::printCardInfo()
|
|
||||||
{
|
|
||||||
if (!mounted)
|
|
||||||
{
|
|
||||||
Serial.println("[Storage] Card info unavailable (not mounted)");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
uint64_t total = backend->totalBytes();
|
|
||||||
uint64_t used = backend->usedBytes();
|
|
||||||
uint64_t free = (total > used) ? (total - used) : 0;
|
|
||||||
|
|
||||||
Serial.println("----------------------------------------");
|
|
||||||
Serial.println("SD card information");
|
|
||||||
Serial.println("----------------------------------------");
|
|
||||||
Serial.printf(" Type : %s\n", cardTypeName(backend->cardType()));
|
|
||||||
Serial.printf(" Total : %llu bytes\n", (unsigned long long)total);
|
|
||||||
Serial.printf(" Used : %llu bytes\n", (unsigned long long)used);
|
|
||||||
Serial.printf(" Free : %llu bytes\n", (unsigned long long)free);
|
|
||||||
Serial.println("----------------------------------------");
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void SDManager::listFiles()
|
|
||||||
{
|
|
||||||
if (!mounted)
|
|
||||||
{
|
|
||||||
Serial.println("[Storage] Cannot list files (not mounted)");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
Serial.println("Files on SD card:");
|
|
||||||
Serial.println("----------------------------------------");
|
|
||||||
|
|
||||||
char path[256];
|
|
||||||
snprintf(path, sizeof(path), "/");
|
|
||||||
|
|
||||||
listFilesRecursive(backend->fs(), path, sizeof(path), 0);
|
|
||||||
|
|
||||||
Serial.println("----------------------------------------");
|
|
||||||
Serial.println("End of listing");
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void SDManager::listFilesRecursive(fs::FS& files, char* path, size_t pathSize, uint8_t depth)
|
|
||||||
{
|
|
||||||
if (depth > STORAGE_LIST_MAX_DEPTH)
|
|
||||||
{
|
|
||||||
Serial.printf(" ... (max depth %u reached)\n", STORAGE_LIST_MAX_DEPTH);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
File dir = files.open(path);
|
|
||||||
|
|
||||||
if (!dir)
|
|
||||||
{
|
|
||||||
Serial.printf(" [error] cannot open: %s\n", path);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!dir.isDirectory())
|
|
||||||
{
|
|
||||||
Serial.printf(" %s (%llu bytes)\n", path, (unsigned long long)dir.size());
|
|
||||||
dir.close();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
File entry;
|
|
||||||
while ((entry = dir.openNextFile()))
|
|
||||||
{
|
|
||||||
size_t base = strlen(path);
|
|
||||||
|
|
||||||
if (entry.isDirectory())
|
|
||||||
{
|
|
||||||
snprintf(path + base, pathSize - base, "/%s", entry.name());
|
|
||||||
Serial.printf(" %s/\n", path);
|
|
||||||
listFilesRecursive(files, path, pathSize, depth + 1);
|
|
||||||
path[base] = '\0';
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
snprintf(path + base, pathSize - base, "/%s", entry.name());
|
|
||||||
Serial.printf(" %s (%llu bytes)\n", path, (unsigned long long)entry.size());
|
|
||||||
}
|
|
||||||
|
|
||||||
entry.close();
|
|
||||||
}
|
|
||||||
|
|
||||||
dir.close();
|
|
||||||
}
|
|
||||||
@@ -1,72 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
#include <FS.h>
|
|
||||||
|
|
||||||
// StorageCardType decouples the rest of the system from the SD/SD_MMC
|
|
||||||
// library enum so nothing outside sd_manager.cpp needs to know which
|
|
||||||
// hardware interface is in use.
|
|
||||||
enum class StorageCardType
|
|
||||||
{
|
|
||||||
None,
|
|
||||||
MMC,
|
|
||||||
SD,
|
|
||||||
SDHC,
|
|
||||||
Unknown
|
|
||||||
};
|
|
||||||
|
|
||||||
// Hardware abstraction for the SD card transport. The concrete backend
|
|
||||||
// (SPI today, SDIO later) is selected at build time in storage_config.h.
|
|
||||||
// Everything downstream (StorageTask, future DataLogger) talks only to
|
|
||||||
// the `fs::FS` reference, so swapping SPI for SDIO touches no other code.
|
|
||||||
class StorageBackend
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
virtual ~StorageBackend() {}
|
|
||||||
|
|
||||||
virtual bool begin() = 0;
|
|
||||||
virtual void end() = 0;
|
|
||||||
|
|
||||||
virtual fs::FS& fs() = 0;
|
|
||||||
|
|
||||||
virtual StorageCardType cardType() = 0;
|
|
||||||
virtual uint64_t totalBytes() = 0;
|
|
||||||
virtual uint64_t usedBytes() = 0;
|
|
||||||
};
|
|
||||||
|
|
||||||
// Owns the selected backend, mounts the card, and provides the boot-time
|
|
||||||
// info/listing report. The mount and any file access are blocking and are
|
|
||||||
// expected to be driven from the dedicated StorageTask on core 0.
|
|
||||||
class SDManager
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
SDManager();
|
|
||||||
|
|
||||||
bool begin();
|
|
||||||
void end();
|
|
||||||
|
|
||||||
bool isMounted() const;
|
|
||||||
|
|
||||||
fs::FS& fs();
|
|
||||||
|
|
||||||
StorageCardType cardType() const;
|
|
||||||
const char* cardTypeName() const;
|
|
||||||
uint64_t totalBytes() const;
|
|
||||||
uint64_t usedBytes() const;
|
|
||||||
uint64_t freeBytes() const;
|
|
||||||
|
|
||||||
// Writes a scratch file of STORAGE_SPEED_MEASURE_BYTES and returns the
|
|
||||||
// achieved write throughput in bytes/second (0 on failure). The scratch
|
|
||||||
// file is deleted before returning.
|
|
||||||
uint32_t measureWriteSpeed();
|
|
||||||
|
|
||||||
void printCardInfo();
|
|
||||||
void listFiles();
|
|
||||||
|
|
||||||
private:
|
|
||||||
StorageBackend* backend;
|
|
||||||
bool mounted;
|
|
||||||
|
|
||||||
void listFilesRecursive(fs::FS& files, char* path, size_t pathSize, uint8_t depth);
|
|
||||||
static const char* cardTypeName(StorageCardType type);
|
|
||||||
};
|
|
||||||
@@ -1,145 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
// ============================================================================
|
|
||||||
// Storage configuration
|
|
||||||
// ============================================================================
|
|
||||||
//
|
|
||||||
// This header is the single place to configure how the Hub talks to the
|
|
||||||
// SD card. Two hardware interfaces are supported by the Arduino-ESP32
|
|
||||||
// framework, and both are implemented behind the StorageBackend interface
|
|
||||||
// in sd_manager.cpp:
|
|
||||||
//
|
|
||||||
// STORAGE_IFACE_SPI - Uses the `SD` library (SPI protocol). This is the
|
|
||||||
// bring-up path for the current breakout module,
|
|
||||||
// which only breaks out the 4 SPI lines.
|
|
||||||
// Max practical throughput: ~1-2 MB/s. NOT enough
|
|
||||||
// for the 6-8 MB/s audio logging target.
|
|
||||||
//
|
|
||||||
// STORAGE_IFACE_SDMMC - Uses the `SD_MMC` library (SDIO protocol, the
|
|
||||||
// SDMMC peripheral). This is the production path.
|
|
||||||
// 4-bit mode @ 40 MHz (SDMMC_FREQ_HIGHSPEED) gives
|
|
||||||
// roughly 8-12 MB/s, which comfortably meets the
|
|
||||||
// target. REQUIRED for the final design.
|
|
||||||
//
|
|
||||||
// TO MIGRATE TO SDIO (the new SDIO-capable board that is being shipped):
|
|
||||||
//
|
|
||||||
// 1. Change STORAGE_IFACE below from STORAGE_IFACE_SPI to
|
|
||||||
// STORAGE_IFACE_SDMMC.
|
|
||||||
//
|
|
||||||
// 2. Wire the SD card to the SDIO lines listed in the STORAGE_SDMMC_*
|
|
||||||
// defines below. On the classic ESP32 the SDMMC peripheral is routed
|
|
||||||
// through the GPIO matrix, so these pins can be changed to any free
|
|
||||||
// GPIO simply by editing the defines.
|
|
||||||
//
|
|
||||||
// 3. Keep STORAGE_SDMMC_MODE_1BIT as `false` (4-bit bus is required for
|
|
||||||
// the write throughput) and keep STORAGE_SDMMC_FORMAT_IF_FAILED as
|
|
||||||
// `false` (a foreign/unformatted card must never be auto-formatted).
|
|
||||||
//
|
|
||||||
// 4. Everything downstream (StorageTask, SDManager, and the future
|
|
||||||
// DataLogger) talks only through the `fs::FS` interface, so no other
|
|
||||||
// code changes are needed.
|
|
||||||
// ============================================================================
|
|
||||||
|
|
||||||
// --- Backend selection ------------------------------------------------------
|
|
||||||
|
|
||||||
#define STORAGE_IFACE_SPI 1
|
|
||||||
#define STORAGE_IFACE_SDMMC 2
|
|
||||||
|
|
||||||
// SPI until the SDIO-capable board arrives.
|
|
||||||
#ifndef STORAGE_IFACE
|
|
||||||
#define STORAGE_IFACE STORAGE_IFACE_SPI
|
|
||||||
#endif
|
|
||||||
|
|
||||||
// --- Common -----------------------------------------------------------------
|
|
||||||
|
|
||||||
// Single canonical mount point so file paths never change when the backend
|
|
||||||
// is switched (the SD lib defaults to "/sd", SD_MMC to "/sdcard").
|
|
||||||
#define STORAGE_MOUNT_POINT "/sdcard"
|
|
||||||
|
|
||||||
#define STORAGE_MAX_OPEN_FILES 5
|
|
||||||
|
|
||||||
// --- SPI (bring-up only) ----------------------------------------------------
|
|
||||||
|
|
||||||
// Default VSPI pins on the classic ESP32 DevKitC (variant/pins_arduino.h).
|
|
||||||
#define STORAGE_SPI_CS 5
|
|
||||||
#define STORAGE_SPI_SCK 18
|
|
||||||
#define STORAGE_SPI_MOSI 23
|
|
||||||
#define STORAGE_SPI_MISO 19
|
|
||||||
|
|
||||||
// 10 MHz is a reliable default for breadboard/jumper-wire bring-up. The SD
|
|
||||||
// init handshake always runs at 400 kHz regardless (see sd_diskio.cpp), so a
|
|
||||||
// mount failure is NOT a frequency problem - check power, wiring, pull-ups
|
|
||||||
// and card seating first. Raise to 20 MHz once the wiring is proven.
|
|
||||||
#define STORAGE_SPI_FREQ 10000000UL
|
|
||||||
|
|
||||||
// --- SDIO / SD_MMC (production, 4-bit) --------------------------------------
|
|
||||||
|
|
||||||
// Default ESP32 SDMMC slot-1 pins (GPIO matrix, freely re-routable).
|
|
||||||
#define STORAGE_SDMMC_CLK 6
|
|
||||||
#define STORAGE_SDMMC_CMD 11
|
|
||||||
#define STORAGE_SDMMC_D0 7
|
|
||||||
#define STORAGE_SDMMC_D1 8
|
|
||||||
#define STORAGE_SDMMC_D2 9
|
|
||||||
#define STORAGE_SDMMC_D3 10
|
|
||||||
|
|
||||||
// false = 4-bit wide bus (required for >8 MB/s). Do NOT enable 1-bit mode.
|
|
||||||
#define STORAGE_SDMMC_MODE_1BIT false
|
|
||||||
|
|
||||||
// NEVER auto-format: an unformatted/foreign card must never be destroyed.
|
|
||||||
#define STORAGE_SDMMC_FORMAT_IF_FAILED false
|
|
||||||
|
|
||||||
// 40 MHz == SDMMC_FREQ_HIGHSPEED. Written as a literal to keep this header
|
|
||||||
// free of driver includes.
|
|
||||||
#define STORAGE_SDMMC_FREQ_HZ 40000000
|
|
||||||
|
|
||||||
// --- Boot-time file listing -------------------------------------------------
|
|
||||||
|
|
||||||
// Maximum directory depth printed during the recursive boot listing. Guards
|
|
||||||
// the StorageTask stack against pathological directory nesting.
|
|
||||||
#define STORAGE_LIST_MAX_DEPTH 10
|
|
||||||
|
|
||||||
// --- Write-speed estimation -------------------------------------------------
|
|
||||||
|
|
||||||
// Path of the temporary scratch file used to measure write throughput. It is
|
|
||||||
// created, written, measured, then deleted, so it never appears in listings.
|
|
||||||
#define STORAGE_SPEED_MEASURE_PATH STORAGE_MOUNT_POINT "/.writespeed.tmp"
|
|
||||||
|
|
||||||
// Size of the scratch file written per measurement (bytes).
|
|
||||||
#define STORAGE_SPEED_MEASURE_BYTES (0.5 * 1024 * 1024)
|
|
||||||
|
|
||||||
// How often to re-measure write speed (ms). This is a bring-up ESTIMATE only:
|
|
||||||
// it writes STORAGE_SPEED_MEASURE_BYTES to the card on every tick. Once real
|
|
||||||
// logging exists, throughput should be derived from actual logging writes and
|
|
||||||
// this benchmark disabled by setting the interval to 0 (measure once at boot).
|
|
||||||
#define STORAGE_SPEED_MEASURE_INTERVAL_MS 0
|
|
||||||
|
|
||||||
// --- Audio logging ----------------------------------------------------------
|
|
||||||
//
|
|
||||||
// The DataLogger (docs/audio_logging.md) writes one multichannel WAV file
|
|
||||||
// per rotation. The Node collection task feeds it interleaved 40-channel PCM
|
|
||||||
// as chunks; the logger appends them and only the consumer touches the card.
|
|
||||||
//
|
|
||||||
// Stream rate: 40 ch x 48 kHz x 2 B = 3,840,000 B/s. The 128 KB pool holds
|
|
||||||
// ~34 ms of audio, so collection rounds must stay <= ~25 ms (<= 96 KB) or
|
|
||||||
// the pool must grow (see docs/audio_logging.md section 7).
|
|
||||||
|
|
||||||
// WAV parameters (must match what the Nodes produce).
|
|
||||||
#define STORAGE_AUDIO_SAMPLE_RATE_HZ 48000
|
|
||||||
#define STORAGE_AUDIO_CHANNELS 40 // 10 nodes x 4 mics
|
|
||||||
#define STORAGE_AUDIO_BITS 16
|
|
||||||
|
|
||||||
// Chunk pool: STORAGE_LOG_POOL_SIZE chunks of STORAGE_LOG_CHUNK_SIZE bytes,
|
|
||||||
// allocated with MALLOC_CAP_DMA. 128 KB total.
|
|
||||||
#define STORAGE_LOG_CHUNK_SIZE (16 * 1024)
|
|
||||||
#define STORAGE_LOG_POOL_SIZE 8
|
|
||||||
|
|
||||||
// Recording directory and rotation policy.
|
|
||||||
#define STORAGE_LOG_DIR STORAGE_MOUNT_POINT "/audio"
|
|
||||||
#define STORAGE_LOG_ROTATE_BYTES (1024LL * 1024 * 1024) // 1 GiB / file
|
|
||||||
#define STORAGE_LOG_FLUSH_BYTES (16LL * 1024 * 1024) // f_sync cadence
|
|
||||||
|
|
||||||
// Overflow warning LED: lights while chunks are being dropped because the SD
|
|
||||||
// card cannot keep up. GPIO 4 is unused by the SD card lines (SPI 5/18/19/23,
|
|
||||||
// SDMMC 6-11). Change both values if a different LED is wired.
|
|
||||||
#define STORAGE_WARN_LED_GPIO 4
|
|
||||||
#define STORAGE_WARN_LED_ACTIVE_HIGH true
|
|
||||||
@@ -1,98 +0,0 @@
|
|||||||
#include "storage_task.h"
|
|
||||||
|
|
||||||
#include "../storage/storage_config.h"
|
|
||||||
|
|
||||||
|
|
||||||
StorageTask::StorageTask(SDManager& manager, StorageState& state)
|
|
||||||
:
|
|
||||||
storage(manager),
|
|
||||||
storageState(state),
|
|
||||||
taskHandle(nullptr)
|
|
||||||
{
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void StorageTask::start()
|
|
||||||
{
|
|
||||||
// Dedicated task on core 0 so SD card work never blocks the services
|
|
||||||
// running on core 1 (SystemTask / DiagnosticsTask). The 8 KB stack is
|
|
||||||
// sized for the recursive boot-time file listing; the eventual
|
|
||||||
// producer/consumer logging loop will also live here.
|
|
||||||
xTaskCreatePinnedToCore(
|
|
||||||
taskEntry,
|
|
||||||
"StorageTask",
|
|
||||||
8192,
|
|
||||||
this,
|
|
||||||
3,
|
|
||||||
&taskHandle,
|
|
||||||
0
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void StorageTask::taskEntry(void* parameter)
|
|
||||||
{
|
|
||||||
StorageTask* task =
|
|
||||||
static_cast<StorageTask*>(parameter);
|
|
||||||
|
|
||||||
task->run();
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
void StorageTask::run()
|
|
||||||
{
|
|
||||||
Serial.println("[Storage] Initializing SD card...");
|
|
||||||
|
|
||||||
storageState.begin();
|
|
||||||
|
|
||||||
// While the mount has not succeeded the dashboard must show "SD not found".
|
|
||||||
storageState.setMounted(false);
|
|
||||||
|
|
||||||
// Retry the mount until it succeeds. This keeps the card dead-pin-capable:
|
|
||||||
// reseating the card, fixing a wire, or powering the module will bring it
|
|
||||||
// up without rebooting the Hub. (The SD init handshake runs at 400 kHz, so
|
|
||||||
// a failure here is wiring/power/seating/pull-up related, not speed.)
|
|
||||||
while (!storage.begin())
|
|
||||||
{
|
|
||||||
Serial.println("[Storage] SD card mount FAILED.");
|
|
||||||
Serial.println("[Storage] Check: 3.3V power + common ground, CS/SCK/MOSI/MISO wiring,");
|
|
||||||
Serial.println("[Storage] card fully seated (click), and module pull-ups.");
|
|
||||||
Serial.println("[Storage] Retrying in 5 s...");
|
|
||||||
vTaskDelay(pdMS_TO_TICKS(5000));
|
|
||||||
}
|
|
||||||
|
|
||||||
storageState.setMounted(true);
|
|
||||||
storageState.setCardType(storage.cardTypeName());
|
|
||||||
|
|
||||||
storage.printCardInfo();
|
|
||||||
storage.listFiles();
|
|
||||||
|
|
||||||
// Initial write-speed estimate for the dashboard.
|
|
||||||
storageState.setWriteSpeedBps(storage.measureWriteSpeed());
|
|
||||||
storageState.setCapacity(storage.totalBytes(), storage.usedBytes());
|
|
||||||
|
|
||||||
TickType_t lastWake =
|
|
||||||
xTaskGetTickCount();
|
|
||||||
|
|
||||||
uint32_t lastSpeedMeasure = millis();
|
|
||||||
|
|
||||||
while (true)
|
|
||||||
{
|
|
||||||
// Periodically re-estimate write speed and refresh capacity so the
|
|
||||||
// dashboard stays current. An interval of 0 disables re-measuring.
|
|
||||||
if (STORAGE_SPEED_MEASURE_INTERVAL_MS != 0 &&
|
|
||||||
millis() - lastSpeedMeasure >= STORAGE_SPEED_MEASURE_INTERVAL_MS)
|
|
||||||
{
|
|
||||||
lastSpeedMeasure = millis();
|
|
||||||
storageState.setWriteSpeedBps(storage.measureWriteSpeed());
|
|
||||||
storageState.setCapacity(storage.totalBytes(), storage.usedBytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
// Future integration point: a producer/consumer queue will feed
|
|
||||||
// audio data here to be flushed to the card.
|
|
||||||
vTaskDelayUntil(
|
|
||||||
&lastWake,
|
|
||||||
pdMS_TO_TICKS(1000)
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
|
|
||||||
#include "../core/storage_state.h"
|
|
||||||
#include "../storage/sd_manager.h"
|
|
||||||
|
|
||||||
|
|
||||||
class StorageTask
|
|
||||||
{
|
|
||||||
public:
|
|
||||||
StorageTask(SDManager& manager, StorageState& state);
|
|
||||||
|
|
||||||
void start();
|
|
||||||
|
|
||||||
private:
|
|
||||||
static void taskEntry(void* parameter);
|
|
||||||
void run();
|
|
||||||
|
|
||||||
SDManager& storage;
|
|
||||||
StorageState& storageState;
|
|
||||||
|
|
||||||
TaskHandle_t taskHandle = nullptr;
|
|
||||||
};
|
|
||||||
Reference in New Issue
Block a user