Initial project commit

2026-08-08 20:58:06 -06:00
commit ea34c399d7
18 changed files with 1039 additions and 0 deletions
@@ -0,0 +1,3 @@
 .pio
 .clang_complete
 .ccls
@@ -0,0 +1,506 @@
 # High level overview
 This application is less like an Arduino sketch and more like an **embeded application**. It should include the patterns of seperation of concerns, services, interfaces, event-driven communication, configuration management and dependency boundries. Here are the design requirements:
 This project is the **Hub** in a Hub and spoke architechure. The Hub is responsible for creating an access point that all of the spokes (Nodes) can connect to. The Hub will use a round-robbin scheduling to collect sensor data from the spokes. The spokes are collecting audio data and storing it in a buffer that is read by the Hub. This means that we have to be on time and can not fall behind schedule. This means that we should either use or lean heavily toward a real time system. 
 The Hub is also responsible for storing the audio data it is collecting onto an SD card. This **must** be done using 4 bit SDIO to be able to achieve the write speeds required to save all of the audio data. We estimate to support up to 10 Nodes each with 4 microphones. The raw audio should be saved to the SD card for post processing.
 The Hub also will act as the interface with the users. It will contain a web dashboard that users can use to view the status of the Hub and the Nodes. 
 ## Define Hub responsibilities
 My Hub is going to have several independent responsibilities:
 1. **Network Management**
   * Creates WiFi network
   * Handles connected devices
   * Manages IP addresses
   * Handles OTA
   * Handles Node authentication
 2. **Device Management**
   * Knows what sensor nodes exist
   * Tracks online/offline state
   * Stores metadata:
     * device ID
     * last seen time
     * firmware version
     * sensor capabilities
     * signal strength
 3. **Sensor Data Collection**
   * Polls other ESP32 nodes
   * Parses responses
   * Validates data
   * Converts into a common format
 4. **Data Storage**
   * SD card management
   * Logging
   * File rotation
   * Handling SD card failures
 5. **Web Interface**
   * Dashboard
   * API endpoints
   * Configuration pages
   * OTA interface
 6. **System Monitoring**
   * Free heap
   * Uptime
   * CPU usage
   * Connected clients
   * Errors
 7. **Configuration**
   * WiFi settings
   * Sensor definitions
   * Logging intervals
   * Device names
 # Proposed High-Level Structure
 I would lean toward something like this:
 ```
 src/
 │
 ├── main.cpp
 │
 ├── core/
 │   ├── system_manager.cpp
 │   ├── event_bus.cpp
 │   └── config.cpp
 │
 ├── network/
 │   ├── wifi_manager.cpp
 │   ├── ota_manager.cpp
 │   └── api_server.cpp
 │
 ├── sensors/
 │   ├── sensor_manager.cpp
 │   ├── sensor_node.cpp
 │   └── sensor_protocol.cpp
 │
 ├── storage/
 │   ├── sd_manager.cpp
 │   └── data_logger.cpp
 │
 ├── web/
 │   ├── web_server.cpp
 │   ├── dashboard.cpp
 │   └── web_assets/
 │
 └── models/
    ├── sensor_data.h
    ├── device_info.h
    └── system_status.h
 ```
 The goal is that `main.cpp` becomes boring.
 Ideally:
 ```text
 initialize system
 start services
 run loop
 ```
 # How Modules Communicate
 ```
             +----------------+
             |  Web Interface |
             +-------+--------+
                     |
                     |
             +-------v--------+
             |  System State  |
             +-------+--------+
                     |
       +-------------+-------------+
       |             |             |
       v             v             v
 Sensor Manager  SD Logger   Network Manager
 ```
 The web interface does not know how sensors work.
 It asks:
 > "Give me current system state."
 Not:
 > "Go ask the sensor manager, which talks to WiFi, which parses packets..."
 ---
 # Introduce a Data Model Layer
 Instead of passing random variables around:
 ```cpp
 temperature
 humidity
 battery
 deviceName
 ```
 Create common structures.
 Conceptually:
 ```
 SensorReading
 {
    deviceId,
    timestamp,
    temperature,
    humidity,
    batteryVoltage
 }
 ```
 Then everything speaks this language:
 ```
 Sensor Node
      |
      |
      v
 SensorReading
      |
      +------> SD Logger
      |
      +------> Dashboard
      |
      +------> API
 ```
 Now if I change from ESP-NOW to WiFi HTTP to MQTT later, the rest of the system does not care.
 ---
 # Consider a Service-Oriented Architecture
 I would like a service oriented architechure like this:
 ```
 WifiService
 SensorService
 StorageService
 WebService
 OTAService
 ```
 Each service has:
 ```
 begin()
 loop()
 status()
 ```
 Conceptually:
 ```
 setup()
 {
    wifi.begin();
    storage.begin();
    sensors.begin();
    web.begin();
 }
 loop()
 {
    wifi.update();
    sensors.update();
    storage.update();
    web.update();
 }
 ```
 This is very ESP32-friendly.
 ---
 # Use Interfaces Where Things Might Change
 For example, how do sensors communicate?
 Today:
 ```
 Hub ---- HTTP ----> Sensor ESP32
 ```
 Tomorrow:
 ```
 Hub ---- ESP-NOW ----> Sensor ESP32
 ```
 Later:
 ```
 Hub ---- MQTT ----> Sensor ESP32
 ```
 If we build our sensor system around:
 ```
 SensorTransport
 ```
 we can swap the communication method without rewriting our sensor manager.
 Example concept:
 ```
             SensorManager
                    |
                    |
          SensorTransport Interface
          /          |          \
       HTTP       ESP-NOW      MQTT
 ```
 This is a very powerful pattern.
 ---
 # Event-Driven vs Polling
 My system naturally has events:
 Examples:
 * Sensor came online
 * Sensor stopped responding
 * New reading received
 * SD card full
 * OTA started
 Instead of:
 ```cpp
 if(sensorOffline)
 {
   updateWebPage();
 }
 ```
 you could have:
 ```
 SensorManager
      |
      |
      emits:
      SensorOfflineEvent
              |
              |
       +------+------+
       |             |
 Web Interface   Logger
 ```
 This prevents modules from knowing about each other.
 For an ESP32 project, I would not over-engineer this with a full framework, but a simple event queue could be very clean.
 ---
 # Configuration Management
 Avoid hardcoding:
 ```cpp
 const char* wifiName = "MyNetwork";
 ```
 Instead:
 ```
 /config.json
 ```
 Example:
 ```
 {
  "Hub_name": "greenhouse-Hub",
  "logging_interval": 60,
  "sensors": [
      {
        "id": "sensor01",
        "location": "north bed"
      }
  ]
 }
 ```
 Then your web dashboard could eventually modify configuration.
 ---
 # Think About Failure Modes Early
 Embedded systems fail differently than servers.
 Plan for:
 ## SD card removed
 If the SD card is removed, it is acceptable to crash/halt (of course with a
 nice error message) We do not have enough RAM to store the data so it is not
 feasable to buffer until the SD card comes back.
 ## Sensor disappears
 If one of the Nodes dissappear, we need to alert the user. This should be done
 with a warning light on the device however I do not think it is a critical
 failure. As we are recording audio, we can just insert silence in that section
 if the Node drops off.
 ## Power loss
 With power loss I want the stored audio capture to not be corrupted. That is
 about the only requirement. If the system dies mid recording I want to be able
 to still use whatever was recorded before it died.
 ## Corrupt configuration
 I want to have the device configurable through USB. So I can give it a default
 configuration however then end users can use a yaml file to reconfigure the
 device. This should then reboot the device and start it up again.
 # Logging Architecture
 I would avoid:
 ```
 sensor_manager.cpp
 writeFile()
 ```
 Instead:
 ```
 SensorManager
 produces:
 SensorReading
        |
        v
 DataLogger
        |
        v
 SDManager
 ```
 Then the logger can do it's own thing and the sensor system doesn't care.
 ---
 # Mental Model
 Think of the Hub as a small operating system:
 ```
                 HUB
        +----------------+
        | Web Dashboard  |
        +----------------+
        +----------------+
        | System State   |
        +----------------+
 +----------+ +----------+ +----------+
 | Sensors  | | Storage  | | Network  |
 +----------+ +----------+ +----------+
        +----------------+
        | Hardware Layer |
        +----------------+
 ```
 ---
 # Node Hub communication
 This will be done using UDP packets. The Hub will send out a packet to ask for
 data and the Node will reply with a packet containnig the binary audio data.
 ---
 # Node count
 I estimate having 10 nodes sampling 4 microphones each. Doing the math this
 ends up with about 6 MB/sec which is doable if I use 4 bit SDIO and wifi is
 totally capable of handling those speeds.
 ---
 ## Data frequency
 The nodes do not have much RAM so I want to collect frequently. I am thinking
 that when the Nodes connect to the Hub they are assigned a "time slot" so that
 they can dump their data during that time.
 ---
 ## Power
 The Hub is batery powered however it will be a large power bank. The nodes
 however are not and will be kept light weight.
 ---
 My initial recommendation would be:
 * **Service-oriented architecture**
 * **Strong data models**
 * **Central system state**
 * **Event-based communication where useful**
 * **Hardware abstraction boundaries**
 * **Separate firmware projects with shared libraries**
 The next thing I would design is the **communication protocol between the Hub and sensor nodes**, because that decision will influence almost every other piece.
@@ -0,0 +1,37 @@
 This directory is intended for project header files.
 A header file is a file containing C declarations and macro definitions
 to be shared between several project source files. You request the use of a
 header file in your project source file (C, C++, etc) located in `src` folder
 by including it, with the C preprocessing directive `#include'.
 ```src/main.c
 #include "header.h"
 int main (void)
 {
 ...
 }
 ```
 Including a header file produces the same results as copying the header file
 into each source file that needs it. Such copying would be time-consuming
 and error-prone. With a header file, the related declarations appear
 in only one place. If they need to be changed, they can be changed in one
 place, and programs that include the header file will automatically use the
 new version when next recompiled. The header file eliminates the labor of
 finding and changing all the copies as well as the risk that a failure to
 find one copy will result in inconsistencies within a program.
 In C, the convention is to give header files names that end with `.h'.
 Read more about using header files in official GCC documentation:
 * Include Syntax
 * Include Operation
 * Once-Only Headers
 * Computed Includes
 https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html
@@ -0,0 +1,46 @@
 This directory is intended for project specific (private) libraries.
 PlatformIO will compile them to static libraries and link into the executable file.
 The source code of each library should be placed in a separate directory
 ("lib/your_library_name/[Code]").
 For example, see the structure of the following example libraries `Foo` and `Bar`:
 |--lib
 |  |
 |  |--Bar
 |  |  |--docs
 |  |  |--examples
 |  |  |--src
 |  |     |- Bar.c
 |  |     |- Bar.h
 |  |  |- library.json (optional. for custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html
 |  |
 |  |--Foo
 |  |  |- Foo.c
 |  |  |- Foo.h
 |  |
 |  |- README --> THIS FILE
 |
 |- platformio.ini
 |--src
   |- main.c
 Example contents of `src/main.c` using Foo and Bar:
 ```
 #include <Foo.h>
 #include <Bar.h>
 int main (void)
 {
  ...
 }
 ```
 The PlatformIO Library Dependency Finder will find automatically dependent
 libraries by scanning project source files.
 More information about PlatformIO Library Dependency Finder
 - https://docs.platformio.org/page/librarymanager/ldf.html
@@ -0,0 +1,20 @@
 ; PlatformIO Project Configuration File
 ;
 ;   Build options: build flags, source filter
 ;   Upload options: custom upload port, speed and extra flags
 ;   Library options: dependencies, extra library storages
 ;   Advanced options: extra scripting
 ;
 ; Please visit documentation for the other options and examples
 ; https://docs.platformio.org/page/projectconf.html
 [env:esp32dev]
 platform = espressif32
 board = esp32dev
 framework = arduino
 ;lib_deps =
 ; upload via OTA
 upload_protocol = espota
 upload_port = 192.168.4.1
@@ -0,0 +1,10 @@
 # Hub board
 Responsibilities:
 - Host the network
 - Round Robin collection of audio data from nodes
 - Simple user interface via web
 - Fast writing to an SD card using SDIO
 board: esp32dev
@@ -0,0 +1,2 @@
 pio
 platformio
@@ -0,0 +1,36 @@
 // Nathan Hinton
 // Main file for the hub node. Starting with the WIFI initialization.
 #include <Arduino.h>
 #include "services/service_manager.h"
 #include "services/wifi_service.h"
 #include "services/web_service.h"
 #include "services/ota_service.h"
 ServiceManager services;
 WiFiService wifi;
 WebService web;
 OTAService ota;
 void setup()
 {
    Serial.begin(115200);
    services.add(&wifi);
    services.add(&ota);
    services.add(&web);
    services.begin();
 }
 void loop()
 {
    services.update();
 }
@@ -0,0 +1,25 @@
 #include "ota_service.h"
 void OTAService::begin()
 {
    ArduinoOTA.setHostname("ESP32-Dashboard");
    ArduinoOTA.onStart([](){
        Serial.println("OTA started");
    });
    ArduinoOTA.onEnd([](){
        Serial.println("OTA finished");
    });
    ArduinoOTA.begin();
    Serial.println("OTA ready");
 }
 void OTAService::update()
 {
    ArduinoOTA.handle();
 }
@@ -0,0 +1,16 @@
 #pragma once
 #include "service.h"
 #include <ArduinoOTA.h>
 class OTAService : public Service {
 public:
    OTAService()
        : Service("OTA", 10)
    {}
    void begin() override;
    void update() override;
 };
@@ -0,0 +1,46 @@
 #pragma once
 #include <Arduino.h>
 class Service {
 public:
    Service(const char* name, uint32_t intervalMs)
        : name(name),
          intervalMs(intervalMs),
          lastRun(0),
          lastExecutionUs(0)
    {}
    virtual void begin() = 0;
    virtual void update() = 0;
    void run()
    {
        uint32_t now = millis();
        if (now - lastRun < intervalMs)
            return;
        lastRun = now;
        uint32_t start = micros();
        update();
        lastExecutionUs = micros() - start;
    }
    const char* getName() {
        return name;
    }
    uint32_t getExecutionTimeUs() {
        return lastExecutionUs;
    }
 protected:
    const char* name;
    uint32_t intervalMs;
    uint32_t lastRun;
    uint32_t lastExecutionUs;
 };
@@ -0,0 +1,27 @@
 #include "service_manager.h"
 void ServiceManager::add(Service* service)
 {
    if(count < MAX_SERVICES)
    {
        services[count++] = service;
    }
 }
 void ServiceManager::begin()
 {
    for(uint8_t i = 0; i < count; i++)
    {
        services[i]->begin();
    }
 }
 void ServiceManager::update()
 {
    for(uint8_t i = 0; i < count; i++)
    {
        services[i]->run();
    }
 }
@@ -0,0 +1,20 @@
 #pragma once
 #include "service.h"
 #define MAX_SERVICES 10
 class ServiceManager {
 public:
    void add(Service* service);
    void begin();
    void update();
 private:
    Service* services[MAX_SERVICES];
    uint8_t count = 0;
 };
@@ -0,0 +1,161 @@
 #include "web_service.h"
 #define FIRMWARE_VERSION "1.0.0"
 WebService::WebService()
    :
    Service("Web", 10),
    server(80)
 {
 }
 String formatUptime() {
  unsigned long seconds = millis() / 1000;
  unsigned long days = seconds / 86400;
  seconds %= 86400;
  unsigned long hours = seconds / 3600;
  seconds %= 3600;
  unsigned long minutes = seconds / 60;
  seconds %= 60;
  char buffer[64];
  snprintf(buffer, sizeof(buffer),
           "%lu days %02lu:%02lu:%02lu",
           days, hours, minutes, seconds);
  return String(buffer);
 }
 String ROOT_HTML = R"rawliteral(
 <!DOCTYPE html>
 <html>
 <head>
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <title>ESP32 Dashboard</title>
 <style>
 body {
  font-family: Arial, sans-serif;
  text-align: center;
  margin-top: 50px;
  background: #111;
  color: white;
 }
 .card {
  background: #222;
  padding: 25px;
  margin: auto;
  width: 80%;
  max-width: 400px;
  border-radius: 15px;
 }
 h1 {
  color: #00ff99;
 }
 #uptime {
  font-size: 28px;
  margin-top: 20px;
 }
 </style>
 <script>
 function updateUptime() {
  fetch('/uptime')
    .then(response => response.text())
    .then(data => {
      document.getElementById("uptime").innerHTML = data;
    });
 }
 setInterval(updateUptime, 1000);
 window.onload = updateUptime;
 </script>
 <script>
 function getVersion() {
  fetch('/version')
    .then(response => response.text())
    .then(data => {
      document.getElementById("firmware_version").innerHTML = data;
    });
 }
 window.onload = getVersion;
 </script>
 </head>
 <body>
 <div class="card">
 <h1>ESP32 Dashboard OTA</h1>
 <p>Device uptime:</p>
 <div id="uptime">Loading...</div>
 </div>
 <footer>
 <div id="firmware_version">Loading...</div>
 </footer>
 </body>
 </html>
 )rawliteral";
 //   server.send(200, "text/html", html);
 // }
 // 
 // void handleUptime() {
 //   server.send(200, "text/plain", formatUptime());
 // }
 // 
 // void handleVersion() {
 //   server.send(200, "text/plain", FIRMWARE_VERSION);
 // }
 // 
 void WebService::begin() {
    // Root path
    server.on("/", [this](){
        server.send(
                    200,
                    "text/html",
                    ROOT_HTML
                    );    
    });
    // Version path
    server.on("/version", [this](){
        server.send(
                    200,
                    "text/plain",
                    FIRMWARE_VERSION
                    );
    });
    // Uptime path
    server.on("/uptime", [this]() {
        server.send(
                    200,
                    "text/plain",
                    formatUptime()
                    );
    });
    server.begin();
    Serial.println("Web server started");
 }
 void WebService::update()
 {
    server.handleClient();
 }
@@ -0,0 +1,19 @@
 #pragma once
 #include "service.h"
 #include <WebServer.h>
 class WebService : public Service {
 public:
    WebService();
    void begin() override;
    void update() override;
 private:
    WebServer server;
 };
@@ -0,0 +1,39 @@
 #include "wifi_service.h"
 #include <WiFi.h>
 const char* ssid = "ESP32-Dashboard";
 const char* password = "esp32password";
 WiFiService::WiFiService()
 :
 Service("WiFi", 1000)
 {
 }
 void WiFiService::begin()
 {
    WiFi.mode(WIFI_AP);
    WiFi.softAP(
        ssid,
        password
    );
    Serial.print("IP: ");
    Serial.println(WiFi.softAPIP());
 }
 void WiFiService::update()
 {
    // Future:
    // monitor connected clients
    // reconnect logic
    // diagnostics
 }
@@ -0,0 +1,15 @@
 #pragma once
 #include "service.h"
 class WiFiService : public Service {
 public:
    WiFiService();
    void begin() override;
    void update() override;
 };
@@ -0,0 +1,11 @@
 This directory is intended for PlatformIO Test Runner and project tests.
 Unit Testing is a software testing method by which individual units of
 source code, sets of one or more MCU program modules together with associated
 control data, usage procedures, and operating procedures, are tested to
 determine whether they are fit for use. Unit testing finds problems early
 in the development cycle.
 More information about PlatformIO Unit Testing:
 - https://docs.platformio.org/en/latest/advanced/unit-testing/index.html