audio_project_hub/design.md

# High level overview

This application is less like an Arduino sketch and more like an **embeded application**. It should include the patterns of seperation of concerns, services, interfaces, event-driven communication, configuration management and dependency boundries. Here are the design requirements:


This project is the **Hub** in a Hub and spoke architechure. The Hub is responsible for creating an access point that all of the spokes (Nodes) can connect to. The Hub will use a round-robbin scheduling to collect sensor data from the spokes. The spokes are collecting audio data and storing it in a buffer that is read by the Hub. This means that we have to be on time and can not fall behind schedule. This means that we should either use or lean heavily toward a real time system. 

The Hub is also responsible for storing the audio data it is collecting onto an SD card. This **must** be done using 4 bit SDIO to be able to achieve the write speeds required to save all of the audio data. We estimate to support up to 10 Nodes each with 4 microphones. The raw audio should be saved to the SD card for post processing.

The Hub also will act as the interface with the users. It will contain a web dashboard that users can use to view the status of the Hub and the Nodes. 


## Define Hub responsibilities


My Hub is going to have several independent responsibilities:

1. **Network Management**

   * Creates WiFi network
   * Handles connected devices
   * Manages IP addresses
   * Handles OTA
   * Handles Node authentication

2. **Device Management**

   * Knows what sensor nodes exist
   * Tracks online/offline state
   * Stores metadata:

     * device ID
     * last seen time
     * firmware version
     * sensor capabilities
     * signal strength

3. **Sensor Data Collection**

   * Polls other ESP32 nodes
   * Parses responses
   * Validates data
   * Converts into a common format

4. **Data Storage**

   * SD card management
   * Logging
   * File rotation
   * Handling SD card failures

5. **Web Interface**

   * Dashboard
   * API endpoints
   * Configuration pages
   * OTA interface

6. **System Monitoring**

   * Free heap
   * Uptime
   * CPU usage
   * Connected clients
   * Errors

7. **Configuration**

   * WiFi settings
   * Sensor definitions
   * Logging intervals
   * Device names


# Proposed High-Level Structure

I would lean toward something like this:

```
src/
│
├── main.cpp
│
├── core/
│   ├── system_manager.cpp
│   ├── event_bus.cpp
│   └── config.cpp
│
├── network/
│   ├── wifi_manager.cpp
│   ├── ota_manager.cpp
│   └── api_server.cpp
│
├── sensors/
│   ├── sensor_manager.cpp
│   ├── sensor_node.cpp
│   └── sensor_protocol.cpp
│
├── storage/
│   ├── sd_manager.cpp
│   └── data_logger.cpp
│
├── web/
│   ├── web_server.cpp
│   ├── dashboard.cpp
│   └── web_assets/
│
└── models/
    ├── sensor_data.h
    ├── device_info.h
    └── system_status.h
```

The goal is that `main.cpp` becomes boring.

Ideally:

```text
initialize system
start services
run loop
```


# How Modules Communicate


```
             +----------------+
             |  Web Interface |
             +-------+--------+
                     |
                     |
             +-------v--------+
             |  System State  |
             +-------+--------+
                     |
       +-------------+-------------+
       |             |             |
       v             v             v

 Sensor Manager  SD Logger   Network Manager
```

The web interface does not know how sensors work.

It asks:

> "Give me current system state."

Not:

> "Go ask the sensor manager, which talks to WiFi, which parses packets..."

---

# Introduce a Data Model Layer

Instead of passing random variables around:

```cpp
temperature
humidity
battery
deviceName
```

Create common structures.

Conceptually:

```
SensorReading

{
    deviceId,
    timestamp,
    temperature,
    humidity,
    batteryVoltage
}
```

Then everything speaks this language:

```
Sensor Node
      |
      |
      v
SensorReading
      |
      +------> SD Logger
      |
      +------> Dashboard
      |
      +------> API
```

Now if I change from ESP-NOW to WiFi HTTP to MQTT later, the rest of the system does not care.

---

# Consider a Service-Oriented Architecture

I would like a service oriented architechure like this:

```
WifiService
SensorService
StorageService
WebService
OTAService
```

Each service has:

```
begin()
loop()
status()
```

Conceptually:

```
setup()
{
    wifi.begin();
    storage.begin();
    sensors.begin();
    web.begin();
}


loop()
{
    wifi.update();
    sensors.update();
    storage.update();
    web.update();
}
```

This is very ESP32-friendly.

---

# Use Interfaces Where Things Might Change

For example, how do sensors communicate?

Today:

```
Hub ---- HTTP ----> Sensor ESP32
```

Tomorrow:

```
Hub ---- ESP-NOW ----> Sensor ESP32
```

Later:

```
Hub ---- MQTT ----> Sensor ESP32
```

If we build our sensor system around:

```
SensorTransport
```

we can swap the communication method without rewriting our sensor manager.

Example concept:

```
             SensorManager

                    |
                    |

          SensorTransport Interface

          /          |          \

       HTTP       ESP-NOW      MQTT
```

This is a very powerful pattern.

---

# Event-Driven vs Polling

My system naturally has events:

Examples:

* Sensor came online
* Sensor stopped responding
* New reading received
* SD card full
* OTA started

Instead of:

```cpp
if(sensorOffline)
{
   updateWebPage();
}
```

you could have:

```
SensorManager
      |
      |
      emits:
      SensorOfflineEvent

              |
              |
       +------+------+
       |             |
 Web Interface   Logger
```

This prevents modules from knowing about each other.

For an ESP32 project, I would not over-engineer this with a full framework, but a simple event queue could be very clean.

---

# Configuration Management

Avoid hardcoding:

```cpp
const char* wifiName = "MyNetwork";
```

Instead:

```
/config.json
```

Example:

```
{
  "Hub_name": "greenhouse-Hub",
  "logging_interval": 60,
  "sensors": [
      {
        "id": "sensor01",
        "location": "north bed"
      }
  ]
}
```

Then your web dashboard could eventually modify configuration.

---

# Think About Failure Modes Early

Embedded systems fail differently than servers.

Plan for:

## SD card removed

If the SD card is removed, it is acceptable to crash/halt (of course with a
nice error message) We do not have enough RAM to store the data so it is not
feasable to buffer until the SD card comes back.

## Sensor disappears

If one of the Nodes dissappear, we need to alert the user. This should be done
with a warning light on the device however I do not think it is a critical
failure. As we are recording audio, we can just insert silence in that section
if the Node drops off.

## Power loss

With power loss I want the stored audio capture to not be corrupted. That is
about the only requirement. If the system dies mid recording I want to be able
to still use whatever was recorded before it died.

## Corrupt configuration

I want to have the device configurable through USB. So I can give it a default
configuration however then end users can use a yaml file to reconfigure the
device. This should then reboot the device and start it up again.

# Logging Architecture

I would avoid:

```
sensor_manager.cpp

writeFile()
```

Instead:

```
SensorManager

produces:

SensorReading

        |
        v

DataLogger

        |
        v

SDManager
```

Then the logger can do it's own thing and the sensor system doesn't care.

---

# Mental Model

Think of the Hub as a small operating system:

```
                 HUB

        +----------------+
        | Web Dashboard  |
        +----------------+

        +----------------+
        | System State   |
        +----------------+

 +----------+ +----------+ +----------+
 | Sensors  | | Storage  | | Network  |
 +----------+ +----------+ +----------+

        +----------------+
        | Hardware Layer |
        +----------------+
```

---

# Node Hub communication

This will be done using UDP packets. The Hub will send out a packet to ask for
data and the Node will reply with a packet containnig the binary audio data.

---

# Node count

I estimate having 10 nodes sampling 4 microphones each. Doing the math this
ends up with about 6 MB/sec which is doable if I use 4 bit SDIO and wifi is
totally capable of handling those speeds.


---

## Data frequency

The nodes do not have much RAM so I want to collect frequently. I am thinking
that when the Nodes connect to the Hub they are assigned a "time slot" so that
they can dump their data during that time.

---

## Power

The Hub is batery powered however it will be a large power bank. The nodes
however are not and will be kept light weight.

---


My initial recommendation would be:

* **Service-oriented architecture**
* **Strong data models**
* **Central system state**
* **Event-based communication where useful**
* **Hardware abstraction boundaries**
* **Separate firmware projects with shared libraries**

The next thing I would design is the **communication protocol between the Hub and sensor nodes**, because that decision will influence almost every other piece.