9.3 KiB
High level overview
This application is less like an Arduino sketch and more like an embeded application. It should include the patterns of seperation of concerns, services, interfaces, event-driven communication, configuration management and dependency boundries. Here are the design requirements:
This project is the Hub in a Hub and spoke architechure. The Hub is responsible for creating an access point that all of the spokes (Nodes) can connect to. The Hub will use a round-robbin scheduling to collect sensor data from the spokes. The spokes are collecting audio data and storing it in a buffer that is read by the Hub. This means that we have to be on time and can not fall behind schedule. This means that we should either use or lean heavily toward a real time system.
The Hub is also responsible for storing the audio data it is collecting onto an SD card. This must be done using 4 bit SDIO to be able to achieve the write speeds required to save all of the audio data. We estimate to support up to 10 Nodes each with 4 microphones. The raw audio should be saved to the SD card for post processing.
The Hub also will act as the interface with the users. It will contain a web dashboard that users can use to view the status of the Hub and the Nodes.
Define Hub responsibilities
My Hub is going to have several independent responsibilities:
-
Network Management
- Creates WiFi network
- Handles connected devices
- Manages IP addresses
- Handles OTA
- Handles Node authentication
-
Device Management
-
Knows what sensor nodes exist
-
Tracks online/offline state
-
Stores metadata:
- device ID
- last seen time
- firmware version
- sensor capabilities
- signal strength
-
-
Sensor Data Collection
- Polls other ESP32 nodes
- Parses responses
- Validates data
- Converts into a common format
-
Data Storage
- SD card management
- Logging
- File rotation
- Handling SD card failures
-
Web Interface
- Dashboard
- API endpoints
- Configuration pages
- OTA interface
-
System Monitoring
- Free heap
- Uptime
- CPU usage
- Connected clients
- Errors
-
Configuration
- WiFi settings
- Sensor definitions
- Logging intervals
- Device names
Proposed High-Level Structure
I would lean toward something like this:
src/
│
├── main.cpp
│
├── core/
│ ├── system_manager.cpp
│ ├── event_bus.cpp
│ └── config.cpp
│
├── network/
│ ├── wifi_manager.cpp
│ ├── ota_manager.cpp
│ └── api_server.cpp
│
├── sensors/
│ ├── sensor_manager.cpp
│ ├── sensor_node.cpp
│ └── sensor_protocol.cpp
│
├── storage/
│ ├── sd_manager.cpp
│ └── data_logger.cpp
│
├── web/
│ ├── web_server.cpp
│ ├── dashboard.cpp
│ └── web_assets/
│
└── models/
├── sensor_data.h
├── device_info.h
└── system_status.h
The goal is that main.cpp becomes boring.
Ideally:
initialize system
start services
run loop
How Modules Communicate
+----------------+
| Web Interface |
+-------+--------+
|
|
+-------v--------+
| System State |
+-------+--------+
|
+-------------+-------------+
| | |
v v v
Sensor Manager SD Logger Network Manager
The web interface does not know how sensors work.
It asks:
"Give me current system state."
Not:
"Go ask the sensor manager, which talks to WiFi, which parses packets..."
Introduce a Data Model Layer
Instead of passing random variables around:
temperature
humidity
battery
deviceName
Create common structures.
Conceptually:
SensorReading
{
deviceId,
timestamp,
temperature,
humidity,
batteryVoltage
}
Then everything speaks this language:
Sensor Node
|
|
v
SensorReading
|
+------> SD Logger
|
+------> Dashboard
|
+------> API
Now if I change from ESP-NOW to WiFi HTTP to MQTT later, the rest of the system does not care.
Consider a Service-Oriented Architecture
I would like a service oriented architechure like this:
WifiService
SensorService
StorageService
WebService
OTAService
Each service has:
begin()
loop()
status()
Conceptually:
setup()
{
wifi.begin();
storage.begin();
sensors.begin();
web.begin();
}
loop()
{
wifi.update();
sensors.update();
storage.update();
web.update();
}
This is very ESP32-friendly.
Use Interfaces Where Things Might Change
For example, how do sensors communicate?
Today:
Hub ---- HTTP ----> Sensor ESP32
Tomorrow:
Hub ---- ESP-NOW ----> Sensor ESP32
Later:
Hub ---- MQTT ----> Sensor ESP32
If we build our sensor system around:
SensorTransport
we can swap the communication method without rewriting our sensor manager.
Example concept:
SensorManager
|
|
SensorTransport Interface
/ | \
HTTP ESP-NOW MQTT
This is a very powerful pattern.
Event-Driven vs Polling
My system naturally has events:
Examples:
- Sensor came online
- Sensor stopped responding
- New reading received
- SD card full
- OTA started
Instead of:
if(sensorOffline)
{
updateWebPage();
}
you could have:
SensorManager
|
|
emits:
SensorOfflineEvent
|
|
+------+------+
| |
Web Interface Logger
This prevents modules from knowing about each other.
For an ESP32 project, I would not over-engineer this with a full framework, but a simple event queue could be very clean.
Configuration Management
Avoid hardcoding:
const char* wifiName = "MyNetwork";
Instead:
/config.json
Example:
{
"Hub_name": "greenhouse-Hub",
"logging_interval": 60,
"sensors": [
{
"id": "sensor01",
"location": "north bed"
}
]
}
Then your web dashboard could eventually modify configuration.
Think About Failure Modes Early
Embedded systems fail differently than servers.
Plan for:
SD card removed
If the SD card is removed, it is acceptable to crash/halt (of course with a nice error message) We do not have enough RAM to store the data so it is not feasable to buffer until the SD card comes back.
Sensor disappears
If one of the Nodes dissappear, we need to alert the user. This should be done with a warning light on the device however I do not think it is a critical failure. As we are recording audio, we can just insert silence in that section if the Node drops off.
Power loss
With power loss I want the stored audio capture to not be corrupted. That is about the only requirement. If the system dies mid recording I want to be able to still use whatever was recorded before it died.
Corrupt configuration
I want to have the device configurable through USB. So I can give it a default configuration however then end users can use a yaml file to reconfigure the device. This should then reboot the device and start it up again.
Logging Architecture
I would avoid:
sensor_manager.cpp
writeFile()
Instead:
SensorManager
produces:
SensorReading
|
v
DataLogger
|
v
SDManager
Then the logger can do it's own thing and the sensor system doesn't care.
Mental Model
Think of the Hub as a small operating system:
HUB
+----------------+
| Web Dashboard |
+----------------+
+----------------+
| System State |
+----------------+
+----------+ +----------+ +----------+
| Sensors | | Storage | | Network |
+----------+ +----------+ +----------+
+----------------+
| Hardware Layer |
+----------------+
Node Hub communication
This will be done using UDP packets. The Hub will send out a packet to ask for data and the Node will reply with a packet containnig the binary audio data.
Node count
I estimate having 10 nodes sampling 4 microphones each. Doing the math this ends up with about 6 MB/sec which is doable if I use 4 bit SDIO and wifi is totally capable of handling those speeds.
Data frequency
The nodes do not have much RAM so I want to collect frequently. I am thinking that when the Nodes connect to the Hub they are assigned a "time slot" so that they can dump their data during that time.
Power
The Hub is batery powered however it will be a large power bank. The nodes however are not and will be kept light weight.
My initial recommendation would be:
- Service-oriented architecture
- Strong data models
- Central system state
- Event-based communication where useful
- Hardware abstraction boundaries
- Separate firmware projects with shared libraries
The next thing I would design is the communication protocol between the Hub and sensor nodes, because that decision will influence almost every other piece.