Working harvesting detection, working manufacturing detection.
This commit is contained in:
@@ -0,0 +1,144 @@
|
||||
# AGENTS.md
|
||||
|
||||
Guidance for AI agents working in this repo. Read before editing. The goal of
|
||||
these rules is to keep the code clean, consistent, and easy to extend without
|
||||
breaking the JSONL log contract or the test suite.
|
||||
|
||||
## Project in one line
|
||||
|
||||
Capture the Eternal Lands X11 window, detect harvesting + the 5 HUD bar
|
||||
numbers, track the draggable Manufacturing/Inventory windows, OCR the craft
|
||||
status message, and append one JSON object per frame to a JSONL log for a
|
||||
future "observer" to consume.
|
||||
|
||||
## Commands
|
||||
|
||||
Run everything through the project virtualenv — the system Python is
|
||||
externally managed (PEP 668) and lacks the deps:
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate # then plain python/pytest work
|
||||
# or, without activating:
|
||||
.venv/bin/python -m pytest tests/ -v
|
||||
.venv/bin/python main.py
|
||||
.venv/bin/python main.py --replay tests/images/harvesting
|
||||
```
|
||||
|
||||
- Always run the full test suite before finishing: `.venv/bin/python -m pytest tests/`
|
||||
- If you add a Python dependency, add it to `requirements.txt` and install it
|
||||
into `.venv`.
|
||||
|
||||
## Architecture and responsibilities
|
||||
|
||||
- `capture.py` — the ONLY module that touches X11. Returns PIL RGBA images.
|
||||
- `analyze.py` — pure functions only (no I/O, no side effects). Every function
|
||||
takes a PIL image (or path) and returns data. Includes window detection
|
||||
(`verify_window`, `find_windows`), craft-message OCR
|
||||
(`read_manufacturing_message`), and `classify_message`.
|
||||
- `main.py` — CLI, capture loop, replay mode, JSON record construction, and
|
||||
logging. Owns all file/stdout I/O. Also owns the window-position cache
|
||||
(`WindowTracker`): verify the cached position each frame, rescan the full
|
||||
frame only when a window moves.
|
||||
- `tests/` — folder-driven pytest tests plus fixture images.
|
||||
|
||||
Keep this split. New analysis logic goes in `analyze.py`. New ways to run the
|
||||
loop go in `main.py`. Never put capture logic in `analyze.py` or analysis in
|
||||
`capture.py`.
|
||||
|
||||
## Code style
|
||||
|
||||
- **No comments unless they explain WHY** (non-obvious decisions, measured
|
||||
constants). Docstrings on public functions and modules are expected. Match
|
||||
the existing tone: short, declarative docstrings; a brief comment only where
|
||||
the reasoning is not obvious (e.g. "y0 starts at 1033 because row 1032 is
|
||||
the panel border, which confuses OCR").
|
||||
- Standard library + the deps in `requirements.txt` only. Do not introduce
|
||||
new heavy dependencies without a strong reason and a requirements.txt
|
||||
update.
|
||||
- Return plain dicts/lists of primitives (JSON-serializable) from `analyze.py`
|
||||
functions — never objects or PIL types — so results drop straight into the
|
||||
log.
|
||||
- Function names are snake_case and descriptive (`is_harvesting`,
|
||||
`read_bar_values`, `load_image`).
|
||||
- Module-level constants are UPPER_SNAKE_CASE; private helpers/constants are
|
||||
`_leading_underscore`.
|
||||
- No type hints required, but if you add them keep them consistent across a
|
||||
file.
|
||||
|
||||
## Screen-coordinate calibration
|
||||
|
||||
All pixel coordinates are measured values for a **1920x1080** window and live
|
||||
as constants at the top of `analyze.py` (`HARVEST_PIXEL`,
|
||||
`HARVEST_LUMA_THRESHOLD`, `NUMBER_REGIONS`).
|
||||
|
||||
Window coordinates are **relative to the window title bar**, never fixed
|
||||
screen positions — the player can drag the Manufacturing/Inventory windows.
|
||||
Calibrated constants: `MESSAGE_SEARCH` (the craft message sits at
|
||||
`(title_x - 161, title_y + 121)`, verified at 5 different window positions)
|
||||
and the `TITLE_VERIFY_CROP`/`FIND_TITLE_OFFSETS` tuning.
|
||||
|
||||
- When you change a coordinate or threshold, you MUST re-run the tests. If you
|
||||
have a screenshot proving the new value, drop it in the right
|
||||
`tests/images/` folder first so the tests verify your change.
|
||||
- Never "adjust" a coordinate in a test to make it pass. Verify against the
|
||||
actual images.
|
||||
- If the game window resolution changes, the calibration must be redone from
|
||||
real screenshots — do not guess coordinates.
|
||||
|
||||
## Tests
|
||||
|
||||
- Tests are **folder-driven**:
|
||||
- `tests/images/harvesting/*.png` → `is_harvesting` must be True
|
||||
- `tests/images/not_harvesting/*.png` → must be False
|
||||
- `tests/images/corrupt/*.png` → `load_image` must return None, no crash
|
||||
- `tests/images/manufacturing/<category>/*.png` → the craft message read at
|
||||
the pinned title position must classify to `<category>`
|
||||
- `tests/images/inventory/*.png` → `verify_window` at the pinned position
|
||||
- Adding an edge case = dropping a PNG into the right folder. Do that instead
|
||||
of writing one-off test code, unless a new *expected value* must be pinned
|
||||
(then update `tests/test_bars.py`, e.g. `INVENTORY_EXPECTED`, or
|
||||
`tests/test_windows.py`, e.g. `MFG_POSITION`).
|
||||
- Keep fixture images small in number and real (actual captured frames), not
|
||||
synthetic.
|
||||
- Pinned OCR values in `test_bars.py` are ground truth from real frames. If a
|
||||
frame's expected value must change, verify the number by reading the image
|
||||
— never just update the test to match a new wrong OCR result.
|
||||
|
||||
## The JSONL log contract
|
||||
|
||||
`main.py` appends one JSON object per line to `analysis.jsonl` (default).
|
||||
Shape:
|
||||
|
||||
```json
|
||||
{"time": "…", "frame": 4, "is_harvesting": true, "harvest_pixel": "#f2c285", "harvest_luma": 201, "bars": {"mana": 32, "food": 39, "health": 40, "inventory": 106, "action_points": 140}, "manufacturing_window": [1104, 327], "manufacturing_message": {"text": "you successfully created 1 potion of minor healing", "color": "green", "category": "success"}, "inventory_window": [1629, 686]}
|
||||
```
|
||||
|
||||
- The future observer parses this log. **Do not change field names, types, or
|
||||
add/remove fields casually.** If you must extend it, add new keys (never
|
||||
repurpose existing ones) and update `design.md`.
|
||||
- `bars.*` may be `null` when OCR fails — consumers must tolerate that; keep
|
||||
it that way.
|
||||
- `manufacturing_window` / `inventory_window` are `[x, y]` title-bar
|
||||
coordinates, or `null` when the window is not currently verified.
|
||||
`manufacturing_message` is `{text, color, category}` (`text`/`color` are
|
||||
`null` when no message shows), or `null` when the window position is
|
||||
unverified. Message categories: `working`, `success`, `stopped`, `failed`,
|
||||
`failed_lost`, `hungry`, `overloaded`, `none`, `unknown`. Consumers must
|
||||
tolerate all `null` cases.
|
||||
- Keep the log append-only and line-per-record (JSONL). It is tail-friendly.
|
||||
|
||||
## Error handling
|
||||
|
||||
- Analysis functions must never crash the capture loop. `analyze.py` catches
|
||||
per-item failures (e.g. OCR returns `None` on error) and returns `None`
|
||||
rather than raising.
|
||||
- Optional dependency pattern: `pytesseract` is imported inside
|
||||
`try/except ImportError` and the module degrades gracefully
|
||||
(`pytesseract = None`). Follow this pattern for any new optional dep.
|
||||
- Corrupt images → `load_image` returns `None`; callers skip them.
|
||||
|
||||
## Docs
|
||||
|
||||
- `design.md` — the source of truth for capabilities, calibration,
|
||||
architecture, and the log format. Keep it in sync when you change behavior.
|
||||
- `AGENTS.md` — these rules. Update it if a convention changes.
|
||||
Reference in New Issue
Block a user