145 lines
6.8 KiB
Markdown
145 lines
6.8 KiB
Markdown
# AGENTS.md
|
|
|
|
Guidance for AI agents working in this repo. Read before editing. The goal of
|
|
these rules is to keep the code clean, consistent, and easy to extend without
|
|
breaking the JSONL log contract or the test suite.
|
|
|
|
## Project in one line
|
|
|
|
Capture the Eternal Lands X11 window, detect harvesting + the 5 HUD bar
|
|
numbers, track the draggable Manufacturing/Inventory windows, OCR the craft
|
|
status message, and append one JSON object per frame to a JSONL log for a
|
|
future "observer" to consume.
|
|
|
|
## Commands
|
|
|
|
Run everything through the project virtualenv — the system Python is
|
|
externally managed (PEP 668) and lacks the deps:
|
|
|
|
```bash
|
|
source .venv/bin/activate # then plain python/pytest work
|
|
# or, without activating:
|
|
.venv/bin/python -m pytest tests/ -v
|
|
.venv/bin/python main.py
|
|
.venv/bin/python main.py --replay tests/images/harvesting
|
|
```
|
|
|
|
- Always run the full test suite before finishing: `.venv/bin/python -m pytest tests/`
|
|
- If you add a Python dependency, add it to `requirements.txt` and install it
|
|
into `.venv`.
|
|
|
|
## Architecture and responsibilities
|
|
|
|
- `capture.py` — the ONLY module that touches X11. Returns PIL RGBA images.
|
|
- `analyze.py` — pure functions only (no I/O, no side effects). Every function
|
|
takes a PIL image (or path) and returns data. Includes window detection
|
|
(`verify_window`, `find_windows`), craft-message OCR
|
|
(`read_manufacturing_message`), and `classify_message`.
|
|
- `main.py` — CLI, capture loop, replay mode, JSON record construction, and
|
|
logging. Owns all file/stdout I/O. Also owns the window-position cache
|
|
(`WindowTracker`): verify the cached position each frame, rescan the full
|
|
frame only when a window moves.
|
|
- `tests/` — folder-driven pytest tests plus fixture images.
|
|
|
|
Keep this split. New analysis logic goes in `analyze.py`. New ways to run the
|
|
loop go in `main.py`. Never put capture logic in `analyze.py` or analysis in
|
|
`capture.py`.
|
|
|
|
## Code style
|
|
|
|
- **No comments unless they explain WHY** (non-obvious decisions, measured
|
|
constants). Docstrings on public functions and modules are expected. Match
|
|
the existing tone: short, declarative docstrings; a brief comment only where
|
|
the reasoning is not obvious (e.g. "y0 starts at 1033 because row 1032 is
|
|
the panel border, which confuses OCR").
|
|
- Standard library + the deps in `requirements.txt` only. Do not introduce
|
|
new heavy dependencies without a strong reason and a requirements.txt
|
|
update.
|
|
- Return plain dicts/lists of primitives (JSON-serializable) from `analyze.py`
|
|
functions — never objects or PIL types — so results drop straight into the
|
|
log.
|
|
- Function names are snake_case and descriptive (`is_harvesting`,
|
|
`read_bar_values`, `load_image`).
|
|
- Module-level constants are UPPER_SNAKE_CASE; private helpers/constants are
|
|
`_leading_underscore`.
|
|
- No type hints required, but if you add them keep them consistent across a
|
|
file.
|
|
|
|
## Screen-coordinate calibration
|
|
|
|
All pixel coordinates are measured values for a **1920x1080** window and live
|
|
as constants at the top of `analyze.py` (`HARVEST_PIXEL`,
|
|
`HARVEST_LUMA_THRESHOLD`, `NUMBER_REGIONS`).
|
|
|
|
Window coordinates are **relative to the window title bar**, never fixed
|
|
screen positions — the player can drag the Manufacturing/Inventory windows.
|
|
Calibrated constants: `MESSAGE_SEARCH` (the craft message sits at
|
|
`(title_x - 161, title_y + 121)`, verified at 5 different window positions)
|
|
and the `TITLE_VERIFY_CROP`/`FIND_TITLE_OFFSETS` tuning.
|
|
|
|
- When you change a coordinate or threshold, you MUST re-run the tests. If you
|
|
have a screenshot proving the new value, drop it in the right
|
|
`tests/images/` folder first so the tests verify your change.
|
|
- Never "adjust" a coordinate in a test to make it pass. Verify against the
|
|
actual images.
|
|
- If the game window resolution changes, the calibration must be redone from
|
|
real screenshots — do not guess coordinates.
|
|
|
|
## Tests
|
|
|
|
- Tests are **folder-driven**:
|
|
- `tests/images/harvesting/*.png` → `is_harvesting` must be True
|
|
- `tests/images/not_harvesting/*.png` → must be False
|
|
- `tests/images/corrupt/*.png` → `load_image` must return None, no crash
|
|
- `tests/images/manufacturing/<category>/*.png` → the craft message read at
|
|
the pinned title position must classify to `<category>`
|
|
- `tests/images/inventory/*.png` → `verify_window` at the pinned position
|
|
- Adding an edge case = dropping a PNG into the right folder. Do that instead
|
|
of writing one-off test code, unless a new *expected value* must be pinned
|
|
(then update `tests/test_bars.py`, e.g. `INVENTORY_EXPECTED`, or
|
|
`tests/test_windows.py`, e.g. `MFG_POSITION`).
|
|
- Keep fixture images small in number and real (actual captured frames), not
|
|
synthetic.
|
|
- Pinned OCR values in `test_bars.py` are ground truth from real frames. If a
|
|
frame's expected value must change, verify the number by reading the image
|
|
— never just update the test to match a new wrong OCR result.
|
|
|
|
## The JSONL log contract
|
|
|
|
`main.py` appends one JSON object per line to `analysis.jsonl` (default).
|
|
Shape:
|
|
|
|
```json
|
|
{"time": "…", "frame": 4, "is_harvesting": true, "harvest_pixel": "#f2c285", "harvest_luma": 201, "bars": {"mana": 32, "food": 39, "health": 40, "inventory": 106, "action_points": 140}, "manufacturing_window": [1104, 327], "manufacturing_message": {"text": "you successfully created 1 potion of minor healing", "color": "green", "category": "success"}, "inventory_window": [1629, 686]}
|
|
```
|
|
|
|
- The future observer parses this log. **Do not change field names, types, or
|
|
add/remove fields casually.** If you must extend it, add new keys (never
|
|
repurpose existing ones) and update `design.md`.
|
|
- `bars.*` may be `null` when OCR fails — consumers must tolerate that; keep
|
|
it that way.
|
|
- `manufacturing_window` / `inventory_window` are `[x, y]` title-bar
|
|
coordinates, or `null` when the window is not currently verified.
|
|
`manufacturing_message` is `{text, color, category}` (`text`/`color` are
|
|
`null` when no message shows), or `null` when the window position is
|
|
unverified. Message categories: `working`, `success`, `stopped`, `failed`,
|
|
`failed_lost`, `hungry`, `overloaded`, `none`, `unknown`. Consumers must
|
|
tolerate all `null` cases.
|
|
- Keep the log append-only and line-per-record (JSONL). It is tail-friendly.
|
|
|
|
## Error handling
|
|
|
|
- Analysis functions must never crash the capture loop. `analyze.py` catches
|
|
per-item failures (e.g. OCR returns `None` on error) and returns `None`
|
|
rather than raising.
|
|
- Optional dependency pattern: `pytesseract` is imported inside
|
|
`try/except ImportError` and the module degrades gracefully
|
|
(`pytesseract = None`). Follow this pattern for any new optional dep.
|
|
- Corrupt images → `load_image` returns `None`; callers skip them.
|
|
|
|
## Docs
|
|
|
|
- `design.md` — the source of truth for capabilities, calibration,
|
|
architecture, and the log format. Keep it in sync when you change behavior.
|
|
- `AGENTS.md` — these rules. Update it if a convention changes.
|