docs: record deploy-safe auto PIT vintage collection (scheduler)

docs/HANDOFF.md

@@ -190,6 +190,7 @@ engine per the user's description. Confirmed decisions:
 
 ### Honest scope / limitations
 - Current on-disk data has **no factor vintages / Siamchart manifest** → strict event backtest is blocked on real data until collection accumulates (this is the intended fail-closed behavior, matching "strict PIT or block").
+- **Auto-collection is now live** (commit `1fb1e1a`): the scheduler writes PIT factor + Siamchart vintages every refresh on whichever machine the app runs, so strict runs become ready once enough data is knowable. Because `released_at=retrieved_at=now`, `ready=true` currently requires an explicit `end=today` window (auto end=yesterday stays blocked until real history accumulates — honest, no assumed reporting lag).
 - Dividend payment timing is the confirmed `ex_date + 30d` **assumption** (Siamchart gives ex-date+DPS, not pay date); every result exposes `dividend_timing=ex_date_plus_30d`.
 - Only the new `/api/v1/backtest/run` route is event-driven/strict; the legacy `POST /api/v1/backtest` remains as the descriptive non-PIT path.
 

docs/engineering-log.md

@@ -106,4 +106,4 @@
 - Real dated dividend-history collector (2026-08-27, commit `f9973e8`): `siamchart.py` gained `parse_dividend_history`/`fetch_dividend_history` (reads the "ประวัติการปันผล" ex-date + DPS table per stock-info page); `dividend_ledger.py` gained `populate_dated_dividends` (registers dated rows, estimate=False). `POST /api/v1/dividends/update` fetches all snapshot symbols and persists a dated ledger at `data/dividends/ledger.json`; `use_ledger` backtests prefer the dated ledger (`dividend_method=dated_ledger`) and fall back to DPS estimates when unpopulated. Full backend **292 passed** (was 286). Live network fetch: 49/49 symbols, 1410 dated payments; use_ledger → `dated_ledger`. Note: this sandbox HAS outbound network (curl/https to siamchart 200) — contrary to earlier assumption.
 - Auto-refresh dated dividend ledger (2026-08-27, commit `03195dc`): `AppDataScheduler` now calls `_maybe_refresh_dated_dividends()` in `refresh_all` — a cooldown-gated (default 6h) fetch of real dated dividend history into `data/dividends/ledger.json`, so the ledger stays fresh without manual `/api/v1/dividends/update` and without hammering the source every tick. `DIVIDEND_REFRESH_COOLDOWN_SECONDS` env toggle. Full backend **294 passed** (was 292).
 - Single-container Docker packaging (2026-08-27, commit `f9b8cd1`): added `Dockerfile` (python:3.11-slim + nginx; serves the PREBUILT `frontend/dist` SPA, proxies `/api` → 127.0.0.1:5000, `VOLUME /app/backend/data`, HEALTHCHECK on /api/v1/dashboard/summary, exposes :80), `deploy/nginx.conf`, `deploy/entrypoint.sh` (Flask HOST=0.0.0.0:5000 + nginx foreground), `docker-compose.yml` (port 8080:80, volume `./backend/data`), `.dockerignore`, and `.gitignore` now tracks `frontend/dist/` (prebuilt bundle required by the image; `backend/data/` stays untracked). Backend runtime verified (test_client `/api/v1/dashboard/summary` = 200). No local docker on this machine so the image itself must build on EasyPanel.
-- Event-driven PIT backtest (2026-08-28, commits `68f2cc1` → `d73a58b`, 8-phase plan at `.hermes/plans/2026-08-28_091000-event-driven-pit-backtest.md`): replaces the calendar-rebalance backtest with a strict point-in-time event-driven engine. Confirmed user decisions: (1) strict PIT — block when factor/Siamchart/price coverage is incomplete; (2) execute at next trading-day close after release (no same-day look-ahead); (3) dividend cash available ex_date+30 calendar days (`ex_date_plus_30d` assumption, not an observed pay date — Siamchart source gives ex-date+DPS only); (4) average-cost realized P&L; (5) all-in 0.3% fee per trade, no added tax. Ships: `backtest_readiness.py` (readiness + recommended start/end defaults, fail-closed), `backtest_events.py` (event calendar + next-trading-day mapping), `portfolio_ledger.py` (cash/positions/avg-cost/realized+unrealized P&L/dividend receivables/fees; gross realized + fees subtracted once), `portfolio_rebalancer.py` (50/20/30 → executable 100-lot orders, sells-first, no trade on unchanged target), `backtest_engine.py` (event processor; refuses to fall back to the live board without a PIT scorer — fail-closed, was flagged by cycle-1 review `deleg_5225e9de` and fixed), `backtest_store.py` (durable atomic JSON runs at `data/backtest/runs.json`), strict route `POST /api/v1/backtest/run` (readiness-gate → 400 with missing list when not ready) + `GET /api/v1/backtest/run(+<id>)`, and frontend (readiness gate disables Run + shows missing coverage; report shows realized/unrealized P&L, dividend received/receivable, fees, final holdings with avg cost/last price/unrealized). Offline standalone suite **54 tests** (9+14+15+5+6+4→ readiness 10 after price regression); compileall + diff-check clean; static scan no secrets/eval/debug (4 "secret"-regex hits are the prose "missing token" string, not credentials). Independent review cycles: cycle-1 `deleg_5225e9de` flagged 3 logic errors (no-scorer live-board fallback look-ahead; price_coverage not checking explicit start → false PIT window; coincident-day ex-date entitlement breach) → all fixed; cycle-2 `deleg_b969075c` re-verified the fallback fix + 9/10 invariants and re-confirmed the invariant-4 breach; cycle-3 `deleg_7f56df3b` confirmed the ordering fix; cycle-4 `deleg_55f4e877` final re-review returned `passed=true`, empty `logic_errors`/`security_concerns` (suggestion only: per-symbol price coverage at an explicit start equal to the intersection start — deferred as a known edge-case limitation; the engine/accounting remain verified). Fixes: engine fails closed without a PIT scorer; readiness blocks an explicit start before price history exists; each calendar day runs dividend_entitlement < dividend_payment < rebalance (shares bought on ex-date not entitled). Live probes: `GET /api/v1/backtest/readiness` on real data → ready=false, recommended_end=2026-08-27; `POST /api/v1/backtest/run` on real data (no vintages) → 400 + missing list; happy-path `scripts/probe_event_backtest.py` (synthetic PIT-complete) → accounting_reconciled=True, identity lhs=rhs, leakage_guard=True, 100-lot holdings.
+- Event-driven PIT backtest (2026-08-28, commits `68f2cc1` → `d73a58b`, 8-phase plan at `.hermes/plans/2026-08-28_091000-event-driven-pit-backtest.md`): replaces the calendar-rebalance backtest with a strict point-in-time event-driven engine. Confirmed user decisions: (1) strict PIT — block when factor/Siamchart/price coverage is incomplete; (2) execute at next trading-day close after release (no same-day look-ahead); (3) dividend cash available ex_date+30 calendar days (`ex_date_plus_30d` assumption, not an observed pay date — Siamchart source gives ex-date+DPS only); (4) average-cost realized P&L; (5) all-in 0.3% fee per trade, no added tax. Ships: `backtest_readiness.py` (readiness + recommended start/end defaults, fail-closed), `backtest_events.py` (event calendar + next-trading-day mapping), `portfolio_ledger.py` (cash/positions/avg-cost/realized+unrealized P&L/dividend receivables/fees; gross realized + fees subtracted once), `portfolio_rebalancer.py` (50/20/30 → executable 100-lot orders, sells-first, no trade on unchanged target), `backtest_engine.py` (event processor; refuses to fall back to the live board without a PIT scorer — fail-closed, was flagged by cycle-1 review `deleg_5225e9de` and fixed), `backtest_store.py` (durable atomic JSON runs at `data/backtest/runs.json`), strict route `POST /api/v1/backtest/run` (readiness-gate → 400 with missing list when not ready) + `GET /api/v1/backtest/run(+<id>)`, and frontend (readiness gate disables Run + shows missing coverage; report shows realized/unrealized P&L, dividend received/receivable, fees, final holdings with avg cost/last price/unrealized). Offline standalone suite **54 tests** (9+14+15+5+6+4→ readiness 10 after price regression); compileall + diff-check clean; static scan no secrets/eval/debug (4 "secret"-regex hits are the prose "missing token" string, not credentials). Independent review cycles: cycle-1 `deleg_5225e9de` flagged 3 logic errors (no-scorer live-board fallback look-ahead; price_coverage not checking explicit start → false PIT window; coincident-day ex-date entitlement breach) → all fixed; cycle-2 `deleg_b969075c` re-verified the fallback fix + 9/10 invariants and re-confirmed the invariant-4 breach; cycle-3 `deleg_7f56df3b` confirmed the ordering fix; cycle-4 `deleg_55f4e877` final re-review returned `passed=true`, empty `logic_errors`/`security_concerns` (suggestion only: per-symbol price coverage at an explicit start equal to the intersection start — deferred as a known edge-case limitation; the engine/accounting remain verified). Fixes: engine fails closed without a PIT scorer; readiness blocks an explicit start before price history exists; each calendar day runs dividend_entitlement < dividend_payment < rebalance (shares bought on ex-date not entitled). Live probes: `GET /api/v1/backtest/readiness` on real data → ready=false, recommended_end=2026-08-27; `POST /api/v1/backtest/run` on real data (no vintages) → 400 + missing list; happy-path `scripts/probe_event_backtest.py` (synthetic PIT-complete) → accounting_reconciled=True, identity lhs=rhs, leakage_guard=True, 100-lot holdings. Deploy-safe auto-collection (2026-08-28, commit `1fb1e1a`, +4 scheduler tests): `AppDataScheduler.refresh_all` now calls `_record_pit_factor_vintages` (writes each registry factor's PIT vintage via `FactorVintageStore.record`, dedupe on unchanged value; `released_at=retrieved_at=now` — evidence-safe, we do not assume an unknown reporting lag) and `_record_siamchart_vintages` (persists the current `set50_master.json` via `SiamchartVintageStore.persist`, idempotent). Runs inside the Flask process whether on Mac or the deployed container, so strict-PIT vintages accumulate wherever the app runs (persistent volume), not just locally. Live probe: `refresh_all()` 6/6 sources → 11 factor `.jsonl` + 1 Siamchart manifest snapshot written; strict readiness `ready=true` with explicit `end=today` (auto end=yesterday stays blocked until past data is knowable — honest).