From 5667e96c4061e7e9936e7e5008e33955b5cd3e64 Mon Sep 17 00:00:00 2001 From: Kunthawat Greethong Date: Thu, 27 Aug 2026 12:35:01 +0700 Subject: [PATCH] docs: record real dated dividend collector session (2026-08-27) --- docs/HANDOFF.md | 24 ++++++++++++++++++++++++ docs/engineering-log.md | 1 + 2 files changed, 25 insertions(+) diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index 566d160..8ef99a1 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -193,6 +193,30 @@ Implement dated dividend events and a provenance-validated PIT score provider be --- +## Session 2026-08-27 — Real dated dividend-history collector (verified + pushed) + +**Branch:** main · **HEAD:** f9973e8 (pushed `59c97b5..f9973e8`) + +### Completed this session (verified + pushed) — the last PIT milestone +- **Dated dividend collector** `backend/app/siamchart.py`: `parse_dividend_history(html)` extracts the "ประวัติการปันผล" dividend table (ex-date + per-share DPS) from each stock-info page; `fetch_dividend_history(symbol)` fetches it live. +- **Ledger upgrade** `backend/app/dividend_ledger.py`: `populate_dated_dividends(ledger, symbols, fetcher)` registers every dated payment as a real row (`estimate=False`, `source=siamchart_dated`); one symbol failing never aborts the rest. +- **API** `backend/app/__init__.py`: `DividendLedger` persisted at `data/dividends/ledger.json`; `POST /api/v1/dividends/update` fetches all snapshot symbols and saves; `use_ledger` backtests prefer the dated ledger (`dividend_method=dated_ledger`) and fall back to DPS estimates when unpopulated. +- **Tests**: parser (4) + populate (2) — full backend **292 passed** (was 286). + +### Verified (real network!) +- **The sandbox HAS outbound network** (contrary to the earlier "needs external access" deferral): curl/https to siamchart.com returns 200. +- Live `POST /api/v1/dividends/update`: fetched **49/49 symbols, 1410 dated payments** in ~11s. +- Live `use_ledger` backtest then reports **`dividend_method=dated_ledger`** (real ex-date × qty) instead of DPS proxy. + +### Honest scope / notes +- Every snapshot symbol now has real dated dividend history (PTT: 45 rows from 2002; ADVANC: 51; AOT: 22; AWC: 6, etc.). +- `use_pit` scoring remains non-PIT when a run is fed current-board scores; the dated ledger is orthogonal (cash-flow model), not a PIT-scorer flag. + +### Exact next action +All requested milestones are complete and pushed. Optional follow-ups: (a) schedule `/api/v1/dividends/update` into the data refresh loop, (b) surface the dated-ledger synthesis (dated vs proxy) more explicitly in the backtest UI. Do not reset or stage unrelated pre-existing working-tree changes. + +--- + ## Session 2026-08-27 — Factor-learning validation gate (verified + pushed) **Branch:** main · **HEAD:** ae814c3 (pushed `9700f6b..ae814c3`) diff --git a/docs/engineering-log.md b/docs/engineering-log.md index 17480ba..f966661 100644 --- a/docs/engineering-log.md +++ b/docs/engineering-log.md @@ -103,3 +103,4 @@ - PIT siamchart vintage store (2026-08-27, commit `887e9c9`): added `siamchart_vintages.py` (append-only hash-chained store of every collected siamchart snapshot; `snapshot_at(as_of)` returns the newest snapshot with `retrieved_at <= as_of` — anti-look-ahead, fail-closed). `PitScoreProvider` accepts `siamchart_store`; when wired the fundamental dimension reads the snapshot knowable at as_of (`pit_grade='pit'`) and `partial_pit` no longer forces true. `/api/v1/backtest use_pit` seeds the first vintage from the current snapshot. Full backend **273 passed** (was 266). Honest scope: snapshot reconstruction forward only; EPS year-keys not tied to calendar years (growth = latest-vs-prior). - Real forward-test lifecycle (2026-08-27, commits `6d9c283` + `80c6d79`): the cosmetic "forward" (same single-pass backtest, different mode string) is replaced with a durable, frozen-signal paper-portfolio lifecycle. `forward_test.py` `ForwardTestStore` (thread-safe JSON store) with status flow: frozen (signals snapshotted immutable) → executed (fills 50/20/30 at post-freeze prices) → marked (mark-to-market equity series) → matured (net_return). New routes: `GET /api/v1/forward(+)`, `POST /api/v1/forward` (create+execute, `use_pit` freeze), `POST //mark`, `POST //mature`; store at `data/forward/runs.json` (survives restarts). UI simulation tab: forward calls `/api/v1/forward`, loads runs, shows status/non-PIT/holdings + Mark/Mature per run. Full backend **280 passed** (was 273). Honest scope: score source at CREATE may be current board (`non_pit=true` tagged); paper-only. - Factor-learning validation gate (2026-08-27, commit `ae814c3`): closed the P4 "no auto-apply" loop — `weight_learning.py` now splits a chronological IC series into train + holdout via `apply_validation_gate`, and a factor is `validated=True` only when total sample >= 12, each window >= its min, train & holdout IC both beat baseline (BASELINE_IC=0) and agree in sign, and pooled |t| > 1.0. `apply_weight_update` keeps the weight unchanged for any unvalidated factor (no auto-apply); only validated factors move. `learn_momentum_gated` wired into `/api/v1/learning/momentum`, surfacing `ic_train`/`ic_holdout`/`validated`/`gate_notes`. Full backend **286 passed** (was 280). Live probe: momentum validated=false, gate_note "IC not above baseline (0.0711/-0.1143)" — weight unchanged. +- Real dated dividend-history collector (2026-08-27, commit `f9973e8`): `siamchart.py` gained `parse_dividend_history`/`fetch_dividend_history` (reads the "ประวัติการปันผล" ex-date + DPS table per stock-info page); `dividend_ledger.py` gained `populate_dated_dividends` (registers dated rows, estimate=False). `POST /api/v1/dividends/update` fetches all snapshot symbols and persists a dated ledger at `data/dividends/ledger.json`; `use_ledger` backtests prefer the dated ledger (`dividend_method=dated_ledger`) and fall back to DPS estimates when unpopulated. Full backend **292 passed** (was 286). Live network fetch: 49/49 symbols, 1410 dated payments; use_ledger → `dated_ledger`. Note: this sandbox HAS outbound network (curl/https to siamchart 200) — contrary to earlier assumption.