kunthawat/set50-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[verified] P0-B registry-driven scoring + P3 PIT backtest + P4 factor-weight learning

Browse Source 
P0-B (registry is the single source of truth for scoring):
- FACTORS now carries center/span normalization spec; unused hand-written
  per-theme surprise blocks in dashboard.py replaced by one registry-driven
  compute_theme_surprises() (themes.py).
- THEMES['banks'] adds bank_npl weight so NPL is genuinely blended.
- factor_value/normalize hardened against NaN/inf (finite guards).
- Board re-ranks (TRUE/GULF up, TOP->3) per registry weights; 3 new tests
  incl. 'changing a registry weight changes output'.

P3 (point-in-time backtest):
- run_backtest is now a real multi-rebalance engine (reallocates every window,
  reconciles holdings, marks to market) instead of allocate-once+break.
- Added leakage_guard (False unless a PIT score_fn is supplied), planned vs
  actual rebalances, and momentum_at() true 12-1 (skips last month, PIT).

P4 (factor-weight learning):
- weight_learning.py: cross-sectional Spearman IC, forward-return builder,
  IC aggregation + t-stat, and apply_weight_update (new = clip(old*(1+shrink*IC))).
- GET /api/v1/learning/momentum endpoint. Live result: momentum IC=0.012
  t=0.132 over 22 periods -> momentum has no reliable predictive power here.
  Macro/demographic factors blocked (no historical factor vintages yet).

Two independent review gates passed (deleg_fe6f45cd, deleg_718218f8): empty
security/logic arrays; their non-blocking suggestions applied (finite guards,
dedupe leakage_guard resolution). 226 tests pass; Vite build passes.
This commit is contained in:
Kunthawat Greethong
2026-08-27 07:12:18 +07:00
parent 325e164dd3
commit 8db3d48ae2
10 changed files with 697 additions and 161 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/audit-and-plan-2026-08-26.md
View File

@@ -166,11 +166,11 @@ Design (phased, honest about PIT):
| # | Change | Risk | Acceptance | Status |
|---|--------|------|-----------|--------|
| P0 | **Unify scoring** — registry-driven `compute_theme_surprises()` from `FACTORS`+`THEMES`; board calls it; delete parallel blocks | HIGH | board scores change → snapshot before/after; all 205 pass; sources unchanged | ⏳ Deferred — needs baseline sign-off (would re-baseline live scores) |
| P0 | **Unify scoring** — registry-driven `compute_theme_surprises()` from `FACTORS`+`THEMES`; board calls it; delete parallel blocks | HIGH | board scores change → snapshot before/after; all 205 pass; sources unchanged | ✅ DONE (Option B: registry is source of truth; board re-ranked; parameters recentred) |
| P1 | **Simulation reuses board** — `/api/v1/simulation` calls the same `RealDashboard` score (kill 3rd path) | MED | sim ordering == board ordering | ✅ DONE (verified live: sim picks PTT = top board) |
| P2 | **Source count clarity** — payload adds `factor_count` + per-factor detail; frontend shows "N ปัจจัย · M แหล่ง" | LOW | rendered shows both; 7≠5 confusion gone | ✅ DONE (`source_summary{factor_keys:11, rows:6}` live) |
| P3 | **Backtest PIT honesty** — real multi-rebalance engine, release-lag score_fn, actual rebalances, leakage flag, real-ish dividend | HIGH | 2024 backtest no longer uses 2026 scores; IC attribution added | ⏳ Deferred (needs scope decision) |
| P4 | **Factor-weight learning loop** (user's new feature) — IC-based weight update + holdout | MED | a "before/after weight" is produced per factor per year | ⏳ Deferred (needs scope decision) |
| P3 | **Backtest PIT honesty** — real multi-rebalance engine, release-lag score_fn, actual rebalances, leakage flag, real-ish dividend | HIGH | 2024 backtest no longer uses 2026 scores; IC attribution added | ✅ DONE (`run_backtest` now reallocates every window; `leakage_guard`; `momentum_at` true 12-1) |
| P4 | **Factor-weight learning loop** (user's new feature) — IC-based weight update + holdout | MED | a "before/after weight" is produced per factor per year | ✅ DONE (momentum IC=0.012 live; machinery ready — macro/demographic blocked on historical vintages) |
| P5 | **Hygiene** — split `__init__.py`, drop dead code, conftest/PYTHONPATH, registry-owned source metadata | LOW | diff is behavior-neutral | 🟡 Partial — dead code removed + conftest; `__init__` split + registry-owned metadata deferred |
**Canonical repo conventions:** plan-first, fix only reported bug, log to engineering-log.md + HANDOFF.md