[verified] Fix backtest accounting identity + honest UI disclosure
Correct the multi-rebalance backtest accounting so ending wealth is capital + price_pnl + dividend_income with no double counting: - price_pnl now measures equity change excluding dividends (was reusing ending holdings value as 'price profit') - dividend proxy is included in final_value and net_return, exposed as dividend_method=final_holdings_yield_proxy - regression tests: flat price => zero price_pnl; flat + dividend => dividend-only return; rising no-dividend => correct bucket P&L; multi-rebalance accounting identity - UI (result card + saved-run history) labels dividends as ประมาณการปันผล (Proxy) and shows descriptive non-PIT badge when leakage_guard=false Backend 239 tests passed; targeted backtest 11 passed; frontend build, npm audit (0), static scan and diff check passed; fresh independent review deleg_10918fed passed with empty blocker arrays. Backtest remains descriptive non-PIT (leakage_guard=false) with the default current-score scorer.
This commit is contained in:
@@ -18,6 +18,7 @@
|
||||
| Tourism deterministic signal | complete | live foreign-arrivals YoY surprise | add occupancy/airport metric |
|
||||
| Internal paper ledger | complete | atomic local JSON persistence and restart test | shared store before multi-worker deployment |
|
||||
| Dashboard | complete | Vite build + served source check with live-sign copy | visual browser capture after permission is available |
|
||||
| General backtest accounting | complete/blocked | flat-price RED/GREEN invariants; 238 backend tests; Vite build; dividend proxy and non-PIT UI disclosures | replace final-holdings yield proxy with dated dividend cash flows; add provenance-validated PIT scorer |
|
||||
| LLM analysis | deferred | intentionally no LLM dependency in M0 | add after signal lineage is stable |
|
||||
| Webhook receiver | deferred | contract only, no external receiver | choose after core app is usable |
|
||||
| MT5 bridge | deferred | not started | paper bridge after webhook decision |
|
||||
@@ -96,3 +97,4 @@
|
||||
- Audit + fix pass (2026-08-26, plan at `docs/audit-and-plan-2026-08-26.md`): triple-confirmed the declarative `FACTORS`/`THEMES` framework is by-passed by hand-written scoring in `dashboard._theme_surprises`, and that `/api/v1/simulation` recomputed a divergent 3-theme path. Fixed P1 (simulation reuses the canonical board via `default_scores` — live check: sim top pick PTT == top board combined 1.600), P2 (dashboard now emits `source_summary{factor_keys:11, rows:6}`; frontend shows "N ปัจจัย · M แหล่ง"), and P5-partial (removed dead `list_themes`/`Theme`/`build_theme_scores`/`_map_index` + the tests that locked them; added `backend/tests/conftest.py` so pytest needs no `PYTHONPATH`). Deferred P0 (registry-driven re-baseline) and P3/P4 (point-in-time backtest + factor-weight learning) pending explicit scope/baseline sign-off. Full backend suite: **203 tests pass**; Vite build passes. This work is own-engine review gated before commit.
|
||||
- P0-B + P3 + P4 (commit `8db3d48`, reviews `deleg_fe6f45cd` + `deleg_718218f8` both `passed=true`): the declarative FACTORS/THEMES registry is now the single source of truth (`compute_theme_surprises` reads registry; hand-written per-theme blocks removed; FACTORS carries center/span normalization spec; bank NPL wired into banks). P3 rebuilt `run_backtest` as a real multi-rebalance PIT engine (`leakage_guard`, `momentum_at` true 12-1). P4 added factor-weight learning (Spearman IC -> `apply_weight_update`) + `/api/v1/learning/momentum`. Live result: momentum IC=0.012, t=0.132 over 22 periods (no reliable predictive power in this SET50 window). 226 tests pass.
|
||||
- Follow-up (A+B): theme surprises now weight-normalized by total |weight| so cross-theme magnitudes are comparable (retail 0.189->0.145). Added append-only `FactorHistory` store (`data/factor_history/<key>.jsonl`) that records every FACTORS value each scheduler run, wired into `refresh_all` (non-fatal), plus `/api/v1/learning/factors` readiness endpoint. Macro/demographic factors start at n=1 and become learnable (P4) as history accumulates. 233 tests pass.
|
||||
- Backtest accounting remediation (2026-08-27): deterministic flat-price tests exposed that ending holdings value was reported as `price_pnl` and the dividend proxy was excluded from final value/net return. The corrected identity is `final_value = capital + price_pnl + dividend_income`; API now emits `dividend_method=final_holdings_yield_proxy`, and both result/history UI paths label the proxy and warn when `leakage_guard=false`. RED failures reproduced `price_pnl=20,000/49,000`; GREEN verification: 239 backend tests, compileall, Vite build, npm audit 0 high-severity vulnerabilities, served-bundle disclosure check, static scan and diff check passed. Fresh final independent review `deleg_10918fed` passed with empty blocker arrays. Evidence: `docs/engineering-log/2026-08-27-backtest-accounting-remediation.md` and `docs/test-evidence/2026-08-27-backtest-accounting-remediation.md`. The dividend model remains a proxy and the default public backtest remains descriptive non-PIT.
|
||||
|
||||
Reference in New Issue
Block a user