A. Cross-theme comparability:
- compute_theme_surprises now weight-normalizes by total |weight| (weighted
average), so every theme surprise on same [-1,1] scale regardless of factor
count/weight (retail 0.189->0.145; auto_credit 1.0->0.64).
B. Historical factor store (enables learning macro/demographic factors):
- New factor_history.py: append-only per-factor JSONL, dedupes unchanged
values, rejects non-finite, records every FACTORS value each scheduler run.
- scheduler.py: jobs carry fetch_module; refresh_all records factor history
(non-fatal); added bank_npl job.
- GET /api/v1/learning/factors?min_points= reports n_points/learnable per
factor so users see when P4 learning unlocks (validated query parsing).
- weight_learning: generic learn_factor_series() aggregator (momentum reuses).
Independent review deleg_5dd358e3 passed=true (empty security/logic arrays);
its two robustness suggestions applied (finite guard in record(), clean 400 on
bad min_points). 234 tests pass; Vite build passes.
P0-B (registry is the single source of truth for scoring):
- FACTORS now carries center/span normalization spec; unused hand-written
per-theme surprise blocks in dashboard.py replaced by one registry-driven
compute_theme_surprises() (themes.py).
- THEMES['banks'] adds bank_npl weight so NPL is genuinely blended.
- factor_value/normalize hardened against NaN/inf (finite guards).
- Board re-ranks (TRUE/GULF up, TOP->3) per registry weights; 3 new tests
incl. 'changing a registry weight changes output'.
P3 (point-in-time backtest):
- run_backtest is now a real multi-rebalance engine (reallocates every window,
reconciles holdings, marks to market) instead of allocate-once+break.
- Added leakage_guard (False unless a PIT score_fn is supplied), planned vs
actual rebalances, and momentum_at() true 12-1 (skips last month, PIT).
P4 (factor-weight learning):
- weight_learning.py: cross-sectional Spearman IC, forward-return builder,
IC aggregation + t-stat, and apply_weight_update (new = clip(old*(1+shrink*IC))).
- GET /api/v1/learning/momentum endpoint. Live result: momentum IC=0.012
t=0.132 over 22 periods -> momentum has no reliable predictive power here.
Macro/demographic factors blocked (no historical factor vintages yet).
Two independent review gates passed (deleg_fe6f45cd, deleg_718218f8): empty
security/logic arrays; their non-blocking suggestions applied (finite guards,
dedupe leakage_guard resolution). 226 tests pass; Vite build passes.
- P1: /api/v1/simulation now uses the canonical board score (default_scores)
instead of a divergent 3-theme recompute -> 'จำลอง' can't disagree with board
(live check: sim top pick PTT == top board combined 1.600). Removes binary
auto/en signs, restores quality+momentum+dividend screen consistency.
- P2: dashboard emits source_summary{factor_keys, rows}; frontend shows
'N ปัจจัย · M แหล่ง' so the 7-vs-5 count confusion is impossible.
- P5: removed dead themes.list_themes()/Theme/build_theme_scores/_map_index and
the tests that locked them; added tests/conftest.py so pytest needs no PYTHONPATH.
- docs: audit-and-plan-2026-08-26.md (full P0-P5 plan) + engineering-log entry.
- 203 backend tests pass; Vite build passes. Independent reviewer: no security or
logic blockers (minor error-leak suggestion applied: 503 message no longer leaks
exception detail).
- backtest.py: buy-and-hold backtest over [start,end] — allocates 50/20/30 at first available rebalance date, marks to market to end, accrues dividend, reports {final_value, price_pnl, dividend_income, net_return, trades, holdings}
- Fixed double-spend bug (was allocating full capital every rebalance -> negative cash)
- dashboard.default_scores(): per-symbol combined/dividend/yield baseline for backtest
- POST /api/v1/backtest + GET /api/v1/backtest/runs (results persisted in app state -> survive refresh)
- Frontend: backtest section w/ start/end/capital/freq inputs + P&L KPIs + run history table
- Honest note: uses current combined scores as static baseline (non-PIT); PIT score_fn pluggable
- Verified: 1M -> 1.088M (+8.80%) over 2024-06..2026-06; history persists across refresh
- Signal threshold no longer hardcoded 0.15: now quartile-based (LONG>=Q3, SHORT<=Q1, else NEUTRAL) over the whole SET50 board, recomputed each refresh.
- Market-regime gate: if >=4 themes have negative surprise -> risk-off bear regime -> tighten LONG bar + pull more into SHORT/avoid, so 'best of a falling board' isn't LONG (answers user 'ตลาดตกควรขายทิ้ง').
- SHORT semantics (user confirmed) = 'หลีก/ไม่ถือ' -> cash, NOT short-selling.
- reason_codes + regime now on factor rows (transparent).
- Verified: LONG 12 / SHORT 12 / NEUTRAL 25 in normal regime (Q1=-0.044 Q3=0.407).
- Fixed test_factors_endpoint_signal_join (was asserting AOT LONG from old tourism). Full suite 202 OK.
- /api/v1/factors signal no longer from tourism-only signals; derived from RealDashboard combined score: LONG>=0.15, SHORT<=-0.15, else NEUTRAL. One source of truth.
- Verified: LONG 25 / NEUTRAL 16 / SHORT 8; BANPU LONG (3 themes), KTB LONG (banks), CPN LONG (3 themes), JMART SHORT (nonbank) — reflects all 13 themes + per-symbol quality, not flat tourism.
- reason_codes + combined_score on each factor row.
- THEME_SYMBOLS expanded: added banks, retail, telecom_it, property, healthcare, petrochem_materials, consumer_staples, utilities, nonbank_finance, exploration -> all 49 SET50 names now in a theme
- THEME_LABELS_TH Thai labels; THEME_FREQUENCY per theme
- symbol_breakdown now lists EVERY theme the symbol belongs to (label_th + surprise, or 'ยังไม่มีข้อมูล'), so theme_score is transparent per source
- frontend: theme column maps all 49 symbols (mirrors backend); modal shows per-theme score detail
- Fixed test for BANPU multi-theme; full suite 199 OK
- Verified: 49/49 rows have theme chip; AOT modal shows ท่องเที่ยว 0.57σ + full calc
- scheduler.py: daemon thread inside Flask refreshes all real Thai collectors on interval (default 60min, REFRESH_INTERVAL_SECONDS) via shared daily cache + writes timestamped marker
- create_app starts scheduler (skipped in TESTING); shared daily_cache now an extension
- GET /api/v1/data/last-refresh: automation status + last refresh (every N hours)
- Live verified: refresh_all pulls 5/5 real sources (tourism/auto/NPL/energy/macro)
- 2 tests; full suite 197 OK
- auto_npl.py: parse BOT Gross NPLs by business (reportID=794); extract auto loan NPL (20,602 mn THB, 3.95% of NPLs, 2.06% of loans)
- /api/v1/themes now exposes auto_npl_pct + auto_npl_amount alongside car-sales volume
- 3 new tests; full suite 188 OK; live verified (themes shows auto_npl_pct 3.95)
- Theme panel (ธีม): shows tourism/auto_credit/refining_energy with Thai labels, frequency chips (monthly/quarterly), and real factor reads (tourism surprise, car sales YoY, TOP net profit) from /api/v1/themes
- Combined count badge; responsive 1-col on mobile
- Loads /api/v1/themes in loadDashboard (already wired backend)
- Verified: npm build passes, browser shows 3 Thai cards with live values
- DailyCache: JSON file cache keyed by (source/as_of), TTL 24h, atomic tmp+rename write, persists across runs
- fetch_or_stale: returns fresh cache, else refetch+cache, else falls back to stale so dashboard is never blanked
- 7 tests; full suite pass
- energy_thai.py: scrape Thai Oil (TOP) investor financial-highlights -> quarterly + annual EBITDA/Net Profit/Sales (Million Baht), largest Thai refinery
- Thai-specific factor per user (energy must reflect Thai companies, not US EIA proxy); Krungsri was projection-only, TOP gives real quarterly actuals
- Frequency: quarterly (documented in research note)
- 4 tests; full backend suite 156 OK; compileall ok; static scan clean
- header month-cell detection now matches any 4-digit BE year (25XX) not just 256X
- northeast reportID 955 disabled: returns 0.0 BE-2570 placeholder page, no real data yet (documented)
- north 954 stays enabled (live idx 100.3); tests + full suite 152 OK
- bot_regional.py: parse BOT BTWS_STAT regional report (reportID per region) -> private consumption index + nondurable index + monthly series
- North reportID=954 verified live (idx 100.3, series 6mo); region-keyed for extension to other regions
- collect_bot_regional.py CLI -> JSON snapshot
- 4 tests; full backend suite 152 OK; compileall ok
