Commit Graph

35 Commits

Author SHA1 Message Date
Kunthawat Greethong
325e164dd3 [verified] Fix P1-P2-P5 audit findings: simulation reuses board, source_summary clarity, dead-code removal + conftest
- P1: /api/v1/simulation now uses the canonical board score (default_scores)
  instead of a divergent 3-theme recompute -> 'จำลอง' can't disagree with board
  (live check: sim top pick PTT == top board combined 1.600). Removes binary
  auto/en signs, restores quality+momentum+dividend screen consistency.
- P2: dashboard emits source_summary{factor_keys, rows}; frontend shows
  'N ปัจจัย · M แหล่ง' so the 7-vs-5 count confusion is impossible.
- P5: removed dead themes.list_themes()/Theme/build_theme_scores/_map_index and
  the tests that locked them; added tests/conftest.py so pytest needs no PYTHONPATH.
- docs: audit-and-plan-2026-08-26.md (full P0-P5 plan) + engineering-log entry.
- 203 backend tests pass; Vite build passes. Independent reviewer: no security or
  logic blockers (minor error-leak suggestion applied: 503 message no longer leaks
  exception detail).
2026-08-27 03:21:22 +07:00
Kunthawat Greethong
6e78b6acb5 [verified] Apply R1-R5 (factor formula) + real bank-sector NPL collector
(a) R1-R5 (factor-refinement, grounded in methodology-research.md):
- R1 (PEAD): EPS-growth weight raised 1.0->1.5 in build_siamchart_score / symbol_breakdown (Bernard-Thomas 1990, Livnat-Mendenhall 2006)
- R2 (momentum): 12-1 momentum factor from Yahoo price snapshot (Jegadeesh-Titman 93; lite weight 0.5)
- R3 (regime): binary bear gate -> continuous stress = negative-themes fraction, smooth LONG/SHORT shift
- R5 (dividend screen): non-dividend / cut-yield names no longer go LONG (screen-off)
- R4 (earnings-revision) deferred: no free EPS-forecast source yet (documented)

(b) bank-sector NPL collector (BOT reportID 794, financial&insurance sector):
- refactored auto_npl to expose shared _parse_sector; new bank_npl.py reuses it
- registered bank_npl FACTOR -> auto-appears in sources table (6 rows) + blends into banks theme surprise (real NPL)
- +unit tests (test_bank_npl), test_dashboard updated (6 sources)

205 tests pass; verified live API (banks surprise incl. NPL 1.07, 6 sources).
2026-08-26 19:56:39 +07:00
Kunthawat Greethong
fc592d8aa9 [verified] Real backtest engine + backtest UI section (start/end dates, P&L, persisted)
- backtest.py: buy-and-hold backtest over [start,end] — allocates 50/20/30 at first available rebalance date, marks to market to end, accrues dividend, reports {final_value, price_pnl, dividend_income, net_return, trades, holdings}
- Fixed double-spend bug (was allocating full capital every rebalance -> negative cash)
- dashboard.default_scores(): per-symbol combined/dividend/yield baseline for backtest
- POST /api/v1/backtest + GET /api/v1/backtest/runs (results persisted in app state -> survive refresh)
- Frontend: backtest section w/ start/end/capital/freq inputs + P&L KPIs + run history table
- Honest note: uses current combined scores as static baseline (non-PIT); PIT score_fn pluggable
- Verified: 1M -> 1.088M (+8.80%) over 2024-06..2026-06; history persists across refresh
2026-08-26 16:00:17 +07:00
Kunthawat Greethong
2da73b8a8c [verified] Sources table auto-derived from FACTORS registry + next-update column
- _build_sources now iterates FACTORS registry (grouped by fetch module) instead of hardcoded 5-row list -> adding a FACTOR auto-appends its source row (single source of truth, answers 'เพิ่มแหล่งควรอัตโนมัติ')
- Each source row gains ความถี่ + อัปเดตครั้งต่อไป (from frequency cadence)
- Frontend sources table: 6 columns (ข้อมูล/แหล่ง/ช่วงข้อมูล/ความถี่/อัปเดตครั้งต่อไป/อัปเดตล่าสุด), all from /api/v1/dashboard (not mock)
- Verified live: 5 sources auto-derived w/ next-update; browser shows 6 cols
2026-08-26 15:55:30 +07:00
Kunthawat Greethong
5516fc51a0 [verified] LONG/SHORT/NEUTRAL via quartile + market-regime gate (user choice B)
- Signal threshold no longer hardcoded 0.15: now quartile-based (LONG>=Q3, SHORT<=Q1, else NEUTRAL) over the whole SET50 board, recomputed each refresh.
- Market-regime gate: if >=4 themes have negative surprise -> risk-off bear regime -> tighten LONG bar + pull more into SHORT/avoid, so 'best of a falling board' isn't LONG (answers user 'ตลาดตกควรขายทิ้ง').
- SHORT semantics (user confirmed) = 'หลีก/ไม่ถือ' -> cash, NOT short-selling.
- reason_codes + regime now on factor rows (transparent).
- Verified: LONG 12 / SHORT 12 / NEUTRAL 25 in normal regime (Q1=-0.044 Q3=0.407).
- Fixed test_factors_endpoint_signal_join (was asserting AOT LONG from old tourism). Full suite 202 OK.
2026-08-26 15:34:31 +07:00
Kunthawat Greethong
375682d2dc [verified] Signal column now derives from theme engine (combined 60/40 + quality), not tourism_result
- /api/v1/factors signal no longer from tourism-only signals; derived from RealDashboard combined score: LONG>=0.15, SHORT<=-0.15, else NEUTRAL. One source of truth.
- Verified: LONG 25 / NEUTRAL 16 / SHORT 8; BANPU LONG (3 themes), KTB LONG (banks), CPN LONG (3 themes), JMART SHORT (nonbank) — reflects all 13 themes + per-symbol quality, not flat tourism.
- reason_codes + combined_score on each factor row.
2026-08-26 15:26:34 +07:00
Kunthawat Greethong
d850955c44 [verified] Declarative factor engine + per-symbol stock selection (full-app consistency)
- factors.py: FACTORS registry (10 declarative entries: source/fetch/frequency/sign/weight) + normalize/z-score helpers. Add a source = one dict entry, no scoring-function edit.
- themes.THEMES: 13 themes reference FACTORS with per-theme weights (flexible), replacing hardcoded _theme_surprises/_theme_narrative.
- themes.quality_within_theme(): per-symbol quality vs theme cohort (ROE/EPS) -> real stock picking. dashboard board now surprise×quality (BBL 0.5 vs KTB 1.5 in banks).
- board rows carry per-symbol themes[]; /api/v1/themes delegates to RealDashboard.build() -> 13-theme consistency with /api/v1/dashboard (removed 115 lines dead dup logic).
- frontend: deleted THEME_BY_SYMBOL/themeLabelById hardcode; theme column + modal labels+quality all from API. Modal shows surprise×quality=theme_score.
- Tests: 202 OK (quality selection, breakdown quality, themes/dashboard consistency).
- Verified: BBL modal 1.00σ×0.5=0.50σ; KTB 1.5 vs BBL 0.5, PTT 2 themes; 49/49 rows theme from API.
2026-08-26 15:17:41 +07:00
Kunthawat Greethong
e7819a35dd [verified] All 13 SET50 themes now have real surprise (macro-proxy) — no more 'ยังไม่มีข้อมูล'
- dashboard._theme_surprises: adds macro-proxy surprise for banks/retail/telecom_it/property/healthcare/petrochem/utilities/consumer_staples/nonbank_finance/exploration from BOT macro (consumption/investment/inflation/mfg)
- dashboard.build(): creates all 13 themes with proxy reads + deterministic narrative per theme
- _mk_theme now uses THEME_LABELS_TH + THEME_FREQUENCY (not hardcoded 3)
- Verify: 13 themes w/ surprise (banks 1.0, retail 0.19, utilities -0.31, exploration 1.62); BBL modal (ธนาคาร 1.00σ, combined 0.467)
- Full suite 199 OK; fixed test_build themes=13
2026-08-26 14:19:59 +07:00
Kunthawat Greethong
55b3574040 [verified] Cover full SET50 with 13 themes + per-theme score detail in symbol view
- THEME_SYMBOLS expanded: added banks, retail, telecom_it, property, healthcare, petrochem_materials, consumer_staples, utilities, nonbank_finance, exploration -> all 49 SET50 names now in a theme
- THEME_LABELS_TH Thai labels; THEME_FREQUENCY per theme
- symbol_breakdown now lists EVERY theme the symbol belongs to (label_th + surprise, or 'ยังไม่มีข้อมูล'), so theme_score is transparent per source
- frontend: theme column maps all 49 symbols (mirrors backend); modal shows per-theme score detail
- Fixed test for BANPU multi-theme; full suite 199 OK
- Verified: 49/49 rows have theme chip; AOT modal shows ท่องเที่ยว 0.57σ + full calc
2026-08-26 14:05:10 +07:00
Kunthawat Greethong
6c26bf99dc [verified] Fix dashboard: real-data badge + sortable คะแนนรวม + modal calc steps
- badge now driven by /api/v1/dashboard availability (ข้อมูลจริงจากแหล่งไทย), not stale fixture summary (#request)
- factorValue handles 'combined' -> คะแนนรวม column now sorts + toggles asc/desc
- symbol modal shows full calculation: combined_formula, per-step calc (ธีม/พื้นฐาน + weights), z-score derivation with population mean/stdev/universe size
- panel-header layout fixes (full-width subtitle, header breathing room)
- Verified: badge real, sort 0.11->1.43, modal calc steps render
2026-08-26 06:22:52 +07:00
Kunthawat Greethong
33a4662cd4 [verified] Add per-symbol analysis breakdown endpoint + per-theme narrative
- themes.symbol_breakdown(): transparent scoring derivation (theme_score, siamchart_score components, combined = 0.6*theme + 0.4*siamchart npolut)
- GET /api/v1/symbols/<symbol>: themes + theme surprise contributions + fundamentals + price + weights (ข้อ 7)
- dashboard.py _theme_narrative(): long-form Thai explanation of each theme's analysis outcome + implication for its stocks (ข้อ 5)
- 2 tests; full suite OK; live verified (AOT: combined 0.107 = 0.6*0.571 + 0.4*(-0.588))
2026-08-25 21:18:20 +07:00
Kunthawat Greethong
bfa9b08af6 [verified] Scheduler refreshes once on boot (no 1h wait for first pull) 2026-08-25 21:03:52 +07:00
Kunthawat Greethong
abc06af3a1 [verified] Add in-app automatic data scheduler (runs on its own server, no Hermes)
- scheduler.py: daemon thread inside Flask refreshes all real Thai collectors on interval (default 60min, REFRESH_INTERVAL_SECONDS) via shared daily cache + writes timestamped marker
- create_app starts scheduler (skipped in TESTING); shared daily_cache now an extension
- GET /api/v1/data/last-refresh: automation status + last refresh (every N hours)
- Live verified: refresh_all pulls 5/5 real sources (tourism/auto/NPL/energy/macro)
- 2 tests; full suite 197 OK
2026-08-25 21:00:13 +07:00
Kunthawat Greethong
9739849f68 [verified] Add real multi-theme dashboard (3 themes + macro + board + sources) — req #6/#8/#9/#10
- dashboard.py: RealDashboard assembles real Thai data (tourism + auto+NPL + energy TOP + macro BOT) with uniform z-score surprise per theme, per-theme thesis, sources provenance table, 49-symbol combined board
- macro_thai.py: BOT Thai Economy macro backdrop (consumption +4.9%, inflation 1.95%, unemployment 0.93%, tourists 16.2mn)
- GET /api/v1/dashboard endpoint (real data, no fixture fallback per user)
- 7 new tests; full suite 195 OK; live verified (3 theme surprise: 0.571/0.81/1.623)
2026-08-25 20:30:14 +07:00
Kunthawat Greethong
166a885fb9 [verified] Add Thai macro backdrop collector (BOT Thai Economy) — real consumption/inflation/unemployment
- macro_thai.py scrapes bot.or.th/en/thai-economy.html (server-rendered, no auth)
- private consumption +4.9%, private investment +18.1%, mfg -3.1%, headline inflation 1.95%, core 1.34%, unemployment 0.93%, tourists YTD 16.2mn
- 3 tests; live verified; macro backdrop layer (req #6)
2026-08-25 20:27:15 +07:00
Kunthawat Greethong
1e75377732 [verified] Add BOT auto NPL (credit-quality) factor; deepen auto_credit theme
- auto_npl.py: parse BOT Gross NPLs by business (reportID=794); extract auto loan NPL (20,602 mn THB, 3.95% of NPLs, 2.06% of loans)
- /api/v1/themes now exposes auto_npl_pct + auto_npl_amount alongside car-sales volume
- 3 new tests; full suite 188 OK; live verified (themes shows auto_npl_pct 3.95)
2026-08-25 19:06:48 +07:00
Kunthawat Greethong
2e492b375a [verified] Extend price universe to full SET50 (49 symbols + index) so simulation allocates across all names
- DEFAULT_SYMBOL_MAP now covers full SET50 (from Siamchart master snapshot) instead of 8 names
- Collected real Yahoo price snapshot: 49 symbols + SET50, ~646 bars each (2024-01-01 → 2026-08-24), quality=revised_vendor_history point_in_time=false
- Full suite 185 OK; live simulation HTTP 200
2026-08-25 17:04:06 +07:00
Kunthawat Greethong
aae1d132c8 [verified] Fix bucket3 to rank by dividend yield (ignore score) — reviewer blocker
- allocate_capital _fill now takes sort_by; bucket3 uses dividend_yield, buckets 1/2 use combined_score
- Previously bucket3 wrongly ranked by score (picked CPN/PTT); now picks highest-yield CRC — matches user rule 'bucket3 = highest dividend yield, ignoring score'
- Added test_bucket3_ranks_by_yield_ignoring_score
- Re-reviewed (deleg_43b165e0) passed=true, no logic/security errors; full suite 185 OK
2026-08-25 16:32:12 +07:00
Kunthawat Greethong
c3a1461932 [verified] Add capital-allocation simulation engine + MT5 bridge (both dry-run/gated)
- simulation.py: price-series loader (Yahoo snapshot) + allocate_capital 50/20/30 with min-100 shares, bucket3 excludes bucket1, cash fallback
- /api/v1/simulation POST: combines theme 60/40 score + Siamchart dividend + Yahoo price; labels output paper/backtest non-PIT (never validated)
- mt5_bridge.py: MT5 order interface, dry-run default; live dispatch needs MT5_SEND_ORDERS=1 AND approval (Windows-only MetaTrader5)
- 12 new tests (simulation/mt5/api); full suite 184 OK; live verified (1M -> buckets)
2026-08-25 16:24:22 +07:00
Kunthawat Greethong
7643764679 [verified] Add GET /api/v1/themes multi-theme combined board (60% theme / 40% Siamchart)
- Aggregates 3 Thai themes: tourism signals (real), auto_credit (TradingEcon car sales YoY), refining_energy (Thai Oil quarterly net profit/EBITDA) via daily cache
- Combines per-symbol theme scores with Siamchart fundamental score (60/40), sorts by combined score, reports per-theme frequency (monthly/quarterly)
- Added test_themes_endpoint (mocked collectors); full suite 172 OK; compileall ok
2026-08-25 15:33:06 +07:00
Kunthawat Greethong
63058ecfc9 [verified] Add daily cache layer (fetch-once/day, stale fallback, atomic write)
- DailyCache: JSON file cache keyed by (source/as_of), TTL 24h, atomic tmp+rename write, persists across runs
- fetch_or_stale: returns fresh cache, else refetch+cache, else falls back to stale so dashboard is never blanked
- 7 tests; full suite pass
2026-08-25 15:29:15 +07:00
Kunthawat Greethong
affc29a3be [verified] Add multi-theme registry + combined scoring engine (60% theme / 40% Siamchart)
- themes.py: 3-theme registry (tourism monthly, auto_credit monthly, refining_energy quarterly) with Thai labels + frequency; curated SET50 symbol->theme exposure map; z-normalized theme scoring and Siamchart fundamental score; 60/40 combined score (multi-theme mean)
- Frequency recorded per theme so consumers don't mix different-cadence factors as same-timestamp
- 8 tests; full suite pass
2026-08-25 15:19:39 +07:00
Kunthawat Greethong
3f7fccd25b [verified] Add Thai Energy/Refining collector (TOP quarterly financials) replacing EIA US crack spread
- energy_thai.py: scrape Thai Oil (TOP) investor financial-highlights -> quarterly + annual EBITDA/Net Profit/Sales (Million Baht), largest Thai refinery
- Thai-specific factor per user (energy must reflect Thai companies, not US EIA proxy); Krungsri was projection-only, TOP gives real quarterly actuals
- Frequency: quarterly (documented in research note)
- 4 tests; full backend suite 156 OK; compileall ok; static scan clean
2026-08-25 15:13:36 +07:00
Kunthawat Greethong
fdfd832fae [verified] Generalize BOT regional header detection (BE year 25XX); only enable regions with real data
- header month-cell detection now matches any 4-digit BE year (25XX) not just 256X
- northeast reportID 955 disabled: returns 0.0 BE-2570 placeholder page, no real data yet (documented)
- north 954 stays enabled (live idx 100.3); tests + full suite 152 OK
2026-08-25 14:27:40 +07:00
Kunthawat Greethong
2fa805554e [verified] Add BOT regional private-consumption collector (North) for regional tourism/consumption factor
- bot_regional.py: parse BOT BTWS_STAT regional report (reportID per region) -> private consumption index + nondurable index + monthly series
- North reportID=954 verified live (idx 100.3, series 6mo); region-keyed for extension to other regions
- collect_bot_regional.py CLI -> JSON snapshot
- 4 tests; full backend suite 152 OK; compileall ok
2026-08-25 14:14:33 +07:00
Kunthawat Greethong
f6dd6304cc [verified] Add Auto Credit (Trading Economics) + Refining/Energy (EIA) factor collectors
- auto_credit.py: scrape Trading Economics Thailand total-vehicle-sales HTML -> total sales + new car sales YoY + vehicle production/passenger/exports
- refining_energy.py: scrape EIA prices.php 3:2:1 crack spread (Gulf LLS) + WTI/Brent/gasoline
- Both server-rendered HTML scrapers (har-derived-api-client pattern); EIA used instead of RBN (RBN is Cloudflare-challenged, 403 via urllib; EIA is open US gov data, HTTP 200 direct)
- collect_auto_credit.py / collect_refining.py CLI -> JSON snapshot
- 8 tests; full backend suite 148 OK; live verified (auto 59198/20% YoY; crack 71.10 $/bbl); static scan clean
2026-08-25 11:05:13 +07:00
Kunthawat Greethong
8a6991b7dd [verified] Add Siamchart factor view + redesigned SET50 dashboard stock board
Backend:
- siamchart_factors.py: build per-symbol factor view from Siamchart snapshot (PE, EPS latest, EPS growth YoY derived from series, dividend yield, P/BV, ROE, is_dividend). eps_latest now returns the most recent year.
- /api/v1/factors endpoint: merge Siamchart fundamentals with the tourism signal (side/score), signal-led sorting.
- test_siamchart_factors.py: 5 tests incl. regression asserting eps == year5 value.

Frontend:
- App.vue/style.css: new 'Stock board' dashboard table (Signal, Symbol, P/E, EPS, EPS YoY, Yield%, P/BV, ROE) with a Dividend-only filter and click-to-sort columns.

Verified: full backend suite 140 tests OK, frontend build OK, static scan clean, live /api/v1/factors 200 (49 factors/46 dividends), rendered table filter+sort verified in browser. Independent review deleg_969513e5 caught+fixed eps bug; re-review deleg_e6bd80db passed=true.
2026-08-25 09:03:28 +07:00
Kunthawat Greethong
632216ac27 [verified] Add Siamchart SET50 fundamental scraper (financial table + stock-info ratios/income)
- fetch_financial(group): parse Siamchart JS store_real_data array -> EPS/Rev/NP (5yr) + PE for every symbol
- parse_stock_info_html: head1/body key ratios (PE/P/BV/D/E/DPS/EPS/ROAA/ROAE/NPM/Yield) + income statement (QoQ/YoY)
- collect_siamchart.py CLI: --group/--stock-info/--with-info, timestamped JSON snapshot
- build_url validates group against ^[A-Z0-9]+(?:-[A-Z0-9]+)*$ (blocker closed)
- 10 unit tests; full backend suite 135 OK; independent review deleg_ad2b8dc7 passed=true

[verified] tags from requesting-code-review pipeline.
2026-08-25 08:43:00 +07:00
Kunthawat Greethong
ba9114d2e8 [verified] bind snapshots to manifests and raw hashes 2026-08-23 14:58:15 +07:00
Kunthawat Greethong
f55ff69c31 [verified] add frozen research runner and durable paper ledger 2026-08-23 14:42:02 +07:00
Kunthawat Greethong
7f7a6145cd [verified] add SET price snapshot adapter 2026-08-23 12:38:17 +07:00
Kunthawat Greethong
0b47a06238 [verified] add event-study readiness gate 2026-08-23 12:02:22 +07:00
Kunthawat Greethong
d1ba6efc68 [verified] add vintage collector and point-in-time API 2026-08-23 11:11:53 +07:00
Kunthawat Greethong
a113a51589 [verified] add BOT tourism source adapter 2026-08-23 10:19:40 +07:00
Kunthawat Greethong
3e978c5948 [verified] build tourism signal dashboard 2026-08-23 07:41:31 +07:00