docs: data-source expansion plan + engineering log (te_thailand, sign fix, ui fix)

docs/engineering-log/2026-08-29-data-source-expansion-and-ui-fix.md

@@ -0,0 +1,50 @@
+# 2026-08-29 — Data-source expansion (Phase A) + frontend padding fix + sign bug
+
+## Scope
+Two-part request from the owner:
+1. **More + genuinely-used data sources per theme** — the owner felt the source
+   count was too few and insisted any newly-fetched data must feed the analysis
+   (theme surprise), not just the provenance table.
+2. **Frontend padding/margin issues** — several panels had misaligned spacing.
+
+## What changed (all verified)
+| Area | Change | Evidence |
+|---|---|---|
+| `factors.py` | Wired 2 **fetched-but-unused** BOT fields into the registry: `macro_core_inflation` (core inflation, sign −1) and `macro_unemployment` (sign −1) → they now actually move themes | Before, `macro_thai.to_dict()` emitted 7 fields but only 5 were registered factors — 2 were dead display data |
+| `themes.py` | Deepened banks/retail/consumer_staples/healthcare/nonbank_finance with the new factors (positive weights) | direction tests |
+| **Sign bug fix** | Found + fixed a **double-negative**: several factors carry `sign: -1` (NPL, inflation, unemployment) while theme weights were ALSO negative → higher NPL/inflation RAISED the theme score (inverted). Flipped all 16 theme weights to positive; direction now lives only in `sign`. Proven: auto-NPL 3.95→7.0 previously raised auto_credit 0.64→0.799; now lowers it. | `test_bearish_factors_move_score_the_right_way` |
+| **NEW `te_thailand.py`** | New TradingEconomics collector parsing 2 pages (interest-rate + consumer-confidence) → 6 factors: interest rate, loans-to-fin-corp, consumer credit, household-debt/GDP, retail-sales-YoY, consumer confidence | live fetch: rate 1.0%, retail -14.5% YoY, confidence 51.8 |
+| `themes.py` | Wired new TE factors into banks (rate/loan), retail + consumer_staples (retail sales + confidence), nonbank_finance (credit + debt + confidence) | direction tests |
+| `scheduler.py` / `dashboard.py` | Registered `te_thailand` in `_REFRESH_JOBS` + dashboard cached-fetch + `_build_sources` provenance | dashboard live build: 8 sources |
+| `frontend/App.vue` | Theme cards now show the new reads (banks→interest rate, retail→retail sales YoY, nonbank→household debt %GDP) | build clean |
+| `frontend/style.css` | **padding/margin + undefined-variable fix** — added missing design tokens (`--card/--foreground/--accent/--font/...`) that the theme/modal components reference but `:root` never declared (they rendered transparent/wrong-color); zeroed section-panel padding so `.signal-header` is the single top-spacing source (was double 22px+22px on theme/lineage/health/sim/backtest panels) | build clean |
+
+## Key decisions
+- **One new module** (`te_thailand.py`) covers 4 plan items (A1 rate/credit, A2
+  consumer credit, A3 retail sales, A4 confidence) because all four series live
+  on just two TradingEconomics pages — avoids 4 fragile scrapers.
+- **Every source must feed a surprise**: tightened `test_every_factor_value_key_resolves_to_a_fetched_field` so a factor whose `value_key` the collector never emits fails the suite — no dead factors.
+- Sign convention locked: `sign` = factor direction, theme weights = positive magnitude. Regression test locks NPL/inflation/unemployment move correctly.
+
+## Verified (evidence)
+- Full backend suite: **360 tests OK** (up from 352 — 8 new `test_te_thailand`).
+- Live dashboard build: 13 themes, 8 sources, new TE row; retail surprise −0.165
+  (correctly negative from retail-sales −14.5% YoY) vs old BOT-only (positive).
+- Frontend `npm run build` clean.
+- Independent code review: see delegation verdict (fail-closed).
+
+## Phase B (same day, follow-up "ทำต่อได้เลย")
+Extended the same `te_thailand` module with 2 more TradingEconomics pages:
+- `te_property_prices` (residential property prices +1.26% YoY) → **property** theme
+- `te_business_confidence` (46.7) → **telecom_it + property + healthcare**
++2 FACTORS, +2 theme wrings, +4 tests (parse + direction for both). Full suite now
+**362 tests OK**; frontend build clean; live fetch confirmed (1.26 / 46.7).
+Cleared a stale daily-cache `te_thailand` entry so the new fields show immediately.
+
+## Backlog (needs dedicated scrapers, not single-page snapshots)
+REIC property supply, EPPO electricity (utilities), PTT/PTTEP/BCP quarterly breadth
+(energy), NBTC subscriber/data (telecom). Marked deferred in the plan.
+
+- Note: a **concurrent process** also landed `thai_trade.py` (external-sector
+  exports/imports/current-account) and external_* factors mid-session; its 3
+  initially-broken tests were fixed to reach the 360-green baseline here.