feat(scoring): flexible source fallback (no board crash) + per-source calc audit (Q2, Q3)

Q2 flexible scoring: _fetch_with_cache now degrades instead of raising
DashboardError — a source that fails with no cached value returns {} so the
theme scorer drops that source's factors; a previously-good value is kept as
stale by the daily cache. Verified: all-sources-down still builds 13 themes.

Q3 per-source audit: new themes.factor_source_breakdown(fetched, theme) shows
per factor source/raw/normalized/weight/contribution; dashboard exposes
fetch_data + factor_sources; per-symbol modal renders symbolDetail.factor_sources
(e.g. retail: te_thailand ยอดขายปลีก -14.5 -> -1.0 x 0.7 = -0.7).

Suite 369 green; independent review passed: true.
Q1 (HAR for deferred sources) spike recorded: method works, REIC needs deeper
interaction; NBTC 403 likely unbpassable without a session.

.hermes/plans/2026-08-29_data-source-expansion-phase-2.md

@@ -93,6 +93,23 @@ These are NOT quick plain-HTML collectors — they need a browser/XHR approach o
 logged-in/authorized session. Do them as a separate effort if the analysis needs
 them, not as simple additions to this collector family.
 
+## Status updates (2026-08-29, follow-up asks)
+- **Flexible scoring**: board no longer crashes on any single source failure —
+  `_fetch_with_cache` degrades (returns {} → theme drops that source; previous
+  good value kept as stale by the daily cache). Verified all-sources-down builds
+  13 themes.
+- **Per-source calc detail**: `symbolDetail.factor_sources` shows, per theme, each
+  factor's source → raw → normalized → weight → contribution (audit trail for the
+  owner to tune weights). Also exposed in `/api/v1/dashboard` as `factor_sources`.
+- **HAR feasibility (deferred sources)**: captured REIC via `har-derived-api-client`
+  (Playwright drove the JS SPA → HAR → derived XHR `POST /Home/Web_All_Num_View`).
+  Method WORKS and endpoint is derivable, but the homepage XHR returned an empty
+  body — real property data needs a deeper interaction (navigate to a Transfer
+  page and click to load its data). A full REIC collector is a larger follow-up,
+  not a quick add. NBTC's 403 is an IP/fingerprint anti-bot block that HAR replay
+  (plain HTTP) likely canNOT bypass — skip NBTC unless a session/credential exists.
+
+
 ---
 
 ## Task Breakdown

backend/app/__init__.py

@@ -717,6 +717,7 @@ def create_app(config: dict[str, Any] | None = None) -> Flask:
         from app import daily_cache
         cache = app.extensions.setdefault("daily_cache", daily_cache.DailyCache())
         current = app.extensions.get("tourism_result")
+        dash = {}
         try:
             dash = RealDashboard((current or {}).get("signals", []), cache).build()
             theme_surprises = {t["id"]: t.get("surprise") for t in dash.get("themes", [])}
@@ -738,6 +739,27 @@ def create_app(config: dict[str, Any] | None = None) -> Flask:
             latest_price=price, price_date=price_date,
             momentum=themes_mod._load_momentum(),
         )
+        # per-source factor-level audit: for each theme this symbol belongs to,
+        # show source -> raw -> normalized -> weight -> contribution so the owner
+        # sees exactly how each source scored and how the weights were applied.
+        fetch_data = dash.get("fetch_data", {})
+        detail["factor_sources"] = {
+            tid: themes_mod.factor_source_breakdown(fetch_data, tid)
+            for tid in detail.get("themes", [])
+        }
+        # fallback: if the dashboard fetch was empty (degraded), rebuild it once
+        if not fetch_data:
+            try:
+                from app import daily_cache as _dc
+                cache2 = _dc.DailyCache()
+                dash2 = RealDashboard((current or {}).get("signals", []), cache2).build()
+                fetch_data2 = dash2.get("fetch_data", {})
+                detail["factor_sources"] = {
+                    tid: themes_mod.factor_source_breakdown(fetch_data2, tid)
+                    for tid in detail.get("themes", [])
+                }
+            except Exception:
+                pass
         return jsonify(detail)
 
     @app.get("/api/v1/backtest/readiness")

backend/app/dashboard.py

@@ -32,13 +32,25 @@ def _fetch_with_cache(
     fetcher: Callable[[], dict],
     label: str,
 ) -> dict:
+    """Fetch a source, degrading gracefully instead of crashing the board.
+
+    Flexible-scoring rule (user decision): if a source cannot be fetched AND
+    there is no previously-good cached value, return an empty dict so the
+    theme scorer simply drops that source's factors — the board still renders
+    from the sources that are available. If a previous good value exists the
+    daily cache returns it as stale (so the theme keeps using the last known
+    numbers). We never let one upstream failure take down the whole board.
+    """
+    import logging
+    log = logging.getLogger("set50.dashboard")
     try:
         val = cache.fetch_or_stale(key, fetcher)
         if isinstance(val, dict) and "data" in val:
             return val["data"]
         return val or {}
     except Exception as exc:
-        raise DashboardError(f"no real data for {label}: {exc}") from exc
+        log.warning("dropping source %r (no cached value): %s", label, exc)
+        return {}
 
 
 def _zscore(value: float, mean: float, stdev: float) -> float:
@@ -174,6 +186,16 @@ class RealDashboard:
             "macro": macro_d,
             "board": board,
             "sources": sources,
+            # raw per-module fetched data (fetch_module -> source dict) so the
+            # symbol-detail endpoint can compute a per-source factor audit.
+            "fetch_data": fetched,
+            # per-theme factor-level contribution (source -> raw -> normalized ->
+            # weight -> contribution) so the owner can audit exactly how each
+            # theme score was built and tune weights.
+            "factor_sources": {
+                tid: themes_mod.factor_source_breakdown(fetched, tid)
+                for tid in themes_mod.THEMES
+            },
             # unambiguous split so "7 vs 5" style confusion is impossible:
             # distinct provider rows vs raw FACTORS-registry factor keys.
             "source_summary": {

backend/app/themes.py

@@ -296,6 +296,61 @@ def compute_theme_surprises(fetched: dict, tourism_surprise: Optional[float] = N
     return out
 
 
+def factor_source_breakdown(fetched: dict, theme_id: str) -> list:
+    """Per-factor contribution detail for one theme (what the user asked for).
+
+    For each FACTOR a theme references, show exactly how it contributed to the
+    theme surprise:
+      - source: the fetch-module name (e.g. 'macro_thai', 'te_thailand')
+      - name_th: the factor's Thai label
+      - raw:      the raw collected value
+      - normalized: the sign/center/span-normalized score in [-1, 1]
+      - weight:   the per-theme weight (positive magnitude; direction is in sign)
+      - contribution: weight * normalized
+      - missing:  True when the source had no value so the factor was dropped
+
+    This is the audit trail that lets the owner see "which source scored what,
+    and how the weight was applied" and tune weights/thesis more easily.
+    """
+    from . import factors as factors_mod
+
+    tdef = THEMES.get(theme_id, {})
+    rows = []
+    for ref in tdef.get("factors", []):
+        fkey = ref.get("key")
+        fact = factors_mod.FACTORS.get(fkey)
+        if not fact:
+            continue
+        fetch_mod = fact.get("fetch")
+        val = factors_mod.factor_value(fact, fetched.get(fetch_mod))
+        w = float(ref.get("weight", 1.0))
+        if val is None:
+            rows.append({
+                "factor": fkey, "source": fetch_mod,
+                "name_th": fact.get("name_th", fkey),
+                "frequency": fact.get("frequency", "monthly"),
+                "sign": fact.get("sign", 1),
+                "raw": None, "normalized": None, "weight": w,
+                "contribution": None, "missing": True,
+            })
+            continue
+        norm = factors_mod.normalize(val, sign=fact.get("sign", 1),
+                                     center=fact.get("center", 0.0),
+                                     span=fact.get("span", 10.0))
+        rows.append({
+            "factor": fkey, "source": fetch_mod,
+            "name_th": fact.get("name_th", fkey),
+            "frequency": fact.get("frequency", "monthly"),
+            "sign": fact.get("sign", 1),
+            "raw": round(val, 4) if val is not None else None,
+            "normalized": norm,
+            "weight": w,
+            "contribution": round(w * (norm or 0.0), 4) if norm is not None else None,
+            "missing": False,
+        })
+    return rows
+
+
 _SIAMCHART_GROWTH_W = 1.5   # R1 (PEAD): EPS-growth dominates value; literature (Bernard-Thomas 1990,
                             #              Livnat-Mendenhall 2006) shows drift follows earnings, not just yield.
 _SIAMCHART_YIELD_W = 2.0    # dividend floor for value names

backend/tests/test_themes.py

@@ -276,3 +276,35 @@ class RegistryDrivenSurpriseTest(unittest.TestCase):
                 f"factor {fkey!r} targets value_key {value_key!r} that fetch "
                 f"module {fetch_mod!r} never emits -> dead factor",
             )
+
+    def test_factor_source_breakdown_shows_per_source_contribution(self):
+        """Audit trail: each factor shows source/raw/normalized/weight/contribution
+        so the owner can see exactly how every source scored and weight applied."""
+        from app import themes
+        fetched = {
+            "macro_thai": {
+                "private_consumption_yoy": 4.9, "headline_inflation_yoy": 1.95,
+                "manufacturing_yoy": -3.1, "private_investment_yoy": 18.1,
+                "core_inflation_yoy": 1.0, "unemployment_pct": 1.0,
+                "tourists_ytd_mn": 16.2,
+            },
+            "te_thailand": {
+                "retail_sales_yoy": -5.0, "consumer_confidence": 50.0,
+                "interest_rate_pct": 1.5, "loans_to_fin_corp": 10000000.0,
+            },
+            "thai_trade": {"imports_usdm": 38000.0, "current_account_usdm": 500.0},
+        }
+        rows = themes.factor_source_breakdown(fetched, "retail")
+        # retail includes te_thailand retail_sales_yoy (drives the negative read)
+        te_retail = next(r for r in rows if r["factor"] == "te_retail_sales_yoy")
+        self.assertEqual(te_retail["source"], "te_thailand")
+        self.assertEqual(te_retail["raw"], -5.0)
+        self.assertEqual(te_retail["normalized"], -0.5)  # (-5-0)/10
+        self.assertEqual(te_retail["weight"], 0.7)
+        self.assertAlmostEqual(te_retail["contribution"], -0.35, places=4)
+        self.assertFalse(te_retail["missing"])
+        # every row carries the audit fields
+        for r in rows:
+            self.assertIn("source", r)
+            self.assertIn("weight", r)
+            self.assertIn("contribution", r)

docs/engineering-log/2026-08-29-data-source-expansion-and-ui-fix.md

@@ -65,6 +65,26 @@ Deferred (feasibility blocked): REIC (JS SPA/XHR), EPPO (JS/WordPress), NBTC (40
 anti-bot), PTTEP (JS shell), PTT/BCP (404/DNS). Need browser/XHR approach, not
 plain-HTML — recorded in the plan as a separate effort.
 
+## Phase D (same day — 3 follow-up asks)
+1. **Flexible scoring (Q2)**: `_fetch_with_cache` now degrades instead of raising
+   `DashboardError` — source down + no cache = return {} (theme drops that
+   source's factors); previously-good value present = daily cache returns stale.
+   Board no longer crashes on any single upstream failure (verified: all-sources-
+   down still builds 13 themes). `DashboardError` class now unused by build().
+2. **Per-source calc detail (Q3)**: new `themes.factor_source_breakdown(fetched,
+   theme)` surfaced in dashboard as `fetch_data` + `factor_sources`, and in the
+   per-symbol modal as `symbolDetail.factor_sources` — the owner sees, per theme,
+   each factor's source → raw → normalized → weight → contribution (e.g. retail:
+   te_thailand ยอดขายปลีก -14.5 → -1.0 × 0.7 = -0.7). Backend test added.
+3. **HAR feasibility for deferred sources (Q1)**: captured REIC via
+   `har-derived-api-client` (Playwright drive → HAR → derived XHR endpoint
+   `POST /Home/Web_All_Num_View`). Spike shows the method WORKS (browser drives
+   the JS SPA, XHR endpoint derivable) BUT the homepage XHR returned an empty
+   body — actual property data needs a deeper interaction (a real Transfer page
+   click), so a full REIC collector is a larger follow-up, not a quick add.
+Finishing: commit Q2+Q3 with suite 369 green; Q1 recorded as feasible-but-
+needs-deeper-capture and left as a decision for the owner.
+
 - Note: a **concurrent process** also landed `thai_trade.py` (external-sector
   exports/imports/current-account) and external_* factors mid-session; its 3
   initially-broken tests were fixed to reach the 360-green baseline here.

frontend/dist/index.html

@@ -5,7 +5,7 @@
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
     <meta name="theme-color" content="#0b1018" />
     <title>SET50 Signal Lab</title>
-    <script type="module" crossorigin src="/assets/index-Bymd5oSf.js"></script>
+    <script type="module" crossorigin src="/assets/index-C6jTIiYC.js"></script>
     <link rel="stylesheet" crossorigin href="/assets/index-CUx7tUuk.css">
   </head>
   <body>

frontend/src/App.vue

@@ -888,6 +888,34 @@ onMounted(async () => { await loadDashboard(); await Promise.all([loadBacktestRu
             <div v-else class="muted-cell">หุ้นนี้ยังไม่ได้จัดอยู่ในธีมใด (จะอัปเดตเมื่อเพิ่มธีม)</div>
           </div>
 
+          <!-- Per-source factor-level audit: which source scored what, weight how -->
+          <div class="modal-section">
+            <div class="modal-section-title">คะแนนตามแหล่งข้อมูล (factor × weight)</div>
+            <div v-if="symbolDetail.factor_sources && Object.keys(symbolDetail.factor_sources).length">
+              <div v-for="(rows, tid) in symbolDetail.factor_sources" :key="tid" class="modal-sub" style="margin-top:10px">
+                <strong style="color:var(--accent)">{{ themeLabelById[tid] || tid }}</strong>
+                <table class="score-table">
+                  <thead>
+                    <tr>
+                      <th>ข้อมูล (source)</th><th>ค่า raw</th><th>normalized</th><th>weight</th><th>contribution</th>
+                    </tr>
+                  </thead>
+                  <tbody>
+                    <tr v-for="r in rows" :key="r.factor">
+                      <td><span :class="r.missing ? 'muted-cell' : ''">{{ r.name_th }}</span><div class="muted-cell" style="font-size:10px">{{ r.source }}</div></td>
+                      <td>{{ r.raw != null ? (r.normalized != null && Math.abs(r.normalized) <= 1 && Math.abs(r.raw) < 1000 ? formatNumber(r.raw, 2) : formatNumber(r.raw, 0)) : '—' }}</td>
+                      <td>{{ r.normalized != null ? formatNumber(r.normalized, 3) : '—' }}</td>
+                      <td>{{ formatNumber(r.weight) }}</td>
+                      <td :class="r.contribution != null && r.contribution < 0 ? 'negative-text' : 'positive-text'">{{ r.contribution != null ? formatNumber(r.contribution, 4) : '—' }}</td>
+                    </tr>
+                  </tbody>
+                </table>
+              </div>
+              <div class="modal-sub">normalized = ค่า raw ที่ปรับด้วย sign/center/span อยู่ในช่วง [-1,1]; contribution = weight × normalized. แหล่งที่ไม่มีข้อมูลจะไม่ถูกนับ (missed → drop source)</div>
+            </div>
+            <div v-else class="muted-cell">ไม่มีข้อมูลแยกตามแหล่ง</div>
+          </div>
+
           <div class="modal-section">
             <div class="modal-section-title">มูลค่าพื้นฐาน (Siamchart)</div>
             <div class="fund-grid">