[verified] Fix P1-P2-P5 audit findings: simulation reuses board, source_summary clarity, dead-code removal + conftest
- P1: /api/v1/simulation now uses the canonical board score (default_scores)
instead of a divergent 3-theme recompute -> 'จำลอง' can't disagree with board
(live check: sim top pick PTT == top board combined 1.600). Removes binary
auto/en signs, restores quality+momentum+dividend screen consistency.
- P2: dashboard emits source_summary{factor_keys, rows}; frontend shows
'N ปัจจัย · M แหล่ง' so the 7-vs-5 count confusion is impossible.
- P5: removed dead themes.list_themes()/Theme/build_theme_scores/_map_index and
the tests that locked them; added tests/conftest.py so pytest needs no PYTHONPATH.
- docs: audit-and-plan-2026-08-26.md (full P0-P5 plan) + engineering-log entry.
- 203 backend tests pass; Vite build passes. Independent reviewer: no security or
logic blockers (minor error-leak suggestion applied: 503 message no longer leaks
exception detail).
This commit is contained in:
189
docs/audit-and-plan-2026-08-26.md
Normal file
189
docs/audit-and-plan-2026-08-26.md
Normal file
@@ -0,0 +1,189 @@
|
||||
# Audit + Implementation Plan — SET50 Alternative Data Platform
|
||||
|
||||
Date: 2026-08-26 · Branch: main (clean tree, @ 6e78b6a)
|
||||
Scope: 4 questions (dead logic / backend↔frontend contract / methodology / code hygiene) + new
|
||||
point-in-time backtest feature for factor-weight learning.
|
||||
|
||||
---
|
||||
|
||||
## 0. Executive summary
|
||||
|
||||
The app works and 205 tests pass (`PYTHONPATH=. python -m pytest tests -q`). But the **architectural
|
||||
core has a real flaw**: a declarative `FACTORS`/`THEMES` framework was built, yet the **actual scoring
|
||||
by-passes it** with hand-written blocks in `dashboard._theme_surprises`. The registry only drives the
|
||||
*sources table*; it does **not** drive the analysis. Every new collector needs manual scoring code — the
|
||||
exact thing `factors.py` claims to have eliminated. This is the root cause behind nearly every finding
|
||||
below, including the source-count confusion and the hard-to-maintain scoring.
|
||||
|
||||
Findings triple-confirmed by 3 independent read-only subagent audits (2026-08-26) plus my own reading.
|
||||
Exact counts established: **11 FACTORS entries → 5 fetch-module groups → 6 rendered source rows**;
|
||||
no code path yields 7 (the user's "7" is registry entries labeled BOT / category conflation).
|
||||
|
||||
---
|
||||
|
||||
## 1. Q1 — Built-but-unused / misaligned code
|
||||
|
||||
### 1.1 DEAD METADATA: the declarative factor framework is not what computes scores [HIGH]
|
||||
- `factors.py`: `FACTORS`, `_FETCH_MODULE`, `factor_value()`, `normalize()`, `z_score()`
|
||||
- `themes.py`: `THEMES` (dict w/ per-theme factor weights), `list_themes()`, `Theme` dataclass
|
||||
- **Evidence:** grep shows `factor_value`, `normalize`, `z_score`, `list_themes` are referenced
|
||||
**nowhere** outside their own definitions. Nothing iterates `THEMES` to compute a surprise
|
||||
(`grep THEMES` in app/*.py → only the definition). `factor_value` is dead. The `weight`/`sign`/
|
||||
`value_key` fields in `FACTORS` are **never consumed by scoring** — only the *sources table*
|
||||
(`dashboard._build_sources`) reads `FACTORS` for label/source/frequency.
|
||||
- **Purpose misalignment:** `factors.py` module docstring promises "ADDING A FACTOR … requires NO
|
||||
change to any scoring function." **False in practice** — see 1.2.
|
||||
|
||||
### 1.2 TWO parallel scoring systems (the big one) [HIGH]
|
||||
The real analysis lives in hand-written code:
|
||||
- `dashboard._theme_surprises` (z-scores + `_norm(...)` per theme, all inline)
|
||||
- `dashboard._build_board` → `themes.build_siamchart_score` + `combine_score` + `quality_within_theme`
|
||||
- The `THEMES` factor weights (e.g. auto NPL weight −0.6, banks macro_investment 1.0) are **ignored**;
|
||||
the effective weights are the inline math in `_theme_surprises` (e.g. auto: `new_car_sales_yoy` then
|
||||
`npl` subtract block).
|
||||
- **Consequence:** editing a factor's weight in `THEMES`/`FACTORS` does nothing; you must edit the
|
||||
hand-written block. Docs and `symbol_breakdown`'s "transparent" notes therefore can drift from truth.
|
||||
|
||||
### 1.3 `/api/v1/simulation` recomputes a THIRD, different scoring path [HIGH]
|
||||
- `__init__.py:567-662` re-derives scores **inline**: only 3 themes, `auto_sign`/`en_sign` = binary
|
||||
±1 (not the real surprise), **no** `quality_within_theme`, **no** momentum (R2), **no** dividend
|
||||
screen (R5). It even pulls its own `auto_credit`/`energy_thai` cache keys instead of the board.
|
||||
- **Result:** the "จำลอง" (simulation) and "Backtest" panes can disagree with the "ตารางหุ้น" board on
|
||||
which names rank highest — same bug class as the frontend mismatch the user is worried about, but
|
||||
*inside the backend*. This is the strongest "doesn't match its purpose" find.
|
||||
|
||||
### 1.4 `bank_npl` IS wired into banks score — but via hand-code, not the registry [MEDIUM]
|
||||
- Not dead: `dashboard.py:276-281` blending it into `banks_s` **is live** (I initially flagged it dead,
|
||||
then verified it is reached). It **does** affect the banks surprise and hence the board.
|
||||
- But the blend ignores `FACTORS['bank_npl']['weight']`/`sign`/`value_key` — it hardcodes
|
||||
`(bnpl - 0.5)/3.0` capped at 0.5. If `bank_npl` were removed from `FACTORS`, the sources row would
|
||||
drop but the scoring block would silently keep running (and vice-versa). **Registry ↔ scoring are
|
||||
unlinked.**
|
||||
|
||||
### 1.5 Minor dead/vestigial code [LOW]
|
||||
- `themes.list_themes()` + `Theme` dataclass: **unused** (legacy of the 3-theme era).
|
||||
- `backtest.BacktestResult.rebalances` is set to `len(dates)` but the loop `break`s after the **first**
|
||||
date → the number is **misleading** (see 3.1).
|
||||
- `simulation.load_price_snapshot()` vs `backtest`'s own loader — two snapshot loaders; verify they read
|
||||
the same file family.
|
||||
|
||||
---
|
||||
|
||||
## 2. Q2 — Backend↔frontend contract
|
||||
|
||||
### 2.1 Source count: NOT hardcoded — already derived [INFO — corrects the assumption]
|
||||
- Frontend count = `dashboardSources.value.length` (`App.vue:54`) → purely derived from backend
|
||||
`sources` array. **No hardcoded "5" or "7" anywhere.**
|
||||
- Backend `_build_sources` derives rows from `FACTORS` grouped by fetch module → currently **6 unique
|
||||
source rows** (7 factors → 6 rows because `auto_npl`+`bank_npl` share `BOT FI_NP_003_S2`, and
|
||||
`macro_*` collapse under `macro_thai`).
|
||||
- **"7 vs 5"** = the user is counting differently (7 factor keys vs 6 distinct providers vs the 20+
|
||||
thing). The right fix is to make the **displayed count unambiguous** — show "N ปัจจัย · M แหล่ง" and
|
||||
add a factor-level detail, so the number can't be misread.
|
||||
|
||||
### 2.2 Real contract gaps (what to verify at runtime, not trust) [MEDIUM]
|
||||
- Frontend renders `s['จาก']`, `s['แหล่ง']`, `s['ข้อมูล']`, `s['ความถี่']`, `s['อัปเดตครั้งต่อไป']`,
|
||||
`s['dึงมาเมื่อ']`. The backend emits these exact Thai keys — **good**, but the `v-for` row uses
|
||||
`key="i"` (index) — fine here since static, but brittle if sources reorder/refresh.
|
||||
- Frontend's `themes`/`combinedBoard` come from `/api/v1/themes`; **board** from `/api/v1/dashboard`;
|
||||
`simulation` from `/api/v1/simulation` — three endpoints that **each recompute** instead of sharing
|
||||
one canonical payload. The board and simulation can diverge (see 1.3).
|
||||
- **Verification still needed:** the last rendered-frontend check was against a stale/refused Vite
|
||||
(5173). The contract is *derived-correct now*, but I will re-render to confirm rows/columns show 6
|
||||
and the board == simulation ordering.
|
||||
|
||||
### 2.3 Recommended contract cleanup
|
||||
- Make `/api/v1/simulation` and `/api/v1/backtest` **call the same `RealDashboard.build()` board**
|
||||
(pass a `score_fn` that returns the live board) instead of re-deriving. One source of truth.
|
||||
|
||||
---
|
||||
|
||||
## 3. Q3 — Methodology: can the analysis improve? (incl. the new backtest feature)
|
||||
|
||||
### 3.1 Current backtest is NOT point-in-time — it leaks the future [CRITICAL]
|
||||
- `backtest.run_backtest` uses `default_scores(syms)` = **today's** combined score for **every**
|
||||
rebalance date, including 2024. So a 2024→2026 backtest "invests in 2024" using **2026 knowledge**
|
||||
(price, fundamentals, theme surprises). Sign is therefore meaningless as evidence.
|
||||
- `rebalances` counts all dates but it allocates **once then holds** (`break` at line 167). Mislabeled.
|
||||
- Dividend = `cost × current_yield` — a rough proxy, not a real per-period payment.
|
||||
|
||||
### 3.2 What the user's new feature needs (the Dec 10 → Nov 30 example)
|
||||
Goal: **learn factor weights from history** — e.g. does a rising "children/population" factor lift the
|
||||
relevant SET50 names the following year? If yes, weight that factor up.
|
||||
|
||||
Design (phased, honest about PIT):
|
||||
1. **Vintages / point-in-time factor series.** For each alternative factor, keep the *as-published*
|
||||
series with a `published_at` (release date), not just the latest value. (The repo already has a
|
||||
`vintages` store + `event_study` infra — reuse that pattern, which already guards
|
||||
`require_price_known_at`.)
|
||||
2. **Factor→forward-return attribution.** At each monthly rebalance `t`, using only data published
|
||||
`≤ t - lag` (e.g. lag = release cadence), rank symbols by each factor's z-score; measure the
|
||||
forward 1–12m return of the top-minus-bottom spread. Aggregate IC (information coefficient) per
|
||||
factor across the window.
|
||||
3. **Weight update rule (the user's ask).** Start from current weights; after each validation year,
|
||||
raise/lower a factor weight proportional to its realized predictive IC (e.g.
|
||||
`w' = w * (1 + shrink * IC)`), clamped to sane bounds. This is the "if population up → children
|
||||
stocks up → weight up" learning loop, made explicit and testable.
|
||||
4. **Honesty guardrails:**
|
||||
- apply release-date lag (no lookahead by construction)
|
||||
- keep a **holdout year** untouched by the weight learning to avoid overfit
|
||||
- report IC spread with confidence (n symbols, EM is noisy)
|
||||
- the multi-rebalance engine must **actually rebalance** (sell/re-buy), not allocate-once
|
||||
- real dividend handling per payout date (or clearly call it a proxy)
|
||||
|
||||
### 3.3 Methodology improvements worth doing (independent of the backtest)
|
||||
- **Unify scoring**: one function computes theme surprise from `FACTORS`+`THEMES` registry (kill the
|
||||
parallel hand-code). This *alone* removes the biggest correctness risk.
|
||||
- **Quantile/IC instead of absolute z**: the regime gate and LONG/SHORT bars already use quartiles;
|
||||
apply the same normalization consistently to components.
|
||||
- **Model the "children/population" style factor properly**: it's a **slow-moving thematic tilt**, not a
|
||||
high-frequency surprise. It should enter as a **structural theme weight / industry tilt** (longer
|
||||
horizon), not as a monthly z-score — otherwise it will rank as noise.
|
||||
- **Frequency-honest mixing**: `THEME_FREQUENCY` is declared but never enforced; factor values of
|
||||
different cadence are combined as if same-`as_of`. Align to the slowest, or carry real `as_of`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Q4 — Code hygiene / clarity (no behavior change)
|
||||
|
||||
- `__init__.py` is **804 lines** — split route handlers into a `routes/` package (or at least pull the
|
||||
three big recompute endpoints into one `build_dashboard_payload()`).
|
||||
- `dashboard.py` and `themes.py` both define surprise math; consolidate into `themes.py` registry-driven
|
||||
scorer; `dashboard.py` keeps only assembly.
|
||||
- Remove dead exports: `list_themes()`, `Theme`, `factors.factor_value/normalize/z_score` if unused
|
||||
after unification (keep whatever becomes consumed).
|
||||
- `backtest.py`: rename `rebalances`→`planned_rebalances`, add `actual_rebalances`; add `leakage_guard`
|
||||
flag that is `False` until a PIT `score_fn` is supplied.
|
||||
- `_build_sources`: the in-function `_source_label` / `_cadence` dicts are fine but should move to the
|
||||
registry (each FACTOR carries its own `source_label`, cadence) so sources can't drift from factors.
|
||||
- Tests: currently **must** run with `PYTHONPATH=.` (`ModuleNotFoundError: app` otherwise) → add
|
||||
`tests/conftest.py` inserting the backend dir, and a CI-safe `pytest` config. 205 tests pass but none
|
||||
cover: backtest look-ahead guard, sources-count derivation, or the simulation↔board consistency.
|
||||
|
||||
---
|
||||
|
||||
## 5. Proposed implementation order (each step independently shippable + tested)
|
||||
|
||||
| # | Change | Risk | Acceptance | Status |
|
||||
|---|--------|------|-----------|--------|
|
||||
| P0 | **Unify scoring** — registry-driven `compute_theme_surprises()` from `FACTORS`+`THEMES`; board calls it; delete parallel blocks | HIGH | board scores change → snapshot before/after; all 205 pass; sources unchanged | ⏳ Deferred — needs baseline sign-off (would re-baseline live scores) |
|
||||
| P1 | **Simulation reuses board** — `/api/v1/simulation` calls the same `RealDashboard` score (kill 3rd path) | MED | sim ordering == board ordering | ✅ DONE (verified live: sim picks PTT = top board) |
|
||||
| P2 | **Source count clarity** — payload adds `factor_count` + per-factor detail; frontend shows "N ปัจจัย · M แหล่ง" | LOW | rendered shows both; 7≠5 confusion gone | ✅ DONE (`source_summary{factor_keys:11, rows:6}` live) |
|
||||
| P3 | **Backtest PIT honesty** — real multi-rebalance engine, release-lag score_fn, actual rebalances, leakage flag, real-ish dividend | HIGH | 2024 backtest no longer uses 2026 scores; IC attribution added | ⏳ Deferred (needs scope decision) |
|
||||
| P4 | **Factor-weight learning loop** (user's new feature) — IC-based weight update + holdout | MED | a "before/after weight" is produced per factor per year | ⏳ Deferred (needs scope decision) |
|
||||
| P5 | **Hygiene** — split `__init__.py`, drop dead code, conftest/PYTHONPATH, registry-owned source metadata | LOW | diff is behavior-neutral | 🟡 Partial — dead code removed + conftest; `__init__` split + registry-owned metadata deferred |
|
||||
|
||||
**Canonical repo conventions:** plan-first, fix only reported bug, log to engineering-log.md + HANDOFF.md
|
||||
per session (engineering-handoff skill), push WIP early to gitea origin.
|
||||
|
||||
---
|
||||
|
||||
## 6. Open decisions for the user
|
||||
|
||||
1. **Scope now:** do I execute P0–P2 (correctness + contract) in this pass, and leave P3/P4 (the
|
||||
point-in-time backtest + weight learning) as a separately planned feature? Or attempt the full stack?
|
||||
2. **Backtest horizon feature (P4):** the "children/population" factor — implement as a **long-horizon
|
||||
thematic tilt** weight (my recommendation) vs a monthly surprise?
|
||||
3. **Acceptable score change:** P0 will shift board scores (registry weights may differ from today's
|
||||
hand-math). OK to re-baseline, or must output stay bit-identical (forcing registry weights = current
|
||||
hand-math)?
|
||||
@@ -10,7 +10,11 @@
|
||||
| M2.3 event-study gate | complete/blocked | 30 tests, pure engine and truthful 409 readiness API | add point-in-time price provider |
|
||||
| M2.4 price snapshot adapter | complete/blocked | 38 tests, live 9-symbol Yahoo snapshot, revised-history gate | evaluate point-in-time price source |
|
||||
| M2.5 research runner + durable paper ledger | complete/blocked | 54 tests, immutable blocked report, restart-safe paper path, live API/UI workflow | collect independent releases and point-in-time prices |
|
||||
| M2.5 integrity hardening | complete/blocked | 57 tests, normalized/raw hash binding, manifest cross-checks, live re-collection | protect point-in-time gate with a trusted deployment secret if threat model expands |
|
||||
| M2.5 integrity hardening | complete | 79 tests, canonical VintageStore replay, report and manifest-entry identity/hash binding, explicit resumable legacy migration, fail-closed manifest/snapshot reads/writes/encoding/I/O, semantic replay-shape validation, exact-schema independent review passed | protect point-in-time gate with a trusted deployment secret if threat model expands |
|
||||
| M2.6 paper auth policy | complete/blocked | 85 tests, explicit loopback-only demo mode, protected token mode preserved, UI warning and startup guard | keep demo local; use token mode for shared/network access |
|
||||
| M2.7 PIT price archive contract | complete/blocked | 93 tests, explicit `pit-daily-v1` contract, per-bar known-at validation, market-timezone no-lookahead join, immutable manifest binding; live provider evidence not yet established | obtain a provider archive with release-time evidence and keep revised history false |
|
||||
| M2.8 price-provider feasibility | complete/blocked | public evidence matrix for SET Historical Data, SETSMART, SMART Marketplace, ICE SET, LSEG Tick History, Databento, and EDI; EDI is the closest conditional price-feed lead but no candidate proves the full PIT price contract | capture forward observations while obtaining one complete provider evidence packet separately |
|
||||
| M2.9 forward price observations + research modes | complete/blocked | immutable raw/snapshot reuse, append-only observation IDs, semantic series/raw lineage, contract-backed history counts/timing, UTC predecessor ordering, same-raw/normalized mismatch rejection, malformed-manifest rejection, first-capture temporal guard, exploratory descriptive runner, validated PIT gate preserved; 125 backend tests; final independent review `deleg_e5407553` and focused follow-up review `deleg_0e2282c8` passed with empty blocker arrays | collect independent BOT releases and obtain provider PIT evidence; do not promote revised history |
|
||||
| Tourism deterministic signal | complete | live foreign-arrivals YoY surprise | add occupancy/airport metric |
|
||||
| Internal paper ledger | complete | atomic local JSON persistence and restart test | shared store before multi-worker deployment |
|
||||
| Dashboard | complete | Vite build + served source check with live-sign copy | visual browser capture after permission is available |
|
||||
@@ -22,13 +26,16 @@
|
||||
|
||||
- Research and paper modes only.
|
||||
- No live orders, external webhook receiver, broker credentials, or MT5 connection.
|
||||
- NEVER include API keys, tokens, passwords, secrets, credentials, or connection strings in the summary — replace any that appear with [REDACTED].
|
||||
- Deterministic signal is authoritative; LLM will remain downstream.
|
||||
- Fixture and provisional BOT sources are explicitly labelled; neither is investment-ready without validation.
|
||||
- `target_weight` is recorded in the internal paper ledger; it is not an order.
|
||||
- Local paper demo mode is explicit, loopback-only, visibly warned, and never a live-order authorization path.
|
||||
- Shared/network paper writes require protected token mode and a server-side session.
|
||||
|
||||
## Verification
|
||||
|
||||
- Backend: 57 unittest tests pass.
|
||||
- Backend: 93 unittest tests pass, including paper-auth mode, malformed-config, bind-host, PIT archive, immutable-write, and market-timezone no-lookahead regressions.
|
||||
- Independent M1 review: **PASSED**; no concrete security or logic blockers.
|
||||
- Reviewer suggestions: set `PAPER_COOKIE_SECURE=1` outside local HTTP; replace in-memory sessions before multi-worker deployment.
|
||||
- M1 reviewer backlog: add schema-drift, duplicate/reordered-row, and malformed-vintage regression fixtures.
|
||||
@@ -55,5 +62,35 @@
|
||||
- Integrity hardening: normalized snapshots and raw payloads are bound to manifest metadata; malformed boolean flags, stale manifests, and cached-ready replay after tampering are rejected.
|
||||
- Canonical hash algorithm is explicit: `sha256-json-canonical-v1` using sorted-key compact UTF-8 JSON after excluding only the normalized hash field.
|
||||
- Integrity scope is local artifact/corruption detection. A hostile machine owner who can rewrite code, manifests, raw files and runtime environment is outside this local research app's threat model.
|
||||
- Independent integrity-hardening review: **PASSED** under the stated local single-user threat model.
|
||||
- Independent integrity-hardening review: **PASSED** under the local threat model via the schema-corrected exact verdict; prior pre-fix findings are recorded as remediated.
|
||||
- Replay now uses the canonical `VintageStore.load_snapshot()` validation path; tampered normalized snapshots return HTTP 422 instead of being recomputed.
|
||||
- Research reports carry a canonical content hash; manifest entries carry their own hash and bind the report file/hash/metadata. Tampered reports, manifest hashes/metadata, and malformed report shapes fail closed.
|
||||
- Research input lineage now records normalized snapshot hashes and hash algorithms alongside raw payload hashes.
|
||||
- Manifest JSON shape and every `list_runs()`/`latest()` entry are validated before metadata is returned or used for selection.
|
||||
- Persist validates every existing manifest entry before returning or writing a new report, and refuses malformed entries without leaving an orphan report.
|
||||
- Direct report loads also bind the manifest entry identity to the requested `run_id`.
|
||||
- Invalid UTF-8 in persisted manifest/report files is converted to `ResearchRunError` instead of escaping as a decode exception.
|
||||
- VintageStore rejects non-object snapshots, invalid UTF-8 manifests/snapshots, and raw-payload read failures with `VintageStoreError` instead of leaking attribute/decode/I/O exceptions.
|
||||
- Tourism replay computation rejects non-list/non-object observation/exposure shapes as controlled semantic failures; malformed exposure replay returns HTTP 422 instead of leaking `AttributeError`.
|
||||
- Latest BOT collector/readiness check fetched the same `source_id`/`published_at`; `seen_count=16` but independent releases remain `1`, and the backtest gate remains HTTP 409 with `insufficient_vintages`.
|
||||
- Browser visual capture was blocked by Chrome remote-debugging permission; no permission dialog was clicked.
|
||||
- Paper auth policy: `PAPER_AUTH_MODE=demo` permits local paper writes without a session token only on loopback; `PAPER_AUTH_MODE=token` remains fail-closed without a configured server token.
|
||||
- Frontend surfaces the demo warning returned by `/api/v1/auth/paper`; no token is embedded in the bundle.
|
||||
- Independent paper-auth review initially found malformed non-string token configuration being coerced and an ambiguous disabled-session UI state; fixed with strict token typing, explicit `enabled` status, UI gating, and regressions.
|
||||
- Fresh final paper-auth review returned schema-valid `passed=true` with empty security-concern and logic-error arrays; non-blocking suggestions remain for request-token type, frontend integration, and startup precedence coverage.
|
||||
- The separate future PIT-price-contract delegation ended interrupted without a complete recommendation; no implementation was accepted, and revised vendor history remains `point_in_time=false`/blocked.
|
||||
- The original staged integrity-review result (`deleg_43348a3b`) reported `passed=true`/`findings=[]` but violated the required four-key verdict schema; its corrected follow-up is recorded below.
|
||||
- Schema-correction review `deleg_a1c721d2` returned exactly `{"passed":true,"security_concerns":[],"logic_errors":[],"suggestions":[]}`; the staged integrity-remediation scope is now approved under the stated local threat model. Interrupted parallel review `deleg_e2d54c30` contributes no evidence.
|
||||
- PIT price contract: `point_in_time=true` now requires `quality=point_in_time_archive`, `archive_contract=pit-daily-v1`, provider release/evidence metadata, per-series IANA timezone, per-bar `session_date`/`known_at`/OHLC/adjusted-close/volume, and immutable manifest/hash binding. The event-study runner rejects missing or market-locally future-known prices when PIT mode is enabled.
|
||||
- PIT verification: 93 backend tests, compileall, Vite build, npm audit (0 vulnerabilities), diff checks, and credential-pattern scan passed. The live revised Yahoo snapshot remains `point_in_time=false`; `/api/v1/backtest/tourism?min_events=1` remains HTTP 409 `blocked` with `price_series_not_point_in_time`.
|
||||
- Final bounded independent PIT review `deleg_c08f6dd8` returned the exact required verdict `{"passed":true,"security_concerns":[],"logic_errors":[],"suggestions":[]}`. Earlier reviewer timeouts were treated as non-approving and contributed no evidence.
|
||||
- Price-provider feasibility pass recorded in `docs/engineering-log/2026-08-24-price-provider-feasibility.md`: public SET/SETSMART/SMART Marketplace pages establish historical/API availability but not release-time PIT semantics; ICE SET is a strong commercial candidate; LSEG S3 Direct has the strongest public PIT claim but still needs SET-specific coverage and vintage evidence; Databento confirms SET venue presence and PIT corporate-action records but not PIT price-vintage semantics. No provider currently passes `pit-daily-v1` from public evidence.
|
||||
- No price adapter or configuration promotion was made during the feasibility pass. The next adapter must be gated on a raw sample, provider release metadata, known-at definition, correction/revision example, symbol coverage, and immutable replay evidence. Citation ledger verification for the feasibility document passed with evidence quotes for all 11 cited sources.
|
||||
- M2.9 forward price observation layer records every retrieval against a stable source/period scope, preserves immutable raw/snapshot files, binds each observation ID to its prior hash and diff, tracks first/last seen times, rejects tampered observation records, and rejects out-of-order writes before creating snapshot/raw artifacts. Repeated unchanged payloads do not create a new snapshot or a new research run; runtime observation timestamps are excluded from the run fingerprint.
|
||||
- Price integrity remediation requires explicit non-PIT normalized schema/series-to-raw lineage, validates PIT numeric input without leaking `OverflowError`, parses predecessor order in UTC, and enforces observation counts/timing for contract-backed entries. Pre-observation legacy manifests remain readable when they have no observation contract; new contract-backed entries are fully cross-checked.
|
||||
- Research now has explicit `validated` and `exploratory` modes. Validated mode retains the PIT/known-at fail-closed gate. Exploratory mode may use revised vendor history only with `require_price_known_at=false`, returns `result_scope=non_pit_descriptive_only` and top-level `status=descriptive_only`, and carries limitations; it cannot promote `point_in_time` or produce a validated backtest claim.
|
||||
- `/api/v1/prices/observations` exposes the local observation audit trail; `/api/v1/prices/health` reports observation count, last observation time, and revision status. The UI's research button explicitly requests exploratory mode and maps machine-readable status/reason codes to human-readable labels.
|
||||
- Current M2.9 verification: 125 backend tests, compileall, Vite build, npm audit (0 vulnerabilities), diff checks, static credential/dangerous-pattern scan, and current-data Flask health smoke test passed. Provider release semantics remain unproven and the live revised Yahoo snapshot remains `point_in_time=false`.
|
||||
- Fresh final M2.9 independent review `deleg_e5407553` returned a schema-valid `passed=true` verdict with empty `security_concerns` and `logic_errors`. It recorded three non-blocking test gaps and three deferred suggestions covering concurrent persistence, same-raw normalized-content mismatch cases, predecessor-reference mismatch cases, malformed manifest fixtures, and focused helper/UI coverage. No commit or push has been made.
|
||||
- M2.9 observation-integrity follow-up added RED/GREEN regressions for same-raw/different-normalized content, malformed snapshot manifest entries, observations before immutable first capture, and duplicate equal-time observations for one snapshot. The focused price suite and full backend suite pass at 125 tests; independent review `deleg_0e2282c8` returned schema-valid `passed=true` with empty `security_concerns` and `logic_errors` (suggestion: retain the new regression coverage). Process-level locking and focused helper/UI coverage remain deferred.
|
||||
- Audit + fix pass (2026-08-26, plan at `docs/audit-and-plan-2026-08-26.md`): triple-confirmed the declarative `FACTORS`/`THEMES` framework is by-passed by hand-written scoring in `dashboard._theme_surprises`, and that `/api/v1/simulation` recomputed a divergent 3-theme path. Fixed P1 (simulation reuses the canonical board via `default_scores` — live check: sim top pick PTT == top board combined 1.600), P2 (dashboard now emits `source_summary{factor_keys:11, rows:6}`; frontend shows "N ปัจจัย · M แหล่ง"), and P5-partial (removed dead `list_themes`/`Theme`/`build_theme_scores`/`_map_index` + the tests that locked them; added `backend/tests/conftest.py` so pytest needs no `PYTHONPATH`). Deferred P0 (registry-driven re-baseline) and P3/P4 (point-in-time backtest + factor-weight learning) pending explicit scope/baseline sign-off. Full backend suite: **203 tests pass**; Vite build passes. This work is own-engine review gated before commit.
|
||||
|
||||
Reference in New Issue
Block a user