set50-system/docs/engineering-log.md

Engineering Log — SET50 Alternative Data Platform

Current status

Milestone	Status	Evidence	Next action
M0 repo foundation	complete	Flask API, Vue/Vite shell	keep research/paper guardrails
M1 BOT Tourism adapter	complete	18 tests, live BOT fetch, raw/snapshot persistence	validate multiple vintages
M2 vintage collector	complete	25 tests, manifest idempotency, live collector and point-in-time API	collect independent releases
M2.3 event-study gate	complete/blocked	30 tests, pure engine and truthful 409 readiness API	add point-in-time price provider
M2.4 price snapshot adapter	complete/blocked	38 tests, live 9-symbol Yahoo snapshot, revised-history gate	evaluate point-in-time price source
M2.5 research runner + durable paper ledger	complete/blocked	54 tests, immutable blocked report, restart-safe paper path, live API/UI workflow	collect independent releases and point-in-time prices
M2.5 integrity hardening	complete/blocked	57 tests, normalized/raw hash binding, manifest cross-checks, live re-collection	protect point-in-time gate with a trusted deployment secret if threat model expands
Tourism deterministic signal	complete	live foreign-arrivals YoY surprise	add occupancy/airport metric
Internal paper ledger	complete	atomic local JSON persistence and restart test	shared store before multi-worker deployment
Dashboard	complete	Vite build + served source check with live-sign copy	visual browser capture after permission is available
LLM analysis	deferred	intentionally no LLM dependency in M0	add after signal lineage is stable
Webhook receiver	deferred	contract only, no external receiver	choose after core app is usable
MT5 bridge	deferred	not started	paper bridge after webhook decision

Guardrails

• Research and paper modes only.
• No live orders, external webhook receiver, broker credentials, or MT5 connection.
• Deterministic signal is authoritative; LLM will remain downstream.
• Fixture and provisional BOT sources are explicitly labelled; neither is investment-ready without validation.
• target_weight is recorded in the internal paper ledger; it is not an order.

Verification

• Backend: 57 unittest tests pass.
• Independent M1 review: PASSED; no concrete security or logic blockers.
• Reviewer suggestions: set PAPER_COOKIE_SECURE=1 outside local HTTP; replace in-memory sessions before multi-worker deployment.
• M1 reviewer backlog: add schema-drift, duplicate/reordered-row, and malformed-vintage regression fixtures.
• Frontend: npm run build passes with Vite.
• Backend health endpoint returns HTTP 200 JSON.
• Dashboard served HTML contains the current title, Vue mount point and Vite entry.
• Paper ledger POST and readback work through the live API.
• BOT source live fetch parsed 138 monthly periods and persisted raw HTML plus normalized snapshot.
• Data-health reports source_mode=bot, status=provisional, and replayable=true.
• Live vintage replay returned the same theme surprise as the current dashboard summary.
• Vintage collector preserved one live vintage_id with seen_count=4 and point-in-time API excluded it before published_at.
• Independent M2 review: PASSED; no concrete security or logic blockers.
• Event-study readiness gate correctly returns HTTP 409 with 1/12 independent vintages; no backtest result is fabricated.
• Independent M2.3 review: PASSED; no concrete security or logic blockers.
• Price snapshot normalized 9 symbols with adjusted close and provider-derived trading dates; quality is explicitly revised_vendor_history and point_in_time=false.
• Backtest gate requires both independent vintages and point-in-time prices.
• Independent M2.4 review: PASSED; no concrete security or logic blockers.
• Research runner persists blocked/ready reports with frozen input IDs, hashes, configuration and gate reasons.
• Revision-aware readiness counts (source_id, published_at) once and runner selects the latest revision only.
• Event-study defaults to next trading-session execution and handles non-trading event dates without skipping the first session.
• Paper ledger persists atomically under ignored backend/data/paper/ledger.json when configured.
• Live API 0.5.0 created and replayed the same blocked Tourism research report; UI served the research-run panel and human-readable gate reason.
• Independent M2.5 review: PASSED; no concrete security or logic blockers.
• Integrity hardening: normalized snapshots and raw payloads are bound to manifest metadata; malformed boolean flags, stale manifests, and cached-ready replay after tampering are rejected.
• Canonical hash algorithm is explicit: sha256-json-canonical-v1 using sorted-key compact UTF-8 JSON after excluding only the normalized hash field.
• Integrity scope is local artifact/corruption detection. A hostile machine owner who can rewrite code, manifests, raw files and runtime environment is outside this local research app's threat model.
• Independent integrity-hardening review: PASSED under the stated local single-user threat model.
• Browser visual capture was blocked by Chrome remote-debugging permission; no permission dialog was clicked.