[verified] add SET price snapshot adapter
This commit is contained in:
@@ -4,8 +4,8 @@
|
||||
|
||||
- Path: `/Users/kunthawat/Gitea/set50-alternative-data-platform`
|
||||
- Branch: `main`
|
||||
- Verified code commit: `0b47a06` — `[verified] add event-study readiness gate`
|
||||
- Current milestone: M2.3 event-study foundation complete; real backtest blocked
|
||||
- Verified code commit: `0b47a06` — `[verified] add event-study readiness gate` (price adapter pending commit)
|
||||
- Current milestone: M2.4 price snapshot foundation complete; real backtest blocked
|
||||
- Mode: research + paper only
|
||||
- Frontend: Vue 3 + Vite
|
||||
- Backend: Flask `0.4.0`
|
||||
@@ -24,6 +24,8 @@
|
||||
- Point-in-time vintage query: `GET /api/v1/vintages?as_of=<ISO-8601>`.
|
||||
- Deterministic event-study engine with window, benchmark and cost calculations.
|
||||
- Backtest readiness gate: `GET /api/v1/backtest/tourism?min_events=12`.
|
||||
- Yahoo-backed daily price snapshot with SET symbol mapping and adjusted-close bars.
|
||||
- Price health: `GET /api/v1/prices/health`.
|
||||
- Ranked target weights and LONG/SHORT/NEUTRAL classification.
|
||||
- English dashboard with live/provisional source label, sign-aware surprise copy and lineage fields.
|
||||
- HttpOnly paper session and internal paper ledger.
|
||||
@@ -71,6 +73,9 @@ HTTP 200; count=1; manifest seen_count=4
|
||||
|
||||
GET /api/v1/backtest/tourism?min_events=12
|
||||
HTTP 409; status=blocked, available_events=1, required_events=12, price_series_required=true
|
||||
|
||||
GET /api/v1/prices/health
|
||||
HTTP 200; available=true, quality=revised_vendor_history, point_in_time=false, symbols=9
|
||||
```
|
||||
|
||||
Paper writes use a server-side token exchange and HttpOnly `paper_session` cookie; the token is not embedded in the frontend bundle.
|
||||
@@ -82,9 +87,9 @@ Paper writes use a server-side token exchange and HttpOnly `paper_session` cooki
|
||||
- Snapshot storage is local filesystem and single-process; shared persistence is required before multi-worker deployment.
|
||||
- No investment edge, transaction-cost result, or backtest conclusion has been established.
|
||||
- One independent source release is not enough for a valid event study; current historical rows are not treated as point-in-time vintages.
|
||||
- The event-study engine is deterministic and tested, but no real price provider is connected yet.
|
||||
- The event-study engine is deterministic and tested. Yahoo price history is connected for plumbing, but it is revised vendor history, not point-in-time data.
|
||||
- Browser screenshot verification remains blocked by the Chrome remote-debugging permission prompt; served HTML/source, live API, fresh Vite build and replay integrity were verified instead.
|
||||
|
||||
## Exact next action
|
||||
|
||||
Collect independent BOT releases over time and add a point-in-time daily price provider. Only then raise the readiness gate and run the event study. Add another metric only when its historical release coverage is real.
|
||||
Collect independent BOT releases over time and replace/supplement revised vendor history with a point-in-time daily price source. Only then raise the readiness gate and run the event study. Add another metric only when its historical release coverage is real.
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
| M1 BOT Tourism adapter | complete | 18 tests, live BOT fetch, raw/snapshot persistence | validate multiple vintages |
|
||||
| M2 vintage collector | complete | 25 tests, manifest idempotency, live collector and point-in-time API | collect independent releases |
|
||||
| M2.3 event-study gate | complete/blocked | 30 tests, pure engine and truthful 409 readiness API | add point-in-time price provider |
|
||||
| M2.4 price snapshot adapter | complete/blocked | 38 tests, live 9-symbol Yahoo snapshot, revised-history gate | evaluate point-in-time price source |
|
||||
| Tourism deterministic signal | complete | live foreign-arrivals YoY surprise | add occupancy/airport metric |
|
||||
| Internal paper ledger | complete | POST/readback through live API | persist in PostgreSQL later |
|
||||
| Dashboard | complete | Vite build + served source check with live-sign copy | visual browser capture after permission is available |
|
||||
@@ -25,7 +26,7 @@
|
||||
|
||||
## Verification
|
||||
|
||||
- Backend: 30 unittest tests pass.
|
||||
- Backend: 38 unittest tests pass.
|
||||
- Independent M1 review: **PASSED**; no concrete security or logic blockers.
|
||||
- Reviewer suggestions: set `PAPER_COOKIE_SECURE=1` outside local HTTP; replace in-memory sessions before multi-worker deployment.
|
||||
- M1 reviewer backlog: add schema-drift, duplicate/reordered-row, and malformed-vintage regression fixtures.
|
||||
@@ -40,4 +41,7 @@
|
||||
- Independent M2 review: **PASSED**; no concrete security or logic blockers.
|
||||
- Event-study readiness gate correctly returns HTTP 409 with 1/12 independent vintages; no backtest result is fabricated.
|
||||
- Independent M2.3 review: **PASSED**; no concrete security or logic blockers.
|
||||
- Price snapshot normalized 9 symbols with adjusted close and provider-derived trading dates; quality is explicitly `revised_vendor_history` and `point_in_time=false`.
|
||||
- Backtest gate requires both independent vintages and point-in-time prices.
|
||||
- Independent M2.4 review: **PASSED**; no concrete security or logic blockers.
|
||||
- Browser visual capture was blocked by Chrome remote-debugging permission; no permission dialog was clicked.
|
||||
|
||||
72
docs/engineering-log/2026-08-23-price-snapshot.md
Normal file
72
docs/engineering-log/2026-08-23-price-snapshot.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# 2026-08-23 — SET price snapshot adapter
|
||||
|
||||
## Plan status
|
||||
|
||||
- SET symbol mapping: complete.
|
||||
- Daily OHLCV/adjusted-close snapshot contract: complete.
|
||||
- Yahoo Chart API provider: complete and live-tested.
|
||||
- Trading-date extraction from provider bars: complete.
|
||||
- Point-in-time price quality: intentionally blocked; Yahoo history is labelled revised vendor history.
|
||||
|
||||
## Changed files
|
||||
|
||||
- `backend/app/prices.py` — Yahoo Chart API client, SET symbol map, payload normalization, adjusted-close handling, trading dates, raw/snapshot store and quality metadata.
|
||||
- `backend/scripts/collect_prices.py` — one-shot daily price collector.
|
||||
- `backend/app/__init__.py` — price store wiring, `/api/v1/prices/health`, and backtest gate price-quality check.
|
||||
- `backend/tests/test_prices.py` — mapping, normalized bars, adjusted prices, incomplete-row handling, storage/hash tests.
|
||||
- `backend/tests/test_api.py` — missing/revised price gate tests.
|
||||
- `frontend/src/App.vue` — Backtest gate shows price quality beside vintage count.
|
||||
- `README.md` — price collector and revised-vendor-history boundary.
|
||||
|
||||
## Live source evidence
|
||||
|
||||
Provider: Yahoo Finance Chart API, used as a research plumbing source only.
|
||||
|
||||
```text
|
||||
snapshot_id: prices-yahoo-chart-2024-01-01-2026-08-24-a8b5c7abae7c
|
||||
source_id: yahoo.chart
|
||||
period_start: 2024-01-01
|
||||
period_end: 2026-08-24
|
||||
symbols: 9 (AOT, AWC, BEM, CPALL, CPN, CRC, MINT, PTT, SET50)
|
||||
quality: revised_vendor_history
|
||||
point_in_time: false
|
||||
```
|
||||
|
||||
The provider returned 645 bars for each exposure symbol and 614 bars for the SET50 index series after incomplete rows were skipped. The snapshot is available to inspect, but it cannot unlock a real backtest because its historical values were retrieved/revised as a current vendor response.
|
||||
|
||||
## Readiness behavior
|
||||
|
||||
With the live price snapshot and one BOT vintage:
|
||||
|
||||
```text
|
||||
GET /api/v1/prices/health
|
||||
HTTP 200; available=true, quality=revised_vendor_history, point_in_time=false
|
||||
|
||||
GET /api/v1/backtest/tourism?min_events=12
|
||||
HTTP 409; status=blocked, reason=insufficient_vintages
|
||||
```
|
||||
|
||||
If the vintage threshold were met while prices remained non-point-in-time, the gate would still return `reason=price_series_not_point_in_time`.
|
||||
|
||||
## Verification
|
||||
|
||||
- `PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v` — **38 tests passed**.
|
||||
- `npm run build` — passed.
|
||||
- `npm audit --omit=dev --audit-level=high` — 0 vulnerabilities.
|
||||
- Yahoo live fetch — 9 symbols normalized and persisted.
|
||||
- API price health and readiness gate — passed.
|
||||
- Served Vue source contains the Backtest gate and price-quality copy.
|
||||
|
||||
## Independent review
|
||||
|
||||
```text
|
||||
passed: true
|
||||
security_concerns: []
|
||||
logic_errors: []
|
||||
```
|
||||
|
||||
Non-blocking backlog: add explicit revised/non-point-in-time gate coverage and document adjusted-close semantics for downstream calculations.
|
||||
|
||||
## Risks and exact next action
|
||||
|
||||
Yahoo historical data is not a point-in-time price source. Do not interpret the snapshot as a valid historical execution series. Next, evaluate a price source with release/availability timestamps or explicitly maintain a separate research-only revised-history mode.
|
||||
47
docs/test-evidence/2026-08-23-price-snapshot.md
Normal file
47
docs/test-evidence/2026-08-23-price-snapshot.md
Normal file
@@ -0,0 +1,47 @@
|
||||
# Test evidence — 2026-08-23 SET price snapshot
|
||||
|
||||
## Automated
|
||||
|
||||
```text
|
||||
PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v
|
||||
Ran 38 tests ... OK
|
||||
|
||||
npm run build
|
||||
Vite build completed successfully.
|
||||
|
||||
npm audit --omit=dev --audit-level=high
|
||||
found 0 vulnerabilities
|
||||
```
|
||||
|
||||
## Live Yahoo snapshot
|
||||
|
||||
```text
|
||||
snapshot_id: prices-yahoo-chart-2024-01-01-2026-08-24-a8b5c7abae7c
|
||||
symbols: 9
|
||||
quality: revised_vendor_history
|
||||
point_in_time: false
|
||||
```
|
||||
|
||||
## Live API
|
||||
|
||||
```text
|
||||
GET /api/v1/prices/health
|
||||
HTTP 200; available=true
|
||||
|
||||
GET /api/v1/backtest/tourism?min_events=12
|
||||
HTTP 409; blocked because only 1 independent BOT vintage exists
|
||||
```
|
||||
|
||||
## Frontend served verification
|
||||
|
||||
After rebuild/restart, the served Vue source contains the `Backtest gate` KPI and price-quality value path. Screenshot capture remains unavailable because the Chrome remote-debugging permission prompt has not been approved.
|
||||
|
||||
## Independent review
|
||||
|
||||
```text
|
||||
passed: true
|
||||
security_concerns: []
|
||||
logic_errors: []
|
||||
```
|
||||
|
||||
Non-blocking backlog: add explicit revised/non-point-in-time gate coverage and document adjusted-close semantics for downstream calculations.
|
||||
Reference in New Issue
Block a user