[verified] add BOT tourism source adapter

This commit is contained in:
Kunthawat Greethong
2026-08-23 10:19:40 +07:00
parent 3e978c5948
commit a113a51589
15 changed files with 800 additions and 38 deletions

View File

@@ -3,45 +3,71 @@
## Project
- Path: `/Users/kunthawat/Gitea/set50-alternative-data-platform`
- Branch: `main`
- Current milestone: M1 complete and independently reviewed
- Mode: research + paper only
- Frontend: Vue 3 + Vite
- Backend: Flask
- Current data: deterministic fixture with lineage metadata
- Backend: Flask `0.2.0`
- Current runtime source: BOT Tourism Indicators (`TOURISM_SOURCE=bot`)
## Completed
- Tourism snapshot schema with source, publication, retrieval and vintage fields.
- Tourism snapshot schema with source, publication, retrieval, vintage and raw hash fields.
- Deterministic Tourism Pulse score: standardized surprise × exposure × confidence.
- BOT ASP.NET form adapter with monthly history parsing and trailing 12-point YoY baseline.
- Atomic raw HTML and normalized snapshot persistence under ignored `backend/data/`.
- Read-only `/api/v1/data-health` endpoint.
- Safe `/api/v1/replay/tourism?vintage_id=...` endpoint with identity and path validation.
- Ranked target weights and LONG/SHORT/NEUTRAL classification.
- Flask endpoints for health, summary, observations, signals and paper ledger.
- English dashboard with live API data, lineage panel, signal table and paper-entry form.
- English dashboard with live/provisional source label, sign-aware surprise copy and lineage fields.
- HttpOnly paper session and internal paper ledger.
- No external webhook or MT5 integration.
## Verified commands
Independent M1 review passed with no concrete security or logic blockers. Non-blocking backlog: add schema-drift, duplicate/reordered-row, and malformed-vintage regression fixtures.
## Current live vintage
```text
PYTHONPATH=backend .venv/bin/python -m unittest discover -s backend/tests -v
Ran 10 tests ... OK
source_id: bot.ec_ei_028_s2
published_at: 2026-07-31T14:30:00+07:00
as_of: 2026-06-30
status: provisional
available_periods: 138
history_points: 12
vintage_id: bot.ec_ei_028_s2-2026-07-31-665981f88b4a
raw_payload_hash: 665981f88b4a30c5bd30026cf1e96279c244ad83725558f4136d952b29756a31
theme_surprise: -0.35114754
```
Paper writes use a server-side token exchange and HttpOnly `paper_session` cookie; the token is not embedded in the frontend bundle.
## Verified commands and live checks
```text
PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v
Ran 18 tests ... OK
npm run build
Vite build completed successfully.
GET /api/v1/health
{"mode":"research","status":"ok","version":"0.1.0"}
HTTP 200; {"mode":"research","status":"ok","version":"0.2.0"}
POST /api/v1/paper/ledger + GET /api/v1/paper/ledger
PAPER_RECORDED and readback verified.
GET /api/v1/data-health
HTTP 200; source_mode=bot, status=provisional, replayable=true
Independent review
PASSED — no concrete security or logic blockers.
GET /api/v1/replay/tourism?vintage_id=bot.ec_ei_028_s2-2026-07-31-665981f88b4a
HTTP 200; replay theme surprise matched live summary exactly.
```
## Known limitation
Paper writes use a server-side token exchange and HttpOnly `paper_session` cookie; the token is not embedded in the frontend bundle.
The data adapter is a fixture. It demonstrates the contract and calculation, not live tourism data quality. Browser visual screenshot verification was blocked by a Chrome remote-debugging permission prompt; served HTML, API response and frontend build were verified instead.
## Known limitations
## Next action
- BOT data is provisional and may be revised. The vintage/hash contract preserves the fetched version, but this is not a final-data guarantee.
- Only foreign-arrival YoY is live in this slice; occupancy and airport passenger metrics are not yet connected.
- Snapshot storage is local filesystem and single-process; shared persistence is required before multi-worker deployment.
- No investment edge, transaction-cost result, or backtest conclusion has been established.
- Browser screenshot verification remains blocked by the Chrome remote-debugging permission prompt; served HTML/source, live API, fresh Vite build and replay integrity were verified instead.
Implement one replayable Tourism source adapter without changing the snapshot contract. Add parser fixture tests, publication timestamp handling, raw snapshot hash, and a data-health failure state before adding LLM analysis.
## Exact next action
Validate multiple BOT vintages and add the next real Tourism metric without changing the snapshot contract. Run event-study/backtest checks before adding LLM analysis.