set50-system/README.md

# SET50 Alternative Data Platform

Tourism-first vertical slice for a deterministic SET50 alternative-data research system. The app can run against a clearly-labelled fixture or fetch a point-in-time Tourism Indicators vintage from the Bank of Thailand report backed by the Ministry of Tourism and Sports.

Current scope:

```text
fixture or BOT source observation
  → Tourism Pulse surprise
  → versioned exposure score
  → ranked target weights
  → English dashboard
  → internal paper ledger
```

No external webhook receiver and no live MT5 execution are enabled.

## Run the backend

```bash
python -m venv .venv
.venv/bin/pip install -r backend/requirements.txt
PAPER_AUTH_MODE=demo HOST=127.0.0.1 TOURISM_SOURCE=fixture PYTHONPATH=backend .venv/bin/python backend/run.py
```

Health check:

```bash
curl http://127.0.0.1:5000/api/v1/health
```

## Run the dashboard

In a second terminal:

```bash
cd frontend
npm install
npm run dev -- --host 127.0.0.1
```

Open `http://127.0.0.1:5173`.

The frontend reads the live API through Vite's `/api` proxy. The local command above uses explicit loopback-only `PAPER_AUTH_MODE=demo`: paper writes do not require a session token, and the dashboard shows a demo warning. This mode is for a single local operator only; it must not be bound to a network interface or used for live execution. For shared or network access, use `PAPER_AUTH_MODE=token`, set `PAPER_WRITE_TOKEN` in the server environment, and require the HttpOnly browser session; the token is never embedded in the frontend bundle. The paper-entry action records an assumed fill in the local paper ledger only; it never sends an order.

For HTTPS/non-local deployment, use protected token mode and set `PAPER_COOKIE_SECURE=1`. Paper entries persist atomically under `backend/data/paper/ledger.json` by default. Sessions remain in-memory and single-process; use a shared session store before running multiple workers or replicas.

## Run with the real BOT Tourism source

Use the BOT-backed adapter when network access is available:

```bash
PAPER_AUTH_MODE=demo HOST=127.0.0.1 TOURISM_SOURCE=bot PYTHONPATH=backend .venv/bin/python backend/run.py
```

For a shared/network BOT deployment, replace `PAPER_AUTH_MODE=demo` with `PAPER_AUTH_MODE=token` and provide `PAPER_WRITE_TOKEN` through the server environment. The backend rejects demo mode when `HOST`/`PAPER_BIND_HOST` is not loopback.

At startup the adapter performs a read-only GET/POST against the BOT Tourism Indicators report, parses the available monthly history, computes the latest year-over-year arrival observation against a trailing 12-point baseline, and stores the raw HTML plus normalized snapshot under `backend/data/` (ignored by git). The dashboard labels provisional BOT data as `provisional`, not `high`.

Data-health and replay endpoints:

```text
GET /api/v1/data-health
GET /api/v1/vintages?as_of=<ISO-8601 timestamp>
GET /api/v1/replay/tourism?vintage_id=<vintage_id>
GET /api/v1/backtest/tourism?min_events=12
GET /api/v1/prices/health
POST /api/v1/research/tourism/run
GET /api/v1/research/tourism/latest
```

Source: `https://app.bot.or.th/BTWS_STAT/statistics/ReportPage.aspx?reportID=875&language=eng`

## Collect a vintage manually

The collector is intentionally one-shot and idempotent. Run it after a source update; it preserves raw bytes, normalized snapshots, and a manifest under `backend/data/`:

```bash
PYTHONPATH=backend .venv/bin/python backend/scripts/collect_tourism_vintage.py --root backend/data
```

Repeated collection of the same source hash keeps one `vintage_id` and increments `seen_count` without changing `first_seen_at`. A new hash for the same publication timestamp is recorded as a separate `revised` vintage.

## Tests and build

```bash
PYTHONPATH=backend .venv/bin/python -m unittest discover -s backend/tests -v
cd frontend && npm run build
```

## Current M2 boundary

- English UI and analysis vocabulary
- Research mode and paper mode only
- Tourism Pulse fixture adapter and BOT Tourism Indicators adapter
- Data lineage: source, publication time, retrieval time, vintage
- Raw response hash and normalized snapshot persistence
- Immutable vintage manifest with first-seen/revision metadata
- Read-only data-health, vintage timeline and vintage replay endpoints
- Yahoo-backed daily price snapshot contract with SET symbol mapping
- Deterministic event-study engine with benchmark and cost inputs
- Next-trading-session execution anchor by default; revisions are deduplicated by source and publication timestamp
- Backtest readiness gate that blocks without independent vintages and prices
- Frozen, replayable Tourism research-run reports with immutable input manifests
- Durable local paper ledger across backend restarts
- Deterministic surprise × exposure × confidence score
- Paper ledger endpoint
- No LLM call yet; the deterministic result is the source of truth
- No webhook receiver yet
- No MT5 bridge yet

The PIT archive contract is implemented, but a real provider archive and release-time evidence are still required. Continue collecting independent BOT releases and do not promote revised vendor history by configuration alone.

The event-study gate is now exposed through `/api/v1/backtest/tourism`. It returns HTTP `409` with `status=blocked` when the independent-vintage minimum is not met, and it explicitly reports that a point-in-time daily price series is still required. The pure engine accepts events, daily prices, benchmark prices, event windows, and cost assumptions; it does not fetch or invent market prices.

The accepted PIT contract is `pit-daily-v1`. It requires explicit provider release/evidence metadata, an IANA market timezone per series, timezone-aware per-bar `session_date`/`known_at`/OHLC/adjusted-close/volume, and immutable raw/normalized/manifest hash binding. PIT event-study mode rejects missing or market-locally future-known prices. The current Yahoo collector intentionally remains `quality=revised_vendor_history` and `point_in_time=false`.

## Run a frozen research check

The research runner persists either a deterministic event-study result or a blocked report. A blocked report is still useful: it records the exact vintage IDs, hashes, price snapshot, configuration, and gate reason that prevented the study.

From the CLI:

```bash
PYTHONPATH=backend .venv/bin/python backend/scripts/run_tourism_research.py \
  --data-root backend/data \
  --min-events 12 \
  --windows 1 3 5 20 \
  --cost-bps 20 \
  --execution-lag-sessions 1
```

From the API:

```text
POST /api/v1/research/tourism/run
GET  /api/v1/research/tourism/latest
```

The default live state intentionally returns `status=blocked`: the current archive has one independent BOT release and the Yahoo price snapshot is marked `point_in_time=false`. The runner never converts that data into a backtest by inference.

Research reports written before integrity hashes were introduced are intentionally not auto-migrated. Migrate a trusted local research store explicitly before loading it:

```bash
PYTHONPATH=backend .venv/bin/python backend/scripts/migrate_research_report_hashes.py \
  --root backend/data/research
```

The migration validates report identity and manifest metadata, writes the canonical report and manifest-entry hashes, and fails closed on malformed or partially migrated entries.

## Collect daily price snapshots

The initial research price provider uses Yahoo Finance Chart API with SET ticker mappings. It stores OHLCV plus adjusted close for the eight exposure names and `^SET.BK` as `SET50`. This is **revised vendor history**, not point-in-time market data, so the snapshot is visible for data plumbing but cannot unlock the backtest gate:

```bash
PYTHONPATH=backend .venv/bin/python backend/scripts/collect_prices.py \
  --root backend/data/prices \
  --start 2024-01-01 \
  --end 2026-08-24
```

Inspect the latest price snapshot:

```text
GET /api/v1/prices/health
```