kunthawat/set50-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: pre-existing in-tree work (event-study/research/vintages/prices + migration script + integrity docs)

Browse Source 
Committing the prior uncommitted working-tree state that predates this session's
data-source work (was already modified/untracked at session start) so the tree
is clean before push. Includes: event-study + research report integrity/forward
observation work, prices tests, research hash migration script, and the
2026-08-23/24 engineering-log + test-evidence notes. Verified green as part of
the full 362-test suite.
This commit is contained in:
Kunthawat Greethong
2026-08-29 09:19:24 +07:00
parent dbb787c50a
commit ead9aeb25c
22 changed files with 2326 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
docs/engineering-log/2026-08-23-replay-report-integrity-remediation.md Normal file
View File

@@ -0,0 +1,69 @@
# 2026-08-23 — replay and research-report integrity remediation
## Plan status
- Canonical replay validation: complete.
- Research report content-hash binding: complete.
- Research manifest-entry hash binding: complete.
- Malformed report fail-closed handling: complete.
- VintageStore snapshot/manifest loader fail-closed hardening: complete.
- Semantic replay-shape validation: complete; non-list/non-object observations and exposures now fail closed through controlled `ValueError` handling.
- Independent review gate: no valid approval for the current tree. `deleg_c227c439` found that `persist()` did not validate unrelated manifest entries before writing a new report; this was fixed with a regression test. `deleg_cf1da27d` then timed out after 600 seconds without a schema-valid JSON verdict; it is inconclusive, not an approval. `deleg_feb40653` was dispatched against the pre-semantic-fix tree, reported truncated cached diffs, and is stale/non-approving. `deleg_87a0f686` is the current bounded post-fix exact-diff review; no schema-complete current-tree approval has been surfaced yet.
- Backtest gate: still blocked; this work does not create additional evidence.
## Changed files
- `backend/app/__init__.py` — replay and replayability now use `VintageStore.load_snapshot()` rather than reading normalized JSON directly.
- `backend/app/research.py` — report hashes, manifest-entry hashes, strict object validation, explicit resumable legacy migration, and normalized snapshot lineage fields.
- `backend/app/vintages.py` — malformed snapshot/manifest shape, encoding, and raw-payload I/O failures now map to `VintageStoreError`.
- `backend/app/tourism.py` — malformed observation/exposure containers and entries now raise controlled semantic validation errors.
- `backend/scripts/migrate_research_report_hashes.py` — explicit migration command for trusted pre-hash research stores.
- `backend/tests/test_api.py` — canonical-store replay setup, tampered/semantically invalid snapshots including malformed exposure entries, and replay storage-I/O regressions.
- `backend/tests/test_research.py` — report-file, manifest-hash, manifest-metadata, legacy migration, malformed-report, and persist-wide manifest-validation regressions.
- `backend/tests/test_vintages.py` — non-object, invalid-UTF-8, and raw-payload I/O regression coverage.
## Root cause
The replay endpoint bypassed the canonical vintage loader, so a modified normalized snapshot could be recomputed as if it were trusted. Persisted research reports also had no self-hash, and the manifest metadata was not bound to its recorded entry. A changed report or `generated_at` value could therefore enter a replay path without integrity detection.
## Fix
- Replay loads only through `VintageStore.load_snapshot()`, which verifies raw payload, normalized payload, manifest metadata, and identity.
- Reports persist `report_hash_algorithm=sha256-json-canonical-v1` and a canonical `report_hash`.
- Each manifest entry persists and verifies its own canonical `manifest_entry_hash`, binding report filename, report hash, status, reason, and generated timestamp.
- JSON values that are not report objects are rejected with `ResearchRunError` instead of causing an attribute error.
- Non-object manifests and malformed manifest entries are rejected by `load_manifest()`, `list_runs()`, and `latest()`.
- `list_runs()` verifies each report and manifest-entry hash before returning metadata.
- `persist()` validates every existing manifest entry before returning or writing a new report, and refuses malformed entries without leaving an orphan report.
- Direct report loads bind the manifest entry identity to the requested `run_id`.
- Invalid UTF-8 in persisted manifest/report files is converted to `ResearchRunError` instead of escaping as a decode exception.
- VintageStore rejects non-object snapshots, invalid UTF-8 manifests/snapshots, and raw-payload read failures with `VintageStoreError` instead of leaking attribute/decode/I/O exceptions.
- Research input lineage now includes normalized snapshot hashes and their hash algorithm.
- Replay computation rejects non-list/non-object observation/exposure shapes as controlled semantic failures, so the replay endpoint returns HTTP 422 rather than leaking `AttributeError`.
## Verification
```text
PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v
Ran 79 tests ... OK
PYTHONPATH=backend .venv/bin/python -m compileall -q backend/app backend/scripts backend/tests
npm run build
npm audit --omit=dev --audit-level=high
found 0 vulnerabilities
git diff --check
static scan of added lines: no findings
```
Runtime smoke:
```text
GET /api/v1/research/tourism/latest → HTTP 200; report_hash present
GET /api/v1/backtest/tourism?min_events=12 → HTTP 409; reason=insufficient_vintages
```
## Boundary and remaining work
These hashes detect local artifact corruption and accidental tampering; they are not signatures against a hostile operator who can rewrite the application, data, and runtime. The validated backtest remains blocked until at least 12 independent BOT releases and a point-in-time price archive exist.
The post-fix BOT collector run fetched the same release (`source_id=bot.ec_ei_028_s2`, `published_at=2026-07-31T14:30:00+07:00`), increasing observation count to 14 without increasing independent releases. The gate correctly remained `status=blocked`, `reason=insufficient_vintages`.

65
docs/engineering-log/2026-08-24-forward-price-observation-modes.md Normal file
View File

@@ -0,0 +1,65 @@
# M2.9 — Forward price observations and explicit research modes
## Plan status
- Status: **complete/blocked**
- Complete: the system can record forward observations, detect local revisions, and run an explicitly labelled exploratory descriptive study.
- Blocked: validated historical PIT backtest remains closed until a provider passes `pit-daily-v1` and BOT has at least 12 independent releases.
## Decision
The strict PIT requirement is a gate for validated historical backtests, not a launch blocker for exploratory or forward paper work. Revised vendor history remains usable only when the result is labelled `non_pit_descriptive_only`.
## Root cause addressed
The price store already preserved raw payloads and immutable normalized snapshots, but repeated retrievals had no durable observation history. The research runner also treated the PIT requirement as a universal gate, which prevented descriptive research from using an available revised history.
## Changes
- `backend/app/prices.py`
- Added an optional, backward-compatible manifest `observations` list.
- Records one observation per distinct retrieval identity with source/period scope, retrieval time, raw hash, prior snapshot ID/hash, revision status, and structured diff paths.
- Preserves the original raw and normalized snapshot on unchanged retrievals; an actual content change creates a new snapshot/vintage instead of overwriting the prior one.
- Tracks `first_seen_at`, `last_seen_at`, and `observation_count` on snapshot manifest entries.
- Binds observation IDs to their content and rejects tampered or unknown-snapshot observation records at load time.
- Keeps revised Yahoo history at `quality=revised_vendor_history` and `point_in_time=false`.
- `backend/app/research.py`
- Added explicit `mode=validated|exploratory`.
- `validated` remains fail-closed on missing PIT archive evidence and requires per-bar known-at validation.
- `exploratory` may proceed with revised history only as `result_scope=non_pit_descriptive_only`; it disables PIT known-at enforcement and emits limitations.
- Runtime observation timestamps are excluded from the research input fingerprint, so an unchanged feed reuses the same frozen research run.
- `backend/app/__init__.py`
- Price health now exposes observation count, last observation time, and revision status.
- Added `GET /api/v1/prices/observations` for the local audit trail.
- The research endpoint accepts the explicit mode and defaults exploratory requests to one event unless overridden; validated requests retain the 12-event default.
- `frontend/src/App.vue`
- The research action explicitly requests exploratory mode with `min_events=1`.
- UI copy and result metadata distinguish a descriptive study from a validated backtest.
- Tests cover unchanged retrievals, revision diffs, immutable prior snapshots, observation tampering, exploratory execution, validated regression behavior, API mode validation, and stable run reuse.
## Verification evidence
- `PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v`
- Baseline before this slice: 93 tests passed.
- First full run after the slice: 98 tests passed.
- Focused run after observation-integrity hardening: 67 tests passed.
- Final full run: 121 tests passed with `-W error`.
- `PYTHONPATH=backend .venv/bin/python -m compileall -q backend` passed.
- `npm run build` from `frontend/` passed.
- `npm audit --omit=dev --audit-level=high` reported 0 vulnerabilities.
- `git diff --check` passed.
- Static scan over the changed code found zero hardcoded-secret assignments, shell execution, `eval`/`exec`, `pickle`, or SQL-f-string patterns.
- Final bounded independent reviewer `deleg_e5407553`: schema-valid `passed=true`; `security_concerns=[]` and `logic_errors=[]`. Non-blocking gaps: concurrent persistence/atomic manifest-update coverage, same-raw normalized-content mismatch coverage, and invalid/missing/mismatched predecessor-reference coverage. Suggestions are deferred because the store remains single-process and validated PIT deployment is still blocked.
## Risks and limits
- Observation history proves only what this system observed after capture began. It does not prove provider release time, historical knowledge state, or absence of revisions before first capture.
- The local filesystem store remains single-process; concurrent-writer safety and shared database migration are deferred.
- No provider has passed the complete PIT evidence packet. Do not set `point_in_time=true` for revised vendor history.
- No live execution, webhook, broker, MT5, or external order path was added.
## Exact next action
Keep collecting independent BOT releases until 12 genuinely distinct `(source_id, published_at)` releases exist. In parallel, request one complete market-data provider evidence packet with SET coverage, provider release identity/time, explicit known-at semantics, correction/revision replay, immutable raw/archive/manifest evidence, and timezone/session rules. Only then implement or promote a PIT adapter.
NEVER include API keys, tokens, passwords, secrets, credentials, or connection strings in the summary — replace any that appear with [REDACTED].

30
docs/engineering-log/2026-08-24-integrity-review.md Normal file
View File

@@ -0,0 +1,30 @@
# Integrity Remediation Review — 2026-08-24
## Status
Complete for the staged integrity-remediation scope under the stated local single-user threat model. No commit or push was performed.
## Scope
The review covered canonical report-content hashing, manifest-entry identity/hash binding, replay through canonical loaders, manifest-wide validation before writes, raw/normalized snapshot binding, legacy migration boundaries, strict UTF-8/object/path/raw-hash validation, and tourism replay semantic validation.
## Independent Review Gate
- Original independent review `deleg_43348a3b` returned `passed=true` with empty `findings`, but its payload did not use the required `security_concerns` and `logic_errors` keys.
- Schema-correction review `deleg_a1c721d2` returned exactly:
`{"passed":true,"security_concerns":[],"logic_errors":[],"suggestions":[]}`
- The corrected payload was parsed and verified to contain exactly the four required keys, with `passed=true` and all arrays empty.
- Parallel review `deleg_e2d54c30` ended interrupted and contributes no evidence.
The corrected verdict closes the integrity-review gate for the stated threat model. It does not cover hostile machine owners, multi-user production deployment, live execution, or the separate point-in-time price-data readiness gate.
## Existing Verification Evidence
The parent verification already recorded 85 backend tests passing, compileall passing, Vite build passing, high-severity npm audit with zero vulnerabilities, and both working-tree and staged `git diff --check` passing. No new code was changed while processing this verdict.
## Remaining Work
- Revised vendor price history remains explicitly `point_in_time=false`.
- Independent BOT releases remain below the 12-event research threshold.
- PIT price archive design and evidence are still required before event-study/backtest readiness can be raised.
- No commit or push has been performed.

55
docs/engineering-log/2026-08-24-m29-final-review.md Normal file
View File

@@ -0,0 +1,55 @@
# M2.9 Final Independent Review — 2026-08-24
## Status
- Review gate: **passed** for the bounded M2.9 remediation scope under the declared local single-process threat model.
- Milestone: **complete/blocked** — exploratory and forward paper work is available; validated historical PIT research remains blocked.
- Commit/push: none performed.
## Independent verdict
Reviewer `deleg_e5407553` returned the required schema-valid verdict:
```json
{
"passed": true,
"security_concerns": [],
"logic_errors": [],
"test_gaps": [
"concurrent read-modify-write persistence and atomic manifest updates",
"same-raw-payload observation with differing normalized content",
"invalid, missing, or mismatched predecessor references"
],
"suggestions": [
"add process-level locking or transactional manifest updates before multi-worker deployment",
"add malformed-manifest, equal-time, lineage, and predecessor adversarial fixtures",
"add focused helper and frontend mapping tests"
]
}
```
The security and logic blocker arrays are empty. The listed gaps and suggestions are non-blocking for the current single-process local store and are deferred until the deployment scope expands or focused coverage is prioritized.
## Verification basis
- Backend: 121 tests passed with `-W error`.
- Backend compileall: passed.
- Frontend Vite build: passed.
- Production dependency audit: 0 vulnerabilities.
- `git diff --check`: passed.
- Static dangerous-pattern scan: no hardcoded-secret assignments, shell execution, `eval`/`exec`, `pickle`, or SQL-f-string matches.
- Live smoke: health and observations returned HTTP 200; validated backtest remained HTTP 409 and fail-closed.
## Remaining gates
- Revised vendor price history remains `point_in_time=false`.
- BOT has only 1/12 independent releases.
- No provider has supplied the evidence required by `pit-daily-v1`.
- Do not claim a validated historical backtest or alpha result.
- Add locking/transactional persistence before multi-worker or shared-filesystem deployment.
## Next action
Continue collecting genuinely independent BOT releases and request a complete PIT provider evidence packet. Keep exploratory output explicitly `status=descriptive_only` with `result_scope=non_pit_descriptive_only`.
NEVER include API keys, tokens, passwords, secrets, credentials, or connection strings in the summary — replace any that appear with [REDACTED].

46
docs/engineering-log/2026-08-24-m29-observation-integrity-followup.md Normal file
View File

@@ -0,0 +1,46 @@
# M2.9 Observation Integrity Follow-up — 2026-08-24
## Status
- Milestone: **complete/blocked** for the local single-process observation-integrity scope.
- Commit/push: none performed.
- Validated PIT research remains blocked; revised vendor history remains `point_in_time=false`.
## Scope
Closed three non-blocking local integrity gaps identified by the prior M2.9 review:
1. Reject a new snapshot that reuses the same raw payload hash while changing normalized content.
2. Reject malformed snapshot manifest entries instead of silently skipping them.
3. Harden observation predecessor chronology: observations cannot precede the manifest snapshot's immutable first capture, and equal-UTC duplicate observations for one snapshot are rejected.
The existing valid repeated-observation path remains supported: later observations of an unchanged immutable snapshot continue to append without creating a new snapshot or research run.
## TDD evidence
Each new behavior followed RED → GREEN:
- `test_price_store_rejects_same_raw_payload_with_changed_normalized_content` failed before the guard, then passed after the minimal raw/normalized consistency check.
- `test_price_store_rejects_malformed_snapshot_manifest_entry` failed before manifest-entry type validation, then passed after fail-closed validation.
- `test_price_store_rejects_observation_before_snapshot_first_capture` failed before first-capture temporal binding, then passed after binding to manifest `retrieved_at`.
- `test_price_store_rejects_duplicate_equal_time_observations_for_one_snapshot` failed before equal-time ambiguity detection, then passed after strict predecessor ordering.
## Verification
- Price tests: 37 passed.
- Full backend suite: 125 passed with `PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v`.
- Backend compileall: passed.
- Frontend `npm run build`: passed.
- `npm audit --omit=dev --audit-level=high`: `found 0 vulnerabilities`.
- `git diff --check`: passed.
- Added-line static scan: no hardcoded-secret assignments, shell execution, `eval`/`exec`, unsafe pickle, or SQL-f-string matches.
- `ruff`, `mypy`, `eslint`, and `tsc` are not installed in this environment.
- Fresh focused independent review `deleg_0e2282c8` returned schema-valid `passed=true` with `security_concerns=[]` and `logic_errors=[]`. Its only suggestion is to retain regression coverage for later repeated observations and deterministic equal-time ordering across distinct snapshot IDs.
## Remaining work
- Process-level locking/transactional manifest updates remain deferred until multi-worker or shared-filesystem deployment is in scope.
- Focused helper/UI mapping coverage remains non-blocking follow-up work.
- Continue collecting independent BOT releases and obtain a provider evidence packet satisfying `pit-daily-v1` before enabling validated PIT research.
NEVER include API keys, tokens, passwords, secrets, credentials, or connection strings in the summary — replace any that appear with [REDACTED].

47
docs/engineering-log/2026-08-24-paper-auth-policy.md Normal file
View File

@@ -0,0 +1,47 @@
# Paper Auth Policy — 2026-08-24
## Status
Complete for the paper-auth scope. The separate staged integrity-remediation review remains open. No commit or push was performed.
## Goal
Make the paper-only write path explicit and safe for the local single-user demo while preserving fail-closed protected mode for shared/network deployments.
## Changes
- Added explicit `demo` and `token` paper-auth modes.
- Restricted demo mode to a loopback bind; unknown and non-loopback bind hosts fail closed.
- Passed the actual `HOST`/bind value from `backend/run.py` into the app factory.
- Kept protected token mode as the default and fail-closed when its configured credential is missing or malformed.
- Added truthful API/UI state for “configured but not yet authenticated” versus “paper writes disabled by server configuration.”
- Added regression coverage for demo writes, non-loopback rejection, unknown bind rejection, missing credentials, malformed credential configuration, and pre-session enabled state.
- Updated README and handoff documentation with the two operating modes.
## Review Finding and Root-Cause Fix
The first independent review found that coercing a non-string configured credential to `str(...)` could turn malformed configuration into an accepted value. It also found that GET auth status conflated an unauthenticated browser session with a disabled server configuration. The fix validates configured credentials without coercion, reports an explicit `enabled` state, and prevents the UI from presenting a dead unlock/write path.
## Verification Evidence
- `PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v` — 85 tests passed.
- `PYTHONPATH=backend .venv/bin/python -m compileall -q backend` — passed.
- `npm run build` — passed.
- `npm audit --omit=dev --audit-level=high` — 0 vulnerabilities.
- `git diff --check` and `git diff --cached --check` — passed.
- Static diff scan for credential-shaped literals and dangerous patterns — no findings.
- Environment smoke checks — loopback demo allowed; unknown and non-loopback demo binds rejected.
- Fresh static-server verification on port 5175 — HTTP 200; served bundle contained the disabled-state copy and auth warning marker. Temporary server was stopped afterward.
- `ruff`, `mypy`, and `eslint` are unavailable in this environment; no project lint script is configured for them.
## Data Safety
The smoke test created only synthetic paper rows in the runtime ledger; those exact test rows were removed afterward. The ledger was verified to contain no remaining smoke-test entries. No credentials, tokens, secrets, or connection strings were recorded.
## Independent Review
Fresh bounded review returned schema-valid `passed=true` with empty security-concern and logic-error arrays. Non-blocking suggestions: add one request-token type regression, add frontend integration coverage for both auth states, and add a startup precedence regression for `HOST`/`PAPER_BIND_HOST`.
## Remaining Gate
The separate staged integrity-remediation review is now closed for this scope: schema correction `deleg_a1c721d2` returned the exact required four-key verdict with `passed=true` and empty security/logic-error arrays. Earlier schema-invalid, timeout, and interrupted responses remain non-approving historical artifacts.

52
docs/engineering-log/2026-08-24-pit-price-contract.md Normal file
View File

@@ -0,0 +1,52 @@
# PIT Daily Price Archive Contract — 2026-08-24
## Status
- Implementation: complete for the local research/paper vertical slice.
- Provider evidence: not established yet.
- Backtest/event-study readiness: blocked until a real archive satisfies this contract and the BOT source reaches the independent-vintage threshold.
- Existing revised vendor history remains `point_in_time=false`.
## Contract
`point_in_time=true` is accepted only when all of the following are present and bound into the immutable price manifest:
- `quality=point_in_time_archive`.
- `archive_contract=pit-daily-v1`.
- Non-empty `provider_release_id`, `source_url`, `parser_version`, and timezone-aware `retrieved_at`.
- `point_in_time_evidence` with `known_at_field=known_at`, `known_at_semantics=provider_release_time`, a matching provider release ID, and timezone-aware `release_published_at`.
- Every series has a matching `canonical_symbol`, an IANA `timezone`, and ordered unique bars.
- Every bar has `session_date`, timezone-aware `known_at`, OHLC, `adjusted_close`, and `volume` values. `known_at` must not be after the session date in the market timezone or after retrieval.
- Snapshot raw-payload hashes, normalized hash, manifest identity, and contract metadata must agree on reload.
The generic Yahoo collector does not produce this evidence and continues to persist `revised_vendor_history` with `point_in_time=false`.
## No-lookahead behavior
The event-study runner has an explicit `require_price_known_at=True` mode. The research runner enables it for PIT snapshots. It rejects missing timestamps and rejects any price whose `known_at` falls on a later market-local session date, including UTC timestamps that cross the market-timezone boundary.
## Changed files
- `backend/app/prices.py` — PIT contract validation, per-bar known-at checks, timezone validation, manifest binding.
- `backend/app/event_study.py` — known-at-aware price joining and market-timezone no-lookahead checks.
- `backend/app/research.py` — carries session date, known-at, and market timezone into the event-study join; enables the PIT check.
- `backend/app/__init__.py` — exposes contract metadata in price health and keeps backtest readiness fail-closed.
- `backend/tests/test_prices.py`, `backend/tests/test_event_study.py`, `backend/tests/test_research.py` — RED/GREEN regression coverage for missing contract evidence, missing known-at, manifest binding, revised-history preservation, and timezone lookahead.
## Verification
- `PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v` — **93 tests passed**.
- `PYTHONPATH=backend .venv/bin/python -m compileall -q backend` — passed.
- `npm run build` — passed.
- `npm audit --omit=dev --audit-level=high` — 0 vulnerabilities.
- `git diff HEAD --check` and `git diff --cached --check` — passed.
- Static credential-pattern scan over `git diff HEAD` — 0 findings.
- Final bounded independent review `deleg_c08f6dd8` — exact schema returned `passed=true` with empty security-concern, logic-error, and suggestion arrays. Two earlier reviewer attempts timed out and were not counted.
- App smoke check: `/api/v1/prices/health` returned HTTP 200 for revised history; `/api/v1/backtest/tourism?min_events=1` returned HTTP 409 with `blocked` / `price_series_not_point_in_time`.
## Remaining work
- Obtain an actual provider archive/release contract and evidence for PIT semantics; do not flip the live Yahoo flag by configuration alone.
- Collect at least 12 independent BOT releases; repeated observations of one release do not count.
- Only after both gates pass should a frozen event-study/backtest result be treated as available. No alpha or investment claim is established here.
- No commit or push was performed in this session.

85
docs/engineering-log/2026-08-24-price-provider-feasibility.md Normal file
View File

@@ -0,0 +1,85 @@
# Price Provider Feasibility — 2026-08-24
## Decision
**No candidate passes `pit-daily-v1` from public evidence currently available.**
The repository must not add a provider adapter that emits `point_in_time=true` yet. Existing Yahoo/revised vendor history remains `quality=revised_vendor_history` and `point_in_time=false`.
This is a source-evidence decision, not a statement that the candidates cannot provide the required semantics under a commercial contract.
## Gate used
A provider must be able to supply an evidence-bearing archive with all of the following:
- stable `provider_release_id` for each release;
- `release_published_at` or equivalent provider release time;
- explicit `known_at` semantics that can be mapped to the provider release time;
- daily bars with session date, OHLC, adjusted close, volume, and market timezone;
- raw payload and normalized snapshot retained immutably;
- a manifest binding release identity, period, symbols, raw hash, normalized hash, parser version, and source URL;
- correction/revision behavior documented well enough to replay what was knowable at the decision time;
- evidence that the SET symbols and SET50 benchmark requested by this project are covered.
The local implementation enforces this as `archive_contract=pit-daily-v1`; the implementation itself is not provider evidence.
## Candidate matrix
| Candidate | Publicly verified capability | PIT evidence gap | Decision |
|---|---|---|---|
| SET Historical Data Request | SET offers historical trading/statistics data as a one-time service, with data sent by email after payment. [1] | No public release ID per historical vintage, release publication time, known-at semantics, immutable archive contract, or correction replay semantics. | **Reject for PIT claim**. Useful as a historical data content source only until SET supplies a release/archive contract. |
| SETSMART | SETSMART contains real-time and historical trading information for securities, derivatives, and indices. [2] | Public page does not establish timestamped historical snapshots, release identity, known-at semantics, or immutable archive replay. | **Not sufficient**. Do not build a PIT adapter from the web UI description alone. |
| SMART Marketplace | SET describes an API platform with historical intraday data and equity EOD trading price/statistics. [3] | Public page does not establish per-release identity, release publication time, historical corrections/revisions, or immutable PIT archive semantics. | **Promising but unproven**. Request the API/archive data contract before implementation. |
| ICE SET data | ICE advertises historical and end-of-day SET data, native exchange timestamps plus ICE UTC system timestamps, and an ICE historical archive. [4] [7] | The public pages do not prove that the SET daily archive exposes provider release IDs, release publication times, or point-in-time correction/replay semantics. Coverage and fields for this project also require confirmation. | **Best commercial candidate; conditional**. Ask ICE for a concrete sample archive and contract. |
| LSEG Tick History / S3 Direct | LSEG explicitly advertises full historical market data and point-in-time updates through S3 Direct, with persistent access in the customer's S3 environment. [5] [6] | Public material still does not prove that SET is covered for the requested symbols or expose the release/update metadata needed to map each daily bar to `known_at=provider_release_time`. The point-in-time claim is strong, but the SET-specific evidence packet is missing. | **Strongest PIT candidate; not yet accepted**. Request SET venue coverage, sample files, release/update metadata, and revision replay evidence before writing an adapter. |
| Databento Historical / Corporate Actions | Databento's venue page lists the Stock Exchange of Thailand, and its market-data pages describe aggregates through day-level bars, point-in-time instrument definitions, and nanosecond event timestamps. [8] [10] [11] | Databento's explicit PIT/replay contract is for corporate-action records: records are retained by `ts_record` and described as known at points in time. [9] The public market-data material does not establish price-bar release IDs, release publication times, client availability/`known_at`, or correction-vintage replay for SET prices. | **Reject for PIT price claim**. Retain as a possible SET coverage/reference-data lead; do not build a price adapter or promote its event timestamps to `known_at`. |
| Exchange Data International (EDI) | EDI advertises daily EOD pricing across 170+ exchanges, OHLCV-style closing data, API/S3/SFTP delivery, and Thailand Stock Exchange coverage identified as `XBKK`; its July 2025 ETA sheet lists confirmed Thailand equity delivery at 14:30 GMT. [12] [14] | EDI documents unconfirmed-to-confirmed republish and correction files in `Repush1/2/3`, but its FAQ says there is currently no time-series database for EOD prices. [12] [13] Public material does not establish stable provider release IDs, actual release-published timestamps per file, immutable retention, or retrievable vintage manifests. | **Closest conditional candidate; not accepted**. Request a customer evidence packet before writing an adapter. |
## Required provider evidence packet
Before writing a production adapter, obtain one complete historical sample covering at least one SET symbol and the SET50 benchmark. The sample should include:
1. provider documentation defining `provider_release_id`, `release_published_at`, and the meaning of `known_at`;
2. the exact raw response/file as delivered by the provider;
3. the provider's release manifest or metadata file;
4. a correction/revision example showing whether a later revision changes a prior bar and how both vintages are retrieved;
5. the provider timezone and session-calendar definition;
6. a symbol mapping file for the requested SET instruments;
7. delivery metadata proving when the archive was available to the client, not only when the market session occurred;
8. licensing/retention terms that permit immutable local replay for internal research and paper testing.
The fixture is accepted only if the complete packet can be normalized into the existing contract and replayed after changing neither the raw payload nor its manifest.
## Implementation decision
No files under `backend/app/` or `backend/scripts/` were changed for this feasibility pass. This avoids creating a fake adapter around a source whose PIT semantics are still unverified.
The next implementation action is one of:
- obtain a **market-data provider evidence packet** from EDI first, because its public docs expose SET coverage and correction/re-push behavior; this means provider documentation plus a representative market-data sample, not customer/vendor business data; or
- build the smallest ICE adapter if ICE supplies a valid evidence packet; or
- build the smallest LSEG adapter if LSEG supplies a valid evidence packet; or
- keep the gate blocked and document the provider rejection if neither does.
Databento was checked as an additional candidate. Its public documentation is useful evidence for SET venue presence and PIT corporate-action history, but it does not close the PIT **price-vintage** gate. No Databento price adapter is justified from the public material currently available. [8] [9] [11]
EDI is currently the closest public price-feed lead: its correction/re-push workflow and confirmed Thailand ETA are relevant to vintage capture, but an ETA is not an actual per-release publication timestamp and the public contract still lacks the immutable release metadata required by `pit-daily-v1`. Treat EDI as a request-for-evidence path, not as a qualified provider. [12] [13] [14]
In parallel, continue collecting BOT **independent** releases. Repeated observations of one published release still do not count toward the 12-release threshold.
## Sources
[1] https://www.set.or.th/en/services/connectivity-and-data/data/historical
[2] https://www.set.or.th/en/services/connectivity-and-data/data/web-based
[3] https://www.set.or.th/en/services/connectivity-and-data/data/smart-marketplace
[4] https://developer.ice.com/fixed-income-data-services/catalog/stock-exchange-thailand-set
[5] https://www.lseg.com/en/data-analytics/market-data/data-feeds/tick-history
[6] https://www.lseg.com/en/data-analytics/market-data/data-feeds/tick-history/s3-direct
[7] https://www.ice.com/fixed-income-data-services/access-and-delivery/connectivity-and-feeds/ice-consolidated-history
[8] https://databento.com/venues
[9] https://databento.com/docs/venues-and-datasets/corporate-actions
[10] https://databento.com/historical
[11] https://databento.com/equities
[12] https://www.exchange-data.com/product/end-day-pricing-data
[13] https://www.exchange-data.com/faqs-end-of-day-prices
[14] https://www.exchange-data.com/wp-content/uploads/2025/09/ETA-Coverage-17-July-2025.xls

60
docs/test-evidence/2026-08-23-replay-report-integrity-remediation.md Normal file
View File

@@ -0,0 +1,60 @@
# Test evidence — 2026-08-23 replay and research-report integrity remediation
## Automated
```text
PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v
Ran 79 tests ... OK
PYTHONPATH=backend .venv/bin/python -m compileall -q backend/app backend/scripts backend/tests
npm run build
Vite build completed successfully.
npm audit --omit=dev --audit-level=high
found 0 vulnerabilities
git diff --check
static scan of added lines: no findings
```
Ruff, mypy, and ESLint were not installed/configured in this checkout and were skipped without masking an exit code.
## Regression coverage
- Replay reads through `VintageStore.load_snapshot()`.
- Tampered normalized vintage snapshots return HTTP 422 from the replay endpoint.
- Research report content tampering is rejected by the report hash.
- Research manifest report-hash tampering is rejected.
- Research manifest metadata tampering is rejected by the manifest-entry hash.
- Non-object report JSON is rejected as invalid rather than raising an uncaught attribute error.
- Non-object manifests and malformed manifest entries are rejected by `load_manifest()`, `list_runs()`, and `latest()`.
- `list_runs()` verifies each report and manifest-entry hash before returning metadata.
- `persist()` validates every existing manifest entry before returning or writing a new report, and refuses malformed entries without leaving an orphan report.
- Source replayability uses the same verified load-and-compute helper as the replay endpoint, so semantically invalid snapshots are marked unreplayable.
- Tourism replay computation rejects non-list and non-object observation/exposure shapes as controlled semantic validation failures instead of leaking `AttributeError`.
- Replay storage I/O failures return HTTP 422 instead of leaking an uncontrolled 500.
- Explicit legacy report migration validates identity/metadata, is resumable after an interrupted report write, and is not automatic.
- Direct report loads bind the manifest entry identity to the requested `run_id`.
- Invalid UTF-8 in persisted manifest/report files is converted to `ResearchRunError` instead of escaping as a decode exception.
- VintageStore rejects non-object snapshots with `VintageStoreError`.
- Invalid UTF-8 in vintage manifests/snapshots is converted to `VintageStoreError`.
- Raw vintage payload read failures are converted to `VintageStoreError`.
- Research lineage includes normalized snapshot hashes and algorithms.
## Runtime smoke
```text
GET /api/v1/research/tourism/latest → HTTP 200; report_hash present
GET /api/v1/backtest/tourism?min_events=12 → HTTP 409; reason=insufficient_vintages
BOT collector rerun → same published release, `seen_count=14`, independent release count remains 1, gate remains blocked.
```
## Independent review
No valid approval for the current tree. `deleg_c227c439` found a persist-wide manifest-validation gap in the prior staged tree; the gap is now covered by a regression test and fixed before writing new reports. `deleg_cf1da27d` timed out after 600 seconds with no schema-valid JSON verdict, so it is inconclusive and does not approve the tree. `deleg_feb40653` was dispatched against the pre-semantic-fix tree, reported truncated cached diffs, and is stale/non-approving. `deleg_87a0f686` is the current bounded post-fix exact-diff review; no schema-complete current-tree approval has been surfaced yet.
## Scope boundary
This remediation improves local artifact integrity only. It does not unlock the backtest: the current evidence remains below 12 independent releases and price data remains `point_in_time=false`.

67
docs/test-evidence/2026-08-24-forward-price-observation-modes.md Normal file
View File

@@ -0,0 +1,67 @@
# Test Evidence — Forward Price Observations and Research Modes
- Date: 2026-08-24
- Scope: M2.9 forward price observation archive, exploratory research mode, validated PIT regression
- Branch: `main`
- Commit: pending; no commit or push performed
## Commands and results
```text
PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v
Ran 125 tests in 0.351s
OK
PYTHONPATH=backend .venv/bin/python -m compileall -q backend
exit 0
(cd frontend && npm run build)
Vite production build completed successfully
(cd frontend && npm audit --omit=dev --audit-level=high)
found 0 vulnerabilities
git diff --check
exit 0
```
Static scan over the changed code reported zero matches for:
```text
secret_assignment=0
shell=0
eval=0
pickle=0
sql_fstring=0
```
## Focused behavior verified
- Identical provider payloads reuse the immutable snapshot and append an `unchanged` observation.
- Changed provider payloads create a new snapshot and append a `revised` observation with prior raw hash and structured diff paths.
- Original raw/snapshot files remain readable after a revision.
- Tampered observation audit entries fail closed on manifest load.
- Contract-backed snapshot entries reject missing observation references, count/timing metadata tampering, and out-of-order/equal-time predecessor insertion without creating artifacts.
- Non-PIT normalized series require canonical identity, strictly increasing sessions, and provider-symbol/raw-payload lineage.
- Legacy pre-observation price manifests remain readable, while new contract-backed entries require complete manifest metadata cross-checks.
- Observation time changes do not create a new research run when the price snapshot content is unchanged.
- Same raw payload bytes with changed normalized content across a new snapshot ID are rejected.
- Malformed snapshot manifest entries are rejected instead of silently skipped.
- Observation timestamps before a snapshot's immutable first capture are rejected.
- Duplicate equal-UTC observations for one snapshot are rejected while later repeated observations remain valid.
- Explicit exploratory mode can run on revised history only with `status=descriptive_only`, `result_scope=non_pit_descriptive_only`, and limitations.
- Validated mode still rejects non-PIT price history and requires the existing PIT/known-at contract.
- API exposes `/api/v1/prices/observations`; invalid research modes return HTTP 400.
## Independent review status
The final bounded independent reviewer `deleg_e5407553` returned a schema-valid `passed=true` verdict with `security_concerns=[]` and `logic_errors=[]`. This follow-up closes the same-raw, malformed-manifest, first-capture timing, and equal-time duplicate gaps with RED/GREEN regressions. Fresh focused independent review `deleg_0e2282c8` returned schema-valid `passed=true` with empty blocker arrays; its only suggestion is to retain regression coverage for later repeated observations and deterministic equal-time ordering across distinct snapshot IDs. Process-level locking/transactional persistence and direct helper/UI tests remain deferred. No commit or push has been performed.
## Known limits
- Cache/diff evidence begins at first capture and does not prove provider release time or historical knowledge state before capture.
- The live revised Yahoo history remains `quality=revised_vendor_history` and `point_in_time=false`.
- No validated historical event study/backtest or alpha claim is enabled.
- Local filesystem writes remain single-process; concurrent-writer and shared database verification are deferred.
NEVER include API keys, tokens, passwords, secrets, credentials, or connection strings in the summary — replace any that appear with [REDACTED].

78
docs/test-evidence/2026-08-24-pit-price-contract.md Normal file
View File

@@ -0,0 +1,78 @@
# Test Evidence — PIT Daily Price Contract — 2026-08-24
## Scope
PIT daily-price archive validation and no-lookahead event-study joining. This evidence covers the implementation contract only; it does not claim that the live revised vendor source is PIT.
## Commands
```text
PYTHONPATH=backend .venv/bin/python -W error -m unittest discover -s backend/tests -v
Ran 93 tests in 0.214s — OK
PYTHONPATH=backend .venv/bin/python -m compileall -q backend
passed
cd frontend && npm run build
exit 0
cd frontend && npm audit --omit=dev --audit-level=high
found 0 vulnerabilities
git diff HEAD --check && git diff --cached --check
passed
```
## Focused evidence
- A `point_in_time=true` snapshot without `pit-daily-v1` and provider-release evidence is rejected.
- A PIT bar without `known_at` is rejected.
- A valid PIT archive persists and reloads only when raw hash, normalized hash, manifest identity, release metadata, and contract metadata agree.
- Existing snapshot IDs are immutable; changed raw or normalized content cannot overwrite an already manifested archive.
- Malformed manifests, snapshots, and raw payload symbols fail as controlled `PriceSourceError` values before persistence.
- Revised vendor history remains accepted only as `quality=revised_vendor_history` and `point_in_time=false`.
- Event-study PIT mode rejects missing `known_at` and a timestamp after the session date.
- Event-study PIT mode checks the session date in the declared market timezone, not only the UTC calendar date.
- The research runner enables the known-at check when the PIT gate is active.
## Live app check
```text
GET /api/v1/prices/health
HTTP 200; current source is revised vendor history, point_in_time=false.
GET /api/v1/backtest/tourism?min_events=1
HTTP 409; status=blocked; reason=price_series_not_point_in_time.
```
- Final bounded independent review `deleg_c08f6dd8` returned `passed=true` with empty `security_concerns`, `logic_errors`, and `suggestions` arrays.
## Provider evidence gate
- Databento public material confirms SET venue presence and point-in-time corporate-action records, but does not establish point-in-time SET price-bar release IDs, client availability/`known_at`, or correction-vintage replay. [8][9][11]
- Exchange Data International (EDI) public material confirms SET/XBKK equity coverage, a July 2025 confirmed ETA of 14:30 GMT, and correction/re-push behavior. Its public contract still does not establish actual per-file release timestamps, immutable retention, release IDs, or retrievable vintage manifests; its FAQ also states that it has no EOD time-series database. [12][13][14]
- ICE and LSEG remain conditional commercial candidates; no candidate passed `pit-daily-v1`, so no provider adapter was added and the live API remains fail-closed. [4][5][6]
- The broader matrix retains public references for SET Historical Data, SETSMART, and SMART Marketplace; none supplies the missing PIT release/vintage contract. [1][2][3]
- ICE SET and Databento historical timestamp materials remain non-qualifying without SET price-vintage evidence. [7][10]
- Full feasibility evidence and citations: `docs/engineering-log/2026-08-24-price-provider-feasibility.md`.
## Remaining gate
A real archive provider and release-time evidence are still required. The contract implementation must not be treated as evidence that any current vendor history is PIT. BOT remains at one independent release out of the required twelve.
## Sources
[1] https://www.set.or.th/en/services/connectivity-and-data/data/historical
[2] https://www.set.or.th/en/services/connectivity-and-data/data/web-based
[3] https://www.set.or.th/en/services/connectivity-and-data/data/smart-marketplace
[4] https://developer.ice.com/fixed-income-data-services/catalog/stock-exchange-thailand-set
[5] https://www.lseg.com/en/data-analytics/market-data/data-feeds/tick-history
[6] https://www.lseg.com/en/data-analytics/market-data/data-feeds/tick-history/s3-direct
[7] https://www.ice.com/fixed-income-data-services/access-and-delivery/connectivity-and-feeds/ice-consolidated-history
[8] https://databento.com/venues
[9] https://databento.com/docs/venues-and-datasets/corporate-actions
[10] https://databento.com/historical
[11] https://databento.com/equities
[12] https://www.exchange-data.com/product/end-day-pricing-data
[13] https://www.exchange-data.com/faqs-end-of-day-prices
[14] https://www.exchange-data.com/wp-content/uploads/2025/09/ETA-Coverage-17-July-2025.xls