1e75377732
- auto_npl.py: parse BOT Gross NPLs by business (reportID=794); extract auto loan NPL (20,602 mn THB, 3.95% of NPLs, 2.06% of loans) - /api/v1/themes now exposes auto_npl_pct + auto_npl_amount alongside car-sales volume - 3 new tests; full suite 188 OK; live verified (themes shows auto_npl_pct 3.95)
92 lines
6.7 KiB
Markdown
92 lines
6.7 KiB
Markdown
# Alternative Factor Data Source Research — Auto Credit + Refining/Energy
|
||
|
||
> Date: 2026-08-25
|
||
> Goal: Identify real Thai national data sources for the themes before building collectors. Per user: research sources first, build with `har-derived-api-client`.
|
||
|
||
## Scope (confirmed 2026-08-25)
|
||
- **NATIONAL (ภาพรวมประเทศไทย) only** — NO province/region-level scraping. DLT by-province work is **cancelled**.
|
||
- **All alternative factors must be Thailand-specific** (Thai auto sales, Thai tourism, Thai consumption, Thai GRM) — NOT US/global proxies.
|
||
|
||
## Energy factor — Thai GRM (replaces EIA)
|
||
The US EIA 3-2-1 crack spread is a **US** proxy; the user wants **Thai** refinery margin instead (traded companies are Thai).
|
||
**Note:** Thai refinery GRM (TOP/PTT) is published **quarterly** (analyst meetings / results / Krungsri), not daily. Candidate Thai sources:
|
||
- **Krungsri Research** — Thailand refinery GRM average (e.g. THB 3.8/liter Q1, projected 2.0–3.0), aggregate national macro factor.
|
||
- **TOP (Thai Oil)** analyst-meeting PDFs (`top.listedcompany.com`) — GRM/GIM per quarter, $/bbl.
|
||
- **EPPO / MOPH** — Thai retail fuel prices (more frequent, but that's fuel price, not GRM).
|
||
|
||
**Decision (final):** Use **Thai Oil (TOP) quarterly financial highlights** (`investor.thaioilgroup.com/en/financial-information/financial-highlights`) — real quarterly EBITDA/Net Profit/Sales of the largest Thai refinery (Million Baht), scraped from server-rendered HTML. Implemented in `backend/app/energy_thai.py`. This is Thai-specific and replaces both the US EIA crack spread and the Krungsri outlook projection. Frequency: **quarterly**. (Krungsri Research remains a qualitative backdrop; EIA is a US proxy — both superseded by TOP for the factor.)
|
||
|
||
## Auto NPL (credit-quality) — added 2026-08-25
|
||
BOT Gross NPLs by business type (`ReportPage.aspx?reportID=794`, FI_NP_003_S2). Extracts **auto-loan NPL** (20,602 mn THB, **3.95% of NPLs**, 2.06% of loans, Q2/2568). Deepens the auto_credit theme beyond new-car sales with credit quality. Implemented in `backend/app/auto_npl.py`; merged into the auto_credit theme read.
|
||
|
||
|
||
|
||
|
||
## Method
|
||
|
||
For each candidate I browsed the live site, checked whether the data arrives as (a) a JSON/XHR API (ideal — derive a client with `har-derived-api-client`) or (b) server-rendered HTML (scrape the `<td>`/chart markup, same pattern as Siamchart). I confirmed the data is real and visible, not a fixture.
|
||
|
||
---
|
||
|
||
## THEME: Refining / Energy ← strongest candidates
|
||
|
||
### 1. RBN Energy — 3-2-1 Crack Spread ✅ HAR-scrape-able (HTML)
|
||
- **URL:** `https://rbnenergy.com/market-data/3-2-1-crack-spread`
|
||
- **Data:** 3-2-1 crack spread in $/bbl (e.g. **69.61, +3.35**), plus ethane/gas ratios, Brent-WTI, petchem margins. A direct refinery-margin indicator — exactly the "refining spread" factor.
|
||
- **Delivery:** server-rendered HTML — the value appears in the page text (`3-2-1 Crack Spread 69.61 3.35`). No JSON API (only analytics + a tracking pixel). → **scrape HTML** like Siamchart.
|
||
- **Notes:** US Gulf Coast indicator, not Thailand-specific, but it is the standard refining-margin proxy and is freely accessible without login.
|
||
|
||
### 2. Krungsri Research — Refinery Industry Outlook
|
||
- **URL:** `krungsri.com/en/research/industry/industry-outlook/.../refinery/`
|
||
- **Data:** Thailand refinery margin outlook (USD 5.0-6.0/bbl avg 2024), Thailand-specific.
|
||
- **Delivery:** PDF/article, not structured — **low** HAR value, better as a reference/commentary than a machine-read factor.
|
||
|
||
### 3. EIA — crack spread education/data
|
||
- Reference concept only; not a live Thai factor source.
|
||
|
||
---
|
||
|
||
## THEME: Auto Credit ← candidates need a decision
|
||
|
||
### 1. Trading Economics — Thailand Domestic Car Sales ⚠️ HTML, ad-heavy
|
||
- **URL:** `tradingeconomics.com/thailand/total-vehicle-sales`
|
||
- **Data:** monthly domestic car sales (e.g. +17.3% YoY in Jun 2026); recent-data table visible.
|
||
- **Delivery:** server-rendered HTML; HAR capture found **no JSON API** (only ads/analytics). The monthly series is in HTML/embedded chart data → scrape HTML, but the page is very ad-heavy and data granularity (monthly series extraction) needs care.
|
||
- **License/ToS:** Trading Economics data is **commercial/paid** for redistribution; scraping for internal research is grey-area. Flag: prefer a free primary source.
|
||
|
||
### 2. Federation of Thai Industries (FTI) — primary monthly vehicle data
|
||
- The **primary source** cited by news (Reuters/MarkLines rely on FTI). If we can find an FTI page or the monthly production/sales release, that is the correct, license-safe primary source for Thai vehicle data.
|
||
- **Status:** needs deeper search for a direct FTI/monthly-release URL (a `.or.th` or press page) that is HAR-scrapeable.
|
||
|
||
### 3. MarkLines — Thailand automotive production (monthly)
|
||
- **URL:** `marklines.com/en/statistics/flash_prod/automotive-production-in-thailand-by-month`
|
||
- **Data:** monthly production/sales figures with periodic updates.
|
||
- **Delivery:** likely JS chart + HTML; partially paywalled (some data for registered/logged-in). Needs verification for free access.
|
||
|
||
### 4. Auto loan growth / auto NPL (for the auto-loan component of the factor)
|
||
- Public series for Thai auto loan growth and auto NPL are **not freely available** from a single free API. Candidates:
|
||
- Bank of Thailand (BOT) statistical data — the correct primary source for loan/NPL series, but access is a downloadable portal (needs HAR inspection).
|
||
- Bank financial disclosures (annual) — not monthly.
|
||
- **Status:** most likely gap; the Auto Credit factor may need to lean on vehicle sales (FTI/Trading Economics/MarkLines) + a BOT loan-NPL series harvested via HAR.
|
||
|
||
---
|
||
|
||
## Recommendation (decision needed before building)
|
||
|
||
**Refining/Energy:** Use **RBN Energy 3-2-1 crack spread** (HTML scrape, free, real) as the factor input. Proxy for refinery margin.
|
||
|
||
**Auto Credit:** Two sub-parts:
|
||
- **Vehicle sales** — best free/primary: FTI monthly release (to confirm) OR Trading Economics (workable but commercial/TOS-grey + ad-heavy).
|
||
- **Auto loan / NPL growth** — BOT statistical portal, needs HAR inspection; if not accessible, the auto-credit factor can start with the vehicle-sales (sales surprise) component only and add loan/NPL later.
|
||
|
||
**Open items I did not resolve (would need your steer or a deeper dig):**
|
||
1. Confirm a free, HAR-scrapeable **FTI** monthly vehicle-sales page (vs Trading Economics).
|
||
2. Whether **auto loan / NPL** from BOT is reachable via HAR without login, or whether we accept a vehicle-sales-only auto factor for v1.
|
||
|
||
---
|
||
|
||
## How progress will proceed (per your instructions)
|
||
1. User picks the source per theme.
|
||
2. I capture HAR → derive/scrape script with `har-derived-api-client` (same as Siamchart).
|
||
3. Build collector + cache + theme scoring, then the dashboard/simulation.
|