3f7fccd25b
- energy_thai.py: scrape Thai Oil (TOP) investor financial-highlights -> quarterly + annual EBITDA/Net Profit/Sales (Million Baht), largest Thai refinery - Thai-specific factor per user (energy must reflect Thai companies, not US EIA proxy); Krungsri was projection-only, TOP gives real quarterly actuals - Frequency: quarterly (documented in research note) - 4 tests; full backend suite 156 OK; compileall ok; static scan clean
6.4 KiB
Alternative Factor Data Source Research — Auto Credit + Refining/Energy
Date: 2026-08-25 Goal: Identify real Thai national data sources for the themes before building collectors. Per user: research sources first, build with
har-derived-api-client.
Scope (confirmed 2026-08-25)
- NATIONAL (ภาพรวมประเทศไทย) only — NO province/region-level scraping. DLT by-province work is cancelled.
- All alternative factors must be Thailand-specific (Thai auto sales, Thai tourism, Thai consumption, Thai GRM) — NOT US/global proxies.
Energy factor — Thai GRM (replaces EIA)
The US EIA 3-2-1 crack spread is a US proxy; the user wants Thai refinery margin instead (traded companies are Thai). Note: Thai refinery GRM (TOP/PTT) is published quarterly (analyst meetings / results / Krungsri), not daily. Candidate Thai sources:
- Krungsri Research — Thailand refinery GRM average (e.g. THB 3.8/liter Q1, projected 2.0–3.0), aggregate national macro factor.
- TOP (Thai Oil) analyst-meeting PDFs (
top.listedcompany.com) — GRM/GIM per quarter, $/bbl. - EPPO / MOPH — Thai retail fuel prices (more frequent, but that's fuel price, not GRM).
Decision (final): Use Thai Oil (TOP) quarterly financial highlights (investor.thaioilgroup.com/en/financial-information/financial-highlights) — real quarterly EBITDA/Net Profit/Sales of the largest Thai refinery (Million Baht), scraped from server-rendered HTML. Implemented in backend/app/energy_thai.py. This is Thai-specific and replaces both the US EIA crack spread and the Krungsri outlook projection. Frequency: quarterly. (Krungsri Research remains a qualitative backdrop; EIA is a US proxy — both superseded by TOP for the factor.)
Method
For each candidate I browsed the live site, checked whether the data arrives as (a) a JSON/XHR API (ideal — derive a client with har-derived-api-client) or (b) server-rendered HTML (scrape the <td>/chart markup, same pattern as Siamchart). I confirmed the data is real and visible, not a fixture.
THEME: Refining / Energy ← strongest candidates
1. RBN Energy — 3-2-1 Crack Spread ✅ HAR-scrape-able (HTML)
- URL:
https://rbnenergy.com/market-data/3-2-1-crack-spread - Data: 3-2-1 crack spread in $/bbl (e.g. 69.61, +3.35), plus ethane/gas ratios, Brent-WTI, petchem margins. A direct refinery-margin indicator — exactly the "refining spread" factor.
- Delivery: server-rendered HTML — the value appears in the page text (
3-2-1 Crack Spread 69.61 3.35). No JSON API (only analytics + a tracking pixel). → scrape HTML like Siamchart. - Notes: US Gulf Coast indicator, not Thailand-specific, but it is the standard refining-margin proxy and is freely accessible without login.
2. Krungsri Research — Refinery Industry Outlook
- URL:
krungsri.com/en/research/industry/industry-outlook/.../refinery/ - Data: Thailand refinery margin outlook (USD 5.0-6.0/bbl avg 2024), Thailand-specific.
- Delivery: PDF/article, not structured — low HAR value, better as a reference/commentary than a machine-read factor.
3. EIA — crack spread education/data
- Reference concept only; not a live Thai factor source.
THEME: Auto Credit ← candidates need a decision
1. Trading Economics — Thailand Domestic Car Sales ⚠️ HTML, ad-heavy
- URL:
tradingeconomics.com/thailand/total-vehicle-sales - Data: monthly domestic car sales (e.g. +17.3% YoY in Jun 2026); recent-data table visible.
- Delivery: server-rendered HTML; HAR capture found no JSON API (only ads/analytics). The monthly series is in HTML/embedded chart data → scrape HTML, but the page is very ad-heavy and data granularity (monthly series extraction) needs care.
- License/ToS: Trading Economics data is commercial/paid for redistribution; scraping for internal research is grey-area. Flag: prefer a free primary source.
2. Federation of Thai Industries (FTI) — primary monthly vehicle data
- The primary source cited by news (Reuters/MarkLines rely on FTI). If we can find an FTI page or the monthly production/sales release, that is the correct, license-safe primary source for Thai vehicle data.
- Status: needs deeper search for a direct FTI/monthly-release URL (a
.or.thor press page) that is HAR-scrapeable.
3. MarkLines — Thailand automotive production (monthly)
- URL:
marklines.com/en/statistics/flash_prod/automotive-production-in-thailand-by-month - Data: monthly production/sales figures with periodic updates.
- Delivery: likely JS chart + HTML; partially paywalled (some data for registered/logged-in). Needs verification for free access.
4. Auto loan growth / auto NPL (for the auto-loan component of the factor)
- Public series for Thai auto loan growth and auto NPL are not freely available from a single free API. Candidates:
- Bank of Thailand (BOT) statistical data — the correct primary source for loan/NPL series, but access is a downloadable portal (needs HAR inspection).
- Bank financial disclosures (annual) — not monthly.
- Status: most likely gap; the Auto Credit factor may need to lean on vehicle sales (FTI/Trading Economics/MarkLines) + a BOT loan-NPL series harvested via HAR.
Recommendation (decision needed before building)
Refining/Energy: Use RBN Energy 3-2-1 crack spread (HTML scrape, free, real) as the factor input. Proxy for refinery margin.
Auto Credit: Two sub-parts:
- Vehicle sales — best free/primary: FTI monthly release (to confirm) OR Trading Economics (workable but commercial/TOS-grey + ad-heavy).
- Auto loan / NPL growth — BOT statistical portal, needs HAR inspection; if not accessible, the auto-credit factor can start with the vehicle-sales (sales surprise) component only and add loan/NPL later.
Open items I did not resolve (would need your steer or a deeper dig):
- Confirm a free, HAR-scrapeable FTI monthly vehicle-sales page (vs Trading Economics).
- Whether auto loan / NPL from BOT is reachable via HAR without login, or whether we accept a vehicle-sales-only auto factor for v1.
How progress will proceed (per your instructions)
- User picks the source per theme.
- I capture HAR → derive/scrape script with
har-derived-api-client(same as Siamchart). - Build collector + cache + theme scoring, then the dashboard/simulation.