kunthawat/MeanRevisionEA
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(oi): update scraper for new QuikStrike website structure

Browse Source 
- Replace direct product URL navigation with fixed heatmap URL and UI product selection
- Implement cookie validation with automatic session cleanup
- Update login flow to use SSO authentication and new form selectors
- Improve data extraction with iframe context and better table parsing
- Add multiple fallback selectors for gold price scraping
- Enhance error handling, logging, and timeout management
This commit is contained in:
Kunthawat Greethong
2026-01-06 12:16:53 +07:00
parent 28a4546cd8
commit 2e8e07ed17
5 changed files with 1411 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
oi_scraper/README.md
View File

@@ -92,6 +92,7 @@ FuturePrice,4345.50
Edit `.env` to customize:
- `PRODUCT_URL` - QuikStrike product page URL (requires login)
- `CME_LOGIN_URL` - CME login page URL (default: SSO URL)
- `TOP_N_STRIKES` - Number of top strikes to export (default: 3)
- `HEADLESS` - Run browser in headless mode (default: false for debugging)
- `CSV_OUTPUT_PATH` - Output CSV file path