feat(oi): add open interest scraper module

add new oi_scraper directory for collecting open interest data
and update the main EA to integrate with the scraper functionality

oi_scraper/README.md

@@ -0,0 +1,178 @@
+# CME OI Scraper
+
+Python scraper to pull Open Interest data from CME Group QuikStrike and current gold price from investing.com.
+
+## What It Extracts
+
+1. **OI Levels (from CME QuikStrike):**
+   - Top 3 CALL strikes by OI volume
+   - Top 3 PUT strikes by OI volume
+
+2. **Gold Price (from investing.com):**
+   - Current gold futures price (e.g., 4345.50)
+
+## Prerequisites
+
+- Python 3.9+
+- CME Group QuikStrike account with login credentials
+
+## Installation
+
+1. Copy environment variables:
+```bash
+cp .env.example .env
+```
+
+2. Edit `.env` and add your CME credentials:
+```bash
+CME_USERNAME=your_username
+CME_PASSWORD=your_password
+```
+
+3. Install dependencies:
+```bash
+pip install -r requirements.txt
+playwright install chromium
+```
+
+## Usage
+
+### Basic Scraping
+
+```bash
+python main.py
+```
+
+This will:
+- Login to CME QuikStrike
+- Navigate to OI Heatmap
+- Extract top 3 CALL and PUT strikes by OI volume
+- Scrape current gold price from investing.com
+- Export to `oi_data.csv`
+
+### Session Persistence
+
+The scraper automatically saves your login session to `cookies.json`. This means:
+
+- **First run**: Logs in with your credentials, saves cookies
+- **Subsequent runs**: Uses saved cookies if session is still valid
+- **Session expired**: Automatically logs in again and saves new cookies
+
+Benefits for scheduled runs:
+- Faster execution (skips login when session is valid)
+- Reduces login attempts to CME servers
+- CME sessions typically last several days/weeks
+
+To force a fresh login, delete `cookies.json`:
+```bash
+rm cookies.json
+```
+
+### Output Format
+
+The CSV output is compatible with the EA's `LoadOIFromCSV()` and `LoadFuturePriceFromCSV()` functions:
+
+```csv
+Type,Strike,OI
+CALL,4345,155398
+CALL,4350,229137
+CALL,4360,90649
+PUT,4300,227936
+PUT,4290,270135
+PUT,4280,65839
+
+[Price]
+FuturePrice,4345.50
+```
+
+**Note:** The `[Price]` section contains the current gold futures price scraped from investing.com. The EA reads this value for Delta calculation.
+
+## Configuration
+
+Edit `.env` to customize:
+
+- `PRODUCT_URL` - QuikStrike product page URL (requires login)
+- `TOP_N_STRIKES` - Number of top strikes to export (default: 3)
+- `HEADLESS` - Run browser in headless mode (default: false for debugging)
+- `CSV_OUTPUT_PATH` - Output CSV file path
+- `TIMEOUT_SECONDS` - Page load timeout
+
+### Available Products
+
+**Gold (XAUUSD/COMEX Gold - OG|GC):**
+```
+PRODUCT_URL=https://cmegroup.quikstrike.net/User/QuikStrikeView.aspx?pid=40&viewitemid=IntegratedOpenInterestTool
+```
+
+**Silver:**
+```
+PRODUCT_URL=https://cmegroup.quikstrike.net/User/QuikStrikeView.aspx?pid=41&viewitemid=IntegratedOpenInterestTool
+```
+
+**SOFR (3M SOFR):**
+```
+PRODUCT_URL=https://cmegroup.quikstrike.net/User/QuikStrikeView.aspx?pid=476&viewitemid=IntegratedOpenInterestTool
+```
+
+**Note:** You must be logged in to access QuikStrike data. The scraper will automatically login using credentials from `.env`.
+
+## Integration with EA
+
+The EA reads OI data from CSV when `InpOISource = OI_SOURCE_CSV_FILE`.
+
+Place the generated `oi_data.csv` in MetaTrader's `MQL5/Files` directory.
+
+## Scheduling
+
+Use cron or Windows Task Scheduler to run periodically:
+
+```bash
+# Run every hour
+0 * * * * cd /path/to/oi_scraper && python main.py
+```
+
+## Troubleshooting
+
+**Login fails:**
+- Verify credentials in `.env`
+- Check if CME requires 2FA
+- Set `HEADLESS=false` to see what's happening
+- Check screenshots: `login_failed.png`, `login_error.png`, `login_success.png`
+
+**No data extracted:**
+- Check if table structure changed
+- Increase `TIMEOUT_SECONDS`
+- Check logs for detailed errors
+- Screenshot saved as `login_debug.png` or `login_failed.png`
+
+**Login page selectors changed:**
+- If the scraper can't find username/password inputs, CME may have updated their login page
+- Update the selectors in `login_to_cme()` function in `main.py`:
+  ```python
+  # Example: update to match current CME login form
+  page.fill('input[id="username"]', CME_USERNAME)
+  page.fill('input[id="password"]', CME_PASSWORD)
+  page.click('button[type="submit"]')
+  ```
+
+**Browser issues:**
+- Install Chromium dependencies: `playwright install chromium`
+- Try different browser: Change `p.chromium.launch()` to `p.firefox.launch()`
+
+## Notes
+
+- The scraper targets the OI Heatmap table structure
+- Only exports top N strikes by OI volume
+- Login session is not persisted (login each run)
+- Cookies could be saved for faster subsequent runs
+
+### Finding Product IDs
+
+To find product IDs for other instruments:
+1. Visit https://www.cmegroup.com/tools-information/quikstrike/open-interest-heatmap.html
+2. Login to your CME account
+3. Select a product from the "Products" menu
+4. The URL will update with the `pid` parameter
+5. Copy that URL to your `.env` file
+
+Example: `https://www.cmegroup.com/tools-information/quikstrike/open-interest-heatmap.html?pid=40` (Gold)