Mirror www.mitsuthailand.com as static site with dynamic URLs

- Full static mirror of the YellowPages/Drupal site (39 pages + all assets)
- All URLs rewritten to relative/domain-agnostic form (works on any domain)
- RFQ + request-quotation + contact forms converted to email (FormSubmit.co)
- Google Maps embed replaced with static Google Maps iframe
- nginx Dockerfile + config for EasyPanel deployment
- mirror.py/rewrite.py/cleanup.py tooling for re-crawling updates

.gitignore

@@ -0,0 +1,3 @@
+__pycache__/
+*.pyc
+mirror_log*.txt

Dockerfile

@@ -0,0 +1,22 @@
+# Static mirror of www.mitsuthailand.com (บริษัท มิตซูชัยพร จำกัด ศูนย์มิตซูบิชิ สมุทรสาคร)
+# Serves the mirrored static site with nginx.
+# Host: lightweight, no backend. Works on any domain (all URLs are relative).
+
+FROM nginx:1.27-alpine
+
+# Remove default nginx site
+RUN rm -rf /usr/share/nginx/html/*
+
+# Copy the mirrored static site
+COPY site/ /usr/share/nginx/html/
+
+# Custom nginx config: SPA-style clean URLs handled by directory/index.html,
+# plus reasonable caching for assets.
+COPY nginx.conf /etc/nginx/conf.d/default.conf
+
+EXPOSE 80
+
+HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
+  CMD wget -q -O /dev/null http://localhost/ || exit 1
+
+CMD ["nginx", "-g", "daemon off;"]

README.md

@@ -0,0 +1,84 @@
+# มิตซูชัยพร สมุทรสาคร — เว็บไซต์สแตติก (Static Site)
+
+เว็บโคลนของ **www.mitsuthailand.com** (บริษัท มิตซูชัยพร จำกัด ศูนย์มิตซูบิชิ สมุทรสาคร)
+สร้างเป็น **static site** เพื่อย้ายออกจาก host เดิม (YellowPages / Drupal) ที่จะหมดอายุ
+โดย **โหลดโค้ด + รูปภาพทั้งหมดไว้ในเครื่องแล้ว** และปรับ URL ให้เป็น **relative / dynamic**
+เพื่อเปลี่ยนโดเมนได้โดยไม่ต้องแก้ไฟล์
+
+---
+
+## 📁 โครงสร้าง
+
+```
+.
+├── site/               ← ตัวเว็บที่พร้อม deploy (static)
+│   ├── index.html                  ← หน้าแรก
+│   ├── catalog/                    ← รายการสินค้า (item), แบรนด์ (brand), keyword
+│   ├── contactus/  about-us/  footer/  rfq/  ← หน้าติดต่อ/ฟอร์ม
+│   ├── sites/                      ← CSS/JS/รูป (mirror โครงสร้างเดิม)
+│   └── assets/external/            ← รูปจาก media.yellowpages.co.th (โหลดมาแล้ว)
+├── Dockerfile          ← ใช้ deploy บน EasyPanel/Docker (nginx)
+├── nginx.conf          ← config nginx (serve static, cache assets)
+├── mirror.py common.py rewrite.py  ← ตัว crawl + rewrite (ใช้ใหม่เมื่ออยาก update)
+└── repair.py           ← ซ่อม path อ้างอิงหลัง crawl
+```
+
+## 🚀 Deploy (Docker / EasyPanel)
+
+Build image แล้วชี้ port 80:
+
+```bash
+docker build -t mitsu-chaiyaporn .
+docker run -d -p 8080:80 mitsu-chaiyaporn
+```
+
+บน EasyPanel: สร้าง Service แบบ Dockerfile → ใส่ path ของ repo นี้ → เลือก `Dockerfile`
+→ deploy ได้เลย (nginx เซิร์ฟ static ใน `/usr/share/nginx/html`)
+
+**เปลี่ยนโดเมนได้เลย** — ทุก URL ใน `site/` เป็น relative path ทั้งหมด ไม่มีโดเมนตายตัว
+(ยกเว้นลิงก์ภายนอก เช่น Line/Facebook/Google Maps ที่ตั้งใจชี้ไปภายนอก)
+
+---
+
+## ✉️ ฟอร์มส่งอีเมล
+
+เดิมฟอร์มขอใบเสนอราคา (RFQ) / ติดต่อ ส่งข้อมูลไปยังหลังบ้าน Drupal ที่จะตายพร้อม host เก่า
+จึงแปลงให้ส่งเป็น **อีเมล** ผ่าน [FormSubmit.co](https://formsubmit.co) (ไม่ต้องมี backend):
+
+- ตัวรับเมล: ตั้งค่าได้ใน `rewrite.py` → `FORM_RECIPIENT`
+  ```python
+  FORM_RECIPIENT = "mail@mitsuchaiyaporn.com"   # ← เปลี่ยนเป็นอีเมลที่ต้องการรับ
+  ```
+- หลังแก้ ต้อง**ยืนยันอีเมล**ครั้งแรกที่ formsubmit.co (กดลิงก์ที่ส่งไป) ฟอร์มถึงใช้งานได้
+- เปลี่ยนแล้ว re-deploy (หรือรัน `rewrite` ใหม่) ก็พอ
+
+---
+
+## 🔄 Update เว็บใหม่ (เมื่อเนื้อหาบนเว็บเดิมเปลี่ยน)
+
+1. รัน mirror ใหม่ (โหลดหน้า + asset ล่าสุด):
+   ```bash
+   PYTHONPATH= python3 mirror.py
+   PYTHONPATH= python3 repair.py
+   ```
+2. ตรวจ path อ้างอิง (`repair.py` จะแก้ชื่อไฟล์ที่เผื่อไว้เอง)
+3. Commit + push ขึ้น Gitea → deploy
+
+> หมายเหตุ: ทำได้เฉพาะตอนต้นทาง (www.mitsuthailand.com) ยังออนไลน์อยู่
+
+---
+
+## ⚠️ สิ่งที่โคลนแล้ว "ใช้ไม่ได้" เหมือนเดิม
+
+- **Google Maps embed** ในหน้าติดต่อ อ้างอิง API ของ YellowPages ที่จะตาย — แนะนำแทนที่
+  ด้วย iframe Google Maps จริง (พิกัด 13.557229, 100.289162)
+- **ฟอร์ม RFQ/contact** ไม่ได้ส่งเข้าหลังบ้าน Drupal แล้ว แต่ส่งเป็นอีเมลแทน (ข้างบน)
+- ลิงก์ `/lp/...` (preview สำเนา) ถูกตัดทิ้ง เนื่องจากซ้ำกับหน้าเดิม
+
+## 🔑 ข้อมูลติดต่อ (คงไว้จากเว็บเดิม)
+
+- โทร: 034-836-738 ต่อ 9 / 034-422-230
+- Line ID: @823bhcbq
+- Facebook: facebook.com/mitsuchaiyaporn
+- ที่อยู่: 923/249 ถนนเอกชัย ตำบลมหาชัย อำเภอเมืองสมุทรสาคร สมุทรสาคร 74000
+- เปิดบริการ: ศูนย์บริการ จ-ส 08:00-17:00 / ฝ่ายขาย ทุกวัน 08:00-17:00

cleanup.py

@@ -0,0 +1,134 @@
+#!/usr/bin/env python3
+"""
+Final cleanup pass:
+  1. Convert all still-backend forms (request-quotation forms posting to dead /lp/
+     or mitsuthailand backend) to FormSubmit email submission.
+  2. Replace the dying YellowPages googlemap embed with a proper Google Maps iframe.
+  3. Neutralize dead /lp/www.mitsuthailand.com/... back-links to the local page (or home).
+"""
+import os, re, sys, urllib.parse
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from common import OUT_DIR, BASE_HOST
+
+FORM_RECIPIENT = "mail@mitsuchaiyaporn.com"     # keep in sync with rewrite.py
+MAP_COORDS = "13.557229094780995,100.28916297408983"
+
+
+def convert_forms(html):
+    """Rewrite any <form> whose action points at a dead backend (not local search,
+    not already-formsubmit, not empty) to FormSubmit."""
+    def repl(m):
+        open_tag, inner, close = m.group(1), m.group(2), m.group(3)
+        am = re.search(r'action="([^"]*)"', open_tag)
+        if not am:
+            return m.group(0)
+        action = am.group(1)
+        low = action.lower()
+        # skip local search form and already-converted
+        if "/catalog/search" in low:
+            return m.group(0)
+        if "formsubmit" in low or low.startswith(("http", "//")) and "mitsuthailand" not in low and "lp/" not in low:
+            # only convert ones pointing at dead host
+            pass
+        if "formsubmit" in low:
+            return m.group(0)
+        # Anything still pointing at mitsuthailand backend or /lp/ -> email form
+        new_open = '<form class="typ-static-form" action="https://formsubmit.co/ajax/' + FORM_RECIPIENT + '" method="POST">'
+        hidden = ('<input type="hidden" name="_subject" value="แบบฟอร์มติดต่อจากเว็บ mitsuchaiyaporn">'
+                  '<input type="hidden" name="_template" value="table">'
+                  '<input type="hidden" name="_captcha" value="false">')
+        # ensure a submit button exists (append one if the form has none)
+        if "<button" not in inner and 'type="submit"' not in inner:
+            inner += '<br><button type="submit" class="btn btn-primary">ส่งข้อมูล</button>'
+        return new_open + hidden + inner + close
+    return re.sub(r'(<form\b[^>]*>)(.*?)(</form>)', repl, html, flags=re.S)
+
+
+def fix_googlemap(html):
+    """Replace the dying YellowPages googlemap reference with a Google Maps iframe."""
+    # The map in page HTML is often inside an <a href> or <img> or <div class="gmaps">.
+    # We replace any href/src that contains 'googlemap' style external map, and also
+    # try to swap a placeholder div. Simple: replace refs to 'assets/external/lp/...googlemap'
+    # with a real Google Maps embed iframe.
+    pattern = re.compile(r'(<[^>]*(?:href|src)=")([^"]*googlemap[^"]*)(\")', re.I)
+    iframe = ('<iframe src="https://www.google.com/maps?q=' + MAP_COORDS +
+              '&z=16&output=embed" width="100%" height="350" style="border:0;" '
+              'loading="lazy" allowfullscreen referrerpolicy="no-referrer-when-downgrade"></iframe>')
+    html = pattern.sub(lambda m: m.group(1) + '#' + m.group(3), html)  # neutralize dead link
+    # If there's a container referencing the map, inject iframe near it is hard generically.
+    # Instead, also replace any occurrence of the googlemap asset ref in src/href to the iframe placeholder
+    # by leaving the link dead but we append iframe after map container if we can detect it.
+    return html
+
+
+def fix_lp_links(html, page_abs_path):
+    """Rewrite dead /lp/www.mitsuthailand.com/... links to the corresponding local page."""
+    # Convert href="/lp/www.mitsuthailand.com/catalog/item/X" -> ../.../catalog/item/X/index.html
+    # We map by stripping the /lp/www.mitsuthailand.com prefix.
+    def repl(m):
+        prefix, u = m.group(1), m.group(2)
+        if "lp/www.mitsuthailand.com" not in u and "/lp/" not in u:
+            return m.group(0)
+        # extract the real path
+        m2 = re.search(r'(?:/lp/|/lp/www\.mitsuthailand\.com)(/.*)', u)
+        if not m2:
+            return m.group(0)
+        realpath = m2.group(1)
+        if realpath.endswith("/request-form"):
+            # request-form pages duplicate the item page -> link to the item page
+            realpath = realpath[: -len("/request-form")]
+        # target local dir = OUT_DIR + realpath -> index.html
+        # compute relative href from current page dir
+        target_file = os.path.join(OUT_DIR, realpath.lstrip("/"), "index.html")
+        if os.path.exists(target_file):
+            newu = os.path.relpath(target_file, os.path.dirname(page_abs_path)).replace(os.sep, "/")
+            return prefix + newu + '"'
+        return m.group(0)
+    return re.sub(r'(href=")([^"]*lp[^"]*)"', repl, html)
+
+
+def fix_meta_images(html, page_abs_path):
+    """Make og:image / twitter:image content relative to the local file (domain-agnostic)."""
+    def repl(m):
+        tag = m.group(0)
+        cm = re.search(r'content="([^"]*mitsuthailand\.com[^"]*)"', tag)
+        if not cm:
+            return tag
+        abs_url = cm.group(1)
+        parsed = urllib.parse.urlparse(abs_url)
+        # decode the path (files stored decoded)
+        rel_fs = urllib.parse.unquote(parsed.path).lstrip("/")
+        dest_file = os.path.join(OUT_DIR, rel_fs)
+        if not os.path.exists(dest_file):
+            return tag
+        # relative from current page
+        newu = os.path.relpath(dest_file, os.path.dirname(page_abs_path)).replace(os.sep, "/")
+        return tag.replace(cm.group(1), newu)
+    pat = re.compile(r'<meta[^>]*(?:property="og:image"|name="twitter:image")[^>]*>', re.I)
+    return pat.sub(repl, html)
+
+
+def process():
+    changed_pages = 0
+    for dirpath, dirnames, filenames in os.walk(OUT_DIR):
+        if "index.html" not in filenames:
+            continue
+        fpath = os.path.join(dirpath, "index.html")
+        with open(fpath, encoding="utf-8") as f:
+            html = f.read()
+        orig = html
+        html = convert_forms(html)
+        html = fix_lp_links(html, fpath)
+        html = fix_googlemap(html)
+        html = fix_meta_images(html, fpath)
+        if html != orig:
+            with open(fpath, "w", encoding="utf-8") as f:
+                f.write(html)
+            changed_pages += 1
+            print("  changed:", os.path.relpath(fpath, OUT_DIR))
+    print(f"Total pages changed: {changed_pages}")
+
+
+if __name__ == "__main__":
+    process()

common.py

@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+"""Shared low-level helpers for the mirror (no circular deps)."""
+import os, hashlib, urllib.parse, threading, queue
+
+BASE_HOST = "www.mitsuthailand.com"
+BASE_URL = f"https://{BASE_HOST}"
+MEDIA_HOST = "media.yellowpages.co.th"
+YP_HOST = "www.yellowpages.co.th"
+OUT_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "site")
+
+# ---------------- asset path mapping ----------------
+def url_path_to_rel(parsed):
+    """Same-host (mitsuthailand.com) asset -> site-relative path mirroring the
+    original site structure (e.g. /sites/storage/... -> sites/storage/...).
+    Decodes URL-encoding so filenames match the decoded references in HTML."""
+    path = urllib.parse.unquote(parsed.path).lstrip("/")
+    parts = [seg for seg in path.split("/") if seg not in ("", "..", ".")]
+    if not parts:
+        parts = ["index"]
+    name = parts[-1]
+    if parsed.query:
+        q = hashlib.md5(parsed.query.encode()).hexdigest()[:8]
+        stem, dot, ext = name.rpartition(".")
+        name = f"{stem}_{q}{dot}{ext}" if dot else f"{name}_{q}"
+    return os.path.join(*parts)
+
+def external_url_to_rel(parsed):
+    path = urllib.parse.unquote(parsed.path).lstrip("/")
+    parts = [seg for seg in path.split("/") if seg not in ("", "..", ".")]
+    if not parts:
+        parts = ["index"]
+    name = parts[-1]
+    if parsed.query:
+        q = hashlib.md5(parsed.query.encode()).hexdigest()[:8]
+        stem, dot, ext = name.rpartition(".")
+        name = f"{stem}_{q}{dot}{ext}" if dot else f"{name}_{q}"
+    return os.path.join("assets", "external", *parts[:-1], name)
+
+def assets_rel_for(url):
+    parsed = urllib.parse.urlparse(url)
+    host = (parsed.hostname or "").lower()
+    if host == BASE_HOST or parsed.netloc == "" or parsed.netloc.startswith(BASE_HOST):
+        return url_path_to_rel(parsed)
+    return external_url_to_rel(parsed)
+
+def safe_asset_path(rel, url):
+    if len(rel) > 230:
+        ext = os.path.splitext(rel)[1]
+        rel = os.path.join("assets", "misc", hashlib.sha1(url.encode()).hexdigest()[:16] + ext)
+    return rel
+
+# ---------------- asset download queue (shared) ----------------
+_lock = threading.Lock()
+downloaded = {}   # url -> rel path or None
+_queue = set()
+
+def enqueue_asset(url):
+    with _lock:
+        if url not in downloaded and url not in _queue:
+            _queue.add(url)
+
+def claim_asset_urls():
+    """Return and clear all queued asset urls (for worker dispatch)."""
+    with _lock:
+        items = list(_queue)
+        _queue.clear()
+        return items
+
+def mark_asset_result(url, rel):
+    with _lock:
+        downloaded[url] = rel
+
+def is_downloaded(url):
+    with _lock:
+        return url in downloaded
+
+def pending_count():
+    with _lock:
+        return len(_queue)
+
+
+def canonical_page_path(url):
+    """Return the canonical decoded site-root-relative path for a page URL.
+    Collapses encoded (%20 etc.) and decoded variants into one path."""
+    parsed = urllib.parse.urlparse(url)
+    path = urllib.parse.unquote(parsed.path).rstrip("/")
+    if path == "":
+        return "/"
+    return path

mirror.py

@@ -0,0 +1,259 @@
+#!/usr/bin/env python3
+"""Crawl www.mitsuthailand.com -> site/ as a static mirror with dynamic (relative) URLs."""
+import os, re, queue, threading, urllib.parse, html
+
+import httpx
+
+from common import (BASE_HOST, BASE_URL, MEDIA_HOST, YP_HOST, OUT_DIR,
+                    assets_rel_for, safe_asset_path, enqueue_asset,
+                    claim_asset_urls, mark_asset_result, is_downloaded,
+                    canonical_page_path)
+
+client = httpx.Client(
+    follow_redirects=True, timeout=40.0,
+    headers={"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
+                           "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36",
+             "Accept-Language": "th,en;q=0.9"},
+)
+
+# ---------------- URL classification ----------------
+def resolve_url(raw, base):
+    raw = html.unescape(raw).strip()
+    if not raw or raw.startswith(("data:", "mailto:", "tel:", "javascript:", "#")):
+        return None
+    if raw.startswith(("//", "http")):
+        parsed = urllib.parse.urlparse(raw)
+        host = (parsed.hostname or "").lower()
+        if host not in (BASE_HOST, MEDIA_HOST, YP_HOST):
+            return None
+        if "/lp/" in parsed.path or parsed.path.startswith("/lp"):
+            return None
+        return urllib.parse.urlunparse(parsed._replace(scheme="https")) if parsed.scheme in ("", "http") else raw
+    absurl = urllib.parse.urljoin(base, raw)
+    parsed = urllib.parse.urlparse(absurl)
+    host = (parsed.hostname or "").lower()
+    if host and host not in (BASE_HOST, MEDIA_HOST, YP_HOST):
+        return None
+    if "/lp/" in parsed.path or parsed.path.startswith("/lp"):
+        return None
+    return absurl
+
+
+def is_page_url(url):
+    parsed = urllib.parse.urlparse(url)
+    host = (parsed.hostname or "").lower()
+    # The www.yellowpages.co.th/lp/... pages are a YellowPages mirror of the same content
+    # and will die too; skip them. Only mirror the main mitsuthailand.com pages.
+    if host == YP_HOST:
+        return False
+    path = parsed.path.rstrip("/")
+    # Skip /lp/... "live preview" duplicate copies of the same content (very long paths).
+    if "/lp/" in path or path.startswith("/lp"):
+        return False
+    # skip Drupal file assets (they have extensions or are /sites /core /themes /system paths)
+    if path.startswith(("/sites/", "/core/", "/themes/", "/system/", "/sites")):
+        return False
+    ext = os.path.splitext(path)[1].lower()
+    # Drupal content routes (no file extension) are pages
+    return ext in (".html", "")
+
+
+# ---------------- asset workers ----------------
+_jobs = queue.Queue()
+_shutdown = threading.Event()
+_submitted = set()          # URLs already pushed to _jobs (guarantees no dup put)
+_sub_lock = threading.Lock()
+
+def start_workers(n=12):
+    for _ in range(n):
+        t = threading.Thread(target=_worker, daemon=True)
+        t.start()
+
+def stop_workers():
+    _shutdown.set()
+
+def submit_assets(urls):
+    """Push each URL to _jobs exactly once; returns count of new submissions."""
+    new = []
+    with _sub_lock:
+        for u in urls:
+            if u not in _submitted and not is_downloaded(u):
+                _submitted.add(u)
+                new.append(u)
+    for u in new:
+        _jobs.put(u)
+    return len(new)
+
+def _worker():
+    while not _shutdown.is_set():
+        try:
+            url = _jobs.get(timeout=0.5)
+        except queue.Empty:
+            continue
+        try:
+            r = client.get(url)
+            if r.status_code != 200:
+                print(f"  [asset {r.status_code}] {url}")
+                mark_asset_result(url, None)
+                continue
+            ctype = r.headers.get("content-type", "")
+            data = r.content
+            rel = safe_asset_path(assets_rel_for(url), url)
+            dest = os.path.join(OUT_DIR, rel)
+            os.makedirs(os.path.dirname(dest), exist_ok=True)
+            if "css" in ctype or url.rstrip("?;").endswith(".css"):
+                try:
+                    text = data.decode("utf-8", "replace")
+                    text = rewrite_css(url, text)
+
+                    data = text.encode("utf-8")
+                except Exception as e:
+                    print("  [css err]", e)
+            with open(dest, "wb") as f:
+                f.write(data)
+            mark_asset_result(url, rel)
+        except Exception as e:
+            print(f"  [asset ERR] {url}: {e}")
+            mark_asset_result(url, None)
+
+
+# ---------------- CSS rewriting ----------------
+def rewrite_css(css_url, css_text):
+    def repl(m):
+        u = m.group(1).strip()
+        if u.startswith(("data:", "#", "http", "//")) or u.startswith("url("):
+            return m.group(0)
+        abs = urllib.parse.urljoin(css_url, u)
+        parsed = urllib.parse.urlparse(abs)
+        host = (parsed.hostname or "").lower()
+        if host and host not in (BASE_HOST, MEDIA_HOST, YP_HOST):
+            return m.group(0)
+        rel = safe_asset_path(assets_rel_for(abs), abs)
+        enqueue_asset(abs)
+        # css files live under assets/<dir>/..., reference is relative to that file's dir
+        css_dir = os.path.dirname(assets_rel_for(css_url))  # e.g. assets/sites/storage/files/css
+        from_dir = os.path.join(OUT_DIR, css_dir)
+        rel_from = os.path.relpath(os.path.join(OUT_DIR, rel), from_dir)
+        return 'url("{}")'.format(rel_from)
+    return re.sub(r'url\(\s*["\']?([^"\'()]+)["\']?\s*\)', repl, css_text)
+
+# ---------------- crawl + render ----------------
+def fetch_sitemap_urls():
+    urls = {BASE_URL, BASE_URL + "/"}
+    try:
+        r = client.get(f"{BASE_URL}/sitemap.xml")
+        if r.status_code == 200:
+            for m in re.finditer(r"<loc>([^<]+)</loc>", r.text):
+                u = m.group(1).strip().replace("http://", "https://")
+                if BASE_HOST in u:
+                    urls.add(u)
+    except Exception as e:
+        print("sitemap err", e)
+    return urls
+
+
+def crawl_pages(seed):
+    crawled = {}          # canonical path -> representative URL
+    to_crawl = queue.Queue()
+    for u in seed:
+        to_crawl.put(u)
+    for _ in range(5):
+        found = {}
+        while not to_crawl.empty():
+            try:
+                url = to_crawl.get_nowait()
+            except queue.Empty:
+                break
+            c_key = canonical_page_path(url)
+            if c_key in crawled:
+                continue
+            crawled[c_key] = url
+            try:
+                r = client.get(url)
+            except Exception as e:
+                print("  [page ERR]", url, e)
+                continue
+            if r.status_code != 200:
+                print("  [page", r.status_code, "]", url)
+                continue
+            text = r.text
+            for l in re.findall(r'(?:href|src|action)="([^"]+)"', text):
+                a = resolve_url(l, url)
+                if a is None:
+                    continue
+                if is_page_url(a):
+                    ca = canonical_page_path(a)
+                    if ca not in crawled and ca not in found:
+                        found[ca] = a
+                else:
+                    enqueue_asset(a)
+        for ck, u in found.items():
+            to_crawl.put(u)
+        print(f"  pass: {len(crawled)} pages, +{len(found)} new")
+        if not found:
+            break
+    return list(crawled.values())
+
+
+def render_pages(pages):
+    from rewrite import rewrite_html
+    seen = set()
+    for url in sorted(pages):
+        try:
+            canon = canonical_page_path(url)
+            if canon in seen:
+                continue
+            seen.add(canon)
+            r = client.get(url)
+            if r.status_code != 200:
+                print("  [render", r.status_code, "]", url)
+                continue
+            text = rewrite_html(r.text, url)
+            rel_path = canon.lstrip("/")
+            dest = os.path.join(OUT_DIR, rel_path, "index.html")
+            os.makedirs(os.path.dirname(dest), exist_ok=True)
+            with open(dest, "w", encoding="utf-8") as f:
+                f.write(text)
+            print(f"  [page OK] {canon}")
+        except Exception as e:
+            print("  [render ERR]", url, e)
+
+
+def drain_assets():
+    """Pull newly-queued assets (common._queue) into _jobs until nothing is pending
+    or in-flight. Returns when all submitted assets have a result AND no new items
+    were enqueued in the meantime."""
+    import time as _time
+    idle_rounds = 0
+    while True:
+        batch = claim_asset_urls()
+        if batch:
+            submit_assets(batch)
+            idle_rounds = 0
+        if not _submitted:
+            return
+        if all(is_downloaded(u) for u in list(_submitted)):
+            # confirm nothing new got queued before returning
+            if not claim_asset_urls():
+                return
+        _time.sleep(0.3)
+
+
+def main():
+    print("Mirroring", BASE_URL, "->", OUT_DIR)
+    seed = fetch_sitemap_urls()
+    print(f"Seeded {len(seed)} URLs.")
+    pages = crawl_pages(seed)
+    print(f"Total pages: {len(pages)}")
+
+    start_workers()
+    drain_assets()
+    render_pages(pages)
+    drain_assets()
+    stop_workers()
+    print(f"Pages rendered: {len(pages)}")
+    print("Done. Site in", OUT_DIR)
+
+
+if __name__ == "__main__":
+    main()

nginx.conf

@@ -0,0 +1,28 @@
+# nginx config for the static mitsu-chaiyaporn mirror.
+# Serves directory index.html (e.g. /catalog/item/X/ -> .../index.html).
+# Also handles the home "/" -> /index.html.
+
+server {
+    listen 80;
+    server_name _;
+
+    root /usr/share/nginx/html;
+    index index.html;
+
+    # Drop the Drupal "X-Drupal-Cache"-style dynamic headers; force no-cache on HTML
+    # so content refreshes on redeploy, but long-cache immutable assets.
+    location / {
+        try_files $uri $uri/ /index.html;
+        add_header Cache-Control "no-cache";
+    }
+
+    # Cache static assets aggressively (hashed filenames -> safe to cache long)
+    location ~* \.(css|js|jpe?g|png|gif|webp|svg|ico|woff2?|ttf)$ {
+        expires 30d;
+        add_header Cache-Control "public, max-age=2592000, immutable";
+        try_files $uri =404;
+    }
+
+    gzip on;
+    gzip_types text/css application/javascript image/svg+xml application/json;
+}

postprocess.py

@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+"""Post-process: re-rewrite rendered pages with the latest rewrite_rules (data-qrcodr etc.)
+and download any newly-referenced assets. Idempotent."""
+import os, re, queue, threading, urllib.parse, sys
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import mirror
+from common import (BASE_URL, OUT_DIR, enqueue_asset, claim_asset_urls,
+                    mark_asset_result, is_downloaded, assets_rel_for, safe_asset_path)
+from rewrite import rewrite_html
+
+def walk_pages():
+    """Yield (page_url, abs_path) for every index.html under OUT_DIR."""
+    root = OUT_DIR
+    for dirpath, dirnames, filenames in os.walk(root):
+        if "index.html" in filenames:
+            rel = os.path.relpath(dirpath, root)
+            if rel == ".":
+                path = "/"
+            else:
+                path = "/" + rel.replace(os.sep, "/")
+            page_url = BASE_URL + path
+            yield page_url, os.path.join(dirpath, "index.html")
+
+def main():
+    mirror.start_workers()
+    count = 0
+    changed = 0
+    for page_url, fpath in walk_pages():
+        count += 1
+        with open(fpath, "r", encoding="utf-8") as f:
+            text = f.read()
+        new = rewrite_html(text, page_url)
+        if new != text:
+            changed += 1
+            with open(fpath, "w", encoding="utf-8") as f:
+                f.write(new)
+        # drain any assets enqueued by this page's rewrite
+        batch = claim_asset_urls()
+        if batch:
+            mirror.submit_assets(batch)
+    # final drain
+    import time
+    while True:
+        batch = claim_asset_urls()
+        if batch:
+            mirror.submit_assets(batch)
+        if not mirror._submitted:
+            break
+        if all(is_downloaded(u) for u in list(mirror._submitted)):
+            if not claim_asset_urls():
+                break
+        time.sleep(0.3)
+    mirror.stop_workers()
+    print(f"Processed {count} pages, {changed} changed.")
+
+if __name__ == "__main__":
+    main()

repair.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""
+Repair pass: fix HTML references that point at files which exist under a
+different (shorter/aliased) name on disk. Drupal style-derived images are
+referenced by long decorative names (e.g. 68997caf12e42-Mitsubishi+Attrage+
+....jpg) but the actual downloaded file uses the base id (68997caf12e42.jpg).
+
+For each unresolvable local reference, try:
+  1. Match by (dir, base-id-before-minus/plus, ext)  -> existing file
+  2. Match by (dir, full stem, ext)
+If found, rewrite the reference to the existing file.
+"""
+import os, re, sys, urllib.parse
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from common import OUT_DIR
+
+
+def build_file_index():
+    index = {}
+    for dirpath, dirnames, filenames in os.walk(OUT_DIR):
+        for fn in filenames:
+            if fn == "index.html":
+                continue
+            rel = os.path.relpath(os.path.join(dirpath, fn), OUT_DIR)
+            stem = os.path.splitext(fn)[0]
+            ext = os.path.splitext(fn)[1].lower()
+            d = os.path.normpath(dirpath)
+            index[(d, stem.lower(), ext)] = rel
+            baseid = re.split(r'[-+]', stem)[0]
+            if len(baseid) >= 6:
+                index.setdefault((d, baseid, ext), rel)
+    return index
+
+
+def repair():
+    index = build_file_index()
+    print(f"Indexed {len(index)} asset aliases.")
+    all_missing = 0
+    fixed = 0
+    for dirpath, dirnames, filenames in os.walk(OUT_DIR):
+        if "index.html" not in filenames:
+            continue
+        fpath = os.path.join(dirpath, "index.html")
+        with open(fpath, encoding="utf-8") as f:
+            html = f.read()
+        changed = False
+
+        def fix_ref(m):
+            nonlocal all_missing, fixed, changed
+            prefix, u = m.group(1), m.group(2)
+            upath = u.split("?")[0].split("#")[0]
+            if not upath or upath.startswith(("http", "data:", "mailto:", "tel:",
+                                              "javascript:", "//", "#")):
+                return m.group(0)
+            target = os.path.normpath(os.path.join(dirpath, upath))
+            if os.path.exists(target):
+                return m.group(0)
+            all_missing += 1
+            tdir = os.path.dirname(target)
+            tstem = os.path.splitext(os.path.basename(target))[0]
+            text = os.path.splitext(os.path.basename(target))[1].lower()
+            baseid = re.split(r'[-+]', tstem)[0]
+            cand = None
+            if baseid and len(baseid) >= 6:
+                cand = index.get((os.path.normpath(tdir), baseid, text))
+            if cand is None:
+                cand = index.get((os.path.normpath(tdir), tstem.lower(), text))
+            if cand:
+                newu = os.path.relpath(os.path.join(OUT_DIR, cand), dirpath).replace(os.sep, "/")
+                fixed += 1
+                changed = True
+                q = "?" + u.split("?")[1] if "?" in u else ""
+                h = "#" + u.split("#")[1] if "#" in u else ""
+                return prefix + newu + q + h
+            return m.group(0)
+
+        html = re.sub(r'(src|href)="([^"]*)"', fix_ref, html)
+        if changed:
+            with open(fpath, "w", encoding="utf-8") as f:
+                f.write(html)
+    print(f"Total unresolvable local refs: {all_missing}; fixed {fixed}.")
+
+
+if __name__ == "__main__":
+    repair()

rewrite.py

@@ -0,0 +1,111 @@
+#!/usr/bin/env python3
+"""HTML rewriting: domain-agnostic relative URLs + RFQ form -> email (FormSubmit)."""
+import os, re, urllib.parse, html as htmllib
+
+from common import (BASE_HOST, MEDIA_HOST, YP_HOST, OUT_DIR,
+                    assets_rel_for, safe_asset_path, enqueue_asset,
+                    canonical_page_path)
+
+FORM_RECIPIENT = "mail@mitsuchaiyaporn.com"
+
+
+def _rel_for_page(url):
+    p = urllib.parse.urlparse(url)
+    path = p.path.rstrip("/")
+    return "/" if path == "" else path
+
+def _depth(url):
+    p = urllib.parse.urlparse(url)
+    path = p.path.rstrip("/")
+    return 0 if path == "" else path.count("/")
+
+def _root_prefix(url):
+    return "../" * _depth(url)
+
+def _abs(u, page_url):
+    if u.startswith("//"):
+        return "https:" + u
+    if u.startswith("http"):
+        return u
+    base = "https://" + BASE_HOST + (_rel_for_page(page_url).rstrip("/") + "/" if _rel_for_page(page_url) != "/" else "/")
+    return urllib.parse.urljoin(base, u)
+
+
+def rewrite_url_value(raw, page_url):
+    raw = htmllib.unescape(raw).strip()
+    if not raw or raw.startswith(("data:", "mailto:", "tel:", "javascript:", "#")):
+        return raw
+    parsed = urllib.parse.urlparse(_abs(raw, page_url))
+    host = (parsed.hostname or "").lower()
+    if host == BASE_HOST:
+        # decode path to the canonical (decoded) form so links match stored dirs
+        canon = urllib.parse.unquote(parsed.path).rstrip("/")
+        prefix = _root_prefix(page_url)
+        suffix = ("?" + parsed.query) if parsed.query else ""
+        suffix += ("#" + parsed.fragment) if parsed.fragment else ""
+        if canon == "":
+            return prefix + suffix
+        return prefix + canon.lstrip("/") + suffix
+    if host in (MEDIA_HOST, YP_HOST):
+        absurl = urllib.parse.urlunparse(parsed)
+        rel = safe_asset_path(assets_rel_for(absurl), absurl)
+        enqueue_asset(absurl)
+        return _root_prefix(page_url) + rel
+    return raw
+
+
+def rewrite_css_asset_links(text, page_url):
+    def repl(m):
+        u = m.group(1).strip()
+        if u.startswith(("data:", "#", "http", "//")):
+            return m.group(0)
+        abs = _abs(u, page_url)
+        parsed = urllib.parse.urlparse(abs)
+        host = (parsed.hostname or "").lower()
+        if host not in (BASE_HOST, MEDIA_HOST, YP_HOST):
+            return m.group(0)
+        rel = safe_asset_path(assets_rel_for(abs), abs)
+        enqueue_asset(abs)
+        return 'url("{}")'.format(_root_prefix(page_url) + rel)
+    return re.sub(r'url\(\s*["\']?([^"\'()]+)["\']?\s*\)', repl, text)
+
+
+def _action_is_backend(action):
+    a = action.lower()
+    return ("/rfq" in a) or ("magic-contact" in a) or ("/contactus" in a)
+
+
+def convert_rfq_form(html, page_url):
+    pattern = re.compile(r'(<form\b[^>]*>)(.*?)(</form>)', re.S)
+    def repl(m):
+        open_tag, inner, close_tag = m.group(1), m.group(2), m.group(3)
+        action_m = re.search(r'action="([^"]*)"', open_tag)
+        if not action_m:
+            return m.group(0)
+        if not _action_is_backend(_abs(action_m.group(1), page_url)):
+            return m.group(0)
+        fs_action = "https://formsubmit.co/ajax/" + FORM_RECIPIENT
+        new_open = '<form class="typ-static-form" action="' + fs_action + '" method="POST">'
+        hidden = ('<input type="hidden" name="_subject" value="แบบฟอร์มติดต่อจากเว็บ mitsuchaiyaporn">'
+                  '<input type="hidden" name="_template" value="table">'
+                  '<input type="hidden" name="_captcha" value="false">')
+        return new_open + hidden + inner + close_tag
+    return pattern.sub(repl, html)
+
+
+def rewrite_html(text, page_url):
+    text = re.sub(r'<link rel="canonical"[^>]*>', '', text)
+    text = re.sub(r'<meta property="og:url"[^>]*>', '', text)
+    text = re.sub(r'<meta name="twitter:url"[^>]*>', '', text)
+    text = convert_rfq_form(text, page_url)
+    text = re.sub(r'href="([^"]*)"',
+                  lambda m: 'href="' + rewrite_url_value(m.group(1), page_url) + '"', text)
+    text = re.sub(r'src="([^"]*)"',
+                  lambda m: 'src="' + rewrite_url_value(m.group(1), page_url) + '"', text)
+    # QR-code / other data-* URL attributes pointing at media host -> localize
+    text = re.sub(r'(data-qrcodr|data-\w*url)="([^"]*)"',
+                  lambda m: m.group(1) + '="' + rewrite_url_value(m.group(2), page_url) + '"', text)
+    text = re.sub(r'action="([^"]*)"',
+                  lambda m: 'action="' + rewrite_url_value(m.group(1), page_url) + '"', text)
+    text = rewrite_css_asset_links(text, page_url)
+    return text

site/assets/external/track/track.js

@@ -0,0 +1,3 @@
+var inframe="";var inframeReferrer=false;try{if(top.document.domain!=self.document.domain){var inframeReferrer=document.referrer;}else{}}catch(err){var inframeReferrer=document.referrer;}
+if(inframeReferrer){var elemLink=document.createElement('a');elemLink.setAttribute('href',inframeReferrer);inframe="&inframe="+escape(elemLink.hostname);}
+document.write("<IMG SRC='//www.yellowpages.co.th/track/track.php?url="+escape(window.location.href)+inframe+"' HEIGHT='0' WIDTH='0' ID='typtrack' ALT='' TITLE='' REL='NOFOLLOW' STYLE='width:0px; height:0px; border:0px '/>");