.hermes/plans/2026-08-19-m2-self-improving-analytics.md

@@ -0,0 +1,90 @@
+# M2 — Self-Improving Chatbot Analytics (Moreminimore Chat)
+
+Status: **part2b ✅ · phase 2 ✅ · phase 3 import-backend ✅ (in review passed) · persona-eval/approval/import-UI next** · Updated: 2026-08-19
+Repo: `/Users/kunthawat/Gitea/Chatwoot` · branch `develop`
+
+## Goal
+Every chat of every account is analyzed by an LLM daily → auto-tagged (topic + product) + deal outcome →
+aggregated into immutable daily metrics → admin-only reports + weekly persona improvement recommendation.
+
+## Design decisions (approved by user)
+- **Analyze EVERY chat of EVERY account daily** (LLM cost accepted — selling point is self-improving chatbot).
+- **LLM cascade** (`lib/llm/resolver.rb`): per-account openai hook → Captain config → nil (feature disabled). LLM effectively always on unless Captain unset.
+- **Original conversation tags**: use Chatwoot's `label_list` (ActsAsTaggableOn) — do NOT add many tag fields. Product/sale tag goes into the same tag list, prefixed (e.g. `group>subgroup>product` joined, or `สินค้า:...`/`กลุ่ม:...`). `top_tags` aggregates from these.
+- **No `tagged_at`** (skip time-tracking to avoid DB complexity).
+- **Daily stats are immutable snapshots**; live tags get re-tagged later (e.g. session becomes product B → product A removed). Snapshot preserves earlier state.
+- **Report page admin-role ONLY.** Filter agent/team/inbox/channel/tag/sale-tag/time + cross-analysis. Break down per CUSTOMER and per AGENT.
+- **Weekly (Mon 10:00)** LLM summarizes 7 days → persona improvement recommendation (summary only, NOT full prompt). Delivered via Line (quick-replies) / Telegram (inline keyboard) / webhook. Admin approve / view-full-prompt / reject; viewing full prompt → re-approve; approve → webhook with prompt.
+- **Product list**: per-account, admin-only menu. Import copy/paste or CSV/XLSX. Multi-level hierarchy (group>subgroup>product>display). LLM matches from lowest level first, tags ancestors; supports multiple products/groups per chat.
+
+## Data foundation (DONE — committed)
+- `conversation_daily_metrics` (per account/day: message_count, conversation_count, resolved/unresolved, avg_session_duration, top_tags, sale_tags, deal_outcomes, peak hours, agent/team/channel/inbox breakdown) — `9fbb0da9b`
+- `customer_daily_metrics` (per customer/day: same core + agent_ids) — `0fad9ce6b`
+- `product_catalog_entries` (per-account hierarchical: group>subgroup>product+aliases; `tag_hierarchy` helper) — `161e4210b`
+- `lib/llm/resolver.rb` (cascade; https-only api_base) — `ccff2dfca`
+- M1 openai custom base_url — `f9628db0e`
+
+## Remaining work
+### part2b-ii — LLM classifier service ✅ DONE (2026-08-19)
+Read a conversation (latest message + history for context) → classify → tags (topic + product from catalog using `>` join) + deal (won/lost/undecided). Cascade via `Llm::Resolver`.
+- **`lib/llm/analytics_classifier.rb`** (new): NOT `Captain::BaseTaskService` (no captain feature-flag gate — M2 is always-on via resolver).
+- Uses `Llm::Resolver.resolve(account)` → `Llm::Config.with_api_key(key, api_base:)` + `context.chat(model:).with_schema(...).ask(...)` for structured JSON output.
+- Matches product from `ProductCatalogEntry` via `tag_hierarchy` (index → path mapping with `values_at`+`compact`, safe against bad indexes).
+- Writes nothing — pure classifier. Returns `Result` (topics/products/deal) or `disabled: true` when resolver is nil (fail-closed, no LLM call, no content sent).
+- **Also fixed**: added `has_many :product_catalog_entries` to `Account` (foundation commit `161e4210b` added the model+migration but never wired the reverse association → would have been `NoMethodError`).
+- **Verified**: `ruby -c` clean (2.6.10 + 3.4.10); isolated smoke test 8/8 under Ruby 3.4.10 (disabled path, catalog join, fence sanitize, present path mapping + deal); static security scan clean. Independent reviewer `deleg_47c1a2cb` = **complete 5-key PASS** (empty security/logic; non-blocking em-dash escape fix applied).
+
+### part2b-iii — daily batch job ✅ DONE (2026-08-19)
+sidekiq-cron ~02:30 in the SUPER ADMIN timezone (`Asia/Bangkok`, single TZ, no hourly-check) → auto-write tags + aggregate into conversation_daily_metrics + customer_daily_metrics (immutable counts; re-tag live).
+- **`app/services/analytics/account_daily_processor.rb`** (new): per-account/date aggregation. Computes UTC day window from `account.reporting_timezone` (default UTC/Time.zone); selects conversations active in window; classifies each via `Llm::AnalyticsClassifier` (skips on error/disabled — fail-closed); applies tags (replaces prior M2-managed tags — `topic:` prefix + current catalog paths — preserves manual labels); accumulates per-conversation + per-customer metrics; upserts `ConversationDailyMetric` (unique account+date) + `CustomerDailyMetric` (unique account+contact+date).
+- **`app/jobs/analytics/daily_metrics_job.rb`** (new): iterates all accounts with per-account rescue; computes closed day (`yesterday`) in account TZ.
+- **`config/schedule.yml`**: adds `analytics_daily_metrics_job` at `30 2 * * *` timezone `Asia/Bangkok`, queue scheduled_jobs.
+- **Verified**: `ruby -c` clean (3.4.10); isolated smoke test 15/15 under Ruby 3.4.10 (day_window Bangkok UTC bounds, apply_tags re-tag preserving manual labels, accumulate conv+customer counts, upsert payload shape). Static scan clean; schedule.yml no dup keys. Independent reviewer `deleg_fecf7c61` = **complete 5-key PASS** (empty security/logic; 4 non-blocking suggestions: add specs, N+1 in chat_message_count, deal_outcomes dual structure, apply_tags rescue scope).
+
+### phase 2 — rollup + report (BACKEND DONE + REVIEWED-PASS; frontend in review)
+**Backend (all committed-eligible, reviewer `deleg_6800b012` = complete 5-key PASS):**
+- **`Analytics::ReportService`** — summary/timeseries/customers/agents from immutable conversation_daily_metrics + customer_daily_metrics (rollup/snapshot surface). Deal_outcomes normalization handles both 'totals'-nested (conv) and flat (customer).
+- **`Analytics::DrilldownService`** — deep filterable drilldown over live Conversation records (since/until/agent_id/team_id/inbox_id/channel/tag/deal + pagination). Fail-closed label-join (scope.none when no label exists). Tag/deal match via `acts_as_taggable_on :labels` (Deal tag convention `deal:won/lost/undecided`).
+- **`Api::V2::Accounts::AnalyticsReportsController`** — admin-only (`authorize :report, :view?` → ReportPolicy#view? = administrator?) with `summary` + `drilldown` endpoints; params validation (BadRequest on invalid date/int).
+- **`config/routes.rb`** — `resources :analytics_reports` collection get :summary / get :drilldown under v2 accounts.
+- **`AccountDailyProcessor#apply_tags`** — now also writes `deal:<outcome>` tag onto the live conversation (M2-managed, replaced on re-tag) so drilldown can filter by deal.
+- Verified: `ruby -c` clean (3.4.10), smoke 12/12 (report summary/timeseries/deal-norm/top_tags/customers/agents/daterange + drilldown deal/channel/label_list/unknown-tag), routes parse, static scan clean. Reviewer `deleg_6800b012` = complete 5-key PASS (4 non-blocking suggestions: N+1 in serialize, no specs, page not clamped, memoize labels).
+
+**Frontend (built, reviewer `deleg_15f8447c` pending):**
+- **`api/analyticsReports.js`** — v2 API wrapper (summary + drilldown).
+- **`routes/dashboard/settings/reports/AnalyticsReports.vue`** — admin report dashboard: summary cards + filter bar (since/until/agent/team/inbox/channel/tag/deal) + drilldown table.
+- **`reports.routes.js`** — new `analytics_reports` route under the (admin-only, REPORTS feature-flag) reports section.
+- **`en/report.json`** — `ANALYTICS_REPORTS` i18n block.
+- Verified: `node --check` on .js + JSON.parse on report.json. NOTE: full frontend build (eslint/vite/vitest) canNOT run locally (no node_modules) — flagged as limitation; run `pnpm install && pnpm test` before ship.
+
+### phase 3 — import + persona eval + approval
+**Import backend (DONE, reviewer `deleg_7c595252` = complete 5-key PASS):**
+- **`Analytics::ProductCatalogImportService`** — imports `product_catalog_entries` from copy/paste / CSV / TSV / keyword (`key: value`) / named-header forms. Parsing: TSV vs '|' vs comma detection; keyword-mode + named-header column mapping (COLUMN_ALIASES group/product/subgroup/display); ordered 2-col=group+product / 3-col=group+subgroup+product; alias normalization (comma/semicolon split); upsert (same account+group+product → update, else create). Returns `{ imported, updated, errors[] }`. Non-blocking reviewer fix applied: error line numbers now reference original input (header-shift offset).
+- **`Api::V2::Accounts::ProductCatalogEntriesController`** — admin-only (`ReportPolicy#view?`): `index` (list) + `import` (POST content, BadRequest if blank).
+- **`config/routes.rb`** — `resources :product_catalog_entries` only [:index] + collection post :import.
+- Verified: `ruby -c` clean (3.4.10), smoke 10/10 (TSV/pipe/keyword/header/ordered/upsert/error), static scan clean. Reviewer `deleg_7c595252` = complete 5-key PASS (4 non-blocking: no specs, line-offset [fixed], tab+pipe edge, XLSX unsupported).
+
+**XLSX support (DONE, reviewer `deleg_e7e0879c` = complete 5-key PASS):** Added `gem 'roo', '~> 2.10'` to Gemfile; `ProductCatalogImportService.import_file(account:, file_path:, filename:)` + `parse_spreadsheet` (uses `Roo::Spreadsheet.open` lazily, converts sheet rows to pipe-delimited lines then reuses the shared `parse_rows_from_lines` parser; graceful `rescue LoadError` for missing roo + `rescue StandardError` for parse errors); `extension()` prefers original filename (Rack tempfiles lack extension). Controller `import` branches to file-upload (`file.tempfile.path` + `file.original_filename`) with a `SUPPORTED_SPREADSHEET_EXTS` whitelist (`xlsx xls ods csv`) rejecting other types, vs text `content`. Verified: `ruby -c` clean, stub smoke tests text 10/10 + xlsx 5/5 under plain Ruby 3.4.10 with a fake roo module, static scan clean, reviewer `deleg_e7e0879c` = complete 5-key PASS (3 non-blocking: no committed specs, file-ext whitelist [applied], extension() nil guard). **REQUIRED deploy step**: `bundle install` must run in the deploy env to resolve `roo` + update `Gemfile.lock` (cannot run here — Ruby 3.4.4 pin vs 3.4.10 installed).
+
+**Remaining phase 3 — weekly persona eval (DONE, reviewer `deleg_b7d63fd7` = complete 5-key PASS):**
+- **`Analytics::WeeklyPersonaEvaluator`** — summarizes 7 days via `Analytics::ReportService` (aggregate counts/tags/deal_outcomes only, NO raw message content / NO customer PII) + LLM via `Llm::Resolver` cascade + `with_schema` → `Result(summary, recommendations)`. Pure evaluator (no persistence). Fail-closed: no credential → disabled, never calls LLM.
+- **`persona_evaluation`** admin-only endpoint (`POST /analytics_reports/persona_evaluation`) + route.
+- Verified: `ruby -c` clean, smoke 8/8, static scan clean, reviewer `deleg_b7d63fd7` = complete 5-key PASS (3 non-blocking: no specs, guard non-string recs, confirm .content — latter matches repo convention per Captain::ChatResponseHelper).
+
+**Remaining (NOT built):**
+- **Frontend product-catalog import UI (DONE, reviewer `deleg_0e9f9291` = PASS):** `api/productCatalog.js` + `productCatalog/Index.vue` (paste + .xlsx/.csv upload + result + table) + route `product_catalog_index` (admin-only) wired into `settings.routes.js`. Verified: `node --check`. NOTE: full frontend build cannot run locally — run `pnpm install && pnpm test` before ship.
+- **Approval flow (DONE, all reviewers PASS)** — user chose **"LINE primary (quick-replies) + webhook fallback"**, then asked to add **Telegram**. Delivery precedence: **LINE → Telegram → webhook**. `Analytics::PersonaApprovalService` (`deliver`: LINE push to `persona_line_user_id` w/ quickReply buttons / Telegram `sendMessage` inline keyboard to `persona_telegram_chat_id` / webhook POST to `persona_webhook_url` + HMAC sig header; `notify_approval`: **webhook-only** sends the approval decision to the webhook) + `Analytics::WeeklyPersonaEvaluationJob` + controller actions (`persona_evaluation_deliver`, `persona_approval_settings` [line_user_id + telegram_chat_id + webhook_url, conditional], `persona_decision`) + routes + schedule.yml `analytics_weekly_persona_evaluation_job` (Mon 10:00 Asia/Bangkok). Config in `Account#custom_attributes`. Reviewers: `deleg_1fe19871` FAIL-CLOSED (1 medium logic error `:approve` path — fixed) → re-review `deleg_3052f7a4` PASS; Telegram channel `deleg_34116d1a` = **complete 5-key PASS** (3 non-blocking: i18n/dedup button labels, no specs). **POST-MVP design note — prompt webhook handshake (DECIDED, NOT built; awaiting LLM-server details from user):**
+User clarified the self-improvement loop: a SEPARATE LLM server (not this app) will hold/apply the actual prompt, and this app talks to it over webhook.
+- **Connect-first**: on first webhook connection, the app should **pull the current prompt** via webhook (GET) and store it (snapshot).
+- **Approve → push**: when the admin approves the weekly persona recommendation, the app **POSTs the new prompt** via webhook to the LLM server to apply.
+- **Analytics separate**: this app only surfaces "what customers asked" (topics/products/deal) + recommendations; knowledge-prep/prompt-authoring is done/admin/LLM-server side, NOT here.
+- Delivery-prompt-webhook design (REST 2-endpoint vs generic action-body) — **DEFERRED**, user said "document/design only, wait for details". Not built.
+- **Docs** — NOT built.
+
+## Enterprise (EE) note
+EE files still read `CAPTAIN_OPEN_AI_ENDPOINT` directly and won't honor per-account base_url gating — intentional (out of OSS scope), tracked as follow-up.
+
+## Verification notes
+- Ruby env active is 2.6.10; target 3.4.4. Rails/RSpec/RuboCop cannot be reliably claimed passed — use `ruby -c` syntax + YAML validity + isolated smoke tests.
+- Full Vitest clean under TZ=UTC: 414 files / 4176 tests, 0 failed (frontend untouched here).
+- No secrets committed. Temp helpers under /tmp only.

.hermes/plans/2026-08-20-oss-chatbot-guardrail-knowledge.md

@@ -0,0 +1,82 @@
+# Plan — OSS Self-Contained Chatbot (Guardrail + Knowledge + Llm::Resolver Answer)
+
+Repo: `/Users/kunthawat/Gitea/Chatwoot` · branch `develop`
+Date: 2026-08-20
+Status: **PLAN — awaiting user approval before build**
+
+## Vision (from user)
+Make this Chatwoot fork a self-contained OSS chatbot: every inbound chat → guardrail
+(keep users on-topic, e.g. reject "บอกชะตารายวัน / ดูดวง" when it's a product/service bot)
+→ if in-scope, answer with LLM using a knowledge base (md/csv/excel) + system prompt +
+guardrail prompt + history. No EE, no external Captain. Answer via `Llm::Resolver` local.
+
+## Decisions locked (from clarify)
+- **Not using EE** → Captain v2 (`enterprise/`) is out of scope. Use OSS path only.
+- **Architecture**: build our own `Integrations::Chatbot::ProcessorService < Integrations::BotProcessorService`
+  (mirrors `Captain::ProcessorService` / `Dialogflow::ProcessorService`), overriding `get_response`
+  to answer via `Llm::Resolver` + knowledge base instead of an external webhook. Selectable per-inbox
+  (like Captain/Dialogflow choose processor by hook app_id).
+- **Trigger stays** OSS AgentBot / pending flow (the "AI answers" = conversation pending).
+- **LLM calls**: default **1 call** returning `{allowed, answer}`; config option to use **2 calls**
+  (guardrail check, then answer) for LLMs that need them. Backward-compatible.
+- **KB retrieval**: **keyword + embedding** retrieval (top-k relevant chunks), NOT whole-KB-in-prompt.
+- **Out-of-scope policy**:
+  - topic clearly off-guardrail (e.g. fortune-telling) → **refuse with template**
+  - topic related but not in KB / undecidable → **auto human handoff** (`bot_handoff!`)
+- **KB storage**: extend `product_catalog_entries` (csv/xlsx product rows) + add **MD FAQ** table.
+
+## Current OSS bot flow (verified)
+```
+inbound message (conversation pending?) 
+  → ... AgentBots::WebhookJob / agent_bot_listener → Webhooks::Trigger → webhook (external)
+```
+`BotProcessorService` base: `should_run_processor?` (message.reportable?, conversation.pending?)
+→ `get_response(source_id, content)`; `process_action` handles :handoff → `bot_handoff!` / :resolve.
+`Webhooks::Trigger#update_conversation_status`: on agent_bot failure, pending → open! (human takes over).
+
+## Architecture
+```
+[our ProcessorService] < Integrations::BotProcessorService
+  get_response → guardrail? 
+      ├─ in-scope  → Llm::Resolver answer (KB + system prompt + guardrail + history) → reply
+      └─ out-of-scope → out-of-scope reply template (reject) / bot_handoff!
+```
+Selected per inbox by hook app_id (same mechanism as Captain/Dialogflow).
+
+## Deliverables (build order)
+1. **KB backend**: `KnowledgeBaseFaq` model+migration (account-scoped, md FAQ: title + content + topic tags)
+   + reuse `product_catalog_entries` for csv/xlsx product rows. Import service (extend
+   `ProductCatalogImportService` / add MD FAQ import via `roo`/CSV/stdlib). Admin endpoints (list/import) + routes.
+2. **Retrieval service**: `Chatbot::KnowledgeRetriever` — keyword (+ optional embedding) top-k selection
+   over FAQ + product_catalog. Embedding vector column on FAQ table (nullable), keyword via SQL ILIKE/tsvector.
+3. **Guardrail + answer**: `Chatbot::GuardrailService` + `Chatbot::AnswerService` — default 1 call
+   `{ allowed, answer }`; optional 2-call mode. Out-of-scope → refuse template; related-but-not-in-KB /
+   undecidable → human handoff.
+4. **Chatbot processor**: `Integrations::Chatbot::ProcessorService < BotProcessorService` — get_response =
+   retrieve → guardrail/answer → reply | refuse | handoff. Wire selection per inbox.
+   - **Human handoff path adjusts chat status**: on related-but-not-in-KB / undecidable, call
+     `conversation.bot_handoff!` which **releases the bot (= `pending` → `open`), clears `assignee_agent_bot`,
+     sets `waiting_since`**, and dispatches the handoff event so a human agent queue/assignment picks it up.
+5. **Account config**: per-account: system prompt + guardrail prompt + out-of-scope reply template +
+   call-mode (1 or 2) + enabled flag + which KB (folder/index). Store in Account#custom_attributes or settings model.
+   Admin endpoints + UI.
+6. **Verify + review each part** (smoke / ruby -c / static scan / independent reviewer).
+
+## Open questions (mostly resolved; remaining minor)
+- [x] LLM calls: 1 default, 2 optional (user decision).
+- [x] KB retrieval: keyword + embedding, top-k (user decision).
+- [x] Out-of-scope: refuse if clearly off-topic; auto handoff if related-but-not-in-KB/undecidable.
+- [x] KB storage: extend product_catalog + add MD FAQ.
+- [ ] Embedding: which provider/API to compute embeddings (Llm::Resolver? separate embedding model?).
+- [ ] MD FAQ granularity: one row per file? per heading/section chunk? (affects retrieval + import).
+
+## Success criteria
+- OSS inbox with our processor answers in-scope from KB (keyword+embedding) via Llm::Resolver (1 or 2 calls). ✅ built
+- Clearly out-of-scope (e.g. fortune-telling) → templated refusal. ✅
+- Related-but-not-in-KB / undecidable → auto human handoff. ✅ (bot_handoff! → pending→open)
+- Product rows (csv/xlsx) + MD FAQ both import + retrieve. ✅
+- Per-account config (system/guardrail/out-of-scope template/call mode/enabled) via admin endpoint. ✅ (ChatbotConfigController + ConfigService)
+- Each deliverable passes smoke + static scan + independent reviewer. ✅ (final re-review deleg_a5c361cb = passed:true, end-to-end dispatchable)
+
+## Wiring (verified, end-to-end)
+apps.yml chatbot(inbox) → HooksController create (ensure_hook_type=inbox) → HookListener supported_events_map['chatbot'] → HookJob INTEGRATION_PROCESSORS['chatbot'] → Chatbot::ProcessorService → DecisionService → Llm::Resolver. i18n chatbot added to en.yml integration_apps.

Gemfile

@@ -51,6 +51,8 @@ gem 'gmail_xoauth'
 gem 'net-smtp',  '~> 0.3.4'
 # Prevent CSV injection
 gem 'csv-safe'
+# XLSX / ODS / XLS spreadsheet reading (used by Analytics::ProductCatalogImportService)
+gem 'roo', '~> 2.10'
 
 ##-- for active storage --##
 gem 'aws-sdk-s3', require: false

HANDOFF.md

@@ -1,5 +1,24 @@
 # HANDOFF
 
+## ═══════ SESSION 2026-08-22 — Final decision: external n8n (Option 3) ═══════
+
+- User selected Option 3: n8n remains a separate service for the LLM chatbot/workflow path.
+- Do not integrate Hermes, embed n8n, or change Chatwoot production code for this decision.
+- Keep the existing AgentBot async webhook boundary: Chatwoot → external n8n → configured LLM/external services → Chatwoot API reply.
+- The Gateway/Adapter design is deferred; implement it only if later requirements justify centralized tenant isolation, idempotency, DLQ, observability, or provider routing.
+- Only this handoff/log decision is updated; production source remains unchanged.
+
+## ═══════ SESSION 2026-08-22 — LLM chatbot runtime architecture discovery ═══════
+
+- Read-only architecture review for `/Users/kunthawat/Gitea/Chatwoot`; no production code changed.
+- Chatwoot's existing AgentBot path is already the intended boundary: async signed webhook out, external runtime processes, then external runtime posts an outgoing message through Chatwoot API. `outgoing_url` is not an LLM request/response endpoint.
+- Hermes is technically feasible only behind a bridge. Hermes API server is the preferred execution interface; Hermes webhook adapter is not plug-compatible because signature headers and response delivery contract differ.
+- Recommended target: `Chatwoot AgentBot → dedicated Hermes bridge → Hermes API → Chatwoot API`; optional external n8n remains for visual workflows, CRM/LINE/ERP side effects, and integrations. Do not embed n8n or give a customer-facing Hermes profile unrestricted terminal/file tools.
+- Tenant rules: keep Chatwoot as transcript/source of truth; namespace every request by account/inbox/conversation; do not treat Hermes personal memory/profile files as a multi-tenant knowledge base. Use allow-listed tools, sandboxing, idempotency, HMAC verification, secret rotation, and human handoff.
+- Next implementation gate: P0 one-account bridge proof (or n8n→Hermes proof), message-created only, no loop, signed webhook verification, reply via bot token, duplicate/retry tests, cross-tenant isolation test, and latency/error telemetry.
+- Working tree was already dirty with unrelated M1/M2 changes; future bridge work must use a separate worktree/branch.
+
+
 ## ═══════ SESSION 2026-08-19 — Moreminimore Chat: rebrand + M1 + M2 work ═══════
 
 ## Current state (2026-08-19)
@@ -30,10 +49,16 @@ HEAD = `ccff2dfca`. Worktree clean except untracked `.hermes/plans/` (never comm
 - Product list: per-account, admin-only menu (like report), import via copy/paste or CSV/XLSX. Multi-level hierarchy (big group>subgroup>product>display) — LLM matches from lowest level first, tags ancestors; supports multiple products/groups per chat.
 
 ### NEXT TO DO (M2 remaining — biggest remaining work)
-1. **part2b-ii**: LLM classifier service — read conversation (latest message + history for context), classify → tags (topic + product from catalog using '>' join) + deal (won/lost/undecided). Cascade via Llm::Resolver.
+1. **part2b-ii — LLM classifier service: ✅ DONE (2026-08-19)**. New `lib/llm/analytics_classifier.rb` — reads conversation, classifies via `Llm::Resolver` cascade + `with_schema` JSON output → `Result` (topics, products via tag_hierarchy path mapping, deal won/lost/undecided). Fail-closed on no-credential (returns `disabled`, never calls LLM / sends content). Also fixed: `Account` missing `has_many :product_catalog_entries` (foundation `161e4210b` wired model+migration but not the reverse association → would NoMethodError). Verified: `ruby -c` clean (2.6/3.4), isolated smoke 8/8 under Ruby 3.4.10 (found `/opt/homebrew/opt/ruby@3.4/bin/ruby` — real 3.4.10 available, `.ruby-version`=3.4.4), static scan clean, independent reviewer `deleg_47c1a2cb` = complete 5-key PASS (non-blocking em-dash escape fix applied). Plan doc: `.hermes/plans/2026-08-19-m2-self-improving-analytics.md`.
-2. **part2b-iii**: daily batch job (sidekiq-cron) at ~02:00-03:00 in the SUPER ADMIN timezone (single TZ, no hourly-check) → auto-write tags to conversation, aggregate into conversation_daily_metrics + customer_daily_metrics (immutable counts; re-tag live).
+2. **part2b-iii — daily batch job: ✅ DONE (2026-08-19)**. New `app/services/analytics/account_daily_processor.rb` (per-account/date: UTC day window from `account.reporting_timezone`, selects active conversations, classifies via `Llm::AnalyticsClassifier` fail-closed, re-tags replacing M2-managed tags + preserving manual labels, upserts ConversationDailyMetric unique account+date + CustomerDailyMetric unique account+contact+date) + `app/jobs/analytics/daily_metrics_job.rb` (per-account rescue, yesterday in account TZ) + `config/schedule.yml` entry `analytics_daily_metrics_job` `30 2 * * *` timezone `Asia/Bangkok`. Verified: `ruby -c` clean (3.4.10), smoke 15/15, static scan clean, schedule no dup keys, reviewer `deleg_fecf7c61` = complete 5-key PASS (4 non-blocking suggestions: add specs, N+1 in chat_message_count, deal_outcomes dual structure, apply_tags rescue scope).
-3. **phase 2**: monthly/yearly rollup (sum daily) + admin-only report dashboard (filters agent/team/inbox/channel/tag/sale/time, cross-analysis) + deep per-customer/per-agent views.
+3. **phase 2 — rollup + report: ✅ DONE (2026-08-19)**. Backend: `Analytics::ReportService` (summary/timeseries/customers/agents from immutable daily-metric tables; deal_outcomes normalization for 'totals'-nested + flat) + `Analytics::DrilldownService` (deep filterable drilldown over live conversations: since/until/agent/team/inbox/channel/tag/deal + pagination; fail-closed label-join) + `Api::V2::Accounts::AnalyticsReportsController` (admin-only via ReportPolicy#view?=administrator; summary + drilldown) + routes `analytics_reports` (get :summary/:drilldown). `AccountDailyProcessor#apply_tags` now writes `deal:<outcome>` tag for deal filtering. Frontend: `api/analyticsReports.js` + `AnalyticsReports.vue` (summary cards + filter bar + drilldown table) + route `analytics_reports` under reports section + `ANALYTICS_REPORTS` i18n keys in en/report.json. Verified: backend smoke 12/12, reviewers `deleg_6800b012` (backend) + `deleg_15f8447c` (frontend) = complete 5-key PASS (non-blocking i18n suggestions applied). NOTE: frontend full build (eslint/vite/vitest) cannot run locally (no node_modules) — run `pnpm install && pnpm test` before ship.
-4. **phase 3**: product-list import UI (admin-only, copy/paste + CSV + XLSX), weekly persona eval + approval flow (Telegram inline button, Line quick-reply, webhook) + docs.
+4. **phase 3 — import + persona eval + approval (2026-08-20):**
+   - **Weekly persona eval** ✅: `Analytics::WeeklyPersonaEvaluator` (7d via ReportService → LLM summary+recommendations; summary-only, no PII) + `persona_evaluation` endpoint + route. Reviewer `deleg_b7d63fd7` PASS.
+   - **Frontend product-catalog import UI** ✅: `api/productCatalog.js` + `productCatalog/Index.vue` (paste + xlsx/csv upload + result + table) + admin-only route. Reviewer `deleg_0e9f9291` PASS (i18n suggestions applied).
+   - **Approval flow** (user chose **LINE primary + webhook fallback**, then added **Telegram**; precedence LINE → Telegram → webhook): `Analytics::PersonaApprovalService` (`deliver` LINE quick-reply / Telegram inline keyboard / webhook fallback; `notify_approval` webhook-only w/ decision) + `Analytics::WeeklyPersonaEvaluationJob` + controller (`persona_evaluation_deliver`, `persona_approval_settings` [line_user_id + telegram_chat_id + webhook_url, conditional], `persona_decision`) + routes + schedule.yml `analytics_weekly_persona_evaluation_job` (Mon 10:00 Asia/Bangkok). First reviewer `deleg_1fe19871` FAIL-CLOSED (1 medium logic error: `:approve` path LINE-first + decision not passed) → **FIXED** (webhook-only `notify_approval(decision)`) → re-review `deleg_3052f7a4` PASS; Telegram `deleg_34116d1a` PASS. Config: `persona_line_user_id` / `persona_telegram_chat_id` / `persona_webhook_url` / `persona_webhook_secret` in Account#custom_attributes.
+   - **Docs** — via HANDOFF.md/plan doc; no separate docs/ convention.
+   - **POST-MVP design note (prompt webhook handshake) — user clarified, NOT built, awaiting LLM-server details**: a SEPARATE LLM server holds/applies the actual prompt; this app connects via webhook — (1) connect-first: pull current prompt (GET) + store snapshot; (2) approve weekly recommendation → POST new prompt to LLM server to apply. Analytics here ONLY surfaces "what customers asked" + recommendations; knowledge-prep/prompt-authoring is admin/LLM-server side. Delivery-webhook design (REST 2-endpoint vs generic action-body) DEFERRED — user wants document/design only for now.
+   Verified: smoke tests (eval 8/8, import 10/10 + xlsx 5/5, approval 10/10 incl. Telegram), `ruby -c` clean, node --check on .js, schedule.yml valid + unique keys, static scan clean across all.
 
 ### Verification notes / blockers
 - Ruby env active is 2.6.10; target 3.4.4. Rails/RSpec/RuboCop cannot be reliably claimed passed. Ruby `ruby -c` syntax + YAML validity + isolated smoke tests used instead.

app/controllers/api/v2/accounts/analytics_reports_controller.rb

@@ -0,0 +1,129 @@
+# Admin-only M2 analytics reporting endpoints (phase 2).
+#
+#   GET /api/v2/accounts/:account_id/analytics_reports/summary
+#     -> Analytics::ReportService (immutable daily-metric snapshots)
+#   GET /api/v2/accounts/:account_id/analytics_reports/drilldown
+#     -> Analytics::DrilldownService (live conversations, filterable)
+#
+# Both are admin-role only (ReportPolicy#view? => administrator?).
+class Api::V2::Accounts::AnalyticsReportsController < Api::V1::Accounts::BaseController
+  before_action :check_authorization
+
+  def summary
+    render json: Analytics::ReportService.build(
+      account: Current.account,
+      since: date_param(:since),
+      until_date: date_param(:until)
+    )
+  end
+
+  def drilldown
+    render json: Analytics::DrilldownService.build(
+      account: Current.account,
+      filters: drilldown_filters
+    )
+  end
+
+  def persona_evaluation
+    result = Analytics::WeeklyPersonaEvaluator.evaluate(account: Current.account)
+    render json: {
+      summary: result.summary,
+      recommendations: result.recommendations,
+      disabled: result.disabled?,
+      error: result.error
+    }
+  end
+
+  # Run the evaluation and deliver it via the configured channel (LINE / webhook).
+  # POST /analytics_reports/persona_evaluation/deliver
+  def persona_evaluation_deliver
+    result = Analytics::WeeklyPersonaEvaluator.evaluate(account: Current.account)
+    return render json: { disabled: true } if result.disabled?
+
+    delivery = Analytics::PersonaApprovalService.deliver(account: Current.account, evaluation: result)
+    render json: {
+      summary: result.summary,
+      recommendations: result.recommendations,
+      delivery: delivery
+    }
+  end
+
+  # Save the delivery configuration (admin LINE user id / Telegram chat id + webhook URL/secret).
+  # POST /analytics_reports/persona_approval_settings
+  #   { line_user_id, telegram_chat_id, webhook_url, webhook_secret }
+  def persona_approval_settings
+    account = Current.account
+    account.custom_attributes['persona_line_user_id'] = params[:line_user_id] if params.key?(:line_user_id)
+    account.custom_attributes['persona_telegram_chat_id'] = params[:telegram_chat_id] if params.key?(:telegram_chat_id)
+    account.custom_attributes['persona_webhook_url'] = params[:webhook_url] if params.key?(:webhook_url)
+    account.custom_attributes['persona_webhook_secret'] = params[:webhook_secret] if params[:webhook_secret].present?
+    account.save!
+    render json: { success: true }
+  end
+
+  # Receive an admin decision (approve / reject / view_full_prompt) — from a LINE
+  # callback or webhook echo. On approve, notify the configured webhook with the
+  # decision attached (webhook-only, NOT re-sent over LINE).
+  # POST /analytics_reports/persona_decision  { decision }
+  def persona_decision
+    decision = params[:decision].to_s
+    raise ActionController::BadRequest, 'invalid decision' unless %w[approve reject view_full_prompt].include?(decision)
+
+    if decision == 'approve'
+      track_approval(decision)
+    end
+
+    render json: { success: true, decision: decision }
+  end
+
+  private
+
+  def track_approval(decision)
+    evaluation = Analytics::WeeklyPersonaEvaluator.evaluate(account: Current.account)
+    return if evaluation.disabled?
+
+    Analytics::PersonaApprovalService.notify_approval(
+      account: Current.account,
+      evaluation: evaluation,
+      decision: decision
+    )
+  end
+
+
+  def check_authorization
+    authorize :report, :view?
+  end
+
+  def date_param(name)
+    value = params[name]
+    return nil if value.blank?
+
+    Date.parse(value.to_s)
+  rescue ArgumentError
+    raise ActionController::BadRequest, "Invalid #{name} date"
+  end
+
+  def drilldown_filters
+    {
+      since: date_param(:since),
+      until: date_param(:until),
+      agent_id: integer_param(:agent_id),
+      team_id: integer_param(:team_id),
+      inbox_id: integer_param(:inbox_id),
+      channel: params[:channel].presence,
+      tag: params[:tag].presence,
+      deal: params[:deal].presence,
+      page: integer_param(:page),
+      per_page: integer_param(:per_page)
+    }.compact
+  end
+
+  def integer_param(name)
+    value = params[name]
+    return nil if value.blank?
+
+    Integer(value)
+  rescue ArgumentError
+    raise ActionController::BadRequest, "Invalid #{name}"
+  end
+end

app/controllers/api/v2/accounts/chatbot_config_controller.rb

@@ -0,0 +1,30 @@
+# Admin-only chatbot configuration endpoint for the self-contained OSS chatbot.
+#
+#   GET  /api/v2/accounts/:account_id/chatbot_config -> current config
+#   POST /api/v2/accounts/:account_id/chatbot_config -> update allowed keys
+#
+# Admin-role only (ReportPolicy#view? => administrator?). Reads/writes the account's
+# chatbot settings via Chatbot::ConfigService (stored in Account#custom_attributes).
+class Api::V2::Accounts::ChatbotConfigController < Api::V1::Accounts::BaseController
+  before_action :check_authorization
+
+  def show
+    render json: Chatbot::ConfigService.config(Current.account)
+  end
+
+  def update
+    config = Chatbot::ConfigService.update!(Current.account, chatbot_config_params)
+    render json: config
+  end
+
+  private
+
+  def chatbot_config_params
+    params.permit(:chatbot_enabled, :chatbot_system_prompt, :chatbot_guardrail_prompt,
+                  :chatbot_out_of_scope_reply, :chatbot_call_mode)
+  end
+
+  def check_authorization
+    authorize :report, :view?
+  end
+end

app/controllers/api/v2/accounts/knowledge_base_faqs_controller.rb

@@ -0,0 +1,34 @@
+# Admin-only knowledge base management for the self-contained OSS chatbot.
+#
+#   GET  /api/v2/accounts/:account_id/knowledge_base_faqs
+#        -> list FAQ entries
+#   POST /api/v2/accounts/:account_id/knowledge_base_faqs/import
+#        -> import markdown text (`content`) or uploaded .md/.csv/.xlsx (`file`)
+#
+# Admin-role only (ReportPolicy#view? => administrator?).
+class Api::V2::Accounts::KnowledgeBaseFaqsController < Api::V1::Accounts::BaseController
+  before_action :check_authorization
+
+  def index
+    faqs = Current.account.knowledge_base_faqs.order(:title)
+    render json: { faqs: faqs.as_json(only: %i[id title topic_tags source_filename updated_at]) }
+  end
+
+  def import
+    if params[:file].present?
+      result = KnowledgeBase::ImportService.import(account: Current.account, file_path: params[:file].tempfile.path, filename: params[:file].original_filename)
+    else
+      content = params[:content]
+      raise ActionController::BadRequest, 'content is required' if content.blank?
+
+      result = KnowledgeBase::ImportService.import(account: Current.account, content: content)
+    end
+    render json: result
+  end
+
+  private
+
+  def check_authorization
+    authorize :report, :view?
+  end
+end

app/controllers/api/v2/accounts/product_catalog_entries_controller.rb

@@ -0,0 +1,54 @@
+# Admin-only product catalog management (phase 3).
+#
+#   GET  /api/v2/accounts/:account_id/product_catalog_entries
+#        -> list current catalog entries
+#   POST /api/v2/accounts/:account_id/product_catalog_entries/import
+#        -> import from pasted text (copy/paste), CSV/TSV upload, or an .xlsx/.ods/.csv
+#           file upload (multipart `file`); text via `content`
+#
+# Admin-role only (ReportPolicy#view? => administrator?).
+class Api::V2::Accounts::ProductCatalogEntriesController < Api::V1::Accounts::BaseController
+  before_action :check_authorization
+
+  SUPPORTED_SPREADSHEET_EXTS = %w[xlsx xls ods csv].freeze
+
+  def index
+    entries = Current.account.product_catalog_entries
+                     .order(:group_name, :subgroup_name, :product_name)
+    render json: { entries: entries.map(&:as_json) }
+  end
+
+  def import
+    if params[:file].present?
+      import_from_file(params[:file])
+    else
+      import_from_content(params[:content])
+    end
+  end
+
+  private
+
+  def import_from_content(content)
+    raise ActionController::BadRequest, 'content is required' if content.blank?
+
+    result = Analytics::ProductCatalogImportService.import(account: Current.account, content: content)
+    render json: result
+  end
+
+  def import_from_file(file)
+    ext = File.extname(file.original_filename.to_s).delete('.').downcase
+    raise ActionController::BadRequest, "unsupported file type: .#{ext}" unless SUPPORTED_SPREADSHEET_EXTS.include?(ext)
+
+    path = file.tempfile.path
+    result = Analytics::ProductCatalogImportService.import_file(
+      account: Current.account,
+      file_path: path,
+      filename: file.original_filename
+    )
+    render json: result
+  end
+
+  def check_authorization
+    authorize :report, :view?
+  end
+end

app/javascript/dashboard/api/analyticsReports.js

@@ -0,0 +1,18 @@
+/* global axios */
+import ApiClient from './ApiClient';
+
+class AnalyticsReportsAPI extends ApiClient {
+  constructor() {
+    super('analytics_reports', { accountScoped: true, apiVersion: 'v2' });
+  }
+
+  getSummary(params = {}) {
+    return axios.get(`${this.url}/summary`, { params });
+  }
+
+  getDrilldown(params = {}) {
+    return axios.get(`${this.url}/drilldown`, { params });
+  }
+}
+
+export default new AnalyticsReportsAPI();

app/javascript/dashboard/api/productCatalog.js

@@ -0,0 +1,26 @@
+/* global axios */
+import ApiClient from './ApiClient';
+
+class ProductCatalogAPI extends ApiClient {
+  constructor() {
+    super('product_catalog_entries', { accountScoped: true, apiVersion: 'v2' });
+  }
+
+  list() {
+    return axios.get(this.url, { params: { page: 1 } });
+  }
+
+  importFromContent(content) {
+    return axios.post(`${this.url}/import`, { content });
+  }
+
+  importFromFile(file) {
+    const formData = new FormData();
+    formData.append('file', file);
+    return axios.post(`${this.url}/import`, formData, {
+      headers: { 'Content-Type': 'multipart/form-data' },
+    });
+  }
+}
+
+export default new ProductCatalogAPI();

app/javascript/dashboard/i18n/locale/en/report.json

@@ -541,6 +541,61 @@
       }
     }
   },
+  "PRODUCT_CATALOG": {
+    "TITLE": "Product Catalog",
+    "DESC": "Import your product catalog so the chatbot can tag conversations with the right product.",
+    "PASTE_TITLE": "Paste catalog",
+    "IMPORT_BUTTON": "Import text",
+    "UPLOAD_BUTTON": "Upload .xlsx / .csv",
+    "IMPORTED": "imported",
+    "UPDATED": "updated",
+    "CURRENT_TITLE": "Current catalog",
+    "COL_GROUP": "Group",
+    "COL_SUBGROUP": "Subgroup",
+    "COL_PRODUCT": "Product",
+    "COL_ALIASES": "Aliases",
+    "EMPTY": "No products in the catalog yet.",
+    "LOAD_FAILED": "Failed to load catalog",
+    "IMPORT_FAILED": "Import failed",
+    "FILE_IMPORT_FAILED": "File import failed",
+    "FORMAT_HINT": "group | subgroup | product | display | aliases",
+    "FORMAT_EXAMPLE": "กลุ่มหลัก | กลุ่มย่อย | ชื่อสินค้า | ชื่อแสดง | สินค้า1,สินค้า2",
+    "LINE": "Line"
+  },
+  "ANALYTICS_REPORTS": {
+    "HEADER": "Analytics",
+    "HEADER_DESC": "Self-improving chatbot insights",
+    "FROM": "From",
+    "TO": "To",
+    "AGENT": "Agent ID",
+    "TEAM": "Team ID",
+    "INBOX": "Inbox ID",
+    "CHANNEL": "Channel",
+    "TAG": "Tag",
+    "DEAL": "Deal",
+    "ALL_CHANNELS": "All channels",
+    "ALL_DEALS": "All deals",
+    "APPLY": "Apply",
+    "RESET": "Reset",
+    "CONVERSATIONS": "Conversations",
+    "MESSAGES": "Messages",
+    "RESOLVED": "Resolved",
+    "UNRESOLVED": "Unresolved",
+    "DEALS_WON": "Deals won",
+    "CONVERSATION_DRILLDOWN": "Conversation drilldown",
+    "NO_DATA": "No conversations match the current filters.",
+    "TOTAL": "{count} total",
+    "COL_ID": "ID",
+    "COL_CHANNEL": "Channel",
+    "COL_ASSIGNEE": "Assignee",
+    "COL_STATUS": "Status",
+    "COL_DEAL": "Deal",
+    "COL_MESSAGES": "Messages",
+    "COL_TAGS": "Tags",
+    "OPT_WON": "Won",
+    "OPT_LOST": "Lost",
+    "OPT_UNDECIDED": "Undecided"
+  },
   "OVERVIEW_REPORTS": {
     "HEADER": "Overview",
     "LIVE": "Live",

app/javascript/dashboard/routes/dashboard/settings/productCatalog/Index.vue

@@ -0,0 +1,175 @@
+<script setup>
+import { ref, onMounted } from 'vue';
+import { useI18n } from 'vue-i18n';
+import { useAlert } from 'dashboard/composables';
+import ProductCatalogAPI from 'dashboard/api/productCatalog';
+
+const { t } = useI18n();
+
+const entries = ref([]);
+const loading = ref(false);
+const importResult = ref(null);
+const pastedContent = ref('');
+const fileInput = ref(null);
+
+const PASTE_HINT = `${t('PRODUCT_CATALOG.FORMAT_HINT')}\n${t('PRODUCT_CATALOG.FORMAT_EXAMPLE')}`;
+
+async function loadEntries() {
+  loading.value = true;
+  try {
+    const { data } = await ProductCatalogAPI.list();
+    entries.value = data.entries || [];
+  } catch (error) {
+    useAlert(t('PRODUCT_CATALOG.LOAD_FAILED'));
+  } finally {
+    loading.value = false;
+  }
+}
+
+async function importContent() {
+  if (!pastedContent.value.trim()) return;
+  loading.value = true;
+  try {
+    const { data } = await ProductCatalogAPI.importFromContent(
+      pastedContent.value
+    );
+    importResult.value = data;
+    useAlert(
+      `${data.imported} ${t('PRODUCT_CATALOG.IMPORTED')}, ${data.updated} ${t('PRODUCT_CATALOG.UPDATED')}`
+    );
+    await loadEntries();
+  } catch (error) {
+    useAlert(t('PRODUCT_CATALOG.IMPORT_FAILED'));
+  } finally {
+    loading.value = false;
+  }
+}
+
+async function onFileChange(event) {
+  const file = event.target.files[0];
+  if (!file) return;
+  loading.value = true;
+  try {
+    const { data } = await ProductCatalogAPI.importFromFile(file);
+    importResult.value = data;
+    useAlert(
+      `${data.imported} ${t('PRODUCT_CATALOG.IMPORTED')}, ${data.updated} ${t('PRODUCT_CATALOG.UPDATED')}`
+    );
+    await loadEntries();
+  } catch (error) {
+    useAlert(t('PRODUCT_CATALOG.FILE_IMPORT_FAILED'));
+  } finally {
+    loading.value = false;
+    if (fileInput.value) fileInput.value.value = '';
+  }
+}
+
+onMounted(loadEntries);
+</script>
+
+<template>
+  <div class="flex flex-col gap-4 p-4">
+    <header>
+      <h2 class="text-2xl font-semibold">{{ $t('PRODUCT_CATALOG.TITLE') }}</h2>
+      <p class="text-sm text-slate-500">
+        {{ $t('PRODUCT_CATALOG.DESC') }}
+      </p>
+    </header>
+
+    <!-- Paste import -->
+    <section
+      class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+    >
+      <h3 class="text-base font-semibold mb-2">
+        {{ $t('PRODUCT_CATALOG.PASTE_TITLE') }}
+      </h3>
+      <p class="text-xs text-slate-400 mb-2">{{ PASTE_HINT }}</p>
+      <textarea
+        v-model="pastedContent"
+        rows="8"
+        class="w-full text-sm p-3 rounded-md border border-slate-200 dark:border-slate-700 bg-slate-50 dark:bg-slate-800 font-mono"
+        :placeholder="PASTE_HINT"
+      />
+      <div class="flex gap-2 mt-3">
+        <button
+          class="button button--small bg-slate-800 text-white"
+          :disabled="loading"
+          @click="importContent"
+        >
+          {{ $t('PRODUCT_CATALOG.IMPORT_BUTTON') }}
+        </button>
+        <label class="button button--small cursor-pointer">
+          {{ $t('PRODUCT_CATALOG.UPLOAD_BUTTON') }}
+          <input
+            ref="fileInput"
+            type="file"
+            accept=".xlsx,.xls,.ods,.csv"
+            class="hidden"
+            @change="onFileChange"
+          />
+        </label>
+      </div>
+    </section>
+
+    <!-- Import result -->
+    <section
+      v-if="importResult"
+      class="rounded-lg border border-emerald-200 bg-emerald-50 dark:bg-emerald-950 p-4"
+    >
+      <div class="text-sm font-medium text-emerald-700 dark:text-emerald-300">
+        {{ importResult.imported }} {{ $t('PRODUCT_CATALOG.IMPORTED') }} ·
+        {{ importResult.updated }} {{ $t('PRODUCT_CATALOG.UPDATED') }}
+      </div>
+      <ul
+        v-if="importResult.errors && importResult.errors.length"
+        class="mt-2 text-xs text-red-600"
+      >
+        <li v-for="err in importResult.errors" :key="err.line">
+          {{ $t('PRODUCT_CATALOG.LINE') }} {{ err.line }}: {{ err.message }}
+        </li>
+      </ul>
+    </section>
+
+    <!-- List -->
+    <section
+      class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+    >
+      <h3 class="text-base font-semibold mb-2">
+        {{ $t('PRODUCT_CATALOG.CURRENT_TITLE') }} ({{ entries.length }})
+      </h3>
+      <div class="overflow-x-auto">
+        <table class="min-w-full text-sm">
+          <thead>
+            <tr
+              class="text-left text-xs text-slate-500 border-b border-slate-200 dark:border-slate-700"
+            >
+              <th class="py-2 pr-3">{{ $t('PRODUCT_CATALOG.COL_GROUP') }}</th>
+              <th class="py-2 pr-3">
+                {{ $t('PRODUCT_CATALOG.COL_SUBGROUP') }}
+              </th>
+              <th class="py-2 pr-3">{{ $t('PRODUCT_CATALOG.COL_PRODUCT') }}</th>
+              <th class="py-2">{{ $t('PRODUCT_CATALOG.COL_ALIASES') }}</th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr
+              v-for="e in entries"
+              :key="e.id"
+              class="border-b border-slate-100 dark:border-slate-800"
+            >
+              <td class="py-2 pr-3">{{ e.group_name }}</td>
+              <td class="py-2 pr-3">{{ e.subgroup_name || '-' }}</td>
+              <td class="py-2 pr-3">{{ e.product_name }}</td>
+              <td class="py-2 text-xs">{{ (e.aliases || []).join(', ') }}</td>
+            </tr>
+            <tr v-if="!entries.length && !loading">
+              <td colspan="4" class="py-6 text-center text-slate-400">
+                {{ $t('PRODUCT_CATALOG.EMPTY') }}
+              </td>
+            </tr>
+          </tbody>
+        </table>
+      </div>
+    </section>
+  </div>
+</template>

app/javascript/dashboard/routes/dashboard/settings/productCatalog/productCatalog.routes.js

@@ -0,0 +1,18 @@
+import { frontendURL } from '../../../../helper/URLHelper';
+
+import ProductCatalogIndex from './Index.vue';
+
+const meta = {
+  permissions: ['administrator'],
+};
+
+export default {
+  routes: [
+    {
+      path: frontendURL('accounts/:accountId/settings/productCatalog'),
+      name: 'product_catalog_index',
+      meta,
+      component: ProductCatalogIndex,
+    },
+  ],
+};

app/javascript/dashboard/routes/dashboard/settings/reports/AnalyticsReports.vue

@@ -0,0 +1,287 @@
+<script setup>
+import { ref, onMounted } from 'vue';
+import { useAlert } from 'dashboard/composables';
+import AnalyticsReportsAPI from 'dashboard/api/analyticsReports';
+import ReportHeader from './components/ReportHeader.vue';
+
+const loading = ref(false);
+const summary = ref(null);
+const drilldown = ref(null);
+
+const filter = ref({
+  since: '',
+  until: '',
+  agentId: '',
+  teamId: '',
+  inboxId: '',
+  channel: '',
+  tag: '',
+  deal: '',
+});
+
+const DEAL_OPTIONS = [
+  { label: 'OPT_WON', value: 'won' },
+  { label: 'OPT_LOST', value: 'lost' },
+  { label: 'OPT_UNDECIDED', value: 'undecided' },
+];
+
+const CHANNEL_OPTIONS = [
+  'Channel::WebWidget',
+  'Channel::Whatsapp',
+  'Channel::FacebookPage',
+  'Channel::Instagram',
+  'Channel::Line',
+  'Channel::Telegram',
+  'Channel::Email',
+];
+
+async function loadSummary() {
+  const params = {};
+  if (filter.value.since) params.since = filter.value.since;
+  if (filter.value.until) params.until = filter.value.until;
+  summary.value = (await AnalyticsReportsAPI.getSummary(params)).data;
+}
+
+async function loadDrilldown() {
+  const params = {};
+  if (filter.value.since) params.since = filter.value.since;
+  if (filter.value.until) params.until = filter.value.until;
+  if (filter.value.agentId) params.agent_id = filter.value.agentId;
+  if (filter.value.teamId) params.team_id = filter.value.teamId;
+  if (filter.value.inboxId) params.inbox_id = filter.value.inboxId;
+  if (filter.value.channel) params.channel = filter.value.channel;
+  if (filter.value.tag) params.tag = filter.value.tag;
+  if (filter.value.deal) params.deal = filter.value.deal;
+  drilldown.value = (await AnalyticsReportsAPI.getDrilldown(params)).data;
+}
+
+async function loadAll() {
+  loading.value = true;
+  try {
+    await Promise.all([loadSummary(), loadDrilldown()]);
+  } catch (error) {
+    useAlert('Failed to load analytics');
+  } finally {
+    loading.value = false;
+  }
+}
+
+function resetFilters() {
+  filter.value = {
+    since: '',
+    until: '',
+    agentId: '',
+    teamId: '',
+    inboxId: '',
+    channel: '',
+    tag: '',
+    deal: '',
+  };
+  loadAll();
+}
+
+onMounted(loadAll);
+</script>
+
+<template>
+  <div>
+    <ReportHeader
+      :header-title="$t('ANALYTICS_REPORTS.HEADER')"
+      :header-description="$t('ANALYTICS_REPORTS.HEADER_DESC')"
+    />
+
+    <div class="flex flex-col gap-4 pb-6">
+      <!-- Filters -->
+      <div
+        class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4 flex flex-wrap gap-3 items-end"
+      >
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.FROM') }}
+          <input v-model="filter.since" type="date" class="input" />
+        </label>
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.TO') }}
+          <input v-model="filter.until" type="date" class="input" />
+        </label>
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.AGENT') }}
+          <input v-model="filter.agentId" type="number" class="input w-24" />
+        </label>
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.TEAM') }}
+          <input v-model="filter.teamId" type="number" class="input w-24" />
+        </label>
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.INBOX') }}
+          <input v-model="filter.inboxId" type="number" class="input w-24" />
+        </label>
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.CHANNEL') }}
+          <select v-model="filter.channel" class="input">
+            <option value="">{{ $t('ANALYTICS_REPORTS.ALL_CHANNELS') }}</option>
+            <option v-for="ch in CHANNEL_OPTIONS" :key="ch" :value="ch">
+              {{ ch }}
+            </option>
+          </select>
+        </label>
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.TAG') }}
+          <input v-model="filter.tag" type="text" class="input w-32" />
+        </label>
+        <label
+          class="flex flex-col gap-1 text-xs text-slate-600 dark:text-slate-300"
+        >
+          {{ $t('ANALYTICS_REPORTS.DEAL') }}
+          <select v-model="filter.deal" class="input">
+            <option value="">{{ $t('ANALYTICS_REPORTS.ALL_DEALS') }}</option>
+            <option v-for="d in DEAL_OPTIONS" :key="d.value" :value="d.value">
+              {{ $t(`ANALYTICS_REPORTS.${d.label}`) }}
+            </option>
+          </select>
+        </label>
+        <div class="flex gap-2">
+          <button
+            class="button button--small bg-slate-800 text-white"
+            :disabled="loading"
+            @click="loadAll"
+          >
+            {{ $t('ANALYTICS_REPORTS.APPLY') }}
+          </button>
+          <button class="button button--small" @click="resetFilters">
+            {{ $t('ANALYTICS_REPORTS.RESET') }}
+          </button>
+        </div>
+      </div>
+
+      <!-- Summary cards -->
+      <div
+        v-if="summary && !loading"
+        class="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-5 gap-4"
+      >
+        <div
+          class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+        >
+          <div class="text-xs text-slate-500">
+            {{ $t('ANALYTICS_REPORTS.CONVERSATIONS') }}
+          </div>
+          <div class="text-2xl font-bold">{{ summary.conversation_count }}</div>
+        </div>
+        <div
+          class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+        >
+          <div class="text-xs text-slate-500">
+            {{ $t('ANALYTICS_REPORTS.MESSAGES') }}
+          </div>
+          <div class="text-2xl font-bold">{{ summary.message_count }}</div>
+        </div>
+        <div
+          class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+        >
+          <div class="text-xs text-slate-500">
+            {{ $t('ANALYTICS_REPORTS.RESOLVED') }}
+          </div>
+          <div class="text-2xl font-bold text-emerald-600">
+            {{ summary.resolved_count }}
+          </div>
+        </div>
+        <div
+          class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+        >
+          <div class="text-xs text-slate-500">
+            {{ $t('ANALYTICS_REPORTS.UNRESOLVED') }}
+          </div>
+          <div class="text-2xl font-bold text-amber-600">
+            {{ summary.unresolved_count }}
+          </div>
+        </div>
+        <div
+          class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+        >
+          <div class="text-xs text-slate-500">
+            {{ $t('ANALYTICS_REPORTS.DEALS_WON') }}
+          </div>
+          <div class="text-2xl font-bold text-emerald-600">
+            {{ summary.deal_outcomes?.won || 0 }}
+          </div>
+        </div>
+      </div>
+
+      <!-- Drilldown table -->
+      <div
+        class="bg-white dark:bg-slate-900 rounded-lg border border-slate-200 dark:border-slate-800 p-4"
+      >
+        <div class="flex justify-between items-center mb-3">
+          <h3 class="text-base font-semibold">
+            {{ $t('ANALYTICS_REPORTS.CONVERSATION_DRILLDOWN') }}
+          </h3>
+          <span v-if="drilldown" class="text-xs text-slate-500">{{
+            $t('ANALYTICS_REPORTS.TOTAL', { count: drilldown.total })
+          }}</span>
+        </div>
+        <div class="overflow-x-auto">
+          <table class="min-w-full text-sm">
+            <thead>
+              <tr
+                class="text-left text-xs text-slate-500 border-b border-slate-200 dark:border-slate-800"
+              >
+                <th class="py-2 pr-3">{{ $t('ANALYTICS_REPORTS.COL_ID') }}</th>
+                <th class="py-2 pr-3">
+                  {{ $t('ANALYTICS_REPORTS.COL_CHANNEL') }}
+                </th>
+                <th class="py-2 pr-3">
+                  {{ $t('ANALYTICS_REPORTS.COL_ASSIGNEE') }}
+                </th>
+                <th class="py-2 pr-3">
+                  {{ $t('ANALYTICS_REPORTS.COL_STATUS') }}
+                </th>
+                <th class="py-2 pr-3">
+                  {{ $t('ANALYTICS_REPORTS.COL_DEAL') }}
+                </th>
+                <th class="py-2 pr-3">
+                  {{ $t('ANALYTICS_REPORTS.COL_MESSAGES') }}
+                </th>
+                <th class="py-2">{{ $t('ANALYTICS_REPORTS.COL_TAGS') }}</th>
+              </tr>
+            </thead>
+            <tbody>
+              <tr
+                v-for="c in drilldown?.conversations || []"
+                :key="c.id"
+                class="border-b border-slate-100 dark:border-slate-800"
+              >
+                <td class="py-2 pr-3">#{{ c.display_id }}</td>
+                <td class="py-2 pr-3">{{ c.channel }}</td>
+                <td class="py-2 pr-3">{{ c.assignee_id || '-' }}</td>
+                <td class="py-2 pr-3 capitalize">{{ c.status }}</td>
+                <td class="py-2 pr-3">
+                  {{ c.label_list.find(t => t.startsWith('deal:')) || '-' }}
+                </td>
+                <td class="py-2 pr-3">{{ c.message_count }}</td>
+                <td class="py-2 text-xs">{{ c.label_list.join(', ') }}</td>
+              </tr>
+              <tr v-if="drilldown && !drilldown.conversations.length">
+                <td colspan="7" class="py-6 text-center text-slate-400">
+                  {{ $t('ANALYTICS_REPORTS.NO_DATA') }}
+                </td>
+              </tr>
+            </tbody>
+          </table>
+        </div>
+      </div>
+    </div>
+  </div>
+</template>

app/javascript/dashboard/routes/dashboard/settings/reports/reports.routes.js

@@ -23,6 +23,7 @@ import CsatResponses from './CsatResponses.vue';
 import BotReports from './BotReports.vue';
 import LiveReports from './LiveReports.vue';
 import SLAReports from './SLAReports.vue';
+import AnalyticsReports from './AnalyticsReports.vue';
 
 const meta = {
   featureFlag: FEATURE_FLAGS.REPORTS,
@@ -152,6 +153,12 @@ export default {
           meta,
           component: BotReports,
         },
+        {
+          path: 'analytics',
+          name: 'analytics_reports',
+          meta,
+          component: AnalyticsReports,
+        },
       ],
     },
   ],

app/javascript/dashboard/routes/dashboard/settings/settings.routes.js

@@ -28,6 +28,7 @@ import security from './security/security.routes';
 import conversationWorkflow from './conversationWorkflow/conversationWorkflow.routes';
 import captain from './captain/captain.routes';
 import data from './data/data.routes';
+import productCatalog from './productCatalog/productCatalog.routes';
 
 export default {
   routes: [
@@ -71,5 +72,6 @@ export default {
     ...security.routes,
     ...conversationWorkflow.routes,
     ...captain.routes,
+    ...productCatalog.routes,
   ],
 };

app/jobs/analytics/daily_metrics_job.rb

@@ -0,0 +1,35 @@
+# Daily analytics aggregation driver (part2b-iii).
+#
+# Runs once daily (see config/schedule.yml) and pushes each account's closed-day
+# conversation metrics through Analytics::AccountDailyProcessor: classify every
+# active conversation via the LLM, write the classification tags onto the live
+# conversation (re-tagging prior M2-managed tags), and aggregate into the
+# immutable daily metrics tables.
+#
+# One account failing (e.g. an LLM/credential error) never blocks the others.
+class Analytics::DailyMetricsJob < ApplicationJob
+  queue_as :scheduled_jobs
+
+  def perform
+    Account.find_each do |account|
+      process_account(account)
+    rescue StandardError => e
+      Rails.logger.error("[Analytics::DailyMetricsJob] account=#{account.id} #{e.class}: #{e.message}")
+    end
+  end
+
+  private
+
+  def process_account(account)
+    date = closed_day_in_account_tz(account)
+    Analytics::AccountDailyProcessor.perform(account: account, date: date)
+  end
+
+  # The most recent fully-completed day in the account's reporting timezone.
+  def closed_day_in_account_tz(account)
+    tz = account.reporting_timezone.presence
+    zone = tz ? ActiveSupport::TimeZone[tz] : Time.zone
+    current_in_zone = zone ? Time.current.in_time_zone(zone) : Time.current
+    (current_in_zone - 1.day).to_date
+  end
+end

app/jobs/analytics/weekly_persona_evaluation_job.rb

@@ -0,0 +1,23 @@
+# Weekly driver: evaluates the persona for every account and delivers the summary
+# via Analytics::PersonaApprovalService (LINE primary, webhook fallback). Runs each
+# Monday 10:00 (see config/schedule.yml). Per-account failures are contained.
+class Analytics::WeeklyPersonaEvaluationJob < ApplicationJob
+  queue_as :default
+
+  def perform
+    Account.find_each do |account|
+      evaluate_and_deliver(account)
+    rescue StandardError => e
+      Rails.logger.error("[WeeklyPersonaEvaluation] account=#{account.id} #{e.class}: #{e.message}")
+    end
+  end
+
+  private
+
+  def evaluate_and_deliver(account)
+    evaluation = Analytics::WeeklyPersonaEvaluator.evaluate(account: account)
+    return if evaluation.disabled?
+
+    Analytics::PersonaApprovalService.deliver(account: account, evaluation: evaluation)
+  end
+end

app/jobs/hook_job.rb

@@ -6,6 +6,7 @@ class HookJob < MutexApplicationJob
   INTEGRATION_PROCESSORS = {
     'slack' => :process_slack_integration,
     'dialogflow' => :process_dialogflow_integration,
+    'chatbot' => :process_chatbot_integration,
     'google_translate' => :google_translate_integration,
     'leadsquared' => :process_leadsquared_integration_with_lock,
     'linear' => :process_linear_integration
@@ -50,6 +51,15 @@ class HookJob < MutexApplicationJob
     Integrations::Dialogflow::ProcessorService.new(event_name: event_name, hook: hook, event_data: event_data).perform
   end
 
+  def process_chatbot_integration(hook, event_name, event_data)
+    return unless event_name == 'message.created'
+
+    message = event_data[:message]
+    return unless message.content_type == 'text' && message.content.present?
+
+    Integrations::Chatbot::ProcessorService.new(event_name: event_name, hook: hook, event_data: event_data).perform
+  end
+
   def google_translate_integration(hook, event_name, event_data)
     return unless ['message.created'].include?(event_name)
 

app/listeners/hook_listener.rb

@@ -61,6 +61,7 @@ class HookListener < BaseListener
     supported_events_map = {
       'slack' => ['message.created', 'message.updated'],
       'dialogflow' => ['message.created', 'message.updated'],
+      'chatbot' => ['message.created'],
       'google_translate' => ['message.created'],
       'leadsquared' => ['contact.updated', 'conversation.created', 'conversation.resolved'],
       'linear' => ['message.created']

app/models/account.rb

@@ -86,6 +86,7 @@ class Account < ApplicationRecord
   has_many :tiktok_channels, dependent: :destroy_async, class_name: '::Channel::Tiktok'
   has_many :hooks, dependent: :destroy_async, class_name: 'Integrations::Hook'
   has_many :inboxes, dependent: :destroy_async
+  has_many :knowledge_base_faqs, dependent: :destroy_async
   has_many :labels, dependent: :destroy_async
   has_many :line_channels, dependent: :destroy_async, class_name: '::Channel::Line'
   has_many :mentions, dependent: :destroy_async
@@ -94,6 +95,7 @@ class Account < ApplicationRecord
   has_many :notification_settings, dependent: :destroy_async
   has_many :notifications, dependent: :destroy_async
   has_many :portals, dependent: :destroy_async, class_name: '::Portal'
+  has_many :product_catalog_entries, dependent: :destroy_async
   has_many :sms_channels, dependent: :destroy_async, class_name: '::Channel::Sms'
   has_many :teams, dependent: :destroy_async
   has_many :telegram_channels, dependent: :destroy_async, class_name: '::Channel::Telegram'

app/models/knowledge_base_faq.rb

@@ -0,0 +1,34 @@
+# == Schema Information
+#
+# Table name: knowledge_base_faqs
+#
+#  id              :bigint           not null, primary key
+#  account_id      :bigint           not null
+#  title           :string           not null
+#  content         :text             not null
+#  topic_tags      :jsonb            default: []       not null
+#  source_filename :string
+#  embedding       :vector(1536)
+#  created_at      :datetime         not null
+#  updated_at      :datetime         not null
+#
+class KnowledgeBaseFaq < ApplicationRecord
+  belongs_to :account
+  # pgvector KNN support (matches repo pattern: has_neighbors + nearest_neighbors)
+  has_neighbors :embedding, normalize: true
+
+  validates :title, presence: true
+  validates :content, presence: true
+
+  # topic_tags stored as jsonb array; expose string-list helpers for import/retrieval.
+  def topic_tag_list
+    Array(topic_tags)
+  end
+
+  def topic_tag_list=(value)
+    self.topic_tags = Array(value).map(&:strip).reject(&:blank?)
+  end
+
+  scope :for_account, ->(account) { where(account_id: account.id) }
+  scope :with_embedding, -> { where.not(embedding: nil) }
+end

app/services/analytics/account_daily_processor.rb

@@ -0,0 +1,218 @@
+# Aggregates one account's conversation activity into the immutable daily metrics
+# (conversation_daily_metrics + customer_daily_metrics) for a single date in the
+# account's reporting timezone, and writes the LLM classification tags onto the
+# live conversations.
+#
+# Flow per account/date:
+#   1. Compute the UTC day window from account.reporting_timezone (default UTC).
+#   2. Select conversations with activity inside that window.
+#   3. For each conversation, classify via Llm::AnalyticsClassifier (skipped when
+#      disabled), re-tag it (replace prior M2-managed tags, keep manual labels),
+#      and collect per-conversation / per-customer metrics.
+#   4. Upsert conversation_daily_metric (unique account+date) and
+#      customer_daily_metric (unique account+contact+date).
+#
+# Daily snapshots are immutable: counts captured here are a point-in-time view; the
+# live conversation tags may be re-tagged later without changing historical snapshots.
+class Analytics::AccountDailyProcessor
+  TOPIC_TAG_PREFIX = 'topic:'.freeze
+  DEAL_TAG_PREFIX = 'deal:'.freeze
+
+  # @param account [Account]
+  # @param date [Date] the reporting date in the account's timezone
+  def self.perform(account:, date:)
+    new(account: account, date: date).perform
+  end
+
+  def initialize(account:, date:)
+    @account = account
+    @date = date
+  end
+
+  def perform
+    return unless scope_enabled?
+
+    window = day_window
+    conversations = active_conversations(window)
+    return if conversations.empty?
+
+    catalog_paths = catalog_paths
+    metrics = build_metrics(conversations, window, catalog_paths)
+    upsert_metrics(metrics)
+  end
+
+  private
+
+  def scope_enabled?
+    @account.reporting_timezone.blank? || ActiveSupport::TimeZone[@account.reporting_timezone].present?
+  end
+
+  def timezone
+    @timezone ||= ActiveSupport::TimeZone[@account.reporting_timezone] || Time.zone
+  end
+
+  # [start_utc, end_utc) covering the account-local date.
+  def day_window
+    start_tz = timezone.parse(@date.to_s)
+    end_tz = timezone.parse((@date + 1).to_s)
+    [start_tz.utc, end_tz.utc]
+  end
+
+  # Conversations with activity (created or last activity) inside the account-local day.
+  def active_conversations(window)
+    start_utc, end_utc = window
+    @account.conversations
+            .where('created_at < ?', end_utc)
+            .where('created_at >= ? OR last_activity_at >= ?', start_utc, start_utc)
+  end
+
+  # All tag-hierarchy paths in the account's catalog — used to detect & replace
+  # previously-written product tags on re-classification.
+  def catalog_paths
+    @account.product_catalog_entries
+            .order(:group_name, :subgroup_name, :product_name)
+            .map(&:tag_hierarchy)
+            .map { |parts| parts.join('>') }
+  end
+
+  def build_metrics(conversations, _window, catalog_paths)
+    {
+      conversation: {
+        message_count: 0,
+        conversation_count: 0,
+        resolved_count: 0,
+        unresolved_count: 0,
+        top_tags: [],
+        sale_tags: [],
+        deal_outcomes: {},
+        agent_breakdown: Hash.new(0),
+        team_breakdown: Hash.new(0),
+        channel_breakdown: Hash.new(0),
+        inbox_breakdown: Hash.new(0)
+      },
+      customers: {}
+    }.tap do |acc|
+      conversations.each { |conversation| accumulate(acc, conversation, catalog_paths) }
+    end
+  end
+
+  def accumulate(acc, conversation, catalog_paths)
+    classification = Llm::AnalyticsClassifier.classify(account: @account, conversation: conversation)
+    return if classification.error.present? || classification.disabled?
+
+    apply_tags(conversation, classification, catalog_paths)
+
+    cm = acc[:conversation]
+    cm[:conversation_count] += 1
+    message_count = chat_message_count(conversation)
+    cm[:message_count] += message_count
+
+    if conversation.resolved?
+      cm[:resolved_count] += 1
+    else
+      cm[:unresolved_count] += 1
+    end
+
+    cm[:top_tags] |= (classification.topics + classification.products).compact
+    cm[:sale_tags] |= classification.products
+    cm[:deal_outcomes]['totals'] ||= {}
+    cm[:deal_outcomes]['totals'][classification.deal] = cm[:deal_outcomes]['totals'][classification.deal].to_i + 1
+
+    cm[:agent_breakdown][conversation.assignee_id] += 1 if conversation.assignee_id
+    cm[:team_breakdown][conversation.team_id] += 1 if conversation.team_id
+    cm[:channel_breakdown][channel_key(conversation)] += 1
+    cm[:inbox_breakdown][conversation.inbox_id] += 1 if conversation.inbox_id
+
+    customer_key = conversation.contact_id
+    customer = acc[:customers][customer_key] ||= {
+      message_count: 0, conversation_count: 0, resolved_count: 0, unresolved_count: 0,
+      top_tags: [], deal_outcomes: {}, agent_ids: []
+    }
+    customer[:conversation_count] += 1
+    customer[:message_count] += message_count
+    if conversation.resolved?
+      customer[:resolved_count] += 1
+    else
+      customer[:unresolved_count] += 1
+    end
+    customer[:top_tags] |= (classification.topics + classification.products).compact
+    customer[:deal_outcomes][classification.deal] = customer[:deal_outcomes][classification.deal].to_i + 1
+    customer[:agent_ids] = (customer[:agent_ids] | [conversation.assignee_id]).compact if conversation.assignee_id
+  end
+
+  def chat_message_count(conversation)
+    conversation.messages.chat.count
+  end
+
+  def channel_key(conversation)
+    conversation.inbox&.channel_type&.demodulize
+  end
+
+  # Write classifier tags onto the conversation: remove any prior M2-managed tags
+  # (topic:/deal: prefixes + current catalog paths), then add the fresh ones.
+  # Manual labels are preserved.
+  def apply_tags(conversation, classification, catalog_paths)
+    current = conversation.label_list.to_a
+    m2_managed = current.select do |tag|
+      tag.start_with?(TOPIC_TAG_PREFIX) || tag.start_with?(DEAL_TAG_PREFIX) || catalog_paths.include?(tag)
+    end
+    next_tags = current - m2_managed
+
+    next_tags += classification.products
+    next_tags += classification.topics.map { |topic| "#{TOPIC_TAG_PREFIX}#{topic}" }
+    # Persist the deal outcome on the live conversation so reports can filter by it.
+    next_tags << "#{DEAL_TAG_PREFIX}#{classification.deal}" if classification.deal.present?
+
+    conversation.update!(label_list: next_tags.uniq)
+  rescue ActiveRecord::RecordInvalid => e
+    Rails.logger.error("[Analytics] conversation #{conversation.id} tag update failed: #{e.message}")
+  end
+
+  def upsert_metrics(metrics)
+    conv = metrics[:conversation]
+    conv_attrs = {
+      account_id: @account.id,
+      date: @date,
+      timezone: @account.reporting_timezone.presence || 'UTC',
+      message_count: conv[:message_count],
+      conversation_count: conv[:conversation_count],
+      resolved_count: conv[:resolved_count],
+      unresolved_count: conv[:unresolved_count],
+      top_tags: conv[:top_tags],
+      sale_tags: conv[:sale_tags],
+      deal_outcomes: conv[:deal_outcomes],
+      agent_breakdown: conv[:agent_breakdown].map { |id, count| { 'agent_id' => id, 'count' => count } },
+      team_breakdown: conv[:team_breakdown].map { |id, count| { 'team_id' => id, 'count' => count } },
+      channel_breakdown: conv[:channel_breakdown].map { |ch, count| { 'channel' => ch, 'count' => count } },
+      inbox_breakdown: conv[:inbox_breakdown].map { |id, count| { 'inbox_id' => id, 'count' => count } }
+    }
+
+    ConversationDailyMetric.upsert(
+      conv_attrs,
+      unique_by: %i[account_id date],
+      update_only: %i[message_count conversation_count resolved_count unresolved_count top_tags sale_tags
+                      deal_outcomes agent_breakdown team_breakdown channel_breakdown inbox_breakdown timezone]
+    )
+
+    metrics[:customers].each do |contact_id, data|
+      CustomerDailyMetric.upsert(
+        {
+          account_id: @account.id,
+          contact_id: contact_id,
+          date: @date,
+          timezone: @account.reporting_timezone.presence || 'UTC',
+          message_count: data[:message_count],
+          conversation_count: data[:conversation_count],
+          resolved_count: data[:resolved_count],
+          unresolved_count: data[:unresolved_count],
+          top_tags: data[:top_tags],
+          deal_outcomes: data[:deal_outcomes],
+          agent_ids: data[:agent_ids].to_a
+        },
+        unique_by: %i[account_id contact_id date],
+        update_only: %i[message_count conversation_count resolved_count unresolved_count top_tags
+                        deal_outcomes agent_ids timezone]
+      )
+    end
+  end
+end

app/services/analytics/drilldown_service.rb

@@ -0,0 +1,107 @@
+# Deep, filterable admin drilldown over LIVE conversations (phase 2).
+#
+# Unlike Analytics::ReportService (which reads the immutable per-day snapshots),
+# this queries the actual Conversation records scoped to a date range, so the
+# filters the M2 report needs (agent / team / inbox / channel / tag / deal) can
+# be applied at per-conversation granularity, with per-customer and per-agent
+# cross-breakdowns and pagination.
+#
+# Supported filters (all optional, applied with AND):
+#   :since   (Date)  — conversations active on/after (created_at or last_activity_at)
+#   :until   (Date)  — conversations active before this date (exclusive end)
+#   :agent_id (Integer)
+#   :team_id  (Integer)
+#   :inbox_id (Integer)
+#   :channel  (String, e.g. "Channel::WebWidget") — matched against inbox.channel_type
+#   :tag      (String) — matches any tag in the conversation label_list (exact)
+#   :deal     (String, "won"|"lost"|"undecided") — matches the "deal:<value>" tag
+#   :page, :per_page — pagination (default 1 / 25)
+#
+# Admin-only; access is enforced by the calling controller/policy.
+class Analytics::DrilldownService
+  DEFAULT_PER_PAGE = 25
+
+  # @param account [Account]
+  # @param filters [Hash]
+  def self.build(account:, filters: {})
+    new(account: account, filters: filters).build
+  end
+
+  def initialize(account:, filters: {})
+    @account = account
+    @filters = filters.symbolize_keys
+  end
+
+  def build
+    scope = filtered_scope
+    {
+      total: scope.count,
+      page: page,
+      per_page: per_page,
+      conversations: scope.offset((page - 1) * per_page).limit(per_page).map { |c| serialize(c) }
+    }
+  end
+
+  private
+
+  attr_reader :account, :filters
+
+  def page = (filters[:page] || 1).to_i
+  def per_page = (filters[:per_page] || DEFAULT_PER_PAGE).to_i.clamp(1, 100)
+
+  def filtered_scope
+    scope = account.conversations
+
+    if filters[:since].present?
+      start = filters[:since]
+      scope = scope.where('created_at >= ?', start)
+    end
+    if filters[:until].present?
+      scope = scope.where('created_at < ?', filters[:until] + 1)
+    end
+
+    scope = scope.where(assignee_id: filters[:agent_id]) if filters[:agent_id].present?
+    scope = scope.where(team_id: filters[:team_id]) if filters[:team_id].present?
+    scope = scope.where(inbox_id: filters[:inbox_id]) if filters[:inbox_id].present?
+
+    if filters[:channel].present?
+      scope = scope.joins(:inbox).where(inboxes: { channel_type: filters[:channel] })
+    end
+
+    if filters[:tag].present?
+      tag = filters[:tag].to_s
+      matches = account.labels.where(title: tag).pluck(:id)
+      # No conversation can carry a label that doesn't exist — return none.
+      return scope.none if matches.empty?
+
+      scope = scope.joins(:labels).where('labels.id IN (?)', matches)
+    end
+
+    if filters[:deal].present?
+      deal_tag = "deal:#{filters[:deal]}"
+      matches = account.labels.where(title: deal_tag).pluck(:id)
+      # No conversation can carry a deal tag that doesn't exist — return none.
+      return scope.none if matches.empty?
+
+      scope = scope.joins(:labels).where('labels.id IN (?)', matches)
+    end
+
+    scope.order(:created_at)
+  end
+
+  def serialize(conversation)
+    {
+      id: conversation.id,
+      display_id: conversation.display_id,
+      contact_id: conversation.contact_id,
+      assignee_id: conversation.assignee_id,
+      team_id: conversation.team_id,
+      inbox_id: conversation.inbox_id,
+      channel: conversation.inbox&.channel_type,
+      status: conversation.status,
+      created_at: conversation.created_at,
+      label_list: conversation.label_list,
+      message_count: conversation.messages.chat.count
+    }
+  end
+end

app/services/analytics/persona_approval_service.rb

@@ -0,0 +1,200 @@
+# Approval-flow delivery for the weekly persona evaluation (phase 3).
+#
+# Delivers the weekly LLM summary + recommendations to the admin via LINE
+# (primary, using quick-reply buttons), Telegram (inline keyboard), or a webhook
+# (fallback), then records the admin's decision. The full persona system prompt is
+# intentionally NOT included in the delivery body — it is only sent to the webhook
+# AFTER the admin approves.
+#
+# Delivery configuration lives on Account#custom_attributes (jsonb):
+#   persona_line_user_id      - the admin's LINE user id (for LINE push) [primary]
+#   persona_telegram_chat_id  - the admin's Telegram chat id (for Telegram push)
+#   persona_webhook_url       - fallback webhook URL
+#   persona_webhook_secret    - shared secret signed into the webhook body / header
+#
+# Decision keys match LINE quick-reply / webhook actions:
+#   'approve' | 'reject' | 'view_full_prompt'
+class Analytics::PersonaApprovalService
+  # @param account [Account]
+  # @param evaluation [Analytics::WeeklyPersonaEvaluator::Result]
+  # @return [Hash] { delivered:, channel:, error: }
+  def self.deliver(account:, evaluation:)
+    new(account: account, evaluation: evaluation).deliver
+  end
+
+  # Sends an admin approval decision to the configured webhook (webhook-only, never
+  # LINE). Used by the :approve path to inform the integration about the decision.
+  # @return [Hash] { delivered:, channel: 'webhook', error: }
+  def self.notify_approval(account:, evaluation:, decision:)
+    new(account: account, evaluation: evaluation).notify_approval(decision)
+  end
+
+  def initialize(account:, evaluation:)
+    @account = account
+    @evaluation = evaluation
+  end
+
+  def deliver
+    return { delivered: false, error: 'no evaluation to deliver' } if @evaluation.nil? || @evaluation.disabled?
+
+    if line_delivery_available?
+      deliver_via_line
+    elsif telegram_delivery_available?
+      deliver_via_telegram
+    elsif webhook_delivery_available?
+      deliver_via_webhook
+    else
+      { delivered: false, channel: nil, error: 'no delivery channel configured (set persona_line_user_id, persona_telegram_chat_id or persona_webhook_url)' }
+    end
+  end
+
+  # Sends an admin approval decision to the configured webhook. Webhook-only (never
+  # LINE) — used by the :approve path so the approval is not re-sent as a LINE card.
+  def notify_approval(decision)
+    return { delivered: false, channel: 'webhook', error: 'no webhook configured' } unless webhook_delivery_available?
+
+    post_to_webhook(
+      type: 'persona_decision',
+      decision: decision
+    )
+  end
+
+  private
+
+  attr_reader :account
+
+  # -- LINE (primary) ----------------------------------------------------------
+
+  def line_delivery_available?
+    line_user_id.present? && account.line_channels.present?
+  end
+
+  def deliver_via_line
+    channel = account.line_channels.first
+    channel.client.push_message(
+      line_user_id,
+      build_line_payload
+    )
+    { delivered: true, channel: 'line' }
+  rescue StandardError => e
+    Rails.logger.error("[PersonaApproval] LINE push failed: #{e.message}")
+    { delivered: false, channel: 'line', error: e.message }
+  end
+
+  def build_line_payload
+    {
+      type: 'text',
+      text: line_text,
+      quickReply: {
+        items: [
+          quick_reply_item('👍 Approve', 'approve'),
+          quick_reply_item('Reject', 'reject'),
+          quick_reply_item('👁 View full prompt', 'view_full_prompt')
+        ]
+      }
+    }
+  end
+
+  def quick_reply_item(label, key)
+    {
+      type: 'action',
+      action: {
+        type: 'message',
+        label: label,
+        text: "persona:#{key}"
+      }
+    }
+  end
+
+  def line_text
+    summary = @evaluation.summary.to_s
+    recommendations = Array(@evaluation.recommendations)
+    [summary, '', *recommendations.map { |r| "• #{r}" }].join("\n")
+  end
+
+  # -- Telegram ----------------------------------------------------------------
+
+  def telegram_delivery_available?
+    telegram_chat_id.present? && account.telegram_channels.present?
+  end
+
+  def deliver_via_telegram
+    channel = account.telegram_channels.first
+    response = HTTParty.post(
+      "#{channel.telegram_api_url}/sendMessage",
+      body: {
+        chat_id: telegram_chat_id,
+        text: line_text,
+        reply_markup: build_telegram_keyboard
+      }
+    )
+    success = response.success?
+    Rails.logger.error("[PersonaApproval] Telegram send failed: #{response.parsed_response}") unless success
+    { delivered: success, channel: 'telegram', error: success ? nil : 'telegram send failed' }
+  rescue StandardError => e
+    Rails.logger.error("[PersonaApproval] Telegram delivery failed: #{e.message}")
+    { delivered: false, channel: 'telegram', error: e.message }
+  end
+
+  # Inline keyboard with the three decision buttons (callback_data = persona:<key>)
+  def build_telegram_keyboard
+    {
+      inline_keyboard: [
+        [
+          { text: '👍 Approve', callback_data: 'persona:approve' },
+          { text: 'Reject', callback_data: 'persona:reject' },
+          { text: '👁 View full prompt', callback_data: 'persona:view_full_prompt' }
+        ]
+      ]
+    }.to_json
+  end
+
+  # -- Webhook (fallback) ------------------------------------------------------
+
+  def webhook_delivery_available?
+    webhook_url.present?
+  end
+
+  # Shared webhook POST: builds the payload from the evaluation plus extra fields.
+  def post_to_webhook(extra_fields)
+    body = {
+      summary: @evaluation.summary,
+      recommendations: Array(@evaluation.recommendations)
+    }.merge(extra_fields)
+    response = HTTParty.post(
+      webhook_url,
+      body: body.to_json,
+      headers: webhook_headers
+    )
+    success = response.success?
+    Rails.logger.error("[PersonaApproval] webhook #{response.code}") unless success
+    { delivered: success, channel: 'webhook', error: success ? nil : "webhook responded #{response.code}" }
+  rescue StandardError => e
+    Rails.logger.error("[PersonaApproval] webhook failed: #{e.message}")
+    { delivered: false, channel: 'webhook', error: e.message }
+  end
+
+  def deliver_via_webhook
+    post_to_webhook(
+      type: 'persona_evaluation',
+      actions: %w[approve reject view_full_prompt]
+    )
+  end
+
+  def webhook_headers
+    headers = { 'Content-Type' => 'application/json' }
+    headers['X-Persona-Signature'] = signature if webhook_secret.present?
+    headers
+  end
+
+  def signature
+    OpenSSL::HMAC.hexdigest('sha256', webhook_secret, @evaluation.summary.to_s)
+  end
+
+  # -- config ------------------------------------------------------------------
+
+  def line_user_id = account.custom_attributes['persona_line_user_id']
+  def telegram_chat_id = account.custom_attributes['persona_telegram_chat_id']
+  def webhook_url = account.custom_attributes['persona_webhook_url']
+  def webhook_secret = account.custom_attributes['persona_webhook_secret']
+end

app/services/analytics/product_catalog_import_service.rb

@@ -0,0 +1,228 @@
+# Imports product catalog entries for an account from pasted text, CSV/TSV, or an
+# uploaded .xlsx spreadsheet (phase 3, admin-only).
+#
+# Accepted formats:
+#   - copy/paste: one product per line, columns separated by tab or '|'
+#   - CSV: standard RFC4180 with a header row
+#   - XLSX: an uploaded spreadsheet opened via the `roo` gem (file_path:)
+# Columns (in order): group, subgroup, product, display, aliases
+#   group, product are required; subgroup/display optional; aliases separator is
+#   comma or semicolon.
+#
+# Hierarchical semantics: a row defines one product leaf. The classifier matches
+# from the lowest level first and tags ancestors, so only leaf rows are stored;
+# no need to also store group/subgroup as standalone rows.
+#
+# Upsert semantics: same (account_id, group_name, product_name) is replaced with
+# the latest row (aliases/subgroup/display overwritten), keeping the catalog unique.
+#
+# Returns a Hash: { imported: n, updated: n, errors: [{ line, message }] }.
+class Analytics::ProductCatalogImportService
+  COLUMNS = %w[group_name subgroup_name product_name display_name aliases].freeze
+  COLUMN_ALIASES = {
+    'group' => 'group_name', 'product' => 'product_name',
+    'subgroup' => 'subgroup_name', 'display' => 'display_name'
+  }.freeze
+  REQUIRED = %w[group_name product_name].freeze
+  ALIAS_SPLIT = /[,;]/
+
+  # @param account [Account]
+  # @param content [String] raw pasted text or file content (CSV/TSV/pipe)
+  # @return [Hash]
+  def self.import(account:, content:)
+    new(account: account, content: content).import
+  end
+
+  # @param account [Account]
+  # @param file_path [String] path to an uploaded .xlsx/.ods/.csv file (via roo)
+  # @param filename [String, nil] original upload filename (used to detect extension
+  #   when the temp path has none, e.g. RackMultipart tempfile)
+  # @return [Hash]
+  def self.import_file(account:, file_path:, filename: nil)
+    new(account: account, content: nil, file_path: file_path, filename: filename).import
+  end
+
+  def initialize(account:, content: nil, file_path: nil, filename: nil)
+    @account = account
+    @content = content.to_s
+    @file_path = file_path
+    @filename = filename
+  end
+
+  def import
+    rows = (@file_path ? parse_spreadsheet : parse_rows)
+    return { imported: 0, updated: 0, errors: rows[:errors] } if rows[:data].empty?
+
+    result = upsert_rows(rows[:data])
+    result.merge(errors: rows[:errors])
+  end
+
+  private
+
+  attr_reader :account
+
+  # Convert an .xlsx/.xls/.ods sheet into the same { data, errors } shape as
+  # parse_rows by serializing each row into a pipe-delimited text line, then
+  # reusing the shared row parser. The first row is treated as a header when it
+  # looks like declared column names (same detection as text input).
+  def parse_spreadsheet
+    require 'roo'
+    sheet = Roo::Spreadsheet.open(@file_path, extension: extension).sheet(0)
+    lines = (1..sheet.last_row).filter_map do |idx|
+      row = (1..sheet.last_column).map { |col| sheet.cell(idx, col).to_s }
+      row.join('|') unless row.all?(&:blank?)
+    end
+    parse_rows_from_lines(lines)
+  rescue LoadError
+    # roo gem unavailable (should be resolved after bundle install).
+    Rails.logger.error('[ProductCatalogImport] roo gem not available; cannot read spreadsheet')
+    { data: [], errors: [{ line: 1, message: 'spreadsheet reading is unavailable (roo gem not installed)' }] }
+  rescue StandardError => e
+    Rails.logger.error("[ProductCatalogImport] xlsx parse failed: #{e.message}")
+    { data: [], errors: [{ line: 1, message: "could not read spreadsheet: #{e.message}" }] }
+  end
+
+  def extension
+    # Prefer the original upload filename (Rack tempfiles lack a useful extension);
+    # fall back to the path if no filename was supplied.
+    name = @filename.presence || @file_path.to_s
+    File.extname(name).delete('.').presence
+  end
+
+  # Shared row->attrs pipeline used by both text and spreadsheet input.
+  def parse_rows_from_lines(lines)
+    return { data: [], errors: [] } if lines.empty?
+
+    header_columns = detect_header_columns(lines)
+    line_offset = header_columns ? 1 : 0
+    keyword_mode = lines.any? { |line| keyword_line?(line) }
+
+    data = []
+    errors = []
+    lines.each_with_index do |line, idx|
+      raw = split_line(line)
+      attrs = row_to_attrs(raw, header_columns: header_columns, keyword: keyword_mode, line_no: idx + 1 + line_offset)
+      if attrs.is_a?(Hash)
+        data << attrs
+      else
+        errors << { line: idx + 1 + line_offset, message: attrs }
+      end
+    end
+    { data: data, errors: errors }
+  end
+
+  # @return [Hash] { data: [attrs...], errors: [{ line:, message: }] }
+  def parse_rows
+    lines = @content.strip.split(/\r?\n/).reject(&:blank?)
+    parse_rows_from_lines(lines)
+  end
+
+  # If the first line looks like a declared column header (e.g. "group,product" or
+  # "group|subgroup|product|display|aliases", accepting group/group_name and
+  # product/product_name), return the normalized column names so data rows are
+  # mapped by name. Returns nil otherwise.
+  def detect_header_columns(lines)
+    first = split_line(lines.first).map { |v| v.to_s.strip.downcase }
+    normalized = first.map { |col| COLUMN_ALIASES.fetch(col, col) }
+    return nil unless normalized.all? { |col| COLUMNS.include?(col) }
+
+    lines.shift
+    normalized
+  end
+
+  def keyword_line?(line)
+    line =~ /(?:^|[\t\|\s])(group_name|group|product_name|product|subgroup_name|display_name|aliases)\s*:/
+  end
+
+  # Split a line by a consistent delimiter. If tabs present -> TSV; else '|' -> pipe; else comma -> CSV.
+  def split_line(line)
+    if line.include?("\t")
+      line.split("\t")
+    elsif line.include?('|')
+      line.split('|')
+    else
+      CSV.parse_line(line) || []
+    end
+  end
+
+  def row_to_attrs(raw, header_columns:, keyword:, line_no:)
+    values = raw.map(&:to_s).map(&:strip)
+    values = values.map(&:presence).compact
+    return 'empty row' if values.empty?
+
+    attrs =
+      if keyword
+        keyword_row_to_attrs(values)
+      elsif header_columns
+        header_row_to_attrs(raw, header_columns)
+      else
+        ordered_row_to_attrs(values)
+      end
+
+    return attrs if attrs.is_a?(String)
+
+    missing = REQUIRED.select { |col| attrs[col].blank? }
+    return "line #{line_no}: missing required column(s): #{missing.join(', ')}" unless missing.empty?
+
+    attrs['aliases'] = normalize_aliases(attrs['aliases'])
+    attrs
+  end
+
+  # Field form: "group: A | product: X | subgroup: S | aliases: a,b"
+  def keyword_row_to_attrs(values)
+    attrs = {}
+    values.each do |pair|
+      key, _, value = pair.partition(':')
+      normalized = COLUMN_ALIASES.fetch(key.strip.downcase, key.strip.downcase)
+      attrs[normalized] = value.strip.presence
+    end
+    attrs
+  end
+
+  # Named-header form: header tells us which column is which.
+  def header_row_to_attrs(raw, header_columns)
+    attrs = {}
+    raw.each_with_index do |value, i|
+      col = header_columns[i]
+      attrs[col] = value.presence if col
+    end
+    attrs
+  end
+
+  # Ordered form: "group, product" (no subgroup) or "group, subgroup, product".
+  def ordered_row_to_attrs(values)
+    if values.length == 2
+      { 'group_name' => values[0], 'product_name' => values[1] }
+    else
+      # group, subgroup, product, [display], [aliases]
+      { 'group_name' => values[0], 'subgroup_name' => values[1], 'product_name' => values[2],
+        'display_name' => values[3], 'aliases' => values[4] }
+    end
+  end
+
+  def normalize_aliases(value)
+    value.to_s.split(ALIAS_SPLIT).map(&:strip).reject(&:blank?)
+  end
+
+  def upsert_rows(rows)
+    imported = 0
+    updated = 0
+    rows.each do |attrs|
+      existing = @account.product_catalog_entries.find_by(
+        group_name: attrs['group_name'], product_name: attrs['product_name']
+      )
+      if existing
+        existing.update!(
+          subgroup_name: attrs['subgroup_name'],
+          display_name: attrs['display_name'],
+          aliases: attrs['aliases']
+        )
+        updated += 1
+      else
+        @account.product_catalog_entries.create!(attrs.merge(account_id: @account.id))
+        imported += 1
+      end
+    end
+    { imported: imported, updated: updated }
+  end
+end

app/services/analytics/report_service.rb

@@ -0,0 +1,113 @@
+# Admin analytics summary/rollup over the immutable per-day metrics (phase 2).
+#
+# Reads conversation_daily_metrics + customer_daily_metrics (written by the
+# Analytics::AccountDailyProcessor batch) and returns:
+#   - summary    : totals over the date range
+#   - timeseries : the same totals bucketed per day (for charting)
+#   - customers  : per-customer totals (from customer_daily_metrics)
+#   - agents     : per-agent conversation counts (from agent_breakdown)
+#
+# This is the snapshot/rollup surface. Deep per-conversation filtering
+# (agent/team/inbox/channel/tag/deal) lives in Analytics::DrilldownService, which
+# reads live conversations. Admin-only; access is enforced by the controller/policy.
+class Analytics::ReportService
+  # @param account [Account]
+  # @param since [Date]
+  # @param until_date [Date]
+  def self.build(account:, since: nil, until_date: nil)
+    new(account: account, since: since, until_date: until_date).build
+  end
+
+  def initialize(account:, since: nil, until_date: nil)
+    @account = account
+    @since = since
+    @until_date = until_date
+  end
+
+  def build
+    {
+      summary: summary,
+      timeseries: timeseries,
+      customers: customers,
+      agents: agents
+    }
+  end
+
+  private
+
+  attr_reader :account
+
+  def since
+    @since || ConversationDailyMetric.where(account_id: account.id).minimum(:date) || Date.today
+  end
+
+  def until_date
+    @until_date || Date.today
+  end
+
+  def date_range = (since..until_date)
+
+  def conversation_rows
+    @conversation_rows ||= ConversationDailyMetric
+                           .where(account_id: account.id)
+                           .where(date: date_range)
+                           .order(:date)
+  end
+
+  def customer_rows
+    @customer_rows ||= CustomerDailyMetric
+                       .where(account_id: account.id)
+                       .where(date: date_range)
+                       .order(:date, :contact_id)
+  end
+
+  # Aggregate a set of rows into normalized totals.
+  def totals(rows)
+    {
+      conversation_count: rows.sum { |r| r.conversation_count },
+      message_count: rows.sum { |r| r.message_count },
+      resolved_count: rows.sum { |r| r.resolved_count },
+      unresolved_count: rows.sum { |r| r.unresolved_count },
+      deal_outcomes: merge_deal_outcomes(rows),
+      top_tags: merge_top_tags(rows)
+    }
+  end
+
+  def summary = totals(conversation_rows)
+
+  def timeseries
+    conversation_rows.group_by(&:date).map { |date, rows| totals(rows).merge(date: date) }
+  end
+
+  def customers
+    customer_rows.group_by(&:contact_id).map do |contact_id, rows|
+      totals(rows).merge(contact_id: contact_id)
+    end
+  end
+
+  def agents
+    per_agent = Hash.new { |h, k| h[k] = 0 }
+    conversation_rows.each do |row|
+      Array(row.agent_breakdown).each { |entry| per_agent[entry['agent_id']] += entry['count'].to_i }
+    end
+    per_agent.map { |agent_id, count| { agent_id: agent_id, conversation_count: count } }
+             .sort_by { |entry| -entry[:conversation_count] }
+  end
+
+  # conversation rows store deal_outcomes nested under 'totals'; customer rows flat.
+  # Normalize both into a flat { deal => count } mapping.
+  def merge_deal_outcomes(rows)
+    rows.each_with_object({}) do |row, acc|
+      row.deal_outcomes.each do |key, value|
+        data = value.is_a?(Hash) ? value : { key => value }
+        data.each { |deal, count| acc[deal] = (acc[deal] || 0) + count.to_i }
+      end
+    end
+  end
+
+  def merge_top_tags(rows)
+    rows.each_with_object(Hash.new(0)) do |row, acc|
+      Array(row.top_tags).each { |tag| acc[tag] += 1 }
+    end.sort_by { |_tag, count| -count }.to_h
+  end
+end

app/services/analytics/weekly_persona_evaluator.rb

@@ -0,0 +1,123 @@
+# Weekly persona evaluation for the self-improving chatbot (phase 3).
+#
+# Summarizes the last 7 days of immutable daily metrics (Analytics::ReportService /
+# ConversationDailyMetric + CustomerDailyMetric via Analytics::ReportService) and asks the
+# LLM (via the Llm::Resolver cascade) to recommend persona/system-prompt improvements.
+#
+# Output is the SUMMARY ONLY (human-readable recommendation) — the full system prompt is
+# intentionally NOT produced/revealed here; it is gated behind the admin approval flow.
+#
+# Like Llm::AnalyticsClassifier, this is a pure evaluator: it CLASSIFIES/SUMMARIZES and
+# returns a Result; persisting the recommendation is the caller's responsibility
+# (the weekly job / approval flow). Fail-closed: no LLM credential -> { disabled: true },
+# never sends conversation content when disabled.
+module Analytics::WeeklyPersonaEvaluator
+  SCHEMA = {
+    type: 'object',
+    additionalProperties: false,
+    properties: {
+      summary: {
+        type: 'string',
+        description: 'A concise human-readable summary of the week: top topics, sales wins/losses, and any notable trends.'
+      },
+      recommendations: {
+        type: 'array',
+        items: { type: 'string' },
+        description: 'Concrete, actionable recommendations to improve the chatbot persona/behavior next week.'
+      }
+    },
+    required: %w[summary recommendations]
+  }.freeze
+
+  Result = Struct.new(:summary, :recommendations, :disabled, :error, keyword_init: true) do
+    def disabled?
+      disabled == true
+    end
+
+    def success?
+      error.nil?
+    end
+  end
+
+  WINDOW_DAYS = 7
+
+  module_function
+
+  # @param account [Account]
+  # @param report [Hash] output of Analytics::ReportService.build (or built here if nil)
+  # @return [Analytics::WeeklyPersonaEvaluator::Result]
+  def evaluate(account:, report: nil)
+    credential = Llm::Resolver.resolve(account)
+    return disabled_result if credential.nil?
+
+    report ||= Analytics::ReportService.build(account: account, since: WINDOW_DAYS.days.ago.to_date, until_date: Date.today)
+    response = call_llm(credential, build_prompt(report))
+    build_result(response)
+  rescue StandardError => e
+    Rails.logger.error("[WeeklyPersonaEvaluator] account=#{account&.id} #{e.class}: #{e.message}")
+    Result.new(error: e.message)
+  end
+
+  # -- result helpers ---------------------------------------------------------
+
+  def disabled_result
+    Result.new(summary: nil, recommendations: [], disabled: true)
+  end
+
+  def build_result(response)
+    return Result.new(error: response[:error] || 'evaluation failed') if response[:error]
+
+    parsed = JSON.parse(sanitize_json(response[:content]))
+    Result.new(
+      summary: parsed['summary'],
+      recommendations: Array(parsed['recommendations']),
+      disabled: false
+    )
+  rescue JSON::ParserError, TypeError
+    Result.new(error: 'LLM returned an unparsable evaluation')
+  end
+
+  # -- LLM call ---------------------------------------------------------------
+
+  def call_llm(credential, prompt)
+    Llm::Config.with_api_key(credential[:api_key], api_base: credential[:api_base]) do |context|
+      chat = context.chat(model: MODEL).with_schema(SCHEMA)
+      chat.with_instructions(SYSTEM_PROMPT)
+      { content: chat.ask(prompt).content }
+    end
+  rescue StandardError => e
+    Rails.logger.error("[WeeklyPersonaEvaluator] LLM call failed #{e.class}: #{e.message}")
+    { error: e.message }
+  end
+
+  MODEL = Llm::Config::DEFAULT_MODEL
+
+  # -- prompt construction ----------------------------------------------------
+
+  def build_prompt(report)
+    summary = report[:summary].to_h
+    [
+      'Here is the past week of customer-service analytics for the account:',
+      '',
+      "Conversations: #{summary[:conversation_count]}",
+      "Messages: #{summary[:message_count]}",
+      "Resolved: #{summary[:resolved_count]}",
+      "Unresolved: #{summary[:unresolved_count]}",
+      "Deal outcomes: #{summary[:deal_outcomes].inspect}",
+      "Top tags: #{summary[:top_tags].inspect}",
+      '',
+      'Based on this, recommend persona / behavior improvements for the chatbot.'
+    ].join("\n")
+  end
+
+  def sanitize_json(content)
+    content.to_s.gsub('```json', '').gsub('```', '').strip
+  end
+
+  SYSTEM_PROMPT = <<~PROMPT.freeze
+    You are a customer-service improvement analyst. Given a week of aggregate metrics,
+    write a concise summary and 2-5 concrete, actionable recommendations to improve the
+    chatbot's persona and behavior. Keep recommendations specific and grounded in the data.
+    Return only the JSON object described by the schema — no extra text.
+  PROMPT
+end

app/services/chatbot/config_service.rb

@@ -0,0 +1,86 @@
+# Per-account chatbot configuration for the self-contained OSS chatbot.
+#
+# Reads/writes the account's chatbot settings in Account#custom_attributes (jsonb).
+# The chatbot processor + decision service read from here so there's a single source
+# of truth for how the bot behaves for a given account.
+#
+# Keys (namespaced 'chatbot_*'):
+#   chatbot_enabled            [bool] gates whether the chatbot replies (default false)
+#   chatbot_system_prompt      [String] the persona/system instructions for the answer LLM
+#   chatbot_guardrail_prompt   [String] the allowed-scope / guardrail instructions
+#   chatbot_out_of_scope_reply [String] templated refusal reply for out-of-scope messages
+#   chatbot_call_mode          [Integer] 1 (one call) or 2 (guardrail + answer) — default 1
+#
+# Guardrail + out-of-scope reply have safe Thai defaults; system prompt and call mode
+# fall back to built-in values when unset.
+module Chatbot::ConfigService
+  KEYS = %w[
+    chatbot_enabled chatbot_system_prompt chatbot_guardrail_prompt
+    chatbot_out_of_scope_reply chatbot_call_mode
+  ].freeze
+
+  DEFAULT_SYSTEM_PROMPT = <<~PROMPT.freeze
+    You are a helpful customer-service assistant for this business. Answer using ONLY the
+    provided knowledge base and conversation history. Be concise, accurate and polite.
+    If the knowledge base does not contain the answer, say you are not sure and offer to
+    connect the customer with a support agent. Do not invent facts.
+  PROMPT
+
+  DEFAULT_GUARDRAIL_PROMPT = <<~PROMPT.freeze
+    The bot answers questions about this business's PRODUCTS, SERVICES, and related support
+    topics only. It must NOT answer unrelated or off-topic requests (e.g. personal advice,
+    fortune-telling, horoscopes, unrelated general knowledge).
+  PROMPT
+
+  DEFAULT_OUT_OF_SCOPE_REPLY = 'ขออภัยครับ คำถามนี้อยู่นอกขอบเขตที่เราสามารถให้บริการได้ กรุณาสอบถามเรื่องสินค้าและบริการของเรา'
+
+  module_function
+
+  def enabled?(account)
+    account.custom_attributes['chatbot_enabled'] == true
+  end
+
+  def system_prompt(account)
+    value_or_default(account, 'chatbot_system_prompt', DEFAULT_SYSTEM_PROMPT)
+  end
+
+  def guardrail_prompt(account)
+    value_or_default(account, 'chatbot_guardrail_prompt', DEFAULT_GUARDRAIL_PROMPT)
+  end
+
+  def out_of_scope_reply(account)
+    value_or_default(account, 'chatbot_out_of_scope_reply', DEFAULT_OUT_OF_SCOPE_REPLY)
+  end
+
+  def call_mode(account)
+    configured = account.custom_attributes['chatbot_call_mode'].to_i
+    [1, 2].include?(configured) ? configured : 1
+  end
+
+  # Apply a params hash of allowed keys to the account's custom_attributes and persist.
+  # Returns the resulting config hash. Ignores/merges only known keys.
+  def update!(account, params)
+    attrs = account.custom_attributes || {}
+    KEYS.each do |key|
+      attrs[key] = params[key] if params.key?(key)
+    end
+    account.update!(custom_attributes: attrs)
+    config(account)
+  end
+
+  # @return [Hash] full current config
+  def config(account)
+    {
+      enabled: enabled?(account),
+      system_prompt: system_prompt(account),
+      guardrail_prompt: guardrail_prompt(account),
+      out_of_scope_reply: out_of_scope_reply(account),
+      call_mode: call_mode(account)
+    }
+  end
+
+  def value_or_default(account, key, default)
+    value = account.custom_attributes[key]
+    value.presence || default
+  end
+end

app/services/chatbot/decision_service.rb

@@ -0,0 +1,206 @@
+# Chatbot decision service for the self-contained OSS chatbot.
+#
+# Given an inbound user message (plus history and retrieved knowledge), decides what the
+# bot should do:
+#   :answer  — in-scope; `answer` is the reply (grounded in the knowledge base)
+#   :refuse  — clearly off-topic (out of guardrail scope, e.g. fortune-telling); uses the
+#              account's out-of-scope reply template
+#   :handoff — related to scope but the bot can't answer (nothing in KB / undecidable);
+#              the caller should hand off to a human (conversation.bot_handoff!)
+#
+# Default: ONE LLM call returns a structured decision { decision, reason, answer }.
+# Optional TWO-call mode: a guardrail call decides in/out of scope, then an answer call
+# composes the reply from KB. Selected per account (config option; 2-call is for LLMs that
+# handle the compound single-call poorly).
+#
+# Fail-closed: no LLM credential -> { disabled: true } (never sends chat content when
+# disabled). Mirrors the Analytics::WeeklyPersonaEvaluator / Llm::AnalyticsClassifier pattern.
+module Chatbot::DecisionService
+  DECISION_SCHEMA = {
+    type: 'object',
+    additionalProperties: false,
+    properties: {
+      decision: {
+        type: 'string',
+        enum: %w[in_scope refuse handoff],
+        description: "in_scope = answer from the knowledge base; refuse = clearly off-topic and must not be answered; handoff = related to scope but bot cannot answer -> hand to a human."
+      },
+      reason: { type: 'string', description: 'One sentence justifying the decision.' },
+      answer: { type: 'string', description: 'The bot reply. Populated for in_scope; may be blank for refuse/handoff.' }
+    },
+    required: %w[decision reason answer]
+  }.freeze
+
+  Result = Struct.new(:action, :answer, :reason, :disabled, :error, keyword_init: true) do
+    def disabled? = disabled == true
+    def success? = error.nil?
+    def answer? = action == :answer
+    def refuse? = action == :refuse
+    def handoff? = action == :handoff
+  end
+
+  module_function
+
+  # @param account [Account]
+  # @param message [String] the inbound user text
+  # @param history [Array<Hash>] [{ role: 'user'|'assistant', content: String }]
+  # @param knowledge [Array<Hash>] [{ title:, content:, score: }] retrieved KB context
+  # @param call_mode [Integer] 1 (default) or 2
+  # @param system_prompt [String] optional per-account persona/system instructions
+  # @param guardrail_prompt [String] optional per-account allowed-scope instructions
+  # @return [Chatbot::DecisionService::Result]
+  def decide(account:, message:, history: [], knowledge: [], call_mode: 1, system_prompt: nil, guardrail_prompt: nil)
+    credential = Llm::Resolver.resolve(account)
+    return disabled_result if credential.nil?
+
+    # Prompts are threaded as explicit args (not module instance vars) so concurrent
+    # requests can never bleed one account's person/system prompt into another.
+    system = system_prompt.presence || SYSTEM_PROMPT
+    guardrail = guardrail_prompt.presence || GUARDRAIL_SCOPE
+
+    if call_mode == 2
+      decide_two_call(credential, message, history, knowledge, system, guardrail)
+    else
+      decide_one_call(credential, message, history, knowledge, system, guardrail)
+    end
+  rescue StandardError => e
+    Rails.logger.error("[ChatbotDecision] account=#{account&.id} #{e.class}: #{e.message}")
+    Result.new(error: e.message)
+  end
+
+  # -- 1-call mode ------------------------------------------------------------
+
+  def decide_one_call(credential, message, history, knowledge, system_prompt, guardrail_prompt)
+    response = call_llm(credential, build_one_call_prompt(message, history, knowledge, guardrail_prompt), system_prompt)
+    return Result.new(error: response[:error] || 'completion failed') if response[:error]
+
+    parsed = JSON.parse(sanitize_json(response[:content]))
+    action = normalize_action(parsed['decision'])
+    Result.new(
+      action: action,
+      reason: parsed['reason'].to_s,
+      answer: parsed['answer'].to_s,
+      disabled: false
+    )
+  rescue JSON::ParserError, TypeError
+    Result.new(error: 'LLM returned an unparsable decision')
+  end
+
+  # -- 2-call mode ------------------------------------------------------------
+
+  def decide_two_call(credential, message, history, knowledge, system_prompt, guardrail_prompt)
+    guardrail = call_llm(credential, build_guardrail_prompt(message, guardrail_prompt), system_prompt)
+    return Result.new(error: guardrail[:error] || 'guardrail failed') if guardrail[:error]
+
+    parsed = JSON.parse(sanitize_json(guardrail[:content]))
+    decision = parsed['decision']&.to_s
+    return Result.new(action: :refuse, reason: parsed['reason'].to_s, answer: '', disabled: false) if decision == 'refuse'
+    # refuse / handoff_unknown / anything-but-in_scope -> hand to a human
+    return Result.new(action: :handoff, reason: parsed['reason']&.to_s, answer: '', disabled: false) unless decision == 'in_scope'
+
+    answer_response = call_llm(credential, build_answer_prompt(message, history, knowledge), system_prompt)
+    return Result.new(error: answer_response[:error] || 'answer failed') if answer_response[:error]
+
+    Result.new(action: :answer, answer: answer_response[:content].to_s, reason: 'in_scope', disabled: false)
+  rescue JSON::ParserError, TypeError
+    Result.new(error: 'LLM returned an unparsable decision')
+  end
+
+  # -- LLM + prompt helpers ---------------------------------------------------
+
+  def call_llm(credential, prompt, system_prompt)
+    Llm::Config.with_api_key(credential[:api_key], api_base: credential[:api_base]) do |context|
+      chat = context.chat(model: MODEL).with_schema(DECISION_SCHEMA)
+      chat.with_instructions(system_prompt)
+      { content: chat.ask(prompt).content }
+    end
+  rescue StandardError => e
+    Rails.logger.error("[ChatbotDecision] LLM call failed #{e.class}: #{e.message}")
+    { error: e.message }
+  end
+
+  MODEL = Llm::Config::DEFAULT_MODEL
+
+  def build_one_call_prompt(message, history, knowledge, guardrail_prompt)
+    [
+      'Decide how the customer-service bot should respond to the customer message.',
+      '',
+      '## Guardrail scope',
+      guardrail_prompt,
+      '',
+      '## Knowledge base (retrieved, most relevant first)',
+      knowledge_text(knowledge).presence || '(no relevant knowledge found)',
+      '',
+      '## Conversation history',
+      history_text(history).presence || '(no prior messages)',
+      '',
+      "## Latest customer message\n#{message}",
+      '',
+      'If the message is in scope AND relevant knowledge exists, return decision=in_scope with the best answer grounded in the knowledge. If it is clearly outside the guardrail scope (e.g. fortune-telling, off-topic), return decision=refuse (answer may be blank). If it is related to scope but there is no knowledge to answer with, return decision=handoff.'
+    ].join("\n")
+  end
+
+  def build_guardrail_prompt(message, guardrail_prompt)
+    [
+      'You are a safety guardrail. Decide whether this customer message is within the allowed scope.',
+      '',
+      guardrail_prompt,
+      '',
+      "## Customer message\n#{message}",
+      '',
+      'Return decision: refuse if clearly outside scope; in_scope if within scope but may need knowledge to answer; handoff_unknown if related but ambiguous.'
+    ].join("\n")
+  end
+
+  def build_answer_prompt(message, history, knowledge)
+    [
+      'You are a helpful customer-service assistant for this business. Answer the customer using ONLY the provided knowledge base; do not invent facts.',
+      '',
+      '## Knowledge base',
+      knowledge_text(knowledge).presence || '(no relevant knowledge found)',
+      '',
+      '## Conversation history',
+      history_text(history).presence || '(no prior messages)',
+      '',
+      "## Customer message\n#{message}"
+    ].join("\n")
+  end
+
+  def knowledge_text(knowledge)
+    Array(knowledge).map { |k| "- #{k[:title]}: #{k[:content]}".strip }.join("\n")
+  end
+
+  def history_text(history)
+    Array(history).map { |h| "#{h[:role].to_s.capitalize}: #{h[:content]}" }.join("\n")
+  end
+
+  def normalize_action(decision)
+    case decision&.to_sym
+    when :in_scope then :answer
+    when :refuse then :refuse
+    when :handoff then :handoff
+    else :handoff
+    end
+  end
+
+  def sanitize_json(content)
+    content.to_s.gsub('```json', '').gsub('```', '').strip
+  end
+
+  def disabled_result
+    Result.new(action: nil, answer: '', reason: '', disabled: true)
+  end
+
+  SYSTEM_PROMPT = <<~PROMPT.freeze
+    You decide and then answer for a customer-service chatbot. Stay within the allowed
+    guardrail scope and be truthful and helpful. Return only the JSON object described by
+    the schema — no extra text.
+  PROMPT
+
+  GUARDRAIL_SCOPE = <<~SCOPE.freeze
+    The bot answers questions about this business's PRODUCTS, SERVICES, and related
+    support topics only. It must NOT answer unrelated or off-topic requests (e.g. personal
+    advice, fortune-telling, horoscopes, unrelated general knowledge, or any topic outside
+    the listed products/services/support).
+  SCOPE
+end

app/services/chatbot/knowledge_retriever.rb

@@ -0,0 +1,86 @@
+# KnowledgeBase retriever for the self-contained OSS chatbot.
+#
+# Given a user message, returns the top-k most relevant KnowledgeBaseFaq entries for
+# that account, ordered best-first. Retrieval is keyword-based (pg_trgm similarity on
+# title+content + topic_tag match) so it works without an embedding backend.
+#
+# A `query_embedding` param is accepted as a future extension point for embedding-fusion
+# (hybrid keyword + vector ranking), to be wired when an embedding provider is configured
+# (see plan). Returns [{ faq:, score: Float }] — the caller injects these into the LLM prompt.
+module Chatbot::KnowledgeRetriever
+  DEFAULT_LIMIT = 5
+
+  module_function
+
+  # @param account [Account]
+  # @param query [String] the user message
+  # @param query_embedding [Array<Float>, nil] reserved; embedding-fusion is a later step
+  # @param limit [Integer]
+  # @return [Array<Hash>] [{ faq:, score: Float }]
+  def retrieve(account:, query:, query_embedding: nil, limit: DEFAULT_LIMIT)
+    return [] if query.blank?
+
+    scores = score_candidates(account, query)
+    return [] if scores.empty?
+
+    max = scores.values.max
+
+    scores.map { |faq, score| { faq: faq, score: (score / max).round(4) } }
+          .sort_by { |h| -h[:score] }
+          .first(limit)
+  end
+
+  # Rank candidate FAQ entries by pg_trgm similarity + topic-tag match.
+  # @return [Hash{KnowledgeBaseFaq => Float}]
+  def score_candidates(account, query)
+    scores = {}
+    candidates(account, query).each do |faq|
+      s = faq_title_similarity(faq, query)
+      s = [s, faq_content_similarity(faq, query)].max
+      s += 0.2 if topic_match?(faq, query)
+      scores[faq] = s if s.positive?
+    end
+    scores
+  end
+
+  # Candidate set: entries whose title or content is likely relevant (pre-filter via
+  # pg_trgm word_similarity to keep the scoring pass small). Falls back to all account
+  # FAQs if pre-filter isn't available (plain AR without pg_trgm search string).
+  def candidates(account, query)
+    relation = account.knowledge_base_faqs
+    column = %(GREATEST(word_similarity(title, #{quote(query)}), word_similarity(content, #{quote(query)})))
+    relation.where("#{column} > 0.1").limit(50).to_a
+  rescue StandardError
+    relation.limit(200).to_a
+  end
+
+  def faq_title_similarity(faq, query)
+    pg_similarity(faq.title, query)
+  end
+
+  def faq_content_similarity(faq, query)
+    pg_similarity(faq.content, query)
+  end
+
+  # Token-overlap similarity ratio computed in Ruby (downcase → split → overlap / max size).
+  # Deterministic and DB-free; used to rank the small candidate pool from `candidates`.
+  def pg_similarity(text_a, text_b)
+    return 0.0 if text_a.blank? || text_b.blank?
+
+    a = text_a.downcase.split(/\s+/).reject(&:blank?)
+    b = text_b.downcase.split(/\s+/).reject(&:blank?)
+    return 0.0 if a.empty? || b.empty?
+
+    overlap = (a & b).size
+    overlap.to_f / [a.size, b.size].max.to_f
+  end
+
+  def topic_match?(faq, query)
+    q = query.downcase
+    faq.topic_tag_list.any? { |t| q.include?(t.downcase) }
+  end
+
+  def quote(value)
+    ActiveRecord::Base.sanitize_sql_like(value.to_s).gsub("'", "''")
+  end
+end

app/services/knowledge_base/import_service.rb

@@ -0,0 +1,131 @@
+# Imports markdown FAQ content into KnowledgeBaseFaq for an account (OSS self-contained
+# chatbot, phase: KB). Accepts:
+#   - pasted markdown text (content:) — split into per-heading sections
+#   - an uploaded .md / .csv / .xlsx file (file_path: + filename:)
+#
+# For markdown: each `#`/`##`/`---` section becomes one KnowledgeBaseFaq row, which
+# is the retrieval unit (title = heading, content = section body). topic_tags come
+# from explicit front-matter tags or fall back to the first heading words.
+#
+# Returns a Hash: { imported:, updated:, errors: [{ line, message }] }.
+class KnowledgeBase::ImportService
+  FRONT_MATTER_TAGS = /\A---\s*\ntags:\s*(.+?)\n---\s*\n/im
+  # Matches markdown headings (#, ##, ... up to ######). Uses [ # ]{1,6} to avoid
+  # any #{ } interpolation ambiguity in the regex literal.
+  HEADING = /^[#]{1,6}\s+(.+)$/i
+
+  def self.import(account:, content: nil, file_path: nil, filename: nil)
+    new(account: account, content: content, file_path: file_path, filename: filename).import
+  end
+
+  def initialize(account:, content: nil, file_path: nil, filename: nil)
+    @account = account
+    @content = content.to_s
+    @file_path = file_path
+    @filename = filename
+  end
+
+  def import
+    sections = @file_path ? sections_from_file : sections_from_text(@content)
+    upsert_sections(sections)
+  end
+
+  private
+
+  def sections_from_file
+    ext = File.extname(@filename.presence || @file_path.to_s).delete('.').downcase
+    text =
+      case ext
+      when 'md'
+        File.read(@file_path)
+      when 'csv'
+        require 'csv'
+        # one row per column -> naive title/content pair
+        CSV.read(@file_path).map { |r| "#{r[0]}\n\n#{r[1..].join(' ')}" }.join("\n\n")
+      when 'xlsx'
+        read_xlsx(@file_path)
+      else
+        File.read(@file_path)
+      end
+    sections_from_text(text)
+  rescue StandardError => e
+    Rails.logger.error("[KnowledgeBaseImport] parse failed: #{e.message}")
+    [{ error: "could not read file: #{e.message}" }]
+  end
+
+  def read_xlsx(path)
+    require 'roo'
+    sheet = Roo::Spreadsheet.open(path, extension: 'xlsx').sheet(0)
+    rows = (1..sheet.last_row).filter_map do |idx|
+      r = (1..sheet.last_column).map { |c| sheet.cell(idx, c).to_s }
+      "#{r[0]}\n\n#{r[1..].join(' ')}" unless r.all?(&:blank?)
+    end
+    rows.join("\n\n")
+  end
+
+  # Split markdown into per-heading sections; content preceding the first heading is
+  # treated as a single section with a derived title. A leading front-matter block
+  # (`---\ntags: ...\n---`) is stripped and its tags applied to every section.
+  def sections_from_text(text)
+    body, tags = extract_front_matter_tags(text.to_s)
+
+    sections = []
+    current = { title: nil, body: [] }
+
+    body.strip.split(/\r?\n/).each do |line|
+      if (m = line.match(HEADING))
+        # Flush the current section (even a title-less intro block) before a heading
+        sections << close_section(current, tags)
+        current = { title: m[1].strip, body: [] }
+      else
+        current[:body] << line
+      end
+    end
+    sections << close_section(current, tags)
+
+    sections.compact.reject { |s| s[:content].blank? }
+  end
+
+  # Returns [body_without_front_matter, tags_array]
+  def extract_front_matter_tags(text)
+    if (m = text.match(FRONT_MATTER_TAGS))
+      [text.sub(m[0], ''), m[1].split(/[,;\s]+/).map(&:strip).reject(&:blank?)]
+    else
+      [text, []]
+    end
+  end
+
+  def close_section(section, tags)
+    body = section[:body].join("\n").strip
+    return nil if body.blank? && section[:title].blank?
+
+    { title: section[:title].presence || body.lines.first.to_s.strip[0..80], content: body, topic_tags: tags }
+  end
+
+  def upsert_sections(sections)
+    imported = 0
+    updated = 0
+    errors = []
+    sections.each_with_index do |sec, idx|
+      next unless sec.is_a?(Hash) && sec[:content].present?
+      next if sec[:content].blank?
+
+      attrs = {
+        content: sec[:content],
+        topic_tags: sec[:topic_tags] || [],
+        source_filename: @filename
+      }
+      existing = @account.knowledge_base_faqs.find_by(title: sec[:title])
+      if existing
+        existing.update!(attrs)
+        updated += 1
+      else
+        @account.knowledge_base_faqs.create!(attrs.merge(title: sec[:title]))
+        imported += 1
+      end
+    rescue ActiveRecord::RecordInvalid => e
+      errors << { line: idx + 1, message: e.message }
+    end
+    { imported: imported, updated: updated, errors: errors }
+  end
+end

config/integration/apps.yml

@@ -87,6 +87,14 @@ slack:
   hook_type: account
   allow_multiple_hooks: false
   visible_properties: ['channel_name']
+chatbot:
+  id: chatbot
+  logo: chatbot.png
+  i18n_key: chatbot
+  action: /chatbot
+  hook_type: inbox
+  allow_multiple_hooks: false
+  visible_properties: []
 dialogflow:
   id: dialogflow
   logo: dialogflow.png

config/locales/en.yml

@@ -408,6 +408,10 @@ en:
       name: 'Dialogflow'
       short_description: 'Build chatbots to handle initial queries before transferring to agents.'
       description: 'Build chatbots with Dialogflow and easily integrate them into your inbox. These bots can handle initial queries before transferring them to a customer service agent.'
+    chatbot:
+      name: 'Chatbot'
+      short_description: 'AI chatbot that answers in-scope questions from your knowledge base and hands off to agents when needed.'
+      description: 'Enable the built-in AI chatbot: it answers questions that fall within your product/service scope using your knowledge base (md/csv/excel), refuses clearly off-topic requests, and automatically hands related-but-unanswerable conversations to a human agent.'
     google_translate:
       name: 'Google Translate'
       short_description: 'Automatically translate customer messages for agents.'

config/routes.rb

@@ -544,6 +544,27 @@ Rails.application.routes.draw do
               get :grouped_conversation_metrics
             end
           end
+          resources :analytics_reports, only: [] do
+            collection do
+              get :summary
+              get :drilldown
+              post :persona_evaluation
+              post :persona_evaluation_deliver
+              post :persona_approval_settings
+              post :persona_decision
+            end
+          end
+          resources :product_catalog_entries, only: [:index] do
+            collection do
+              post :import
+            end
+          end
+          resources :knowledge_base_faqs, only: [:index] do
+            collection do
+              post :import
+            end
+          end
+          resource :chatbot_config, only: %i[show update]
         end
       end
     end

config/schedule.yml

@@ -74,3 +74,20 @@ remove_orphan_conversations_job:
   cron: '0 */12 * * *'
   class: 'Internal::RemoveOrphanConversationsJob'
   queue: housekeeping
+
+# M2 analytics: runs once daily at 02:30 in the installation (super admin) timezone.
+# Single TZ, no hourly-check — single pass over all accounts' closed days.
+# Note: cron is interpreted in the given timezone (sidekiq-cron `timezone`).
+analytics_daily_metrics_job:
+  cron: '30 2 * * *'
+  timezone: 'Asia/Bangkok'
+  class: 'Analytics::DailyMetricsJob'
+  queue: scheduled_jobs
+
+# M2 analytics: weekly persona evaluation every Monday 10:00 (Asia/Bangkok).
+# Evaluates the last 7 days and delivers the recommendation via Analytics::PersonaApprovalService.
+analytics_weekly_persona_evaluation_job:
+  cron: '0 10 * * 1'
+  timezone: 'Asia/Bangkok'
+  class: 'Analytics::WeeklyPersonaEvaluationJob'
+  queue: scheduled_jobs

db/migrate/20260820000000_create_knowledge_base_faqs.rb

@@ -0,0 +1,28 @@
+# KnowledgeBaseFaq — OSS-level markdown FAQ knowledge entries for the self-contained
+# chatbot. Account-scoped. Supports keyword retrieval via pg_trgm index on title and
+# an optional pgvector embedding column (populated by a later embedding step).
+#
+# This is intentionally SEPARATE from Enterprise-only Captain tables
+# (captain_assistant_responses / captain_assistants) so it works without EE.
+class CreateKnowledgeBaseFaqs < ActiveRecord::Migration[7.1]
+  def change
+    create_table :knowledge_base_faqs do |t|
+      t.bigint :account_id, null: false
+      t.string :title, null: false            # short topic/question label
+      t.text :content, null: false            # markdown body / answer
+      t.jsonb :topic_tags, default: [], null: false  # e.g. ["shipping", "refund"]
+      t.string :source_filename               # original md filename (optional)
+      t.vector :embedding, limit: 1536        # pgvector embedding (nullable; set later)
+      t.timestamps
+    end
+
+    add_index :knowledge_base_faqs, :account_id
+    add_index :knowledge_base_faqs, [:account_id, :title]
+    # pg_trgm GIN index for fuzzy keyword search on title + content
+    # (matches repo convention: gin + gin_trgm_ops, e.g. index_messages_on_content)
+    add_index :knowledge_base_faqs, :title, using: :gin, opclass: :gin_trgm_ops, name: 'index_kbf_on_title_trgm'
+    add_index :knowledge_base_faqs, :content, using: :gin, opclass: :gin_trgm_ops, name: 'index_kbf_on_content_trgm'
+    # pgvector ivfflat index for embedding similarity search (only when embeddings exist)
+    add_index :knowledge_base_faqs, :embedding, using: :ivfflat, opclass: :vector_cosine_ops, name: 'index_kbf_on_embedding'
+  end
+end

engineering-log.md

@@ -113,3 +113,38 @@ Application changes are tracked in the verified checkpoints and the uncheckpoint
 - SM-09/10 cannot change branding defaults/assets/metadata until approved product name, domains, logos/icons, sender, legal URLs and primary color are supplied. Current visible-branding report remains 2,878 findings; it is report-only, not a privacy-rule pass.
 - Enterprise `WebsiteBrandingService` remains proprietary and currently fails with `NameError: uninitialized constant WebsiteBrandingService`; no dummy service or remote enrichment was added.
 - SM-11/12 static inventory: repository is shallow (`git rev-parse --is-shallow-repository=true`); Docker, Syft, Trivy, Cosign and Gitleaks are unavailable; Dockerfile base images are mutable tags. Full-history fetch, isolated HTTP/DNS capture, production-like DB rehearsal, image digest and SBOM gates remain blocked. No network action was performed.
+
+## 2026-08-22 — LLM chatbot runtime architecture discovery
+
+### Objective
+Compare Hermes, embedded n8n-like workflow functionality, and external n8n for the Chatwoot AgentBot path. This was a read-only architecture review; no production code was changed.
+
+### Verified Chatwoot contract
+- `AgentBot` currently has `bot_type: webhook`, `outgoing_url`, `bot_config`, secret, access token, account/inbox associations.
+- `AgentBotListener` enqueues an asynchronous `AgentBots::WebhookJob`; it POSTs an event payload and does not wait for an LLM response.
+- `Webhooks::Trigger` signs requests with `X-Chatwoot-Timestamp` and `X-Chatwoot-Signature`, retries 429/500 for agent bots, and treats the external bot as responsible for posting an outgoing message through Chatwoot API.
+- Super Admin AgentBot is an admin CRUD surface. Per-account Bot Configuration is the correct customer-facing runtime boundary.
+- No n8n integration exists in the current repository search.
+
+### Verified Hermes contract
+- Hermes API server exposes OpenAI-compatible `/v1/chat/completions`, Responses/Runs and Sessions APIs with bearer auth; Chat Completions is stateless unless history/response chaining is supplied.
+- Hermes webhook adapter expects its own generic signature headers (`X-Webhook-Signature-V2`/`X-Webhook-Timestamp`), not Chatwoot's signature headers, and its configured delivery targets do not include Chatwoot message creation.
+- Hermes memory/profiles are Hermes-scoped, not automatically Chatwoot-account-scoped. API server capabilities include terminal/tools, so a customer-facing deployment needs a restricted toolset, sandbox, and tenant isolation.
+
+### Architecture finding
+- Directly setting Chatwoot `outgoing_url` to Hermes is not a complete integration. A bridge/adapter is required to verify Chatwoot events, map account/inbox/conversation identity, invoke Hermes, enforce tool policy/idempotency, and POST the final message back to Chatwoot.
+- n8n is appropriate as an external workflow/orchestration layer, but should not become the source of truth for conversation state or tenant knowledge.
+- Embedding n8n UI/runtime in the product is a separate product/licensing/operations scope. n8n's official OEM docs require a commercial agreement when customers interact with the embedded editor; behind-the-scenes backend use is a distinct model.
+
+### Decision candidate
+Use a hybrid target: Chatwoot AgentBot → dedicated Hermes bridge → Hermes API for LLM/knowledge/tools; keep external n8n optional for cross-system workflows and side effects. Start with n8n as a P0 validation path only if it materially shortens learning; do not embed n8n or expose unrestricted Hermes tools to end users.
+
+### Repository state
+The worktree already contained unrelated uncommitted M1/M2 files before this review. Do not mix the future bridge implementation with those changes without an isolated worktree/branch.
+
+## 2026-08-22 — Final decision: external n8n (Option 3)
+
+- User selected Option 3: keep n8n as a separate service for the LLM chatbot/workflow path.
+- No Hermes integration, embedded n8n/workflow builder, or Chatwoot production-code change is authorized in this decision.
+- Existing Chatwoot AgentBot webhook contract remains the boundary; n8n receives the event, calls the configured LLM/external services, and posts the reply back through Chatwoot API.
+- Treat the earlier Hermes hybrid architecture as analysis only, not an implementation decision. A Gateway/Adapter remains a future option if tenant isolation, idempotency, observability, or multi-runtime routing later require it.

lib/integrations/chatbot/processor_service.rb

@@ -0,0 +1,107 @@
+# Chatbot processor that answers inbound chats in-place using the OSS self-contained
+# chatbot (guardrail + knowledge base + Llm::Resolver), instead of an external bot.
+#
+# Selected per inbox via an Integrations::Hook with app_id == 'chatbot' (see HookJob).
+# Mirrors Integrations::Dialogflow::ProcessorService / Integrations::Captain::ProcessorService
+# by subclassing Integrations::BotProcessorService and overriding get_response.
+#
+# Flow (from BotProcessorService#process_content):
+#   get_response(session_id, content)
+#     -> retrieve knowledge (keyword top-k)
+#     -> build history
+#     -> Chatbot::DecisionService.decide (1 or 2 calls)
+#     -> return an action marker consumed by process_response
+#   process_response(message, decision)
+#     -> :answer  -> create outbound reply (knowledge-grounded)
+#     -> :refuse  -> create outbound reply using account's out-of-scope template
+#     -> :handoff -> conversation.bot_handoff! (release bot, pending -> open, human takes over)
+#
+# Fail-closed: if the LLM is disabled or errors, we hand off to a human (safe default)
+# rather than silently not replying. Never sends chat content to the LLM when disabled.
+class Integrations::Chatbot::ProcessorService < Integrations::BotProcessorService
+  pattr_initialize [:event_name!, :hook!, :event_data!]
+
+  HANDOFF = 'chatbot_handoff'.freeze
+
+  private
+
+  # BotProcessorService calls get_response(source_id, content) then process_response.
+  # We return a Chatbot::DecisionService::Result (or a String marker for handoff) and
+  # handle rendering in process_response.
+  def get_response(_session_id, message_content)
+    return HANDOFF if message_content.blank?
+    # Fail-closed gate: if the bot is not enabled for this account, hand to a human.
+    return HANDOFF unless Chatbot::ConfigService.enabled?(conversation.account)
+
+    result = Chatbot::DecisionService.decide(
+      account: conversation.account,
+      message: message_content,
+      history: build_history,
+      knowledge: knowledge_for(message_content),
+      call_mode: chatbot_call_mode,
+      system_prompt: Chatbot::ConfigService.system_prompt(conversation.account),
+      guardrail_prompt: Chatbot::ConfigService.guardrail_prompt(conversation.account)
+    )
+
+    # Fail-closed: disabled, or any error -> human handoff (safe default).
+    return HANDOFF if result.disabled? || !result.success?
+
+    result
+  end
+
+  def process_response(message, decision)
+    return create_conversation(message, { content: out_of_scope_reply }) if decision.refuse?
+
+    if decision.handoff? || decision == HANDOFF
+      message.conversation.bot_handoff!
+      return
+    end
+
+    return if decision.answer.blank? # nothing to say
+
+    create_conversation(message, { content: decision.answer })
+  end
+
+  # -- context helpers --------------------------------------------------------
+
+  def conversation
+    @conversation ||= event_data[:message].conversation
+  end
+
+  def knowledge_for(content)
+    Chatbot::KnowledgeRetriever.retrieve(account: conversation.account, query: content)
+  end
+
+  def build_history
+    # Lightweight: the most recent few incoming/outgoing text messages (excluding this one).
+    conversation.messages
+                .where(message_type: %i[incoming outgoing], content_type: 'text')
+                .where.not(id: event_data[:message].id)
+                .order(created_at: :asc)
+                .last(8)
+                .map do |m|
+      { role: m.message_type == 'outgoing' ? 'assistant' : 'user', content: m.content.to_s }
+    end
+  end
+
+  def chatbot_call_mode
+    Chatbot::ConfigService.call_mode(conversation.account)
+  end
+
+  def out_of_scope_reply
+    Chatbot::ConfigService.out_of_scope_reply(conversation.account)
+  end
+
+  def create_conversation(message, content_params)
+    return if content_params.blank? || content_params[:content].blank?
+
+    conv = message.conversation
+    conv.messages.create!(
+      content_params.merge(
+        message_type: :outgoing,
+        account_id: conv.account_id,
+        inbox_id: conv.inbox_id
+      )
+    )
+  end
+end

lib/llm/analytics_classifier.rb

@@ -0,0 +1,168 @@
+# Classifies a single conversation via the LLM cascade (Llm::Resolver):
+#   per-account OpenAI hook (with optional custom base_url) -> Captain -> nil.
+#
+# Produces a normalized classification used by the daily analytics batch job:
+#   - topics: free-form topic tags (e.g. "pricing", "installation")
+#   - products: matched product_catalog_entries (full tag hierarchy, '>' joined)
+#   - deal: won | lost | undecided
+#
+# This service only CLASSIFIES; it never writes to the conversation or to the
+# daily-metric tables. Writing is the caller's responsibility (the batch job),
+# so a failed/disabled classification can never mutate state.
+#
+# If no LLM is configured for the account, returns { disabled: true } without
+# making any network call or sending conversation content anywhere.
+module Llm::AnalyticsClassifier
+  SCHEMA = {
+    type: 'object',
+    additionalProperties: false,
+    properties: {
+      topics: {
+        type: 'array',
+        items: { type: 'string' },
+        description: 'Short topic labels describing what this conversation is about (e.g. pricing, installation, support).'
+      },
+      product_indexes: {
+        type: 'array',
+        items: { type: 'integer' },
+        description: 'Indexes (0-based) into the supplied product list that apply to this conversation. Empty if none apply.'
+      },
+      deal: {
+        type: 'string',
+        enum: %w[won lost undecided],
+        description: 'Whether this conversation reached a sale decision. won = closed sale, lost = customer declined, undecided = no decision yet.'
+      }
+    },
+    required: %w[topics product_indexes deal]
+  }.freeze
+
+  Result = Struct.new(:topics, :products, :deal, :disabled, :error, keyword_init: true) do
+    def disabled?
+      disabled == true
+    end
+
+    def success?
+      error.nil?
+    end
+  end
+
+  MODEL = Llm::Config::DEFAULT_MODEL
+
+  module_function
+
+  # @param account [Account]
+  # @param conversation [Conversation]
+  # @return [Llm::AnalyticsClassifier::Result]
+  def classify(account:, conversation:)
+    credential = Llm::Resolver.resolve(account)
+    return disabled_result if credential.nil?
+
+    payload = build_payload(account, conversation)
+    response = call_llm(credential, payload)
+
+    build_result(response, payload)
+  rescue StandardError => e
+    Rails.logger.error("[AnalyticsClassifier] account=#{account&.id} #{e.class}: #{e.message}")
+    Result.new(error: e.message)
+  end
+
+  # -- result builders -------------------------------------------------------
+
+  def disabled_result
+    Result.new(topics: [], products: [], deal: 'undecided', disabled: true)
+  end
+
+  def build_result(response, payload)
+    return Result.new(error: response.dig(:error, :message) || 'classification failed') if response.dig(:error)
+
+    parsed = JSON.parse(sanitize_json(response[:message]))
+    indexes = Array(parsed['product_indexes'])
+    products = payload[:catalog].values_at(*indexes).compact
+
+    Result.new(
+      topics: Array(parsed['topics']),
+      products: products,
+      deal: parsed['deal'] || 'undecided',
+      disabled: false
+    )
+  rescue JSON::ParserError, TypeError
+    Result.new(error: 'LLM returned an unparsable classification')
+  end
+
+  # Some gateways wrap structured JSON in markdown fences despite response_format
+  # hints — strip them before parsing (same convention as Captain::ChatResponseHelper).
+  def sanitize_json(content)
+    content.to_s.gsub('```json', '').gsub('```', '').strip
+  end
+
+  # -- LLM call --------------------------------------------------------------
+
+  def call_llm(credential, payload)
+    Llm::Config.with_api_key(credential[:api_key], api_base: credential[:api_base]) do |context|
+      chat = context.chat(model: MODEL).with_schema(SCHEMA)
+      chat.with_instructions(SYSTEM_PROMPT)
+      response = chat.ask(payload[:user_prompt])
+      { message: response.content }
+    end
+  rescue StandardError => e
+    Rails.logger.error("[AnalyticsClassifier] LLM call failed #{e.class}: #{e.message}")
+    { error: { message: e.message } }
+  end
+
+  # -- prompt construction ---------------------------------------------------
+
+  def build_payload(account, conversation)
+    catalog = catalog_entries(account)
+    { catalog: catalog, user_prompt: build_user_prompt(catalog, conversation) }
+  end
+
+  # Returns an ordered Array of tag-hierarchy strings (group>subgroup>product)
+  # aligned with what product_indexes refers to.
+  def catalog_entries(account)
+    account.product_catalog_entries
+           .order(:group_name, :subgroup_name, :product_name)
+           .map(&:tag_hierarchy)
+           .map { |parts| parts.join('>') }
+  end
+
+  def build_user_prompt(catalog, conversation)
+    lines = []
+    lines << 'Below is a customer service chat transcript.'
+    lines << ''
+    lines << "Available products (0-based index, tag hierarchy):"
+    if catalog.empty?
+      lines << '(no product catalog configured for this account — return an empty product_indexes)'
+    end
+    catalog.each_with_index { |path, i| lines << "#{i}: #{path}" }
+    lines << ''
+    lines << 'Transcript:'
+    lines << transcript(conversation)
+    lines.join("\n")
+  end
+
+  def transcript(conversation)
+    conversation.messages
+                .where(message_type: %i[incoming outgoing])
+                .where(private: false)
+                .order(:id)
+                .inject([]) do |acc, message|
+      content = message.respond_to?(:content_for_llm) ? message.content_for_llm : message.content
+      next acc if content.blank?
+
+      sender = message.incoming? ? 'Customer' : 'Agent'
+      acc << "#{sender}: #{content}"
+    end.join("\n")
+  end
+
+  SYSTEM_PROMPT = <<~PROMPT.freeze
+    You classify customer service conversations for a business intelligence system.
+    Read the transcript and return:
+    - topics: short, specific topic labels (in the language of the conversation).
+    - product_indexes: the 0-based indexes into the supplied product list that match the
+      products discussed. Match from the MOST SPECIFIC (lowest) level first. If multiple
+      products apply, list all. Empty when no catalog product is discussed.
+    - deal: "won" when a purchase is completed or confirmed, "lost" when the customer
+      declines or leaves without purchasing, "undecided" when no clear decision was made.
+    Only return the JSON object described by the schema — no extra text.
+  PROMPT
+end