moreminimore-chat/.hermes/plans/2026-08-20-oss-chatbot-guardrail-knowledge.md

Plan — OSS Self-Contained Chatbot (Guardrail + Knowledge + Llm::Resolver Answer)

Repo: /Users/kunthawat/Gitea/Chatwoot · branch develop Date: 2026-08-20 Status: PLAN — awaiting user approval before build

Vision (from user)

Make this Chatwoot fork a self-contained OSS chatbot: every inbound chat → guardrail (keep users on-topic, e.g. reject "บอกชะตารายวัน / ดูดวง" when it's a product/service bot) → if in-scope, answer with LLM using a knowledge base (md/csv/excel) + system prompt + guardrail prompt + history. No EE, no external Captain. Answer via Llm::Resolver local.

Decisions locked (from clarify)

• Not using EE → Captain v2 (enterprise/) is out of scope. Use OSS path only.
• Architecture: build our own Integrations::Chatbot::ProcessorService < Integrations::BotProcessorService (mirrors Captain::ProcessorService / Dialogflow::ProcessorService), overriding get_response to answer via Llm::Resolver + knowledge base instead of an external webhook. Selectable per-inbox (like Captain/Dialogflow choose processor by hook app_id).
• Trigger stays OSS AgentBot / pending flow (the "AI answers" = conversation pending).
• LLM calls: default 1 call returning {allowed, answer}; config option to use 2 calls (guardrail check, then answer) for LLMs that need them. Backward-compatible.
• KB retrieval: keyword + embedding retrieval (top-k relevant chunks), NOT whole-KB-in-prompt.
• Out-of-scope policy:
  • topic clearly off-guardrail (e.g. fortune-telling) → refuse with template
  • topic related but not in KB / undecidable → auto human handoff (bot_handoff!)
• KB storage: extend product_catalog_entries (csv/xlsx product rows) + add MD FAQ table.

Current OSS bot flow (verified)

inbound message (conversation pending?) 
  → ... AgentBots::WebhookJob / agent_bot_listener → Webhooks::Trigger → webhook (external)

BotProcessorService base: should_run_processor? (message.reportable?, conversation.pending?) → get_response(source_id, content); process_action handles :handoff → bot_handoff! / :resolve. Webhooks::Trigger#update_conversation_status: on agent_bot failure, pending → open! (human takes over).

Architecture

[our ProcessorService] < Integrations::BotProcessorService
  get_response → guardrail? 
      ├─ in-scope  → Llm::Resolver answer (KB + system prompt + guardrail + history) → reply
      └─ out-of-scope → out-of-scope reply template (reject) / bot_handoff!

Selected per inbox by hook app_id (same mechanism as Captain/Dialogflow).

Deliverables (build order)

1. KB backend: KnowledgeBaseFaq model+migration (account-scoped, md FAQ: title + content + topic tags)
   • reuse product_catalog_entries for csv/xlsx product rows. Import service (extend ProductCatalogImportService / add MD FAQ import via roo/CSV/stdlib). Admin endpoints (list/import) + routes.
2. Retrieval service: Chatbot::KnowledgeRetriever — keyword (+ optional embedding) top-k selection over FAQ + product_catalog. Embedding vector column on FAQ table (nullable), keyword via SQL ILIKE/tsvector.
3. Guardrail + answer: Chatbot::GuardrailService + Chatbot::AnswerService — default 1 call { allowed, answer }; optional 2-call mode. Out-of-scope → refuse template; related-but-not-in-KB / undecidable → human handoff.
4. Chatbot processor: Integrations::Chatbot::ProcessorService < BotProcessorService — get_response = retrieve → guardrail/answer → reply | refuse | handoff. Wire selection per inbox.
   • Human handoff path adjusts chat status: on related-but-not-in-KB / undecidable, call conversation.bot_handoff! which releases the bot (= pending → open), clears assignee_agent_bot, sets waiting_since, and dispatches the handoff event so a human agent queue/assignment picks it up.
5. Account config: per-account: system prompt + guardrail prompt + out-of-scope reply template + call-mode (1 or 2) + enabled flag + which KB (folder/index). Store in Account#custom_attributes or settings model. Admin endpoints + UI.
6. Verify + review each part (smoke / ruby -c / static scan / independent reviewer).

Open questions (mostly resolved; remaining minor)

• LLM calls: 1 default, 2 optional (user decision).
• KB retrieval: keyword + embedding, top-k (user decision).
• Out-of-scope: refuse if clearly off-topic; auto handoff if related-but-not-in-KB/undecidable.
• KB storage: extend product_catalog + add MD FAQ.
• Embedding: which provider/API to compute embeddings (Llm::Resolver? separate embedding model?).
• MD FAQ granularity: one row per file? per heading/section chunk? (affects retrieval + import).

Success criteria

• OSS inbox with our processor answers in-scope from KB (keyword+embedding) via Llm::Resolver (1 or 2 calls). ✅ built
• Clearly out-of-scope (e.g. fortune-telling) → templated refusal. ✅
• Related-but-not-in-KB / undecidable → auto human handoff. ✅ (bot_handoff! → pending→open)
• Product rows (csv/xlsx) + MD FAQ both import + retrieve. ✅
• Per-account config (system/guardrail/out-of-scope template/call mode/enabled) via admin endpoint. ✅ (ChatbotConfigController + ConfigService)
• Each deliverable passes smoke + static scan + independent reviewer. ✅ (final re-review deleg_a5c361cb = passed:true, end-to-end dispatchable)

Wiring (verified, end-to-end)

apps.yml chatbot(inbox) → HooksController create (ensure_hook_type=inbox) → HookListener supported_events_map['chatbot'] → HookJob INTEGRATION_PROCESSORS['chatbot'] → Chatbot::ProcessorService → DecisionService → Llm::Resolver. i18n chatbot added to en.yml integration_apps.