feat: SaaS foundation for CrowdSight
Elevate MiroFish/CrowdSight from single-container dev to a SaaS foundation: - Local memory backend (Zep-compatible): memory services/models, local graph builder + updater, AgentActivity seam, import-boundary isolation; Zep stays default, local is opt-in behind MEMORY_BACKEND. Semantic parity not yet proven. - Durable product persistence: projects/simulations/reports schema (migration 0007) + tenant/owner-scoped ProductRepository + dual-write + scoped_project read-first + ArtifactStore abstraction; durable JobQueue + worker.py. - SaaS hardening: durable RateLimiter (wired to login), UsageService (LLM accounting), redacted AuditService, idempotency, CORS allowlist, safe API errors, single-use PasswordResetService + endpoints (covers invite-pending). - Exactly 3 roles (super_admin/admin/user) with tenant authz policy. - Admin UI: GET/POST/PATCH /api/admin/users + GET/PUT /api/admin/settings (super-admin only, encrypted/masked); AdminView.vue + SettingsView.vue with admin/super-admin route guards, th/en i18n. - Production deploy topology: multi-stage Dockerfile (frontend build + gunicorn wsgi + nginx SPA-proxy + supervisord worker), backend/wsgi.py, gunicorn dep. Backend 197 passed; frontend 10 tests + build green. ruff unavailable (gap). No commit of credentials; secrets handled via env/.env.example. Deferred: Zep semantic A/B parity, object storage cutover, mobile QA, EasyPanel container build of deploy topology.
This commit is contained in:
494
.hermes/plans/2026-08-23_110451-mirofish-saas-migration.md
Normal file
494
.hermes/plans/2026-08-23_110451-mirofish-saas-migration.md
Normal file
@@ -0,0 +1,494 @@
|
||||
# MiroFish → Company SaaS Architecture & Migration Plan
|
||||
|
||||
**สถานะ:** Implementation in progress — bounded SaaS foundation and local-memory E2E gates are implemented; production migration remains incomplete
|
||||
**Repo:** `/Users/kunthawat/Gitea/MiroFish`
|
||||
**HEAD ที่ตรวจ:** `89d04e7` (`fix: replace hero logo with inline use cases grid`)
|
||||
**วันที่ตรวจ:** 2026-08-23 (+07:00)
|
||||
|
||||
## 1. เป้าหมาย
|
||||
|
||||
เปลี่ยน MiroFish/CrowdSight จากแอป single-user ที่พึ่งพา Zep Cloud และไฟล์ในเครื่อง ให้เป็น SaaS ของบริษัท โดยมีผลลัพธ์ที่ยอมรับได้ดังนี้:
|
||||
|
||||
1. **Frontend ไม่มีภาษาจีนใน product surface** — UI chrome, locale fallback, API error ที่แสดงบนจอ, prompt/runtime ที่ระบบสร้าง, route metadata และ build artifact ใช้เฉพาะ Thai หรือ English
|
||||
2. **ไม่มี Zep dependency/runtime** — ใช้ LLM ทำ extraction/merge/summarization/reasoning และใช้ local durable graph repository ทำ storage/index/query แทน
|
||||
3. **มี authentication + tenant isolation + 3 roles เท่านั้น** — `super_admin`, `admin`, `user`
|
||||
4. **Super admin ตั้งค่าระบบเชิงลึกได้** — provider, base URL, model, generation/runtime parameters และ policy ที่เกี่ยวข้อง โดยไม่เปิด secret ให้ browser
|
||||
5. **Admin ใช้งานระบบและจัดการ users ในองค์กรของตนได้** — ห้ามเปลี่ยน/มอบ `super_admin`
|
||||
6. **User ใช้งาน simulation/report ของตนได้เท่านั้น**
|
||||
7. **ระบบรองรับการ deploy แบบ SaaS จริง** — ไม่พึ่ง in-memory state หรือ background thread ภายใน web process เป็นหลัก
|
||||
|
||||
## 2. สิ่งที่ตรวจพบจาก baseline
|
||||
|
||||
### 2.1 Baseline ที่ผ่าน/ไม่ผ่าน
|
||||
|
||||
- `git status --short --branch`: clean, branch `main` ตรงกับ `origin/main`
|
||||
- `npm run build`: **ผ่าน**; Vite build สำเร็จ แต่เตือน chunk หลักเกิน 500 kB และ dynamic/static import ของ `pendingUpload.js`
|
||||
- `python3 -m compileall -q backend/app backend/run.py backend/scripts`: **ผ่าน** (syntax-only; เครื่องปัจจุบันเป็น Python 3.14 แต่โปรเจกต์กำหนด `<3.13` จึงยังไม่ใช่ runtime validation)
|
||||
- `git diff --check`: **ผ่าน**
|
||||
- ไม่พบ test suite จริงที่ครอบคลุม backend/frontend; พบเพียง `backend/scripts/test_profile_format.py`
|
||||
- Backend มี **64 routes** ใน 5 blueprints และยังไม่มี authentication middleware
|
||||
- Frontend มี 16 `.vue` files; source frontend 22 files มี CJK code points รวมจำนวนมาก และ build artifact ยังมี CJK อยู่ จึง **ยังไม่ผ่าน** acceptance ของข้อ 1
|
||||
|
||||
### 2.2 ปัญหาหลักที่ยืนยันจาก source
|
||||
|
||||
| พื้นที่ | หลักฐาน | ผลกระทบ |
|
||||
|---|---|---|
|
||||
| i18n default/fallback | `frontend/src/i18n/index.js:17,22` ตั้ง default/fallback เป็น `zh` | ผู้ใช้ใหม่/ผู้ใช้เก่าที่มี `localStorage.locale=zh` อาจกลับไป Chinese |
|
||||
| HTML metadata/font | `frontend/index.html:2,4,7,11-12` ใช้ `lang=zh`, default `zh`, `Noto Sans SC`, title/description Chinese | browser chrome/metadata และ build ยังเผย Chinese |
|
||||
| locale registry | `locales/languages.json:2-4` มี `zh` และ instruction ให้ตอบ Chinese | LLM output และ language switcher ยังมี Chinese path |
|
||||
| backend locale fallback | `backend/app/utils/locale.py:30-32,36-49,66-69` fallback เป็น `zh` | API/background task อาจสร้างข้อความ Chinese แม้ frontend เลือกภาษาอื่น |
|
||||
| hard-coded frontend UI | `frontend/src/views/Process.vue` มี rendered Chinese หลายจุด; `frontend/src/components/Step4Report.vue:1349+` มี parser ที่ผูกกับ Chinese headings | แค่แก้ locale JSON ไม่พอ และ parser จะพังเมื่อ output เป็น Thai/English |
|
||||
| hard-coded frontend prompt | `frontend/src/components/Step5Interaction.vue:731-733` สร้าง prompt ด้วย Chinese labels | ระบบส่ง Chinese เข้า LLM จาก frontend runtime |
|
||||
| old Zep terminology | `frontend/src/views/Process.vue:320`, `locales/en.json`/`th.json` มี Zep strings | product UI ยังสื่อว่าต้องมี Zep |
|
||||
| Zep package/config | `backend/pyproject.toml:19-20`, `backend/requirements.txt:16-17`, `backend/uv.lock` | dependency และ lockfile ยังบังคับ Zep |
|
||||
| Zep graph build | `backend/app/services/graph_builder.py:13-18,121-180,205-291,294-505` | ontology, batch episodes, async processing, temporal graph info ผูกกับ SDK |
|
||||
| Zep read/search | `backend/app/services/zep_entity_reader.py:10,127-180,215-331`; `backend/app/services/zep_tools.py:425-544,650-1090,1145-1270` | profile generation/report agent ต้องการ node/edge/search/temporal result |
|
||||
| Zep runtime updates | `backend/app/services/zep_graph_memory_updater.py:15,202-246,396-455` | ทุก activity ถูกแปลงเป็น text แล้วส่งเข้า Zep |
|
||||
| ReportAgent coupling | `backend/app/services/report_agent.py:25-30,883-907,1156-1178` | report tools รับ `ZepToolsService` โดยตรง |
|
||||
| ไม่มี auth | `backend/app/__init__.py:65-73`; route เช่น `backend/app/api/graph.py:36-67` | ทุกคนที่เข้าถึง API รู้ project ID ก็อ่าน/แก้/ลบ resource ได้ |
|
||||
| CORS กว้างเกินไป | `backend/app/__init__.py:42-43` ใช้ `origins: "*"` | ไม่เหมาะกับ authenticated SaaS |
|
||||
| insecure defaults | `backend/app/config.py:65-69` มี default secret และ DEBUG=True | เสี่ยง production และ session/auth ในอนาคต |
|
||||
| single-user storage | `backend/app/models/project.py:101-219` ใช้ directory เดียวทั้งระบบ | ไม่มี owner/org scope, query, transaction หรือ multi-instance safety |
|
||||
| ephemeral state | `backend/app/models/task.py:56-72`; `backend/app/services/simulation_runner.py:219-224` | restart/หลาย worker ทำให้ task/process state หายหรือแยกกัน |
|
||||
| production runtime | `Dockerfile:18-29` ติดตั้งและรัน `npm run dev`; `docker-compose.yml:1-14` มี service เดียว/volume | ยังไม่ใช่ web/API/worker/data topology สำหรับ SaaS |
|
||||
| raw request logging | `backend/app/__init__.py:51-57` log JSON request body | อาจบันทึก source document, simulation requirement, chat history หรือ secret ลง log ต้อง redaction/ปิดใน production |
|
||||
| raw exception/path exposure | หลาย route คืน `str(e)`; resource paths ต่อจาก caller id เช่น `project.py:113-120`, `simulation_manager.py:139-143`, `report_agent.py:1910-1918` | อาจเปิด traceback, filesystem path หรือข้อมูลภายใน และเสี่ยง path traversal ต้องใช้ scoped repository + safe opaque IDs + generic error envelope |
|
||||
| retry/idempotency | `frontend/src/api/index.js:64-76` retry POST บางประเภท เช่น graph/simulation/report | response หลุดหลัง server ทำงานสำเร็จอาจสร้าง project/job/LLM cost ซ้ำ ต้องมี idempotency key และ durable job deduplication |
|
||||
| contract drift | `frontend/src/api/report.js:15-16` เรียก status ด้วย GET/query แต่ backend `report.py:203-230` รับ POST/body | ต้องทำ API contract tests ก่อนเพิ่ม role-specific UI |
|
||||
| browser-only draft state | `frontend/src/store/pendingUpload.js:7-31` เก็บ pending upload ใน memory | reload/tab close ทำให้ไฟล์และ intent หาย; SaaS ควรใช้ server-side draft หรือ presigned upload session |
|
||||
| branding/deployment mismatch | `package.json:2-4`, `README.md:3-9`, `docker-compose.yml:3` ใช้ CrowdSight ขณะที่ repo/task ใช้ MiroFish | ต้องตัดสินใจชื่อ product/canonical identifiers ก่อนทำ SaaS auth, domains และเอกสาร |
|
||||
|
||||
## 3. หลักการตัดสินใจ
|
||||
|
||||
### 3.1 ข้อเสนอที่ควรยึด
|
||||
|
||||
- **อย่าใช้ LLM แทน database/search engine โดยตรง**: LLM ทำ extraction, entity resolution, summary, query decomposition และ reranking ได้ แต่ไม่ควรรับผิดชอบ durable storage, pagination, exact ID lookup หรือ authorization
|
||||
- **ใช้ PostgreSQL เป็น system of record** ตั้งแต่ต้นสำหรับ SaaS; ใช้ SQLite ได้เฉพาะ local test/dev หากต้องการ
|
||||
- **ออกแบบ tenant boundary ตั้งแต่วันแรก** แม้เริ่มจากบริษัทเดียว เพื่อไม่ต้องรื้อ schema ภายหลัง
|
||||
- **เก็บ internal enum/field names เป็น English stable identifiers**; label ที่ user เห็นให้มาจาก `en/th` locale
|
||||
- **LLM settings ต้อง snapshot ตอนเริ่ม job** เพื่อให้ report/simulation reproducible แม้ super admin เปลี่ยน model ระหว่างรัน
|
||||
- **Frontend ไม่ควร parse Markdown ที่ผูกกับภาษา**; API ควรส่ง structured result แล้วให้ frontend render ด้วย locale
|
||||
- **ห้ามส่ง raw exception ให้ผู้ใช้**; log รายละเอียดไว้ server-side และส่ง `error_code` + localized message
|
||||
|
||||
### 3.2 Non-goals ของ v1
|
||||
|
||||
- Billing/subscription/usage metering
|
||||
- SSO/SAML/SCIM
|
||||
- Public self-signup แบบเปิดกว้าง
|
||||
- สิทธิ์แบบ custom role นอกเหนือจาก 3 role
|
||||
- การรับประกันว่า LLM ให้ผล semantic เหมือน Zep 100% — จะทำ parity ที่ interface/behavior และวัดด้วย golden fixtures แทน
|
||||
|
||||
## 4. Target architecture
|
||||
|
||||
```text
|
||||
Browser (Vue SPA)
|
||||
├─ Login / Auth store / route guards
|
||||
├─ User workspace
|
||||
├─ Admin user management
|
||||
└─ Super-admin settings
|
||||
│ HTTPS, HttpOnly auth cookie, Accept-Language: th|en
|
||||
▼
|
||||
Flask API (stateless web process)
|
||||
├─ auth + role + tenant policy
|
||||
├─ project/simulation/report API
|
||||
├─ memory API adapter
|
||||
├─ settings/audit API
|
||||
└─ job enqueue/status API
|
||||
│
|
||||
├──────── PostgreSQL ──────── users, orgs, projects, graph, jobs, reports, settings, audit
|
||||
├──────── Redis/queue ─────── graph extraction, report generation, simulation orchestration
|
||||
└──────── Object storage ──── uploads, reports, simulation artifacts/logs
|
||||
|
||||
Worker(s)
|
||||
├─ Memory extraction/merge worker (LLM)
|
||||
├─ Report worker (LLM + memory tools)
|
||||
└─ OASIS simulation worker
|
||||
|
||||
LLM Provider Gateway
|
||||
├─ provider/base_url/model selected by effective settings
|
||||
├─ server-side secret resolution
|
||||
├─ retry/timeout/rate/cost policy
|
||||
└─ structured output validation
|
||||
```
|
||||
|
||||
## 5. Domain/data model ที่เสนอ
|
||||
|
||||
### 5.1 Identity/tenant
|
||||
|
||||
- `organizations`: `id`, `name`, `slug`, `status`, timestamps
|
||||
- `users`: `id`, `email_normalized`, `password_hash` or external identity subject, `status`, `auth_version`, `locale`, `last_login_at`, timestamps
|
||||
- `memberships`: `user_id`, `organization_id`, `role` (`super_admin|admin|user`), status, timestamps; unique `(user_id, organization_id)`
|
||||
- `sessions` หรือ `refresh_tokens`: hashed token, user, expiry, revoked_at, rotation metadata
|
||||
|
||||
ใช้ memberships แทนการผูก user กับองค์กรเดียวแบบถาวร เพื่อรองรับ super-admin/platform scope และการเพิ่มหลายองค์กรภายหลัง โดยยังคงมี role เพียง 3 ค่า
|
||||
- `audit_logs`: actor, organization, action, target type/id, metadata, timestamp, IP/user-agent ที่จำเป็น
|
||||
|
||||
### 5.2 Product resources
|
||||
|
||||
- `projects`: `organization_id`, `owner_user_id`, name/status, source metadata, ontology JSON, language, timestamps
|
||||
- `simulations`: `organization_id`, `project_id`, `created_by`, status, config snapshot, worker/job id, timestamps
|
||||
- `reports`: `organization_id`, `project_id`, `simulation_id`, created_by, status, outline/sections/content metadata, timestamps
|
||||
- `jobs`: durable status/progress/error/result reference, `idempotency_key`, retry count, usage/cost metadata; ห้ามพึ่ง in-memory `TaskManager`
|
||||
- `artifacts`: object key, checksum, content type, size, owner resource, retention metadata
|
||||
- `usage_events`: organization/user, operation, model, input/output tokens, estimated cost, timestamps
|
||||
- `audit_logs`: actor, organization, action, target and redacted metadata
|
||||
|
||||
Report agent logs contain prompts, tool results and model responses; treat them as tenant data, redact secrets and expose only through scoped role-aware endpoints
|
||||
|
||||
### 5.3 Local graph/memory ที่มาแทน Zep
|
||||
|
||||
- `memory_graphs`: graph id, organization, project, ontology JSON, build status/version
|
||||
- `memory_episodes`: graph id, source type (`document|simulation_action|manual`), source reference, raw/normalized text, processing status, timestamps
|
||||
- `memory_nodes`: graph id, canonical name, normalized name, labels, attributes JSONB, summary, aliases JSONB, confidence, created/updated timestamps
|
||||
- `memory_edges`: graph id, source node, target node, relation name, fact, attributes JSONB, confidence, `valid_at`, `invalid_at`, `expired_at`, created timestamp
|
||||
- `memory_evidence`: episode-to-node/edge links, evidence span/reference, extractor version
|
||||
- optional `memory_embeddings`: pgvector or external vector index; defer until baseline local search is measured
|
||||
|
||||
Unique/index rules:
|
||||
|
||||
- unique `(graph_id, normalized_canonical_name)` where applicable
|
||||
- indexes on graph, labels, relation, source/target, temporal fields
|
||||
- full-text/trigram index for quick search
|
||||
- all repository methods require `organization_id`/graph scope; never accept an unscoped graph id from a route
|
||||
|
||||
### 5.4 Settings
|
||||
|
||||
- `platform_settings`: global active LLM provider/model/runtime settings, version, updated_by
|
||||
- optional `organization_settings`: only if later allowing per-org overrides
|
||||
- secret values are encrypted server-side or referenced from a secret manager; API returns masked status only
|
||||
- job stores a settings snapshot/version, not secret plaintext
|
||||
|
||||
## 6. Zep replacement design
|
||||
|
||||
### 6.1 Preserve a compatibility interface, replace implementation
|
||||
|
||||
Create an internal interface such as `MemoryRepository` / `MemorySearchService` whose output contracts preserve what the current UI/report code needs:
|
||||
|
||||
- `NodeInfo`: uuid, name, labels, summary, attributes
|
||||
- `EdgeInfo`: uuid, name, fact, source/target ids, temporal fields
|
||||
- `EntityNode` / `FilteredEntities`
|
||||
- `SearchResult`: facts, edges, nodes, query, total_count
|
||||
- `PanoramaResult`: active/historical facts, all nodes/edges, counts
|
||||
- `InsightForgeResult`: sub_queries, semantic facts, entity insights, relationship chains, counts
|
||||
|
||||
Then change consumers:
|
||||
|
||||
- `GraphBuilderService` → `GraphMemoryBuilder`
|
||||
- `ZepEntityReader` → `MemoryEntityReader`
|
||||
- `ZepToolsService` → `MemoryToolsService`
|
||||
- `ZepGraphMemoryUpdater` → `MemoryEventProcessor`
|
||||
- `OasisProfileGenerator` receives `MemorySearchService`
|
||||
- `ReportAgent` depends on an abstract memory tools interface, not a Zep-named class
|
||||
|
||||
This keeps API/frontend changes bounded while removing the external provider.
|
||||
|
||||
### 6.2 LLM extraction contract
|
||||
|
||||
LLM must return JSON only and pass Pydantic validation. Proposed shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"entities": [
|
||||
{
|
||||
"mention": "text span",
|
||||
"canonical_name": "stable name",
|
||||
"labels": ["Person"],
|
||||
"aliases": [],
|
||||
"attributes": {},
|
||||
"summary": "short evidence-grounded summary",
|
||||
"confidence": 0.0
|
||||
}
|
||||
],
|
||||
"edges": [
|
||||
{
|
||||
"source_entity_ref": "entity-1",
|
||||
"target_entity_ref": "entity-2",
|
||||
"relation": "WORKS_FOR",
|
||||
"fact": "evidence-grounded fact",
|
||||
"attributes": {},
|
||||
"valid_at": null,
|
||||
"invalid_at": null,
|
||||
"expired_at": null,
|
||||
"confidence": 0.0,
|
||||
"evidence": ["episode-id or span-id"]
|
||||
}
|
||||
],
|
||||
"episode_summary": "short summary",
|
||||
"unresolved_mentions": []
|
||||
}
|
||||
```
|
||||
|
||||
Prompt requirements:
|
||||
|
||||
1. System prompt is English and neutral; put the selected Thai/English output instruction at the **start**.
|
||||
2. Never invent a fact not supported by the episode/context.
|
||||
3. Use only ontology labels/relation names; field names and enum values are stable English identifiers.
|
||||
4. Preserve temporal semantics; use `null` when dates are not evidenced.
|
||||
5. Return entity references inside the same response, never database ids guessed by the model.
|
||||
6. Keep evidence references for audit/debugging.
|
||||
7. Return valid JSON with bounded array/string lengths.
|
||||
|
||||
Server-side post-processing must:
|
||||
|
||||
- normalize names/aliases
|
||||
- resolve references against deterministic candidate search first, then let LLM choose among candidates or create a new node
|
||||
- reject unknown labels/relations
|
||||
- clamp confidence and numeric fields
|
||||
- deduplicate edges
|
||||
- upsert transactionally
|
||||
- record extractor prompt/version/model in job metadata, never API key
|
||||
|
||||
### 6.3 Entity resolution and temporal update
|
||||
|
||||
1. Extract candidates from a chunk.
|
||||
2. Retrieve possible existing nodes using normalized name, aliases, trigram/full-text, and optionally embeddings.
|
||||
3. Ask LLM only to choose `existing_node_id` or `new_entity`, with evidence.
|
||||
4. Upsert node and edges in one transaction.
|
||||
5. When a new fact contradicts an active fact, mark the previous edge `invalid_at`/`expired_at`; do not delete history.
|
||||
6. Rebuild summaries from canonical facts/events through a separate LLM prompt.
|
||||
|
||||
### 6.4 Search parity
|
||||
|
||||
- **Quick search**: local full-text/trigram retrieval of facts/nodes; optional LLM query rewrite only.
|
||||
- **Panorama search**: deterministic graph scan with active/historical classification and bounded result set.
|
||||
- **InsightForge**: LLM decomposes the question into subqueries; local retrieval gathers candidates; deterministic dedupe; optional LLM reranker/synthesizer can rank candidate IDs but may not create unsupported facts.
|
||||
- **Entity context**: fetch node + adjacent edges + related nodes from repository.
|
||||
- **Interview agents**: remains an OASIS operation, not graph search; keep separate from memory repository.
|
||||
|
||||
The user-facing contract can be equivalent to current Zep-backed dataclasses, but the semantic result will only be considered acceptable after golden-fixture comparison.
|
||||
|
||||
### 6.5 Dynamic simulation memory
|
||||
|
||||
Do not generate Chinese natural-language episodes in the frontend/backend. Convert OASIS actions to canonical event JSON first:
|
||||
|
||||
```json
|
||||
{
|
||||
"simulation_id": "...",
|
||||
"platform": "twitter",
|
||||
"agent_id": 12,
|
||||
"action_type": "CREATE_POST",
|
||||
"action_args": {},
|
||||
"round_num": 4,
|
||||
"timestamp": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Use deterministic mappings for obvious actions (`FOLLOW`, `LIKE_POST`, `REPOST`) and LLM extraction only for content/stance/context enrichment. Queue batches through the worker and persist failure/retry status.
|
||||
|
||||
### 6.6 Migration decision
|
||||
|
||||
- **Recommended if no production Zep data exists:** clean break; rebuild graph from stored source documents under the new repository.
|
||||
- **If existing Zep data matters:** before removing credentials, run an export/import job for nodes, edges, temporal fields, episodes and evidence; verify counts/checksums and sample search behavior. Keep import code as a one-time script, not runtime dependency.
|
||||
|
||||
## 7. Prompt architecture
|
||||
|
||||
Centralize prompts in a versioned module, for example `backend/app/services/prompts/`:
|
||||
|
||||
- `ontology.py`
|
||||
- `memory_extraction.py`
|
||||
- `entity_resolution.py`
|
||||
- `memory_summary.py`
|
||||
- `query_decomposition.py`
|
||||
- `report.py`
|
||||
- `profile.py`
|
||||
|
||||
Every prompt receives an explicit `OutputLanguage` (`th` or `en`) and a `prompt_version`. Use structured output validators and retry with a repair prompt that contains the validation error, not an unbounded second generation.
|
||||
|
||||
Important current gaps to fix during this work:
|
||||
|
||||
- `backend/app/services/ontology_generator.py:284-309` still builds the user message with Chinese headings.
|
||||
- `backend/app/services/zep_tools.py:1138-1143` has Chinese fallback subqueries.
|
||||
- several config/profile/report prompts append language instructions inconsistently; move to one prompt builder that prepends a strong instruction.
|
||||
- API error handlers currently sometimes return `str(e)`; use stable codes/localized messages instead.
|
||||
|
||||
## 8. SaaS authorization model
|
||||
|
||||
### 8.1 Role matrix
|
||||
|
||||
| Capability | `super_admin` | `admin` | `user` |
|
||||
|---|---:|---:|---:|
|
||||
| Login/use simulation | yes | yes | yes |
|
||||
| View own projects/reports | yes | yes | yes |
|
||||
| View all projects in own org | yes | yes | no (recommended) |
|
||||
| Manage users in own org | yes | yes | no |
|
||||
| Assign `user` role | yes | yes | no |
|
||||
| Grant/revoke `admin` | yes | **no, recommended** | no |
|
||||
| Grant/revoke `super_admin` | yes | no | no |
|
||||
| Platform/org settings | yes | no | no |
|
||||
| LLM provider/model/base URL | yes | no | no |
|
||||
| Audit logs | all/platform scope | own org scope | no |
|
||||
|
||||
The backend is authoritative. Hiding a menu in Vue is not authorization.
|
||||
|
||||
### 8.2 Auth recommendation
|
||||
|
||||
- Login with email + password.
|
||||
- Argon2id (or Werkzeug scrypt) password hash; never store raw password.
|
||||
- Short-lived access session in `HttpOnly`, `Secure`, `SameSite` cookie; refresh/session rotation server-side.
|
||||
- CSRF protection for cookie-authenticated state-changing requests.
|
||||
- Rate-limit login, password reset and admin user mutations.
|
||||
- Normalize email and enforce unique `(organization_id, email_normalized)`.
|
||||
- On role/status/password change, revoke sessions through `auth_version` or token revocation.
|
||||
- Do not store auth tokens in `localStorage`.
|
||||
|
||||
### 8.3 Resource policy
|
||||
|
||||
Every project/simulation/report/graph/job route must:
|
||||
|
||||
1. authenticate request
|
||||
2. load resource through repository with tenant scope
|
||||
3. apply role policy
|
||||
4. only then read/write files or start a worker
|
||||
|
||||
Never trust `project_id`, `simulation_id` or `graph_id` supplied by the browser as proof of ownership.
|
||||
|
||||
### 8.4 API surface
|
||||
|
||||
Add:
|
||||
|
||||
- `POST /api/auth/login`
|
||||
- `POST /api/auth/logout`
|
||||
- `GET /api/auth/me`
|
||||
- `POST /api/auth/refresh` or session refresh
|
||||
- `POST /api/auth/change-password`
|
||||
- `GET/POST /api/users` — admin/super admin; scoped
|
||||
- `GET/PATCH/DELETE /api/users/<id>` — policy-enforced; soft delete/deactivate
|
||||
- `POST /api/users/<id>/reset-password` or invite flow
|
||||
- `GET/PATCH /api/admin/settings/llm` — super admin only
|
||||
- `POST /api/admin/settings/llm/test` — super admin only, redacted response
|
||||
- `GET /api/admin/audit-logs` — scoped by role
|
||||
|
||||
Protect all existing graph/simulation/report/template/agent-group routes. Keep `/health` unauthenticated but do not expose config/secrets.
|
||||
|
||||
## 9. Frontend migration
|
||||
|
||||
1. Add auth store and `/login`; add route guard and role-aware navigation.
|
||||
2. Add `/admin/users` for admin/super admin.
|
||||
3. Add `/admin/settings` for super admin only.
|
||||
4. Keep user workspace flow but scope API calls to authenticated identity; never put role authority only in Vue.
|
||||
5. Limit supported locales to `th` and `en`; recommend default `th`, with English switcher.
|
||||
6. Normalize legacy `localStorage.locale=zh` to `th`/`en` during boot and overwrite it.
|
||||
7. Remove CJK from `frontend/index.html`, `App.vue` font stack, hard-coded templates/strings, regex/parser labels, prompts and generated build.
|
||||
8. Replace language-specific report/tool parsing with structured API fields. For old Chinese report artifacts, either migrate/translate before serving or mark them as legacy and do not display unprocessed content if the strict no-Chinese requirement applies to historical data too.
|
||||
9. Rename/remove unused Chinese-named assets and repository docs as a separate cleanup pass; do not assume an image filename is harmless if it is later imported into the SPA.
|
||||
|
||||
Acceptance gate for frontend:
|
||||
|
||||
```text
|
||||
- no `zh` locale, `zh-CN`, `Noto Sans SC`, Chinese language label, or Chinese fallback in shipped frontend
|
||||
- no CJK code point in user-facing frontend source/locale/meta/build artifact, excluding explicitly approved user-uploaded content fixtures
|
||||
- all 16 Vue views/components render with `th` and `en`
|
||||
- API errors shown on screen are localized `th`/`en`
|
||||
- browser storage containing legacy `zh` self-heals to an allowed locale
|
||||
```
|
||||
|
||||
## 10. Settings design for super admin
|
||||
|
||||
Safe editable fields:
|
||||
|
||||
- provider preset and display name
|
||||
- model name
|
||||
- base URL allowlist/custom endpoint policy
|
||||
- temperature, max tokens, timeout, retry count/backoff
|
||||
- ontology/memory extraction batch size and token budget
|
||||
- report max tool calls/reflection rounds/token budget
|
||||
- OASIS max rounds/concurrency/retention limits
|
||||
- default UI/output language (`th|en`)
|
||||
- feature flags for memory search modes
|
||||
|
||||
Guardrails:
|
||||
|
||||
- server validates ranges and URL scheme/allowlist
|
||||
- API key field is write-only/masked; never return it
|
||||
- save creates version + audit log
|
||||
- test connection uses the pending settings without persisting unless explicitly saved
|
||||
- each job captures effective settings version/model/base URL (not secret)
|
||||
- admin/user cannot mutate these settings
|
||||
|
||||
## 11. Implementation phases and gates
|
||||
|
||||
### M0 — Decision lock and contracts
|
||||
|
||||
- Confirm language scope, tenancy, Zep data migration, onboarding, canonical product branding and deployment topology.
|
||||
- Freeze API/error/graph schemas, role constants and HTTP method/body contracts.
|
||||
- Define idempotency semantics for every LLM-triggering POST before enabling frontend retries.
|
||||
- Preserve the current clean baseline.
|
||||
- Record and fix known frontend/backend contract drift, including report status method/body mismatch, before adding route guards.
|
||||
|
||||
**Gate:** decisions recorded; no implementation starts against unresolved identity/data assumptions.
|
||||
|
||||
### M1 — Persistence, auth and tenant foundation
|
||||
|
||||
- Add migrations and PostgreSQL repository.
|
||||
- Add org/user/session/audit/job/resource tables.
|
||||
- Add auth endpoints, password hashing or managed identity integration, cookies/CSRF, role decorators/policies.
|
||||
- Attach `organization_id` and `owner_user_id` to product resources.
|
||||
- Replace unscoped file manager calls with scoped repositories and explicit path confinement.
|
||||
- Add idempotency keys, per-user/org rate limits, concurrency limits, input/output caps and LLM usage/cost accounting.
|
||||
- Redact request bodies, raw exceptions, filesystem paths, tracebacks and sensitive report logs from user-facing responses/logs.
|
||||
- Add contract tests for method/body mismatches and child-resource relationship checks.
|
||||
|
||||
**Gate:** automated role × endpoint × cross-tenant matrix passes; IDOR attempts return 404/403 without leakage.
|
||||
|
||||
### M2 — No-Zep memory repository
|
||||
|
||||
- Add graph/episode/node/edge/evidence tables and repository.
|
||||
- Implement structured LLM extraction, validation, merge, temporal update and summaries.
|
||||
- Implement quick/panorama/insight/entity-context adapters.
|
||||
- Replace profile/report/updater dependencies.
|
||||
- Remove Zep package, env var, imports and runtime checks.
|
||||
|
||||
**Gate:** no `zep_cloud` import/dependency/config; golden fixtures compare node/edge counts, entity recall, search recall@k, temporal classification and failure recovery against captured baseline or approved acceptance thresholds.
|
||||
|
||||
### M3 — Frontend Thai/English hardening
|
||||
|
||||
- Remove `zh` registry/fallback/default and legacy browser state.
|
||||
- Translate hard-coded rendered strings and backend error codes.
|
||||
- Replace report parser with structured JSON contract.
|
||||
- Remove Zep product terminology and CJK from shipped frontend.
|
||||
- Add login/role guards and admin navigation shell.
|
||||
|
||||
**Gate:** build + CJK scanner + visual/manual smoke at desktop and 320×568 / 500×768; both locales complete all core journeys.
|
||||
|
||||
### M4 — Admin and super-admin surfaces
|
||||
|
||||
- User list/create/invite/deactivate/reset/role policy.
|
||||
- Super-admin LLM settings, test connection, versioning and audit view.
|
||||
- Redacted settings API and permission tests.
|
||||
|
||||
**Gate:** each forbidden control is blocked server-side and hidden/disabled client-side; audit entries exist for sensitive changes.
|
||||
|
||||
### M5 — Worker/deployment hardening
|
||||
|
||||
- Move graph/report/simulation work to durable jobs/worker processes.
|
||||
- Decide Redis/queue and object storage; make artifact paths tenant-scoped.
|
||||
- Replace process-local pending uploads with server-side draft/upload sessions.
|
||||
- Replace dev Docker command with production frontend/API/worker services.
|
||||
- Add rate limits, structured redacted logs, metrics, usage/cost metering, retention and backup/restore procedure.
|
||||
|
||||
**Gate:** restart web process during a job does not lose job state; two workers do not cross tenant/resource boundaries; deploy/rollback smoke passes.
|
||||
|
||||
### M6 — Migration and release verification
|
||||
|
||||
- Rebuild from source or import Zep data according to M0 decision.
|
||||
- Run security scan, tests, E2E, locale scan, worker smoke, backup/restore and cost/latency benchmark.
|
||||
- Document rollback and known limitations.
|
||||
|
||||
- Project deletion has an explicit, tested cascade/retention policy for project files, graphs, simulations, reports, jobs and artifacts.
|
||||
- The frontend/backend API contract has no untested method/body drift; all retryable mutations are idempotent.
|
||||
- Request and report logs are redacted and cannot expose prompts, API keys, tracebacks or filesystem paths to unauthorized roles.
|
||||
|
||||
**Gate:** release checklist has evidence, not just green intentions.
|
||||
|
||||
## 12. Questions/decisions required before implementation
|
||||
|
||||
1. **ภาษาใน source content:** ต้องการห้าม Chinese เฉพาะ UI/system-generated text หรือแม้แต่ชื่อ entity, post, report ที่มาจากเอกสารที่ผู้ใช้อัปโหลดด้วย? — แนะนำ: UI และ generated system text ต้องไม่มี Chinese; user-provided source data ให้เก็บ original แต่มี display translation/locale policy แยก
|
||||
2. **Tenant scope:** ต้องการหลายบริษัท/หลายองค์กรตั้งแต่ v1 หรือบริษัทเดียวก่อน? — แนะนำ: ทำ `organization_id` ตั้งแต่ v1 แม้เปิดใช้บริษัทเดียว
|
||||
3. **Admin onboarding:** ใช้ email invite/SES หรือให้ admin สร้าง account โดยตรง? — แนะนำ: email invite + one-time setup token; ห้ามส่ง password ถาวรผ่านแชต/อีเมล
|
||||
4. **Zep data:** มี graph/project ที่ต้องรักษาไว้หรือ rebuild ได้? — แนะนำ: ถ้ายังไม่มี production data ให้ clean rebuild; ถ้ามี ให้ export/import ก่อนถอด Zep
|
||||
5. **LLM settings scope:** global ทั้งแพลตฟอร์มหรือแยกต่อองค์กร? — แนะนำ: global ใน v1, แต่ schema รองรับ org override ภายหลัง
|
||||
6. **Deployment:** ยอมรับ PostgreSQL + Redis + object storage/volume แยกหรือไม่? — แนะนำ: production SaaS ต้องแยก; single-container เป็นแค่ local/demo
|
||||
|
||||
## 13. Immediate next action after approval
|
||||
|
||||
ทำ M0 ให้จบด้วยคำตอบ 6 ข้อด้านบน แล้วแตก implementation plan เป็น PR-sized batches โดยเริ่มจาก **M1 persistence/auth contract** และ **M3 locale contract** ก่อนแตะ memory replacement; ห้ามเริ่มจากการลบ Zep imports แบบกระจาย เพราะจะทำให้ไม่มี storage/search contract รองรับและเสี่ยงทำ behavior เดิมหาย.
|
||||
Reference in New Issue
Block a user