docs: refresh HANDOFF + add 2026-08-09 engineering-log entry (idea-flow, live QA, LLM-judge decisions, persona variant)

This commit is contained in:
Macky
2026-08-09 19:18:10 +07:00
parent c92400b195
commit 6811dc1db9
3 changed files with 175 additions and 46 deletions

View File

@@ -2,62 +2,104 @@
> Another AI should be able to resume without chat history.
## Branch / repo
- Repo: `~/Gitea/Sales Trainer/` (local git initialized; **no remote yet**).
- Branch: `main` (default).
## Branch / repo / deploy
- Repo: `~/Gitea/Sales Trainer/` **git repo, remote = Gitea**.
- Remote: `https://git.moreminimore.com/kunthawat/sales-trainer.git` (GITEA_TOKEN via credential
helper; never committed).
- **Live deploy:** `https://moreminimoreapps-saletrainer.ahkhwd.easypanel.host` — EasyPanel,
auto-redeploys from Gitea on push to `main` via webhook (≈3 min). Dockerfile ships prebuilt
`frontend/dist/` (no npm in image). LLM vars set in EasyPanel env.
## What this is
Corporate multi-user sales-training simulator. Vue SPA + Flask API + filesystem JSON storage.
Admins build persona groups from a product (form + upload); trainees chat one-shot against
generated customer personas to practice closing; judge-LLM scores + coaches.
Corporate multi-user **sales-training simulator**: admins create persona groups from a
product/service/idea (โฟกัส "สินค้า/บริการ/ไอเดีย"), app auto-generates **15 customer personas**
(5/tier A/B/C) via LLM; trainees pick a **scenario (social / พบหน้า)**, chat 1:1 one-shot to close a
sale; a **per-turn + final judge LLM** evaluates feelings and scores/coaches. Vue SPA + Flask API +
filesystem JSON storage (no SQL). i18n TH/EN. No self-registration (admin provisions).
## Current state — COMPLETE (M0M7), prototype verified with mock LLM
All backend + frontend built. All 4 backend test suites pass. Frontend builds. Live HTTP smoke
test passes (SPA served, login, group create, register->404).
## Roles
- **super_admin** (bootstrap `admin`) — full recipe (secret persona fields) + tenant admin.
- **admin** — manages groups/users, sees personas with **secret fields stripped** (IP protection).
- **user** (trainee) — trains against personas, own board.
## Current state — COMPLETE core + hardened
All backend + frontend built. **11 test suites green** (mock LLM):
## Verified commands
```bash
# Backend tests (mock LLM, no key needed)
cd backend
uv run python scripts/test_m0.py # auth/roles/no-self-reg
uv run python scripts/test_m1.py # group create + role visibility
uv run python scripts/test_routes.py # 21 routes registered
uv run python scripts/test_e2e.py # full flow (analyze->personas->chat->debrief->one-shot->board->analytics)
uv run python scripts/test_m0.py # auth/roles/no-self-reg
uv run python scripts/test_m1.py # group create + role visibility
uv run python scripts/test_setup.py # first-time admin setup (email+password+ToS)
uv run python scripts/test_security.py # path traversal / IDOR / XSS
uv run python scripts/test_scenario.py # 2 scenarios (social/f2f_call), recontact default
uv run python scripts/test_e2e.py # full flow -> won via judge
uv run python scripts/test_ip_protection.py # secret fields hidden from admin
uv run python scripts/test_saas_tenant.py # tenant isolation + rate-limit + audit
uv run python scripts/test_user_journey.py # idea-flow end-to-end
uv run python scripts/test_variant.py # clone-persona-from-persona
uv run python scripts/test_resume_decision.py # resume + per-turn LLM decision
# Run backend
cd backend && uv run python run.py # Flask :5001 (serves built frontend from frontend/dist)
# Frontend dev
cd frontend && npm install && npm run dev # Vite :3000 proxying /api -> :5001
# Frontend build
cd frontend && npm run build # outputs frontend/dist
# run backend (serves SPA from frontend/dist)
cd backend && uv run python run.py # Flask :5001
```
## Default account
- super_admin: `admin` / `1234` (bootstrap). First login FORCES setting email + changing the
password (must_setup flow) before use.
## Key behaviors (implemented)
- **One-shot:** 1 persona = 1 chat per user; result final (won/lost). `SessionStore` enforces.
- **Resume:** unfinished session resumes on re-entry — **no** scenario re-pick (same session+scenario).
- **Win/loss = per-turn LLM judge** (`Simulator.evaluate_turn`): every customer reply is evaluated →
`{mood, decision(buy|walk|pending), score_delta, reason}`; session ends when decision = buy/walk.
**Not** fixed keywords.
- **2 scenarios only:** `social`, `f2f_call`. Unknown → social.
- **Recontact = persona trait** (not a scenario): chats normally, then at turn ≥ 2 a time-lapse
system note ("⏳ ผ่านไป 2-3 สัปดาห์…"), then re-engages warmer.
- **Persona variant:** `POST /api/groups/<gid>/personas/<pid>/variant` — new persona (new id) that
**locks** pains/objections/levers/tolerance/special/recontact/goal/budget/difficulty/tier/product
**but varies** identity (name/profession/age/location/background/personality/opener). Lets a
trainee re-practice the same challenge (one-shot is per-persona). UI button on finished personas.
- **Auto 15 personas** on create; no "เพิ่มเติม" button (TARGET=15, retry up to 3× + accept ≥ 8 so
real LLM under-count doesn't 500).
- **IP protection:** `SECRET_PERSONA_FIELDS` (pains, objections, negotiation_levers, opener,
tolerance, rootCause, resolutionConditions) stripped for `admin`; full only for `super_admin`.
- **SaaS Phase 13 done:** tenant isolation (`g.org_id` + `assert_tenant`), login/chat rate-limit,
audit log (`data/audit/audit.jsonl`), org plan/seats/active model + `PATCH /api/admin/orgs/<id>`,
ToS consent on setup, org-scoped signed expiring CSV export (5-min HMAC).
- **Reduce raw JSON in UI:** `list_groups` returns lightweight summary (persona_count, no full array);
persona detail rendered as readable form/cards (pain = line-by-line, not `[object Object]`).
## Key gotchas
1. **Do NOT invoke `.venv/bin/python <script>` directly** — the tool lifecycle guard crashes
("embedded null byte"). Always: `uv run python scripts/<name>.py`.
2. LLM creds in `.env` (backend/.env for local; root `.env` for compose). `LLM_API_KEY=replace_me`
is a placeholder → LLM is None → analyze/chat return 500 "LLM not configured".
3. SPA fallback in `app/factory._register_frontend` accepts all HTTP methods and 404s `/api/*`
so no-self-registration holds.
## Credentials / data (testing)
- Bootstrap super-admin `admin` / `1234` → first login forces email + new password + ToS consent.
- Live test users: `testadmin` / `1234` (admin), `testuser` / `1234` (user).
- A test group "CRM ระบบจัดการลูกค้า" exists on live (user keeps it; will delete it themselves).
- **LLM key is a placeholder on local `.env`** (`replace_me`). Real analyze/chat needs a real
`LLM_API_KEY` (+ `LLM_PROVIDER`/`LLM_MODEL`/`LLM_BASE_URL`) in EasyPanel env then redeploy.
## Blockers / open items
- **Real-LLM E2E not yet run** (needs a live API key). This is the #1 item.
- Docker image not built locally (no Docker on this Mac). Validate on EasyPanel.
- No git remote set (Gitea).
## Environment / gotchas
- **Python 3.11 only** (system default 3.14 incompatible). Use `backend/.venv` or
`cd backend && uv run python …`.
- **Tooling guard crash:** commands whose first token is `./.venv/bin/python` trip a lifecycle guard
→ always use `uv run python`. Prefix `PYTHONPATH=` when needed.
- **Frontend build:** `cd frontend && npm run build` (works even with allowScripts restrictions).
Commit `frontend/dist/` with `git add -f` (it's gitignored otherwise); the Dockerfile needs it.
- **Deploy pattern:** commit + push → webhook auto-deploys in ~3 min. Cannot run Docker locally
(no Docker on this Mac) → test with nginx/`python http.server` or `uv run python run.py`.
- **No remote push without asking** unless it's the established auto-deploy cadence.
- `search_files`/`read_file` sometimes misdetect files as binary (e.g. `users.py`, some `.vue`);
use `terminal` + `sed -n 'N,Mp' <file>` or `awk` for those.
## Exact next actions
1. Set real `LLM_PROVIDER` + `LLM_API_KEY` (and optionally base/model) in `backend/.env`.
2. Run a live smoke test: login → create group → analyze → pick persona → chat a few turns → finish → read debrief; confirm judge produces sane output (this exercises real analyzer/persona/chat/judge).
3. Fix any real-model issues surfaced (prompt drift, JSON parsing).
4. Add Gitea remote + push. Optionally wire Gitea Actions / EasyPanel deploy.
5. If EasyPanel: build from root `Dockerfile`, set env vars, map port 5001.
## Wire-authorized / known-limits
- Chrome/Vivaldi drivable via computer-use; current model (DeepSeek V4 Flash) reads screenshots
imperfectly, and window-edge resize to a 320px viewport didn't land — **mobile visual QA at
320/500px still needs the user's eyes** (open live URL on phone, or DevTools device toolbar).
Responsive CSS (640px, single-column, `flex-wrap`, `.btn-back`) is present + deployed.
## Docs
- `docs/PLAN.md` — full design + all confirmed decisions & open questions.
- `docs/engineering-log.md` + `docs/engineering-log/2026-08-07-build-out.md` — milestone record.
- `README.md` — quick start, accounts, tests, LLM config.
## Next actions / backlog (also docs/FUTURE_WORK.md)
- Real `/legal` page (setup links to it), billing/payments, per-tenant storage volume, compressed
persona recipe, export-token polish.
- Mobile visual polish per user feedback on real device.
- Deeper: letting admins bulk-import personas, analytics drill-down per persona variant.
## Related docs
- `docs/PLAN.md`, `docs/SAAS_PLAN.md`, `docs/FUTURE_WORK.md`.
- `docs/engineering-log.md` + `docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md` (this session:
idea-flow UX, 2-scenario + recontact trait, live QA + auto-deploy, per-turn LLM judge, persona
variant, 15-persona auto-gen).

View File

@@ -39,3 +39,4 @@ Informed by MiroFish (CrowdSight engine) + the hermes-brain-and-tools CrowdSight
- `2026-08-07-login-after-setup.md` — can't-login-after-setup = deployment data non-persistence, not login logic (verified).
- `2026-08-07-login-email-fix.md` — REAL fix: verify() resolves username OR email; "wrong password" after logout→login was email-login not resolving a user.
- `2026-08-07-ui-tablayout.md` — 3-tab UI per spec: admin dashboard + date filter, personal dashboard, training w/ difficulty, settings profile.
- `2026-08-09-idea-flow-qa-deploy.md` — idea-flow UX cleanups, 2-scenario model + recontact-as-trait, live QA on EasyPanel + auto-deploy, per-turn LLM judge for win/loss, persona variant, 15-persona auto-gen (this session).

View File

@@ -0,0 +1,86 @@
# 2026-08-09 — Idea-flow, live QA, and hardening
Covers the pivot to **"use the app as designed"** (โฟกัสการใช้งานตามไอเดีย), real-flow QA on the
live deployment, and iterative UX/chat hardening driven by the user's hands-on testing.
## Context
Prior summary had built the full app (M0M7) + **SaaS multi-tenant Phase 13** (tenant isolation,
rate-limit, audit log, plan/seats model, ToS consent, signed export token) and pivoted the UI to
the "idea flow": product label → สินค้า/บริการ/ไอเดีย, scenario (social/พบหน้า) chosen at chat
time instead of a facebook/line channel. This session executed + verified that idea-flow and
delivered it to the live deployment.
## 1. UX clean-ups (from user's bug reports)
- **No raw JSON on the training page.** `list_groups` was returning full group objects incl. the
entire `personas` array → now returns lightweight summaries (id, title, status, `persona_count`,
product) — no long JSON, and less recipe leakage.
- **Chat button visible everywhere.** Added a "แชท" (MessageSquare) button on each persona card in
the GroupEdit (manage) page so admins can start a chat right from post-creation.
- **Removed facebook/line from tutta UI + persona model.** channel default changed `facebook``social`
in create/analyze/store/simulator/`own_persona`. Existing groups keep their stored value (old data).
- **Fixed pain showing `[object Object]`.** PersonaForm now extracts `.description` from pain objects
for a readable line-by-line textarea, and re-wraps as `{description}` on save.
## 2. Scenario model corrected (per user)
- Only **2 scenarios**: `social` and `f2f_call`. Removed the old "recontact" as a scenario.
- **"Recontact" is now a persona trait** (`recontact` bool), NOT a scenario. A recontact persona
**chats normally first**, then after enough info (turn ≥ 2) a time-lapse **system note**
("⏳ ผ่านไป 2-3 สัปดาห์…") is injected, then the customer re-engages warmer. (Mid-chat mechanic,
not "opens by saying I asked before".)
- `_scenarios` map is locale-aware; unknown scenario defaults to social.
## 3. Live QA + the bugs it surfaced
Deployed to `moreminimoreapps-saletrainer.ahkhwd.easypanel.host` (auto-deploy via Gitea webhook on
push to main). Verified via live API (deterministic) + Chrome via computer-use (login testadmin/1234,
testuser/1234).
- **Bug (blocker): analyze 500 when LLM returns < 15 personas.** Real deepseek occasionally returns
14. Fixed `persona_generator.generate`: **retry up to 3×**, then **accept short (≥ 8)** instead of
raising. Verified on live (analyze now returns 15 with retry).
- **Bug: chat resume didn't work.** `start_session` always created a new session → re-entering forced
a scenario re-pick. Fixed: if an active (unfinished) session exists for persona+user, **resume it**
(same session id + original scenario). `/chat/resume` also happy-path.
- **Bug: no win/loss when persona clearly refused.** Real LLMs don't emit structured `meta.decision`;
they say it in text ("ซื้อไม่ไหวแล้ว") which the old code ignored → session stuck active forever.
Fixed by **per-turn judge evaluation** (below).
## 4. Decision = per-turn LLM judge (NOT fixed keywords) ✅ user-requested
- Removed the fixed-value keyword text detector entirely.
- `persona_reply` now returns the customer's natural text reply (no forced JSON).
- Added `Simulator.evaluate_turn()`: after **every** customer reply, a **judge LLM** reads the
transcript + persona + internal state and returns `{mood(-2..+2), decision(buy|walk|pending),
score_delta(-15..+15), reason}`. Context-based: "ซื้อไม่ไหว แต่ว่ามีผ่อนไหม?" stays pending until
the customer truly commits/abandons.
- `send_message` consumes that decision to (a) end the session won/lost + build debrief, and (b) move
the internal score. Mock updated to return buy-on-first-send (keeps E2E deterministic).
## 5. "Create persona from this persona" (variant)
- New `POST /api/groups/<gid>/personas/<pid>/variant` — clones a source persona into a **new** persona
(fresh id) that **LOCKS core traits** (pains, objections, negotiation_levers, tolerance,
special/recontact, goal, budget, difficulty, tier, product_context) but **VARIES identity**
(name, profession, age, location, background, personality, opener). Added to same group.
- One-shot is per-persona, so the variant is chat-able again (practice the same challenge repeatedly,
never an identical copy). UI: finished (won/lost) persona shows "สร้างบุคคลต้นแบบจากต้นแบบนี้".
## 6. 15 personas auto-generated, no "เพิ่มเติม" button
- `TARGET=15` (PER_TIER=5). Removed the "สร้างบุคคลต้นแบบเพิ่มเติม" button + guide step; creation now
auto-generates 15.
## Test suite (all pass, mock LLM)
`test_m0, test_m1, test_setup, test_security, test_scenario, test_e2e, test_ip_protection,
test_saas_tenant, test_user_journey, test_variant, test_resume_decision`**11 suites green.**
## Live state
- Live serves `index-rfeHzF-F.js` = latest build (verified). Auto-deploy on push (Gitea webhook).
- Real LLM (deepseek) requires `LLM_API_KEY` set in EasyPanel env — local `.env` has placeholder.
- testadmin=admin, testuser=user (both pass `1234`); bootstrap super-admin also exists.
- A test group "CRM ระบบจัดการลูกค้า" exists on live (user keeps it; will delete themselves).
## Open / future (see docs/FUTURE_WORK.md)
- `/legal` real page, billing/payments, per-tenant storage volume, compressed recipe, export-token polish.
- Mobile visual QA at 320/500px still needs the user's eyes (or a multimodal model); responsive CSS
(640px, single-column, flex-wrap, .btn-back) is present + deployed.
## Guardrails reaffirmed
- One persona = one chat per user; result final.
- Latent/secret persona fields hidden from admin (IP protection); content visible for super_admin.
- LLM creds in `.env` / EasyPanel env only; never logged.