docs: refresh HANDOFF + add 2026-08-09 engineering-log entry (idea-flow, live QA, LLM-judge decisions, persona variant)
This commit is contained in:
Macky
134
docs/HANDOFF.md
134
docs/HANDOFF.md
@@ -2,62 +2,104 @@
|
|||||||
|
|
||||||
> Another AI should be able to resume without chat history.
|
> Another AI should be able to resume without chat history.
|
||||||
|
|
||||||
## Branch / repo
|
## Branch / repo / deploy
|
||||||
- Repo: `~/Gitea/Sales Trainer/` (local git initialized; **no remote yet**).
|
- Repo: `~/Gitea/Sales Trainer/` — **git repo, remote = Gitea**.
|
||||||
- Branch: `main` (default).
|
- Remote: `https://git.moreminimore.com/kunthawat/sales-trainer.git` (GITEA_TOKEN via credential
|
||||||
|
helper; never committed).
|
||||||
|
- **Live deploy:** `https://moreminimoreapps-saletrainer.ahkhwd.easypanel.host` — EasyPanel,
|
||||||
|
auto-redeploys from Gitea on push to `main` via webhook (≈3 min). Dockerfile ships prebuilt
|
||||||
|
`frontend/dist/` (no npm in image). LLM vars set in EasyPanel env.
|
||||||
|
|
||||||
## What this is
|
## What this is
|
||||||
Corporate multi-user sales-training simulator. Vue SPA + Flask API + filesystem JSON storage.
|
Corporate multi-user **sales-training simulator**: admins create persona groups from a
|
||||||
Admins build persona groups from a product (form + upload); trainees chat one-shot against
|
product/service/idea (โฟกัส "สินค้า/บริการ/ไอเดีย"), app auto-generates **15 customer personas**
|
||||||
generated customer personas to practice closing; judge-LLM scores + coaches.
|
(5/tier A/B/C) via LLM; trainees pick a **scenario (social / พบหน้า)**, chat 1:1 one-shot to close a
|
||||||
|
sale; a **per-turn + final judge LLM** evaluates feelings and scores/coaches. Vue SPA + Flask API +
|
||||||
|
filesystem JSON storage (no SQL). i18n TH/EN. No self-registration (admin provisions).
|
||||||
|
|
||||||
## Current state — COMPLETE (M0–M7), prototype verified with mock LLM
|
## Roles
|
||||||
All backend + frontend built. All 4 backend test suites pass. Frontend builds. Live HTTP smoke
|
- **super_admin** (bootstrap `admin`) — full recipe (secret persona fields) + tenant admin.
|
||||||
test passes (SPA served, login, group create, register->404).
|
- **admin** — manages groups/users, sees personas with **secret fields stripped** (IP protection).
|
||||||
|
- **user** (trainee) — trains against personas, own board.
|
||||||
|
|
||||||
|
## Current state — COMPLETE core + hardened
|
||||||
|
All backend + frontend built. **11 test suites green** (mock LLM):
|
||||||
|
|
||||||
## Verified commands
|
|
||||||
```bash
|
```bash
|
||||||
# Backend tests (mock LLM, no key needed)
|
|
||||||
cd backend
|
cd backend
|
||||||
uv run python scripts/test_m0.py # auth/roles/no-self-reg
|
uv run python scripts/test_m0.py # auth/roles/no-self-reg
|
||||||
uv run python scripts/test_m1.py # group create + role visibility
|
uv run python scripts/test_m1.py # group create + role visibility
|
||||||
uv run python scripts/test_routes.py # 21 routes registered
|
uv run python scripts/test_setup.py # first-time admin setup (email+password+ToS)
|
||||||
uv run python scripts/test_e2e.py # full flow (analyze->personas->chat->debrief->one-shot->board->analytics)
|
uv run python scripts/test_security.py # path traversal / IDOR / XSS
|
||||||
|
uv run python scripts/test_scenario.py # 2 scenarios (social/f2f_call), recontact default
|
||||||
|
uv run python scripts/test_e2e.py # full flow -> won via judge
|
||||||
|
uv run python scripts/test_ip_protection.py # secret fields hidden from admin
|
||||||
|
uv run python scripts/test_saas_tenant.py # tenant isolation + rate-limit + audit
|
||||||
|
uv run python scripts/test_user_journey.py # idea-flow end-to-end
|
||||||
|
uv run python scripts/test_variant.py # clone-persona-from-persona
|
||||||
|
uv run python scripts/test_resume_decision.py # resume + per-turn LLM decision
|
||||||
|
|
||||||
# Run backend
|
# run backend (serves SPA from frontend/dist)
|
||||||
cd backend && uv run python run.py # Flask :5001 (serves built frontend from frontend/dist)
|
cd backend && uv run python run.py # Flask :5001
|
||||||
|
|
||||||
# Frontend dev
|
|
||||||
cd frontend && npm install && npm run dev # Vite :3000 proxying /api -> :5001
|
|
||||||
# Frontend build
|
|
||||||
cd frontend && npm run build # outputs frontend/dist
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Default account
|
## Key behaviors (implemented)
|
||||||
- super_admin: `admin` / `1234` (bootstrap). First login FORCES setting email + changing the
|
- **One-shot:** 1 persona = 1 chat per user; result final (won/lost). `SessionStore` enforces.
|
||||||
password (must_setup flow) before use.
|
- **Resume:** unfinished session resumes on re-entry — **no** scenario re-pick (same session+scenario).
|
||||||
|
- **Win/loss = per-turn LLM judge** (`Simulator.evaluate_turn`): every customer reply is evaluated →
|
||||||
|
`{mood, decision(buy|walk|pending), score_delta, reason}`; session ends when decision = buy/walk.
|
||||||
|
**Not** fixed keywords.
|
||||||
|
- **2 scenarios only:** `social`, `f2f_call`. Unknown → social.
|
||||||
|
- **Recontact = persona trait** (not a scenario): chats normally, then at turn ≥ 2 a time-lapse
|
||||||
|
system note ("⏳ ผ่านไป 2-3 สัปดาห์…"), then re-engages warmer.
|
||||||
|
- **Persona variant:** `POST /api/groups/<gid>/personas/<pid>/variant` — new persona (new id) that
|
||||||
|
**locks** pains/objections/levers/tolerance/special/recontact/goal/budget/difficulty/tier/product
|
||||||
|
**but varies** identity (name/profession/age/location/background/personality/opener). Lets a
|
||||||
|
trainee re-practice the same challenge (one-shot is per-persona). UI button on finished personas.
|
||||||
|
- **Auto 15 personas** on create; no "เพิ่มเติม" button (TARGET=15, retry up to 3× + accept ≥ 8 so
|
||||||
|
real LLM under-count doesn't 500).
|
||||||
|
- **IP protection:** `SECRET_PERSONA_FIELDS` (pains, objections, negotiation_levers, opener,
|
||||||
|
tolerance, rootCause, resolutionConditions) stripped for `admin`; full only for `super_admin`.
|
||||||
|
- **SaaS Phase 1–3 done:** tenant isolation (`g.org_id` + `assert_tenant`), login/chat rate-limit,
|
||||||
|
audit log (`data/audit/audit.jsonl`), org plan/seats/active model + `PATCH /api/admin/orgs/<id>`,
|
||||||
|
ToS consent on setup, org-scoped signed expiring CSV export (5-min HMAC).
|
||||||
|
- **Reduce raw JSON in UI:** `list_groups` returns lightweight summary (persona_count, no full array);
|
||||||
|
persona detail rendered as readable form/cards (pain = line-by-line, not `[object Object]`).
|
||||||
|
|
||||||
## Key gotchas
|
## Credentials / data (testing)
|
||||||
1. **Do NOT invoke `.venv/bin/python <script>` directly** — the tool lifecycle guard crashes
|
- Bootstrap super-admin `admin` / `1234` → first login forces email + new password + ToS consent.
|
||||||
("embedded null byte"). Always: `uv run python scripts/<name>.py`.
|
- Live test users: `testadmin` / `1234` (admin), `testuser` / `1234` (user).
|
||||||
2. LLM creds in `.env` (backend/.env for local; root `.env` for compose). `LLM_API_KEY=replace_me`
|
- A test group "CRM ระบบจัดการลูกค้า" exists on live (user keeps it; will delete it themselves).
|
||||||
is a placeholder → LLM is None → analyze/chat return 500 "LLM not configured".
|
- **LLM key is a placeholder on local `.env`** (`replace_me`). Real analyze/chat needs a real
|
||||||
3. SPA fallback in `app/factory._register_frontend` accepts all HTTP methods and 404s `/api/*`
|
`LLM_API_KEY` (+ `LLM_PROVIDER`/`LLM_MODEL`/`LLM_BASE_URL`) in EasyPanel env then redeploy.
|
||||||
so no-self-registration holds.
|
|
||||||
|
|
||||||
## Blockers / open items
|
## Environment / gotchas
|
||||||
- **Real-LLM E2E not yet run** (needs a live API key). This is the #1 item.
|
- **Python 3.11 only** (system default 3.14 incompatible). Use `backend/.venv` or
|
||||||
- Docker image not built locally (no Docker on this Mac). Validate on EasyPanel.
|
`cd backend && uv run python …`.
|
||||||
- No git remote set (Gitea).
|
- **Tooling guard crash:** commands whose first token is `./.venv/bin/python` trip a lifecycle guard
|
||||||
|
→ always use `uv run python`. Prefix `PYTHONPATH=` when needed.
|
||||||
|
- **Frontend build:** `cd frontend && npm run build` (works even with allowScripts restrictions).
|
||||||
|
Commit `frontend/dist/` with `git add -f` (it's gitignored otherwise); the Dockerfile needs it.
|
||||||
|
- **Deploy pattern:** commit + push → webhook auto-deploys in ~3 min. Cannot run Docker locally
|
||||||
|
(no Docker on this Mac) → test with nginx/`python http.server` or `uv run python run.py`.
|
||||||
|
- **No remote push without asking** unless it's the established auto-deploy cadence.
|
||||||
|
- `search_files`/`read_file` sometimes misdetect files as binary (e.g. `users.py`, some `.vue`);
|
||||||
|
use `terminal` + `sed -n 'N,Mp' <file>` or `awk` for those.
|
||||||
|
|
||||||
## Exact next actions
|
## Wire-authorized / known-limits
|
||||||
1. Set real `LLM_PROVIDER` + `LLM_API_KEY` (and optionally base/model) in `backend/.env`.
|
- Chrome/Vivaldi drivable via computer-use; current model (DeepSeek V4 Flash) reads screenshots
|
||||||
2. Run a live smoke test: login → create group → analyze → pick persona → chat a few turns → finish → read debrief; confirm judge produces sane output (this exercises real analyzer/persona/chat/judge).
|
imperfectly, and window-edge resize to a 320px viewport didn't land — **mobile visual QA at
|
||||||
3. Fix any real-model issues surfaced (prompt drift, JSON parsing).
|
320/500px still needs the user's eyes** (open live URL on phone, or DevTools device toolbar).
|
||||||
4. Add Gitea remote + push. Optionally wire Gitea Actions / EasyPanel deploy.
|
Responsive CSS (640px, single-column, `flex-wrap`, `.btn-back`) is present + deployed.
|
||||||
5. If EasyPanel: build from root `Dockerfile`, set env vars, map port 5001.
|
|
||||||
|
|
||||||
## Docs
|
## Next actions / backlog (also docs/FUTURE_WORK.md)
|
||||||
- `docs/PLAN.md` — full design + all confirmed decisions & open questions.
|
- Real `/legal` page (setup links to it), billing/payments, per-tenant storage volume, compressed
|
||||||
- `docs/engineering-log.md` + `docs/engineering-log/2026-08-07-build-out.md` — milestone record.
|
persona recipe, export-token polish.
|
||||||
- `README.md` — quick start, accounts, tests, LLM config.
|
- Mobile visual polish per user feedback on real device.
|
||||||
|
- Deeper: letting admins bulk-import personas, analytics drill-down per persona variant.
|
||||||
|
|
||||||
|
## Related docs
|
||||||
|
- `docs/PLAN.md`, `docs/SAAS_PLAN.md`, `docs/FUTURE_WORK.md`.
|
||||||
|
- `docs/engineering-log.md` + `docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md` (this session:
|
||||||
|
idea-flow UX, 2-scenario + recontact trait, live QA + auto-deploy, per-turn LLM judge, persona
|
||||||
|
variant, 15-persona auto-gen).
|
||||||
|
|||||||
@@ -39,3 +39,4 @@ Informed by MiroFish (CrowdSight engine) + the hermes-brain-and-tools CrowdSight
|
|||||||
- `2026-08-07-login-after-setup.md` — can't-login-after-setup = deployment data non-persistence, not login logic (verified).
|
- `2026-08-07-login-after-setup.md` — can't-login-after-setup = deployment data non-persistence, not login logic (verified).
|
||||||
- `2026-08-07-login-email-fix.md` — REAL fix: verify() resolves username OR email; "wrong password" after logout→login was email-login not resolving a user.
|
- `2026-08-07-login-email-fix.md` — REAL fix: verify() resolves username OR email; "wrong password" after logout→login was email-login not resolving a user.
|
||||||
- `2026-08-07-ui-tablayout.md` — 3-tab UI per spec: admin dashboard + date filter, personal dashboard, training w/ difficulty, settings profile.
|
- `2026-08-07-ui-tablayout.md` — 3-tab UI per spec: admin dashboard + date filter, personal dashboard, training w/ difficulty, settings profile.
|
||||||
|
- `2026-08-09-idea-flow-qa-deploy.md` — idea-flow UX cleanups, 2-scenario model + recontact-as-trait, live QA on EasyPanel + auto-deploy, per-turn LLM judge for win/loss, persona variant, 15-persona auto-gen (this session).
|
||||||
|
|||||||
86
docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md
Normal file
86
docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
# 2026-08-09 — Idea-flow, live QA, and hardening
|
||||||
|
|
||||||
|
Covers the pivot to **"use the app as designed"** (โฟกัสการใช้งานตามไอเดีย), real-flow QA on the
|
||||||
|
live deployment, and iterative UX/chat hardening driven by the user's hands-on testing.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
Prior summary had built the full app (M0–M7) + **SaaS multi-tenant Phase 1–3** (tenant isolation,
|
||||||
|
rate-limit, audit log, plan/seats model, ToS consent, signed export token) and pivoted the UI to
|
||||||
|
the "idea flow": product label → สินค้า/บริการ/ไอเดีย, scenario (social/พบหน้า) chosen at chat
|
||||||
|
time instead of a facebook/line channel. This session executed + verified that idea-flow and
|
||||||
|
delivered it to the live deployment.
|
||||||
|
|
||||||
|
## 1. UX clean-ups (from user's bug reports)
|
||||||
|
- **No raw JSON on the training page.** `list_groups` was returning full group objects incl. the
|
||||||
|
entire `personas` array → now returns lightweight summaries (id, title, status, `persona_count`,
|
||||||
|
product) — no long JSON, and less recipe leakage.
|
||||||
|
- **Chat button visible everywhere.** Added a "แชท" (MessageSquare) button on each persona card in
|
||||||
|
the GroupEdit (manage) page so admins can start a chat right from post-creation.
|
||||||
|
- **Removed facebook/line from tutta UI + persona model.** channel default changed `facebook`→`social`
|
||||||
|
in create/analyze/store/simulator/`own_persona`. Existing groups keep their stored value (old data).
|
||||||
|
- **Fixed pain showing `[object Object]`.** PersonaForm now extracts `.description` from pain objects
|
||||||
|
for a readable line-by-line textarea, and re-wraps as `{description}` on save.
|
||||||
|
|
||||||
|
## 2. Scenario model corrected (per user)
|
||||||
|
- Only **2 scenarios**: `social` and `f2f_call`. Removed the old "recontact" as a scenario.
|
||||||
|
- **"Recontact" is now a persona trait** (`recontact` bool), NOT a scenario. A recontact persona
|
||||||
|
**chats normally first**, then after enough info (turn ≥ 2) a time-lapse **system note**
|
||||||
|
("⏳ ผ่านไป 2-3 สัปดาห์…") is injected, then the customer re-engages warmer. (Mid-chat mechanic,
|
||||||
|
not "opens by saying I asked before".)
|
||||||
|
- `_scenarios` map is locale-aware; unknown scenario defaults to social.
|
||||||
|
|
||||||
|
## 3. Live QA + the bugs it surfaced
|
||||||
|
Deployed to `moreminimoreapps-saletrainer.ahkhwd.easypanel.host` (auto-deploy via Gitea webhook on
|
||||||
|
push to main). Verified via live API (deterministic) + Chrome via computer-use (login testadmin/1234,
|
||||||
|
testuser/1234).
|
||||||
|
- **Bug (blocker): analyze 500 when LLM returns < 15 personas.** Real deepseek occasionally returns
|
||||||
|
14. Fixed `persona_generator.generate`: **retry up to 3×**, then **accept short (≥ 8)** instead of
|
||||||
|
raising. Verified on live (analyze now returns 15 with retry).
|
||||||
|
- **Bug: chat resume didn't work.** `start_session` always created a new session → re-entering forced
|
||||||
|
a scenario re-pick. Fixed: if an active (unfinished) session exists for persona+user, **resume it**
|
||||||
|
(same session id + original scenario). `/chat/resume` also happy-path.
|
||||||
|
- **Bug: no win/loss when persona clearly refused.** Real LLMs don't emit structured `meta.decision`;
|
||||||
|
they say it in text ("ซื้อไม่ไหวแล้ว") which the old code ignored → session stuck active forever.
|
||||||
|
Fixed by **per-turn judge evaluation** (below).
|
||||||
|
|
||||||
|
## 4. Decision = per-turn LLM judge (NOT fixed keywords) ✅ user-requested
|
||||||
|
- Removed the fixed-value keyword text detector entirely.
|
||||||
|
- `persona_reply` now returns the customer's natural text reply (no forced JSON).
|
||||||
|
- Added `Simulator.evaluate_turn()`: after **every** customer reply, a **judge LLM** reads the
|
||||||
|
transcript + persona + internal state and returns `{mood(-2..+2), decision(buy|walk|pending),
|
||||||
|
score_delta(-15..+15), reason}`. Context-based: "ซื้อไม่ไหว แต่ว่ามีผ่อนไหม?" stays pending until
|
||||||
|
the customer truly commits/abandons.
|
||||||
|
- `send_message` consumes that decision to (a) end the session won/lost + build debrief, and (b) move
|
||||||
|
the internal score. Mock updated to return buy-on-first-send (keeps E2E deterministic).
|
||||||
|
|
||||||
|
## 5. "Create persona from this persona" (variant)
|
||||||
|
- New `POST /api/groups/<gid>/personas/<pid>/variant` — clones a source persona into a **new** persona
|
||||||
|
(fresh id) that **LOCKS core traits** (pains, objections, negotiation_levers, tolerance,
|
||||||
|
special/recontact, goal, budget, difficulty, tier, product_context) but **VARIES identity**
|
||||||
|
(name, profession, age, location, background, personality, opener). Added to same group.
|
||||||
|
- One-shot is per-persona, so the variant is chat-able again (practice the same challenge repeatedly,
|
||||||
|
never an identical copy). UI: finished (won/lost) persona shows "สร้างบุคคลต้นแบบจากต้นแบบนี้".
|
||||||
|
|
||||||
|
## 6. 15 personas auto-generated, no "เพิ่มเติม" button
|
||||||
|
- `TARGET=15` (PER_TIER=5). Removed the "สร้างบุคคลต้นแบบเพิ่มเติม" button + guide step; creation now
|
||||||
|
auto-generates 15.
|
||||||
|
|
||||||
|
## Test suite (all pass, mock LLM)
|
||||||
|
`test_m0, test_m1, test_setup, test_security, test_scenario, test_e2e, test_ip_protection,
|
||||||
|
test_saas_tenant, test_user_journey, test_variant, test_resume_decision` — **11 suites green.**
|
||||||
|
|
||||||
|
## Live state
|
||||||
|
- Live serves `index-rfeHzF-F.js` = latest build (verified). Auto-deploy on push (Gitea webhook).
|
||||||
|
- Real LLM (deepseek) requires `LLM_API_KEY` set in EasyPanel env — local `.env` has placeholder.
|
||||||
|
- testadmin=admin, testuser=user (both pass `1234`); bootstrap super-admin also exists.
|
||||||
|
- A test group "CRM ระบบจัดการลูกค้า" exists on live (user keeps it; will delete themselves).
|
||||||
|
|
||||||
|
## Open / future (see docs/FUTURE_WORK.md)
|
||||||
|
- `/legal` real page, billing/payments, per-tenant storage volume, compressed recipe, export-token polish.
|
||||||
|
- Mobile visual QA at 320/500px still needs the user's eyes (or a multimodal model); responsive CSS
|
||||||
|
(640px, single-column, flex-wrap, .btn-back) is present + deployed.
|
||||||
|
|
||||||
|
## Guardrails reaffirmed
|
||||||
|
- One persona = one chat per user; result final.
|
||||||
|
- Latent/secret persona fields hidden from admin (IP protection); content visible for super_admin.
|
||||||
|
- LLM creds in `.env` / EasyPanel env only; never logged.
|
||||||
Reference in New Issue
Block a user