docs: refresh HANDOFF + add 2026-08-09 engineering-log entry (idea-flow, live QA, LLM-judge decisions, persona variant)
This commit is contained in:
86
docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md
Normal file
86
docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# 2026-08-09 — Idea-flow, live QA, and hardening
|
||||
|
||||
Covers the pivot to **"use the app as designed"** (โฟกัสการใช้งานตามไอเดีย), real-flow QA on the
|
||||
live deployment, and iterative UX/chat hardening driven by the user's hands-on testing.
|
||||
|
||||
## Context
|
||||
Prior summary had built the full app (M0–M7) + **SaaS multi-tenant Phase 1–3** (tenant isolation,
|
||||
rate-limit, audit log, plan/seats model, ToS consent, signed export token) and pivoted the UI to
|
||||
the "idea flow": product label → สินค้า/บริการ/ไอเดีย, scenario (social/พบหน้า) chosen at chat
|
||||
time instead of a facebook/line channel. This session executed + verified that idea-flow and
|
||||
delivered it to the live deployment.
|
||||
|
||||
## 1. UX clean-ups (from user's bug reports)
|
||||
- **No raw JSON on the training page.** `list_groups` was returning full group objects incl. the
|
||||
entire `personas` array → now returns lightweight summaries (id, title, status, `persona_count`,
|
||||
product) — no long JSON, and less recipe leakage.
|
||||
- **Chat button visible everywhere.** Added a "แชท" (MessageSquare) button on each persona card in
|
||||
the GroupEdit (manage) page so admins can start a chat right from post-creation.
|
||||
- **Removed facebook/line from tutta UI + persona model.** channel default changed `facebook`→`social`
|
||||
in create/analyze/store/simulator/`own_persona`. Existing groups keep their stored value (old data).
|
||||
- **Fixed pain showing `[object Object]`.** PersonaForm now extracts `.description` from pain objects
|
||||
for a readable line-by-line textarea, and re-wraps as `{description}` on save.
|
||||
|
||||
## 2. Scenario model corrected (per user)
|
||||
- Only **2 scenarios**: `social` and `f2f_call`. Removed the old "recontact" as a scenario.
|
||||
- **"Recontact" is now a persona trait** (`recontact` bool), NOT a scenario. A recontact persona
|
||||
**chats normally first**, then after enough info (turn ≥ 2) a time-lapse **system note**
|
||||
("⏳ ผ่านไป 2-3 สัปดาห์…") is injected, then the customer re-engages warmer. (Mid-chat mechanic,
|
||||
not "opens by saying I asked before".)
|
||||
- `_scenarios` map is locale-aware; unknown scenario defaults to social.
|
||||
|
||||
## 3. Live QA + the bugs it surfaced
|
||||
Deployed to `moreminimoreapps-saletrainer.ahkhwd.easypanel.host` (auto-deploy via Gitea webhook on
|
||||
push to main). Verified via live API (deterministic) + Chrome via computer-use (login testadmin/1234,
|
||||
testuser/1234).
|
||||
- **Bug (blocker): analyze 500 when LLM returns < 15 personas.** Real deepseek occasionally returns
|
||||
14. Fixed `persona_generator.generate`: **retry up to 3×**, then **accept short (≥ 8)** instead of
|
||||
raising. Verified on live (analyze now returns 15 with retry).
|
||||
- **Bug: chat resume didn't work.** `start_session` always created a new session → re-entering forced
|
||||
a scenario re-pick. Fixed: if an active (unfinished) session exists for persona+user, **resume it**
|
||||
(same session id + original scenario). `/chat/resume` also happy-path.
|
||||
- **Bug: no win/loss when persona clearly refused.** Real LLMs don't emit structured `meta.decision`;
|
||||
they say it in text ("ซื้อไม่ไหวแล้ว") which the old code ignored → session stuck active forever.
|
||||
Fixed by **per-turn judge evaluation** (below).
|
||||
|
||||
## 4. Decision = per-turn LLM judge (NOT fixed keywords) ✅ user-requested
|
||||
- Removed the fixed-value keyword text detector entirely.
|
||||
- `persona_reply` now returns the customer's natural text reply (no forced JSON).
|
||||
- Added `Simulator.evaluate_turn()`: after **every** customer reply, a **judge LLM** reads the
|
||||
transcript + persona + internal state and returns `{mood(-2..+2), decision(buy|walk|pending),
|
||||
score_delta(-15..+15), reason}`. Context-based: "ซื้อไม่ไหว แต่ว่ามีผ่อนไหม?" stays pending until
|
||||
the customer truly commits/abandons.
|
||||
- `send_message` consumes that decision to (a) end the session won/lost + build debrief, and (b) move
|
||||
the internal score. Mock updated to return buy-on-first-send (keeps E2E deterministic).
|
||||
|
||||
## 5. "Create persona from this persona" (variant)
|
||||
- New `POST /api/groups/<gid>/personas/<pid>/variant` — clones a source persona into a **new** persona
|
||||
(fresh id) that **LOCKS core traits** (pains, objections, negotiation_levers, tolerance,
|
||||
special/recontact, goal, budget, difficulty, tier, product_context) but **VARIES identity**
|
||||
(name, profession, age, location, background, personality, opener). Added to same group.
|
||||
- One-shot is per-persona, so the variant is chat-able again (practice the same challenge repeatedly,
|
||||
never an identical copy). UI: finished (won/lost) persona shows "สร้างบุคคลต้นแบบจากต้นแบบนี้".
|
||||
|
||||
## 6. 15 personas auto-generated, no "เพิ่มเติม" button
|
||||
- `TARGET=15` (PER_TIER=5). Removed the "สร้างบุคคลต้นแบบเพิ่มเติม" button + guide step; creation now
|
||||
auto-generates 15.
|
||||
|
||||
## Test suite (all pass, mock LLM)
|
||||
`test_m0, test_m1, test_setup, test_security, test_scenario, test_e2e, test_ip_protection,
|
||||
test_saas_tenant, test_user_journey, test_variant, test_resume_decision` — **11 suites green.**
|
||||
|
||||
## Live state
|
||||
- Live serves `index-rfeHzF-F.js` = latest build (verified). Auto-deploy on push (Gitea webhook).
|
||||
- Real LLM (deepseek) requires `LLM_API_KEY` set in EasyPanel env — local `.env` has placeholder.
|
||||
- testadmin=admin, testuser=user (both pass `1234`); bootstrap super-admin also exists.
|
||||
- A test group "CRM ระบบจัดการลูกค้า" exists on live (user keeps it; will delete themselves).
|
||||
|
||||
## Open / future (see docs/FUTURE_WORK.md)
|
||||
- `/legal` real page, billing/payments, per-tenant storage volume, compressed recipe, export-token polish.
|
||||
- Mobile visual QA at 320/500px still needs the user's eyes (or a multimodal model); responsive CSS
|
||||
(640px, single-column, flex-wrap, .btn-back) is present + deployed.
|
||||
|
||||
## Guardrails reaffirmed
|
||||
- One persona = one chat per user; result final.
|
||||
- Latent/secret persona fields hidden from admin (IP protection); content visible for super_admin.
|
||||
- LLM creds in `.env` / EasyPanel env only; never logged.
|
||||
Reference in New Issue
Block a user