[verified] fix live-QA UX: no JSON leak in chat, greeting openers (wrong_text cools off post-first-reply), auto-close+summarize on buy/walk/try, remove Finish button
This commit is contained in:
@@ -24,6 +24,14 @@ filesystem JSON storage (no SQL). i18n TH/EN. No self-registration (admin provis
|
||||
- **user** (trainee) — trains against personas, own board.
|
||||
|
||||
## Current state — local code/security gate passed; production-operation gate pending
|
||||
> **2026-08-18:** live-QA UX fixes landed (committed locally, not yet pushed): persona
|
||||
> replies can no longer leak raw JSON into the chat bubble; all persona openers are natural
|
||||
> greetings (wrong_text cools off only after the seller's first reply); and the chat now
|
||||
> **auto-closes + auto-summarizes** the moment the customer decides (buy / walk / try-to-trial-first),
|
||||
> with the manual "สรุปผล"/Finish button removed from the UI. Full backend suite **336 passed**
|
||||
> (baseline 330 → +6 new tests), frontend build + unit tests clean, independent reviewer passed.
|
||||
> See `docs/engineering-log/2026-08-18-live-qa-ux-fixes.md`. Blocker unchanged: production
|
||||
> operation (runtime cutover, real-provider QA, Redis, audit apply) still requires operator approval.
|
||||
> **2026-08-16:** the S4.4 JSON-importer + error-handler hardening increment (previously staged on top of `dbfce9a`) was re-verified from a clean `requirements.lock.txt` venv (**330 backend tests**, compileall + frontend build clean) and **committed + pushed**. See `docs/engineering-log/2026-08-16-s4-4-importer-errorhandler-commit.md`. Blocker is unchanged: production operation (runtime cutover, real-provider QA, Redis, audit apply) still requires operator approval.
|
||||
|
||||
The current uncommitted remediation is verified on isolated temporary data: **319 backend tests passed** from a clean `requirements.lock.txt` environment, including **166 focused auth/isolation/export/upload regressions**; **4 frontend unit tests** and **12 Playwright fixture journeys** passed across desktop, 320×568, and 500×768; the production frontend build completed with **1,781 modules** and `npm audit` found **0 vulnerabilities**. Compile, AST, diff, dependency, and added-line security checks passed. The checked-in lock is reproducible. The existing local `backend/.venv` has version drift and `pip check` reports the pre-existing `alibabacloud-tea-openapi 0.4.4` versus `cryptography 50.0.0` conflict; `uv pip sync --dry-run` was inspected but not applied. The final fresh exact-current scoped review returned clean five-key verdicts for auth/storage/rate-limit, tenant/group/session isolation, and analytics/export/parser/upload boundaries. JSON stores remain runtime-authoritative; no production operation has been performed.
|
||||
@@ -57,8 +65,9 @@ cd backend && uv run python run.py # Flask :5001
|
||||
- **One-shot:** 1 persona = 1 chat per user; result final (won/lost). `SessionStore` enforces.
|
||||
- **Resume:** unfinished session resumes on re-entry — **no** scenario re-pick (same session+scenario).
|
||||
- **Win/loss = per-turn LLM judge** (`Simulator.evaluate_turn`): every customer reply is evaluated →
|
||||
`{mood, decision(buy|walk|pending), score_delta, reason}`; session ends when decision = buy/walk.
|
||||
**Not** fixed keywords.
|
||||
`{mood, decision(buy|walk|try|pending), score_delta, reason}`; session **auto-ends (close + summary)**
|
||||
when decision = buy/walk/try — no manual "สรุปผล" button (on `try` a system note says the customer
|
||||
will trial first and come back). **Not** fixed keywords.
|
||||
- **2 scenarios only:** `social`, `f2f_call`. Unknown → social.
|
||||
- **Recontact = persona trait** (not a scenario): chats normally, then at turn ≥ 2 a time-lapse
|
||||
system note ("⏳ ผ่านไป 2-3 สัปดาห์…"), then re-engages warmer.
|
||||
|
||||
@@ -66,3 +66,4 @@ Informed by MiroFish (CrowdSight engine) + the hermes-brain-and-tools CrowdSight
|
||||
- `docs/test-evidence/2026-08-15-postgresql-import.md` — temporary-local PostgreSQL importer evidence and target-operation boundary.
|
||||
- `2026-08-15-s4-4-json-import.md` — fail-closed dry-run/apply importer, idempotency, backup, and parity blockers.
|
||||
- `2026-08-16-s4-4-importer-errorhandler-commit.md` — re-verified from clean lock env (330 tests) and committed the staged S4.4 importer + error-handler hardening increment.
|
||||
- `2026-08-18-live-qa-ux-fixes.md` — live-QA UX fixes: persona JSON never leaks into bubble, natural greeting openers (wrong_text cools off only after first reply), auto-close + auto-summarize on buy/walk/try, no manual "สรุปผล" button (336 tests).
|
||||
|
||||
73
docs/engineering-log/2026-08-18-live-qa-ux-fixes.md
Normal file
73
docs/engineering-log/2026-08-18-live-qa-ux-fixes.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# 2026-08-18 — Live-QA UX fixes: JSON leak, complaint openers, auto-close chat
|
||||
|
||||
Date: 2026-08-18
|
||||
Status: implemented + verification evidence; deploy is the repo's normal Gitea→EasyPanel path (operator runs it)
|
||||
|
||||
## Context
|
||||
|
||||
User tested the hardest-tier personas and reported three UX bugs from a live screenshot:
|
||||
|
||||
1. **Raw JSON leaked into the chat bubble.** A persona's reply rendered as
|
||||
`{"reply". "สวัสดีค่ะ...อยากรู้ว่ามีโปรแกรมช่วยจัดการร้านขายเสื้อผ้ามั้ยคะ" "'decision": "none", "mood"' 0}`
|
||||
instead of just the spoken sentence. Root cause: when the provider emits a
|
||||
valid-looking reply **glued to trailing contract keys** (`"decision": ..., "mood": ...`)
|
||||
with or without braces, the old `_parse_persona_reply` fallback could return that
|
||||
non-protocol-looking-but-still-JSON text verbatim as the bubble.
|
||||
2. **Hardest-tier opener was a complaint**, not a greeting ("ไม่มีเงินซื้อหรอก ของแบบนี้แพง" /
|
||||
wrong_text opener `'never mind, forget it'`). The user wants openers to start with
|
||||
"สวัสดี" / "สนใจ" — and the wrong_text persona should open normally, then only cool off
|
||||
("wrong chat") AFTER the seller's first reply.
|
||||
3. **No automatic close / summary + no visible "สรุปผล" button.** Guide mentioned pressing
|
||||
"สรุปผล" but the user expected the chat to auto-close and auto-summarize when the customer
|
||||
decides, with no manual summarize button.
|
||||
|
||||
## Changes
|
||||
|
||||
- `backend/app/services/simulator.py`
|
||||
- Hardened `_parse_persona_reply`: added `_looks_like_protocol()` and
|
||||
`_strip_trailing_protocol_noise()`. Any output carrying the contract's reserved keys
|
||||
(`"reply"`/`"decision"`/`"mood"`, plain or quoted), an opening `{`/`[`/ fenced block, or a
|
||||
quoted fragment + colon is now rejected → bounded retry → fail-closed (`LLMError`), never
|
||||
rendered. Legitimate natural Thai/EN sentences (even with a time colon e.g. "09:00") still
|
||||
pass clean; a lone quoted sentence unwraps to clean text.
|
||||
- Per-turn judge decision vocabulary now includes `try` in addition to `buy`/`walk`/`pending`.
|
||||
- `_special_instr` for `wrong_text`: opener is a normal greeting; ONLY the first reply to the
|
||||
seller's opening may cool off (wrong chat / never mind).
|
||||
- `backend/app/services/persona_prompts.py` — Rule 6 rewritten (wrong_text opens normally, cools
|
||||
off after first reply) + new Rule 8 OPENER RULE: every opener, for every tier incl. wrong_text,
|
||||
must be a natural polite greeting with interest/question — never a complaint/refusal.
|
||||
- `backend/app/api/chat_routes.py` — `send_message` now auto-closes on `buy`/`walk`/`try`. On
|
||||
`try` (customer decides to trial first + come back) it inserts a localized system note
|
||||
("ลูกค้าตัดสินใจจะลองใช้สินค้า/บริการก่อน แล้วจะกลับมาติดต่ออีกครั้ง...") then finalizes
|
||||
(close + judge summary) exactly like buy/walk.
|
||||
- `frontend/src/views/Chat.vue` — removed the manual "สรุปผล"/Finish button, the `finishing`
|
||||
state, and the `finish()` function; chat now auto-closes/summarizes via `chatSend` → `finished`.
|
||||
- `frontend/src/i18n/index.js` — `chatGuideText` (TH + EN) updated: the chat closes automatically
|
||||
when the customer decides (buy / walk / trial first), no instruction to press "สรุปผล".
|
||||
|
||||
## Tests
|
||||
|
||||
- `backend/tests/test_persona_reply_contract.py` +3: partial-protocol-no-braces never leaks
|
||||
(retry then succeed), malformed-brace payload fails closed (`LLMError`), quoted-sentence
|
||||
unwraps quotes.
|
||||
- `backend/tests/test_auto_close_try.py` (new): route-level `chat/send` with a stubbed simulator
|
||||
asserting `try`/`buy`/`walk` all auto-close (`finished=true`, `status=finished`, debrief
|
||||
present) and that `try` inserts a "ลองใช้"/trial system note.
|
||||
|
||||
## Verification evidence
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Full backend pytest suite (fresh venv via `uv`) | **336 passed** (baseline 330 → +6) |
|
||||
| Frontend `vite build` | clean |
|
||||
| Frontend vitest unit | 4/4 pass |
|
||||
| Static security scan of diff (+lines) | no secrets / shell / eval / pickle / SQL |
|
||||
| Independent reviewer subagent (fail-closed, verified against actual files) | **passed** — parsing is regex/string only, test-fixture credentials recognized as non-secrets |
|
||||
|
||||
## Notes / assumptions
|
||||
|
||||
- The manual backend `chat/finish` endpoint and the `finish`/`finishing`/`finishConfirm` i18n keys
|
||||
are left in place (unused by the UI) as a safe non-breaking safety net; no UI path calls them now.
|
||||
- Sessions the customer has not decided on stay `active` and are resumable (per existing
|
||||
resume-without-re-pick behavior); they only close when a decision is reached.
|
||||
- Deploy remains the repo's normal Gitea→EasyPanel webhook path (≈3 min rebuild) — operator runs it.
|
||||
Reference in New Issue
Block a user