kunthawat/sales-trainer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[verified] harden Sales Trainer and add PostgreSQL foundation

Browse Source
This commit is contained in:
Macky
2026-08-16 02:41:10 +07:00
parent 6811dc1db9
commit dbfce9aa54
179 changed files with 19955 additions and 1505 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
docs/HANDOFF.md
View File

@@ -7,8 +7,9 @@
- Remote: `https://git.moreminimore.com/kunthawat/sales-trainer.git` (GITEA_TOKEN via credential
helper; never committed).
- **Live deploy:** `https://moreminimoreapps-saletrainer.ahkhwd.easypanel.host` — EasyPanel,
auto-redeploys from Gitea on push to `main` via webhook (≈3 min). Dockerfile ships prebuilt
`frontend/dist/` (no npm in image). LLM vars set in EasyPanel env.
auto-redeploys from Gitea on push to `main` via webhook (≈3 min). Dockerfile builds the
frontend in a Node builder stage and serves it from the Python runtime image. LLM vars set in
EasyPanel env.
## What this is
Corporate multi-user **sales-training simulator**: admins create persona groups from a
@@ -22,8 +23,15 @@ filesystem JSON storage (no SQL). i18n TH/EN. No self-registration (admin provis
- **admin** — manages groups/users, sees personas with **secret fields stripped** (IP protection).
- **user** (trainee) — trains against personas, own board.
## Current state — COMPLETE core + hardened
All backend + frontend built. **11 test suites green** (mock LLM):
## Current state — local code/security gate passed; production-operation gate pending
The current uncommitted remediation is verified on isolated temporary data: **319 backend tests passed** from a clean `requirements.lock.txt` environment, including **166 focused auth/isolation/export/upload regressions**; **4 frontend unit tests** and **12 Playwright fixture journeys** passed across desktop, 320×568, and 500×768; the production frontend build completed with **1,781 modules** and `npm audit` found **0 vulnerabilities**. Compile, AST, diff, dependency, and added-line security checks passed. The checked-in lock is reproducible. The existing local `backend/.venv` has version drift and `pip check` reports the pre-existing `alibabacloud-tea-openapi 0.4.4` versus `cryptography 50.0.0` conflict; `uv pip sync --dry-run` was inspected but not applied. The final fresh exact-current scoped review returned clean five-key verdicts for auth/storage/rate-limit, tenant/group/session isolation, and analytics/export/parser/upload boundaries. JSON stores remain runtime-authoritative; no production operation has been performed.
### Newly verified local PostgreSQL evidence — 2026-08-15
- A temporary local PostgreSQL database completed `alembic upgrade head`, the 7-table tenant/uniqueness runtime probe, ORM-vs-migration parity, offline PostgreSQL DDL checks, the JSON importer dry-run/apply/idempotency/conflict-rollback probe, and `alembic downgrade base`; zero application tables remained and all temporary databases/fixtures/backups were cleaned up.
- This closes the **local PostgreSQL schema/runtime and fixture-importer probes** only. Real JSON snapshot parity/rollback, runtime repository cutover, Redis persistence, Docker image/runtime smoke, real-provider QA, authenticated production smoke, and production approval remain open.
**Do not restore public/untrusted access.** An async reviewer packet created before the latest auth-lock state reported a password `auth_version` race; the current tree now serializes `change_password()` and `update_user_fields(password=...)` through the same per-user record lock, and the deterministic cross-path regression passes. A surgical independent review of this auth path returned valid `passed=true` with empty blocking arrays. Legacy-security reviewers `deleg_ffba9adf`, `deleg_03d06f1f`, and strict retry `deleg_0cc095f8` all timed out after 600 seconds without JSON and are no verdicts. Final one-call retry `deleg_e61f99ff` returned complete five-key JSON with `passed=false` because the extraction output exceeded the capture window; it is a limitation/no-approval verdict. The latest full-scope reviewer `deleg_93ef64c5` timed out after 600 seconds without a complete five-key JSON verdict; it is no approval. The replacement batch `deleg_df17f04e` was stopped because it began before the final service-boundary hardening and is not approval. Fresh exact-current review batches `deleg_e835e807` and `deleg_4ec7eb5a` completed for auth/storage/rate-limit, tenant/group/ownership/session isolation, and analytics/export/parser/upload boundaries. All three scoped verdicts returned complete five-key JSON with `passed=true`, `security_concerns=[]`, and `logic_errors=[]`. S4.3 reviewer `deleg_c2e728d8` returned valid `passed=false` after finding the offline Alembic dialect bypass; the shared validator and regression test fixed it, and fresh post-remediation reviewer `deleg_40e8edf9` returned valid `passed=true` with empty blocking arrays. Combined with the earlier dependency/schema review `deleg_e672880a`, the S4.2/S4.3 code-schema review scope is closed. Only complete five-key verdicts with `passed=true`, `security_concerns=[]`, and `logic_errors=[]` close review gates. Docker is unavailable locally. Local fixture-based browser/mobile E2E passes, but real-provider QA, production-authenticated browser QA, and production operations are not run. The restricted deployment checklist (bootstrap credential change, JWT secret rotation, audit inspection, fresh authenticated smoke) remains pending and requires explicit operator approval.
```bash
cd backend
@@ -39,7 +47,7 @@ uv run python scripts/test_user_journey.py # idea-flow end-to-end
uv run python scripts/test_variant.py # clone-persona-from-persona
uv run python scripts/test_resume_decision.py # resume + per-turn LLM decision
# run backend (serves SPA from frontend/dist)
# run backend (serves frontend/dist only when a local build exists; Docker builds it reproducibly)
cd backend && uv run python run.py # Flask :5001
```
@@ -55,11 +63,14 @@ cd backend && uv run python run.py # Flask :5001
- **Persona variant:** `POST /api/groups/<gid>/personas/<pid>/variant` — new persona (new id) that
**locks** pains/objections/levers/tolerance/special/recontact/goal/budget/difficulty/tier/product
**but varies** identity (name/profession/age/location/background/personality/opener). Lets a
trainee re-practice the same challenge (one-shot is per-persona). UI button on finished personas.
trainee re-practice the same challenge (one-shot is per-persona). Trainee-created variants go to
the trainee's owner-private group; admin-created variants extend the shared admin pool. UI button
on finished personas routes trainees to the private group.
- **Auto 15 personas** on create; no "เพิ่มเติม" button (TARGET=15, retry up to 3× + accept ≥ 8 so
real LLM under-count doesn't 500).
- **IP protection:** `SECRET_PERSONA_FIELDS` (pains, objections, negotiation_levers, opener,
tolerance, rootCause, resolutionConditions) stripped for `admin`; full only for `super_admin`.
- **IP protection:** `admin` persona responses use an explicit allowlist and omit secret/process
fields (pains, objections, negotiation_levers, opener, tolerance, rootCause, resolutionConditions)
plus any future unapproved fields; full canonical data is only for `super_admin`.
- **SaaS Phase 1–3 done:** tenant isolation (`g.org_id` + `assert_tenant`), login/chat rate-limit,
audit log (`data/audit/audit.jsonl`), org plan/seats/active model + `PATCH /api/admin/orgs/<id>`,
ToS consent on setup, org-scoped signed expiring CSV export (5-min HMAC).
@@ -67,9 +78,8 @@ cd backend && uv run python run.py # Flask :5001
persona detail rendered as readable form/cards (pain = line-by-line, not `[object Object]`).
## Credentials / data (testing)
- Bootstrap super-admin `admin` / `1234` → first login forces email + new password + ToS consent.
- Live test users: `testadmin` / `1234` (admin), `testuser` / `1234` (user).
- A test group "CRM ระบบจัดการลูกค้า" exists on live (user keeps it; will delete it themselves).
- Production bootstrap super-admin is `admin`; its initial password must come from `BOOTSTRAP_ADMIN_PASSWORD` and is never printed or hard-coded. First login forces email + new password + ToS consent.
- Do not record or repeat live credentials in this handoff. Existing live test accounts/data require operator review after restricted deployment and secret rotation.
- **LLM key is a placeholder on local `.env`** (`replace_me`). Real analyze/chat needs a real
`LLM_API_KEY` (+ `LLM_PROVIDER`/`LLM_MODEL`/`LLM_BASE_URL`) in EasyPanel env then redeploy.
@@ -79,7 +89,8 @@ cd backend && uv run python run.py # Flask :5001
- **Tooling guard crash:** commands whose first token is `./.venv/bin/python` trip a lifecycle guard
→ always use `uv run python`. Prefix `PYTHONPATH=` when needed.
- **Frontend build:** `cd frontend && npm run build` (works even with allowScripts restrictions).
Commit `frontend/dist/` with `git add -f` (it's gitignored otherwise); the Dockerfile needs it.
`frontend/dist/` is generated and ignored; Docker builds it in the image, and CI builds to a
temporary output directory so clean clones never depend on stale hashed bundles.
- **Deploy pattern:** commit + push → webhook auto-deploys in ~3 min. Cannot run Docker locally
(no Docker on this Mac) → test with nginx/`python http.server` or `uv run python run.py`.
- **No remote push without asking** unless it's the established auto-deploy cadence.
@@ -93,6 +104,16 @@ cd backend && uv run python run.py # Flask :5001
Responsive CSS (640px, single-column, `flex-wrap`, `.btn-back`) is present + deployed.
## Next actions / backlog (also docs/FUTURE_WORK.md)
- **Sprint 1 live-operation gate:** deploy behind restricted access, set/verify `BOOTSTRAP_ADMIN_PASSWORD`, rotate `JWT_SECRET` after the patch, inspect audit data, and run a fresh authenticated smoke. Do not open public access before this checklist.
- **Sprint 2:** implementation and local regression gate complete; retain current tests as the contract.
- **P1:** run real-provider QA and browser/mobile E2E; local contract tests use deterministic fakes.
- **P1:** verify final-judge coaching quality with representative provider outputs; correctness and idempotency are locally covered.
- **P1:** remove any stale admin chat entry points in live UI after restricted authenticated smoke.
- **P1/P2 ops:** add PostgreSQL repository adapters, browser E2E, and real-provider QA before production rollout. Upload limits, cleanup, fail-closed JWT/bootstrap configuration, Gunicorn, Docker healthcheck, and `.dockerignore` are implemented and tested locally.
- **S4.2:** schema reviewer findings (nullable audit actor link, CWD-relative Alembic paths, ORM/migration defaults drift, unsupported partial-index dialect, and offline batch rendering) plus dependency reproducibility were remediated locally; combined exact-current evidence from `deleg_e672880a` and post-remediation `deleg_40e8edf9` closes the code/schema/dependency review scope. Temporary-local PostgreSQL execution, ORM/migration parity, offline DDL, and schema rollback now pass; Docker build, importer data parity/rollback, repository cutover, and production-safe rate-limit/audit storage remain blocked.
- **S4.3:** org/users, groups/personas, and sessions/messages repository contracts and SQLAlchemy adapters are local-only; cross-tenant user lookup and offline-dialect remediations are locally verified and independently approved by `deleg_40e8edf9`. PostgreSQL schema parity passes on temporary local databases, while runtime repository cutover remains blocked.
- **S4.4:** JSON importer dry-run, backup, idempotency, conflict rejection, cross-tenant validation, temporary-local PostgreSQL apply, and transaction rollback all pass; real target snapshot parity, retained-backup rollback rehearsal, audit migration, and apply approval remain blocked.
- **2026-08-15 legacy-security remediation:** the latest valid reviewer finding about legacy `ratelimit.json` migration was remediated with fail-closed marker/digest validation, structured keys, duplicate-preserving import, and idempotent import metadata. Current local evidence is 319 backend tests, 166 focused regressions, and 4 frontend unit tests. Three fresh exact-current scoped reviewers returned clean five-key verdicts; see `docs/engineering-log/2026-08-15-final-security-gate.md`.
- Real `/legal` page (setup links to it), billing/payments, per-tenant storage volume, compressed
persona recipe, export-token polish.
- Mobile visual polish per user feedback on real device.
@@ -100,6 +121,18 @@ cd backend && uv run python run.py # Flask :5001
## Related docs
- `docs/PLAN.md`, `docs/SAAS_PLAN.md`, `docs/FUTURE_WORK.md`.
- `docs/engineering-log.md` + `docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md` (this session:
- `docs/engineering-log/2026-08-13-idea-implementation-audit.md` — latest evidence-based audit, runtime probes, and prioritized fixes.
- `docs/engineering-log.md` — status index.
- `docs/engineering-log/2026-08-15-final-security-gate.md` — final local verification and clean exact-current scoped review gate; production-operation limits remain.
- `docs/engineering-log/2026-08-15-postgresql-runtime-gate.md` — temporary-local PostgreSQL schema/runtime, parity, offline DDL, and migration rollback evidence.
- `docs/engineering-log/2026-08-15-postgresql-import-gate.md` — temporary-local PostgreSQL importer dry-run/apply/idempotency/conflict-rollback evidence.
- `docs/test-evidence/2026-08-15-postgresql-runtime.md` — PostgreSQL automated gate evidence and remaining operational boundaries.
- `docs/test-evidence/2026-08-15-postgresql-import.md` — importer evidence separated from real target apply and audit migration.
- `docs/engineering-log/2026-08-09-idea-flow-qa-deploy.md` (this session:
idea-flow UX, 2-scenario + recontact trait, live QA + auto-deploy, per-turn LLM judge, persona
variant, 15-persona auto-gen).
- `docs/engineering-log/2026-08-14-final-review.md` — prior five-finding remediation and review history; current exact-tree approval remains pending.
- `docs/engineering-log/2026-08-15-auth-version-review-reconciliation.md` — current auth-version lock-path verification and pending fresh reviewer gate.
- `docs/engineering-log/2026-08-15-s4-2-schema-foundation.md` — test-first SQLAlchemy/Alembic schema foundation, tenant constraints, and generated-dist cleanup.
- `docs/engineering-log/2026-08-15-s4-3-org-users-repositories.md` — tenant-scoped repository contracts and adapters, with no runtime cutover.
- `docs/engineering-log/2026-08-15-s4-3-offline-dialect-remediation.md` — offline Alembic dialect finding, test-first fix, verification, and pending review.