sales-trainer/docs/test-evidence/2026-08-24-training-security-review.md

Test evidence — 2026-08-24 Training flow/security remediation

Scope

Current uncommitted Training → My Products → Personas → Practice flow, ownership/tenant isolation, lifecycle recovery, session-mode hardening, analytics/export privacy, frontend contracts, and Wave 1 reviewer remediation.

Current-tree automated checks

• Backend: cd backend && ./.venv/bin/python -m pytest -q → 484 passed in 70.52s.
• Latest focused lifecycle/ownership/race set → 74 passed in 12.12s.
• Targeted malformed-owner contract + Wave 1 regression set → 24 passed in 3.55s.
• Frontend unit: cd frontend && npm run test:unit → 24 passed in 6 files.
• Frontend production build: cd frontend && npm run build → passed (1775 modules transformed).
• Browser journeys: E2E_BASE_URL=http://127.0.0.1:3001 npm run test:e2e → 15 passed in 3.8s across desktop, 320×568, and 500×768. The first attempt correctly failed because Playwright reused port 3000, which was serving CrowdSight; rerunning against an isolated Sales Trainer server on port 3001 passed every case.
• Frontend readiness: curl -fsS http://127.0.0.1:3001/ → passed; temporary process proc_3697d2f91f57 was killed after E2E.
• Python compilation: cd backend && ./.venv/bin/python -m compileall -q app tests → passed.
• Whitespace validation: git diff --check → passed.
• Added-line static scan on the latest lifecycle slice: 0 production findings for hardcoded secrets, shell execution, eval/exec, unsafe pickle, formatted SQL, and debug leftovers. Three password-pattern matches were test fixtures.
• Post-deleg_457c2d44 stale-tenant ownership regressions failed twice before remediation, then the focused ownership/security set passed 125 tests in 11.52s. Canonical private ownership now requires a valid matching tenant together with owner identity, including for promoted super-admins.
• ruff, mypy, eslint, and tsc are unavailable in this environment; no result is claimed for those tools.

Wave 1 independent findings and remediation

Batch deleg_4d48caf2 returned two complete, schema-valid negative verdicts:

1. Ownership/tenant isolation: super_admin could list or ID-address owner-private products and start chat against them.
2. Lifecycle/readiness/races: privileged reads trusted malformed persisted ready state, lifecycle status values/transitions were not validated at the service boundary, and admin variant generation could append a stale variant after concurrent source-persona replacement.

Current-tree remediation:

• Owner-private products are excluded from super-admin listings and denied in group and chat authorization; explicitly shared cross-tenant administration remains separate.
• ready records fail closed for every role unless sales kit, report, and canonical Personas are structurally valid.
• GroupStore now allowlists lifecycle states and explicit permitted transitions, including failed → analyzing retry and ready → analyzing reanalysis.
• Variant publication revalidates both source status and the exact source persona under the group record lock after LLM generation.
• Privileged indexes exclude malformed ready and invalid-status records; publish_analysis() accepts only persisted analyzing groups.
• Canonical Persona normalization rejects duplicate IDs, and variant publication compares the complete source Persona, sales kit, and safe input snapshot for both admin and trainee paths.
• Added backend/tests/test_review_wave1_regressions.py and updated stale tests that previously accepted malformed-ready artifacts.

Independent review gate

Remediation re-check batch deleg_8d258a4f returned complete verdicts:

• Ownership/tenant isolation: passed with empty blocking arrays.
• Lifecycle/races/readiness/variants: failed with confirmed malformed-index, publication-transition, duplicate-ID, and stale same-ID/admin+trainee variant gaps.

Batch deleg_0456c47d then returned two complete negative verdicts: privileged owners lost their own private products after promotion; source visibility was not snapshotted; and structurally identical reanalysis was not detectable. Five RED regressions reproduced those gaps. The current remediation gives valid owners precedence without weakening malformed/non-owner denial, snapshots visibility, and increments/revalidates monotonic analysis_revision on every publication. Fresh exact-current re-review deleg_a5931b0d is pending with two reviewers and no overlapping wave. The remaining six scopes are queued and not dispatched.

The gate remains closed until all eight current-tree scopes return complete, schema-valid verdicts with passed: true, security_concerns: [], and logic_errors: []. Interrupted, timed-out, truncated, malformed, stale, or partial verdicts do not approve the tree. Any source remediation makes earlier verdicts stale for affected scopes.

Harness limitation

The Playwright fixture server may emit Vite proxy ECONNREFUSED messages for /api/auth/oauth/config because no Flask backend or OAuth provider is running. This evidence is not a live-provider, authenticated production, or deployment smoke test.

Wave 2 remediation evidence — 2026-08-25

• Failed independent review: deleg_ead5db71 returned complete negative Chat/session and auth/setup/JWT/OAuth/config verdicts with live local probes.
• RED: tests/test_wave2_review_regressions.py → 12 expected failures before production remediation.
• GREEN: the same focused file → 12 passed in 1.16s.
• Focused compatibility run: 197 passed / 1 stale fixture failure, followed by fixture correction; the preceding broader attempt was 157 passed / 29 failures, dominated by one over-strict Persona validator and stale policy expectations, and is not reported as a passing gate.
• Full backend: cd backend && PYTHONPATH= .venv/bin/pytest -q → 496 passed in 71.80s.
• Python compilation and git diff --check: passed.
• Added-line hardcoded-secret/dangerous-execution scan: 0 findings.
• Re-review deleg_6155669d: two complete negative verdicts. It reproduced malformed explicit Persona trait coercion plus three auth gaps: same-user duplicate OAuth bindings, low-period repeated secrets, and malformed timestamp state while Terms were unaccepted.
• Second RED cycle: 6 expected failures / 12 existing passes in the focused file before production remediation.
• Second GREEN cycle: focused Wave-2 file → 18 passed in 1.40s.
• Latest full backend → 502 passed in 72.06s; compileall, diff check, Markdown fence check, and added-line scan all passed with 0 scan findings.
• Re-review deleg_1e21e60c: auth/setup/consent/JWT/OAuth/config passed with exact empty blocking arrays after 226 focused and 502 full tests; Chat remained blocked by malformed explicit intent_tier coexisting with a valid tier.
• Third Chat RED cycle: the dedicated coexistence regression failed as expected.
• Third Chat GREEN cycle: focused Wave-2 file → 19 passed in 1.39s; latest full backend → 503 passed in 71.89s.
• Latest compileall, diff check, and added-line scan passed with 0 findings.
• Incremental current-tree Chat re-review deleg_8c71eff4: passed with exact empty blocking arrays. Independent checks: 136/136 bounded Chat/session/privacy tests, 19/19 Wave-2 regressions, four targeted tests, compileall, and diff checks.
• Wave 3 first review deleg_1733a8bf: both scopes failed and were remediated. Analytics required shared bounded scans and tenant-consistent user/session/group joins. Frontend required disabled analyzing actions and regular-user completed-chat navigation to /my/board.
• Wave 3 re-review deleg_1fed2524: frontend passed; analytics found nested persisted values could be stringified into CSV. A sentinel regression failed before the patch and then passed after scalar-only _csv_cell handling.
• Latest analytics evidence: sentinel regression 1 passed in 0.63s; focused slice 39 passed, 146 deselected in 7.88s; full backend 509 passed in 73.27s; compileall and git diff --check passed.
• Analytics re-review deleg_bcb1510f: passed with exact empty blocking arrays.
• Latest frontend evidence: 26/26 unit tests, production build with 1775 modules transformed, and 15/15 isolated E2E journeys across desktop, 320×568, and 500×768.
• Final Wave 4 deleg_97eb385c: API-contract scope passed with 26/26 frontend and 81/81 focused backend checks; final cross-cutting scope also passed. Both exact verdicts had empty security and logic arrays.
• Independent local code/security gate: 8/8 scopes passed.
• Post-push operator report of missing legacy training data traced to the new fail-closed visibility schema. Legacy records lacking visibility were rejected by authorization. Fixed with migrate-on-read resolved_visibility (owner present => private, absent => public; malformed explicit stays fail-closed; demo/hidden never derived). Applied at every list/authorization/chat/analytics boundary.
• Post-fix evidence: legacy/resolved regression 13 passed; blocker file 55 passed; full backend 517 passed in 80.97s; frontend unit 26 passed; production build passed; compileall and git diff --check passed.
• No live OAuth provider, production runtime, deployment, stage, commit, push, reset, or stash verification is claimed.

Git and safety state

• Branch: main; HEAD remains 8a632b5e6a3b67c9acd2787e1a686ac17ae69486.
• Existing broad user WIP remains uncommitted and unstaged.
• No secrets were read or stored. No reset, stash, cleanup, stage, commit, push, deploy, credential, permission, or production operation was performed.