sales-trainer/docs/engineering-log/2026-08-15-s4-4-json-import.md

S4.4 — JSON-to-relational importer

Date: 2026-08-15 Status: local importer complete; production apply/cutover blocked and not performed

Scope

• Added backend/scripts/migrate_json_to_postgres.py.
• Added backend/tests/test_json_import.py.
• Added docs/runbooks/json-to-postgres.md.

The importer covers orgs, users, groups/personas, sessions/messages, and legacy my_personas converted into deterministic owner-private groups. A non-empty audit/audit.jsonl is rejected rather than silently dropped; audit migration remains part of the S4.6 PostgreSQL audit-store gate.

Safety behavior

• Dry-run is the default.
• --apply requires an explicit target database URL and a new backup directory.
• Full source graph is validated before target writes.
• Existing identical rows are unchanged; conflicting target rows abort without overwrite.
• Source JSON remains untouched; backup is created before the target transaction.
• Cross-tenant references, duplicate IDs/uniqueness keys, malformed scalars, invalid timestamps, oversized files, and symlinks fail closed.
• Output is metadata-only: mode, source checksum, counts, and backup status. Password hashes and payloads are never printed.

TDD / verification

• RED: importer test collection failed before the script existed (ModuleNotFoundError).
• GREEN: importer tests 6 passed.
• Direct CLI smoke: python scripts/migrate_json_to_postgres.py --help passed after adding standalone backend-path bootstrap.
• Covered: dry-run no-write, apply counts, deterministic private group conversion, second-run idempotency, payload redaction, backup creation, cross-tenant rejection, non-empty audit rejection, malformed scalar rejection, and target-conflict no-overwrite.
• Full backend suite after the packet: 193 passed.
• Compileall, diff-check, and added-line security scan passed (added_line_findings=[]).

Explicit blockers

• PostgreSQL service and real target migration are unavailable/unverified locally.
• Audit/rate-limit PostgreSQL/Redis storage is not implemented.
• Importer apply is not production approval; no live data, credentials, deploy, or cutover was touched.