microfish/docs/HANDOFF.md

HANDOFF — MiroFish SaaS Migration

Repo: /Users/kunthawat/Gitea/MiroFish Branch: feat/saas-foundation Base: 89d04e7 Status: SaaS foundation in progress; not production-ready; no commit/push yet

Current outcome

The repository now has a tested identity/authentication foundation, tenant/owner guards around the main graph, simulation, report, and task lookup paths, durable operations schema/jobs/idempotency records, and a bounded local graph → profile → simulation → report persistence path. The original Zep runtime remains the default compatibility backend and legacy file-backed product persistence remains, so this is not yet a completed SaaS migration or production release.

Delivered

• Thai/English locale policy with Thai default; legacy zh values normalize to Thai.
• Frontend API error sanitization and backend structured error envelopes.
• No full JSON request-body logging.
• SQLAlchemy models and Alembic migrations for organizations, users, memberships, and revocable sessions.
• Argon2 password hashing.
• HttpOnly session cookie plus signed CSRF cookie/header validation.
• Explicit credentialed CORS allowlist.
• Three roles only: super_admin, admin, user.
• /api/auth/login, /api/auth/me, /api/auth/logout.
• /api/admin/users list/create with role escalation prevention and response redaction.
• Project organization_id/owner_user_id metadata and scoped project reads/lists.
• Blueprint-wide auth and CSRF checks for simulation/report routes.
• Fail-closed scope checks for project, graph, simulation, report, and task IDs in path/query/body.
• Cross-tenant regression tests for project/simulation/report/task access.
• Frontend auth store using cookie sessions, /login view, protected workspace router guard, and localized Thai/English login copy.
• Production /login white screen root cause fixed locally (2026-09-01): raw @ in auth.emailPlaceholder was invalid vue-i18n linked-message syntax and raised compiler code 10 during render. th/en now use name{'@'}company.com; an all-message compiler regression passes. Frontend 11 tests + production build pass; Chrome rendered DOM contains the login UI. Pending: fresh review, push/redeploy, then live verification.
• Frontend report-status client corrected to POST { task_id, simulation_id }.
• Durable local memory tables and a tenant-scoped SQLAlchemy repository.
• Durable jobs, idempotency records, and audit-log schema with migrations 0004_operations through 0006_job_metadata; TaskManager binds to the current app/session factory, fails closed when an app context has no factory, and has SQL-level tenant/owner filters.
• SQLite engines created by the shared DB helper enable foreign-key enforcement; regression coverage verifies invalid organization/owner references plus SET NULL/CASCADE delete behavior.
• GraphBuilder async adapter now carries organization/session context into durable task metadata.
• Deterministic local graph → profile → simulation-config → report-tools → persisted-report regression.
• Strict Pydantic extraction schema, bounded prompts, JSON validation, and deterministic LLM-result ingestion service.
• Golden parity fixtures and local adapters for entity filtering/context, quick search, panorama temporal inventory, and insight entity/relationship output shapes.
• Explicit MEMORY_BACKEND=local path now serves simulation entity routes and preparation; SimulationManager.create_entity_reader() fails closed when local mode has no injected scoped factory; default remains Zep during migration. Fresh bounded reviewer deleg_029828b6 independently passed this exact reader/fallback slice with no security concerns or logic errors.
• The bounded local panorama/insight/entity-reader parity slice is covered by RED/GREEN tests; conflicting graph IDs fail closed and local simulation activity updater persists tenant-scoped episodes without constructing a Zep client.
• API safe-error regression gate covers simulation, graph, report, agent-group, and template boundaries; task/project/report/run state no longer persists raw exception text.

Verification evidence

• Backend full suite after the latest app/factory isolation fixes, auxiliary/idempotency hardening, simulation/report/graph service import-boundary fixes, durable product-resource schema/repository, tenant-scoped ArtifactStore, durable JobQueue/worker.py, versioned redacted PlatformSettings, durable RateLimiter, durable LLM UsageService, durable redacted AuditService, and durable single-use PasswordResetService: 193 passed with PYTHONPATH= uv run pytest -q from backend (2026-08-24); twenty-four Alembic deprecation warnings remain.
• Schema regression: 16 passed, including Alembic upgrade → downgrade 0003_memory → re-upgrade, operation-table column/index/FK inspection, SQLite PRAGMA enforcement, invalid org/owner rejection, delete actions, cross-app TaskManager binding checks, and background use without Flask context.
• Auxiliary API security regression: 8 passed; every template/agent-group route requires auth, LLM mutations require CSRF + idempotency, safe internal errors are redacted, and both LLM mutations replay completed responses.
• Idempotency API regression: 5 passed; same-route replay/conflict semantics remain, same-key cross-route/body replay is rejected, and multipart file content/repeated-file coverage is verified.
• Local consumer-boundary regression: 9 fresh-import tests passed (test_local_service_import.py); fresh local imports of the updater, SimulationManager, SimulationRunner, ReportAgent, and GraphBuilderService do not eagerly load zep_cloud, add_activity_from_dict uses the shared activity contract without importing zep_cloud on the local path, and the reviewer-requested regression guards for explicit Zep-path imports, shared AgentActivity identity, and the graph_builder.Zep seam were added and pass. The activity-dict boundary regression also passes.
• Durable product-resource persistence: 21 product-persistence tests passed (test_product_persistence.py). Migration 0007_product_resources adds tenant/owner-scoped projects, simulations, and reports tables with FK cascade/SET-NULL, indexes, and a migration round-trip + command.check (no autogenerate drift). ProductRepository provides flush-only tenant-scoped create/get/list for all three with cross-tenant isolation verified. sync_project/sync_simulation/sync_report are idempotent dual-write bridges wired into project-creation, simulation-creation, and report-generation paths; SimulationState carries tenant scope and _save_simulation_state mirrors every status update into the durable table when a session factory is available, fail-closed when unscoped. scoped_project reads the durable table first (mapping back to the legacy Project shape) when the local backend is active, then falls back to filesystem, keeping existing routes intact. Read cutover/dual-write are gated to the local backend so the legacy Zep filesystem path is undisturbed.
• Tenant-scoped artifact store: 12 artifact-store tests passed (test_artifact_store.py). ArtifactStore resolves tenant-scoped paths under a configured root, rejects traversal/absolute components (fail-closed), and exposes a store/read/exists/delete interface (default_artifact_store() is filesystem-backed; swapable for object storage later).
• Durable job queue + worker core: 10 tests passed (test_job_queue.py 7, test_worker.py 3). JobQueue claims/executes/completes/fails durable jobs with optional organization scoping, a handler-registry dispatch (raises on unhandled operations for the worker to fail), and no double-claim; worker.py run_once/run_loop provide the portable worker entry point (no broker dependency yet).
• Versioned, redacted LLM settings: 4 tests passed (test_settings_service.py). Migration 0008_platform_settings + SettingsService store versioned global platform settings; the API key is encrypted with a SECRET_KEY-derived Fernet key (never stored/logged in plaintext, never in job snapshots), public reads return a masked value, and snapshot_for_job() captures the settings version for reproducible work. cryptography added as a direct dependency.
• Durable rate limiting: 6 tests passed (test_rate_limiter.py). Migration 0009_rate_limit + RateLimiter implement a durable sliding-window counter (survives restarts, per-key, no secrets), and are wired into the /api/auth/login route so excessive attempts return 429 (api.tooManyAttempts added in th/en).
• Durable LLM usage accounting: 4 tests passed (test_usage_service.py). Migration 0010_usage_events + UsageService record per-org/user LLM usage (model, input/output tokens, estimated cost) without storing prompt content or secrets, with org-scoped listing and total-cost totals.
• Durable redacted audit service: 3 tests passed (test_audit_service.py). AuditService records tenant-scoped audit events while stripping sensitive keys (secrets, tokens, api keys, prompts) from details, and is wired into the /api/auth/login success path (auth.login event).
• Durable single-use password reset: 6 tests passed (test_password_reset.py). Migration 0011_password_reset_tokens + PasswordResetService issue expiring, single-use, hashed reset tokens (plaintext never stored), bump auth_version on consume to invalidate old sessions, and are exposed as /api/auth/password-reset/request + /api/auth/password-reset/confirm (enrollment-agnostic request prevents account enumeration; also covers invite-pending password setup).
• Frontend contract tests: 10 passed; npm run build: PASS; existing chunk-size/dynamic-import warnings remain.
• Python compileall: PASS.
• Diff security scan: no introduced shell=True, eval/exec, or pickle loading in backend code; git diff --check: PASS.
• ruff is not installed in the backend environment; lint is skipped, not treated as a pass.
• Dev server /login smoke response: PASS. Real 320×568 and 500×768 browser capture is not verified because Chrome requested an OS remote-debugging permission and no permission dialog was clicked.

Important incomplete work

1. Zep replacement is not complete. Local memory storage/extraction contracts and a bounded graph → profile → simulation → report persistence regression exist, but semantic LLM parity and full consumer cutover are not complete; Zep remains the default compatibility backend.
2. Product persistence foundation is in place. Durable tenant/owner-scoped projects/simulations/reports schema (migration 0007_product_resources), a scoped ProductRepository, dual-write for creation and simulation run status, scoped_project read-first cutover (local backend), and a tenant-scoped ArtifactStore abstraction are added and tested. Legacy ProjectManager/SimulationManager/ReportManager filesystem paths remain authoritative-by-default for the Zep backend and for artifact payloads; full read cutover for simulation/report and migrating artifact payloads to object storage are deferred to the production storage/worker phase.
3. Production worker/queue topology core is in place; broker/storage not decided. A durable JobQueue (claim/dispatch/complete/fail, tenant-scoped) and a portable worker.py run-loop entry point exist and are tested, alongside a tenant-scoped ArtifactStore. Production decisions remain: choose a broker (Redis/RabbitMQ) or keep polling the durable table, choose object storage, and define the WSGI/worker/supervisord deployment topology. Background-thread execution in routes is not yet moved onto the durable queue.
4. Admin user-management + super-admin settings UI base in place. Backend GET/POST/PATCH /api/admin/users (role/status policy-guarded, status super-admin only) and GET/PUT /api/admin/settings (super-admin only, secret encrypted+masked via SettingsService). Frontend AdminView.vue (user mgmt) and SettingsView.vue (LLM settings form) are routed at /admin and /admin/settings with admin/super-admin role guards and th/en i18n. Invite self-setup UX, a connection-test endpoint, and full i18n/mobile review remain.
5. LLM settings storage base is in place; admin/bootstrap UI not built. A versioned, encrypted PlatformSettings store (SettingsService + migration 0008) exists and is tested. Super-admin settings UI, provider preset validation, and connection-test endpoints are not yet built; active runtime still reads environment variables.
6. Frontend active UI i18n gates pass; legacy Process.vue retired. Frontend contract tests 10 passed and the production build contains no CJK code points. The unused legacy Process.vue (Chinese copy) was retired — the router's name: 'Process' uses MainView.vue. Real mobile 320×568/500×768 visual capture requires a full-stack + authenticated browser session and is not yet verified; a standalone vite preview of the SPA renders a blank mount (needs backend bootstrap), which is not treated as a pass.
7. Docker deployment drafted as production multi-service; container build not yet proven. Dockerfile is rebuilt as a production multi-service image (multi-stage frontend build, python + gunicorn wsgi:app, nginx serving the built SPA with history fallback and /api→gunicorn proxy, supervisord running backend + durable worker). backend/wsgi.py and gunicorn>=21 were added. Local smoke test (gunicorn + built-dist static serving) passed — /health OK, assets rebuilt — but the nginx SPA-fallback + proxy + worker-poll path must be verified by building the image in EasyPanel; no Docker runtime is available on this Mac.
8. SaaS hardening mostly complete. Durable rate limiting (wired to login), durable LLM usage/cost accounting, a durable redacted audit service, and durable single-use password reset (which also covers invite-pending setup) are added and tested. Still open: a complete resource authorization matrix across every role × endpoint, a full retryable-mutation inventory, and long-running-job review.
9. Auxiliary API hardening is verified. Fresh reviewer deleg_d128494e returned passed=true with empty security_concerns and logic_errors; current auxiliary security tests pass 8/8. This closes only the template/agent-group slice, not the broader SaaS gate.
10. Bounded schema/TaskManager review passed. Fresh reviewer deleg_fb0f80b0 returned passed=true with empty security_concerns and logic_errors after current-code runtime app/factory mismatch probes, SQLite pooled/NullPool foreign-key checks, ORM/migration parity, and migration round-trip checks. Its suggestions are non-blocking: expand mismatch coverage across every TaskManager operation, broaden pooled/NullPool assertions in tests, and assert full ORM/migration signatures. This closes only the exact schema/TaskManager slice; broad reviewer deleg_949ab2f4 timed out and the wider SaaS gate remains open.
11. The current bounded local import-boundary review passed. Fresh reviewer deleg_65bce3de returned passed=true with empty security_concerns and logic_errors; it ran the focused boundary command with 6 passed and isolated probes for five fresh local imports, shared AgentActivity, explicit Zep paths, the graph_builder.Zep seam, and subprocess isolation. This closes only the bounded import-boundary slice; semantic parity, complete consumer cutover, and the wider SaaS gate remain open.

Next implementation order

1. Keep the current staged snapshot and re-run verification after any further edits.
2. Extend golden parity and tenant-negative coverage to the remaining local consumers and simulation-context/report E2E path.
3. Switch graph builder, entity reader, tools, profile generator, and report agent to the local memory adapter.
4. Remove Zep dependency/config only after parity and migration tests pass.
5. Add admin/super-admin UI and settings policy.
6. Finish production worker/deployment hardening and run a fresh end-to-end tenant matrix.

Guardrails

• Exactly three role identifiers: super_admin, admin, user.
• Browser never receives LLM/API secrets.
• Backend authorization is authoritative; frontend guards are convenience only.
• Do not claim semantic LLM/Zep parity without golden-fixture evidence.
• Do not commit or deploy while the independent review gate remains unresolved.