Files
microfish/docs/engineering-log/2026-08-23-local-memory-foundation.md
Kunthawat Greethong 8b84378fe1 feat: SaaS foundation for CrowdSight
Elevate MiroFish/CrowdSight from single-container dev to a SaaS foundation:

- Local memory backend (Zep-compatible): memory services/models, local graph
  builder + updater, AgentActivity seam, import-boundary isolation; Zep stays
  default, local is opt-in behind MEMORY_BACKEND. Semantic parity not yet proven.
- Durable product persistence: projects/simulations/reports schema (migration
  0007) + tenant/owner-scoped ProductRepository + dual-write + scoped_project
  read-first + ArtifactStore abstraction; durable JobQueue + worker.py.
- SaaS hardening: durable RateLimiter (wired to login), UsageService (LLM
  accounting), redacted AuditService, idempotency, CORS allowlist, safe API
  errors, single-use PasswordResetService + endpoints (covers invite-pending).
- Exactly 3 roles (super_admin/admin/user) with tenant authz policy.
- Admin UI: GET/POST/PATCH /api/admin/users + GET/PUT /api/admin/settings
  (super-admin only, encrypted/masked); AdminView.vue + SettingsView.vue with
  admin/super-admin route guards, th/en i18n.
- Production deploy topology: multi-stage Dockerfile (frontend build + gunicorn
  wsgi + nginx SPA-proxy + supervisord worker), backend/wsgi.py, gunicorn dep.

Backend 197 passed; frontend 10 tests + build green. ruff unavailable (gap).
No commit of credentials; secrets handled via env/.env.example.
Deferred: Zep semantic A/B parity, object storage cutover, mobile QA, EasyPanel
container build of deploy topology.
2026-08-31 13:05:21 +07:00

1.6 KiB

Local Memory Foundation — 2026-08-23

Scope

Started the Zep replacement without switching existing consumers yet. The goal of this slice is to separate LLM semantic extraction from durable storage, search, and authorization.

Delivered

  • SQLAlchemy memory tables: memory_graphs, memory_episodes, memory_nodes, memory_edges.
  • Alembic revision 0003_memory with SQLite upgrade/check parity.
  • SqlAlchemyMemoryRepository constrained by organization_id + graph_id.
  • Deterministic node normalization, confidence bounds, graph/node scope checks, and bounded search.
  • Golden parity fixture and local adapters for entity filtering/context and quick search output shapes.
  • Strict Pydantic extraction models with extra="forbid".
  • Bounded Thai/English extraction prompt builder.
  • JSON/code-fence parser with validation.
  • MemoryExtractionService that calls a JSON LLM client, validates output, resolves entity references deterministically, and persists only through the scoped repository.

Evidence

  • Memory repository tests: 2 passed.
  • Extraction contract tests: 3 passed.
  • Memory service test: 1 passed.
  • Full backend suite: 47 passed.
  • Frontend contract suite: 10 passed.
  • Vite build: pass.
  • Python compileall: pass.
  • Alembic revisions 0001_identity0003_memory upgrade/check: pass.

Deliberate non-change

Existing GraphBuilderService, ZepEntityReader, ZepToolsService, OasisProfileGenerator, and ReportAgent still use Zep. Consumer cutover is deferred until golden parity fixtures exist for entity reads, search, temporal edges, and simulation context.