moreminimore-vibe/README-CUSTOM-INTEGRATION.md

Dyad Custom Features Integration Guide

This guide explains how to use the custom features integration script to maintain your remove-limit modifications when updating the main Dyad codebase.

Overview

The integration script (scripts/integrate-custom-features.sh) helps you merge your custom remove-limit features with upstream updates from the original Dyad repository. It automatically:

• Creates backups before making changes
• Detects custom modifications in key files
• Creates missing custom files
• Validates the integration
• Provides restore functionality

Features Integrated

Remove Limit Features

• Smart Context Management: Advanced context handling with rolling summaries
• Rate Limit Simulation: Configurable rate limiting for testing
• Enhanced Chat Streaming: Improved message handling with custom payloads
• Context Snippet Management: Intelligent snippet organization and retrieval

Smart Context System

• Context Caching: Efficient caching with TTL
• Rolling Summaries: Automatic summarization when context exceeds thresholds
• Snippet Importance Scoring: Intelligent ranking of context snippets
• Size Management: Automatic trimming and optimization

Usage

Basic Integration

# Integrate custom features (creates backup automatically)
./scripts/integrate-custom-features.sh integrate

# Validate current integration
./scripts/integrate-custom-features.sh validate

# Show help
./scripts/integrate-custom-features.sh help

Restore from Backup

# List available backups
ls -la backups/

# Restore from a specific backup
./scripts/integrate-custom-features.sh restore backup-20231201-120000

Files Managed

Core Custom Files

• src/components/HelpDialog.tsx - Enhanced help dialog with custom features
• src/components/chat/PromoMessage.tsx - Modified promo messages
• src/ipc/handlers/chat_stream_handlers.ts - Enhanced chat streaming with remove-limit
• src/ipc/ipc_client.ts - Extended IPC client with smart context methods
• src/ipc/ipc_host.ts - Updated IPC host with new handlers
• src/ipc/ipc_types.ts - Extended type definitions
• src/preload.ts - Updated preload script with new channels
• testing/fake-llm-server/chatCompletionHandler.ts - Enhanced testing server

New Custom Files

• src/ipc/handlers/smart_context_handlers.ts - Smart context IPC handlers
• src/ipc/utils/smart_context_store.ts - Context storage and management
• src/hooks/useSmartContext.ts - React hooks for smart context

Complete Step-by-Step Integration Guide

This section provides detailed, copy-paste ready commands for every step of the integration process. Follow these commands exactly as written.

📋 Pre-Update Checklist (5 minutes)

Step 1: Verify Current Working Directory

# Make sure you're in the correct directory
pwd
# Expected output: /Users/kunthawatgreethong/Gitea/moreminimore-vibe

# If not, navigate to the correct directory
cd /Users/kunthawatgreethong/Gitea/moreminimore-vibe

Step 2: Check Git Status

# Check if you have any uncommitted changes
git status
# Expected: "working tree clean" or list your changes

# If you have changes, commit them first
git add .
git commit -m "Save current state before upstream update"

Step 3: Verify Custom Features Are Working

# Validate current integration
./scripts/integrate-custom-features.sh validate
# Expected: [SUCCESS] Integration validation passed

# If validation fails, fix issues before proceeding
./scripts/integrate-custom-features.sh help

Step 4: Check Current Git Branch

# Make sure you're on your main branch
git branch
# Expected: * main or * master (with * indicating current branch)

# If not on main branch, switch to it
git checkout main

🔄 Upstream Update Process (10-15 minutes)

Step 5: Fetch Latest Changes from Upstream

# Fetch all changes from upstream repository
git fetch upstream
# Expected: No output (or shows what was fetched)

# Check what new commits are available
git log main..upstream/main --oneline
# Expected: List of new commits from upstream

Step 6: Stash Your Local Changes (Optional but Recommended)

# If you have any local changes not committed
git stash push -m "Temporary stash before upstream merge"
# Expected: Saved working directory and index state

# To see your stashes
git stash list

Step 7: Merge Upstream Changes

# Merge upstream changes into your main branch
git merge upstream/main
# Expected: Shows merge summary or conflicts

# If there are conflicts, you'll see:
# CONFLICT (content): Merge conflict in file-name

Step 8: Handle Merge Conflicts (If Any)

# If conflicts exist, check which files have conflicts
git status
# Look for files marked as "both modified"

# Open conflicted files and resolve them manually
# Look for these markers in files:
# <<<<<<< HEAD
# Your changes
# =======
# Upstream changes
# >>>>>>> upstream/main

# After resolving conflicts in each file:
git add path/to/resolved/file.ts

# Continue with merge when all conflicts resolved
git merge --continue

Step 9: Restore Stashed Changes (If You Stashed)

# Restore your stashed changes
git stash pop
# Expected: Shows what was restored

# If there are new conflicts, resolve them
git status
git add . (resolve conflicts first)
git commit -m "Merge upstream changes and resolve conflicts"

🔧 Integration Process (5-10 minutes)

Step 10: Run the Integration Script

# Execute the integration script
./scripts/integrate-custom-features.sh integrate
# Expected output:
# [2025-12-18 HH:MM:SS] Starting custom features integration...
# [2025-12-18 HH:MM:SS] Creating backup: backup-YYYYMMDD-HHMMSS
# [SUCCESS] Backup created at: /Users/kunthawatgreethong/Gitea/moreminimore-vibe/backups/backup-YYYYMMDD-HHMMSS
# [2025-12-18 HH:MM:SS] Creating missing custom files...
# [SUCCESS] Missing custom files created
# [2025-12-18 HH:MM:SS] Validating integration...
# [SUCCESS] Integration validation passed
# [SUCCESS] Custom features integration completed successfully!

Step 11: Verify Integration Success

# Check that all expected files exist
ls -la src/ipc/handlers/smart_context_handlers.ts
ls -la src/ipc/utils/smart_context_store.ts
ls -la src/hooks/useSmartContext.ts

# Expected: All three files should exist and show file info

✅ Validation and Testing (10-15 minutes)

Step 12: Run Validation Script

# Validate the integration
./scripts/integrate-custom-features.sh validate
# Expected:
# [2025-12-18 HH:MM:SS] Validating integration...
# [SUCCESS] Integration validation passed

# If you see warnings, note them but continue
# Warnings about missing custom modifications are usually OK

Step 13: Check TypeScript Compilation

# Check for TypeScript errors (may have some MCP-related warnings)
npm run ts
# Expected: May show some MCP-related errors but should complete
# If many new errors appear, investigate before proceeding

Step 14: Test Application Startup

# Install any new dependencies if needed
npm install

# Start the application in development mode
npm start
# Expected: Application should start without critical errors
# Let it run for 30-60 seconds to ensure stability
# Press Ctrl+C to stop when confirmed working

Step 15: Test Custom Features Manually

# Start the application again
npm start

# Test these features in the UI:
# 1. Create a new chat
# 2. Send a message
# 3. Check if smart context is working (no obvious errors)
# 4. Verify help dialog opens correctly
# 5. Check that promo messages display properly

# If everything works, integration is successful!
# If issues occur, proceed to troubleshooting section

🚨 If Something Goes Wrong

Step 16: Identify the Problem

# Check what went wrong
./scripts/integrate-custom-features.sh validate
# Note any errors or warnings

# Check application logs if it won't start
npm start 2>&1 | tee startup-errors.log
# Review startup-errors.log for clues

Step 17: Restore from Backup (If Needed)

# List available backups
ls -la backups/
# Expected: List of backup directories with timestamps

# Find your most recent backup (look for the latest timestamp)
# Example: backup-20231218-154512

# Restore from the most recent working backup
./scripts/integrate-custom-features.sh restore backup-20231218-154512
# Expected:
# [2025-12-18 HH:MM:SS] Restoring from backup: backup-20231218-154512
# [SUCCESS] Restore completed

Step 18: Verify Restoration

# Check that custom features are back
./scripts/integrate-custom-features.sh validate
# Expected: [SUCCESS] Integration validation passed

# Test application startup
npm start
# Should work as it did before the update

Step 19: Manual Investigation (If Restoration Doesn't Work)

# Check Git state
git status
git log --oneline -10

# Check specific files for issues
git diff HEAD~1 src/ipc/ipc_types.ts
git diff HEAD~1 src/ipc/ipc_client.ts

# Look for specific error patterns
grep -r "SmartContext" src/ --include="*.ts" --include="*.tsx"

🔍 Troubleshooting Commands

Check Integration Script Health

# Make script executable (if permission issues)
chmod +x scripts/integrate-custom-features.sh

# Test script help
./scripts/integrate-custom-features.sh help
# Should show usage instructions

Verify File Permissions

# Check script permissions
ls -la scripts/integrate-custom-features.sh
# Should show -rwxr-xr-x (executable)

# Fix permissions if needed
chmod 755 scripts/integrate-custom-features.sh

Check Node.js and npm

# Verify Node.js version
node --version
# Expected: v18.x.x or higher

# Verify npm version
npm --version
# Expected: 9.x.x or higher

# Clear npm cache if needed
npm cache clean --force

Check Git Configuration

# Verify upstream remote is configured
git remote -v
# Should show upstream remote pointing to dyad-sh/dyad

# If upstream is missing, add it
git remote add upstream https://github.com/dyad-sh/dyad.git

📝 Quick Reference Commands

Emergency Restore (One-liner)

# Restore latest backup (replace with actual backup name)
./scripts/integrate-custom-features.sh restore $(ls -t backups/ | head -1)

Check All Backups

# List all backups with details
ls -la backups/ | grep backup

Force Clean Integration

# For problematic integrations, start fresh
git clean -fd
git reset --hard HEAD
./scripts/integrate-custom-features.sh integrate

Verify All Custom Files

# Quick check that all custom files exist
files=("src/ipc/handlers/smart_context_handlers.ts" "src/ipc/utils/smart_context_store.ts" "src/hooks/useSmartContext.ts")
for file in "${files[@]}"; do
  if [[ -f "$file" ]]; then
    echo "✅ $file exists"
  else
    echo "❌ $file missing"
  fi
done

⏱️ Time Estimates

• Pre-Update Checklist: 5 minutes
• Upstream Update: 10-15 minutes (longer if many conflicts)
• Integration Process: 5-10 minutes
• Validation and Testing: 10-15 minutes
• Troubleshooting: 15-30 minutes (if needed)

Total Time: 30-60 minutes for smooth integration, up to 2 hours if problems occur.

🎯 Success Indicators

You'll know the integration was successful when:

1. ✅ Integration script runs without errors
2. ✅ Validation script shows "[SUCCESS] Integration validation passed"
3. ✅ Application starts with npm start
4. ✅ No critical errors in console
5. ✅ Custom features work in the UI
6. ✅ TypeScript compilation completes (may have warnings)

If all these are true, your custom features have been successfully integrated with the latest upstream changes!

---

Integration Workflow Summary

Before Upstream Update

1. Current State: Your custom features are integrated
2. Backup: Script automatically creates backup
3. Validation: Ensure everything is working

After Upstream Update

1. Update Code: Pull latest changes from upstream
2. Run Integration: ./scripts/integrate-custom-features.sh integrate
3. Validate: Check that integration succeeded
4. Test: Verify custom features work correctly

If Something Goes Wrong

1. Restore: ./scripts/integrate-custom-features.sh restore <backup-name>
2. Investigate: Check what conflicts occurred
3. Manual Merge: Resolve conflicts manually if needed
4. Retry: Run integration again

Custom Features Explained

Smart Context System

The smart context system provides intelligent management of chat context to handle longer conversations without hitting token limits.

Key Components

1. SmartContextStore: Manages context caching, summarization, and retrieval
2. Context Snippets: Individual pieces of context with importance scoring
3. Rolling Summaries: Automatically generated summaries of older messages
4. Size Management: Intelligent trimming based on importance and recency

Usage in Components

import { useSmartContext, useUpdateSmartContext } from '@/hooks/useSmartContext';

// Get smart context for a chat
const { data: context, isLoading } = useSmartContext({
  chatId: 123,
  maxTokens: 8000,
  includeSummaries: true
});

// Update context with new content
const updateContext = useUpdateSmartContext();
updateContext.mutate({
  chatId: 123,
  content: "New message content",
  type: "message",
  importance: 1.0
});

Rate Limit Simulation

The system includes configurable rate limiting for testing purposes:

// In chat_stream_handlers.ts
const rateLimitConfig = {
  enabled: true,
  requestsPerMinute: 60,
  burstLimit: 10
};

Enhanced Chat Streaming

Improved message handling with:

• Custom payload processing
• Better error handling
• Enhanced metadata support
• Streaming optimizations

Configuration

Smart Context Settings

You can configure the smart context behavior by modifying the SmartContextStore:

// In src/ipc/utils/smart_context_store.ts
private maxContextSize = 100000; // 100k characters
private maxSnippets = 50;
private summaryThreshold = 20000; // Summarize when context exceeds this

Rate Limit Settings

Configure rate limiting in the chat stream handlers:

// Adjust these values based on your needs
const RATE_LIMIT_CONFIG = {
  requestsPerMinute: 60,
  burstLimit: 10,
  enabled: process.env.NODE_ENV === 'development'
};

Troubleshooting

Common Issues

1. TypeScript Errors: The script automatically fixes MCP-related TypeScript issues during integration.
2. Missing Custom Modifications: The script warns if files don't contain expected custom patterns.
3. Backup Restoration: Always restore from the most recent working backup.

MCP TypeScript Issues

The integration script automatically handles MCP (Model Context Protocol) related TypeScript compilation errors:

Issues Fixed:

• chat_stream_handlers.ts: Adds type assertion (as any) for tool objects
• mcp_handlers.ts: Adds type assertion for tool.description property
• mcp_manager.ts: Replaces problematic imports with stub implementation

Automatic Fixes: The fix_mcp_typescript_issues() function in the script:

1. Detects MCP-related type errors
2. Applies appropriate type assertions
3. Creates stub implementations for missing exports
4. Ensures compilation succeeds

Manual Fix (if needed): If you encounter MCP TypeScript errors after integration:

# Re-run the integration script to fix MCP issues
./scripts/integrate-custom-features.sh integrate

# Or manually fix by adding 'as any' type assertions to tool objects

Validation Warnings

If you see warnings about missing custom modifications:

1. Check if the file actually needs custom changes
2. Verify the custom patterns are being detected correctly
3. Manually add custom modifications if needed

Manual Intervention

Sometimes you may need to manually merge changes:

1. Conflicts: Resolve Git conflicts in key files
2. New Features: Add custom features to new upstream files
3. API Changes: Update custom code to match new APIs

Best Practices

1. Regular Backups: Always create backups before major changes
2. Test Thoroughly: Verify all custom features work after integration
3. Document Changes: Keep notes about manual modifications
4. Version Control: Commit working states regularly
5. Gradual Updates: Update upstream changes incrementally

Development

Adding New Custom Features

1. Update Script: Add new files to the CUSTOM_FILES or NEW_CUSTOM_FILES arrays
2. Create Handlers: Add IPC handlers for new functionality
3. Update Types: Extend type definitions as needed
4. Test Integration: Ensure new features integrate properly

Modifying Existing Features

1. Update Patterns: Add new detection patterns to has_custom_modifications()
2. Test Validation: Ensure the script detects your changes
3. Document Updates: Update this documentation

Support

If you encounter issues:

1. Check Logs: Look for error messages in the script output
2. Restore Backup: Use a recent backup to restore working state
3. Manual Merge: Resolve conflicts manually if needed
4. Test Incrementally: Test changes one at a time

Future Enhancements

Planned improvements to the integration script:

• Automatic Conflict Resolution: Smarter merging of conflicting changes
• Enhanced Validation: More comprehensive testing
• GUI Interface: Visual tool for managing integrations
• Cloud Backups: Optional cloud backup storage
• Rollback System: More granular rollback capabilities

---

Note: This integration script is designed to work with the specific remove-limit features. If you have additional custom modifications, you may need to extend the script accordingly.