# Dyad Custom Features Integration Guide

This guide explains how to use the custom features integration script to maintain your remove-limit modifications when updating the main Dyad codebase.

## Overview

The integration script (`scripts/integrate-custom-features.sh`) helps you merge your custom remove-limit features with upstream updates from the original Dyad repository. It automatically:

- Creates backups before making changes
- Detects custom modifications in key files
- Creates missing custom files
- Validates the integration
- Provides restore functionality

## Features Integrated

### Remove Limit Features
- **Smart Context Management**: Advanced context handling with rolling summaries
- **Rate Limit Simulation**: Configurable rate limiting for testing
- **Enhanced Chat Streaming**: Improved message handling with custom payloads
- **Context Snippet Management**: Intelligent snippet organization and retrieval

### Smart Context System
- **Context Caching**: Efficient caching with TTL
- **Rolling Summaries**: Automatic summarization when context exceeds thresholds
- **Snippet Importance Scoring**: Intelligent ranking of context snippets
- **Size Management**: Automatic trimming and optimization

## Usage

### Basic Integration

```bash
# Integrate custom features (creates backup automatically)
./scripts/integrate-custom-features.sh integrate

# Validate current integration
./scripts/integrate-custom-features.sh validate

# Show help
./scripts/integrate-custom-features.sh help
```

### Restore from Backup

```bash
# List available backups
ls -la backups/

# Restore from a specific backup
./scripts/integrate-custom-features.sh restore backup-20231201-120000
```

## Files Managed

### Core Custom Files
- `src/components/HelpDialog.tsx` - Enhanced help dialog with custom features
- `src/components/chat/PromoMessage.tsx` - Modified promo messages
- `src/ipc/handlers/chat_stream_handlers.ts` - Enhanced chat streaming with remove-limit
- `src/ipc/ipc_client.ts` - Extended IPC client with smart context methods
- `src/ipc/ipc_host.ts` - Updated IPC host with new handlers
- `src/ipc/ipc_types.ts` - Extended type definitions
- `src/preload.ts` - Updated preload script with new channels
- `testing/fake-llm-server/chatCompletionHandler.ts` - Enhanced testing server

### New Custom Files
- `src/ipc/handlers/smart_context_handlers.ts` - Smart context IPC handlers
- `src/ipc/utils/smart_context_store.ts` - Context storage and management
- `src/hooks/useSmartContext.ts` - React hooks for smart context

## Integration Workflow

### Before Upstream Update
1. **Current State**: Your custom features are integrated
2. **Backup**: Script automatically creates backup
3. **Validation**: Ensure everything is working

### After Upstream Update
1. **Update Code**: Pull latest changes from upstream
2. **Run Integration**: `./scripts/integrate-custom-features.sh integrate`
3. **Validate**: Check that integration succeeded
4. **Test**: Verify custom features work correctly

### If Something Goes Wrong
1. **Restore**: `./scripts/integrate-custom-features.sh restore <backup-name>`
2. **Investigate**: Check what conflicts occurred
3. **Manual Merge**: Resolve conflicts manually if needed
4. **Retry**: Run integration again

## Custom Features Explained

### Smart Context System

The smart context system provides intelligent management of chat context to handle longer conversations without hitting token limits.

#### Key Components

1. **SmartContextStore**: Manages context caching, summarization, and retrieval
2. **Context Snippets**: Individual pieces of context with importance scoring
3. **Rolling Summaries**: Automatically generated summaries of older messages
4. **Size Management**: Intelligent trimming based on importance and recency

#### Usage in Components

```typescript
import { useSmartContext, useUpdateSmartContext } from '@/hooks/useSmartContext';

// Get smart context for a chat
const { data: context, isLoading } = useSmartContext({
  chatId: 123,
  maxTokens: 8000,
  includeSummaries: true
});

// Update context with new content
const updateContext = useUpdateSmartContext();
updateContext.mutate({
  chatId: 123,
  content: "New message content",
  type: "message",
  importance: 1.0
});
```

### Rate Limit Simulation

The system includes configurable rate limiting for testing purposes:

```typescript
// In chat_stream_handlers.ts
const rateLimitConfig = {
  enabled: true,
  requestsPerMinute: 60,
  burstLimit: 10
};
```

### Enhanced Chat Streaming

Improved message handling with:
- Custom payload processing
- Better error handling
- Enhanced metadata support
- Streaming optimizations

## Configuration

### Smart Context Settings

You can configure the smart context behavior by modifying the `SmartContextStore`:

```typescript
// In src/ipc/utils/smart_context_store.ts
private maxContextSize = 100000; // 100k characters
private maxSnippets = 50;
private summaryThreshold = 20000; // Summarize when context exceeds this
```

### Rate Limit Settings

Configure rate limiting in the chat stream handlers:

```typescript
// Adjust these values based on your needs
const RATE_LIMIT_CONFIG = {
  requestsPerMinute: 60,
  burstLimit: 10,
  enabled: process.env.NODE_ENV === 'development'
};
```

## Troubleshooting

### Common Issues

1. **TypeScript Errors**: The script skips TypeScript compilation due to existing MCP issues. Focus on functionality first.
2. **Missing Custom Modifications**: The script warns if files don't contain expected custom patterns.
3. **Backup Restoration**: Always restore from the most recent working backup.

### Validation Warnings

If you see warnings about missing custom modifications:

1. Check if the file actually needs custom changes
2. Verify the custom patterns are being detected correctly
3. Manually add custom modifications if needed

### Manual Intervention

Sometimes you may need to manually merge changes:

1. **Conflicts**: Resolve Git conflicts in key files
2. **New Features**: Add custom features to new upstream files
3. **API Changes**: Update custom code to match new APIs

## Best Practices

1. **Regular Backups**: Always create backups before major changes
2. **Test Thoroughly**: Verify all custom features work after integration
3. **Document Changes**: Keep notes about manual modifications
4. **Version Control**: Commit working states regularly
5. **Gradual Updates**: Update upstream changes incrementally

## Development

### Adding New Custom Features

1. **Update Script**: Add new files to the `CUSTOM_FILES` or `NEW_CUSTOM_FILES` arrays
2. **Create Handlers**: Add IPC handlers for new functionality
3. **Update Types**: Extend type definitions as needed
4. **Test Integration**: Ensure new features integrate properly

### Modifying Existing Features

1. **Update Patterns**: Add new detection patterns to `has_custom_modifications()`
2. **Test Validation**: Ensure the script detects your changes
3. **Document Updates**: Update this documentation

## Support

If you encounter issues:

1. **Check Logs**: Look for error messages in the script output
2. **Restore Backup**: Use a recent backup to restore working state
3. **Manual Merge**: Resolve conflicts manually if needed
4. **Test Incrementally**: Test changes one at a time

## Future Enhancements

Planned improvements to the integration script:

- **Automatic Conflict Resolution**: Smarter merging of conflicting changes
- **Enhanced Validation**: More comprehensive testing
- **GUI Interface**: Visual tool for managing integrations
- **Cloud Backups**: Optional cloud backup storage
- **Rollback System**: More granular rollback capabilities

---

**Note**: This integration script is designed to work with the specific remove-limit features. If you have additional custom modifications, you may need to extend the script accordingly.