Alwrity version 0.5.4
This commit is contained in:
ajaysi
362
docs/enhanced_strategy_refactoring_plan.md
Normal file
362
docs/enhanced_strategy_refactoring_plan.md
Normal file
@@ -0,0 +1,362 @@
|
||||
# Enhanced Strategy Refactoring Plan
|
||||
## Least Invasive Module Breakdown Strategy
|
||||
|
||||
### 📋 Overview
|
||||
This document outlines the **least invasive plan** to break down the large `enhanced_strategy_service.py` and `enhanced_strategy_routes.py` modules without breaking the current autofill functionality that achieves **100% success rate**.
|
||||
|
||||
### 🎯 Goals
|
||||
- **Zero Risk**: Maintain 100% autofill success rate throughout refactoring
|
||||
- **Gradual Reduction**: Break down large modules into smaller, manageable pieces
|
||||
- **Independent Testing**: Each extraction is independently testable
|
||||
- **Reversible**: Each step can be rolled back if issues arise
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Critical Protection Zones
|
||||
|
||||
### **NEVER TOUCH (Autofill Core)**
|
||||
```python
|
||||
# These files are the autofill core - NEVER modify during refactoring:
|
||||
❌ backend/api/content_planning/services/content_strategy/autofill/ai_structured_autofill.py
|
||||
❌ backend/api/content_planning/services/content_strategy/autofill/ai_refresh.py
|
||||
❌ backend/api/content_planning/api/enhanced_strategy_routes.py (stream_autofill_refresh endpoint)
|
||||
❌ Any autofill-related imports or dependencies
|
||||
```
|
||||
|
||||
### **Protected Functionality**
|
||||
- ✅ 100% AI autofill success rate (30/30 fields)
|
||||
- ✅ All category completion percentages
|
||||
- ✅ Field type normalization (select, multiselect, numeric)
|
||||
- ✅ Optimized retry logic (stop at 100% success)
|
||||
- ✅ Frontend data flow and display
|
||||
|
||||
---
|
||||
|
||||
## 📁 Phase 1: Enhanced Strategy Service Breakdown
|
||||
|
||||
### **Current State**
|
||||
- **File**: `backend/api/content_planning/services/enhanced_strategy_service.py`
|
||||
- **Size**: ~800+ lines
|
||||
- **Status**: Monolithic, difficult to maintain
|
||||
|
||||
### **Target Structure**
|
||||
```
|
||||
📁 backend/api/content_planning/services/enhanced_strategy/
|
||||
├── 📄 __init__.py (imports from submodules)
|
||||
├── 📁 core/
|
||||
│ ├── 📄 strategy_service.py (main orchestration - keep existing)
|
||||
│ ├── 📄 strategy_validation.py (extract validation logic)
|
||||
│ └── 📄 strategy_utils.py (extract utility functions)
|
||||
├── 📁 data/
|
||||
│ ├── 📄 onboarding_integration.py (extract onboarding logic)
|
||||
│ └── 📄 data_transformation.py (extract data processing)
|
||||
└── 📁 operations/
|
||||
├── 📄 strategy_operations.py (extract CRUD operations)
|
||||
└── 📄 strategy_analytics.py (extract analytics logic)
|
||||
```
|
||||
|
||||
### **Extraction Order (Safest First)**
|
||||
|
||||
#### **1. Strategy Validation (Week 1)**
|
||||
**File**: `core/strategy_validation.py`
|
||||
**Functions to extract**:
|
||||
- `_validate_strategy_data()`
|
||||
- `_validate_field_value()`
|
||||
- `_validate_business_rules()`
|
||||
|
||||
**Risk Level**: 🟢 **LOW** - Pure validation logic, no dependencies
|
||||
|
||||
#### **2. Strategy Utils (Week 1)**
|
||||
**File**: `core/strategy_utils.py`
|
||||
**Functions to extract**:
|
||||
- `_calculate_completion_percentage()`
|
||||
- `_calculate_data_quality_scores()`
|
||||
- `_calculate_confidence_levels()`
|
||||
- `_calculate_data_freshness()`
|
||||
|
||||
**Risk Level**: 🟢 **LOW** - Simple calculations, minimal dependencies
|
||||
|
||||
#### **3. Data Transformation (Week 2)**
|
||||
**File**: `data/data_transformation.py`
|
||||
**Functions to extract**:
|
||||
- `_create_field_mappings()`
|
||||
- `_transform_onboarding_data()`
|
||||
- `_merge_strategy_with_onboarding()`
|
||||
|
||||
**Risk Level**: 🟡 **MEDIUM** - Data processing logic, some dependencies
|
||||
|
||||
#### **4. Onboarding Integration (Week 2)**
|
||||
**File**: `data/onboarding_integration.py`
|
||||
**Functions to extract**:
|
||||
- `_enhance_strategy_with_onboarding_data()`
|
||||
- `_process_onboarding_data()`
|
||||
- `_get_onboarding_data()`
|
||||
|
||||
**Risk Level**: 🟡 **MEDIUM** - Database operations, moderate dependencies
|
||||
|
||||
#### **5. Strategy Operations (Week 3)**
|
||||
**File**: `operations/strategy_operations.py`
|
||||
**Functions to extract**:
|
||||
- `create_enhanced_strategy()`
|
||||
- `update_enhanced_strategy()`
|
||||
- `delete_enhanced_strategy()`
|
||||
- `get_enhanced_strategy()`
|
||||
|
||||
**Risk Level**: 🟠 **HIGH** - Core CRUD operations, many dependencies
|
||||
|
||||
#### **6. Strategy Analytics (Week 3)**
|
||||
**File**: `operations/strategy_analytics.py`
|
||||
**Functions to extract**:
|
||||
- `get_ai_analysis()`
|
||||
- `regenerate_ai_analysis()`
|
||||
- `get_performance_report()`
|
||||
|
||||
**Risk Level**: 🟠 **HIGH** - Analytics operations, external dependencies
|
||||
|
||||
---
|
||||
|
||||
## 📁 Phase 2: Enhanced Strategy Routes Breakdown
|
||||
|
||||
### **Current State**
|
||||
- **File**: `backend/api/content_planning/api/enhanced_strategy_routes.py`
|
||||
- **Size**: ~1000+ lines
|
||||
- **Status**: Monolithic, difficult to maintain
|
||||
|
||||
### **Target Structure**
|
||||
```
|
||||
📁 backend/api/content_planning/api/enhanced_strategy/
|
||||
├── 📄 __init__.py (imports from submodules)
|
||||
├── 📄 routes.py (main router - keep existing)
|
||||
├── 📁 endpoints/
|
||||
│ ├── 📄 strategy_crud.py (extract CRUD endpoints)
|
||||
│ ├── 📄 autofill_endpoints.py (extract autofill endpoints)
|
||||
│ └── 📄 analytics_endpoints.py (extract analytics endpoints)
|
||||
└── 📁 middleware/
|
||||
├── 📄 validation.py (extract validation middleware)
|
||||
└── 📄 error_handling.py (extract error handling)
|
||||
```
|
||||
|
||||
### **Extraction Order (Safest First)**
|
||||
|
||||
#### **1. Strategy CRUD Endpoints (Week 1)**
|
||||
**File**: `endpoints/strategy_crud.py`
|
||||
**Endpoints to extract**:
|
||||
- `get_enhanced_strategies()`
|
||||
- `delete_enhanced_strategy()`
|
||||
- `update_enhanced_strategy()`
|
||||
|
||||
**Risk Level**: 🟢 **LOW** - Read/delete operations, minimal dependencies
|
||||
|
||||
#### **2. Analytics Endpoints (Week 2)**
|
||||
**File**: `endpoints/analytics_endpoints.py`
|
||||
**Endpoints to extract**:
|
||||
- `get_ai_analysis()`
|
||||
- `regenerate_ai_analysis()`
|
||||
- `get_performance_report()`
|
||||
|
||||
**Risk Level**: 🟡 **MEDIUM** - Analytics operations, separate domain
|
||||
|
||||
#### **3. Validation Middleware (Week 2)**
|
||||
**File**: `middleware/validation.py`
|
||||
**Functions to extract**:
|
||||
- `validate_strategy_input()`
|
||||
- `validate_user_permissions()`
|
||||
- `validate_strategy_exists()`
|
||||
|
||||
**Risk Level**: 🟡 **MEDIUM** - Validation logic, moderate dependencies
|
||||
|
||||
#### **4. Error Handling (Week 3)**
|
||||
**File**: `middleware/error_handling.py`
|
||||
**Functions to extract**:
|
||||
- `handle_strategy_errors()`
|
||||
- `handle_validation_errors()`
|
||||
- `handle_database_errors()`
|
||||
|
||||
**Risk Level**: 🟠 **HIGH** - Error handling, many dependencies
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Implementation Strategy
|
||||
|
||||
### **Step-by-Step Process**
|
||||
|
||||
#### **Before Each Extraction**
|
||||
1. **Create Backup**
|
||||
```bash
|
||||
cp enhanced_strategy_service.py enhanced_strategy_service_backup.py
|
||||
```
|
||||
|
||||
2. **Create New Module**
|
||||
```python
|
||||
# Create new file with extracted functions
|
||||
# Keep all existing imports and functionality intact
|
||||
```
|
||||
|
||||
3. **Update Imports**
|
||||
```python
|
||||
# In original file, add import for new module
|
||||
from .core.strategy_validation import validate_strategy_data
|
||||
```
|
||||
|
||||
4. **Test Autofill Functionality**
|
||||
```bash
|
||||
# Test the critical autofill endpoint
|
||||
curl -X POST "http://localhost:8000/api/content-planning/enhanced-strategies/autofill/refresh" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"user_id": 1, "use_ai": true, "ai_only": true}'
|
||||
```
|
||||
|
||||
5. **Verify Success Metrics**
|
||||
- ✅ 100% autofill success rate maintained
|
||||
- ✅ All fields populated correctly
|
||||
- ✅ No breaking changes to existing functionality
|
||||
|
||||
6. **Remove Old Functions**
|
||||
```python
|
||||
# Only after all tests pass
|
||||
# Remove extracted functions from original files
|
||||
```
|
||||
|
||||
### **Testing Checklist**
|
||||
|
||||
#### **Autofill Functionality Test**
|
||||
- [ ] Click "Refresh Data (AI)" button
|
||||
- [ ] Verify 100% success rate in logs
|
||||
- [ ] Verify all 30 fields populated
|
||||
- [ ] Verify proper field types (select, multiselect, numeric)
|
||||
- [ ] Verify frontend displays values correctly
|
||||
|
||||
#### **General Functionality Test**
|
||||
- [ ] Create new strategy
|
||||
- [ ] Update existing strategy
|
||||
- [ ] Delete strategy
|
||||
- [ ] View AI analysis
|
||||
- [ ] Access all endpoints
|
||||
|
||||
---
|
||||
|
||||
## 📊 Success Metrics
|
||||
|
||||
### **Quantitative Metrics**
|
||||
- ✅ **Autofill Success Rate**: Maintain 100% (30/30 fields)
|
||||
- ✅ **Category Completion**: All categories 100% complete
|
||||
- ✅ **Response Time**: No degradation in performance
|
||||
- ✅ **Error Rate**: Zero errors in autofill functionality
|
||||
|
||||
### **Qualitative Metrics**
|
||||
- ✅ **Code Organization**: Improved modularity
|
||||
- ✅ **Maintainability**: Easier to locate and modify code
|
||||
- ✅ **Testability**: Independent testing of modules
|
||||
- ✅ **Readability**: Smaller, focused files
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Risk Mitigation
|
||||
|
||||
### **High-Risk Scenarios**
|
||||
1. **Import Path Issues**: Use absolute imports where possible
|
||||
2. **Circular Dependencies**: Monitor import cycles
|
||||
3. **Breaking Changes**: Test thoroughly before removing old code
|
||||
4. **Performance Degradation**: Monitor response times
|
||||
|
||||
### **Rollback Strategy**
|
||||
1. **Immediate Rollback**: Restore backup files
|
||||
2. **Gradual Rollback**: Revert specific extractions
|
||||
3. **Partial Rollback**: Keep some extractions, revert others
|
||||
|
||||
### **Emergency Procedures**
|
||||
1. **Stop All Refactoring**: If autofill breaks
|
||||
2. **Restore Last Working State**: Use git revert
|
||||
3. **Investigate Root Cause**: Before proceeding
|
||||
4. **Document Issues**: For future reference
|
||||
|
||||
---
|
||||
|
||||
## 📅 Implementation Timeline
|
||||
|
||||
### **Week 1: Foundation**
|
||||
- [ ] Create directory structure
|
||||
- [ ] Extract validation functions
|
||||
- [ ] Extract utility functions
|
||||
- [ ] Test autofill functionality
|
||||
|
||||
### **Week 2: Data Layer**
|
||||
- [ ] Extract data transformation functions
|
||||
- [ ] Extract onboarding integration functions
|
||||
- [ ] Extract CRUD endpoints
|
||||
- [ ] Test autofill functionality
|
||||
|
||||
### **Week 3: Operations Layer**
|
||||
- [ ] Extract strategy operations
|
||||
- [ ] Extract analytics functions
|
||||
- [ ] Extract validation middleware
|
||||
- [ ] Test autofill functionality
|
||||
|
||||
### **Week 4: Cleanup**
|
||||
- [ ] Remove old functions from original files
|
||||
- [ ] Update documentation
|
||||
- [ ] Final testing
|
||||
- [ ] Performance validation
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Monitoring & Validation
|
||||
|
||||
### **Continuous Monitoring**
|
||||
- **Autofill Success Rate**: Must stay at 100%
|
||||
- **Response Times**: No degradation
|
||||
- **Error Logs**: Monitor for new errors
|
||||
- **User Experience**: Frontend functionality intact
|
||||
|
||||
### **Validation Points**
|
||||
- **After Each Extraction**: Test autofill functionality
|
||||
- **Daily**: Run full test suite
|
||||
- **Weekly**: Performance benchmarking
|
||||
- **Before Production**: Complete integration testing
|
||||
|
||||
---
|
||||
|
||||
## 📝 Documentation Updates
|
||||
|
||||
### **Files to Update**
|
||||
- [ ] API documentation
|
||||
- [ ] Service documentation
|
||||
- [ ] README files
|
||||
- [ ] Code comments
|
||||
- [ ] Architecture diagrams
|
||||
|
||||
### **Documentation Standards**
|
||||
- Clear module responsibilities
|
||||
- Import/export documentation
|
||||
- Dependency mapping
|
||||
- Testing instructions
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Success Criteria
|
||||
|
||||
### **Primary Success Criteria**
|
||||
1. **Zero Breaking Changes**: All existing functionality works
|
||||
2. **100% Autofill Success**: Maintain current performance
|
||||
3. **Improved Maintainability**: Easier to locate and modify code
|
||||
4. **Better Organization**: Logical module structure
|
||||
|
||||
### **Secondary Success Criteria**
|
||||
1. **Reduced File Sizes**: No file > 300 lines
|
||||
2. **Clear Dependencies**: Minimal circular dependencies
|
||||
3. **Independent Testing**: Each module testable in isolation
|
||||
4. **Documentation**: Complete and accurate
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Next Steps
|
||||
|
||||
1. **Review Plan**: Stakeholder approval
|
||||
2. **Create Backups**: Before starting
|
||||
3. **Set Up Monitoring**: Track success metrics
|
||||
4. **Begin Phase 1**: Start with validation functions
|
||||
5. **Iterate**: Learn and adjust as needed
|
||||
|
||||
---
|
||||
|
||||
*This plan ensures we maintain the critical autofill functionality while gradually improving code organization and maintainability.*
|
||||
Reference in New Issue
Block a user