Base code

This commit is contained in:
Kunthawat Greethong
2026-01-08 22:39:53 +07:00
parent 697115c61a
commit c35fa52117
2169 changed files with 626670 additions and 0 deletions

View File

@@ -0,0 +1,679 @@
# 🤖 AI Integration Plan for Content Planning System
## 📋 Current Status Analysis
### ❌ **Issues Identified**
1. **Hardcoded Values**: All AI services currently use simulated data instead of real AI calls
2. **Missing AI Integration**: No actual LLM calls in FastAPI services
3. **Unused AI Infrastructure**: Gemini provider exists but not integrated
4. **Missing AI Prompts**: Advanced prompts from legacy system not implemented
### ✅ **Available AI Infrastructure**
1. **Gemini Provider**: `backend/llm_providers/gemini_provider.py`
2. **Main Text Generation**: `backend/llm_providers/main_text_generation.py`
3. **API Key Management**: `backend/services/api_key_manager.py`
4. **AI Prompts**: Available in `CONTENT_GAP_ANALYSIS_DEEP_DIVE.md`
## 🎯 **AI Integration Strategy**
### **Phase 1: Core AI Integration (Week 1)**
#### 1.1 **AI Engine Service Enhancement**
**File**: `backend/services/content_gap_analyzer/ai_engine_service.py`
**Current Issues**:
- All methods use hardcoded responses
- No actual AI calls implemented
- Missing integration with Gemini provider
**Implementation Plan**:
```python
# Add imports
from backend.llm_providers.main_text_generation import llm_text_gen
from backend.llm_providers.gemini_provider import gemini_structured_json_response
# Replace hardcoded responses with AI calls
async def analyze_content_gaps(self, analysis_summary: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze content gaps using AI insights."""
try:
prompt = f"""
As an expert SEO content strategist, analyze this comprehensive content gap analysis data and provide actionable insights:
TARGET ANALYSIS:
- Website: {analysis_summary.get('target_url', 'N/A')}
- Industry: {analysis_summary.get('industry', 'N/A')}
- SERP Opportunities: {analysis_summary.get('serp_opportunities', 0)} keywords not ranking
- Keyword Expansion: {analysis_summary.get('expanded_keywords_count', 0)} additional keywords identified
- Competitors Analyzed: {analysis_summary.get('competitors_analyzed', 0)} websites
DOMINANT CONTENT THEMES:
{json.dumps(analysis_summary.get('dominant_themes', {}), indent=2)}
PROVIDE:
1. Strategic Content Gap Analysis
2. Priority Content Recommendations (top 5)
3. Keyword Strategy Insights
4. Competitive Positioning Advice
5. Content Format Recommendations
6. Technical SEO Opportunities
7. Implementation Timeline (30/60/90 days)
Format as JSON with clear, actionable recommendations.
"""
# Use structured JSON response for better parsing
response = gemini_structured_json_response(
prompt=prompt,
schema={
"type": "object",
"properties": {
"strategic_insights": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {"type": "string"},
"insight": {"type": "string"},
"confidence": {"type": "number"},
"priority": {"type": "string"},
"estimated_impact": {"type": "string"}
}
}
},
"content_recommendations": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {"type": "string"},
"recommendation": {"type": "string"},
"priority": {"type": "string"},
"estimated_traffic": {"type": "string"},
"implementation_time": {"type": "string"}
}
}
},
"performance_predictions": {
"type": "object",
"properties": {
"estimated_traffic_increase": {"type": "string"},
"estimated_ranking_improvement": {"type": "string"},
"estimated_engagement_increase": {"type": "string"},
"estimated_conversion_increase": {"type": "string"},
"confidence_level": {"type": "string"}
}
}
}
}
)
return json.loads(response)
except Exception as e:
logger.error(f"Error in AI content gap analysis: {str(e)}")
return {}
```
#### 1.2 **Keyword Researcher AI Integration**
**File**: `backend/services/content_gap_analyzer/keyword_researcher.py`
**Implementation Plan**:
```python
# Add AI integration for keyword analysis
async def _analyze_keyword_trends(self, industry: str, target_keywords: Optional[List[str]] = None) -> Dict[str, Any]:
"""Analyze keyword trends using AI."""
try:
prompt = f"""
Analyze keyword opportunities for {industry} industry:
Target Keywords: {target_keywords or []}
Provide comprehensive keyword analysis including:
1. Search volume estimates
2. Competition levels
3. Trend analysis
4. Opportunity scoring
5. Content format recommendations
Format as structured JSON with detailed analysis.
"""
response = gemini_structured_json_response(
prompt=prompt,
schema={
"type": "object",
"properties": {
"trends": {
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"search_volume": {"type": "number"},
"difficulty": {"type": "number"},
"trend": {"type": "string"},
"competition": {"type": "string"},
"intent": {"type": "string"},
"cpc": {"type": "number"}
}
}
},
"summary": {
"type": "object",
"properties": {
"total_keywords": {"type": "number"},
"high_volume_keywords": {"type": "number"},
"low_competition_keywords": {"type": "number"},
"trending_keywords": {"type": "number"}
}
}
}
}
)
return json.loads(response)
except Exception as e:
logger.error(f"Error analyzing keyword trends: {str(e)}")
return {}
```
#### 1.3 **Competitor Analyzer AI Integration**
**File**: `backend/services/content_gap_analyzer/competitor_analyzer.py`
**Implementation Plan**:
```python
# Add AI integration for competitor analysis
async def _evaluate_market_position(self, competitors: List[Dict[str, Any]], industry: str) -> Dict[str, Any]:
"""Evaluate market position using AI."""
try:
prompt = f"""
Analyze the market position of competitors in the {industry} industry:
Competitor Analyses:
{json.dumps(competitors, indent=2)}
Provide:
1. Market position analysis
2. Content gaps
3. Competitive advantages
4. Strategic positioning recommendations
Format as structured JSON with detailed analysis.
"""
response = gemini_structured_json_response(
prompt=prompt,
schema={
"type": "object",
"properties": {
"market_leader": {"type": "string"},
"content_leader": {"type": "string"},
"quality_leader": {"type": "string"},
"market_gaps": {
"type": "array",
"items": {"type": "string"}
},
"opportunities": {
"type": "array",
"items": {"type": "string"}
},
"competitive_advantages": {
"type": "array",
"items": {"type": "string"}
},
"strategic_recommendations": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {"type": "string"},
"recommendation": {"type": "string"},
"priority": {"type": "string"},
"estimated_impact": {"type": "string"}
}
}
}
}
}
)
return json.loads(response)
except Exception as e:
logger.error(f"Error evaluating market position: {str(e)}")
return {}
```
### **Phase 2: Advanced AI Features (Week 2)**
#### 2.1 **Content Performance Prediction**
```python
async def predict_content_performance(self, content_data: Dict[str, Any]) -> Dict[str, Any]:
"""Predict content performance using AI."""
try:
prompt = f"""
Predict content performance based on the following data:
Content Data: {json.dumps(content_data, indent=2)}
Provide detailed performance predictions including:
1. Traffic predictions
2. Engagement predictions
3. Ranking predictions
4. Conversion predictions
5. Risk factors
6. Success factors
Format as structured JSON with confidence levels.
"""
response = gemini_structured_json_response(
prompt=prompt,
schema={
"type": "object",
"properties": {
"traffic_predictions": {
"type": "object",
"properties": {
"estimated_monthly_traffic": {"type": "string"},
"traffic_growth_rate": {"type": "string"},
"peak_traffic_month": {"type": "string"},
"confidence_level": {"type": "string"}
}
},
"engagement_predictions": {
"type": "object",
"properties": {
"estimated_time_on_page": {"type": "string"},
"estimated_bounce_rate": {"type": "string"},
"estimated_social_shares": {"type": "string"},
"estimated_comments": {"type": "string"},
"confidence_level": {"type": "string"}
}
},
"ranking_predictions": {
"type": "object",
"properties": {
"estimated_ranking_position": {"type": "string"},
"estimated_ranking_time": {"type": "string"},
"ranking_confidence": {"type": "string"},
"competition_level": {"type": "string"}
}
},
"conversion_predictions": {
"type": "object",
"properties": {
"estimated_conversion_rate": {"type": "string"},
"estimated_lead_generation": {"type": "string"},
"estimated_revenue_impact": {"type": "string"},
"confidence_level": {"type": "string"}
}
},
"risk_factors": {
"type": "array",
"items": {"type": "string"}
},
"success_factors": {
"type": "array",
"items": {"type": "string"}
}
}
}
)
return json.loads(response)
except Exception as e:
logger.error(f"Error in AI performance prediction: {str(e)}")
return {}
```
#### 2.2 **Strategic Intelligence Generation**
```python
async def generate_strategic_insights(self, analysis_data: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Generate strategic insights using AI."""
try:
prompt = f"""
Generate strategic insights based on the following analysis data:
Analysis Data: {json.dumps(analysis_data, indent=2)}
Provide strategic insights covering:
1. Content strategy recommendations
2. Competitive positioning advice
3. Content optimization suggestions
4. Innovation opportunities
5. Risk mitigation strategies
Format as structured JSON with detailed insights.
"""
response = gemini_structured_json_response(
prompt=prompt,
schema={
"type": "object",
"properties": {
"strategic_insights": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {"type": "string"},
"insight": {"type": "string"},
"reasoning": {"type": "string"},
"priority": {"type": "string"},
"estimated_impact": {"type": "string"},
"implementation_time": {"type": "string"}
}
}
}
}
}
)
result = json.loads(response)
return result.get('strategic_insights', [])
except Exception as e:
logger.error(f"Error generating AI strategic insights: {str(e)}")
return []
```
### **Phase 3: AI Prompt Optimization (Week 3)**
#### 3.1 **Enhanced AI Prompts**
Based on the deep dive analysis, implement these advanced prompts:
**Content Gap Analysis Prompt**:
```python
CONTENT_GAP_ANALYSIS_PROMPT = """
As an expert SEO content strategist, analyze this comprehensive content gap analysis data and provide actionable insights:
TARGET ANALYSIS:
- Website: {target_url}
- Industry: {industry}
- SERP Opportunities: {serp_opportunities} keywords not ranking
- Keyword Expansion: {expanded_keywords_count} additional keywords identified
- Competitors Analyzed: {competitors_analyzed} websites
DOMINANT CONTENT THEMES:
{dominant_themes}
PROVIDE:
1. Strategic Content Gap Analysis
2. Priority Content Recommendations (top 5)
3. Keyword Strategy Insights
4. Competitive Positioning Advice
5. Content Format Recommendations
6. Technical SEO Opportunities
7. Implementation Timeline (30/60/90 days)
Format as JSON with clear, actionable recommendations.
"""
```
**Market Position Analysis Prompt**:
```python
MARKET_POSITION_PROMPT = """
Analyze the market position of competitors in the {industry} industry:
Competitor Analyses:
{competitor_analyses}
Provide:
1. Market position analysis
2. Content gaps
3. Competitive advantages
4. Strategic positioning recommendations
Format as JSON with detailed analysis.
"""
```
**Keyword Analysis Prompt**:
```python
KEYWORD_ANALYSIS_PROMPT = """
Analyze keyword opportunities for {industry} industry:
Keyword Trends: {trend_analysis}
Search Intent: {intent_analysis}
Opportunities: {opportunities}
Provide:
1. High-priority keyword recommendations
2. Content format suggestions
3. Topic cluster development
4. Search intent optimization
Format as JSON with detailed analysis.
"""
```
### **Phase 4: AI Service Integration (Week 4)**
#### 4.1 **Create AI Service Manager**
**File**: `backend/services/ai_service_manager.py`
```python
"""
AI Service Manager
Centralized AI service management for content planning system.
"""
from typing import Dict, Any, List, Optional
from loguru import logger
import json
from backend.llm_providers.main_text_generation import llm_text_gen
from backend.llm_providers.gemini_provider import gemini_structured_json_response
class AIServiceManager:
"""Manages AI service interactions and prompt handling."""
def __init__(self):
"""Initialize AI service manager."""
self.logger = logger
self.prompts = self._load_prompts()
def _load_prompts(self) -> Dict[str, str]:
"""Load AI prompts from configuration."""
return {
'content_gap_analysis': CONTENT_GAP_ANALYSIS_PROMPT,
'market_position': MARKET_POSITION_PROMPT,
'keyword_analysis': KEYWORD_ANALYSIS_PROMPT,
'performance_prediction': PERFORMANCE_PREDICTION_PROMPT,
'strategic_insights': STRATEGIC_INSIGHTS_PROMPT
}
async def generate_content_gap_analysis(self, analysis_data: Dict[str, Any]) -> Dict[str, Any]:
"""Generate content gap analysis using AI."""
try:
prompt = self.prompts['content_gap_analysis'].format(**analysis_data)
response = gemini_structured_json_response(
prompt=prompt,
schema=CONTENT_GAP_ANALYSIS_SCHEMA
)
return json.loads(response)
except Exception as e:
self.logger.error(f"Error generating content gap analysis: {str(e)}")
return {}
async def generate_market_position_analysis(self, market_data: Dict[str, Any]) -> Dict[str, Any]:
"""Generate market position analysis using AI."""
try:
prompt = self.prompts['market_position'].format(**market_data)
response = gemini_structured_json_response(
prompt=prompt,
schema=MARKET_POSITION_SCHEMA
)
return json.loads(response)
except Exception as e:
self.logger.error(f"Error generating market position analysis: {str(e)}")
return {}
async def generate_keyword_analysis(self, keyword_data: Dict[str, Any]) -> Dict[str, Any]:
"""Generate keyword analysis using AI."""
try:
prompt = self.prompts['keyword_analysis'].format(**keyword_data)
response = gemini_structured_json_response(
prompt=prompt,
schema=KEYWORD_ANALYSIS_SCHEMA
)
return json.loads(response)
except Exception as e:
self.logger.error(f"Error generating keyword analysis: {str(e)}")
return {}
```
#### 4.2 **Update All Services to Use AI Manager**
```python
# In each service file, replace hardcoded responses with AI calls
from services.ai_service_manager import AIServiceManager
class AIEngineService:
def __init__(self):
self.ai_manager = AIServiceManager()
logger.info("AIEngineService initialized")
async def analyze_content_gaps(self, analysis_summary: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze content gaps using AI insights."""
return await self.ai_manager.generate_content_gap_analysis(analysis_summary)
async def analyze_market_position(self, market_data: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze market position using AI insights."""
return await self.ai_manager.generate_market_position_analysis(market_data)
```
## 📊 **Implementation Timeline**
### **Week 1: Core AI Integration** ✅ **COMPLETED**
- [x] Replace hardcoded responses in AI Engine Service
- [x] Integrate Gemini provider calls
- [x] Implement basic AI prompts
- [x] Test AI functionality
### **Week 2: Advanced AI Features** ✅ **COMPLETED**
- [x] Implement content performance prediction
- [x] Add strategic intelligence generation
- [x] Create comprehensive AI schemas
- [x] Optimize AI prompts
### **Week 3: AI Prompt Optimization** ✅ **COMPLETED**
- [x] Implement advanced prompts from deep dive
- [x] Create structured JSON schemas
- [x] Optimize prompt performance
- [x] Add error handling and fallbacks
**Status Update**: ✅ **AI Prompt Optimizer Service fully implemented**
- Advanced AI prompts from deep dive analysis implemented
- Comprehensive JSON schemas for structured responses
- Optimized prompt performance with expert-level instructions
- Robust error handling and fallback mechanisms
- Integration with existing AI engine service
### **Week 4: AI Service Integration** ✅ **COMPLETED**
- [x] Create AI Service Manager
- [x] Update all services to use AI Manager
- [x] Implement centralized AI configuration
- [x] Add AI performance monitoring
**Status Update**: ✅ **AI Service Manager fully implemented**
- Centralized AI service management with performance monitoring
- All services updated to use AI Service Manager
- Centralized AI configuration with timeout and retry settings
- Comprehensive AI performance monitoring with metrics tracking
- Service breakdown by AI type with success rates and response times
## ✅ **Phase 4 Status Update**
### **Completed Tasks**
1. **✅ AI Service Manager**
- Centralized AI service management with performance monitoring
- Comprehensive AI configuration with timeout and retry settings
- Service breakdown by AI type with success rates and response times
- Performance metrics tracking and health monitoring
- Centralized prompt and schema management
2. **✅ Service Integration**
- AI Engine Service updated to use AI Service Manager
- All AI calls routed through centralized manager
- Performance monitoring and metrics collection
- Error handling and fallback mechanisms
- Health check integration
3. **✅ Performance Monitoring**
- AI call performance metrics tracking
- Service breakdown by AI type
- Success rate monitoring
- Response time tracking
- Error rate monitoring
### **New Features Implemented**
- **Centralized AI Management**: Single point of control for all AI services
- **Performance Monitoring**: Real-time metrics for AI service performance
- **Service Breakdown**: Detailed metrics by AI service type
- **Configuration Management**: Centralized AI configuration settings
- **Health Monitoring**: Comprehensive health checks for AI services
### **Quality Criteria**
- [ ] AI response accuracy > 85%
- [ ] AI response time < 10 seconds
- [ ] AI error rate < 5%
- [ ] AI fallback mechanisms working
- [ ] AI prompts optimized for quality
## 🔧 **Implementation Steps**
### **Step 1: Environment Setup**
1. Verify Gemini API key configuration
2. Test Gemini provider functionality
3. Set up AI service monitoring
4. Configure error handling
### **Step 2: Core Integration**
1. Update AI Engine Service with real AI calls
2. Implement structured JSON responses
3. Add comprehensive error handling
4. Test AI functionality
### **Step 3: Service Updates**
1. Update Keyword Researcher with AI integration
2. Update Competitor Analyzer with AI integration
3. Update Website Analyzer with AI integration
4. Test all services with AI
### **Step 4: Optimization**
1. Optimize AI prompts for better results
2. Implement AI response caching
3. Add AI performance monitoring
4. Create AI fallback mechanisms
## 📈 **Expected Outcomes**
### **Immediate Benefits**
- Real AI-powered insights instead of hardcoded data
- Dynamic content recommendations
- Intelligent keyword analysis
- Strategic competitive intelligence
### **Long-term Benefits**
- Improved content strategy accuracy
- Better keyword targeting
- Enhanced competitive positioning
- Optimized content performance
---
**Status**: Ready for Implementation
**Priority**: High
**Estimated Duration**: 4 weeks
**Dependencies**: Gemini API key, existing AI infrastructure