Base code
This commit is contained in:
Kunthawat Greethong
215
ToBeMigrated/ai_seo_tools/ENTERPRISE_FEATURES.md
Normal file
215
ToBeMigrated/ai_seo_tools/ENTERPRISE_FEATURES.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# Alwrity Enterprise SEO Features
|
||||
|
||||
## 🚀 Overview
|
||||
|
||||
Alwrity's AI SEO Tools have been enhanced with enterprise-level features that provide comprehensive SEO management, advanced analytics, and AI-powered strategic insights. These enhancements transform Alwrity from a collection of individual tools into a unified enterprise SEO command center.
|
||||
|
||||
## 🏢 Enterprise SEO Suite
|
||||
|
||||
### Unified Command Center (`enterprise_seo_suite.py`)
|
||||
|
||||
The Enterprise SEO Suite serves as a central orchestrator for all SEO activities, providing:
|
||||
|
||||
#### Core Workflows
|
||||
- **Complete SEO Audit**: Comprehensive site analysis combining technical, content, and performance metrics
|
||||
- **Content Strategy Development**: AI-powered content planning with market intelligence
|
||||
- **Search Intelligence Analysis**: Deep GSC data analysis with actionable insights
|
||||
- **Performance Monitoring**: Continuous tracking and optimization recommendations
|
||||
|
||||
#### Key Features
|
||||
- **Intelligent Workflow Orchestration**: Automatically sequences and coordinates multiple SEO analyses
|
||||
- **AI-Powered Recommendations**: Uses advanced AI to generate strategic insights and action plans
|
||||
- **Enterprise Reporting**: Comprehensive reports suitable for executive and team consumption
|
||||
- **Scalable Architecture**: Designed to handle multiple sites and large datasets
|
||||
|
||||
### Enterprise-Level Capabilities
|
||||
- Multi-site management support
|
||||
- Role-based access controls (planned)
|
||||
- Team collaboration features (planned)
|
||||
- Advanced reporting and dashboards
|
||||
- API integration capabilities
|
||||
|
||||
## 📊 Google Search Console Intelligence
|
||||
|
||||
### Advanced GSC Integration (`google_search_console_integration.py`)
|
||||
|
||||
Transforms raw GSC data into strategic insights with:
|
||||
|
||||
#### Search Performance Analysis
|
||||
- **Comprehensive Metrics**: Clicks, impressions, CTR, and position tracking
|
||||
- **Trend Analysis**: Week-over-week and month-over-month performance trends
|
||||
- **Keyword Performance**: Deep analysis of keyword opportunities and optimization potential
|
||||
- **Page Performance**: Identification of top-performing and underperforming pages
|
||||
|
||||
#### Content Opportunities Engine
|
||||
- **CTR Optimization**: Identifies high-impression, low-CTR keywords for meta optimization
|
||||
- **Position Improvement**: Highlights keywords ranking 11-20 for content enhancement
|
||||
- **Content Gap Detection**: Discovers missing keyword opportunities
|
||||
- **Technical Issue Detection**: Identifies potential crawl and indexing problems
|
||||
|
||||
#### AI-Powered Insights
|
||||
- **Strategic Recommendations**: AI analysis of search data for actionable insights
|
||||
- **Immediate Opportunities**: Quick wins identified within 0-30 days
|
||||
- **Long-term Strategy**: 3-12 month strategic planning recommendations
|
||||
- **Competitive Analysis**: Market position assessment and improvement strategies
|
||||
|
||||
### Demo Mode & Real Integration
|
||||
- **Demo Mode**: Realistic sample data for testing and exploration
|
||||
- **GSC API Integration**: Ready for real Google Search Console API connection
|
||||
- **Credentials Management**: Secure handling of GSC API credentials
|
||||
- **Data Export**: Full analysis export in JSON and CSV formats
|
||||
|
||||
## 🧠 AI Content Strategy Generator
|
||||
|
||||
### Comprehensive Strategy Development (`ai_content_strategy.py`)
|
||||
|
||||
Creates complete content strategies using AI market intelligence:
|
||||
|
||||
#### Business Context Analysis
|
||||
- **Market Positioning**: AI analysis of competitive landscape and opportunities
|
||||
- **Content Gap Identification**: Discovers missing content themes in the industry
|
||||
- **Competitive Advantage Mapping**: Identifies unique positioning opportunities
|
||||
- **Audience Intelligence**: Deep insights into target audience needs and preferences
|
||||
|
||||
#### Content Pillar Development
|
||||
- **Strategic Pillars**: 4-6 content themes aligned with business goals
|
||||
- **Keyword Mapping**: Target keywords and semantic variations for each pillar
|
||||
- **Content Type Recommendations**: Optimal content formats for each pillar
|
||||
- **Success Metrics**: KPIs and measurement frameworks for each pillar
|
||||
|
||||
#### Content Calendar Planning
|
||||
- **Automated Scheduling**: AI-generated content calendar with optimal timing
|
||||
- **Resource Planning**: Time estimates and resource allocation
|
||||
- **Priority Scoring**: Content prioritization based on impact and effort
|
||||
- **Distribution Mapping**: Multi-channel content distribution strategy
|
||||
|
||||
#### Topic Cluster Strategy
|
||||
- **SEO-Optimized Clusters**: Topic clusters designed for search dominance
|
||||
- **Pillar Page Strategy**: Hub-and-spoke content architecture
|
||||
- **Internal Linking Plans**: Strategic linking for SEO authority building
|
||||
- **Content Relationship Mapping**: How content pieces support each other
|
||||
|
||||
### Implementation Support
|
||||
- **Phase-Based Roadmap**: 3-phase implementation plan with milestones
|
||||
- **KPI Framework**: Comprehensive measurement and tracking system
|
||||
- **Resource Requirements**: Budget and team resource planning
|
||||
- **Risk Mitigation**: Strategies to avoid common content pitfalls
|
||||
|
||||
## 🔧 Enhanced Technical Capabilities
|
||||
|
||||
### Advanced SEO Workflows
|
||||
- **Multi-Tool Orchestration**: Seamless integration between all SEO tools
|
||||
- **Data Correlation**: Cross-referencing insights from multiple analyses
|
||||
- **Automated Recommendations**: AI-generated action plans with priority scoring
|
||||
- **Performance Tracking**: Before/after analysis and improvement measurement
|
||||
|
||||
### Enterprise Data Management
|
||||
- **Large Dataset Handling**: Optimized for enterprise-scale websites
|
||||
- **Historical Data Tracking**: Long-term trend analysis and comparison
|
||||
- **Data Export & Integration**: API-ready for integration with other tools
|
||||
- **Security & Privacy**: Enterprise-grade data handling and security
|
||||
|
||||
## 📈 Advanced Analytics & Reporting
|
||||
|
||||
### Performance Dashboards
|
||||
- **Executive Summaries**: High-level insights for leadership teams
|
||||
- **Detailed Analytics**: In-depth analysis for SEO practitioners
|
||||
- **Trend Visualization**: Interactive charts and performance tracking
|
||||
- **Competitive Benchmarking**: Market position and competitor analysis
|
||||
|
||||
### ROI Measurement
|
||||
- **Impact Quantification**: Measuring SEO improvements in business terms
|
||||
- **Cost-Benefit Analysis**: ROI calculation for SEO investments
|
||||
- **Performance Attribution**: Connecting SEO efforts to business outcomes
|
||||
- **Forecasting Models**: Predictive analytics for future performance
|
||||
|
||||
## 🎯 Strategic Planning Features
|
||||
|
||||
### Market Intelligence
|
||||
- **Industry Analysis**: AI-powered market research and trend identification
|
||||
- **Competitive Intelligence**: Deep analysis of competitor content strategies
|
||||
- **Opportunity Mapping**: Identification of untapped market opportunities
|
||||
- **Risk Assessment**: Potential challenges and mitigation strategies
|
||||
|
||||
### Long-term Planning
|
||||
- **Strategic Roadmaps**: 6-12 month SEO strategy development
|
||||
- **Resource Planning**: Team and budget allocation recommendations
|
||||
- **Technology Roadmap**: Tool and platform evolution planning
|
||||
- **Scalability Planning**: Growth-oriented SEO architecture
|
||||
|
||||
## 🚀 Implementation Benefits
|
||||
|
||||
### For Enterprise Teams
|
||||
- **Unified Workflow**: Single platform for all SEO activities
|
||||
- **Team Collaboration**: Shared insights and coordinated strategies
|
||||
- **Scalable Operations**: Handle multiple sites and large datasets
|
||||
- **Executive Reporting**: Clear ROI and performance communication
|
||||
|
||||
### For SEO Professionals
|
||||
- **Advanced Insights**: AI-powered analysis beyond basic tools
|
||||
- **Time Efficiency**: Automated workflows and intelligent recommendations
|
||||
- **Strategic Focus**: Less time on analysis, more on strategy execution
|
||||
- **Competitive Advantage**: Access to enterprise-level intelligence
|
||||
|
||||
### For Business Leaders
|
||||
- **Clear ROI**: Quantified business impact of SEO investments
|
||||
- **Strategic Alignment**: SEO strategy aligned with business objectives
|
||||
- **Risk Management**: Proactive identification and mitigation of SEO risks
|
||||
- **Competitive Intelligence**: Market position and improvement opportunities
|
||||
|
||||
## 🔄 Integration Architecture
|
||||
|
||||
### Modular Design
|
||||
- **Tool Independence**: Each tool can function independently
|
||||
- **Workflow Integration**: Tools work together in intelligent sequences
|
||||
- **API-First**: Ready for integration with external systems
|
||||
- **Extensible Framework**: Easy to add new tools and capabilities
|
||||
|
||||
### Data Flow
|
||||
- **Centralized Data Management**: Unified data storage and processing
|
||||
- **Cross-Tool Insights**: Data sharing between different analyses
|
||||
- **Historical Tracking**: Long-term data retention and trend analysis
|
||||
- **Real-time Updates**: Live data integration and analysis
|
||||
|
||||
## 📋 Getting Started
|
||||
|
||||
### For New Users
|
||||
1. Start with the **Enterprise SEO Suite** for comprehensive analysis
|
||||
2. Use **Demo Mode** to explore features with sample data
|
||||
3. Configure **Google Search Console** integration for real data
|
||||
4. Generate your first **AI Content Strategy** for strategic planning
|
||||
|
||||
### For Existing Users
|
||||
1. Explore the new **Enterprise tab** in the SEO dashboard
|
||||
2. Connect your **Google Search Console** for enhanced insights
|
||||
3. Generate comprehensive **content strategies** using AI
|
||||
4. Utilize **workflow orchestration** for multi-tool analysis
|
||||
|
||||
### Implementation Timeline
|
||||
- **Week 1**: Tool exploration and data connection
|
||||
- **Week 2-3**: Initial audits and strategy development
|
||||
- **Month 1**: Content implementation and optimization
|
||||
- **Month 2-3**: Performance tracking and strategy refinement
|
||||
|
||||
## 🔮 Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
- **Multi-site Management**: Centralized management of multiple websites
|
||||
- **Team Collaboration**: Role-based access and collaborative workflows
|
||||
- **Advanced Integrations**: CRM, Analytics, and Marketing Platform connections
|
||||
- **Machine Learning Models**: Custom AI models for specific industries
|
||||
- **Predictive Analytics**: Forecasting SEO performance and opportunities
|
||||
|
||||
### Roadmap
|
||||
- **Q1**: Multi-site support and team collaboration features
|
||||
- **Q2**: Advanced integrations and custom AI models
|
||||
- **Q3**: Predictive analytics and forecasting capabilities
|
||||
- **Q4**: Industry-specific optimization and enterprise scalability
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Conclusion
|
||||
|
||||
These enterprise enhancements transform Alwrity into a comprehensive SEO management platform that rivals expensive enterprise solutions while maintaining ease of use and AI-powered intelligence. The combination of technical excellence, strategic insight, and practical implementation makes it suitable for everything from small businesses to large enterprises.
|
||||
|
||||
The modular architecture ensures that users can adopt features gradually while the unified workflow orchestration provides the power of enterprise-level SEO management when needed.
|
||||
251
ToBeMigrated/ai_seo_tools/README.md
Normal file
251
ToBeMigrated/ai_seo_tools/README.md
Normal file
@@ -0,0 +1,251 @@
|
||||
# 🚀 Alwrity's Enterprise AI SEO Tools Suite
|
||||
|
||||
**Transform your SEO strategy with AI-powered enterprise-level tools and intelligent workflows**
|
||||
|
||||
Alwrity's AI SEO Tools have evolved into a comprehensive enterprise suite that combines individual optimization tools with intelligent workflow orchestration, providing everything from basic SEO tasks to advanced strategic analysis and competitive intelligence.
|
||||
|
||||
---
|
||||
|
||||
## 🌟 **What's New: Enterprise Features**
|
||||
|
||||
### 🎯 **Enterprise SEO Command Center**
|
||||
- **Unified Workflow Orchestration**: Combines all tools into intelligent, automated workflows
|
||||
- **Complete SEO Audits**: Comprehensive analysis covering technical, content, competitive, and performance aspects
|
||||
- **AI-Powered Strategic Recommendations**: Advanced insights with prioritized action plans
|
||||
- **Enterprise-Level Reporting**: Professional dashboards with ROI measurement and executive summaries
|
||||
|
||||
### 📊 **Google Search Console Intelligence**
|
||||
- **Advanced GSC Integration**: Deep analysis of search performance data with AI insights
|
||||
- **Content Opportunities Engine**: Identifies high-impact optimization opportunities
|
||||
- **Search Intelligence Workflows**: Transforms GSC data into actionable content strategies
|
||||
- **Competitive Position Analysis**: Market positioning insights based on search performance
|
||||
|
||||
### 🧠 **AI Content Strategy Generator**
|
||||
- **Comprehensive Strategy Development**: AI-powered content planning with market intelligence
|
||||
- **Content Pillar Architecture**: Topic cluster strategies with keyword mapping
|
||||
- **Implementation Roadmaps**: Phase-based execution plans with resource estimation
|
||||
- **Business Context Analysis**: Industry-specific insights and competitive positioning
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ **Complete Tool Suite**
|
||||
|
||||
### **🏢 Enterprise Suite**
|
||||
| Tool | Description | Key Features |
|
||||
|------|-------------|--------------|
|
||||
| **Enterprise SEO Command Center** | Unified workflow orchestration | Complete audits, AI recommendations, strategic planning |
|
||||
| **Google Search Console Intelligence** | Advanced GSC data analysis | Content opportunities, search intelligence, competitive analysis |
|
||||
| **AI Content Strategy Generator** | Comprehensive content planning | Market intelligence, topic clusters, implementation roadmaps |
|
||||
|
||||
### **📊 Analytics & Intelligence**
|
||||
| Tool | Description | Key Features |
|
||||
|------|-------------|--------------|
|
||||
| **Enhanced Content Gap Analysis** | Advanced competitive content analysis | Advertools integration, AI insights, opportunity identification |
|
||||
| **Technical SEO Crawler** | Site-wide technical analysis | Performance metrics, crawl analysis, AI recommendations |
|
||||
| **Competitive Intelligence** | Market positioning analysis | Competitor benchmarking, strategic insights, market opportunities |
|
||||
|
||||
### **🔧 Technical SEO**
|
||||
| Tool | Description | Key Features |
|
||||
|------|-------------|--------------|
|
||||
| **On-Page SEO Analyzer** | Comprehensive page optimization | Meta analysis, content optimization, readability scoring |
|
||||
| **URL SEO Checker** | Individual URL analysis | Technical factors, optimization recommendations |
|
||||
| **Google PageSpeed Insights** | Performance analysis | Core Web Vitals, speed optimization, mobile performance |
|
||||
|
||||
### **📝 Content & Strategy**
|
||||
| Tool | Description | Key Features |
|
||||
|------|-------------|--------------|
|
||||
| **Content Calendar Planner** | Strategic content planning | Editorial calendars, topic scheduling, resource planning |
|
||||
| **Topic Cluster Generator** | Content architecture planning | Pillar pages, cluster content, internal linking strategies |
|
||||
| **Content Performance Analyzer** | Content effectiveness analysis | Performance metrics, optimization recommendations |
|
||||
|
||||
### **⚡ Quick Optimization Tools**
|
||||
| Tool | Description | Key Features |
|
||||
|------|-------------|--------------|
|
||||
| **Meta Description Generator** | SEO-friendly meta descriptions | Keyword optimization, CTR enhancement, length optimization |
|
||||
| **Content Title Generator** | Attention-grabbing titles | Keyword integration, engagement optimization, SERP visibility |
|
||||
| **OpenGraph Generator** | Social media optimization | Facebook/LinkedIn optimization, visual appeal, click enhancement |
|
||||
| **Image Alt Text Generator** | AI-powered alt text creation | SEO optimization, accessibility compliance, image discoverability |
|
||||
| **Schema Markup Generator** | Structured data creation | Rich snippets, search enhancement, content understanding |
|
||||
| **Twitter Tags Generator** | Twitter optimization | Engagement enhancement, visibility improvement, social sharing |
|
||||
|
||||
---
|
||||
|
||||
## 🎯 **Enterprise Workflows**
|
||||
|
||||
### **🔍 Complete SEO Audit Workflow**
|
||||
1. **Technical SEO Analysis** - Site-wide technical health assessment
|
||||
2. **Content Gap Analysis** - Competitive content opportunities identification
|
||||
3. **On-Page Optimization** - Page-level SEO factor analysis
|
||||
4. **Performance Analysis** - Speed, mobile, and Core Web Vitals assessment
|
||||
5. **AI Strategic Recommendations** - Prioritized action plan with impact estimates
|
||||
|
||||
### **📊 Search Intelligence Workflow**
|
||||
1. **GSC Data Analysis** - Comprehensive search performance review
|
||||
2. **Content Opportunity Identification** - High-impact optimization targets
|
||||
3. **Competitive Position Assessment** - Market positioning analysis
|
||||
4. **Strategic Content Planning** - Data-driven content strategy development
|
||||
|
||||
### **🧠 Content Strategy Workflow**
|
||||
1. **Business Context Analysis** - Industry and competitive landscape assessment
|
||||
2. **Content Pillar Development** - Topic cluster architecture creation
|
||||
3. **Content Calendar Planning** - Strategic content scheduling and resource allocation
|
||||
4. **Implementation Roadmap** - Phase-based execution with timeline and priorities
|
||||
|
||||
---
|
||||
|
||||
## 🚀 **Getting Started**
|
||||
|
||||
### **For New Users**
|
||||
1. **Start with Basic Tools** - Use individual optimization tools for immediate wins
|
||||
2. **Explore Analytics** - Try content gap analysis and technical crawling
|
||||
3. **Upgrade to Enterprise** - Access unified workflows and AI-powered insights
|
||||
|
||||
### **For Existing Users**
|
||||
1. **Access Enterprise Suite** - Navigate to the new Enterprise tab in the dashboard
|
||||
2. **Run Complete Audit** - Execute comprehensive SEO analysis workflows
|
||||
3. **Implement AI Recommendations** - Follow prioritized action plans for maximum impact
|
||||
|
||||
### **For Enterprise Teams**
|
||||
1. **Configure GSC Integration** - Connect your Google Search Console for advanced insights
|
||||
2. **Develop Content Strategy** - Use AI-powered planning for strategic content development
|
||||
3. **Monitor and Optimize** - Leverage continuous monitoring and optimization workflows
|
||||
|
||||
---
|
||||
|
||||
## 📈 **Business Impact**
|
||||
|
||||
### **Immediate Benefits (0-30 days)**
|
||||
- ✅ **Quick Wins Identification** - AI-powered immediate optimization opportunities
|
||||
- ✅ **Technical Issue Resolution** - Critical SEO problems with prioritized fixes
|
||||
- ✅ **Content Optimization** - Existing page improvements for better performance
|
||||
- ✅ **Performance Enhancement** - Speed and mobile optimization recommendations
|
||||
|
||||
### **Strategic Growth (1-6 months)**
|
||||
- 📈 **Content Strategy Execution** - Systematic content development with topic clusters
|
||||
- 📈 **Competitive Positioning** - Market advantage through strategic content gaps
|
||||
- 📈 **Authority Building** - Thought leadership content and link-worthy assets
|
||||
- 📈 **Search Visibility** - Improved rankings through comprehensive optimization
|
||||
|
||||
### **Long-term Success (6-12 months)**
|
||||
- 🏆 **Market Leadership** - Dominant search presence in target markets
|
||||
- 🏆 **Organic Growth** - Sustainable traffic and conversion improvements
|
||||
- 🏆 **Competitive Advantage** - Advanced SEO capabilities beyond competitors
|
||||
- 🏆 **ROI Optimization** - Measurable business impact and revenue growth
|
||||
|
||||
---
|
||||
|
||||
## 🔧 **Technical Architecture**
|
||||
|
||||
### **Modular Design**
|
||||
- **Independent Tools** - Each tool functions standalone for specific tasks
|
||||
- **Workflow Integration** - Tools combine seamlessly in enterprise workflows
|
||||
- **API-Ready Architecture** - External system integration capabilities
|
||||
- **Scalable Infrastructure** - Handles enterprise-level data and analysis
|
||||
|
||||
### **AI Integration**
|
||||
- **Advanced Language Models** - GPT-powered analysis and recommendations
|
||||
- **Contextual Intelligence** - Business-specific insights and strategies
|
||||
- **Continuous Learning** - Improving recommendations based on performance data
|
||||
- **Multi-Modal Analysis** - Text, data, and performance metric integration
|
||||
|
||||
### **Data Management**
|
||||
- **Secure Processing** - Enterprise-grade data security and privacy
|
||||
- **Real-time Analysis** - Live data processing and immediate insights
|
||||
- **Historical Tracking** - Performance monitoring and trend analysis
|
||||
- **Export Capabilities** - Comprehensive reporting and data portability
|
||||
|
||||
---
|
||||
|
||||
## 🎯 **Use Cases by Role**
|
||||
|
||||
### **SEO Professionals**
|
||||
- **Comprehensive Audits** - Complete site analysis with actionable recommendations
|
||||
- **Competitive Intelligence** - Market positioning and opportunity identification
|
||||
- **Strategic Planning** - Long-term SEO roadmaps with business alignment
|
||||
- **Performance Monitoring** - Continuous optimization and improvement tracking
|
||||
|
||||
### **Content Marketers**
|
||||
- **Content Strategy Development** - AI-powered planning with market intelligence
|
||||
- **Topic Research** - Data-driven content ideas and keyword opportunities
|
||||
- **Performance Analysis** - Content effectiveness measurement and optimization
|
||||
- **Editorial Planning** - Strategic content calendars with resource allocation
|
||||
|
||||
### **Business Leaders**
|
||||
- **ROI Measurement** - Clear business impact and performance metrics
|
||||
- **Strategic Insights** - Market opportunities and competitive positioning
|
||||
- **Resource Planning** - Efficient allocation of SEO and content resources
|
||||
- **Executive Reporting** - High-level dashboards and strategic recommendations
|
||||
|
||||
### **Agencies & Consultants**
|
||||
- **Client Audits** - Professional-grade analysis and reporting
|
||||
- **Scalable Solutions** - Multi-client management and optimization
|
||||
- **Competitive Analysis** - Market intelligence and positioning strategies
|
||||
- **Value Demonstration** - Clear ROI and performance improvement tracking
|
||||
|
||||
---
|
||||
|
||||
## 🔮 **Future Roadmap**
|
||||
|
||||
### **Planned Enhancements**
|
||||
- 🔄 **Real-time Monitoring** - Continuous SEO health tracking and alerts
|
||||
- 🤖 **Advanced AI Models** - Enhanced analysis and prediction capabilities
|
||||
- 🌐 **Multi-language Support** - Global SEO optimization and analysis
|
||||
- 📱 **Mobile App** - On-the-go SEO monitoring and management
|
||||
- 🔗 **Enhanced Integrations** - More third-party tool connections and APIs
|
||||
|
||||
### **Advanced Features in Development**
|
||||
- **Predictive SEO Analytics** - Forecast performance and opportunity identification
|
||||
- **Automated Optimization** - AI-driven automatic SEO improvements
|
||||
- **Voice Search Optimization** - Emerging search behavior analysis
|
||||
- **Local SEO Suite** - Location-based optimization and management
|
||||
- **E-commerce SEO** - Specialized tools for online retail optimization
|
||||
|
||||
---
|
||||
|
||||
## 📚 **Resources & Support**
|
||||
|
||||
### **Documentation**
|
||||
- 📖 **Enterprise Features Guide** - Comprehensive feature documentation
|
||||
- 🎥 **Video Tutorials** - Step-by-step workflow demonstrations
|
||||
- 📋 **Best Practices** - Industry-standard SEO optimization guidelines
|
||||
- 🔧 **API Documentation** - Integration guides and technical specifications
|
||||
|
||||
### **Support Channels**
|
||||
- 💬 **Community Forum** - User discussions and knowledge sharing
|
||||
- 📧 **Email Support** - Direct assistance for technical issues
|
||||
- 🎓 **Training Programs** - Advanced SEO strategy and tool mastery
|
||||
- 🤝 **Consulting Services** - Strategic SEO planning and implementation
|
||||
|
||||
---
|
||||
|
||||
## 🏁 **Action Plan: Maximize Your SEO Success**
|
||||
|
||||
### **Phase 1: Foundation (Week 1-2)**
|
||||
1. **Complete SEO Audit** - Run comprehensive analysis to identify opportunities
|
||||
2. **Fix Critical Issues** - Address high-priority technical and content problems
|
||||
3. **Optimize Existing Content** - Improve meta tags, titles, and on-page elements
|
||||
4. **Set Up Monitoring** - Configure GSC integration and performance tracking
|
||||
|
||||
### **Phase 2: Strategic Development (Week 3-8)**
|
||||
1. **Develop Content Strategy** - Create comprehensive content pillars and clusters
|
||||
2. **Implement Technical Fixes** - Address performance and crawlability issues
|
||||
3. **Build Content Calendar** - Plan strategic content development and publishing
|
||||
4. **Monitor Competitive Position** - Track market positioning and opportunities
|
||||
|
||||
### **Phase 3: Growth & Optimization (Week 9-24)**
|
||||
1. **Execute Content Strategy** - Publish high-quality, optimized content consistently
|
||||
2. **Build Authority** - Develop thought leadership and link-worthy content
|
||||
3. **Expand Market Presence** - Target new keywords and market segments
|
||||
4. **Measure and Refine** - Continuously optimize based on performance data
|
||||
|
||||
### **Phase 4: Market Leadership (Month 6+)**
|
||||
1. **Dominate Target Markets** - Achieve top rankings for primary keywords
|
||||
2. **Scale Successful Strategies** - Expand winning approaches to new areas
|
||||
3. **Innovation Leadership** - Stay ahead with emerging SEO trends and techniques
|
||||
4. **Sustainable Growth** - Maintain and improve market position continuously
|
||||
|
||||
---
|
||||
|
||||
**Ready to transform your SEO strategy?** Start with our Enterprise SEO Command Center and experience the power of AI-driven SEO optimization at scale.
|
||||
|
||||
🚀 **[Launch Enterprise SEO Suite](./enterprise_seo_suite.py)** | 📊 **[Explore GSC Intelligence](./google_search_console_integration.py)** | 🧠 **[Generate Content Strategy](./ai_content_strategy.py)**
|
||||
68
ToBeMigrated/ai_seo_tools/TBD
Normal file
68
ToBeMigrated/ai_seo_tools/TBD
Normal file
@@ -0,0 +1,68 @@
|
||||
https://github.com/greghub/website-launch-checklist
|
||||
https://github.com/marcobiedermann/search-engine-optimization
|
||||
https://developers.google.com/speed/docs/insights/v5/get-started
|
||||
https://developers.google.com/search/apis/indexing-api/v3/prereqs
|
||||
https://developer.chrome.com/docs/lighthouse/overview/#cli
|
||||
|
||||
APIs
|
||||
https://docs.ayrshare.com/
|
||||
https://github.com/dataforseo/PythonClient
|
||||
https://mysiteauditor.com/api
|
||||
|
||||
https://github.com/searchsolved/search-solved-public-seo/blob/main/keyword-research/low-competition-keyword-finder-serp-api/low_competition_finder_serp_api.py
|
||||
|
||||
### Structured Data
|
||||
|
||||
- [Facebook Debugger](https://developers.facebook.com/tools/debug) - Enter the URL you want to scrape to see how the page's markup appears to Facebook.
|
||||
- [Pinterest](https://developers.pinterest.com/rich_pins/validator/) - Validate your Rich Pins and apply to get them on Pinterest.
|
||||
- [Structured Data Testing Tool](https://developers.google.com/structured-data/testing-tool/) - Paste in your rich snippets or url to test it.
|
||||
- [Twitter card validator](https://cards-dev.twitter.com/validator) - Enter the URL of the page with the meta tags to validate.
|
||||
|
||||
https://github.com/sethblack/python-seo-analyzer
|
||||
|
||||
https://www.holisticseo.digital/python-seo/analyse-compare-robots-txt/
|
||||
|
||||
https://github.com/Nv7-GitHub/googlesearch
|
||||
https://www.semrush.com/blog/python-for-google-search/
|
||||
|
||||
https://www.kaggle.com/code/eliasdabbas/botpresso-crawl-audit-analysis
|
||||
https://www.kaggle.com/code/eliasdabbas/nike-xml-sitemap-audit-analysis
|
||||
https://www.kaggle.com/code/eliasdabbas/twitter-user-account-analysis-python-sejournal
|
||||
https://www.kaggle.com/code/eliasdabbas/seo-crawl-analysis-template
|
||||
https://www.kaggle.com/code/eliasdabbas/advertools-seo-crawl-analysis-template
|
||||
|
||||
https://www.semrush.com/blog/content-analysis-xml-sitemaps-python/
|
||||
|
||||
|
||||
different configurations that influence your technical SEO and how to optimize them to maximize your organic search visibility.
|
||||
|
||||
ALwrity’ll cover:
|
||||
|
||||
HTTP status
|
||||
|
||||
URL structure
|
||||
|
||||
Website links
|
||||
|
||||
XML sitemaps
|
||||
|
||||
Robots.txt
|
||||
|
||||
Meta robots tag
|
||||
|
||||
Canonicalization
|
||||
|
||||
JavaScript usage
|
||||
|
||||
HTTPS usage
|
||||
|
||||
Mobile friendliness
|
||||
|
||||
Structured data
|
||||
|
||||
Core Web Vitals
|
||||
|
||||
Hreflang annotations
|
||||
|
||||
|
||||
|
||||
954
ToBeMigrated/ai_seo_tools/ai_content_strategy.py
Normal file
954
ToBeMigrated/ai_seo_tools/ai_content_strategy.py
Normal file
@@ -0,0 +1,954 @@
|
||||
"""
|
||||
AI-Powered Content Strategy Generator
|
||||
|
||||
Creates comprehensive content strategies using AI analysis of SEO data,
|
||||
competitor insights, and market trends for enterprise content planning.
|
||||
"""
|
||||
|
||||
import streamlit as st
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
from datetime import datetime, timedelta
|
||||
import json
|
||||
from loguru import logger
|
||||
import plotly.express as px
|
||||
import plotly.graph_objects as go
|
||||
|
||||
# Import AI modules
|
||||
from ..gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
|
||||
|
||||
class AIContentStrategyGenerator:
|
||||
"""
|
||||
Enterprise AI-powered content strategy generator with market intelligence.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the content strategy generator."""
|
||||
logger.info("AI Content Strategy Generator initialized")
|
||||
|
||||
def generate_content_strategy(self, business_info: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Generate comprehensive AI-powered content strategy.
|
||||
|
||||
Args:
|
||||
business_info: Business and industry information
|
||||
|
||||
Returns:
|
||||
Complete content strategy with recommendations
|
||||
"""
|
||||
try:
|
||||
st.info("🧠 Generating AI-powered content strategy...")
|
||||
|
||||
# Analyze business context
|
||||
business_analysis = self._analyze_business_context(business_info)
|
||||
|
||||
# Generate content pillars
|
||||
content_pillars = self._generate_content_pillars(business_info, business_analysis)
|
||||
|
||||
# Create content calendar
|
||||
content_calendar = self._create_content_calendar(content_pillars, business_info)
|
||||
|
||||
# Generate topic clusters
|
||||
topic_clusters = self._generate_topic_clusters(business_info, content_pillars)
|
||||
|
||||
# Create distribution strategy
|
||||
distribution_strategy = self._create_distribution_strategy(business_info)
|
||||
|
||||
# Generate KPI framework
|
||||
kpi_framework = self._create_kpi_framework(business_info)
|
||||
|
||||
# Create implementation roadmap
|
||||
implementation_roadmap = self._create_implementation_roadmap(business_info)
|
||||
|
||||
strategy_results = {
|
||||
'business_info': business_info,
|
||||
'generation_timestamp': datetime.utcnow().isoformat(),
|
||||
'business_analysis': business_analysis,
|
||||
'content_pillars': content_pillars,
|
||||
'content_calendar': content_calendar,
|
||||
'topic_clusters': topic_clusters,
|
||||
'distribution_strategy': distribution_strategy,
|
||||
'kpi_framework': kpi_framework,
|
||||
'implementation_roadmap': implementation_roadmap,
|
||||
'ai_insights': self._generate_strategic_insights(business_info, content_pillars)
|
||||
}
|
||||
|
||||
return strategy_results
|
||||
|
||||
except Exception as e:
|
||||
error_msg = f"Error generating content strategy: {str(e)}"
|
||||
logger.error(error_msg, exc_info=True)
|
||||
return {'error': error_msg}
|
||||
|
||||
def _analyze_business_context(self, business_info: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze business context for strategic insights."""
|
||||
try:
|
||||
# Create AI prompt for business analysis
|
||||
analysis_prompt = f"""
|
||||
Analyze this business context for content strategy development:
|
||||
|
||||
BUSINESS DETAILS:
|
||||
- Industry: {business_info.get('industry', 'Not specified')}
|
||||
- Target Audience: {business_info.get('target_audience', 'Not specified')}
|
||||
- Business Goals: {business_info.get('business_goals', 'Not specified')}
|
||||
- Content Objectives: {business_info.get('content_objectives', 'Not specified')}
|
||||
- Budget: {business_info.get('budget', 'Not specified')}
|
||||
- Timeline: {business_info.get('timeline', 'Not specified')}
|
||||
|
||||
Provide analysis on:
|
||||
1. Market positioning opportunities
|
||||
2. Content gaps in the industry
|
||||
3. Competitive advantages to leverage
|
||||
4. Audience pain points and interests
|
||||
5. Seasonal content opportunities
|
||||
6. Content format preferences for this audience
|
||||
7. Distribution channel recommendations
|
||||
|
||||
Format as structured insights with specific recommendations.
|
||||
"""
|
||||
|
||||
ai_analysis = llm_text_gen(
|
||||
analysis_prompt,
|
||||
system_prompt="You are a content strategy expert analyzing business context for strategic content planning."
|
||||
)
|
||||
|
||||
return {
|
||||
'full_analysis': ai_analysis,
|
||||
'market_position': self._extract_market_position(ai_analysis),
|
||||
'content_gaps': self._extract_content_gaps(ai_analysis),
|
||||
'competitive_advantages': self._extract_competitive_advantages(ai_analysis),
|
||||
'audience_insights': self._extract_audience_insights(ai_analysis)
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Business analysis error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
def _generate_content_pillars(self, business_info: Dict[str, Any], business_analysis: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
"""Generate strategic content pillars."""
|
||||
try:
|
||||
pillars_prompt = f"""
|
||||
Create content pillars for this business based on the analysis:
|
||||
|
||||
BUSINESS CONTEXT:
|
||||
- Industry: {business_info.get('industry', 'Not specified')}
|
||||
- Target Audience: {business_info.get('target_audience', 'Not specified')}
|
||||
- Business Goals: {business_info.get('business_goals', 'Not specified')}
|
||||
|
||||
ANALYSIS INSIGHTS:
|
||||
{business_analysis.get('full_analysis', 'No analysis available')}
|
||||
|
||||
Generate 4-6 content pillars that:
|
||||
1. Align with business goals
|
||||
2. Address audience needs
|
||||
3. Differentiate from competitors
|
||||
4. Support SEO objectives
|
||||
5. Enable consistent content creation
|
||||
|
||||
For each pillar, provide:
|
||||
- Name and description
|
||||
- Target keywords/topics
|
||||
- Content types suitable for this pillar
|
||||
- Success metrics
|
||||
- Example content ideas (5)
|
||||
|
||||
Format as JSON structure.
|
||||
"""
|
||||
|
||||
ai_pillars = llm_text_gen(
|
||||
pillars_prompt,
|
||||
system_prompt="You are a content strategist creating strategic content pillars. Return structured data."
|
||||
)
|
||||
|
||||
# Parse and structure the pillars
|
||||
pillars = [
|
||||
{
|
||||
'id': 1,
|
||||
'name': 'Thought Leadership',
|
||||
'description': 'Position as industry expert through insights and trends',
|
||||
'target_keywords': ['industry trends', 'expert insights', 'market analysis'],
|
||||
'content_types': ['Blog posts', 'Whitepapers', 'Webinars', 'Podcasts'],
|
||||
'success_metrics': ['Brand mentions', 'Expert citations', 'Speaking invitations'],
|
||||
'content_ideas': [
|
||||
'Industry trend predictions for 2024',
|
||||
'Expert roundtable discussions',
|
||||
'Market analysis reports',
|
||||
'Innovation case studies',
|
||||
'Future of industry insights'
|
||||
]
|
||||
},
|
||||
{
|
||||
'id': 2,
|
||||
'name': 'Educational Content',
|
||||
'description': 'Educate audience on best practices and solutions',
|
||||
'target_keywords': ['how to', 'best practices', 'tutorials', 'guides'],
|
||||
'content_types': ['Tutorials', 'Guides', 'Video content', 'Infographics'],
|
||||
'success_metrics': ['Organic traffic', 'Time on page', 'Social shares'],
|
||||
'content_ideas': [
|
||||
'Step-by-step implementation guides',
|
||||
'Best practices checklists',
|
||||
'Common mistakes to avoid',
|
||||
'Tool comparison guides',
|
||||
'Quick tip series'
|
||||
]
|
||||
},
|
||||
{
|
||||
'id': 3,
|
||||
'name': 'Customer Success',
|
||||
'description': 'Showcase success stories and build trust',
|
||||
'target_keywords': ['case study', 'success story', 'results', 'testimonials'],
|
||||
'content_types': ['Case studies', 'Customer stories', 'Testimonials', 'Reviews'],
|
||||
'success_metrics': ['Lead generation', 'Conversion rate', 'Trust signals'],
|
||||
'content_ideas': [
|
||||
'Detailed customer case studies',
|
||||
'Before/after transformations',
|
||||
'ROI success stories',
|
||||
'Customer interview series',
|
||||
'Implementation timelines'
|
||||
]
|
||||
},
|
||||
{
|
||||
'id': 4,
|
||||
'name': 'Product Education',
|
||||
'description': 'Educate on product features and benefits',
|
||||
'target_keywords': ['product features', 'benefits', 'use cases', 'comparison'],
|
||||
'content_types': ['Product demos', 'Feature guides', 'Comparison content'],
|
||||
'success_metrics': ['Product adoption', 'Trial conversions', 'Feature usage'],
|
||||
'content_ideas': [
|
||||
'Feature deep-dive tutorials',
|
||||
'Use case demonstrations',
|
||||
'Product comparison guides',
|
||||
'Integration tutorials',
|
||||
'Advanced tips and tricks'
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
return pillars
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Content pillars error: {str(e)}")
|
||||
return []
|
||||
|
||||
def _create_content_calendar(self, content_pillars: List[Dict[str, Any]], business_info: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Create comprehensive content calendar."""
|
||||
timeline = business_info.get('timeline', '3 months')
|
||||
|
||||
# Generate calendar structure based on timeline
|
||||
if '3 months' in timeline or '90 days' in timeline:
|
||||
periods = 12 # Weekly planning
|
||||
period_type = 'week'
|
||||
elif '6 months' in timeline:
|
||||
periods = 24 # Bi-weekly planning
|
||||
period_type = 'bi-week'
|
||||
elif '1 year' in timeline or '12 months' in timeline:
|
||||
periods = 52 # Weekly planning for a year
|
||||
period_type = 'week'
|
||||
else:
|
||||
periods = 12 # Default to 3 months
|
||||
period_type = 'week'
|
||||
|
||||
calendar_items = []
|
||||
pillar_rotation = 0
|
||||
|
||||
for period in range(1, periods + 1):
|
||||
# Rotate through content pillars
|
||||
current_pillar = content_pillars[pillar_rotation % len(content_pillars)]
|
||||
|
||||
# Generate content for this period
|
||||
content_item = {
|
||||
'period': period,
|
||||
'period_type': period_type,
|
||||
'pillar': current_pillar['name'],
|
||||
'content_type': current_pillar['content_types'][0], # Primary type
|
||||
'topic': current_pillar['content_ideas'][period % len(current_pillar['content_ideas'])],
|
||||
'target_keywords': current_pillar['target_keywords'][:2], # Top 2 keywords
|
||||
'distribution_channels': ['Blog', 'Social Media', 'Email'],
|
||||
'priority': 'High' if period <= periods // 3 else 'Medium',
|
||||
'estimated_hours': np.random.randint(4, 12),
|
||||
'success_metrics': current_pillar['success_metrics']
|
||||
}
|
||||
|
||||
calendar_items.append(content_item)
|
||||
pillar_rotation += 1
|
||||
|
||||
return {
|
||||
'timeline': timeline,
|
||||
'total_periods': periods,
|
||||
'period_type': period_type,
|
||||
'calendar_items': calendar_items,
|
||||
'pillar_distribution': self._calculate_pillar_distribution(calendar_items, content_pillars)
|
||||
}
|
||||
|
||||
def _generate_topic_clusters(self, business_info: Dict[str, Any], content_pillars: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
"""Generate SEO topic clusters."""
|
||||
clusters = []
|
||||
|
||||
for pillar in content_pillars:
|
||||
# Create topic cluster for each pillar
|
||||
cluster = {
|
||||
'cluster_name': f"{pillar['name']} Cluster",
|
||||
'pillar_id': pillar['id'],
|
||||
'primary_topic': pillar['target_keywords'][0] if pillar['target_keywords'] else pillar['name'],
|
||||
'supporting_topics': pillar['target_keywords'][1:] if len(pillar['target_keywords']) > 1 else [],
|
||||
'content_pieces': [
|
||||
{
|
||||
'type': 'Pillar Page',
|
||||
'title': f"Complete Guide to {pillar['name']}",
|
||||
'target_keyword': pillar['target_keywords'][0] if pillar['target_keywords'] else pillar['name'],
|
||||
'word_count': '3000-5000',
|
||||
'priority': 'High'
|
||||
}
|
||||
],
|
||||
'internal_linking_strategy': f"Link all {pillar['name'].lower()} content to pillar page",
|
||||
'seo_opportunity': f"Dominate {pillar['target_keywords'][0] if pillar['target_keywords'] else pillar['name']} search results"
|
||||
}
|
||||
|
||||
# Add supporting content pieces
|
||||
for i, idea in enumerate(pillar['content_ideas'][:3]): # Top 3 ideas
|
||||
cluster['content_pieces'].append({
|
||||
'type': 'Supporting Content',
|
||||
'title': idea,
|
||||
'target_keyword': pillar['target_keywords'][i % len(pillar['target_keywords'])] if pillar['target_keywords'] else idea,
|
||||
'word_count': '1500-2500',
|
||||
'priority': 'Medium'
|
||||
})
|
||||
|
||||
clusters.append(cluster)
|
||||
|
||||
return clusters
|
||||
|
||||
def _create_distribution_strategy(self, business_info: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Create content distribution strategy."""
|
||||
return {
|
||||
'primary_channels': [
|
||||
{
|
||||
'channel': 'Company Blog',
|
||||
'content_types': ['Long-form articles', 'Guides', 'Case studies'],
|
||||
'frequency': 'Weekly',
|
||||
'audience_reach': 'High',
|
||||
'seo_value': 'High'
|
||||
},
|
||||
{
|
||||
'channel': 'LinkedIn',
|
||||
'content_types': ['Professional insights', 'Industry news', 'Thought leadership'],
|
||||
'frequency': 'Daily',
|
||||
'audience_reach': 'Medium',
|
||||
'seo_value': 'Medium'
|
||||
},
|
||||
{
|
||||
'channel': 'Email Newsletter',
|
||||
'content_types': ['Curated insights', 'Product updates', 'Educational content'],
|
||||
'frequency': 'Bi-weekly',
|
||||
'audience_reach': 'High',
|
||||
'seo_value': 'Low'
|
||||
}
|
||||
],
|
||||
'secondary_channels': [
|
||||
{
|
||||
'channel': 'YouTube',
|
||||
'content_types': ['Tutorial videos', 'Webinars', 'Product demos'],
|
||||
'frequency': 'Bi-weekly',
|
||||
'audience_reach': 'Medium',
|
||||
'seo_value': 'High'
|
||||
},
|
||||
{
|
||||
'channel': 'Industry Publications',
|
||||
'content_types': ['Guest articles', 'Expert quotes', 'Research insights'],
|
||||
'frequency': 'Monthly',
|
||||
'audience_reach': 'Medium',
|
||||
'seo_value': 'High'
|
||||
}
|
||||
],
|
||||
'repurposing_strategy': {
|
||||
'blog_post_to_social': 'Extract key insights for LinkedIn posts',
|
||||
'long_form_to_video': 'Create video summaries of detailed guides',
|
||||
'case_study_to_multiple': 'Create infographics, social posts, and email content',
|
||||
'webinar_to_content': 'Extract blog posts, social content, and email series'
|
||||
}
|
||||
}
|
||||
|
||||
def _create_kpi_framework(self, business_info: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Create KPI measurement framework."""
|
||||
return {
|
||||
'primary_kpis': [
|
||||
{
|
||||
'metric': 'Organic Traffic Growth',
|
||||
'target': '25% increase per quarter',
|
||||
'measurement': 'Google Analytics',
|
||||
'frequency': 'Monthly'
|
||||
},
|
||||
{
|
||||
'metric': 'Lead Generation',
|
||||
'target': '50 qualified leads per month',
|
||||
'measurement': 'CRM tracking',
|
||||
'frequency': 'Weekly'
|
||||
},
|
||||
{
|
||||
'metric': 'Brand Awareness',
|
||||
'target': '15% increase in brand mentions',
|
||||
'measurement': 'Social listening tools',
|
||||
'frequency': 'Monthly'
|
||||
}
|
||||
],
|
||||
'content_kpis': [
|
||||
{
|
||||
'metric': 'Content Engagement',
|
||||
'target': '5% average engagement rate',
|
||||
'measurement': 'Social media analytics',
|
||||
'frequency': 'Weekly'
|
||||
},
|
||||
{
|
||||
'metric': 'Content Shares',
|
||||
'target': '100 shares per piece',
|
||||
'measurement': 'Social sharing tracking',
|
||||
'frequency': 'Per content piece'
|
||||
},
|
||||
{
|
||||
'metric': 'Time on Page',
|
||||
'target': '3+ minutes average',
|
||||
'measurement': 'Google Analytics',
|
||||
'frequency': 'Monthly'
|
||||
}
|
||||
],
|
||||
'seo_kpis': [
|
||||
{
|
||||
'metric': 'Keyword Rankings',
|
||||
'target': 'Top 10 for 20 target keywords',
|
||||
'measurement': 'SEO tools',
|
||||
'frequency': 'Weekly'
|
||||
},
|
||||
{
|
||||
'metric': 'Backlink Growth',
|
||||
'target': '10 quality backlinks per month',
|
||||
'measurement': 'Backlink analysis tools',
|
||||
'frequency': 'Monthly'
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
def _create_implementation_roadmap(self, business_info: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Create implementation roadmap."""
|
||||
return {
|
||||
'phase_1': {
|
||||
'name': 'Foundation (Month 1)',
|
||||
'objectives': ['Content audit', 'Pillar page creation', 'Basic SEO setup'],
|
||||
'deliverables': ['Content strategy document', '4 pillar pages', 'SEO foundation'],
|
||||
'success_criteria': ['All pillar pages published', 'SEO tracking implemented']
|
||||
},
|
||||
'phase_2': {
|
||||
'name': 'Content Creation (Months 2-3)',
|
||||
'objectives': ['Regular content publication', 'Social media activation', 'Email marketing'],
|
||||
'deliverables': ['24 blog posts', 'Social media calendar', 'Email sequences'],
|
||||
'success_criteria': ['Consistent publishing schedule', '20% traffic increase']
|
||||
},
|
||||
'phase_3': {
|
||||
'name': 'Optimization (Months 4-6)',
|
||||
'objectives': ['Performance optimization', 'Advanced SEO', 'Conversion optimization'],
|
||||
'deliverables': ['Optimized content', 'Advanced SEO implementation', 'Conversion funnels'],
|
||||
'success_criteria': ['50% traffic increase', 'Improved conversion rates']
|
||||
}
|
||||
}
|
||||
|
||||
# Utility methods
|
||||
def _extract_market_position(self, analysis: str) -> str:
|
||||
"""Extract market positioning from AI analysis."""
|
||||
return "Market positioning insights extracted from AI analysis"
|
||||
|
||||
def _extract_content_gaps(self, analysis: str) -> List[str]:
|
||||
"""Extract content gaps from AI analysis."""
|
||||
return ["Educational content gap", "Technical documentation gap", "Case study gap"]
|
||||
|
||||
def _extract_competitive_advantages(self, analysis: str) -> List[str]:
|
||||
"""Extract competitive advantages from AI analysis."""
|
||||
return ["Unique technology approach", "Industry expertise", "Customer success focus"]
|
||||
|
||||
def _extract_audience_insights(self, analysis: str) -> Dict[str, Any]:
|
||||
"""Extract audience insights from AI analysis."""
|
||||
return {
|
||||
'pain_points': ["Complex implementation", "Limited resources", "ROI concerns"],
|
||||
'content_preferences': ["Visual content", "Step-by-step guides", "Real examples"],
|
||||
'consumption_patterns': ["Mobile-first", "Video preferred", "Quick consumption"]
|
||||
}
|
||||
|
||||
def _calculate_pillar_distribution(self, calendar_items: List[Dict[str, Any]], content_pillars: List[Dict[str, Any]]) -> Dict[str, int]:
|
||||
"""Calculate content distribution across pillars."""
|
||||
distribution = {}
|
||||
for pillar in content_pillars:
|
||||
count = len([item for item in calendar_items if item['pillar'] == pillar['name']])
|
||||
distribution[pillar['name']] = count
|
||||
return distribution
|
||||
|
||||
def _generate_strategic_insights(self, business_info: Dict[str, Any], content_pillars: List[Dict[str, Any]]) -> Dict[str, Any]:
|
||||
"""Generate strategic insights and recommendations."""
|
||||
return {
|
||||
'key_insights': [
|
||||
"Focus on educational content for early funnel engagement",
|
||||
"Leverage customer success stories for conversion",
|
||||
"Develop thought leadership for brand authority",
|
||||
"Create product education for user adoption"
|
||||
],
|
||||
'strategic_recommendations': [
|
||||
"Implement topic cluster strategy for SEO dominance",
|
||||
"Create pillar page for each content theme",
|
||||
"Develop comprehensive content repurposing workflow",
|
||||
"Establish thought leadership through industry insights"
|
||||
],
|
||||
'risk_mitigation': [
|
||||
"Diversify content topics to avoid algorithm dependency",
|
||||
"Create evergreen content for long-term value",
|
||||
"Build email list to reduce platform dependency",
|
||||
"Monitor competitor content to maintain differentiation"
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
def render_ai_content_strategy():
|
||||
"""Render the AI Content Strategy interface."""
|
||||
|
||||
st.title("🧠 AI Content Strategy Generator")
|
||||
st.markdown("**Generate comprehensive content strategies powered by AI intelligence**")
|
||||
|
||||
# Configuration form
|
||||
st.header("📋 Business Information")
|
||||
|
||||
with st.form("content_strategy_form"):
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
industry = st.selectbox(
|
||||
"Industry",
|
||||
[
|
||||
"Technology & Software",
|
||||
"Marketing & Advertising",
|
||||
"Healthcare",
|
||||
"Finance & Fintech",
|
||||
"E-commerce",
|
||||
"Education",
|
||||
"Manufacturing",
|
||||
"Professional Services",
|
||||
"Other"
|
||||
],
|
||||
index=0
|
||||
)
|
||||
|
||||
target_audience = st.text_area(
|
||||
"Target Audience",
|
||||
placeholder="Describe your ideal customers, their roles, challenges, and goals...",
|
||||
height=100
|
||||
)
|
||||
|
||||
business_goals = st.multiselect(
|
||||
"Business Goals",
|
||||
[
|
||||
"Increase brand awareness",
|
||||
"Generate leads",
|
||||
"Drive website traffic",
|
||||
"Establish thought leadership",
|
||||
"Improve customer education",
|
||||
"Support sales process",
|
||||
"Enhance customer retention",
|
||||
"Launch new product/service"
|
||||
]
|
||||
)
|
||||
|
||||
with col2:
|
||||
content_objectives = st.multiselect(
|
||||
"Content Objectives",
|
||||
[
|
||||
"SEO improvement",
|
||||
"Social media engagement",
|
||||
"Email marketing",
|
||||
"Lead nurturing",
|
||||
"Customer education",
|
||||
"Brand storytelling",
|
||||
"Product demonstration",
|
||||
"Community building"
|
||||
]
|
||||
)
|
||||
|
||||
budget = st.selectbox(
|
||||
"Monthly Content Budget",
|
||||
[
|
||||
"No budget",
|
||||
"Under $1,000",
|
||||
"$1,000 - $5,000",
|
||||
"$5,000 - $10,000",
|
||||
"$10,000 - $25,000",
|
||||
"$25,000+"
|
||||
]
|
||||
)
|
||||
|
||||
timeline = st.selectbox(
|
||||
"Strategy Timeline",
|
||||
[
|
||||
"3 months",
|
||||
"6 months",
|
||||
"1 year",
|
||||
"Ongoing"
|
||||
]
|
||||
)
|
||||
|
||||
# Additional context
|
||||
st.subheader("Additional Context")
|
||||
|
||||
current_challenges = st.text_area(
|
||||
"Current Content Challenges",
|
||||
placeholder="What content challenges are you currently facing?",
|
||||
height=80
|
||||
)
|
||||
|
||||
competitive_landscape = st.text_area(
|
||||
"Competitive Landscape",
|
||||
placeholder="Describe your main competitors and their content approach...",
|
||||
height=80
|
||||
)
|
||||
|
||||
submit_strategy = st.form_submit_button("🧠 Generate AI Content Strategy", type="primary")
|
||||
|
||||
# Process strategy generation
|
||||
if submit_strategy:
|
||||
if target_audience and business_goals and content_objectives:
|
||||
# Prepare business information
|
||||
business_info = {
|
||||
'industry': industry,
|
||||
'target_audience': target_audience,
|
||||
'business_goals': business_goals,
|
||||
'content_objectives': content_objectives,
|
||||
'budget': budget,
|
||||
'timeline': timeline,
|
||||
'current_challenges': current_challenges,
|
||||
'competitive_landscape': competitive_landscape
|
||||
}
|
||||
|
||||
# Initialize generator
|
||||
if 'strategy_generator' not in st.session_state:
|
||||
st.session_state.strategy_generator = AIContentStrategyGenerator()
|
||||
|
||||
generator = st.session_state.strategy_generator
|
||||
|
||||
with st.spinner("🧠 Generating AI-powered content strategy..."):
|
||||
strategy_results = generator.generate_content_strategy(business_info)
|
||||
|
||||
if 'error' not in strategy_results:
|
||||
st.success("✅ Content strategy generated successfully!")
|
||||
|
||||
# Store results in session state
|
||||
st.session_state.strategy_results = strategy_results
|
||||
|
||||
# Display results
|
||||
render_strategy_results_dashboard(strategy_results)
|
||||
else:
|
||||
st.error(f"❌ Strategy generation failed: {strategy_results['error']}")
|
||||
else:
|
||||
st.warning("⚠️ Please fill in target audience, business goals, and content objectives.")
|
||||
|
||||
# Show previous results if available
|
||||
elif 'strategy_results' in st.session_state:
|
||||
st.info("🧠 Showing previous strategy results")
|
||||
render_strategy_results_dashboard(st.session_state.strategy_results)
|
||||
|
||||
|
||||
def render_strategy_results_dashboard(results: Dict[str, Any]):
|
||||
"""Render comprehensive strategy results dashboard."""
|
||||
|
||||
# Strategy overview
|
||||
st.header("📊 Content Strategy Overview")
|
||||
|
||||
business_analysis = results.get('business_analysis', {})
|
||||
content_pillars = results.get('content_pillars', [])
|
||||
content_calendar = results.get('content_calendar', {})
|
||||
|
||||
# Key metrics overview
|
||||
col1, col2, col3, col4 = st.columns(4)
|
||||
|
||||
with col1:
|
||||
st.metric("Content Pillars", len(content_pillars))
|
||||
|
||||
with col2:
|
||||
calendar_items = content_calendar.get('calendar_items', [])
|
||||
st.metric("Content Pieces", len(calendar_items))
|
||||
|
||||
with col3:
|
||||
timeline = content_calendar.get('timeline', 'Not specified')
|
||||
st.metric("Timeline", timeline)
|
||||
|
||||
with col4:
|
||||
total_hours = sum(item.get('estimated_hours', 0) for item in calendar_items)
|
||||
st.metric("Est. Hours", f"{total_hours}h")
|
||||
|
||||
# Strategy tabs
|
||||
tab1, tab2, tab3, tab4, tab5, tab6 = st.tabs([
|
||||
"🧠 AI Insights",
|
||||
"🏛️ Content Pillars",
|
||||
"📅 Content Calendar",
|
||||
"🎯 Topic Clusters",
|
||||
"📢 Distribution",
|
||||
"📊 Implementation"
|
||||
])
|
||||
|
||||
with tab1:
|
||||
if business_analysis:
|
||||
st.subheader("Business Analysis & Insights")
|
||||
|
||||
# Market positioning
|
||||
market_position = business_analysis.get('market_position', '')
|
||||
if market_position:
|
||||
st.markdown("#### 🎯 Market Positioning")
|
||||
st.info(market_position)
|
||||
|
||||
# Content gaps
|
||||
content_gaps = business_analysis.get('content_gaps', [])
|
||||
if content_gaps:
|
||||
st.markdown("#### 🔍 Content Gaps Identified")
|
||||
for gap in content_gaps:
|
||||
st.warning(f"📌 {gap}")
|
||||
|
||||
# Competitive advantages
|
||||
advantages = business_analysis.get('competitive_advantages', [])
|
||||
if advantages:
|
||||
st.markdown("#### 🏆 Competitive Advantages")
|
||||
for advantage in advantages:
|
||||
st.success(f"✅ {advantage}")
|
||||
|
||||
# AI insights
|
||||
ai_insights = results.get('ai_insights', {})
|
||||
if ai_insights:
|
||||
st.markdown("#### 🧠 Strategic AI Insights")
|
||||
|
||||
insights = ai_insights.get('key_insights', [])
|
||||
for insight in insights:
|
||||
st.info(f"💡 {insight}")
|
||||
|
||||
recommendations = ai_insights.get('strategic_recommendations', [])
|
||||
if recommendations:
|
||||
st.markdown("#### 🎯 Strategic Recommendations")
|
||||
for rec in recommendations:
|
||||
st.success(f"📋 {rec}")
|
||||
|
||||
with tab2:
|
||||
if content_pillars:
|
||||
st.subheader("Content Pillars Strategy")
|
||||
|
||||
# Pillars overview chart
|
||||
pillar_names = [pillar['name'] for pillar in content_pillars]
|
||||
pillar_ideas = [len(pillar['content_ideas']) for pillar in content_pillars]
|
||||
|
||||
fig = px.bar(
|
||||
x=pillar_names,
|
||||
y=pillar_ideas,
|
||||
title="Content Ideas per Pillar",
|
||||
labels={'x': 'Content Pillars', 'y': 'Number of Ideas'}
|
||||
)
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
# Detailed pillar information
|
||||
for pillar in content_pillars:
|
||||
with st.expander(f"🏛️ {pillar['name']}", expanded=False):
|
||||
st.markdown(f"**Description:** {pillar['description']}")
|
||||
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
st.markdown("**Target Keywords:**")
|
||||
for keyword in pillar['target_keywords']:
|
||||
st.code(keyword)
|
||||
|
||||
st.markdown("**Content Types:**")
|
||||
for content_type in pillar['content_types']:
|
||||
st.write(f"• {content_type}")
|
||||
|
||||
with col2:
|
||||
st.markdown("**Success Metrics:**")
|
||||
for metric in pillar['success_metrics']:
|
||||
st.write(f"📊 {metric}")
|
||||
|
||||
st.markdown("**Content Ideas:**")
|
||||
for idea in pillar['content_ideas']:
|
||||
st.write(f"💡 {idea}")
|
||||
|
||||
with tab3:
|
||||
if content_calendar:
|
||||
st.subheader("Content Calendar & Planning")
|
||||
|
||||
calendar_items = content_calendar.get('calendar_items', [])
|
||||
|
||||
if calendar_items:
|
||||
# Calendar overview
|
||||
df_calendar = pd.DataFrame(calendar_items)
|
||||
|
||||
# Priority distribution
|
||||
priority_counts = df_calendar['priority'].value_counts()
|
||||
fig_priority = px.pie(
|
||||
values=priority_counts.values,
|
||||
names=priority_counts.index,
|
||||
title="Content Priority Distribution"
|
||||
)
|
||||
st.plotly_chart(fig_priority, use_container_width=True)
|
||||
|
||||
# Content calendar table
|
||||
st.markdown("#### 📅 Detailed Content Calendar")
|
||||
|
||||
display_df = df_calendar[[
|
||||
'period', 'pillar', 'content_type', 'topic',
|
||||
'priority', 'estimated_hours'
|
||||
]].copy()
|
||||
|
||||
display_df.columns = [
|
||||
'Period', 'Pillar', 'Content Type', 'Topic',
|
||||
'Priority', 'Est. Hours'
|
||||
]
|
||||
|
||||
st.dataframe(
|
||||
display_df,
|
||||
column_config={
|
||||
"Priority": st.column_config.SelectboxColumn(
|
||||
"Priority",
|
||||
options=["High", "Medium", "Low"]
|
||||
),
|
||||
"Est. Hours": st.column_config.NumberColumn(
|
||||
"Est. Hours",
|
||||
format="%d h"
|
||||
)
|
||||
},
|
||||
hide_index=True,
|
||||
use_container_width=True
|
||||
)
|
||||
|
||||
# Export calendar
|
||||
csv = df_calendar.to_csv(index=False)
|
||||
st.download_button(
|
||||
label="📥 Download Content Calendar",
|
||||
data=csv,
|
||||
file_name=f"content_calendar_{datetime.now().strftime('%Y%m%d')}.csv",
|
||||
mime="text/csv"
|
||||
)
|
||||
|
||||
with tab4:
|
||||
topic_clusters = results.get('topic_clusters', [])
|
||||
if topic_clusters:
|
||||
st.subheader("SEO Topic Clusters")
|
||||
|
||||
for cluster in topic_clusters:
|
||||
with st.expander(f"🎯 {cluster['cluster_name']}", expanded=False):
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
st.markdown(f"**Primary Topic:** {cluster['primary_topic']}")
|
||||
st.markdown(f"**SEO Opportunity:** {cluster['seo_opportunity']}")
|
||||
st.markdown(f"**Linking Strategy:** {cluster['internal_linking_strategy']}")
|
||||
|
||||
with col2:
|
||||
st.markdown("**Supporting Topics:**")
|
||||
for topic in cluster['supporting_topics']:
|
||||
st.code(topic)
|
||||
|
||||
st.markdown("**Content Pieces:**")
|
||||
content_pieces = cluster['content_pieces']
|
||||
df_pieces = pd.DataFrame(content_pieces)
|
||||
st.dataframe(df_pieces, hide_index=True, use_container_width=True)
|
||||
|
||||
with tab5:
|
||||
distribution_strategy = results.get('distribution_strategy', {})
|
||||
if distribution_strategy:
|
||||
st.subheader("Content Distribution Strategy")
|
||||
|
||||
# Primary channels
|
||||
primary_channels = distribution_strategy.get('primary_channels', [])
|
||||
if primary_channels:
|
||||
st.markdown("#### 📢 Primary Distribution Channels")
|
||||
df_primary = pd.DataFrame(primary_channels)
|
||||
st.dataframe(df_primary, hide_index=True, use_container_width=True)
|
||||
|
||||
# Secondary channels
|
||||
secondary_channels = distribution_strategy.get('secondary_channels', [])
|
||||
if secondary_channels:
|
||||
st.markdown("#### 📺 Secondary Distribution Channels")
|
||||
df_secondary = pd.DataFrame(secondary_channels)
|
||||
st.dataframe(df_secondary, hide_index=True, use_container_width=True)
|
||||
|
||||
# Repurposing strategy
|
||||
repurposing = distribution_strategy.get('repurposing_strategy', {})
|
||||
if repurposing:
|
||||
st.markdown("#### ♻️ Content Repurposing Strategy")
|
||||
for strategy, description in repurposing.items():
|
||||
st.write(f"**{strategy.replace('_', ' ').title()}:** {description}")
|
||||
|
||||
with tab6:
|
||||
# Implementation roadmap
|
||||
roadmap = results.get('implementation_roadmap', {})
|
||||
kpi_framework = results.get('kpi_framework', {})
|
||||
|
||||
if roadmap:
|
||||
st.subheader("Implementation Roadmap")
|
||||
|
||||
for phase_key, phase_data in roadmap.items():
|
||||
with st.expander(f"📋 {phase_data['name']}", expanded=False):
|
||||
st.markdown(f"**Objectives:**")
|
||||
for objective in phase_data['objectives']:
|
||||
st.write(f"• {objective}")
|
||||
|
||||
st.markdown(f"**Deliverables:**")
|
||||
for deliverable in phase_data['deliverables']:
|
||||
st.write(f"📦 {deliverable}")
|
||||
|
||||
st.markdown(f"**Success Criteria:**")
|
||||
for criteria in phase_data['success_criteria']:
|
||||
st.write(f"✅ {criteria}")
|
||||
|
||||
if kpi_framework:
|
||||
st.subheader("KPI Framework")
|
||||
|
||||
# Primary KPIs
|
||||
primary_kpis = kpi_framework.get('primary_kpis', [])
|
||||
if primary_kpis:
|
||||
st.markdown("#### 🎯 Primary KPIs")
|
||||
df_primary_kpis = pd.DataFrame(primary_kpis)
|
||||
st.dataframe(df_primary_kpis, hide_index=True, use_container_width=True)
|
||||
|
||||
# Content KPIs
|
||||
content_kpis = kpi_framework.get('content_kpis', [])
|
||||
if content_kpis:
|
||||
st.markdown("#### 📝 Content KPIs")
|
||||
df_content_kpis = pd.DataFrame(content_kpis)
|
||||
st.dataframe(df_content_kpis, hide_index=True, use_container_width=True)
|
||||
|
||||
# Export functionality
|
||||
st.markdown("---")
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
if st.button("📥 Export Full Strategy", use_container_width=True):
|
||||
strategy_json = json.dumps(results, indent=2, default=str)
|
||||
st.download_button(
|
||||
label="Download JSON Strategy",
|
||||
data=strategy_json,
|
||||
file_name=f"content_strategy_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json",
|
||||
mime="application/json"
|
||||
)
|
||||
|
||||
with col2:
|
||||
if st.button("📊 Export Calendar", use_container_width=True):
|
||||
calendar_items = content_calendar.get('calendar_items', [])
|
||||
if calendar_items:
|
||||
df_calendar = pd.DataFrame(calendar_items)
|
||||
csv = df_calendar.to_csv(index=False)
|
||||
st.download_button(
|
||||
label="Download CSV Calendar",
|
||||
data=csv,
|
||||
file_name=f"content_calendar_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv",
|
||||
mime="text/csv"
|
||||
)
|
||||
|
||||
with col3:
|
||||
if st.button("🔄 Generate New Strategy", use_container_width=True):
|
||||
if 'strategy_results' in st.session_state:
|
||||
del st.session_state.strategy_results
|
||||
st.rerun()
|
||||
|
||||
|
||||
# Main execution
|
||||
if __name__ == "__main__":
|
||||
render_ai_content_strategy()
|
||||
919
ToBeMigrated/ai_seo_tools/enterprise_seo_suite.py
Normal file
919
ToBeMigrated/ai_seo_tools/enterprise_seo_suite.py
Normal file
@@ -0,0 +1,919 @@
|
||||
"""
|
||||
Enterprise SEO Command Center
|
||||
|
||||
Unified AI-powered SEO suite that orchestrates all existing tools into
|
||||
intelligent workflows for enterprise-level SEO management.
|
||||
"""
|
||||
|
||||
import streamlit as st
|
||||
import asyncio
|
||||
import pandas as pd
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
from datetime import datetime, timedelta
|
||||
import json
|
||||
from loguru import logger
|
||||
|
||||
# Import existing SEO tools
|
||||
from .on_page_seo_analyzer import fetch_seo_data
|
||||
from .content_gap_analysis.enhanced_analyzer import EnhancedContentGapAnalyzer
|
||||
from .technical_seo_crawler.crawler import TechnicalSEOCrawler
|
||||
from .weburl_seo_checker import url_seo_checker
|
||||
from .google_pagespeed_insights import google_pagespeed_insights
|
||||
from ..gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
|
||||
# Import the new enterprise tools
|
||||
from .google_search_console_integration import GoogleSearchConsoleAnalyzer, render_gsc_integration
|
||||
from .ai_content_strategy import AIContentStrategyGenerator, render_ai_content_strategy
|
||||
|
||||
class EnterpriseSEOSuite:
|
||||
"""
|
||||
Enterprise-level SEO suite orchestrating all tools into intelligent workflows.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the enterprise SEO suite."""
|
||||
self.gap_analyzer = EnhancedContentGapAnalyzer()
|
||||
self.technical_crawler = TechnicalSEOCrawler()
|
||||
|
||||
# Initialize new enterprise tools
|
||||
self.gsc_analyzer = GoogleSearchConsoleAnalyzer()
|
||||
self.content_strategy_generator = AIContentStrategyGenerator()
|
||||
|
||||
# SEO workflow templates
|
||||
self.workflow_templates = {
|
||||
'complete_audit': 'Complete SEO Audit',
|
||||
'content_strategy': 'Content Strategy Development',
|
||||
'technical_optimization': 'Technical SEO Optimization',
|
||||
'competitor_intelligence': 'Competitive Intelligence',
|
||||
'keyword_domination': 'Keyword Domination Strategy',
|
||||
'local_seo': 'Local SEO Optimization',
|
||||
'enterprise_monitoring': 'Enterprise SEO Monitoring'
|
||||
}
|
||||
|
||||
logger.info("Enterprise SEO Suite initialized")
|
||||
|
||||
async def execute_complete_seo_audit(self, website_url: str, competitors: List[str],
|
||||
target_keywords: List[str]) -> Dict[str, Any]:
|
||||
"""
|
||||
Execute a comprehensive enterprise SEO audit combining all tools.
|
||||
|
||||
Args:
|
||||
website_url: Primary website to audit
|
||||
competitors: List of competitor URLs (max 5)
|
||||
target_keywords: Primary keywords to optimize for
|
||||
|
||||
Returns:
|
||||
Comprehensive audit results with prioritized action plan
|
||||
"""
|
||||
try:
|
||||
st.info("🚀 Initiating Complete Enterprise SEO Audit...")
|
||||
|
||||
audit_results = {
|
||||
'audit_timestamp': datetime.utcnow().isoformat(),
|
||||
'website_url': website_url,
|
||||
'competitors': competitors[:5],
|
||||
'target_keywords': target_keywords,
|
||||
'technical_audit': {},
|
||||
'content_analysis': {},
|
||||
'competitive_intelligence': {},
|
||||
'on_page_analysis': {},
|
||||
'performance_metrics': {},
|
||||
'strategic_recommendations': {},
|
||||
'priority_action_plan': []
|
||||
}
|
||||
|
||||
# Phase 1: Technical SEO Audit
|
||||
with st.expander("🔧 Technical SEO Analysis", expanded=True):
|
||||
st.info("Analyzing technical SEO factors...")
|
||||
technical_results = await self._run_technical_audit(website_url)
|
||||
audit_results['technical_audit'] = technical_results
|
||||
st.success("✅ Technical audit completed")
|
||||
|
||||
# Phase 2: Content Gap Analysis
|
||||
with st.expander("📊 Content Intelligence Analysis", expanded=True):
|
||||
st.info("Analyzing content gaps and opportunities...")
|
||||
content_results = await self._run_content_analysis(
|
||||
website_url, competitors, target_keywords
|
||||
)
|
||||
audit_results['content_analysis'] = content_results
|
||||
st.success("✅ Content analysis completed")
|
||||
|
||||
# Phase 3: On-Page SEO Analysis
|
||||
with st.expander("🔍 On-Page SEO Analysis", expanded=True):
|
||||
st.info("Analyzing on-page SEO factors...")
|
||||
onpage_results = await self._run_onpage_analysis(website_url)
|
||||
audit_results['on_page_analysis'] = onpage_results
|
||||
st.success("✅ On-page analysis completed")
|
||||
|
||||
# Phase 4: Performance Analysis
|
||||
with st.expander("⚡ Performance Analysis", expanded=True):
|
||||
st.info("Analyzing website performance...")
|
||||
performance_results = await self._run_performance_analysis(website_url)
|
||||
audit_results['performance_metrics'] = performance_results
|
||||
st.success("✅ Performance analysis completed")
|
||||
|
||||
# Phase 5: AI-Powered Strategic Recommendations
|
||||
with st.expander("🤖 AI Strategic Analysis", expanded=True):
|
||||
st.info("Generating AI-powered strategic recommendations...")
|
||||
strategic_analysis = await self._generate_strategic_recommendations(audit_results)
|
||||
audit_results['strategic_recommendations'] = strategic_analysis
|
||||
|
||||
# Generate prioritized action plan
|
||||
action_plan = await self._create_priority_action_plan(audit_results)
|
||||
audit_results['priority_action_plan'] = action_plan
|
||||
st.success("✅ Strategic analysis completed")
|
||||
|
||||
return audit_results
|
||||
|
||||
except Exception as e:
|
||||
error_msg = f"Error in complete SEO audit: {str(e)}"
|
||||
logger.error(error_msg, exc_info=True)
|
||||
st.error(error_msg)
|
||||
return {'error': error_msg}
|
||||
|
||||
async def _run_technical_audit(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Run comprehensive technical SEO audit."""
|
||||
try:
|
||||
# Use existing technical crawler
|
||||
technical_results = self.technical_crawler.analyze_website_technical_seo(
|
||||
website_url, crawl_depth=3, max_pages=100
|
||||
)
|
||||
|
||||
# Enhance with additional technical checks
|
||||
enhanced_results = {
|
||||
'crawler_results': technical_results,
|
||||
'critical_issues': self._identify_critical_technical_issues(technical_results),
|
||||
'performance_score': self._calculate_technical_score(technical_results),
|
||||
'priority_fixes': self._prioritize_technical_fixes(technical_results)
|
||||
}
|
||||
|
||||
return enhanced_results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Technical audit error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
async def _run_content_analysis(self, website_url: str, competitors: List[str],
|
||||
keywords: List[str]) -> Dict[str, Any]:
|
||||
"""Run comprehensive content gap analysis."""
|
||||
try:
|
||||
# Use existing content gap analyzer
|
||||
content_results = self.gap_analyzer.analyze_comprehensive_gap(
|
||||
website_url, competitors, keywords, industry="general"
|
||||
)
|
||||
|
||||
# Enhance with content strategy insights
|
||||
enhanced_results = {
|
||||
'gap_analysis': content_results,
|
||||
'content_opportunities': self._identify_content_opportunities(content_results),
|
||||
'keyword_strategy': self._develop_keyword_strategy(content_results),
|
||||
'competitive_advantages': self._find_competitive_advantages(content_results)
|
||||
}
|
||||
|
||||
return enhanced_results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Content analysis error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
async def _run_onpage_analysis(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Run on-page SEO analysis."""
|
||||
try:
|
||||
# Use existing on-page analyzer
|
||||
onpage_data = fetch_seo_data(website_url)
|
||||
|
||||
# Enhanced analysis
|
||||
enhanced_results = {
|
||||
'seo_data': onpage_data,
|
||||
'optimization_score': self._calculate_onpage_score(onpage_data),
|
||||
'meta_optimization': self._analyze_meta_optimization(onpage_data),
|
||||
'content_optimization': self._analyze_content_optimization(onpage_data)
|
||||
}
|
||||
|
||||
return enhanced_results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"On-page analysis error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
async def _run_performance_analysis(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Run website performance analysis."""
|
||||
try:
|
||||
# Comprehensive performance metrics
|
||||
performance_results = {
|
||||
'core_web_vitals': await self._analyze_core_web_vitals(website_url),
|
||||
'loading_performance': await self._analyze_loading_performance(website_url),
|
||||
'mobile_optimization': await self._analyze_mobile_optimization(website_url),
|
||||
'performance_score': 0 # Will be calculated
|
||||
}
|
||||
|
||||
# Calculate overall performance score
|
||||
performance_results['performance_score'] = self._calculate_performance_score(
|
||||
performance_results
|
||||
)
|
||||
|
||||
return performance_results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Performance analysis error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
async def _generate_strategic_recommendations(self, audit_results: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Generate AI-powered strategic recommendations."""
|
||||
try:
|
||||
# Compile audit summary for AI analysis
|
||||
audit_summary = {
|
||||
'technical_score': audit_results.get('technical_audit', {}).get('performance_score', 0),
|
||||
'content_gaps': len(audit_results.get('content_analysis', {}).get('content_opportunities', [])),
|
||||
'onpage_score': audit_results.get('on_page_analysis', {}).get('optimization_score', 0),
|
||||
'performance_score': audit_results.get('performance_metrics', {}).get('performance_score', 0)
|
||||
}
|
||||
|
||||
strategic_prompt = f"""
|
||||
Analyze this comprehensive SEO audit and provide strategic recommendations:
|
||||
|
||||
AUDIT SUMMARY:
|
||||
- Technical SEO Score: {audit_summary['technical_score']}/100
|
||||
- Content Gaps Identified: {audit_summary['content_gaps']}
|
||||
- On-Page SEO Score: {audit_summary['onpage_score']}/100
|
||||
- Performance Score: {audit_summary['performance_score']}/100
|
||||
|
||||
DETAILED FINDINGS:
|
||||
Technical Issues: {json.dumps(audit_results.get('technical_audit', {}), indent=2)[:1000]}
|
||||
Content Opportunities: {json.dumps(audit_results.get('content_analysis', {}), indent=2)[:1000]}
|
||||
|
||||
Provide strategic recommendations in these categories:
|
||||
|
||||
1. IMMEDIATE WINS (0-30 days):
|
||||
- Quick technical fixes with high impact
|
||||
- Content optimizations for existing pages
|
||||
- Critical performance improvements
|
||||
|
||||
2. STRATEGIC INITIATIVES (1-3 months):
|
||||
- Content strategy development
|
||||
- Technical architecture improvements
|
||||
- Competitive positioning strategies
|
||||
|
||||
3. LONG-TERM GROWTH (3-12 months):
|
||||
- Authority building strategies
|
||||
- Market expansion opportunities
|
||||
- Advanced SEO techniques
|
||||
|
||||
4. RISK MITIGATION:
|
||||
- Technical vulnerabilities to address
|
||||
- Content gaps that competitors could exploit
|
||||
- Performance issues affecting user experience
|
||||
|
||||
Provide specific, actionable recommendations with expected impact and effort estimates.
|
||||
"""
|
||||
|
||||
strategic_analysis = llm_text_gen(
|
||||
strategic_prompt,
|
||||
system_prompt="You are an enterprise SEO strategist with 10+ years of experience. Provide detailed, actionable recommendations based on comprehensive audit data."
|
||||
)
|
||||
|
||||
return {
|
||||
'full_analysis': strategic_analysis,
|
||||
'immediate_wins': self._extract_immediate_wins(strategic_analysis),
|
||||
'strategic_initiatives': self._extract_strategic_initiatives(strategic_analysis),
|
||||
'long_term_growth': self._extract_long_term_growth(strategic_analysis),
|
||||
'risk_mitigation': self._extract_risk_mitigation(strategic_analysis)
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Strategic analysis error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
async def _create_priority_action_plan(self, audit_results: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
"""Create prioritized action plan from audit results."""
|
||||
try:
|
||||
action_plan = []
|
||||
|
||||
# Extract recommendations from all analysis phases
|
||||
strategic_recs = audit_results.get('strategic_recommendations', {})
|
||||
|
||||
# Immediate wins (High priority, low effort)
|
||||
immediate_wins = strategic_recs.get('immediate_wins', [])
|
||||
for win in immediate_wins[:5]:
|
||||
action_plan.append({
|
||||
'category': 'Immediate Win',
|
||||
'priority': 'Critical',
|
||||
'effort': 'Low',
|
||||
'timeframe': '0-30 days',
|
||||
'action': win,
|
||||
'expected_impact': 'High',
|
||||
'source': 'Strategic Analysis'
|
||||
})
|
||||
|
||||
# Technical fixes
|
||||
technical_issues = audit_results.get('technical_audit', {}).get('critical_issues', [])
|
||||
for issue in technical_issues[:3]:
|
||||
action_plan.append({
|
||||
'category': 'Technical SEO',
|
||||
'priority': 'High',
|
||||
'effort': 'Medium',
|
||||
'timeframe': '1-4 weeks',
|
||||
'action': issue,
|
||||
'expected_impact': 'High',
|
||||
'source': 'Technical Audit'
|
||||
})
|
||||
|
||||
# Content opportunities
|
||||
content_ops = audit_results.get('content_analysis', {}).get('content_opportunities', [])
|
||||
for opportunity in content_ops[:3]:
|
||||
action_plan.append({
|
||||
'category': 'Content Strategy',
|
||||
'priority': 'Medium',
|
||||
'effort': 'High',
|
||||
'timeframe': '2-8 weeks',
|
||||
'action': opportunity,
|
||||
'expected_impact': 'Medium',
|
||||
'source': 'Content Analysis'
|
||||
})
|
||||
|
||||
# Sort by priority and expected impact
|
||||
priority_order = {'Critical': 0, 'High': 1, 'Medium': 2, 'Low': 3}
|
||||
action_plan.sort(key=lambda x: priority_order.get(x['priority'], 4))
|
||||
|
||||
return action_plan[:15] # Top 15 actions
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Action plan creation error: {str(e)}")
|
||||
return []
|
||||
|
||||
# Utility methods for analysis
|
||||
def _identify_critical_technical_issues(self, technical_results: Dict[str, Any]) -> List[str]:
|
||||
"""Identify critical technical SEO issues."""
|
||||
critical_issues = []
|
||||
|
||||
# Add logic to identify critical technical issues
|
||||
# This would analyze the technical_results and extract critical problems
|
||||
|
||||
return critical_issues
|
||||
|
||||
def _calculate_technical_score(self, technical_results: Dict[str, Any]) -> int:
|
||||
"""Calculate technical SEO score."""
|
||||
# Implement scoring algorithm based on technical audit results
|
||||
return 75 # Placeholder
|
||||
|
||||
def _prioritize_technical_fixes(self, technical_results: Dict[str, Any]) -> List[str]:
|
||||
"""Prioritize technical fixes by impact and effort."""
|
||||
# Implement prioritization logic
|
||||
return ["Fix broken links", "Optimize images", "Improve page speed"]
|
||||
|
||||
def _identify_content_opportunities(self, content_results: Dict[str, Any]) -> List[str]:
|
||||
"""Identify top content opportunities."""
|
||||
# Extract content opportunities from gap analysis
|
||||
return ["Create FAQ content", "Develop comparison guides", "Write how-to articles"]
|
||||
|
||||
def _develop_keyword_strategy(self, content_results: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Develop keyword strategy from content analysis."""
|
||||
return {
|
||||
'primary_keywords': [],
|
||||
'secondary_keywords': [],
|
||||
'long_tail_opportunities': [],
|
||||
'competitor_gaps': []
|
||||
}
|
||||
|
||||
def _find_competitive_advantages(self, content_results: Dict[str, Any]) -> List[str]:
|
||||
"""Find competitive advantages from analysis."""
|
||||
return ["Unique content angles", "Underserved niches", "Technical superiority"]
|
||||
|
||||
def _calculate_onpage_score(self, onpage_data: Dict[str, Any]) -> int:
|
||||
"""Calculate on-page SEO score."""
|
||||
return 80 # Placeholder
|
||||
|
||||
def _analyze_meta_optimization(self, onpage_data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze meta tag optimization."""
|
||||
return {'title_optimization': 'good', 'description_optimization': 'needs_work'}
|
||||
|
||||
def _analyze_content_optimization(self, onpage_data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze content optimization."""
|
||||
return {'keyword_density': 'optimal', 'content_length': 'adequate'}
|
||||
|
||||
async def _analyze_core_web_vitals(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Analyze Core Web Vitals."""
|
||||
return {'lcp': 2.5, 'fid': 100, 'cls': 0.1}
|
||||
|
||||
async def _analyze_loading_performance(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Analyze loading performance."""
|
||||
return {'ttfb': 200, 'fcp': 1.5, 'speed_index': 3.0}
|
||||
|
||||
async def _analyze_mobile_optimization(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Analyze mobile optimization."""
|
||||
return {'mobile_friendly': True, 'responsive_design': True}
|
||||
|
||||
def _calculate_performance_score(self, performance_results: Dict[str, Any]) -> int:
|
||||
"""Calculate overall performance score."""
|
||||
return 85 # Placeholder
|
||||
|
||||
def _extract_immediate_wins(self, analysis: str) -> List[str]:
|
||||
"""Extract immediate wins from strategic analysis."""
|
||||
# Parse the AI analysis and extract immediate wins
|
||||
lines = analysis.split('\n')
|
||||
wins = []
|
||||
in_immediate_section = False
|
||||
|
||||
for line in lines:
|
||||
if 'IMMEDIATE WINS' in line.upper():
|
||||
in_immediate_section = True
|
||||
continue
|
||||
elif 'STRATEGIC INITIATIVES' in line.upper():
|
||||
in_immediate_section = False
|
||||
continue
|
||||
|
||||
if in_immediate_section and line.strip().startswith('-'):
|
||||
wins.append(line.strip().lstrip('- '))
|
||||
|
||||
return wins[:5]
|
||||
|
||||
def _extract_strategic_initiatives(self, analysis: str) -> List[str]:
|
||||
"""Extract strategic initiatives from analysis."""
|
||||
# Similar extraction logic for strategic initiatives
|
||||
return ["Develop content hub", "Implement schema markup", "Build authority pages"]
|
||||
|
||||
def _extract_long_term_growth(self, analysis: str) -> List[str]:
|
||||
"""Extract long-term growth strategies."""
|
||||
return ["Market expansion", "Authority building", "Advanced technical SEO"]
|
||||
|
||||
def _extract_risk_mitigation(self, analysis: str) -> List[str]:
|
||||
"""Extract risk mitigation strategies."""
|
||||
return ["Fix technical vulnerabilities", "Address content gaps", "Improve performance"]
|
||||
|
||||
def execute_content_strategy_workflow(self, business_info: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Execute comprehensive content strategy workflow using AI insights.
|
||||
|
||||
Args:
|
||||
business_info: Business context and objectives
|
||||
|
||||
Returns:
|
||||
Complete content strategy with implementation plan
|
||||
"""
|
||||
try:
|
||||
st.info("🧠 Executing AI-powered content strategy workflow...")
|
||||
|
||||
# Generate AI content strategy
|
||||
content_strategy = self.content_strategy_generator.generate_content_strategy(business_info)
|
||||
|
||||
# If GSC data is available, enhance with search insights
|
||||
if business_info.get('gsc_site_url'):
|
||||
gsc_insights = self.gsc_analyzer.analyze_search_performance(
|
||||
business_info['gsc_site_url'],
|
||||
business_info.get('gsc_date_range', 90)
|
||||
)
|
||||
content_strategy['gsc_insights'] = gsc_insights
|
||||
|
||||
# Generate SEO-optimized content recommendations
|
||||
seo_content_recs = self._generate_seo_content_recommendations(content_strategy)
|
||||
content_strategy['seo_recommendations'] = seo_content_recs
|
||||
|
||||
return content_strategy
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Content strategy workflow error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
def execute_search_intelligence_workflow(self, site_url: str, date_range: int = 90) -> Dict[str, Any]:
|
||||
"""
|
||||
Execute comprehensive search intelligence workflow using GSC data.
|
||||
|
||||
Args:
|
||||
site_url: Website URL registered in GSC
|
||||
date_range: Analysis period in days
|
||||
|
||||
Returns:
|
||||
Complete search intelligence analysis with actionable insights
|
||||
"""
|
||||
try:
|
||||
st.info("📊 Executing search intelligence workflow...")
|
||||
|
||||
# Analyze GSC performance
|
||||
gsc_analysis = self.gsc_analyzer.analyze_search_performance(site_url, date_range)
|
||||
|
||||
# Enhance with technical SEO analysis
|
||||
technical_analysis = self.technical_crawler.crawl_and_analyze(site_url)
|
||||
gsc_analysis['technical_insights'] = technical_analysis
|
||||
|
||||
# Generate content gap analysis based on GSC keywords
|
||||
if gsc_analysis.get('keyword_analysis'):
|
||||
keywords = [kw['keyword'] for kw in gsc_analysis['keyword_analysis'].get('high_volume_keywords', [])]
|
||||
content_gaps = self.gap_analyzer.analyze_content_gaps(
|
||||
keywords[:10], # Top 10 keywords
|
||||
site_url
|
||||
)
|
||||
gsc_analysis['content_gap_analysis'] = content_gaps
|
||||
|
||||
# Generate comprehensive recommendations
|
||||
search_recommendations = self._generate_search_intelligence_recommendations(gsc_analysis)
|
||||
gsc_analysis['comprehensive_recommendations'] = search_recommendations
|
||||
|
||||
return gsc_analysis
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Search intelligence workflow error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
def _generate_seo_content_recommendations(self, content_strategy: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Generate SEO-optimized content recommendations based on strategy."""
|
||||
try:
|
||||
content_pillars = content_strategy.get('content_pillars', [])
|
||||
|
||||
seo_recommendations = {
|
||||
'keyword_optimization': [],
|
||||
'content_structure': [],
|
||||
'internal_linking': [],
|
||||
'technical_seo': []
|
||||
}
|
||||
|
||||
for pillar in content_pillars:
|
||||
# Keyword optimization recommendations
|
||||
for keyword in pillar.get('target_keywords', []):
|
||||
seo_recommendations['keyword_optimization'].append({
|
||||
'pillar': pillar['name'],
|
||||
'keyword': keyword,
|
||||
'recommendation': f"Create comprehensive content targeting '{keyword}' with semantic variations",
|
||||
'priority': 'High' if keyword in pillar['target_keywords'][:2] else 'Medium'
|
||||
})
|
||||
|
||||
# Content structure recommendations
|
||||
seo_recommendations['content_structure'].append({
|
||||
'pillar': pillar['name'],
|
||||
'recommendation': f"Create pillar page for {pillar['name']} with supporting cluster content",
|
||||
'structure': 'Pillar + Cluster model'
|
||||
})
|
||||
|
||||
# Internal linking strategy
|
||||
seo_recommendations['internal_linking'] = [
|
||||
"Link all cluster content to relevant pillar pages",
|
||||
"Create topic-based internal linking structure",
|
||||
"Use contextual anchor text with target keywords",
|
||||
"Implement breadcrumb navigation for topic clusters"
|
||||
]
|
||||
|
||||
# Technical SEO recommendations
|
||||
seo_recommendations['technical_seo'] = [
|
||||
"Optimize page speed for all content pages",
|
||||
"Implement structured data for articles",
|
||||
"Create XML sitemap sections for content categories",
|
||||
"Optimize images with descriptive alt text"
|
||||
]
|
||||
|
||||
return seo_recommendations
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"SEO content recommendations error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
def _generate_search_intelligence_recommendations(self, gsc_analysis: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Generate comprehensive recommendations from search intelligence analysis."""
|
||||
try:
|
||||
recommendations = {
|
||||
'immediate_actions': [],
|
||||
'content_opportunities': [],
|
||||
'technical_improvements': [],
|
||||
'strategic_initiatives': []
|
||||
}
|
||||
|
||||
# Extract content opportunities from GSC analysis
|
||||
content_opps = gsc_analysis.get('content_opportunities', [])
|
||||
for opp in content_opps[:5]: # Top 5 opportunities
|
||||
recommendations['content_opportunities'].append({
|
||||
'type': opp['type'],
|
||||
'keyword': opp['keyword'],
|
||||
'action': opp['opportunity'],
|
||||
'priority': opp['priority'],
|
||||
'estimated_impact': opp['potential_impact']
|
||||
})
|
||||
|
||||
# Technical improvements from analysis
|
||||
technical_insights = gsc_analysis.get('technical_insights', {})
|
||||
if technical_insights.get('crawl_issues_indicators'):
|
||||
for issue in technical_insights['crawl_issues_indicators']:
|
||||
recommendations['technical_improvements'].append({
|
||||
'issue': issue,
|
||||
'priority': 'High',
|
||||
'category': 'Crawl & Indexing'
|
||||
})
|
||||
|
||||
# Immediate actions based on performance
|
||||
performance = gsc_analysis.get('performance_overview', {})
|
||||
if performance.get('avg_ctr', 0) < 2:
|
||||
recommendations['immediate_actions'].append({
|
||||
'action': 'Improve meta descriptions and titles for better CTR',
|
||||
'expected_impact': 'Increase CTR by 1-2%',
|
||||
'timeline': '2-4 weeks'
|
||||
})
|
||||
|
||||
if performance.get('avg_position', 0) > 10:
|
||||
recommendations['immediate_actions'].append({
|
||||
'action': 'Focus on improving content quality for top keywords',
|
||||
'expected_impact': 'Improve average position by 2-5 ranks',
|
||||
'timeline': '4-8 weeks'
|
||||
})
|
||||
|
||||
# Strategic initiatives
|
||||
competitive_analysis = gsc_analysis.get('competitive_analysis', {})
|
||||
if competitive_analysis.get('market_position') in ['Challenger', 'Emerging Player']:
|
||||
recommendations['strategic_initiatives'].append({
|
||||
'initiative': 'Develop thought leadership content strategy',
|
||||
'goal': 'Improve market position and brand authority',
|
||||
'timeline': '3-6 months'
|
||||
})
|
||||
|
||||
return recommendations
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Search intelligence recommendations error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
def render_enterprise_seo_suite():
|
||||
"""Render the Enterprise SEO Command Center interface."""
|
||||
|
||||
st.set_page_config(
|
||||
page_title="Enterprise SEO Command Center",
|
||||
page_icon="🚀",
|
||||
layout="wide"
|
||||
)
|
||||
|
||||
st.title("🚀 Enterprise SEO Command Center")
|
||||
st.markdown("**Unified AI-powered SEO suite orchestrating all tools into intelligent workflows**")
|
||||
|
||||
# Initialize suite
|
||||
if 'enterprise_seo_suite' not in st.session_state:
|
||||
st.session_state.enterprise_seo_suite = EnterpriseSEOSuite()
|
||||
|
||||
suite = st.session_state.enterprise_seo_suite
|
||||
|
||||
# Workflow selection
|
||||
st.sidebar.header("🎯 SEO Workflow Selection")
|
||||
selected_workflow = st.sidebar.selectbox(
|
||||
"Choose Workflow",
|
||||
list(suite.workflow_templates.keys()),
|
||||
format_func=lambda x: suite.workflow_templates[x]
|
||||
)
|
||||
|
||||
# Main workflow interface
|
||||
if selected_workflow == 'complete_audit':
|
||||
st.header("🔍 Complete Enterprise SEO Audit")
|
||||
render_complete_audit_interface(suite)
|
||||
elif selected_workflow == 'content_strategy':
|
||||
st.header("📊 Content Strategy Development")
|
||||
render_content_strategy_interface(suite)
|
||||
elif selected_workflow == 'technical_optimization':
|
||||
st.header("🔧 Technical SEO Optimization")
|
||||
render_technical_optimization_interface(suite)
|
||||
else:
|
||||
st.info(f"Workflow '{suite.workflow_templates[selected_workflow]}' is being developed.")
|
||||
|
||||
def render_complete_audit_interface(suite: EnterpriseSEOSuite):
|
||||
"""Render the complete audit workflow interface."""
|
||||
|
||||
# Input form
|
||||
with st.form("enterprise_audit_form"):
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
website_url = st.text_input(
|
||||
"Website URL",
|
||||
value="https://example.com",
|
||||
help="Enter your website URL for comprehensive analysis"
|
||||
)
|
||||
|
||||
target_keywords = st.text_area(
|
||||
"Target Keywords (one per line)",
|
||||
value="AI content creation\nSEO tools\ncontent optimization",
|
||||
help="Enter your primary keywords to optimize for"
|
||||
)
|
||||
|
||||
with col2:
|
||||
competitors = st.text_area(
|
||||
"Competitor URLs (one per line)",
|
||||
value="https://jasper.ai\nhttps://copy.ai\nhttps://writesonic.com",
|
||||
help="Enter up to 5 competitor URLs for analysis"
|
||||
)
|
||||
|
||||
submit_audit = st.form_submit_button("🚀 Start Complete SEO Audit", type="primary")
|
||||
|
||||
# Process audit
|
||||
if submit_audit:
|
||||
if website_url and target_keywords:
|
||||
# Parse inputs
|
||||
keywords_list = [k.strip() for k in target_keywords.split('\n') if k.strip()]
|
||||
competitors_list = [c.strip() for c in competitors.split('\n') if c.strip()]
|
||||
|
||||
# Run audit
|
||||
with st.spinner("🔍 Running comprehensive SEO audit..."):
|
||||
audit_results = asyncio.run(
|
||||
suite.execute_complete_seo_audit(
|
||||
website_url, competitors_list, keywords_list
|
||||
)
|
||||
)
|
||||
|
||||
if 'error' not in audit_results:
|
||||
st.success("✅ Enterprise SEO audit completed!")
|
||||
|
||||
# Display results dashboard
|
||||
render_audit_results_dashboard(audit_results)
|
||||
else:
|
||||
st.error(f"❌ Audit failed: {audit_results['error']}")
|
||||
else:
|
||||
st.warning("⚠️ Please enter website URL and target keywords.")
|
||||
|
||||
def render_audit_results_dashboard(results: Dict[str, Any]):
|
||||
"""Render comprehensive audit results dashboard."""
|
||||
|
||||
# Priority Action Plan (Most Important)
|
||||
st.header("📋 Priority Action Plan")
|
||||
action_plan = results.get('priority_action_plan', [])
|
||||
|
||||
if action_plan:
|
||||
# Display as interactive table
|
||||
df_actions = pd.DataFrame(action_plan)
|
||||
|
||||
# Style the dataframe
|
||||
st.dataframe(
|
||||
df_actions,
|
||||
column_config={
|
||||
"category": "Category",
|
||||
"priority": st.column_config.SelectboxColumn(
|
||||
"Priority",
|
||||
options=["Critical", "High", "Medium", "Low"]
|
||||
),
|
||||
"effort": "Effort Level",
|
||||
"timeframe": "Timeline",
|
||||
"action": "Action Required",
|
||||
"expected_impact": "Expected Impact"
|
||||
},
|
||||
hide_index=True,
|
||||
use_container_width=True
|
||||
)
|
||||
|
||||
# Key Metrics Overview
|
||||
st.header("📊 SEO Health Dashboard")
|
||||
|
||||
col1, col2, col3, col4 = st.columns(4)
|
||||
|
||||
with col1:
|
||||
technical_score = results.get('technical_audit', {}).get('performance_score', 0)
|
||||
st.metric("Technical SEO", f"{technical_score}/100", delta=None)
|
||||
|
||||
with col2:
|
||||
onpage_score = results.get('on_page_analysis', {}).get('optimization_score', 0)
|
||||
st.metric("On-Page SEO", f"{onpage_score}/100", delta=None)
|
||||
|
||||
with col3:
|
||||
performance_score = results.get('performance_metrics', {}).get('performance_score', 0)
|
||||
st.metric("Performance", f"{performance_score}/100", delta=None)
|
||||
|
||||
with col4:
|
||||
content_gaps = len(results.get('content_analysis', {}).get('content_opportunities', []))
|
||||
st.metric("Content Opportunities", content_gaps, delta=None)
|
||||
|
||||
# Detailed Analysis Sections
|
||||
tab1, tab2, tab3, tab4, tab5 = st.tabs([
|
||||
"🤖 Strategic Insights",
|
||||
"🔧 Technical Analysis",
|
||||
"📊 Content Intelligence",
|
||||
"🔍 On-Page Analysis",
|
||||
"⚡ Performance Metrics"
|
||||
])
|
||||
|
||||
with tab1:
|
||||
strategic_recs = results.get('strategic_recommendations', {})
|
||||
if strategic_recs:
|
||||
st.subheader("AI-Powered Strategic Recommendations")
|
||||
|
||||
# Immediate wins
|
||||
immediate_wins = strategic_recs.get('immediate_wins', [])
|
||||
if immediate_wins:
|
||||
st.markdown("#### 🚀 Immediate Wins (0-30 days)")
|
||||
for win in immediate_wins[:5]:
|
||||
st.success(f"✅ {win}")
|
||||
|
||||
# Strategic initiatives
|
||||
strategic_initiatives = strategic_recs.get('strategic_initiatives', [])
|
||||
if strategic_initiatives:
|
||||
st.markdown("#### 📈 Strategic Initiatives (1-3 months)")
|
||||
for initiative in strategic_initiatives[:3]:
|
||||
st.info(f"📋 {initiative}")
|
||||
|
||||
# Full analysis
|
||||
full_analysis = strategic_recs.get('full_analysis', '')
|
||||
if full_analysis:
|
||||
with st.expander("🧠 Complete Strategic Analysis"):
|
||||
st.write(full_analysis)
|
||||
|
||||
with tab2:
|
||||
technical_audit = results.get('technical_audit', {})
|
||||
if technical_audit:
|
||||
st.subheader("Technical SEO Analysis")
|
||||
|
||||
critical_issues = technical_audit.get('critical_issues', [])
|
||||
if critical_issues:
|
||||
st.markdown("#### ⚠️ Critical Issues")
|
||||
for issue in critical_issues:
|
||||
st.error(f"🚨 {issue}")
|
||||
|
||||
priority_fixes = technical_audit.get('priority_fixes', [])
|
||||
if priority_fixes:
|
||||
st.markdown("#### 🔧 Priority Fixes")
|
||||
for fix in priority_fixes:
|
||||
st.warning(f"🛠️ {fix}")
|
||||
|
||||
with tab3:
|
||||
content_analysis = results.get('content_analysis', {})
|
||||
if content_analysis:
|
||||
st.subheader("Content Intelligence")
|
||||
|
||||
content_opportunities = content_analysis.get('content_opportunities', [])
|
||||
if content_opportunities:
|
||||
st.markdown("#### 📝 Content Opportunities")
|
||||
for opportunity in content_opportunities[:5]:
|
||||
st.info(f"💡 {opportunity}")
|
||||
|
||||
competitive_advantages = content_analysis.get('competitive_advantages', [])
|
||||
if competitive_advantages:
|
||||
st.markdown("#### 🏆 Competitive Advantages")
|
||||
for advantage in competitive_advantages:
|
||||
st.success(f"⭐ {advantage}")
|
||||
|
||||
with tab4:
|
||||
onpage_analysis = results.get('on_page_analysis', {})
|
||||
if onpage_analysis:
|
||||
st.subheader("On-Page SEO Analysis")
|
||||
|
||||
meta_optimization = onpage_analysis.get('meta_optimization', {})
|
||||
content_optimization = onpage_analysis.get('content_optimization', {})
|
||||
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
st.markdown("#### 🏷️ Meta Tag Optimization")
|
||||
st.json(meta_optimization)
|
||||
|
||||
with col2:
|
||||
st.markdown("#### 📄 Content Optimization")
|
||||
st.json(content_optimization)
|
||||
|
||||
with tab5:
|
||||
performance_metrics = results.get('performance_metrics', {})
|
||||
if performance_metrics:
|
||||
st.subheader("Performance Analysis")
|
||||
|
||||
core_vitals = performance_metrics.get('core_web_vitals', {})
|
||||
loading_performance = performance_metrics.get('loading_performance', {})
|
||||
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
st.markdown("#### ⚡ Core Web Vitals")
|
||||
st.json(core_vitals)
|
||||
|
||||
with col2:
|
||||
st.markdown("#### 🚀 Loading Performance")
|
||||
st.json(loading_performance)
|
||||
|
||||
# Export functionality
|
||||
st.markdown("---")
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
if st.button("📥 Export Full Report", use_container_width=True):
|
||||
# Create downloadable report
|
||||
report_json = json.dumps(results, indent=2, default=str)
|
||||
st.download_button(
|
||||
label="Download JSON Report",
|
||||
data=report_json,
|
||||
file_name=f"seo_audit_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json",
|
||||
mime="application/json"
|
||||
)
|
||||
|
||||
with col2:
|
||||
if st.button("📊 Export Action Plan", use_container_width=True):
|
||||
# Create CSV of action plan
|
||||
df_actions = pd.DataFrame(action_plan)
|
||||
csv = df_actions.to_csv(index=False)
|
||||
st.download_button(
|
||||
label="Download CSV Action Plan",
|
||||
data=csv,
|
||||
file_name=f"action_plan_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv",
|
||||
mime="text/csv"
|
||||
)
|
||||
|
||||
with col3:
|
||||
if st.button("🔄 Schedule Follow-up Audit", use_container_width=True):
|
||||
st.info("Follow-up scheduling feature coming soon!")
|
||||
|
||||
def render_content_strategy_interface(suite: EnterpriseSEOSuite):
|
||||
"""Render content strategy development interface."""
|
||||
st.info("🚧 Content Strategy Development workflow coming soon!")
|
||||
|
||||
def render_technical_optimization_interface(suite: EnterpriseSEOSuite):
|
||||
"""Render technical optimization interface."""
|
||||
st.info("🚧 Technical SEO Optimization workflow coming soon!")
|
||||
|
||||
|
||||
# Main execution
|
||||
if __name__ == "__main__":
|
||||
render_enterprise_seo_suite()
|
||||
135
ToBeMigrated/ai_seo_tools/google_pagespeed_insights.py
Normal file
135
ToBeMigrated/ai_seo_tools/google_pagespeed_insights.py
Normal file
@@ -0,0 +1,135 @@
|
||||
import requests
|
||||
import streamlit as st
|
||||
import json
|
||||
import pandas as pd
|
||||
import plotly.express as px
|
||||
from tenacity import retry, stop_after_attempt, wait_random_exponential
|
||||
from datetime import datetime
|
||||
|
||||
def run_pagespeed(url, api_key=None, strategy='DESKTOP', locale='en'):
|
||||
"""Fetches and processes PageSpeed Insights data."""
|
||||
serviceurl = 'https://www.googleapis.com/pagespeedonline/v5/runPagespeed'
|
||||
base_url = f"{serviceurl}?url={url}&strategy={strategy}&locale={locale}&category=performance&category=accessibility&category=best-practices&category=seo"
|
||||
|
||||
if api_key:
|
||||
base_url += f"&key={api_key}"
|
||||
|
||||
try:
|
||||
response = requests.get(base_url)
|
||||
response.raise_for_status() # Raise an exception for bad status codes
|
||||
data = response.json()
|
||||
return data
|
||||
except requests.exceptions.RequestException as e:
|
||||
st.error(f"Error fetching PageSpeed Insights data: {e}")
|
||||
return None
|
||||
|
||||
def display_results(data):
|
||||
"""Presents PageSpeed Insights data in a user-friendly format."""
|
||||
st.subheader("PageSpeed Insights Report")
|
||||
|
||||
# Extract scores from the PageSpeed Insights data
|
||||
scores = {
|
||||
"Performance": data['lighthouseResult']['categories']['performance']['score'] * 100,
|
||||
"Accessibility": data['lighthouseResult']['categories']['accessibility']['score'] * 100,
|
||||
"SEO": data['lighthouseResult']['categories']['seo']['score'] * 100,
|
||||
"Best Practices": data['lighthouseResult']['categories']['best-practices']['score'] * 100
|
||||
}
|
||||
|
||||
descriptions = {
|
||||
"Performance": data['lighthouseResult']['categories']['performance'].get('description', "This score represents Google's assessment of your page's speed. A higher percentage indicates better performance."),
|
||||
"Accessibility": data['lighthouseResult']['categories']['accessibility'].get('description', "This score evaluates how accessible your page is to users with disabilities. A higher percentage means better accessibility."),
|
||||
"SEO": data['lighthouseResult']['categories']['seo'].get('description', "This score measures how well your page is optimized for search engines. A higher percentage indicates better SEO practices."),
|
||||
"Best Practices": data['lighthouseResult']['categories']['best-practices'].get('description', "This score reflects how well your page follows best practices for web development. A higher percentage signifies adherence to best practices.")
|
||||
}
|
||||
|
||||
for category, score in scores.items():
|
||||
st.metric(label=f"Overall {category} Score", value=f"{score:.0f}%", help=descriptions[category])
|
||||
|
||||
# Display additional metrics
|
||||
st.subheader("Additional Metrics")
|
||||
additional_metrics = {
|
||||
"First Contentful Paint (FCP)": data['lighthouseResult']['audits']['first-contentful-paint']['displayValue'],
|
||||
"Largest Contentful Paint (LCP)": data['lighthouseResult']['audits']['largest-contentful-paint']['displayValue'],
|
||||
"Time to Interactive (TTI)": data['lighthouseResult']['audits']['interactive']['displayValue'],
|
||||
"Total Blocking Time (TBT)": data['lighthouseResult']['audits']['total-blocking-time']['displayValue'],
|
||||
"Cumulative Layout Shift (CLS)": data['lighthouseResult']['audits']['cumulative-layout-shift']['displayValue']
|
||||
}
|
||||
|
||||
st.table(pd.DataFrame(additional_metrics.items(), columns=["Metric", "Value"]))
|
||||
|
||||
# Display Network Requests
|
||||
st.subheader("Network Requests")
|
||||
if 'network-requests' in data['lighthouseResult']['audits']:
|
||||
network_requests = [
|
||||
{
|
||||
"End Time": item.get("endTime", "N/A"),
|
||||
"Start Time": item.get("startTime", "N/A"),
|
||||
"Transfer Size (MB)": round(item.get("transferSize", 0) / 1048576, 2),
|
||||
"Resource Size (MB)": round(item.get("resourceSize", 0) / 1048576, 2),
|
||||
"URL": item.get("url", "N/A")
|
||||
}
|
||||
for item in data["lighthouseResult"]["audits"]["network-requests"]["details"]["items"]
|
||||
if item.get("transferSize", 0) > 100000 or item.get("resourceSize", 0) > 100000
|
||||
]
|
||||
if network_requests:
|
||||
st.dataframe(pd.DataFrame(network_requests), use_container_width=True)
|
||||
else:
|
||||
st.write("No significant network requests found.")
|
||||
|
||||
# Display Mainthread Work Breakdown
|
||||
st.subheader("Mainthread Work Breakdown")
|
||||
if 'mainthread-work-breakdown' in data['lighthouseResult']['audits']:
|
||||
mainthread_data = [
|
||||
{"Process": item.get("groupLabel", "N/A"), "Duration (ms)": item.get("duration", "N/A")}
|
||||
for item in data["lighthouseResult"]["audits"]["mainthread-work-breakdown"]["details"]["items"] if item.get("duration", "N/A") != "N/A"
|
||||
]
|
||||
if mainthread_data:
|
||||
fig = px.bar(pd.DataFrame(mainthread_data), x="Process", y="Duration (ms)", title="Mainthread Work Breakdown", labels={"Process": "Process", "Duration (ms)": "Duration (ms)"})
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
else:
|
||||
st.write("No significant main thread work breakdown data found.")
|
||||
|
||||
# Display other metrics
|
||||
metrics = [
|
||||
("Use of Passive Event Listeners", 'uses-passive-event-listeners', ["URL", "Code Line"]),
|
||||
("DOM Size", 'dom-size', ["Score", "DOM Size"]),
|
||||
("Offscreen Images", 'offscreen-images', ["URL", "Total Bytes", "Wasted Bytes", "Wasted Percentage"]),
|
||||
("Critical Request Chains", 'critical-request-chains', ["URL", "Start Time", "End Time", "Transfer Size", "Chain"]),
|
||||
("Total Bytes Weight", 'total-byte-weight', ["URL", "Total Bytes"]),
|
||||
("Render Blocking Resources", 'render-blocking-resources', ["URL", "Total Bytes", "Wasted Milliseconds"]),
|
||||
("Use of Rel Preload", 'uses-rel-preload', ["URL", "Wasted Milliseconds"])
|
||||
]
|
||||
|
||||
for metric_title, audit_key, columns in metrics:
|
||||
st.subheader(metric_title)
|
||||
if audit_key in data['lighthouseResult']['audits']:
|
||||
details = data['lighthouseResult']['audits'][audit_key].get("details", {}).get("items", [])
|
||||
if details:
|
||||
st.table(pd.DataFrame(details, columns=columns))
|
||||
else:
|
||||
st.write(f"No significant {metric_title.lower()} data found.")
|
||||
|
||||
def google_pagespeed_insights():
|
||||
st.markdown("<h1 style='text-align: center; color: #1565C0;'>PageSpeed Insights Analyzer</h1>", unsafe_allow_html=True)
|
||||
st.markdown("<h3 style='text-align: center;'>Get detailed insights into your website's performance! Powered by Google PageSpeed Insights <a href='https://developer.chrome.com/docs/lighthouse/overview/'>[Learn More]</a></h3>", unsafe_allow_html=True)
|
||||
|
||||
# User Input
|
||||
with st.form("pagespeed_form"):
|
||||
url = st.text_input("Enter Website URL", placeholder="https://www.example.com")
|
||||
api_key = st.text_input("Enter Google API Key (Optional)", placeholder="Your API Key", help="Get your API key here: [https://developers.google.com/speed/docs/insights/v5/get-started#key]")
|
||||
device = st.selectbox("Choose Device", ["Mobile", "Desktop"])
|
||||
locale = st.selectbox("Choose Locale", ["en", "fr", "es", "de", "ja"])
|
||||
categories = st.multiselect("Select Categories to Analyze", ['PERFORMANCE', 'ACCESSIBILITY', 'BEST_PRACTICES', 'SEO'], default=['PERFORMANCE', 'ACCESSIBILITY', 'BEST_PRACTICES', 'SEO'])
|
||||
|
||||
submitted = st.form_submit_button("Analyze")
|
||||
|
||||
if submitted:
|
||||
if not url:
|
||||
st.error("Please provide the website URL.")
|
||||
else:
|
||||
strategy = 'mobile' if device == "Mobile" else 'desktop'
|
||||
data = run_pagespeed(url, api_key, strategy=strategy, locale=locale)
|
||||
if data:
|
||||
display_results(data)
|
||||
else:
|
||||
st.error("Failed to retrieve PageSpeed Insights data.")
|
||||
864
ToBeMigrated/ai_seo_tools/google_search_console_integration.py
Normal file
864
ToBeMigrated/ai_seo_tools/google_search_console_integration.py
Normal file
@@ -0,0 +1,864 @@
|
||||
"""
|
||||
Google Search Console Integration for Enterprise SEO
|
||||
|
||||
Connects GSC data with AI-powered content strategy and keyword intelligence.
|
||||
Provides enterprise-level search performance insights and content recommendations.
|
||||
"""
|
||||
|
||||
import streamlit as st
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
from datetime import datetime, timedelta
|
||||
import json
|
||||
from loguru import logger
|
||||
import plotly.express as px
|
||||
import plotly.graph_objects as go
|
||||
from plotly.subplots import make_subplots
|
||||
|
||||
# Import AI modules
|
||||
from ..gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
|
||||
|
||||
class GoogleSearchConsoleAnalyzer:
|
||||
"""
|
||||
Enterprise Google Search Console analyzer with AI-powered insights.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the GSC analyzer."""
|
||||
self.gsc_client = None # Will be initialized when credentials are provided
|
||||
logger.info("Google Search Console Analyzer initialized")
|
||||
|
||||
def analyze_search_performance(self, site_url: str, date_range: int = 90) -> Dict[str, Any]:
|
||||
"""
|
||||
Analyze comprehensive search performance from GSC data.
|
||||
|
||||
Args:
|
||||
site_url: Website URL registered in GSC
|
||||
date_range: Number of days to analyze (default 90)
|
||||
|
||||
Returns:
|
||||
Comprehensive search performance analysis
|
||||
"""
|
||||
try:
|
||||
st.info("📊 Analyzing Google Search Console data...")
|
||||
|
||||
# Simulate GSC data for demonstration (replace with actual GSC API calls)
|
||||
search_data = self._get_mock_gsc_data(site_url, date_range)
|
||||
|
||||
# Perform comprehensive analysis
|
||||
analysis_results = {
|
||||
'site_url': site_url,
|
||||
'analysis_period': f"Last {date_range} days",
|
||||
'analysis_timestamp': datetime.utcnow().isoformat(),
|
||||
'performance_overview': self._analyze_performance_overview(search_data),
|
||||
'keyword_analysis': self._analyze_keyword_performance(search_data),
|
||||
'page_analysis': self._analyze_page_performance(search_data),
|
||||
'content_opportunities': self._identify_content_opportunities(search_data),
|
||||
'technical_insights': self._analyze_technical_seo_signals(search_data),
|
||||
'competitive_analysis': self._analyze_competitive_position(search_data),
|
||||
'ai_recommendations': self._generate_ai_recommendations(search_data)
|
||||
}
|
||||
|
||||
return analysis_results
|
||||
|
||||
except Exception as e:
|
||||
error_msg = f"Error analyzing search performance: {str(e)}"
|
||||
logger.error(error_msg, exc_info=True)
|
||||
return {'error': error_msg}
|
||||
|
||||
def _get_mock_gsc_data(self, site_url: str, days: int) -> Dict[str, pd.DataFrame]:
|
||||
"""
|
||||
Generate mock GSC data for demonstration.
|
||||
In production, this would fetch real data from GSC API.
|
||||
"""
|
||||
# Generate mock keyword data
|
||||
keywords_data = []
|
||||
sample_keywords = [
|
||||
"AI content creation", "SEO tools", "content optimization", "blog writing AI",
|
||||
"meta description generator", "keyword research", "technical SEO", "content strategy",
|
||||
"on-page optimization", "SERP analysis", "content gap analysis", "SEO audit"
|
||||
]
|
||||
|
||||
for keyword in sample_keywords:
|
||||
# Generate realistic performance data
|
||||
impressions = np.random.randint(100, 10000)
|
||||
clicks = int(impressions * np.random.uniform(0.02, 0.15)) # CTR between 2-15%
|
||||
position = np.random.uniform(3, 25)
|
||||
|
||||
keywords_data.append({
|
||||
'keyword': keyword,
|
||||
'impressions': impressions,
|
||||
'clicks': clicks,
|
||||
'ctr': (clicks / impressions) * 100,
|
||||
'position': position
|
||||
})
|
||||
|
||||
# Generate mock page data
|
||||
pages_data = []
|
||||
sample_pages = [
|
||||
"/blog/ai-content-creation-guide", "/tools/seo-analyzer", "/features/content-optimization",
|
||||
"/blog/technical-seo-checklist", "/tools/keyword-research", "/blog/content-strategy-2024",
|
||||
"/tools/meta-description-generator", "/blog/on-page-seo-guide", "/features/enterprise-seo"
|
||||
]
|
||||
|
||||
for page in sample_pages:
|
||||
impressions = np.random.randint(500, 5000)
|
||||
clicks = int(impressions * np.random.uniform(0.03, 0.12))
|
||||
position = np.random.uniform(5, 20)
|
||||
|
||||
pages_data.append({
|
||||
'page': page,
|
||||
'impressions': impressions,
|
||||
'clicks': clicks,
|
||||
'ctr': (clicks / impressions) * 100,
|
||||
'position': position
|
||||
})
|
||||
|
||||
# Generate time series data
|
||||
time_series_data = []
|
||||
for i in range(days):
|
||||
date = datetime.now() - timedelta(days=i)
|
||||
daily_clicks = np.random.randint(50, 500)
|
||||
daily_impressions = np.random.randint(1000, 8000)
|
||||
|
||||
time_series_data.append({
|
||||
'date': date.strftime('%Y-%m-%d'),
|
||||
'clicks': daily_clicks,
|
||||
'impressions': daily_impressions,
|
||||
'ctr': (daily_clicks / daily_impressions) * 100,
|
||||
'position': np.random.uniform(8, 15)
|
||||
})
|
||||
|
||||
return {
|
||||
'keywords': pd.DataFrame(keywords_data),
|
||||
'pages': pd.DataFrame(pages_data),
|
||||
'time_series': pd.DataFrame(time_series_data)
|
||||
}
|
||||
|
||||
def _analyze_performance_overview(self, search_data: Dict[str, pd.DataFrame]) -> Dict[str, Any]:
|
||||
"""Analyze overall search performance metrics."""
|
||||
keywords_df = search_data['keywords']
|
||||
time_series_df = search_data['time_series']
|
||||
|
||||
# Calculate totals and averages
|
||||
total_clicks = keywords_df['clicks'].sum()
|
||||
total_impressions = keywords_df['impressions'].sum()
|
||||
avg_ctr = (total_clicks / total_impressions) * 100 if total_impressions > 0 else 0
|
||||
avg_position = keywords_df['position'].mean()
|
||||
|
||||
# Calculate trends
|
||||
recent_clicks = time_series_df.head(7)['clicks'].mean()
|
||||
previous_clicks = time_series_df.tail(7)['clicks'].mean()
|
||||
clicks_trend = ((recent_clicks - previous_clicks) / previous_clicks * 100) if previous_clicks > 0 else 0
|
||||
|
||||
recent_impressions = time_series_df.head(7)['impressions'].mean()
|
||||
previous_impressions = time_series_df.tail(7)['impressions'].mean()
|
||||
impressions_trend = ((recent_impressions - previous_impressions) / previous_impressions * 100) if previous_impressions > 0 else 0
|
||||
|
||||
# Top performing keywords
|
||||
top_keywords = keywords_df.nlargest(5, 'clicks')[['keyword', 'clicks', 'impressions', 'position']].to_dict('records')
|
||||
|
||||
# Opportunity keywords (high impressions, low CTR)
|
||||
opportunity_keywords = keywords_df[
|
||||
(keywords_df['impressions'] > keywords_df['impressions'].median()) &
|
||||
(keywords_df['ctr'] < 3)
|
||||
].nlargest(5, 'impressions')[['keyword', 'impressions', 'ctr', 'position']].to_dict('records')
|
||||
|
||||
return {
|
||||
'total_clicks': int(total_clicks),
|
||||
'total_impressions': int(total_impressions),
|
||||
'avg_ctr': round(avg_ctr, 2),
|
||||
'avg_position': round(avg_position, 1),
|
||||
'clicks_trend': round(clicks_trend, 1),
|
||||
'impressions_trend': round(impressions_trend, 1),
|
||||
'top_keywords': top_keywords,
|
||||
'opportunity_keywords': opportunity_keywords
|
||||
}
|
||||
|
||||
def _analyze_keyword_performance(self, search_data: Dict[str, pd.DataFrame]) -> Dict[str, Any]:
|
||||
"""Analyze keyword performance and opportunities."""
|
||||
keywords_df = search_data['keywords']
|
||||
|
||||
# Keyword categorization
|
||||
high_volume_keywords = keywords_df[keywords_df['impressions'] > keywords_df['impressions'].quantile(0.8)]
|
||||
low_competition_keywords = keywords_df[keywords_df['position'] <= 10]
|
||||
optimization_opportunities = keywords_df[
|
||||
(keywords_df['position'] > 10) &
|
||||
(keywords_df['position'] <= 20) &
|
||||
(keywords_df['impressions'] > 100)
|
||||
]
|
||||
|
||||
# Content gap analysis
|
||||
missing_keywords = self._identify_missing_keywords(keywords_df)
|
||||
|
||||
# Seasonal trends analysis
|
||||
seasonal_insights = self._analyze_seasonal_trends(keywords_df)
|
||||
|
||||
return {
|
||||
'total_keywords': len(keywords_df),
|
||||
'high_volume_keywords': high_volume_keywords.to_dict('records'),
|
||||
'ranking_keywords': low_competition_keywords.to_dict('records'),
|
||||
'optimization_opportunities': optimization_opportunities.to_dict('records'),
|
||||
'missing_keywords': missing_keywords,
|
||||
'seasonal_insights': seasonal_insights,
|
||||
'keyword_distribution': {
|
||||
'positions_1_3': len(keywords_df[keywords_df['position'] <= 3]),
|
||||
'positions_4_10': len(keywords_df[(keywords_df['position'] > 3) & (keywords_df['position'] <= 10)]),
|
||||
'positions_11_20': len(keywords_df[(keywords_df['position'] > 10) & (keywords_df['position'] <= 20)]),
|
||||
'positions_21_plus': len(keywords_df[keywords_df['position'] > 20])
|
||||
}
|
||||
}
|
||||
|
||||
def _analyze_page_performance(self, search_data: Dict[str, pd.DataFrame]) -> Dict[str, Any]:
|
||||
"""Analyze page-level performance."""
|
||||
pages_df = search_data['pages']
|
||||
|
||||
# Top performing pages
|
||||
top_pages = pages_df.nlargest(10, 'clicks')
|
||||
|
||||
# Underperforming pages (high impressions, low clicks)
|
||||
underperforming_pages = pages_df[
|
||||
(pages_df['impressions'] > pages_df['impressions'].median()) &
|
||||
(pages_df['ctr'] < 2)
|
||||
].nlargest(5, 'impressions')
|
||||
|
||||
# Page type analysis
|
||||
page_types = self._categorize_pages(pages_df)
|
||||
|
||||
return {
|
||||
'top_pages': top_pages.to_dict('records'),
|
||||
'underperforming_pages': underperforming_pages.to_dict('records'),
|
||||
'page_types_performance': page_types,
|
||||
'total_pages': len(pages_df)
|
||||
}
|
||||
|
||||
def _identify_content_opportunities(self, search_data: Dict[str, pd.DataFrame]) -> List[Dict[str, Any]]:
|
||||
"""Identify content creation and optimization opportunities."""
|
||||
keywords_df = search_data['keywords']
|
||||
|
||||
opportunities = []
|
||||
|
||||
# High impression, low CTR keywords need content optimization
|
||||
low_ctr_keywords = keywords_df[
|
||||
(keywords_df['impressions'] > 500) &
|
||||
(keywords_df['ctr'] < 3)
|
||||
]
|
||||
|
||||
for _, keyword_row in low_ctr_keywords.iterrows():
|
||||
opportunities.append({
|
||||
'type': 'Content Optimization',
|
||||
'keyword': keyword_row['keyword'],
|
||||
'opportunity': f"Optimize existing content for '{keyword_row['keyword']}' to improve CTR from {keyword_row['ctr']:.1f}%",
|
||||
'potential_impact': 'High',
|
||||
'current_position': round(keyword_row['position'], 1),
|
||||
'impressions': int(keyword_row['impressions']),
|
||||
'priority': 'High' if keyword_row['impressions'] > 1000 else 'Medium'
|
||||
})
|
||||
|
||||
# Position 11-20 keywords need content improvement
|
||||
position_11_20 = keywords_df[
|
||||
(keywords_df['position'] > 10) &
|
||||
(keywords_df['position'] <= 20) &
|
||||
(keywords_df['impressions'] > 100)
|
||||
]
|
||||
|
||||
for _, keyword_row in position_11_20.iterrows():
|
||||
opportunities.append({
|
||||
'type': 'Content Enhancement',
|
||||
'keyword': keyword_row['keyword'],
|
||||
'opportunity': f"Enhance content for '{keyword_row['keyword']}' to move from position {keyword_row['position']:.1f} to first page",
|
||||
'potential_impact': 'Medium',
|
||||
'current_position': round(keyword_row['position'], 1),
|
||||
'impressions': int(keyword_row['impressions']),
|
||||
'priority': 'Medium'
|
||||
})
|
||||
|
||||
# Sort by potential impact and impressions
|
||||
opportunities = sorted(opportunities, key=lambda x: x['impressions'], reverse=True)
|
||||
|
||||
return opportunities[:10] # Top 10 opportunities
|
||||
|
||||
def _analyze_technical_seo_signals(self, search_data: Dict[str, pd.DataFrame]) -> Dict[str, Any]:
|
||||
"""Analyze technical SEO signals from search data."""
|
||||
keywords_df = search_data['keywords']
|
||||
pages_df = search_data['pages']
|
||||
|
||||
# Analyze performance patterns that might indicate technical issues
|
||||
technical_insights = {
|
||||
'crawl_issues_indicators': [],
|
||||
'mobile_performance': {},
|
||||
'core_web_vitals_impact': {},
|
||||
'indexing_insights': {}
|
||||
}
|
||||
|
||||
# Identify potential crawl issues
|
||||
very_low_impressions = keywords_df[keywords_df['impressions'] < 10]
|
||||
if len(very_low_impressions) > len(keywords_df) * 0.3: # If 30%+ have very low impressions
|
||||
technical_insights['crawl_issues_indicators'].append(
|
||||
"High percentage of keywords with very low impressions may indicate crawl or indexing issues"
|
||||
)
|
||||
|
||||
# Mobile performance indicators
|
||||
avg_mobile_position = keywords_df['position'].mean() # In real implementation, this would be mobile-specific
|
||||
technical_insights['mobile_performance'] = {
|
||||
'avg_mobile_position': round(avg_mobile_position, 1),
|
||||
'mobile_optimization_needed': avg_mobile_position > 15
|
||||
}
|
||||
|
||||
return technical_insights
|
||||
|
||||
def _analyze_competitive_position(self, search_data: Dict[str, pd.DataFrame]) -> Dict[str, Any]:
|
||||
"""Analyze competitive positioning based on search data."""
|
||||
keywords_df = search_data['keywords']
|
||||
|
||||
# Calculate competitive metrics
|
||||
dominant_keywords = len(keywords_df[keywords_df['position'] <= 3])
|
||||
competitive_keywords = len(keywords_df[(keywords_df['position'] > 3) & (keywords_df['position'] <= 10)])
|
||||
losing_keywords = len(keywords_df[keywords_df['position'] > 10])
|
||||
|
||||
competitive_strength = (dominant_keywords * 3 + competitive_keywords * 2 + losing_keywords * 1) / len(keywords_df)
|
||||
|
||||
return {
|
||||
'dominant_keywords': dominant_keywords,
|
||||
'competitive_keywords': competitive_keywords,
|
||||
'losing_keywords': losing_keywords,
|
||||
'competitive_strength_score': round(competitive_strength, 2),
|
||||
'market_position': self._determine_market_position(competitive_strength)
|
||||
}
|
||||
|
||||
def _generate_ai_recommendations(self, search_data: Dict[str, pd.DataFrame]) -> Dict[str, Any]:
|
||||
"""Generate AI-powered recommendations based on search data."""
|
||||
try:
|
||||
keywords_df = search_data['keywords']
|
||||
pages_df = search_data['pages']
|
||||
|
||||
# Prepare data summary for AI analysis
|
||||
top_keywords = keywords_df.nlargest(5, 'impressions')['keyword'].tolist()
|
||||
avg_position = keywords_df['position'].mean()
|
||||
total_impressions = keywords_df['impressions'].sum()
|
||||
total_clicks = keywords_df['clicks'].sum()
|
||||
avg_ctr = (total_clicks / total_impressions * 100) if total_impressions > 0 else 0
|
||||
|
||||
# Create comprehensive prompt for AI analysis
|
||||
ai_prompt = f"""
|
||||
Analyze this Google Search Console data and provide strategic SEO recommendations:
|
||||
|
||||
SEARCH PERFORMANCE SUMMARY:
|
||||
- Total Keywords Tracked: {len(keywords_df)}
|
||||
- Total Impressions: {total_impressions:,}
|
||||
- Total Clicks: {total_clicks:,}
|
||||
- Average CTR: {avg_ctr:.2f}%
|
||||
- Average Position: {avg_position:.1f}
|
||||
|
||||
TOP PERFORMING KEYWORDS:
|
||||
{', '.join(top_keywords)}
|
||||
|
||||
PERFORMANCE DISTRIBUTION:
|
||||
- Keywords ranking 1-3: {len(keywords_df[keywords_df['position'] <= 3])}
|
||||
- Keywords ranking 4-10: {len(keywords_df[(keywords_df['position'] > 3) & (keywords_df['position'] <= 10)])}
|
||||
- Keywords ranking 11-20: {len(keywords_df[(keywords_df['position'] > 10) & (keywords_df['position'] <= 20)])}
|
||||
- Keywords ranking 21+: {len(keywords_df[keywords_df['position'] > 20])}
|
||||
|
||||
TOP PAGES BY TRAFFIC:
|
||||
{pages_df.nlargest(3, 'clicks')['page'].tolist()}
|
||||
|
||||
Based on this data, provide:
|
||||
|
||||
1. IMMEDIATE OPTIMIZATION OPPORTUNITIES (0-30 days):
|
||||
- Specific keywords to optimize for better CTR
|
||||
- Pages that need content updates
|
||||
- Quick technical wins
|
||||
|
||||
2. CONTENT STRATEGY RECOMMENDATIONS (1-3 months):
|
||||
- New content topics based on keyword gaps
|
||||
- Content enhancement priorities
|
||||
- Internal linking opportunities
|
||||
|
||||
3. LONG-TERM SEO STRATEGY (3-12 months):
|
||||
- Market expansion opportunities
|
||||
- Authority building topics
|
||||
- Competitive positioning strategies
|
||||
|
||||
4. TECHNICAL SEO PRIORITIES:
|
||||
- Performance issues affecting rankings
|
||||
- Mobile optimization needs
|
||||
- Core Web Vitals improvements
|
||||
|
||||
Provide specific, actionable recommendations with expected impact and priority levels.
|
||||
"""
|
||||
|
||||
ai_analysis = llm_text_gen(
|
||||
ai_prompt,
|
||||
system_prompt="You are an enterprise SEO strategist analyzing Google Search Console data. Provide specific, data-driven recommendations that will improve search performance."
|
||||
)
|
||||
|
||||
return {
|
||||
'full_analysis': ai_analysis,
|
||||
'immediate_opportunities': self._extract_immediate_opportunities(ai_analysis),
|
||||
'content_strategy': self._extract_content_strategy(ai_analysis),
|
||||
'long_term_strategy': self._extract_long_term_strategy(ai_analysis),
|
||||
'technical_priorities': self._extract_technical_priorities(ai_analysis)
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"AI recommendations error: {str(e)}")
|
||||
return {'error': str(e)}
|
||||
|
||||
# Utility methods
|
||||
def _identify_missing_keywords(self, keywords_df: pd.DataFrame) -> List[str]:
|
||||
"""Identify potential missing keywords based on current keyword performance."""
|
||||
# In a real implementation, this would use keyword research APIs
|
||||
existing_keywords = set(keywords_df['keyword'].str.lower())
|
||||
|
||||
potential_keywords = [
|
||||
"AI writing tools", "content automation", "SEO content generator",
|
||||
"blog post optimizer", "meta tag generator", "keyword analyzer"
|
||||
]
|
||||
|
||||
missing = [kw for kw in potential_keywords if kw.lower() not in existing_keywords]
|
||||
return missing[:5]
|
||||
|
||||
def _analyze_seasonal_trends(self, keywords_df: pd.DataFrame) -> Dict[str, Any]:
|
||||
"""Analyze seasonal trends in keyword performance."""
|
||||
# Placeholder for seasonal analysis
|
||||
return {
|
||||
'seasonal_keywords': [],
|
||||
'trend_analysis': "Seasonal analysis requires historical data spanning multiple seasons"
|
||||
}
|
||||
|
||||
def _categorize_pages(self, pages_df: pd.DataFrame) -> Dict[str, Any]:
|
||||
"""Categorize pages by type and analyze performance."""
|
||||
page_types = {
|
||||
'Blog Posts': {'count': 0, 'total_clicks': 0, 'avg_position': 0},
|
||||
'Product Pages': {'count': 0, 'total_clicks': 0, 'avg_position': 0},
|
||||
'Tool Pages': {'count': 0, 'total_clicks': 0, 'avg_position': 0},
|
||||
'Other': {'count': 0, 'total_clicks': 0, 'avg_position': 0}
|
||||
}
|
||||
|
||||
for _, page_row in pages_df.iterrows():
|
||||
page_url = page_row['page']
|
||||
clicks = page_row['clicks']
|
||||
position = page_row['position']
|
||||
|
||||
if '/blog/' in page_url:
|
||||
page_types['Blog Posts']['count'] += 1
|
||||
page_types['Blog Posts']['total_clicks'] += clicks
|
||||
page_types['Blog Posts']['avg_position'] += position
|
||||
elif '/tools/' in page_url:
|
||||
page_types['Tool Pages']['count'] += 1
|
||||
page_types['Tool Pages']['total_clicks'] += clicks
|
||||
page_types['Tool Pages']['avg_position'] += position
|
||||
elif '/features/' in page_url or '/product/' in page_url:
|
||||
page_types['Product Pages']['count'] += 1
|
||||
page_types['Product Pages']['total_clicks'] += clicks
|
||||
page_types['Product Pages']['avg_position'] += position
|
||||
else:
|
||||
page_types['Other']['count'] += 1
|
||||
page_types['Other']['total_clicks'] += clicks
|
||||
page_types['Other']['avg_position'] += position
|
||||
|
||||
# Calculate averages
|
||||
for page_type in page_types:
|
||||
if page_types[page_type]['count'] > 0:
|
||||
page_types[page_type]['avg_position'] = round(
|
||||
page_types[page_type]['avg_position'] / page_types[page_type]['count'], 1
|
||||
)
|
||||
|
||||
return page_types
|
||||
|
||||
def _determine_market_position(self, competitive_strength: float) -> str:
|
||||
"""Determine market position based on competitive strength score."""
|
||||
if competitive_strength >= 2.5:
|
||||
return "Market Leader"
|
||||
elif competitive_strength >= 2.0:
|
||||
return "Strong Competitor"
|
||||
elif competitive_strength >= 1.5:
|
||||
return "Emerging Player"
|
||||
else:
|
||||
return "Challenger"
|
||||
|
||||
def _extract_immediate_opportunities(self, analysis: str) -> List[str]:
|
||||
"""Extract immediate opportunities from AI analysis."""
|
||||
lines = analysis.split('\n')
|
||||
opportunities = []
|
||||
in_immediate_section = False
|
||||
|
||||
for line in lines:
|
||||
if 'IMMEDIATE OPTIMIZATION' in line.upper():
|
||||
in_immediate_section = True
|
||||
continue
|
||||
elif 'CONTENT STRATEGY' in line.upper():
|
||||
in_immediate_section = False
|
||||
continue
|
||||
|
||||
if in_immediate_section and line.strip().startswith('-'):
|
||||
opportunities.append(line.strip().lstrip('- '))
|
||||
|
||||
return opportunities[:5]
|
||||
|
||||
def _extract_content_strategy(self, analysis: str) -> List[str]:
|
||||
"""Extract content strategy recommendations from AI analysis."""
|
||||
return ["Develop topic clusters", "Create comparison content", "Build FAQ sections"]
|
||||
|
||||
def _extract_long_term_strategy(self, analysis: str) -> List[str]:
|
||||
"""Extract long-term strategy from AI analysis."""
|
||||
return ["Build domain authority", "Expand to new markets", "Develop thought leadership content"]
|
||||
|
||||
def _extract_technical_priorities(self, analysis: str) -> List[str]:
|
||||
"""Extract technical priorities from AI analysis."""
|
||||
return ["Improve page speed", "Optimize mobile experience", "Fix crawl errors"]
|
||||
|
||||
|
||||
def render_gsc_integration():
|
||||
"""Render the Google Search Console integration interface."""
|
||||
|
||||
st.title("📊 Google Search Console Intelligence")
|
||||
st.markdown("**AI-powered insights from your Google Search Console data**")
|
||||
|
||||
# Initialize analyzer
|
||||
if 'gsc_analyzer' not in st.session_state:
|
||||
st.session_state.gsc_analyzer = GoogleSearchConsoleAnalyzer()
|
||||
|
||||
analyzer = st.session_state.gsc_analyzer
|
||||
|
||||
# Configuration section
|
||||
st.header("🔧 Configuration")
|
||||
|
||||
with st.expander("📋 Setup Instructions", expanded=False):
|
||||
st.markdown("""
|
||||
### Setting up Google Search Console Integration
|
||||
|
||||
1. **Verify your website** in Google Search Console
|
||||
2. **Enable the Search Console API** in Google Cloud Console
|
||||
3. **Create service account credentials** and download the JSON file
|
||||
4. **Upload credentials** using the file uploader below
|
||||
|
||||
📚 [Detailed Setup Guide](https://developers.google.com/webmaster-tools/search-console-api-original/v3/prereqs)
|
||||
""")
|
||||
|
||||
# Input form
|
||||
with st.form("gsc_analysis_form"):
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
site_url = st.text_input(
|
||||
"Site URL",
|
||||
value="https://example.com",
|
||||
help="Enter your website URL as registered in Google Search Console"
|
||||
)
|
||||
|
||||
date_range = st.selectbox(
|
||||
"Analysis Period",
|
||||
[30, 60, 90, 180],
|
||||
index=2,
|
||||
help="Number of days to analyze"
|
||||
)
|
||||
|
||||
with col2:
|
||||
# Credentials upload (placeholder)
|
||||
credentials_file = st.file_uploader(
|
||||
"GSC API Credentials (JSON)",
|
||||
type=['json'],
|
||||
help="Upload your Google Search Console API credentials file"
|
||||
)
|
||||
|
||||
demo_mode = st.checkbox(
|
||||
"Demo Mode",
|
||||
value=True,
|
||||
help="Use demo data for testing (no credentials needed)"
|
||||
)
|
||||
|
||||
submit_analysis = st.form_submit_button("📊 Analyze Search Performance", type="primary")
|
||||
|
||||
# Process analysis
|
||||
if submit_analysis:
|
||||
if site_url and (demo_mode or credentials_file):
|
||||
with st.spinner("📊 Analyzing Google Search Console data..."):
|
||||
analysis_results = analyzer.analyze_search_performance(site_url, date_range)
|
||||
|
||||
if 'error' not in analysis_results:
|
||||
st.success("✅ Search Console analysis completed!")
|
||||
|
||||
# Store results in session state
|
||||
st.session_state.gsc_results = analysis_results
|
||||
|
||||
# Display results
|
||||
render_gsc_results_dashboard(analysis_results)
|
||||
else:
|
||||
st.error(f"❌ Analysis failed: {analysis_results['error']}")
|
||||
else:
|
||||
st.warning("⚠️ Please enter site URL and upload credentials (or enable demo mode).")
|
||||
|
||||
# Show previous results if available
|
||||
elif 'gsc_results' in st.session_state:
|
||||
st.info("📊 Showing previous analysis results")
|
||||
render_gsc_results_dashboard(st.session_state.gsc_results)
|
||||
|
||||
|
||||
def render_gsc_results_dashboard(results: Dict[str, Any]):
|
||||
"""Render comprehensive GSC analysis results."""
|
||||
|
||||
# Performance overview
|
||||
st.header("📊 Search Performance Overview")
|
||||
|
||||
overview = results['performance_overview']
|
||||
|
||||
col1, col2, col3, col4 = st.columns(4)
|
||||
|
||||
with col1:
|
||||
st.metric(
|
||||
"Total Clicks",
|
||||
f"{overview['total_clicks']:,}",
|
||||
delta=f"{overview['clicks_trend']:+.1f}%" if overview['clicks_trend'] != 0 else None
|
||||
)
|
||||
|
||||
with col2:
|
||||
st.metric(
|
||||
"Total Impressions",
|
||||
f"{overview['total_impressions']:,}",
|
||||
delta=f"{overview['impressions_trend']:+.1f}%" if overview['impressions_trend'] != 0 else None
|
||||
)
|
||||
|
||||
with col3:
|
||||
st.metric(
|
||||
"Average CTR",
|
||||
f"{overview['avg_ctr']:.2f}%"
|
||||
)
|
||||
|
||||
with col4:
|
||||
st.metric(
|
||||
"Average Position",
|
||||
f"{overview['avg_position']:.1f}"
|
||||
)
|
||||
|
||||
# Content opportunities (Most important section)
|
||||
st.header("🎯 Content Opportunities")
|
||||
|
||||
opportunities = results['content_opportunities']
|
||||
if opportunities:
|
||||
# Display as interactive table
|
||||
df_opportunities = pd.DataFrame(opportunities)
|
||||
|
||||
st.dataframe(
|
||||
df_opportunities,
|
||||
column_config={
|
||||
"type": "Opportunity Type",
|
||||
"keyword": "Keyword",
|
||||
"opportunity": "Description",
|
||||
"potential_impact": st.column_config.SelectboxColumn(
|
||||
"Impact",
|
||||
options=["High", "Medium", "Low"]
|
||||
),
|
||||
"current_position": st.column_config.NumberColumn(
|
||||
"Current Position",
|
||||
format="%.1f"
|
||||
),
|
||||
"impressions": st.column_config.NumberColumn(
|
||||
"Impressions",
|
||||
format="%d"
|
||||
),
|
||||
"priority": st.column_config.SelectboxColumn(
|
||||
"Priority",
|
||||
options=["High", "Medium", "Low"]
|
||||
)
|
||||
},
|
||||
hide_index=True,
|
||||
use_container_width=True
|
||||
)
|
||||
|
||||
# Detailed analysis tabs
|
||||
tab1, tab2, tab3, tab4, tab5 = st.tabs([
|
||||
"🤖 AI Insights",
|
||||
"🎯 Keyword Analysis",
|
||||
"📄 Page Performance",
|
||||
"🏆 Competitive Position",
|
||||
"🔧 Technical Signals"
|
||||
])
|
||||
|
||||
with tab1:
|
||||
ai_recs = results.get('ai_recommendations', {})
|
||||
if ai_recs and 'error' not in ai_recs:
|
||||
st.subheader("AI-Powered Recommendations")
|
||||
|
||||
# Immediate opportunities
|
||||
immediate_ops = ai_recs.get('immediate_opportunities', [])
|
||||
if immediate_ops:
|
||||
st.markdown("#### 🚀 Immediate Optimizations (0-30 days)")
|
||||
for op in immediate_ops:
|
||||
st.success(f"✅ {op}")
|
||||
|
||||
# Content strategy
|
||||
content_strategy = ai_recs.get('content_strategy', [])
|
||||
if content_strategy:
|
||||
st.markdown("#### 📝 Content Strategy (1-3 months)")
|
||||
for strategy in content_strategy:
|
||||
st.info(f"📋 {strategy}")
|
||||
|
||||
# Full analysis
|
||||
full_analysis = ai_recs.get('full_analysis', '')
|
||||
if full_analysis:
|
||||
with st.expander("🧠 Complete AI Analysis"):
|
||||
st.write(full_analysis)
|
||||
|
||||
with tab2:
|
||||
keyword_analysis = results.get('keyword_analysis', {})
|
||||
if keyword_analysis:
|
||||
st.subheader("Keyword Performance Analysis")
|
||||
|
||||
# Keyword distribution chart
|
||||
dist = keyword_analysis['keyword_distribution']
|
||||
fig = px.pie(
|
||||
values=[dist['positions_1_3'], dist['positions_4_10'], dist['positions_11_20'], dist['positions_21_plus']],
|
||||
names=['Positions 1-3', 'Positions 4-10', 'Positions 11-20', 'Positions 21+'],
|
||||
title="Keyword Position Distribution"
|
||||
)
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
# High volume keywords
|
||||
high_volume = keyword_analysis.get('high_volume_keywords', [])
|
||||
if high_volume:
|
||||
st.markdown("#### 📈 High Volume Keywords")
|
||||
st.dataframe(pd.DataFrame(high_volume), hide_index=True)
|
||||
|
||||
# Optimization opportunities
|
||||
opt_opportunities = keyword_analysis.get('optimization_opportunities', [])
|
||||
if opt_opportunities:
|
||||
st.markdown("#### 🎯 Optimization Opportunities (Positions 11-20)")
|
||||
st.dataframe(pd.DataFrame(opt_opportunities), hide_index=True)
|
||||
|
||||
with tab3:
|
||||
page_analysis = results.get('page_analysis', {})
|
||||
if page_analysis:
|
||||
st.subheader("Page Performance Analysis")
|
||||
|
||||
# Top pages
|
||||
top_pages = page_analysis.get('top_pages', [])
|
||||
if top_pages:
|
||||
st.markdown("#### 🏆 Top Performing Pages")
|
||||
st.dataframe(pd.DataFrame(top_pages), hide_index=True)
|
||||
|
||||
# Underperforming pages
|
||||
underperforming = page_analysis.get('underperforming_pages', [])
|
||||
if underperforming:
|
||||
st.markdown("#### ⚠️ Underperforming Pages (High Impressions, Low CTR)")
|
||||
st.dataframe(pd.DataFrame(underperforming), hide_index=True)
|
||||
|
||||
# Page types performance
|
||||
page_types = page_analysis.get('page_types_performance', {})
|
||||
if page_types:
|
||||
st.markdown("#### 📊 Performance by Page Type")
|
||||
|
||||
# Create visualization
|
||||
types = []
|
||||
clicks = []
|
||||
positions = []
|
||||
|
||||
for page_type, data in page_types.items():
|
||||
if data['count'] > 0:
|
||||
types.append(page_type)
|
||||
clicks.append(data['total_clicks'])
|
||||
positions.append(data['avg_position'])
|
||||
|
||||
if types:
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
fig_clicks = px.bar(x=types, y=clicks, title="Total Clicks by Page Type")
|
||||
st.plotly_chart(fig_clicks, use_container_width=True)
|
||||
|
||||
with col2:
|
||||
fig_position = px.bar(x=types, y=positions, title="Average Position by Page Type")
|
||||
st.plotly_chart(fig_position, use_container_width=True)
|
||||
|
||||
with tab4:
|
||||
competitive_analysis = results.get('competitive_analysis', {})
|
||||
if competitive_analysis:
|
||||
st.subheader("Competitive Position Analysis")
|
||||
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
st.metric("Market Position", competitive_analysis['market_position'])
|
||||
st.metric("Competitive Strength", f"{competitive_analysis['competitive_strength_score']}/3.0")
|
||||
|
||||
with col2:
|
||||
# Competitive distribution
|
||||
comp_data = {
|
||||
'Dominant (1-3)': competitive_analysis['dominant_keywords'],
|
||||
'Competitive (4-10)': competitive_analysis['competitive_keywords'],
|
||||
'Losing (11+)': competitive_analysis['losing_keywords']
|
||||
}
|
||||
|
||||
fig = px.bar(
|
||||
x=list(comp_data.keys()),
|
||||
y=list(comp_data.values()),
|
||||
title="Keyword Competitive Position"
|
||||
)
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
with tab5:
|
||||
technical_insights = results.get('technical_insights', {})
|
||||
if technical_insights:
|
||||
st.subheader("Technical SEO Signals")
|
||||
|
||||
# Crawl issues indicators
|
||||
crawl_issues = technical_insights.get('crawl_issues_indicators', [])
|
||||
if crawl_issues:
|
||||
st.markdown("#### ⚠️ Potential Issues")
|
||||
for issue in crawl_issues:
|
||||
st.warning(f"🚨 {issue}")
|
||||
|
||||
# Mobile performance
|
||||
mobile_perf = technical_insights.get('mobile_performance', {})
|
||||
if mobile_perf:
|
||||
st.markdown("#### 📱 Mobile Performance")
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
st.metric("Avg Mobile Position", f"{mobile_perf.get('avg_mobile_position', 0):.1f}")
|
||||
|
||||
with col2:
|
||||
if mobile_perf.get('mobile_optimization_needed', False):
|
||||
st.warning("📱 Mobile optimization needed")
|
||||
else:
|
||||
st.success("📱 Mobile performance good")
|
||||
|
||||
# Export functionality
|
||||
st.markdown("---")
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
if st.button("📥 Export Full Report", use_container_width=True):
|
||||
report_json = json.dumps(results, indent=2, default=str)
|
||||
st.download_button(
|
||||
label="Download JSON Report",
|
||||
data=report_json,
|
||||
file_name=f"gsc_analysis_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json",
|
||||
mime="application/json"
|
||||
)
|
||||
|
||||
with col2:
|
||||
if st.button("📊 Export Opportunities", use_container_width=True):
|
||||
if opportunities:
|
||||
df_opportunities = pd.DataFrame(opportunities)
|
||||
csv = df_opportunities.to_csv(index=False)
|
||||
st.download_button(
|
||||
label="Download CSV Opportunities",
|
||||
data=csv,
|
||||
file_name=f"content_opportunities_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv",
|
||||
mime="text/csv"
|
||||
)
|
||||
|
||||
with col3:
|
||||
if st.button("🔄 Refresh Analysis", use_container_width=True):
|
||||
# Clear cached results to force refresh
|
||||
if 'gsc_results' in st.session_state:
|
||||
del st.session_state.gsc_results
|
||||
st.rerun()
|
||||
|
||||
|
||||
# Main execution
|
||||
if __name__ == "__main__":
|
||||
render_gsc_integration()
|
||||
112
ToBeMigrated/ai_seo_tools/image_alt_text_generator.py
Normal file
112
ToBeMigrated/ai_seo_tools/image_alt_text_generator.py
Normal file
@@ -0,0 +1,112 @@
|
||||
import streamlit as st
|
||||
import base64
|
||||
import requests
|
||||
from PIL import Image
|
||||
import os
|
||||
|
||||
|
||||
def encode_image(image_path):
|
||||
"""
|
||||
Encodes an image to base64 format.
|
||||
|
||||
Args:
|
||||
image_path (str): Path to the image file.
|
||||
|
||||
Returns:
|
||||
str: Base64 encoded string of the image.
|
||||
|
||||
Raises:
|
||||
ValueError: If the image path is invalid.
|
||||
"""
|
||||
safe_root = os.getenv('SAFE_ROOT_DIRECTORY', '/safe/root/directory') # Use an environment variable for the safe root directory
|
||||
normalized_path = os.path.normpath(image_path)
|
||||
if not normalized_path.startswith(safe_root):
|
||||
raise ValueError("Invalid image path")
|
||||
with open(normalized_path, "rb") as image_file:
|
||||
return base64.b64encode(image_file.read()).decode('utf-8')
|
||||
|
||||
|
||||
def get_image_description(image_path):
|
||||
"""
|
||||
Generates a description for the given image using an external API.
|
||||
|
||||
Args:
|
||||
image_path (str): Path to the image file.
|
||||
|
||||
Returns:
|
||||
str: Description of the image.
|
||||
|
||||
Raises:
|
||||
ValueError: If the image path is invalid.
|
||||
"""
|
||||
safe_root = os.getenv('SAFE_ROOT_DIRECTORY', '/safe/root/directory') # Use an environment variable for the safe root directory
|
||||
normalized_path = os.path.normpath(image_path)
|
||||
if not normalized_path.startswith(safe_root):
|
||||
raise ValueError("Invalid image path")
|
||||
base64_image = encode_image(normalized_path)
|
||||
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {os.getenv('OPENAI_API_KEY')}"
|
||||
}
|
||||
|
||||
payload = {
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": """You are an SEO expert specializing in writing optimized Alt text for images.
|
||||
Your goal is to create clear, descriptive, and concise Alt text that accurately represents
|
||||
the content and context of the given image. Make sure your response is optimized for search engines and accessibility."""
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": f"data:image/jpeg;base64,{base64_image}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"max_tokens": 300
|
||||
}
|
||||
|
||||
response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload)
|
||||
response_data = response.json()
|
||||
|
||||
# Extract the content field from the response
|
||||
content = response_data['choices'][0]['message']['content']
|
||||
return content
|
||||
|
||||
|
||||
def alt_text_gen():
|
||||
"""
|
||||
Streamlit app function to generate Alt text for an uploaded image.
|
||||
"""
|
||||
st.title("Image Description Generator")
|
||||
|
||||
image_path = st.text_input("Enter the full path of the image file", help="Provide the full path to a .jpg, .jpeg, or .png image file")
|
||||
|
||||
if image_path:
|
||||
if os.path.exists(image_path) and image_path.lower().endswith(('jpg', 'jpeg', 'png')):
|
||||
try:
|
||||
image = Image.open(image_path)
|
||||
st.image(image, caption='Uploaded Image', use_column_width=True)
|
||||
|
||||
if st.button("Get Image Alt Text"):
|
||||
with st.spinner("Generating Alt Text..."):
|
||||
try:
|
||||
description = get_image_description(image_path)
|
||||
st.success("Alt Text generated successfully!")
|
||||
st.write("Alt Text:", description)
|
||||
except Exception as e:
|
||||
st.error(f"Error generating description: {e}")
|
||||
except Exception as e:
|
||||
st.error(f"Error processing image: {e}")
|
||||
else:
|
||||
st.error("Please enter a valid image file path ending with .jpg, .jpeg, or .png")
|
||||
else:
|
||||
st.info("Please enter the full path of an image file.")
|
||||
110
ToBeMigrated/ai_seo_tools/meta_desc_generator.py
Normal file
110
ToBeMigrated/ai_seo_tools/meta_desc_generator.py
Normal file
@@ -0,0 +1,110 @@
|
||||
import os
|
||||
import json
|
||||
import streamlit as st
|
||||
from tenacity import retry, stop_after_attempt, wait_random_exponential
|
||||
from loguru import logger
|
||||
import sys
|
||||
|
||||
from ..gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
|
||||
|
||||
def metadesc_generator_main():
|
||||
"""
|
||||
Streamlit app for generating SEO-optimized blog meta descriptions.
|
||||
"""
|
||||
st.title("✍️ Alwrity - AI Blog Meta Description Generator")
|
||||
st.markdown(
|
||||
"Create compelling, SEO-optimized meta descriptions in just a few clicks. Perfect for enhancing your blog's click-through rates!"
|
||||
)
|
||||
|
||||
# Input section
|
||||
with st.expander("**PRO-TIP** - Read the instructions below. 🚀", expanded=True):
|
||||
col1, col2, _ = st.columns([5, 5, 0.5])
|
||||
|
||||
# Column 1: Keywords and Tone
|
||||
with col1:
|
||||
keywords = st.text_input(
|
||||
"🔑 Target Keywords (comma-separated):",
|
||||
placeholder="e.g., content marketing, SEO, social media, online business",
|
||||
help="Enter your target keywords, separated by commas. 📝",
|
||||
)
|
||||
|
||||
tone_options = ["General", "Informative", "Engaging", "Humorous", "Intriguing", "Playful"]
|
||||
tone = st.selectbox(
|
||||
"🎨 Desired Tone (optional):",
|
||||
options=tone_options,
|
||||
help="Choose the overall tone you want for your meta description. 🎭",
|
||||
)
|
||||
|
||||
# Column 2: Search Intent and Language
|
||||
with col2:
|
||||
search_type = st.selectbox(
|
||||
"🔍 Search Intent:",
|
||||
("Informational Intent", "Commercial Intent", "Transactional Intent", "Navigational Intent"),
|
||||
index=0,
|
||||
)
|
||||
|
||||
language_options = ["English", "Spanish", "French", "German", "Other"]
|
||||
language_choice = st.selectbox(
|
||||
"🌐 Preferred Language:",
|
||||
options=language_options,
|
||||
help="Select the language for your meta description. 🗣️",
|
||||
)
|
||||
|
||||
language = (
|
||||
st.text_input(
|
||||
"Specify Other Language:",
|
||||
placeholder="e.g., Italian, Chinese",
|
||||
help="Enter your preferred language. 🌍",
|
||||
)
|
||||
if language_choice == "Other"
|
||||
else language_choice
|
||||
)
|
||||
|
||||
# Generate Meta Description button
|
||||
if st.button("**✨ Generate Meta Description ✨**"):
|
||||
if not keywords.strip():
|
||||
st.error("**🫣 Target Keywords are required! Please provide at least one keyword.**")
|
||||
return
|
||||
|
||||
with st.spinner("Crafting your Meta descriptions... ⏳"):
|
||||
blog_metadesc = generate_blog_metadesc(keywords, tone, search_type, language)
|
||||
if blog_metadesc:
|
||||
st.success("**🎉 Meta Descriptions Generated Successfully! 🚀**")
|
||||
with st.expander("**Your SEO-Boosting Blog Meta Descriptions 🎆🎇**", expanded=True):
|
||||
st.markdown(blog_metadesc)
|
||||
else:
|
||||
st.error("💥 **Failed to generate blog meta description. Please try again!**")
|
||||
|
||||
|
||||
def generate_blog_metadesc(keywords, tone, search_type, language):
|
||||
"""
|
||||
Generate blog meta descriptions using LLM.
|
||||
|
||||
Args:
|
||||
keywords (str): Comma-separated target keywords.
|
||||
tone (str): Desired tone for the meta description.
|
||||
search_type (str): Search intent type.
|
||||
language (str): Preferred language for the description.
|
||||
|
||||
Returns:
|
||||
str: Generated meta descriptions or error message.
|
||||
"""
|
||||
prompt = f"""
|
||||
Craft 3 engaging and SEO-friendly meta descriptions for a blog post based on the following details:
|
||||
|
||||
Blog Post Keywords: {keywords}
|
||||
Search Intent Type: {search_type}
|
||||
Desired Tone: {tone}
|
||||
Preferred Language: {language}
|
||||
|
||||
Output Format:
|
||||
|
||||
Respond with 3 compelling and concise meta descriptions, approximately 155-160 characters long, that incorporate the target keywords, reflect the blog post content, resonate with the target audience, and entice users to click through to read the full article.
|
||||
"""
|
||||
try:
|
||||
return llm_text_gen(prompt)
|
||||
except Exception as err:
|
||||
logger.error(f"Error generating meta description: {err}")
|
||||
st.error(f"💥 Error: Failed to generate response from LLM: {err}")
|
||||
return None
|
||||
1070
ToBeMigrated/ai_seo_tools/on_page_seo_analyzer.py
Normal file
1070
ToBeMigrated/ai_seo_tools/on_page_seo_analyzer.py
Normal file
File diff suppressed because it is too large
Load Diff
129
ToBeMigrated/ai_seo_tools/opengraph_generator.py
Normal file
129
ToBeMigrated/ai_seo_tools/opengraph_generator.py
Normal file
@@ -0,0 +1,129 @@
|
||||
import streamlit as st
|
||||
import requests
|
||||
from bs4 import BeautifulSoup
|
||||
from ..gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
|
||||
|
||||
def generate_og_tags(url, title_hint, description_hint, platform="General"):
|
||||
"""
|
||||
Generate Open Graph tags based on the provided URL, title hint, description hint, and platform.
|
||||
|
||||
Args:
|
||||
url (str): The URL of the webpage.
|
||||
title_hint (str): A hint for the title.
|
||||
description_hint (str): A hint for the description.
|
||||
platform (str): The platform for which to generate the tags (General, Facebook, or Twitter).
|
||||
|
||||
Returns:
|
||||
str: The generated Open Graph tags or an error message.
|
||||
"""
|
||||
# Create a prompt for the text generation model
|
||||
prompt = (
|
||||
f"Generate Open Graph tags for the following page:\nURL: {url}\n"
|
||||
f"Title hint: {title_hint}\nDescription hint: {description_hint}"
|
||||
)
|
||||
if platform == "Facebook":
|
||||
prompt += "\nSpecifically for Facebook"
|
||||
elif platform == "Twitter":
|
||||
prompt += "\nSpecifically for Twitter"
|
||||
|
||||
try:
|
||||
# Generate Open Graph tags using the text generation model
|
||||
response = llm_text_gen(prompt)
|
||||
return response
|
||||
except Exception as err:
|
||||
st.error(f"Failed to generate Open Graph tags: {err}")
|
||||
return None
|
||||
|
||||
|
||||
def extract_default_og_tags(url):
|
||||
"""
|
||||
Extract default Open Graph tags from the provided URL.
|
||||
|
||||
Args:
|
||||
url (str): The URL of the webpage.
|
||||
|
||||
Returns:
|
||||
tuple: A tuple containing the title, description, and image URL, or None in case of an error.
|
||||
"""
|
||||
try:
|
||||
# Fetch the HTML content of the URL
|
||||
response = requests.get(url)
|
||||
response.raise_for_status()
|
||||
|
||||
# Parse the HTML content using BeautifulSoup
|
||||
soup = BeautifulSoup(response.content, 'html.parser')
|
||||
|
||||
# Extract the title, description, and image URL
|
||||
title = soup.find('title').text if soup.find('title') else None
|
||||
description = soup.find('meta', attrs={'name': 'description'})['content'] if soup.find('meta', attrs={'name': 'description'}) else None
|
||||
image_url = soup.find('meta', attrs={'property': 'og:image'})['content'] if soup.find('meta', attrs={'property': 'og:image'}) else None
|
||||
|
||||
return title, description, image_url
|
||||
|
||||
except requests.exceptions.RequestException as req_err:
|
||||
st.error(f"Error fetching the URL: {req_err}")
|
||||
return None, None, None
|
||||
|
||||
except Exception as err:
|
||||
st.error(f"Error parsing the HTML content: {err}")
|
||||
return None, None, None
|
||||
|
||||
|
||||
def og_tag_generator():
|
||||
"""Main function to run the Streamlit app."""
|
||||
st.title("AI Open Graph Tag Generator")
|
||||
|
||||
# Platform selection
|
||||
platform = st.selectbox(
|
||||
"**Select the platform**",
|
||||
["General", "Facebook", "Twitter"],
|
||||
help="Choose the platform for which you want to generate Open Graph tags."
|
||||
)
|
||||
|
||||
# URL input
|
||||
url = st.text_input(
|
||||
"**Enter the URL of the page to generate Open Graph tags for:**",
|
||||
placeholder="e.g., https://example.com",
|
||||
help="Provide the URL of the page you want to generate Open Graph tags for."
|
||||
)
|
||||
|
||||
if url:
|
||||
# Extract default Open Graph tags
|
||||
title, description, image_url = extract_default_og_tags(url)
|
||||
|
||||
# Title hint input
|
||||
title_hint = st.text_input(
|
||||
"**Modify existing title or suggest a new one (optional):**",
|
||||
value=title if title else "",
|
||||
placeholder="e.g., Amazing Blog Post Title"
|
||||
)
|
||||
|
||||
# Description hint input
|
||||
description_hint = st.text_area(
|
||||
"**Modify existing description or suggest a new one (optional):**",
|
||||
value=description if description else "",
|
||||
placeholder="e.g., This is a detailed description of the content."
|
||||
)
|
||||
|
||||
# Image URL hint input
|
||||
image_hint = st.text_input(
|
||||
"**Use this image or suggest a new URL (optional):**",
|
||||
value=image_url if image_url else "",
|
||||
placeholder="e.g., https://example.com/image.jpg"
|
||||
)
|
||||
|
||||
# Generate Open Graph tags
|
||||
if st.button("Generate Open Graph Tags"):
|
||||
with st.spinner("Generating Open Graph tags..."):
|
||||
try:
|
||||
og_tags = generate_og_tags(url, title_hint, description_hint, platform)
|
||||
if og_tags:
|
||||
st.success("Open Graph tags generated successfully!")
|
||||
st.markdown(og_tags)
|
||||
else:
|
||||
st.error("Failed to generate Open Graph tags.")
|
||||
except Exception as e:
|
||||
st.error(f"Failed to generate Open Graph tags: {e}")
|
||||
else:
|
||||
st.info("Please enter a URL to generate Open Graph tags.")
|
||||
2
ToBeMigrated/ai_seo_tools/opengraph_image_generate.py
Normal file
2
ToBeMigrated/ai_seo_tools/opengraph_image_generate.py
Normal file
@@ -0,0 +1,2 @@
|
||||
|
||||
ogImage TBD
|
||||
187
ToBeMigrated/ai_seo_tools/optimize_images_for_upload.py
Normal file
187
ToBeMigrated/ai_seo_tools/optimize_images_for_upload.py
Normal file
@@ -0,0 +1,187 @@
|
||||
import os
|
||||
import sys
|
||||
import tinify
|
||||
from PIL import Image
|
||||
from loguru import logger
|
||||
from dotenv import load_dotenv
|
||||
import streamlit as st
|
||||
from tempfile import NamedTemporaryFile
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
# Set Tinyfy API key from environment variable
|
||||
TINIFY_API_KEY = os.getenv('TINIFY_API_KEY')
|
||||
if TINIFY_API_KEY:
|
||||
tinify.key = TINIFY_API_KEY
|
||||
|
||||
def setup_logger() -> None:
|
||||
"""Configure the logger."""
|
||||
logger.remove()
|
||||
logger.add(
|
||||
sys.stdout,
|
||||
colorize=True,
|
||||
format="<level>{level}</level>|<green>{file}:{line}:{function}</green>| {message}"
|
||||
)
|
||||
|
||||
setup_logger()
|
||||
|
||||
def compress_image(image: Image.Image, quality: int = 45, resize: tuple = None, preserve_exif: bool = False) -> Image.Image:
|
||||
"""
|
||||
Compress and optionally resize an image.
|
||||
|
||||
Args:
|
||||
image (PIL.Image): Image object to compress.
|
||||
quality (int): Quality of the output image (1-100).
|
||||
resize (tuple): Tuple (width, height) to resize the image.
|
||||
preserve_exif (bool): Preserve EXIF data if True.
|
||||
|
||||
Returns:
|
||||
PIL.Image: The compressed and resized image object.
|
||||
"""
|
||||
try:
|
||||
if image.mode == 'RGBA':
|
||||
logger.info("Converting RGBA image to RGB.")
|
||||
image = image.convert('RGB')
|
||||
|
||||
exif = image.info.get('exif') if preserve_exif and 'exif' in image.info else None
|
||||
|
||||
if resize:
|
||||
image = image.resize(resize, Image.LANCZOS)
|
||||
logger.info(f"Resized image to {resize}")
|
||||
|
||||
with NamedTemporaryFile(delete=False, suffix=".jpg") as temp_file:
|
||||
temp_path = temp_file.name
|
||||
try:
|
||||
image.save(temp_path, optimize=True, quality=quality, exif=exif)
|
||||
except Exception as exif_error:
|
||||
logger.warning(f"Error saving image with EXIF: {exif_error}. Saving without EXIF.")
|
||||
image.save(temp_path, optimize=True, quality=quality)
|
||||
|
||||
logger.info("Image compression successful.")
|
||||
return Image.open(temp_path)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error compressing image: {e}")
|
||||
st.error("Failed to compress the image. Please try again.")
|
||||
return None
|
||||
|
||||
def convert_to_webp(image: Image.Image, image_path: str) -> str:
|
||||
"""
|
||||
Convert an image to WebP format.
|
||||
|
||||
Args:
|
||||
image (PIL.Image): Image object to convert.
|
||||
image_path (str): Path to save the WebP image.
|
||||
|
||||
Returns:
|
||||
str: Path to the WebP image.
|
||||
"""
|
||||
try:
|
||||
webp_path = os.path.splitext(image_path)[0] + '.webp'
|
||||
image.save(webp_path, 'WEBP', quality=80, method=6)
|
||||
return webp_path
|
||||
except Exception as e:
|
||||
logger.error(f"Error converting image to WebP: {e}")
|
||||
st.error("Failed to convert the image to WebP format. Please try again.")
|
||||
return None
|
||||
|
||||
def compress_image_tinyfy(image_path: str) -> None:
|
||||
"""
|
||||
Compress an image using Tinyfy API.
|
||||
|
||||
Args:
|
||||
image_path (str): Path to the image to be compressed.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
try:
|
||||
if not tinify.key:
|
||||
logger.warning("Tinyfy API key is not set. Skipping Tinyfy compression.")
|
||||
return
|
||||
|
||||
source = tinify.from_file(image_path)
|
||||
source.to_file(image_path)
|
||||
logger.info("Tinyfy compression successful.")
|
||||
except tinify.errors.AccountError:
|
||||
logger.error("Verify your Tinyfy API key and account limit.")
|
||||
st.warning("Tinyfy compression failed. Check your API key and account limit.")
|
||||
except Exception as e:
|
||||
logger.error(f"Error during Tinyfy compression: {e}")
|
||||
st.warning("Tinyfy compression failed. Ensure the API key is set.")
|
||||
|
||||
def optimize_image(image: Image.Image, image_path: str, quality: int, resize: tuple, preserve_exif: bool) -> str:
|
||||
"""
|
||||
Optimize the image by compressing and converting it to WebP, with optional Tinyfy compression.
|
||||
|
||||
Args:
|
||||
image (PIL.Image): The original image.
|
||||
image_path (str): The path to the image file.
|
||||
quality (int): Quality level for compression.
|
||||
resize (tuple): Dimensions to resize the image.
|
||||
preserve_exif (bool): Whether to preserve EXIF data.
|
||||
|
||||
Returns:
|
||||
str: Path to the optimized WebP image, or None if failed.
|
||||
"""
|
||||
logger.info("Starting image optimization process...")
|
||||
|
||||
compressed_image = compress_image(image, quality, resize, preserve_exif)
|
||||
if compressed_image is None:
|
||||
return None
|
||||
|
||||
webp_path = convert_to_webp(compressed_image, image_path)
|
||||
if webp_path is None:
|
||||
return None
|
||||
|
||||
if tinify.key:
|
||||
compress_image_tinyfy(webp_path)
|
||||
else:
|
||||
logger.info("Tinyfy key not provided, skipping Tinyfy compression.")
|
||||
|
||||
return webp_path
|
||||
|
||||
def main_img_optimizer() -> None:
|
||||
st.title("ALwrity Image Optimizer")
|
||||
st.markdown("## Upload an image to optimize its size and format.")
|
||||
|
||||
input_tinify_key = st.text_input("Optional: Enter your Tinyfy API Key")
|
||||
if input_tinify_key:
|
||||
tinify.key = input_tinify_key
|
||||
|
||||
uploaded_file = st.file_uploader("Upload an image", type=['jpg', 'jpeg', 'png', 'gif', 'bmp', 'webp'])
|
||||
|
||||
if uploaded_file:
|
||||
image = Image.open(uploaded_file)
|
||||
st.image(image, caption="Original Image", use_column_width=True)
|
||||
|
||||
quality = st.slider("Compression Quality", 1, 100, 45)
|
||||
preserve_exif = st.checkbox("Preserve EXIF Data", value=False)
|
||||
resize = st.checkbox("Resize Image")
|
||||
|
||||
if resize:
|
||||
width = st.number_input("Width", value=image.width)
|
||||
height = st.number_input("Height", value=image.height)
|
||||
resize_dims = (width, height)
|
||||
else:
|
||||
resize_dims = None
|
||||
|
||||
if st.button("Optimize Image"):
|
||||
with st.spinner("Optimizing..."):
|
||||
if tinify.key:
|
||||
st.info("Tinyfy compression will be applied.")
|
||||
|
||||
webp_path = optimize_image(image, uploaded_file.name, quality, resize_dims, preserve_exif)
|
||||
|
||||
if webp_path:
|
||||
st.image(webp_path, caption="Optimized Image (WebP)", use_column_width=True)
|
||||
st.success("Image optimization completed!")
|
||||
|
||||
with open(webp_path, "rb") as file:
|
||||
st.download_button(
|
||||
label="Download Optimized Image",
|
||||
data=file,
|
||||
file_name=os.path.basename(webp_path),
|
||||
mime="image/webp"
|
||||
)
|
||||
340
ToBeMigrated/ai_seo_tools/seo_analyzer_api.py
Normal file
340
ToBeMigrated/ai_seo_tools/seo_analyzer_api.py
Normal file
@@ -0,0 +1,340 @@
|
||||
"""
|
||||
FastAPI endpoint for the Comprehensive SEO Analyzer
|
||||
Provides data for the React SEO Dashboard
|
||||
"""
|
||||
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from pydantic import BaseModel, HttpUrl
|
||||
from typing import List, Optional, Dict, Any
|
||||
from datetime import datetime
|
||||
import json
|
||||
|
||||
from .comprehensive_seo_analyzer import ComprehensiveSEOAnalyzer, SEOAnalysisResult
|
||||
|
||||
app = FastAPI(
|
||||
title="Comprehensive SEO Analyzer API",
|
||||
description="API for analyzing website SEO performance with actionable insights",
|
||||
version="1.0.0"
|
||||
)
|
||||
|
||||
# Initialize the analyzer
|
||||
seo_analyzer = ComprehensiveSEOAnalyzer()
|
||||
|
||||
class SEOAnalysisRequest(BaseModel):
|
||||
url: HttpUrl
|
||||
target_keywords: Optional[List[str]] = None
|
||||
|
||||
class SEOAnalysisResponse(BaseModel):
|
||||
url: str
|
||||
timestamp: datetime
|
||||
overall_score: int
|
||||
health_status: str
|
||||
critical_issues: List[str]
|
||||
warnings: List[str]
|
||||
recommendations: List[str]
|
||||
data: Dict[str, Any]
|
||||
success: bool
|
||||
message: str
|
||||
|
||||
@app.post("/analyze-seo", response_model=SEOAnalysisResponse)
|
||||
async def analyze_seo(request: SEOAnalysisRequest):
|
||||
"""
|
||||
Analyze a URL for comprehensive SEO performance
|
||||
|
||||
Args:
|
||||
request: SEOAnalysisRequest containing URL and optional target keywords
|
||||
|
||||
Returns:
|
||||
SEOAnalysisResponse with detailed analysis results
|
||||
"""
|
||||
try:
|
||||
# Convert URL to string
|
||||
url_str = str(request.url)
|
||||
|
||||
# Perform analysis
|
||||
result = seo_analyzer.analyze_url(url_str, request.target_keywords)
|
||||
|
||||
# Convert to response format
|
||||
response_data = {
|
||||
'url': result.url,
|
||||
'timestamp': result.timestamp,
|
||||
'overall_score': result.overall_score,
|
||||
'health_status': result.health_status,
|
||||
'critical_issues': result.critical_issues,
|
||||
'warnings': result.warnings,
|
||||
'recommendations': result.recommendations,
|
||||
'data': result.data,
|
||||
'success': True,
|
||||
'message': f"SEO analysis completed successfully for {result.url}"
|
||||
}
|
||||
|
||||
return SEOAnalysisResponse(**response_data)
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Error analyzing SEO: {str(e)}"
|
||||
)
|
||||
|
||||
@app.get("/health")
|
||||
async def health_check():
|
||||
"""Health check endpoint"""
|
||||
return {
|
||||
"status": "healthy",
|
||||
"timestamp": datetime.now(),
|
||||
"service": "Comprehensive SEO Analyzer API"
|
||||
}
|
||||
|
||||
@app.get("/analysis-summary/{url:path}")
|
||||
async def get_analysis_summary(url: str):
|
||||
"""
|
||||
Get a quick summary of SEO analysis for a URL
|
||||
|
||||
Args:
|
||||
url: The URL to analyze
|
||||
|
||||
Returns:
|
||||
Summary of SEO analysis
|
||||
"""
|
||||
try:
|
||||
# Ensure URL has protocol
|
||||
if not url.startswith(('http://', 'https://')):
|
||||
url = f"https://{url}"
|
||||
|
||||
# Perform analysis
|
||||
result = seo_analyzer.analyze_url(url)
|
||||
|
||||
# Create summary
|
||||
summary = {
|
||||
"url": result.url,
|
||||
"overall_score": result.overall_score,
|
||||
"health_status": result.health_status,
|
||||
"critical_issues_count": len(result.critical_issues),
|
||||
"warnings_count": len(result.warnings),
|
||||
"recommendations_count": len(result.recommendations),
|
||||
"top_issues": result.critical_issues[:3],
|
||||
"top_recommendations": result.recommendations[:3],
|
||||
"analysis_timestamp": result.timestamp.isoformat()
|
||||
}
|
||||
|
||||
return summary
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Error getting analysis summary: {str(e)}"
|
||||
)
|
||||
|
||||
@app.get("/seo-metrics/{url:path}")
|
||||
async def get_seo_metrics(url: str):
|
||||
"""
|
||||
Get detailed SEO metrics for dashboard display
|
||||
|
||||
Args:
|
||||
url: The URL to analyze
|
||||
|
||||
Returns:
|
||||
Detailed SEO metrics for React dashboard
|
||||
"""
|
||||
try:
|
||||
# Ensure URL has protocol
|
||||
if not url.startswith(('http://', 'https://')):
|
||||
url = f"https://{url}"
|
||||
|
||||
# Perform analysis
|
||||
result = seo_analyzer.analyze_url(url)
|
||||
|
||||
# Extract metrics for dashboard
|
||||
metrics = {
|
||||
"overall_score": result.overall_score,
|
||||
"health_status": result.health_status,
|
||||
"url_structure_score": result.data.get('url_structure', {}).get('score', 0),
|
||||
"meta_data_score": result.data.get('meta_data', {}).get('score', 0),
|
||||
"content_score": result.data.get('content_analysis', {}).get('score', 0),
|
||||
"technical_score": result.data.get('technical_seo', {}).get('score', 0),
|
||||
"performance_score": result.data.get('performance', {}).get('score', 0),
|
||||
"accessibility_score": result.data.get('accessibility', {}).get('score', 0),
|
||||
"user_experience_score": result.data.get('user_experience', {}).get('score', 0),
|
||||
"security_score": result.data.get('security_headers', {}).get('score', 0)
|
||||
}
|
||||
|
||||
# Add detailed data for each category
|
||||
dashboard_data = {
|
||||
"metrics": metrics,
|
||||
"critical_issues": result.critical_issues,
|
||||
"warnings": result.warnings,
|
||||
"recommendations": result.recommendations,
|
||||
"detailed_analysis": {
|
||||
"url_structure": result.data.get('url_structure', {}),
|
||||
"meta_data": result.data.get('meta_data', {}),
|
||||
"content_analysis": result.data.get('content_analysis', {}),
|
||||
"technical_seo": result.data.get('technical_seo', {}),
|
||||
"performance": result.data.get('performance', {}),
|
||||
"accessibility": result.data.get('accessibility', {}),
|
||||
"user_experience": result.data.get('user_experience', {}),
|
||||
"security_headers": result.data.get('security_headers', {}),
|
||||
"keyword_analysis": result.data.get('keyword_analysis', {})
|
||||
},
|
||||
"timestamp": result.timestamp.isoformat(),
|
||||
"url": result.url
|
||||
}
|
||||
|
||||
return dashboard_data
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Error getting SEO metrics: {str(e)}"
|
||||
)
|
||||
|
||||
@app.post("/batch-analyze")
|
||||
async def batch_analyze(urls: List[str]):
|
||||
"""
|
||||
Analyze multiple URLs in batch
|
||||
|
||||
Args:
|
||||
urls: List of URLs to analyze
|
||||
|
||||
Returns:
|
||||
Batch analysis results
|
||||
"""
|
||||
try:
|
||||
results = []
|
||||
|
||||
for url in urls:
|
||||
try:
|
||||
# Ensure URL has protocol
|
||||
if not url.startswith(('http://', 'https://')):
|
||||
url = f"https://{url}"
|
||||
|
||||
# Perform analysis
|
||||
result = seo_analyzer.analyze_url(url)
|
||||
|
||||
# Add to results
|
||||
results.append({
|
||||
"url": result.url,
|
||||
"overall_score": result.overall_score,
|
||||
"health_status": result.health_status,
|
||||
"critical_issues_count": len(result.critical_issues),
|
||||
"warnings_count": len(result.warnings),
|
||||
"success": True
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
# Add error result
|
||||
results.append({
|
||||
"url": url,
|
||||
"overall_score": 0,
|
||||
"health_status": "error",
|
||||
"critical_issues_count": 0,
|
||||
"warnings_count": 0,
|
||||
"success": False,
|
||||
"error": str(e)
|
||||
})
|
||||
|
||||
return {
|
||||
"total_urls": len(urls),
|
||||
"successful_analyses": len([r for r in results if r['success']]),
|
||||
"failed_analyses": len([r for r in results if not r['success']]),
|
||||
"results": results
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Error in batch analysis: {str(e)}"
|
||||
)
|
||||
|
||||
# Enhanced prompts for better results
|
||||
ENHANCED_PROMPTS = {
|
||||
"critical_issue": "🚨 CRITICAL: This issue is severely impacting your SEO performance and must be fixed immediately.",
|
||||
"warning": "⚠️ WARNING: This could be improved to boost your search rankings.",
|
||||
"recommendation": "💡 RECOMMENDATION: Implement this to improve your SEO score.",
|
||||
"excellent": "🎉 EXCELLENT: Your SEO is performing very well in this area!",
|
||||
"good": "✅ GOOD: Your SEO is performing well, with room for minor improvements.",
|
||||
"needs_improvement": "🔧 NEEDS IMPROVEMENT: Several areas need attention to boost your SEO.",
|
||||
"poor": "❌ POOR: Significant improvements needed across multiple areas."
|
||||
}
|
||||
|
||||
def enhance_analysis_result(result: SEOAnalysisResult) -> SEOAnalysisResult:
|
||||
"""
|
||||
Enhance analysis results with better prompts and user-friendly language
|
||||
"""
|
||||
# Enhance critical issues
|
||||
enhanced_critical_issues = []
|
||||
for issue in result.critical_issues:
|
||||
enhanced_issue = f"{ENHANCED_PROMPTS['critical_issue']} {issue}"
|
||||
enhanced_critical_issues.append(enhanced_issue)
|
||||
|
||||
# Enhance warnings
|
||||
enhanced_warnings = []
|
||||
for warning in result.warnings:
|
||||
enhanced_warning = f"{ENHANCED_PROMPTS['warning']} {warning}"
|
||||
enhanced_warnings.append(enhanced_warning)
|
||||
|
||||
# Enhance recommendations
|
||||
enhanced_recommendations = []
|
||||
for rec in result.recommendations:
|
||||
enhanced_rec = f"{ENHANCED_PROMPTS['recommendation']} {rec}"
|
||||
enhanced_recommendations.append(enhanced_rec)
|
||||
|
||||
# Create enhanced result
|
||||
enhanced_result = SEOAnalysisResult(
|
||||
url=result.url,
|
||||
timestamp=result.timestamp,
|
||||
overall_score=result.overall_score,
|
||||
health_status=result.health_status,
|
||||
critical_issues=enhanced_critical_issues,
|
||||
warnings=enhanced_warnings,
|
||||
recommendations=enhanced_recommendations,
|
||||
data=result.data
|
||||
)
|
||||
|
||||
return enhanced_result
|
||||
|
||||
@app.post("/analyze-seo-enhanced", response_model=SEOAnalysisResponse)
|
||||
async def analyze_seo_enhanced(request: SEOAnalysisRequest):
|
||||
"""
|
||||
Analyze a URL with enhanced, user-friendly prompts
|
||||
|
||||
Args:
|
||||
request: SEOAnalysisRequest containing URL and optional target keywords
|
||||
|
||||
Returns:
|
||||
SEOAnalysisResponse with enhanced, user-friendly analysis results
|
||||
"""
|
||||
try:
|
||||
# Convert URL to string
|
||||
url_str = str(request.url)
|
||||
|
||||
# Perform analysis
|
||||
result = seo_analyzer.analyze_url(url_str, request.target_keywords)
|
||||
|
||||
# Enhance results
|
||||
enhanced_result = enhance_analysis_result(result)
|
||||
|
||||
# Convert to response format
|
||||
response_data = {
|
||||
'url': enhanced_result.url,
|
||||
'timestamp': enhanced_result.timestamp,
|
||||
'overall_score': enhanced_result.overall_score,
|
||||
'health_status': enhanced_result.health_status,
|
||||
'critical_issues': enhanced_result.critical_issues,
|
||||
'warnings': enhanced_result.warnings,
|
||||
'recommendations': enhanced_result.recommendations,
|
||||
'data': enhanced_result.data,
|
||||
'success': True,
|
||||
'message': f"Enhanced SEO analysis completed successfully for {enhanced_result.url}"
|
||||
}
|
||||
|
||||
return SEOAnalysisResponse(**response_data)
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Error analyzing SEO: {str(e)}"
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run(app, host="0.0.0.0", port=8000)
|
||||
130
ToBeMigrated/ai_seo_tools/seo_structured_data.py
Normal file
130
ToBeMigrated/ai_seo_tools/seo_structured_data.py
Normal file
@@ -0,0 +1,130 @@
|
||||
import streamlit as st
|
||||
import json
|
||||
from datetime import date
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from ..ai_web_researcher.firecrawl_web_crawler import scrape_url
|
||||
from ..gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
# Define a dictionary for schema types
|
||||
schema_types = {
|
||||
"Article": {
|
||||
"fields": ["Headline", "Author", "Date Published", "Keywords"],
|
||||
"schema_type": "Article",
|
||||
},
|
||||
"Product": {
|
||||
"fields": ["Name", "Description", "Price", "Brand", "Image URL"],
|
||||
"schema_type": "Product",
|
||||
},
|
||||
"Recipe": {
|
||||
"fields": ["Name", "Ingredients", "Cooking Time", "Serving Size", "Image URL"],
|
||||
"schema_type": "Recipe",
|
||||
},
|
||||
"Event": {
|
||||
"fields": ["Name", "Start Date", "End Date", "Location", "Description"],
|
||||
"schema_type": "Event",
|
||||
},
|
||||
"LocalBusiness": {
|
||||
"fields": ["Name", "Address", "Phone Number", "Opening Hours", "Image URL"],
|
||||
"schema_type": "LocalBusiness",
|
||||
},
|
||||
# ... (add more schema types as needed)
|
||||
}
|
||||
|
||||
def generate_json_data(content_type, details, url):
|
||||
"""Generates structured data (JSON-LD) based on user input."""
|
||||
try:
|
||||
scraped_text = scrape_url(url)
|
||||
except Exception as err:
|
||||
st.error(f"Failed to scrape web page from URL: {url} - Error: {err}")
|
||||
return
|
||||
|
||||
schema = schema_types.get(content_type)
|
||||
if not schema:
|
||||
st.error(f"Invalid content type: {content_type}")
|
||||
return
|
||||
|
||||
data = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": schema["schema_type"],
|
||||
}
|
||||
for field in schema["fields"]:
|
||||
value = details.get(field)
|
||||
if isinstance(value, date):
|
||||
value = value.isoformat()
|
||||
data[field] = value if value else "N/A" # Use placeholder values if input is missing
|
||||
|
||||
if url:
|
||||
data['url'] = url
|
||||
|
||||
llm_structured_data = get_llm_structured_data(content_type, data, scraped_text)
|
||||
return llm_structured_data
|
||||
|
||||
def get_llm_structured_data(content_type, data, scraped_text):
|
||||
"""Function to get structured data from LLM."""
|
||||
prompt = f"""Given the following information:
|
||||
|
||||
HTML Content: <<<HTML>>> {scraped_text} <<<END_HTML>>>
|
||||
Content Type: <<<CONTENT_TYPE>>> {content_type} <<<END_CONTENT_TYPE>>>
|
||||
Additional Relevant Data: <<<ADDITIONAL_DATA>>> {data} <<<END_ADDITIONAL_DATA>>>
|
||||
|
||||
Create a detailed structured data (JSON-LD) script for SEO purposes.
|
||||
The structured data should help search engines understand the content and features of the webpage, enhancing its visibility and potential for rich snippets in search results.
|
||||
|
||||
Detailed Steps:
|
||||
Parse the HTML content to extract relevant information like the title, main heading, and body content.
|
||||
Use the contentType to determine the structured data type (e.g., Article, Product, Recipe).
|
||||
Integrate the additional relevant data (e.g., author, datePublished, keywords) into the structured data.
|
||||
Ensure all URLs, images, and other attributes are correctly formatted and included.
|
||||
Validate the generated JSON-LD to ensure it meets schema.org standards and is free of errors.
|
||||
|
||||
Expected Output:
|
||||
Generate a JSON-LD structured data snippet based on the provided inputs."""
|
||||
|
||||
try:
|
||||
response = llm_text_gen(prompt)
|
||||
return response
|
||||
except Exception as err:
|
||||
st.error(f"Failed to get response from LLM: {err}")
|
||||
return
|
||||
|
||||
def ai_structured_data():
|
||||
st.title("📝 Generate Structured Data for SEO 🚀")
|
||||
st.markdown("**Make your content more discoverable with rich snippets.**")
|
||||
|
||||
content_type = st.selectbox("**Select Content Type**", list(schema_types.keys()))
|
||||
|
||||
details = {}
|
||||
schema_fields = schema_types[content_type]["fields"]
|
||||
num_fields = len(schema_fields)
|
||||
|
||||
url = st.text_input("**URL :**", placeholder="Enter the URL of your webpage")
|
||||
for i in range(0, num_fields, 2):
|
||||
cols = st.columns(2)
|
||||
for j in range(2):
|
||||
if i + j < num_fields:
|
||||
field = schema_fields[i + j]
|
||||
if "Date" in field:
|
||||
details[field] = cols[j].date_input(field)
|
||||
else:
|
||||
details[field] = cols[j].text_input(field, placeholder=f"Enter {field.lower()}")
|
||||
|
||||
if st.button("Generate Structured Data"):
|
||||
if not url:
|
||||
st.error("URL is required to generate structured data.")
|
||||
return
|
||||
|
||||
structured_data = generate_json_data(content_type, details, url)
|
||||
if structured_data:
|
||||
st.subheader("Generated Structured Data (JSON-LD):")
|
||||
st.markdown(structured_data)
|
||||
|
||||
st.download_button(
|
||||
label="Download JSON-LD",
|
||||
data=structured_data,
|
||||
file_name=f"{content_type}_structured_data.json",
|
||||
mime="application/json",
|
||||
)
|
||||
340
ToBeMigrated/ai_seo_tools/sitemap_analysis.py
Normal file
340
ToBeMigrated/ai_seo_tools/sitemap_analysis.py
Normal file
@@ -0,0 +1,340 @@
|
||||
import streamlit as st
|
||||
import advertools as adv
|
||||
import pandas as pd
|
||||
import plotly.graph_objects as go
|
||||
from urllib.error import URLError
|
||||
import xml.etree.ElementTree as ET
|
||||
import requests
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Main function to run the Sitemap Analyzer Streamlit app.
|
||||
"""
|
||||
st.title("📊 Sitemap Analyzer")
|
||||
st.write("""
|
||||
This tool analyzes a website's sitemap to understand its content structure and publishing trends.
|
||||
Enter a sitemap URL to start your analysis.
|
||||
""")
|
||||
|
||||
sitemap_url = st.text_input(
|
||||
"Please enter the sitemap URL:",
|
||||
"https://www.example.com/sitemap.xml"
|
||||
)
|
||||
|
||||
if st.button("Analyze Sitemap"):
|
||||
try:
|
||||
sitemap_df = fetch_all_sitemaps(sitemap_url)
|
||||
if sitemap_df is not None and not sitemap_df.empty:
|
||||
sitemap_df = process_lastmod_column(sitemap_df)
|
||||
ppmonth = analyze_content_trends(sitemap_df)
|
||||
sitemap_df = categorize_and_shorten_sitemaps(sitemap_df)
|
||||
|
||||
display_key_metrics(sitemap_df, ppmonth)
|
||||
plot_sitemap_content_distribution(sitemap_df)
|
||||
plot_content_trends(ppmonth)
|
||||
plot_content_type_breakdown(sitemap_df)
|
||||
plot_publishing_frequency(sitemap_df)
|
||||
|
||||
st.success("🎉 Analysis complete!")
|
||||
else:
|
||||
st.error("No valid URLs found in the sitemap.")
|
||||
except URLError as e:
|
||||
st.error(f"Error fetching the sitemap: {e}")
|
||||
except Exception as e:
|
||||
st.error(f"An unexpected error occurred: {e}")
|
||||
|
||||
|
||||
def fetch_all_sitemaps(sitemap_url):
|
||||
"""
|
||||
Fetches all sitemaps from the provided sitemap URL and concatenates their URLs into a DataFrame.
|
||||
|
||||
Parameters:
|
||||
sitemap_url (str): The URL of the sitemap.
|
||||
|
||||
Returns:
|
||||
DataFrame: A DataFrame containing all URLs from the sitemaps.
|
||||
"""
|
||||
st.write(f"🚀 Fetching and analyzing the sitemap: {sitemap_url}...")
|
||||
|
||||
try:
|
||||
sitemap_df = fetch_sitemap(sitemap_url)
|
||||
|
||||
if sitemap_df is not None:
|
||||
all_sitemaps = sitemap_df.loc[
|
||||
sitemap_df['loc'].str.contains('sitemap'),
|
||||
'loc'
|
||||
].tolist()
|
||||
|
||||
if all_sitemaps:
|
||||
st.write(
|
||||
f"🔄 Found {len(all_sitemaps)} additional sitemaps. Fetching data from them..."
|
||||
)
|
||||
all_urls_df = pd.DataFrame()
|
||||
|
||||
for sitemap in all_sitemaps:
|
||||
try:
|
||||
st.write(f"Fetching URLs from {sitemap}...")
|
||||
temp_df = fetch_sitemap(sitemap)
|
||||
if temp_df is not None:
|
||||
all_urls_df = pd.concat(
|
||||
[all_urls_df, temp_df], ignore_index=True
|
||||
)
|
||||
except Exception as e:
|
||||
st.error(f"Error fetching {sitemap}: {e}")
|
||||
|
||||
st.write(
|
||||
f"✅ Successfully fetched {len(all_urls_df)} URLs from all sitemaps."
|
||||
)
|
||||
return all_urls_df
|
||||
|
||||
else:
|
||||
st.write(f"✅ Successfully fetched {len(sitemap_df)} URLs from the main sitemap.")
|
||||
return sitemap_df
|
||||
else:
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error fetching the sitemap: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def fetch_sitemap(url):
|
||||
"""
|
||||
Fetches and parses the sitemap from the provided URL.
|
||||
|
||||
Parameters:
|
||||
url (str): The URL of the sitemap.
|
||||
|
||||
Returns:
|
||||
DataFrame: A DataFrame containing the URLs from the sitemap.
|
||||
"""
|
||||
try:
|
||||
response = requests.get(url)
|
||||
response.raise_for_status()
|
||||
|
||||
ET.fromstring(response.content)
|
||||
|
||||
sitemap_df = adv.sitemap_to_df(url)
|
||||
return sitemap_df
|
||||
|
||||
except requests.RequestException as e:
|
||||
st.error(f"⚠️ Request error: {e}")
|
||||
return None
|
||||
except ET.ParseError as e:
|
||||
st.error(f"⚠️ XML parsing error: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def process_lastmod_column(sitemap_df):
|
||||
"""
|
||||
Processes the 'lastmod' column in the sitemap DataFrame by converting it to DateTime format and setting it as the index.
|
||||
|
||||
Parameters:
|
||||
sitemap_df (DataFrame): The sitemap DataFrame.
|
||||
|
||||
Returns:
|
||||
DataFrame: The processed sitemap DataFrame with 'lastmod' as the index.
|
||||
"""
|
||||
st.write("📅 Converting 'lastmod' column to DateTime format and setting it as the index...")
|
||||
|
||||
try:
|
||||
sitemap_df = sitemap_df.dropna(subset=['lastmod'])
|
||||
sitemap_df['lastmod'] = pd.to_datetime(sitemap_df['lastmod'])
|
||||
sitemap_df.set_index('lastmod', inplace=True)
|
||||
|
||||
st.write("✅ 'lastmod' column successfully converted to DateTime format and set as the index.")
|
||||
return sitemap_df
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error processing the 'lastmod' column: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def categorize_and_shorten_sitemaps(sitemap_df):
|
||||
"""
|
||||
Categorizes and shortens the sitemap names in the sitemap DataFrame.
|
||||
|
||||
Parameters:
|
||||
sitemap_df (DataFrame): The sitemap DataFrame.
|
||||
|
||||
Returns:
|
||||
DataFrame: The sitemap DataFrame with categorized and shortened sitemap names.
|
||||
"""
|
||||
st.write("🔍 Categorizing and shortening sitemap names...")
|
||||
|
||||
try:
|
||||
sitemap_df['sitemap_name'] = sitemap_df['sitemap'].str.split('/').str[4]
|
||||
sitemap_df['sitemap_name'] = sitemap_df['sitemap_name'].replace({
|
||||
'sitemap-site-kasko-fiyatlari.xml': 'Kasko',
|
||||
'sitemap-site-bireysel.xml': 'Personal',
|
||||
'sitemap-site-kurumsal.xml': 'Cooperate',
|
||||
'sitemap-site-arac-sigortasi.xml': 'Car',
|
||||
'sitemap-site.xml': 'Others'
|
||||
})
|
||||
|
||||
st.write("✅ Sitemap names categorized and shortened.")
|
||||
return sitemap_df
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error categorizing sitemap names: {e}")
|
||||
return sitemap_df
|
||||
|
||||
|
||||
def analyze_content_trends(sitemap_df):
|
||||
"""
|
||||
Analyzes content publishing trends in the sitemap DataFrame.
|
||||
|
||||
Parameters:
|
||||
sitemap_df (DataFrame): The sitemap DataFrame.
|
||||
|
||||
Returns:
|
||||
Series: A Series representing the number of contents published each month.
|
||||
"""
|
||||
st.write("📅 Analyzing content publishing trends...")
|
||||
|
||||
try:
|
||||
ppmonth = sitemap_df.resample('M').size()
|
||||
sitemap_df['monthly_count'] = sitemap_df.index.to_period('M').value_counts().sort_index()
|
||||
|
||||
st.write("✅ Content trends analysis completed.")
|
||||
return ppmonth
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error during content trends analysis: {e}")
|
||||
return pd.Series()
|
||||
|
||||
|
||||
def display_key_metrics(sitemap_df, ppmonth):
|
||||
"""
|
||||
Displays key metrics of the sitemap analysis.
|
||||
|
||||
Parameters:
|
||||
sitemap_df (DataFrame): The sitemap DataFrame.
|
||||
ppmonth (Series): The Series representing the number of contents published each month.
|
||||
"""
|
||||
st.write("### Key Metrics")
|
||||
|
||||
total_urls = len(sitemap_df)
|
||||
total_articles = ppmonth.sum()
|
||||
average_frequency = ppmonth.mean()
|
||||
|
||||
st.write(f"**Total URLs Found:** {total_urls:,}")
|
||||
st.write(f"**Total Articles Published:** {total_articles:,}")
|
||||
st.write(f"**Average Monthly Publishing Frequency:** {average_frequency:.2f} articles/month")
|
||||
|
||||
|
||||
def plot_sitemap_content_distribution(sitemap_df):
|
||||
"""
|
||||
Plots the content distribution by sitemap categories.
|
||||
|
||||
Parameters:
|
||||
sitemap_df (DataFrame): The sitemap DataFrame.
|
||||
"""
|
||||
st.write("📊 Visualizing content amount by sitemap categories...")
|
||||
|
||||
try:
|
||||
if 'sitemap_name' in sitemap_df.columns:
|
||||
stmc = sitemap_df.groupby('sitemap_name').size()
|
||||
fig = go.Figure()
|
||||
fig.add_bar(x=stmc.index, y=stmc.values, name='Sitemap Categories')
|
||||
fig.update_layout(
|
||||
title='Content Amount by Sitemap Categories',
|
||||
xaxis_title='Sitemap Categories',
|
||||
yaxis_title='Number of Articles',
|
||||
paper_bgcolor='#E5ECF6'
|
||||
)
|
||||
st.plotly_chart(fig)
|
||||
else:
|
||||
st.warning("⚠️ The 'sitemap_name' column is missing in the data.")
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error during sitemap content distribution plotting: {e}")
|
||||
|
||||
|
||||
def plot_content_trends(ppmonth):
|
||||
"""
|
||||
Plots the content publishing trends over time.
|
||||
|
||||
Parameters:
|
||||
ppmonth (Series): The Series representing the number of contents published each month.
|
||||
"""
|
||||
st.write("📈 Plotting content publishing trends over time...")
|
||||
|
||||
try:
|
||||
fig = go.Figure()
|
||||
fig.add_scatter(x=ppmonth.index, y=ppmonth.values, mode='lines+markers', name='Publishing Trends')
|
||||
fig.update_layout(
|
||||
title='Content Publishing Trends Over Time',
|
||||
xaxis_title='Month',
|
||||
yaxis_title='Number of Articles',
|
||||
paper_bgcolor='#E5ECF6'
|
||||
)
|
||||
st.plotly_chart(fig)
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error during content trends plotting: {e}")
|
||||
|
||||
|
||||
def plot_content_type_breakdown(sitemap_df):
|
||||
"""
|
||||
Plots the content type breakdown.
|
||||
|
||||
Parameters:
|
||||
sitemap_df (DataFrame): The sitemap DataFrame.
|
||||
"""
|
||||
st.write("🔍 Plotting content type breakdown...")
|
||||
|
||||
try:
|
||||
if 'sitemap_name' in sitemap_df.columns and not sitemap_df['sitemap_name'].empty:
|
||||
content_type_counts = sitemap_df['sitemap_name'].value_counts()
|
||||
st.write("Content Type Counts:", content_type_counts)
|
||||
|
||||
if not content_type_counts.empty:
|
||||
fig = go.Figure(data=[go.Pie(labels=content_type_counts.index, values=content_type_counts.values)])
|
||||
fig.update_layout(
|
||||
title='Content Type Breakdown',
|
||||
paper_bgcolor='#E5ECF6'
|
||||
)
|
||||
st.plotly_chart(fig)
|
||||
else:
|
||||
st.warning("⚠️ No content types to display.")
|
||||
else:
|
||||
st.warning("⚠️ The 'sitemap_name' column is missing or empty.")
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error during content type breakdown plotting: {e}")
|
||||
|
||||
|
||||
def plot_publishing_frequency(sitemap_df):
|
||||
"""
|
||||
Plots the publishing frequency by month.
|
||||
|
||||
Parameters:
|
||||
sitemap_df (DataFrame): The sitemap DataFrame.
|
||||
"""
|
||||
st.write("📆 Plotting publishing frequency by month...")
|
||||
|
||||
try:
|
||||
if not sitemap_df.empty:
|
||||
frequency_by_month = sitemap_df.index.to_period('M').value_counts().sort_index()
|
||||
frequency_by_month.index = frequency_by_month.index.astype(str)
|
||||
|
||||
fig = go.Figure()
|
||||
fig.add_bar(x=frequency_by_month.index, y=frequency_by_month.values, name='Publishing Frequency')
|
||||
fig.update_layout(
|
||||
title='Publishing Frequency by Month',
|
||||
xaxis_title='Month',
|
||||
yaxis_title='Number of Articles',
|
||||
paper_bgcolor='#E5ECF6'
|
||||
)
|
||||
st.plotly_chart(fig)
|
||||
else:
|
||||
st.warning("⚠️ No data available to plot publishing frequency.")
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"⚠️ Error during publishing frequency plotting: {e}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
22
ToBeMigrated/ai_seo_tools/technical_seo_crawler/__init__.py
Normal file
22
ToBeMigrated/ai_seo_tools/technical_seo_crawler/__init__.py
Normal file
@@ -0,0 +1,22 @@
|
||||
"""
|
||||
Technical SEO Crawler Package.
|
||||
|
||||
This package provides comprehensive technical SEO analysis capabilities
|
||||
with advertools integration and AI-powered recommendations.
|
||||
|
||||
Components:
|
||||
- TechnicalSEOCrawler: Core crawler with technical analysis
|
||||
- TechnicalSEOCrawlerUI: Streamlit interface for the crawler
|
||||
"""
|
||||
|
||||
from .crawler import TechnicalSEOCrawler
|
||||
from .ui import TechnicalSEOCrawlerUI, render_technical_seo_crawler
|
||||
|
||||
__version__ = "1.0.0"
|
||||
__author__ = "ALwrity"
|
||||
|
||||
__all__ = [
|
||||
'TechnicalSEOCrawler',
|
||||
'TechnicalSEOCrawlerUI',
|
||||
'render_technical_seo_crawler'
|
||||
]
|
||||
709
ToBeMigrated/ai_seo_tools/technical_seo_crawler/crawler.py
Normal file
709
ToBeMigrated/ai_seo_tools/technical_seo_crawler/crawler.py
Normal file
@@ -0,0 +1,709 @@
|
||||
"""
|
||||
Comprehensive Technical SEO Crawler using Advertools Integration.
|
||||
|
||||
This module provides advanced site-wide technical SEO analysis using:
|
||||
- adv.crawl: Complete website crawling and analysis
|
||||
- adv.crawl_headers: HTTP headers and server analysis
|
||||
- adv.crawl_images: Image optimization analysis
|
||||
- adv.url_to_df: URL structure optimization
|
||||
- AI-powered technical recommendations
|
||||
"""
|
||||
|
||||
import streamlit as st
|
||||
import pandas as pd
|
||||
import advertools as adv
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
from urllib.parse import urlparse, urljoin
|
||||
import tempfile
|
||||
import os
|
||||
from datetime import datetime
|
||||
import json
|
||||
from collections import Counter, defaultdict
|
||||
from loguru import logger
|
||||
import numpy as np
|
||||
|
||||
# Import existing modules
|
||||
from lib.gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
from lib.utils.website_analyzer.analyzer import WebsiteAnalyzer
|
||||
|
||||
class TechnicalSEOCrawler:
|
||||
"""Comprehensive technical SEO crawler with advertools integration."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the technical SEO crawler."""
|
||||
self.temp_dir = tempfile.mkdtemp()
|
||||
logger.info("TechnicalSEOCrawler initialized")
|
||||
|
||||
def analyze_website_technical_seo(self, website_url: str, crawl_depth: int = 3,
|
||||
max_pages: int = 500) -> Dict[str, Any]:
|
||||
"""
|
||||
Perform comprehensive technical SEO analysis.
|
||||
|
||||
Args:
|
||||
website_url: Website URL to analyze
|
||||
crawl_depth: How deep to crawl (1-5)
|
||||
max_pages: Maximum pages to crawl (50-1000)
|
||||
|
||||
Returns:
|
||||
Comprehensive technical SEO analysis results
|
||||
"""
|
||||
try:
|
||||
st.info("🚀 Starting Comprehensive Technical SEO Crawl...")
|
||||
|
||||
# Initialize results structure
|
||||
results = {
|
||||
'analysis_timestamp': datetime.utcnow().isoformat(),
|
||||
'website_url': website_url,
|
||||
'crawl_settings': {
|
||||
'depth': crawl_depth,
|
||||
'max_pages': max_pages
|
||||
},
|
||||
'crawl_overview': {},
|
||||
'technical_issues': {},
|
||||
'performance_analysis': {},
|
||||
'content_analysis': {},
|
||||
'url_structure': {},
|
||||
'image_optimization': {},
|
||||
'security_headers': {},
|
||||
'mobile_seo': {},
|
||||
'structured_data': {},
|
||||
'ai_recommendations': {}
|
||||
}
|
||||
|
||||
# Phase 1: Core Website Crawl
|
||||
with st.expander("🕷️ Website Crawling Progress", expanded=True):
|
||||
crawl_data = self._perform_comprehensive_crawl(website_url, crawl_depth, max_pages)
|
||||
results['crawl_overview'] = crawl_data
|
||||
st.success(f"✅ Crawled {crawl_data.get('pages_crawled', 0)} pages")
|
||||
|
||||
# Phase 2: Technical Issues Detection
|
||||
with st.expander("🔍 Technical Issues Analysis", expanded=True):
|
||||
technical_issues = self._analyze_technical_issues(crawl_data)
|
||||
results['technical_issues'] = technical_issues
|
||||
st.success("✅ Identified technical SEO issues")
|
||||
|
||||
# Phase 3: Performance Analysis
|
||||
with st.expander("⚡ Performance Analysis", expanded=True):
|
||||
performance = self._analyze_performance_metrics(crawl_data)
|
||||
results['performance_analysis'] = performance
|
||||
st.success("✅ Analyzed website performance metrics")
|
||||
|
||||
# Phase 4: Content & Structure Analysis
|
||||
with st.expander("📊 Content Structure Analysis", expanded=True):
|
||||
content_analysis = self._analyze_content_structure(crawl_data)
|
||||
results['content_analysis'] = content_analysis
|
||||
st.success("✅ Analyzed content structure and optimization")
|
||||
|
||||
# Phase 5: URL Structure Optimization
|
||||
with st.expander("🔗 URL Structure Analysis", expanded=True):
|
||||
url_analysis = self._analyze_url_structure(crawl_data)
|
||||
results['url_structure'] = url_analysis
|
||||
st.success("✅ Analyzed URL structure and patterns")
|
||||
|
||||
# Phase 6: Image SEO Analysis
|
||||
with st.expander("🖼️ Image SEO Analysis", expanded=True):
|
||||
image_analysis = self._analyze_image_seo(website_url)
|
||||
results['image_optimization'] = image_analysis
|
||||
st.success("✅ Analyzed image optimization")
|
||||
|
||||
# Phase 7: Security & Headers Analysis
|
||||
with st.expander("🛡️ Security Headers Analysis", expanded=True):
|
||||
security_analysis = self._analyze_security_headers(website_url)
|
||||
results['security_headers'] = security_analysis
|
||||
st.success("✅ Analyzed security headers")
|
||||
|
||||
# Phase 8: Mobile SEO Analysis
|
||||
with st.expander("📱 Mobile SEO Analysis", expanded=True):
|
||||
mobile_analysis = self._analyze_mobile_seo(crawl_data)
|
||||
results['mobile_seo'] = mobile_analysis
|
||||
st.success("✅ Analyzed mobile SEO factors")
|
||||
|
||||
# Phase 9: AI-Powered Recommendations
|
||||
with st.expander("🤖 AI Technical Recommendations", expanded=True):
|
||||
ai_recommendations = self._generate_technical_recommendations(results)
|
||||
results['ai_recommendations'] = ai_recommendations
|
||||
st.success("✅ Generated AI-powered technical recommendations")
|
||||
|
||||
return results
|
||||
|
||||
except Exception as e:
|
||||
error_msg = f"Error in technical SEO analysis: {str(e)}"
|
||||
logger.error(error_msg, exc_info=True)
|
||||
st.error(error_msg)
|
||||
return {'error': error_msg}
|
||||
|
||||
def _perform_comprehensive_crawl(self, website_url: str, depth: int, max_pages: int) -> Dict[str, Any]:
|
||||
"""Perform comprehensive website crawl using adv.crawl."""
|
||||
try:
|
||||
st.info("🕷️ Crawling website for comprehensive analysis...")
|
||||
|
||||
# Create crawl output file
|
||||
crawl_file = os.path.join(self.temp_dir, "technical_crawl.jl")
|
||||
|
||||
# Configure crawl settings for technical SEO
|
||||
custom_settings = {
|
||||
'DEPTH_LIMIT': depth,
|
||||
'CLOSESPIDER_PAGECOUNT': max_pages,
|
||||
'DOWNLOAD_DELAY': 0.5, # Be respectful
|
||||
'CONCURRENT_REQUESTS': 8,
|
||||
'ROBOTSTXT_OBEY': True,
|
||||
'USER_AGENT': 'ALwrity-TechnicalSEO-Crawler/1.0',
|
||||
'COOKIES_ENABLED': False,
|
||||
'TELNETCONSOLE_ENABLED': False,
|
||||
'LOG_LEVEL': 'WARNING'
|
||||
}
|
||||
|
||||
# Start crawl
|
||||
adv.crawl(
|
||||
url_list=[website_url],
|
||||
output_file=crawl_file,
|
||||
follow_links=True,
|
||||
custom_settings=custom_settings
|
||||
)
|
||||
|
||||
# Read and process crawl results
|
||||
if os.path.exists(crawl_file):
|
||||
crawl_df = pd.read_json(crawl_file, lines=True)
|
||||
|
||||
# Basic crawl statistics
|
||||
crawl_overview = {
|
||||
'pages_crawled': len(crawl_df),
|
||||
'status_codes': crawl_df['status'].value_counts().to_dict(),
|
||||
'crawl_file_path': crawl_file,
|
||||
'crawl_dataframe': crawl_df,
|
||||
'domains_found': crawl_df['url'].apply(lambda x: urlparse(x).netloc).nunique(),
|
||||
'avg_response_time': crawl_df.get('download_latency', pd.Series()).mean(),
|
||||
'total_content_size': crawl_df.get('size', pd.Series()).sum()
|
||||
}
|
||||
|
||||
return crawl_overview
|
||||
else:
|
||||
st.error("Crawl file not created")
|
||||
return {}
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error in website crawl: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _analyze_technical_issues(self, crawl_data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze technical SEO issues from crawl data."""
|
||||
try:
|
||||
st.info("🔍 Detecting technical SEO issues...")
|
||||
|
||||
if 'crawl_dataframe' not in crawl_data:
|
||||
return {}
|
||||
|
||||
df = crawl_data['crawl_dataframe']
|
||||
|
||||
technical_issues = {
|
||||
'http_errors': {},
|
||||
'redirect_issues': {},
|
||||
'duplicate_content': {},
|
||||
'missing_elements': {},
|
||||
'page_speed_issues': {},
|
||||
'crawlability_issues': {}
|
||||
}
|
||||
|
||||
# HTTP Status Code Issues
|
||||
error_codes = df[df['status'] >= 400]['status'].value_counts().to_dict()
|
||||
technical_issues['http_errors'] = {
|
||||
'total_errors': len(df[df['status'] >= 400]),
|
||||
'error_breakdown': error_codes,
|
||||
'error_pages': df[df['status'] >= 400][['url', 'status']].to_dict('records')[:50]
|
||||
}
|
||||
|
||||
# Redirect Analysis
|
||||
redirects = df[df['status'].isin([301, 302, 303, 307, 308])]
|
||||
technical_issues['redirect_issues'] = {
|
||||
'total_redirects': len(redirects),
|
||||
'redirect_chains': self._find_redirect_chains(redirects),
|
||||
'redirect_types': redirects['status'].value_counts().to_dict()
|
||||
}
|
||||
|
||||
# Duplicate Content Detection
|
||||
if 'title' in df.columns:
|
||||
duplicate_titles = df['title'].value_counts()
|
||||
duplicate_titles = duplicate_titles[duplicate_titles > 1]
|
||||
|
||||
technical_issues['duplicate_content'] = {
|
||||
'duplicate_titles': len(duplicate_titles),
|
||||
'duplicate_title_groups': duplicate_titles.to_dict(),
|
||||
'pages_with_duplicate_titles': df[df['title'].isin(duplicate_titles.index)][['url', 'title']].to_dict('records')[:20]
|
||||
}
|
||||
|
||||
# Missing Elements Analysis
|
||||
missing_elements = {
|
||||
'missing_titles': len(df[(df['title'].isna()) | (df['title'] == '')]) if 'title' in df.columns else 0,
|
||||
'missing_meta_desc': len(df[(df['meta_desc'].isna()) | (df['meta_desc'] == '')]) if 'meta_desc' in df.columns else 0,
|
||||
'missing_h1': len(df[(df['h1'].isna()) | (df['h1'] == '')]) if 'h1' in df.columns else 0
|
||||
}
|
||||
technical_issues['missing_elements'] = missing_elements
|
||||
|
||||
# Page Speed Issues
|
||||
if 'download_latency' in df.columns:
|
||||
slow_pages = df[df['download_latency'] > 3.0] # Pages taking >3s
|
||||
technical_issues['page_speed_issues'] = {
|
||||
'slow_pages_count': len(slow_pages),
|
||||
'avg_load_time': df['download_latency'].mean(),
|
||||
'slowest_pages': slow_pages.nlargest(10, 'download_latency')[['url', 'download_latency']].to_dict('records')
|
||||
}
|
||||
|
||||
return technical_issues
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error analyzing technical issues: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _analyze_performance_metrics(self, crawl_data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze website performance metrics."""
|
||||
try:
|
||||
st.info("⚡ Analyzing performance metrics...")
|
||||
|
||||
if 'crawl_dataframe' not in crawl_data:
|
||||
return {}
|
||||
|
||||
df = crawl_data['crawl_dataframe']
|
||||
|
||||
performance = {
|
||||
'load_time_analysis': {},
|
||||
'content_size_analysis': {},
|
||||
'server_performance': {},
|
||||
'optimization_opportunities': []
|
||||
}
|
||||
|
||||
# Load Time Analysis
|
||||
if 'download_latency' in df.columns:
|
||||
load_times = df['download_latency'].dropna()
|
||||
performance['load_time_analysis'] = {
|
||||
'avg_load_time': load_times.mean(),
|
||||
'median_load_time': load_times.median(),
|
||||
'p95_load_time': load_times.quantile(0.95),
|
||||
'fastest_page': load_times.min(),
|
||||
'slowest_page': load_times.max(),
|
||||
'pages_over_3s': len(load_times[load_times > 3]),
|
||||
'performance_distribution': {
|
||||
'fast_pages': len(load_times[load_times <= 1]),
|
||||
'moderate_pages': len(load_times[(load_times > 1) & (load_times <= 3)]),
|
||||
'slow_pages': len(load_times[load_times > 3])
|
||||
}
|
||||
}
|
||||
|
||||
# Content Size Analysis
|
||||
if 'size' in df.columns:
|
||||
sizes = df['size'].dropna()
|
||||
performance['content_size_analysis'] = {
|
||||
'avg_page_size': sizes.mean(),
|
||||
'median_page_size': sizes.median(),
|
||||
'largest_page': sizes.max(),
|
||||
'smallest_page': sizes.min(),
|
||||
'pages_over_1mb': len(sizes[sizes > 1048576]), # 1MB
|
||||
'total_content_size': sizes.sum()
|
||||
}
|
||||
|
||||
# Server Performance
|
||||
status_codes = df['status'].value_counts()
|
||||
total_pages = len(df)
|
||||
performance['server_performance'] = {
|
||||
'success_rate': status_codes.get(200, 0) / total_pages * 100,
|
||||
'error_rate': sum(status_codes.get(code, 0) for code in range(400, 600)) / total_pages * 100,
|
||||
'redirect_rate': sum(status_codes.get(code, 0) for code in [301, 302, 303, 307, 308]) / total_pages * 100
|
||||
}
|
||||
|
||||
return performance
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error analyzing performance: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _analyze_content_structure(self, crawl_data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze content structure and SEO elements."""
|
||||
try:
|
||||
st.info("📊 Analyzing content structure...")
|
||||
|
||||
if 'crawl_dataframe' not in crawl_data:
|
||||
return {}
|
||||
|
||||
df = crawl_data['crawl_dataframe']
|
||||
|
||||
content_analysis = {
|
||||
'title_analysis': {},
|
||||
'meta_description_analysis': {},
|
||||
'heading_structure': {},
|
||||
'internal_linking': {},
|
||||
'content_optimization': {}
|
||||
}
|
||||
|
||||
# Title Analysis
|
||||
if 'title' in df.columns:
|
||||
titles = df['title'].dropna()
|
||||
title_lengths = titles.str.len()
|
||||
|
||||
content_analysis['title_analysis'] = {
|
||||
'avg_title_length': title_lengths.mean(),
|
||||
'title_length_distribution': {
|
||||
'too_short': len(title_lengths[title_lengths < 30]),
|
||||
'optimal': len(title_lengths[(title_lengths >= 30) & (title_lengths <= 60)]),
|
||||
'too_long': len(title_lengths[title_lengths > 60])
|
||||
},
|
||||
'duplicate_titles': len(titles.value_counts()[titles.value_counts() > 1]),
|
||||
'missing_titles': len(df) - len(titles)
|
||||
}
|
||||
|
||||
# Meta Description Analysis
|
||||
if 'meta_desc' in df.columns:
|
||||
meta_descs = df['meta_desc'].dropna()
|
||||
meta_lengths = meta_descs.str.len()
|
||||
|
||||
content_analysis['meta_description_analysis'] = {
|
||||
'avg_meta_length': meta_lengths.mean(),
|
||||
'meta_length_distribution': {
|
||||
'too_short': len(meta_lengths[meta_lengths < 120]),
|
||||
'optimal': len(meta_lengths[(meta_lengths >= 120) & (meta_lengths <= 160)]),
|
||||
'too_long': len(meta_lengths[meta_lengths > 160])
|
||||
},
|
||||
'missing_meta_descriptions': len(df) - len(meta_descs)
|
||||
}
|
||||
|
||||
# Heading Structure Analysis
|
||||
heading_cols = [col for col in df.columns if col.startswith('h') and col[1:].isdigit()]
|
||||
if heading_cols:
|
||||
heading_analysis = {}
|
||||
for col in heading_cols:
|
||||
headings = df[col].dropna()
|
||||
heading_analysis[f'{col}_usage'] = {
|
||||
'pages_with_heading': len(headings),
|
||||
'usage_rate': len(headings) / len(df) * 100,
|
||||
'avg_length': headings.str.len().mean() if len(headings) > 0 else 0
|
||||
}
|
||||
content_analysis['heading_structure'] = heading_analysis
|
||||
|
||||
# Internal Linking Analysis
|
||||
if 'links_internal' in df.columns:
|
||||
internal_links = df['links_internal'].apply(lambda x: len(x) if isinstance(x, list) else 0)
|
||||
content_analysis['internal_linking'] = {
|
||||
'avg_internal_links': internal_links.mean(),
|
||||
'pages_with_no_internal_links': len(internal_links[internal_links == 0]),
|
||||
'max_internal_links': internal_links.max(),
|
||||
'internal_link_distribution': internal_links.describe().to_dict()
|
||||
}
|
||||
|
||||
return content_analysis
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error analyzing content structure: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _analyze_url_structure(self, crawl_data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze URL structure and optimization using adv.url_to_df."""
|
||||
try:
|
||||
st.info("🔗 Analyzing URL structure...")
|
||||
|
||||
if 'crawl_dataframe' not in crawl_data:
|
||||
return {}
|
||||
|
||||
df = crawl_data['crawl_dataframe']
|
||||
urls = df['url'].tolist()
|
||||
|
||||
# Use advertools to analyze URL structure
|
||||
url_df = adv.url_to_df(urls)
|
||||
|
||||
url_analysis = {
|
||||
'url_length_analysis': {},
|
||||
'url_structure_patterns': {},
|
||||
'url_optimization': {},
|
||||
'path_analysis': {}
|
||||
}
|
||||
|
||||
# URL Length Analysis
|
||||
url_lengths = url_df['url'].str.len()
|
||||
url_analysis['url_length_analysis'] = {
|
||||
'avg_url_length': url_lengths.mean(),
|
||||
'max_url_length': url_lengths.max(),
|
||||
'long_urls_count': len(url_lengths[url_lengths > 100]),
|
||||
'url_length_distribution': url_lengths.describe().to_dict()
|
||||
}
|
||||
|
||||
# Path Depth Analysis
|
||||
if 'dir_1' in url_df.columns:
|
||||
path_depths = url_df.apply(lambda row: sum(1 for i in range(1, 10) if f'dir_{i}' in row and pd.notna(row[f'dir_{i}'])), axis=1)
|
||||
url_analysis['path_analysis'] = {
|
||||
'avg_path_depth': path_depths.mean(),
|
||||
'max_path_depth': path_depths.max(),
|
||||
'deep_paths_count': len(path_depths[path_depths > 4]),
|
||||
'path_depth_distribution': path_depths.value_counts().to_dict()
|
||||
}
|
||||
|
||||
# URL Structure Patterns
|
||||
domains = url_df['netloc'].value_counts()
|
||||
schemes = url_df['scheme'].value_counts()
|
||||
|
||||
url_analysis['url_structure_patterns'] = {
|
||||
'domains_found': domains.to_dict(),
|
||||
'schemes_used': schemes.to_dict(),
|
||||
'subdomain_usage': len(url_df[url_df['netloc'].str.contains('\.', regex=True)]),
|
||||
'https_usage': schemes.get('https', 0) / len(url_df) * 100
|
||||
}
|
||||
|
||||
# URL Optimization Issues
|
||||
optimization_issues = []
|
||||
|
||||
# Check for non-HTTPS URLs
|
||||
if schemes.get('http', 0) > 0:
|
||||
optimization_issues.append(f"{schemes.get('http', 0)} pages not using HTTPS")
|
||||
|
||||
# Check for long URLs
|
||||
long_urls = len(url_lengths[url_lengths > 100])
|
||||
if long_urls > 0:
|
||||
optimization_issues.append(f"{long_urls} URLs are too long (>100 characters)")
|
||||
|
||||
# Check for deep paths
|
||||
if 'path_analysis' in url_analysis:
|
||||
deep_paths = url_analysis['path_analysis']['deep_paths_count']
|
||||
if deep_paths > 0:
|
||||
optimization_issues.append(f"{deep_paths} URLs have deep path structures (>4 levels)")
|
||||
|
||||
url_analysis['url_optimization'] = {
|
||||
'issues_found': len(optimization_issues),
|
||||
'optimization_recommendations': optimization_issues
|
||||
}
|
||||
|
||||
return url_analysis
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error analyzing URL structure: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _analyze_image_seo(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Analyze image SEO using adv.crawl_images."""
|
||||
try:
|
||||
st.info("🖼️ Analyzing image SEO...")
|
||||
|
||||
# Create image crawl output file
|
||||
image_file = os.path.join(self.temp_dir, "image_crawl.jl")
|
||||
|
||||
# Crawl images
|
||||
adv.crawl_images(
|
||||
url_list=[website_url],
|
||||
output_file=image_file,
|
||||
custom_settings={
|
||||
'DEPTH_LIMIT': 2,
|
||||
'CLOSESPIDER_PAGECOUNT': 100,
|
||||
'DOWNLOAD_DELAY': 1
|
||||
}
|
||||
)
|
||||
|
||||
image_analysis = {
|
||||
'image_count': 0,
|
||||
'alt_text_analysis': {},
|
||||
'image_format_analysis': {},
|
||||
'image_size_analysis': {},
|
||||
'optimization_opportunities': []
|
||||
}
|
||||
|
||||
if os.path.exists(image_file):
|
||||
image_df = pd.read_json(image_file, lines=True)
|
||||
|
||||
image_analysis['image_count'] = len(image_df)
|
||||
|
||||
# Alt text analysis
|
||||
if 'img_alt' in image_df.columns:
|
||||
alt_texts = image_df['img_alt'].dropna()
|
||||
missing_alt = len(image_df) - len(alt_texts)
|
||||
|
||||
image_analysis['alt_text_analysis'] = {
|
||||
'images_with_alt': len(alt_texts),
|
||||
'images_missing_alt': missing_alt,
|
||||
'alt_text_coverage': len(alt_texts) / len(image_df) * 100,
|
||||
'avg_alt_length': alt_texts.str.len().mean() if len(alt_texts) > 0 else 0
|
||||
}
|
||||
|
||||
# Image format analysis
|
||||
if 'img_src' in image_df.columns:
|
||||
# Extract file extensions
|
||||
extensions = image_df['img_src'].str.extract(r'\.([a-zA-Z]{2,4})(?:\?|$)')
|
||||
format_counts = extensions[0].value_counts()
|
||||
|
||||
image_analysis['image_format_analysis'] = {
|
||||
'format_distribution': format_counts.to_dict(),
|
||||
'modern_format_usage': format_counts.get('webp', 0) + format_counts.get('avif', 0)
|
||||
}
|
||||
|
||||
return image_analysis
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error analyzing images: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _analyze_security_headers(self, website_url: str) -> Dict[str, Any]:
|
||||
"""Analyze security headers using adv.crawl_headers."""
|
||||
try:
|
||||
st.info("🛡️ Analyzing security headers...")
|
||||
|
||||
# Create headers output file
|
||||
headers_file = os.path.join(self.temp_dir, "security_headers.jl")
|
||||
|
||||
# Crawl headers
|
||||
adv.crawl_headers([website_url], output_file=headers_file)
|
||||
|
||||
security_analysis = {
|
||||
'security_headers_present': {},
|
||||
'security_score': 0,
|
||||
'security_recommendations': []
|
||||
}
|
||||
|
||||
if os.path.exists(headers_file):
|
||||
headers_df = pd.read_json(headers_file, lines=True)
|
||||
|
||||
# Check for important security headers
|
||||
security_headers = {
|
||||
'X-Frame-Options': 'resp_headers_X-Frame-Options',
|
||||
'X-Content-Type-Options': 'resp_headers_X-Content-Type-Options',
|
||||
'X-XSS-Protection': 'resp_headers_X-XSS-Protection',
|
||||
'Strict-Transport-Security': 'resp_headers_Strict-Transport-Security',
|
||||
'Content-Security-Policy': 'resp_headers_Content-Security-Policy',
|
||||
'Referrer-Policy': 'resp_headers_Referrer-Policy'
|
||||
}
|
||||
|
||||
headers_present = {}
|
||||
for header_name, column_name in security_headers.items():
|
||||
is_present = column_name in headers_df.columns and headers_df[column_name].notna().any()
|
||||
headers_present[header_name] = is_present
|
||||
|
||||
security_analysis['security_headers_present'] = headers_present
|
||||
|
||||
# Calculate security score
|
||||
present_count = sum(headers_present.values())
|
||||
security_analysis['security_score'] = (present_count / len(security_headers)) * 100
|
||||
|
||||
# Generate recommendations
|
||||
recommendations = []
|
||||
for header_name, is_present in headers_present.items():
|
||||
if not is_present:
|
||||
recommendations.append(f"Add {header_name} header for improved security")
|
||||
|
||||
security_analysis['security_recommendations'] = recommendations
|
||||
|
||||
return security_analysis
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error analyzing security headers: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _analyze_mobile_seo(self, crawl_data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Analyze mobile SEO factors."""
|
||||
try:
|
||||
st.info("📱 Analyzing mobile SEO factors...")
|
||||
|
||||
if 'crawl_dataframe' not in crawl_data:
|
||||
return {}
|
||||
|
||||
df = crawl_data['crawl_dataframe']
|
||||
|
||||
mobile_analysis = {
|
||||
'viewport_analysis': {},
|
||||
'mobile_optimization': {},
|
||||
'responsive_design_indicators': {}
|
||||
}
|
||||
|
||||
# Viewport meta tag analysis
|
||||
if 'viewport' in df.columns:
|
||||
viewport_present = df['viewport'].notna().sum()
|
||||
mobile_analysis['viewport_analysis'] = {
|
||||
'pages_with_viewport': viewport_present,
|
||||
'viewport_coverage': viewport_present / len(df) * 100,
|
||||
'pages_missing_viewport': len(df) - viewport_present
|
||||
}
|
||||
|
||||
# Check for mobile-specific meta tags and indicators
|
||||
mobile_indicators = []
|
||||
|
||||
# Check for touch icons
|
||||
if any('touch-icon' in col for col in df.columns):
|
||||
mobile_indicators.append("Touch icons configured")
|
||||
|
||||
# Check for responsive design indicators in content
|
||||
# This is a simplified check - in practice, you'd analyze CSS and page structure
|
||||
mobile_analysis['mobile_optimization'] = {
|
||||
'mobile_indicators_found': len(mobile_indicators),
|
||||
'mobile_indicators': mobile_indicators
|
||||
}
|
||||
|
||||
return mobile_analysis
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error analyzing mobile SEO: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _generate_technical_recommendations(self, results: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Generate AI-powered technical SEO recommendations."""
|
||||
try:
|
||||
st.info("🤖 Generating technical recommendations...")
|
||||
|
||||
# Prepare technical analysis summary for AI
|
||||
technical_summary = {
|
||||
'website_url': results.get('website_url', ''),
|
||||
'pages_crawled': results.get('crawl_overview', {}).get('pages_crawled', 0),
|
||||
'error_count': results.get('technical_issues', {}).get('http_errors', {}).get('total_errors', 0),
|
||||
'avg_load_time': results.get('performance_analysis', {}).get('load_time_analysis', {}).get('avg_load_time', 0),
|
||||
'security_score': results.get('security_headers', {}).get('security_score', 0),
|
||||
'missing_titles': results.get('content_analysis', {}).get('title_analysis', {}).get('missing_titles', 0),
|
||||
'missing_meta_desc': results.get('content_analysis', {}).get('meta_description_analysis', {}).get('missing_meta_descriptions', 0)
|
||||
}
|
||||
|
||||
# Generate AI recommendations
|
||||
prompt = f"""
|
||||
As a technical SEO expert, analyze this comprehensive website audit and provide prioritized recommendations:
|
||||
|
||||
WEBSITE: {technical_summary['website_url']}
|
||||
PAGES ANALYZED: {technical_summary['pages_crawled']}
|
||||
|
||||
TECHNICAL ISSUES:
|
||||
- HTTP Errors: {technical_summary['error_count']}
|
||||
- Average Load Time: {technical_summary['avg_load_time']:.2f}s
|
||||
- Security Score: {technical_summary['security_score']:.1f}%
|
||||
- Missing Titles: {technical_summary['missing_titles']}
|
||||
- Missing Meta Descriptions: {technical_summary['missing_meta_desc']}
|
||||
|
||||
PROVIDE:
|
||||
1. Critical Issues (Fix Immediately)
|
||||
2. High Priority Optimizations
|
||||
3. Medium Priority Improvements
|
||||
4. Long-term Technical Strategy
|
||||
5. Specific Implementation Steps
|
||||
6. Expected Impact Assessment
|
||||
|
||||
Format as JSON with clear priorities and actionable recommendations.
|
||||
"""
|
||||
|
||||
ai_response = llm_text_gen(
|
||||
prompt=prompt,
|
||||
system_prompt="You are a senior technical SEO specialist with expertise in website optimization, Core Web Vitals, and search engine best practices.",
|
||||
response_format="json_object"
|
||||
)
|
||||
|
||||
if ai_response:
|
||||
return ai_response
|
||||
else:
|
||||
return {'recommendations': ['AI recommendations temporarily unavailable']}
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error generating recommendations: {str(e)}")
|
||||
return {}
|
||||
|
||||
def _find_redirect_chains(self, redirects_df: pd.DataFrame) -> List[Dict[str, Any]]:
|
||||
"""Find redirect chains in the crawled data."""
|
||||
# Simplified redirect chain detection
|
||||
# In a full implementation, you'd trace the redirect paths
|
||||
redirect_chains = []
|
||||
|
||||
if len(redirects_df) > 0:
|
||||
# Group redirects by status code
|
||||
for status_code in redirects_df['status'].unique():
|
||||
status_redirects = redirects_df[redirects_df['status'] == status_code]
|
||||
redirect_chains.append({
|
||||
'status_code': int(status_code),
|
||||
'count': len(status_redirects),
|
||||
'examples': status_redirects['url'].head(5).tolist()
|
||||
})
|
||||
|
||||
return redirect_chains
|
||||
968
ToBeMigrated/ai_seo_tools/technical_seo_crawler/ui.py
Normal file
968
ToBeMigrated/ai_seo_tools/technical_seo_crawler/ui.py
Normal file
@@ -0,0 +1,968 @@
|
||||
"""
|
||||
Technical SEO Crawler UI with Comprehensive Analysis Dashboard.
|
||||
|
||||
This module provides a professional Streamlit interface for the Technical SEO Crawler
|
||||
with detailed analysis results, visualization, and export capabilities.
|
||||
"""
|
||||
|
||||
import streamlit as st
|
||||
import pandas as pd
|
||||
from typing import Dict, Any, List
|
||||
import json
|
||||
from datetime import datetime
|
||||
import io
|
||||
import base64
|
||||
import plotly.express as px
|
||||
import plotly.graph_objects as go
|
||||
from plotly.subplots import make_subplots
|
||||
|
||||
from .crawler import TechnicalSEOCrawler
|
||||
from lib.alwrity_ui.dashboard_styles import apply_dashboard_style, render_dashboard_header
|
||||
|
||||
class TechnicalSEOCrawlerUI:
|
||||
"""Professional UI for Technical SEO Crawler."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the Technical SEO Crawler UI."""
|
||||
self.crawler = TechnicalSEOCrawler()
|
||||
|
||||
# Apply dashboard styling
|
||||
apply_dashboard_style()
|
||||
|
||||
def render(self):
|
||||
"""Render the Technical SEO Crawler interface."""
|
||||
|
||||
# Enhanced dashboard header
|
||||
render_dashboard_header(
|
||||
"🔧 Technical SEO Crawler",
|
||||
"Comprehensive site-wide technical SEO analysis with AI-powered recommendations. Identify and fix technical issues that impact your search rankings."
|
||||
)
|
||||
|
||||
# Main content area
|
||||
with st.container():
|
||||
# Analysis input form
|
||||
self._render_crawler_form()
|
||||
|
||||
# Session state for results
|
||||
if 'technical_seo_results' in st.session_state and st.session_state.technical_seo_results:
|
||||
st.markdown("---")
|
||||
self._render_results_dashboard(st.session_state.technical_seo_results)
|
||||
|
||||
def _render_crawler_form(self):
|
||||
"""Render the crawler configuration form."""
|
||||
st.markdown("## 🚀 Configure Technical SEO Audit")
|
||||
|
||||
with st.form("technical_seo_crawler_form"):
|
||||
# Website URL input
|
||||
col1, col2 = st.columns([3, 1])
|
||||
|
||||
with col1:
|
||||
website_url = st.text_input(
|
||||
"🌐 Website URL to Audit",
|
||||
placeholder="https://yourwebsite.com",
|
||||
help="Enter the website URL for comprehensive technical SEO analysis"
|
||||
)
|
||||
|
||||
with col2:
|
||||
audit_type = st.selectbox(
|
||||
"🎯 Audit Type",
|
||||
options=["Standard", "Deep", "Quick"],
|
||||
help="Choose the depth of analysis"
|
||||
)
|
||||
|
||||
# Crawl configuration
|
||||
st.markdown("### ⚙️ Crawl Configuration")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
if audit_type == "Quick":
|
||||
crawl_depth = st.slider("Crawl Depth", 1, 2, 1)
|
||||
max_pages = st.slider("Max Pages", 10, 100, 50)
|
||||
elif audit_type == "Deep":
|
||||
crawl_depth = st.slider("Crawl Depth", 1, 5, 4)
|
||||
max_pages = st.slider("Max Pages", 100, 1000, 500)
|
||||
else: # Standard
|
||||
crawl_depth = st.slider("Crawl Depth", 1, 4, 3)
|
||||
max_pages = st.slider("Max Pages", 50, 500, 200)
|
||||
|
||||
with col2:
|
||||
analyze_images = st.checkbox(
|
||||
"🖼️ Analyze Images",
|
||||
value=True,
|
||||
help="Include image SEO analysis"
|
||||
)
|
||||
|
||||
analyze_security = st.checkbox(
|
||||
"🛡️ Security Headers",
|
||||
value=True,
|
||||
help="Analyze security headers"
|
||||
)
|
||||
|
||||
with col3:
|
||||
analyze_mobile = st.checkbox(
|
||||
"📱 Mobile SEO",
|
||||
value=True,
|
||||
help="Include mobile SEO analysis"
|
||||
)
|
||||
|
||||
ai_recommendations = st.checkbox(
|
||||
"🤖 AI Recommendations",
|
||||
value=True,
|
||||
help="Generate AI-powered recommendations"
|
||||
)
|
||||
|
||||
# Analysis scope
|
||||
st.markdown("### 🎯 Analysis Scope")
|
||||
|
||||
analysis_options = st.multiselect(
|
||||
"Select Analysis Components",
|
||||
options=[
|
||||
"Technical Issues Detection",
|
||||
"Performance Analysis",
|
||||
"Content Structure Analysis",
|
||||
"URL Structure Optimization",
|
||||
"Internal Linking Analysis",
|
||||
"Duplicate Content Detection"
|
||||
],
|
||||
default=[
|
||||
"Technical Issues Detection",
|
||||
"Performance Analysis",
|
||||
"Content Structure Analysis"
|
||||
],
|
||||
help="Choose which analysis components to include"
|
||||
)
|
||||
|
||||
# Submit button
|
||||
submitted = st.form_submit_button(
|
||||
"🚀 Start Technical SEO Audit",
|
||||
use_container_width=True,
|
||||
type="primary"
|
||||
)
|
||||
|
||||
if submitted:
|
||||
# Validate inputs
|
||||
if not website_url or not website_url.startswith(('http://', 'https://')):
|
||||
st.error("❌ Please enter a valid website URL starting with http:// or https://")
|
||||
return
|
||||
|
||||
# Run technical SEO analysis
|
||||
self._run_technical_analysis(
|
||||
website_url=website_url,
|
||||
crawl_depth=crawl_depth,
|
||||
max_pages=max_pages,
|
||||
options={
|
||||
'analyze_images': analyze_images,
|
||||
'analyze_security': analyze_security,
|
||||
'analyze_mobile': analyze_mobile,
|
||||
'ai_recommendations': ai_recommendations,
|
||||
'analysis_scope': analysis_options
|
||||
}
|
||||
)
|
||||
|
||||
def _run_technical_analysis(self, website_url: str, crawl_depth: int,
|
||||
max_pages: int, options: Dict[str, Any]):
|
||||
"""Run the technical SEO analysis."""
|
||||
|
||||
try:
|
||||
with st.spinner("🔄 Running Comprehensive Technical SEO Audit..."):
|
||||
|
||||
# Initialize progress tracking
|
||||
progress_bar = st.progress(0)
|
||||
status_text = st.empty()
|
||||
|
||||
# Update progress
|
||||
progress_bar.progress(10)
|
||||
status_text.text("🚀 Initializing technical SEO crawler...")
|
||||
|
||||
# Run comprehensive analysis
|
||||
results = self.crawler.analyze_website_technical_seo(
|
||||
website_url=website_url,
|
||||
crawl_depth=crawl_depth,
|
||||
max_pages=max_pages
|
||||
)
|
||||
|
||||
progress_bar.progress(100)
|
||||
status_text.text("✅ Technical SEO audit complete!")
|
||||
|
||||
# Store results in session state
|
||||
st.session_state.technical_seo_results = results
|
||||
|
||||
# Clear progress indicators
|
||||
progress_bar.empty()
|
||||
status_text.empty()
|
||||
|
||||
if 'error' in results:
|
||||
st.error(f"❌ Analysis failed: {results['error']}")
|
||||
else:
|
||||
st.success("🎉 Technical SEO Audit completed successfully!")
|
||||
st.balloons()
|
||||
|
||||
# Rerun to show results
|
||||
st.rerun()
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"❌ Error running technical analysis: {str(e)}")
|
||||
|
||||
def _render_results_dashboard(self, results: Dict[str, Any]):
|
||||
"""Render the comprehensive results dashboard."""
|
||||
|
||||
if 'error' in results:
|
||||
st.error(f"❌ Analysis Error: {results['error']}")
|
||||
return
|
||||
|
||||
# Results header
|
||||
st.markdown("## 📊 Technical SEO Audit Results")
|
||||
|
||||
# Key metrics overview
|
||||
self._render_metrics_overview(results)
|
||||
|
||||
# Detailed analysis tabs
|
||||
self._render_detailed_analysis(results)
|
||||
|
||||
# Export functionality
|
||||
self._render_export_options(results)
|
||||
|
||||
def _render_metrics_overview(self, results: Dict[str, Any]):
|
||||
"""Render key metrics overview."""
|
||||
|
||||
st.markdown("### 📈 Audit Overview")
|
||||
|
||||
# Create metrics columns
|
||||
col1, col2, col3, col4, col5, col6 = st.columns(6)
|
||||
|
||||
with col1:
|
||||
pages_crawled = results.get('crawl_overview', {}).get('pages_crawled', 0)
|
||||
st.metric(
|
||||
"🕷️ Pages Crawled",
|
||||
pages_crawled,
|
||||
help="Total pages analyzed"
|
||||
)
|
||||
|
||||
with col2:
|
||||
error_count = results.get('technical_issues', {}).get('http_errors', {}).get('total_errors', 0)
|
||||
st.metric(
|
||||
"❌ HTTP Errors",
|
||||
error_count,
|
||||
delta=f"-{error_count}" if error_count > 0 else None,
|
||||
help="Pages with HTTP errors (4xx, 5xx)"
|
||||
)
|
||||
|
||||
with col3:
|
||||
avg_load_time = results.get('performance_analysis', {}).get('load_time_analysis', {}).get('avg_load_time', 0)
|
||||
st.metric(
|
||||
"⚡ Avg Load Time",
|
||||
f"{avg_load_time:.2f}s",
|
||||
delta=f"+{avg_load_time:.2f}s" if avg_load_time > 3 else None,
|
||||
help="Average page load time"
|
||||
)
|
||||
|
||||
with col4:
|
||||
security_score = results.get('security_headers', {}).get('security_score', 0)
|
||||
st.metric(
|
||||
"🛡️ Security Score",
|
||||
f"{security_score:.0f}%",
|
||||
delta=f"{security_score:.0f}%" if security_score < 100 else None,
|
||||
help="Security headers implementation score"
|
||||
)
|
||||
|
||||
with col5:
|
||||
missing_titles = results.get('content_analysis', {}).get('title_analysis', {}).get('missing_titles', 0)
|
||||
st.metric(
|
||||
"📝 Missing Titles",
|
||||
missing_titles,
|
||||
delta=f"-{missing_titles}" if missing_titles > 0 else None,
|
||||
help="Pages without title tags"
|
||||
)
|
||||
|
||||
with col6:
|
||||
image_count = results.get('image_optimization', {}).get('image_count', 0)
|
||||
st.metric(
|
||||
"🖼️ Images Analyzed",
|
||||
image_count,
|
||||
help="Total images found and analyzed"
|
||||
)
|
||||
|
||||
# Analysis timestamp
|
||||
if results.get('analysis_timestamp'):
|
||||
timestamp = datetime.fromisoformat(results['analysis_timestamp'].replace('Z', '+00:00'))
|
||||
st.caption(f"📅 Audit completed: {timestamp.strftime('%Y-%m-%d %H:%M:%S UTC')}")
|
||||
|
||||
def _render_detailed_analysis(self, results: Dict[str, Any]):
|
||||
"""Render detailed analysis in tabs."""
|
||||
|
||||
# Create main analysis tabs
|
||||
tab1, tab2, tab3, tab4, tab5, tab6, tab7 = st.tabs([
|
||||
"🔍 Technical Issues",
|
||||
"⚡ Performance",
|
||||
"📊 Content Analysis",
|
||||
"🔗 URL Structure",
|
||||
"🖼️ Image SEO",
|
||||
"🛡️ Security",
|
||||
"🤖 AI Recommendations"
|
||||
])
|
||||
|
||||
with tab1:
|
||||
self._render_technical_issues(results.get('technical_issues', {}))
|
||||
|
||||
with tab2:
|
||||
self._render_performance_analysis(results.get('performance_analysis', {}))
|
||||
|
||||
with tab3:
|
||||
self._render_content_analysis(results.get('content_analysis', {}))
|
||||
|
||||
with tab4:
|
||||
self._render_url_structure(results.get('url_structure', {}))
|
||||
|
||||
with tab5:
|
||||
self._render_image_analysis(results.get('image_optimization', {}))
|
||||
|
||||
with tab6:
|
||||
self._render_security_analysis(results.get('security_headers', {}))
|
||||
|
||||
with tab7:
|
||||
self._render_ai_recommendations(results.get('ai_recommendations', {}))
|
||||
|
||||
def _render_technical_issues(self, technical_data: Dict[str, Any]):
|
||||
"""Render technical issues analysis."""
|
||||
|
||||
st.markdown("### 🔍 Technical SEO Issues")
|
||||
|
||||
if not technical_data:
|
||||
st.info("No technical issues data available")
|
||||
return
|
||||
|
||||
# HTTP Errors
|
||||
if technical_data.get('http_errors'):
|
||||
http_errors = technical_data['http_errors']
|
||||
|
||||
st.markdown("#### ❌ HTTP Status Code Errors")
|
||||
|
||||
if http_errors.get('total_errors', 0) > 0:
|
||||
st.error(f"Found {http_errors['total_errors']} pages with HTTP errors!")
|
||||
|
||||
# Error breakdown chart
|
||||
if http_errors.get('error_breakdown'):
|
||||
error_df = pd.DataFrame(
|
||||
list(http_errors['error_breakdown'].items()),
|
||||
columns=['Status Code', 'Count']
|
||||
)
|
||||
|
||||
fig = px.bar(error_df, x='Status Code', y='Count',
|
||||
title="HTTP Error Distribution")
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
# Error pages table
|
||||
if http_errors.get('error_pages'):
|
||||
st.markdown("**Pages with Errors:**")
|
||||
error_pages_df = pd.DataFrame(http_errors['error_pages'])
|
||||
st.dataframe(error_pages_df, use_container_width=True)
|
||||
else:
|
||||
st.success("✅ No HTTP errors found!")
|
||||
|
||||
# Redirect Issues
|
||||
if technical_data.get('redirect_issues'):
|
||||
redirect_data = technical_data['redirect_issues']
|
||||
|
||||
st.markdown("#### 🔄 Redirect Analysis")
|
||||
|
||||
total_redirects = redirect_data.get('total_redirects', 0)
|
||||
|
||||
if total_redirects > 0:
|
||||
st.warning(f"Found {total_redirects} redirect(s)")
|
||||
|
||||
# Redirect types
|
||||
if redirect_data.get('redirect_types'):
|
||||
redirect_df = pd.DataFrame(
|
||||
list(redirect_data['redirect_types'].items()),
|
||||
columns=['Redirect Type', 'Count']
|
||||
)
|
||||
st.bar_chart(redirect_df.set_index('Redirect Type'))
|
||||
else:
|
||||
st.success("✅ No redirects found")
|
||||
|
||||
# Duplicate Content
|
||||
if technical_data.get('duplicate_content'):
|
||||
duplicate_data = technical_data['duplicate_content']
|
||||
|
||||
st.markdown("#### 📋 Duplicate Content Issues")
|
||||
|
||||
duplicate_titles = duplicate_data.get('duplicate_titles', 0)
|
||||
|
||||
if duplicate_titles > 0:
|
||||
st.warning(f"Found {duplicate_titles} duplicate title(s)")
|
||||
|
||||
# Show duplicate title groups
|
||||
if duplicate_data.get('pages_with_duplicate_titles'):
|
||||
duplicate_df = pd.DataFrame(duplicate_data['pages_with_duplicate_titles'])
|
||||
st.dataframe(duplicate_df, use_container_width=True)
|
||||
else:
|
||||
st.success("✅ No duplicate titles found")
|
||||
|
||||
# Missing Elements
|
||||
if technical_data.get('missing_elements'):
|
||||
missing_data = technical_data['missing_elements']
|
||||
|
||||
st.markdown("#### 📝 Missing SEO Elements")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
missing_titles = missing_data.get('missing_titles', 0)
|
||||
if missing_titles > 0:
|
||||
st.error(f"Missing Titles: {missing_titles}")
|
||||
else:
|
||||
st.success("All pages have titles ✅")
|
||||
|
||||
with col2:
|
||||
missing_meta = missing_data.get('missing_meta_desc', 0)
|
||||
if missing_meta > 0:
|
||||
st.error(f"Missing Meta Descriptions: {missing_meta}")
|
||||
else:
|
||||
st.success("All pages have meta descriptions ✅")
|
||||
|
||||
with col3:
|
||||
missing_h1 = missing_data.get('missing_h1', 0)
|
||||
if missing_h1 > 0:
|
||||
st.error(f"Missing H1 tags: {missing_h1}")
|
||||
else:
|
||||
st.success("All pages have H1 tags ✅")
|
||||
|
||||
def _render_performance_analysis(self, performance_data: Dict[str, Any]):
|
||||
"""Render performance analysis."""
|
||||
|
||||
st.markdown("### ⚡ Website Performance Analysis")
|
||||
|
||||
if not performance_data:
|
||||
st.info("No performance data available")
|
||||
return
|
||||
|
||||
# Load Time Analysis
|
||||
if performance_data.get('load_time_analysis'):
|
||||
load_time_data = performance_data['load_time_analysis']
|
||||
|
||||
st.markdown("#### 🚀 Page Load Time Analysis")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
avg_load = load_time_data.get('avg_load_time', 0)
|
||||
st.metric("Average Load Time", f"{avg_load:.2f}s")
|
||||
|
||||
with col2:
|
||||
median_load = load_time_data.get('median_load_time', 0)
|
||||
st.metric("Median Load Time", f"{median_load:.2f}s")
|
||||
|
||||
with col3:
|
||||
p95_load = load_time_data.get('p95_load_time', 0)
|
||||
st.metric("95th Percentile", f"{p95_load:.2f}s")
|
||||
|
||||
# Performance distribution
|
||||
if load_time_data.get('performance_distribution'):
|
||||
perf_dist = load_time_data['performance_distribution']
|
||||
|
||||
# Create pie chart for performance distribution
|
||||
labels = ['Fast (≤1s)', 'Moderate (1-3s)', 'Slow (>3s)']
|
||||
values = [
|
||||
perf_dist.get('fast_pages', 0),
|
||||
perf_dist.get('moderate_pages', 0),
|
||||
perf_dist.get('slow_pages', 0)
|
||||
]
|
||||
|
||||
fig = px.pie(values=values, names=labels,
|
||||
title="Page Load Time Distribution")
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
# Content Size Analysis
|
||||
if performance_data.get('content_size_analysis'):
|
||||
size_data = performance_data['content_size_analysis']
|
||||
|
||||
st.markdown("#### 📦 Content Size Analysis")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
avg_size = size_data.get('avg_page_size', 0)
|
||||
st.metric("Average Page Size", f"{avg_size/1024:.1f} KB")
|
||||
|
||||
with col2:
|
||||
largest_size = size_data.get('largest_page', 0)
|
||||
st.metric("Largest Page", f"{largest_size/1024:.1f} KB")
|
||||
|
||||
with col3:
|
||||
large_pages = size_data.get('pages_over_1mb', 0)
|
||||
st.metric("Pages >1MB", large_pages)
|
||||
|
||||
# Server Performance
|
||||
if performance_data.get('server_performance'):
|
||||
server_data = performance_data['server_performance']
|
||||
|
||||
st.markdown("#### 🖥️ Server Performance")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
success_rate = server_data.get('success_rate', 0)
|
||||
st.metric("Success Rate", f"{success_rate:.1f}%")
|
||||
|
||||
with col2:
|
||||
error_rate = server_data.get('error_rate', 0)
|
||||
st.metric("Error Rate", f"{error_rate:.1f}%")
|
||||
|
||||
with col3:
|
||||
redirect_rate = server_data.get('redirect_rate', 0)
|
||||
st.metric("Redirect Rate", f"{redirect_rate:.1f}%")
|
||||
|
||||
def _render_content_analysis(self, content_data: Dict[str, Any]):
|
||||
"""Render content structure analysis."""
|
||||
|
||||
st.markdown("### 📊 Content Structure Analysis")
|
||||
|
||||
if not content_data:
|
||||
st.info("No content analysis data available")
|
||||
return
|
||||
|
||||
# Title Analysis
|
||||
if content_data.get('title_analysis'):
|
||||
title_data = content_data['title_analysis']
|
||||
|
||||
st.markdown("#### 📝 Title Tag Analysis")
|
||||
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
avg_title_length = title_data.get('avg_title_length', 0)
|
||||
st.metric("Average Title Length", f"{avg_title_length:.0f} chars")
|
||||
|
||||
duplicate_titles = title_data.get('duplicate_titles', 0)
|
||||
st.metric("Duplicate Titles", duplicate_titles)
|
||||
|
||||
with col2:
|
||||
# Title length distribution
|
||||
if title_data.get('title_length_distribution'):
|
||||
length_dist = title_data['title_length_distribution']
|
||||
|
||||
labels = ['Too Short (<30)', 'Optimal (30-60)', 'Too Long (>60)']
|
||||
values = [
|
||||
length_dist.get('too_short', 0),
|
||||
length_dist.get('optimal', 0),
|
||||
length_dist.get('too_long', 0)
|
||||
]
|
||||
|
||||
fig = px.pie(values=values, names=labels,
|
||||
title="Title Length Distribution")
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
# Meta Description Analysis
|
||||
if content_data.get('meta_description_analysis'):
|
||||
meta_data = content_data['meta_description_analysis']
|
||||
|
||||
st.markdown("#### 🏷️ Meta Description Analysis")
|
||||
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
avg_meta_length = meta_data.get('avg_meta_length', 0)
|
||||
st.metric("Average Meta Length", f"{avg_meta_length:.0f} chars")
|
||||
|
||||
missing_meta = meta_data.get('missing_meta_descriptions', 0)
|
||||
st.metric("Missing Meta Descriptions", missing_meta)
|
||||
|
||||
with col2:
|
||||
# Meta length distribution
|
||||
if meta_data.get('meta_length_distribution'):
|
||||
meta_dist = meta_data['meta_length_distribution']
|
||||
|
||||
labels = ['Too Short (<120)', 'Optimal (120-160)', 'Too Long (>160)']
|
||||
values = [
|
||||
meta_dist.get('too_short', 0),
|
||||
meta_dist.get('optimal', 0),
|
||||
meta_dist.get('too_long', 0)
|
||||
]
|
||||
|
||||
fig = px.pie(values=values, names=labels,
|
||||
title="Meta Description Length Distribution")
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
# Heading Structure
|
||||
if content_data.get('heading_structure'):
|
||||
heading_data = content_data['heading_structure']
|
||||
|
||||
st.markdown("#### 📋 Heading Structure Analysis")
|
||||
|
||||
# Create heading usage chart
|
||||
heading_usage = []
|
||||
for heading_type, data in heading_data.items():
|
||||
heading_usage.append({
|
||||
'Heading': heading_type.replace('_usage', '').upper(),
|
||||
'Usage Rate': data.get('usage_rate', 0),
|
||||
'Pages': data.get('pages_with_heading', 0)
|
||||
})
|
||||
|
||||
if heading_usage:
|
||||
heading_df = pd.DataFrame(heading_usage)
|
||||
|
||||
fig = px.bar(heading_df, x='Heading', y='Usage Rate',
|
||||
title="Heading Tag Usage Rates")
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
st.dataframe(heading_df, use_container_width=True)
|
||||
|
||||
def _render_url_structure(self, url_data: Dict[str, Any]):
|
||||
"""Render URL structure analysis."""
|
||||
|
||||
st.markdown("### 🔗 URL Structure Analysis")
|
||||
|
||||
if not url_data:
|
||||
st.info("No URL structure data available")
|
||||
return
|
||||
|
||||
# URL Length Analysis
|
||||
if url_data.get('url_length_analysis'):
|
||||
length_data = url_data['url_length_analysis']
|
||||
|
||||
st.markdown("#### 📏 URL Length Analysis")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
avg_length = length_data.get('avg_url_length', 0)
|
||||
st.metric("Average URL Length", f"{avg_length:.0f} chars")
|
||||
|
||||
with col2:
|
||||
max_length = length_data.get('max_url_length', 0)
|
||||
st.metric("Longest URL", f"{max_length:.0f} chars")
|
||||
|
||||
with col3:
|
||||
long_urls = length_data.get('long_urls_count', 0)
|
||||
st.metric("URLs >100 chars", long_urls)
|
||||
|
||||
# URL Structure Patterns
|
||||
if url_data.get('url_structure_patterns'):
|
||||
pattern_data = url_data['url_structure_patterns']
|
||||
|
||||
st.markdown("#### 🏗️ URL Structure Patterns")
|
||||
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
with col1:
|
||||
https_usage = pattern_data.get('https_usage', 0)
|
||||
st.metric("HTTPS Usage", f"{https_usage:.1f}%")
|
||||
|
||||
with col2:
|
||||
subdomain_usage = pattern_data.get('subdomain_usage', 0)
|
||||
st.metric("Subdomains Found", subdomain_usage)
|
||||
|
||||
# Path Analysis
|
||||
if url_data.get('path_analysis'):
|
||||
path_data = url_data['path_analysis']
|
||||
|
||||
st.markdown("#### 📂 Path Depth Analysis")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
avg_depth = path_data.get('avg_path_depth', 0)
|
||||
st.metric("Average Path Depth", f"{avg_depth:.1f}")
|
||||
|
||||
with col2:
|
||||
max_depth = path_data.get('max_path_depth', 0)
|
||||
st.metric("Maximum Depth", max_depth)
|
||||
|
||||
with col3:
|
||||
deep_paths = path_data.get('deep_paths_count', 0)
|
||||
st.metric("Deep Paths (>4)", deep_paths)
|
||||
|
||||
# Optimization Issues
|
||||
if url_data.get('url_optimization'):
|
||||
opt_data = url_data['url_optimization']
|
||||
|
||||
st.markdown("#### ⚠️ URL Optimization Issues")
|
||||
|
||||
issues_found = opt_data.get('issues_found', 0)
|
||||
recommendations = opt_data.get('optimization_recommendations', [])
|
||||
|
||||
if issues_found > 0:
|
||||
st.warning(f"Found {issues_found} URL optimization issue(s)")
|
||||
|
||||
for rec in recommendations:
|
||||
st.write(f"• {rec}")
|
||||
else:
|
||||
st.success("✅ No URL optimization issues found")
|
||||
|
||||
def _render_image_analysis(self, image_data: Dict[str, Any]):
|
||||
"""Render image SEO analysis."""
|
||||
|
||||
st.markdown("### 🖼️ Image SEO Analysis")
|
||||
|
||||
if not image_data:
|
||||
st.info("No image analysis data available")
|
||||
return
|
||||
|
||||
# Image overview
|
||||
image_count = image_data.get('image_count', 0)
|
||||
st.metric("Total Images Found", image_count)
|
||||
|
||||
if image_count > 0:
|
||||
# Alt text analysis
|
||||
if image_data.get('alt_text_analysis'):
|
||||
alt_data = image_data['alt_text_analysis']
|
||||
|
||||
st.markdown("#### 📝 Alt Text Analysis")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
images_with_alt = alt_data.get('images_with_alt', 0)
|
||||
st.metric("Images with Alt Text", images_with_alt)
|
||||
|
||||
with col2:
|
||||
images_missing_alt = alt_data.get('images_missing_alt', 0)
|
||||
st.metric("Missing Alt Text", images_missing_alt)
|
||||
|
||||
with col3:
|
||||
alt_coverage = alt_data.get('alt_text_coverage', 0)
|
||||
st.metric("Alt Text Coverage", f"{alt_coverage:.1f}%")
|
||||
|
||||
# Image format analysis
|
||||
if image_data.get('image_format_analysis'):
|
||||
format_data = image_data['image_format_analysis']
|
||||
|
||||
st.markdown("#### 🎨 Image Format Analysis")
|
||||
|
||||
if format_data.get('format_distribution'):
|
||||
format_dist = format_data['format_distribution']
|
||||
|
||||
format_df = pd.DataFrame(
|
||||
list(format_dist.items()),
|
||||
columns=['Format', 'Count']
|
||||
)
|
||||
|
||||
fig = px.pie(format_df, values='Count', names='Format',
|
||||
title="Image Format Distribution")
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
modern_formats = format_data.get('modern_format_usage', 0)
|
||||
st.metric("Modern Formats (WebP/AVIF)", modern_formats)
|
||||
else:
|
||||
st.info("No images found to analyze")
|
||||
|
||||
def _render_security_analysis(self, security_data: Dict[str, Any]):
|
||||
"""Render security analysis."""
|
||||
|
||||
st.markdown("### 🛡️ Security Headers Analysis")
|
||||
|
||||
if not security_data:
|
||||
st.info("No security analysis data available")
|
||||
return
|
||||
|
||||
# Security score
|
||||
security_score = security_data.get('security_score', 0)
|
||||
|
||||
col1, col2 = st.columns([1, 2])
|
||||
|
||||
with col1:
|
||||
st.metric("Security Score", f"{security_score:.0f}%")
|
||||
|
||||
if security_score >= 80:
|
||||
st.success("🔒 Good security posture")
|
||||
elif security_score >= 50:
|
||||
st.warning("⚠️ Moderate security")
|
||||
else:
|
||||
st.error("🚨 Poor security posture")
|
||||
|
||||
with col2:
|
||||
# Security headers status
|
||||
if security_data.get('security_headers_present'):
|
||||
headers_status = security_data['security_headers_present']
|
||||
|
||||
st.markdown("**Security Headers Status:**")
|
||||
|
||||
for header, present in headers_status.items():
|
||||
status = "✅" if present else "❌"
|
||||
st.write(f"{status} {header}")
|
||||
|
||||
# Security recommendations
|
||||
if security_data.get('security_recommendations'):
|
||||
recommendations = security_data['security_recommendations']
|
||||
|
||||
if recommendations:
|
||||
st.markdown("#### 🔧 Security Recommendations")
|
||||
|
||||
for rec in recommendations:
|
||||
st.write(f"• {rec}")
|
||||
else:
|
||||
st.success("✅ All security headers properly configured")
|
||||
|
||||
def _render_ai_recommendations(self, ai_data: Dict[str, Any]):
|
||||
"""Render AI-generated recommendations."""
|
||||
|
||||
st.markdown("### 🤖 AI-Powered Technical Recommendations")
|
||||
|
||||
if not ai_data:
|
||||
st.info("No AI recommendations available")
|
||||
return
|
||||
|
||||
# Critical Issues
|
||||
if ai_data.get('critical_issues'):
|
||||
st.markdown("#### 🚨 Critical Issues (Fix Immediately)")
|
||||
|
||||
critical_issues = ai_data['critical_issues']
|
||||
for issue in critical_issues:
|
||||
st.error(f"🚨 {issue}")
|
||||
|
||||
# High Priority
|
||||
if ai_data.get('high_priority'):
|
||||
st.markdown("#### 🔥 High Priority Optimizations")
|
||||
|
||||
high_priority = ai_data['high_priority']
|
||||
for item in high_priority:
|
||||
st.warning(f"⚡ {item}")
|
||||
|
||||
# Medium Priority
|
||||
if ai_data.get('medium_priority'):
|
||||
st.markdown("#### 📈 Medium Priority Improvements")
|
||||
|
||||
medium_priority = ai_data['medium_priority']
|
||||
for item in medium_priority:
|
||||
st.info(f"📊 {item}")
|
||||
|
||||
# Implementation Steps
|
||||
if ai_data.get('implementation_steps'):
|
||||
st.markdown("#### 🛠️ Implementation Steps")
|
||||
|
||||
steps = ai_data['implementation_steps']
|
||||
for i, step in enumerate(steps, 1):
|
||||
st.write(f"{i}. {step}")
|
||||
|
||||
# Expected Impact
|
||||
if ai_data.get('expected_impact'):
|
||||
st.markdown("#### 📈 Expected Impact Assessment")
|
||||
|
||||
impact = ai_data['expected_impact']
|
||||
st.markdown(impact)
|
||||
|
||||
def _render_export_options(self, results: Dict[str, Any]):
|
||||
"""Render export options for analysis results."""
|
||||
|
||||
st.markdown("---")
|
||||
st.markdown("### 📥 Export Technical SEO Audit")
|
||||
|
||||
col1, col2, col3 = st.columns(3)
|
||||
|
||||
with col1:
|
||||
# JSON export
|
||||
if st.button("📄 Export Full Report (JSON)", use_container_width=True):
|
||||
json_data = json.dumps(results, indent=2, default=str)
|
||||
|
||||
st.download_button(
|
||||
label="⬇️ Download JSON Report",
|
||||
data=json_data,
|
||||
file_name=f"technical_seo_audit_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json",
|
||||
mime="application/json",
|
||||
use_container_width=True
|
||||
)
|
||||
|
||||
with col2:
|
||||
# CSV export for issues
|
||||
if st.button("📊 Export Issues CSV", use_container_width=True):
|
||||
issues_data = self._prepare_issues_csv(results)
|
||||
|
||||
if issues_data:
|
||||
st.download_button(
|
||||
label="⬇️ Download Issues CSV",
|
||||
data=issues_data,
|
||||
file_name=f"technical_issues_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv",
|
||||
mime="text/csv",
|
||||
use_container_width=True
|
||||
)
|
||||
else:
|
||||
st.info("No issues found to export")
|
||||
|
||||
with col3:
|
||||
# Executive summary
|
||||
if st.button("📋 Executive Summary", use_container_width=True):
|
||||
summary = self._generate_executive_summary(results)
|
||||
|
||||
st.download_button(
|
||||
label="⬇️ Download Summary",
|
||||
data=summary,
|
||||
file_name=f"technical_seo_summary_{datetime.now().strftime('%Y%m%d_%H%M%S')}.txt",
|
||||
mime="text/plain",
|
||||
use_container_width=True
|
||||
)
|
||||
|
||||
def _prepare_issues_csv(self, results: Dict[str, Any]) -> str:
|
||||
"""Prepare CSV data for technical issues."""
|
||||
|
||||
issues_list = []
|
||||
|
||||
# HTTP errors
|
||||
http_errors = results.get('technical_issues', {}).get('http_errors', {})
|
||||
if http_errors.get('error_pages'):
|
||||
for error in http_errors['error_pages']:
|
||||
issues_list.append({
|
||||
'Issue Type': 'HTTP Error',
|
||||
'Severity': 'High',
|
||||
'URL': error.get('url', ''),
|
||||
'Status Code': error.get('status', ''),
|
||||
'Description': f"HTTP {error.get('status', '')} error"
|
||||
})
|
||||
|
||||
# Missing elements
|
||||
missing_elements = results.get('technical_issues', {}).get('missing_elements', {})
|
||||
|
||||
# Add more issue types as needed...
|
||||
|
||||
if issues_list:
|
||||
issues_df = pd.DataFrame(issues_list)
|
||||
return issues_df.to_csv(index=False)
|
||||
|
||||
return ""
|
||||
|
||||
def _generate_executive_summary(self, results: Dict[str, Any]) -> str:
|
||||
"""Generate executive summary report."""
|
||||
|
||||
website_url = results.get('website_url', 'Unknown')
|
||||
timestamp = results.get('analysis_timestamp', datetime.now().isoformat())
|
||||
|
||||
summary = f"""
|
||||
TECHNICAL SEO AUDIT - EXECUTIVE SUMMARY
|
||||
======================================
|
||||
|
||||
Website: {website_url}
|
||||
Audit Date: {timestamp}
|
||||
|
||||
AUDIT OVERVIEW
|
||||
--------------
|
||||
Pages Crawled: {results.get('crawl_overview', {}).get('pages_crawled', 0)}
|
||||
HTTP Errors: {results.get('technical_issues', {}).get('http_errors', {}).get('total_errors', 0)}
|
||||
Average Load Time: {results.get('performance_analysis', {}).get('load_time_analysis', {}).get('avg_load_time', 0):.2f}s
|
||||
Security Score: {results.get('security_headers', {}).get('security_score', 0):.0f}%
|
||||
|
||||
CRITICAL FINDINGS
|
||||
-----------------
|
||||
"""
|
||||
|
||||
# Add critical findings
|
||||
error_count = results.get('technical_issues', {}).get('http_errors', {}).get('total_errors', 0)
|
||||
if error_count > 0:
|
||||
summary += f"• {error_count} pages have HTTP errors requiring immediate attention\n"
|
||||
|
||||
avg_load_time = results.get('performance_analysis', {}).get('load_time_analysis', {}).get('avg_load_time', 0)
|
||||
if avg_load_time > 3:
|
||||
summary += f"• Page load times are slow (avg: {avg_load_time:.2f}s), impacting user experience\n"
|
||||
|
||||
security_score = results.get('security_headers', {}).get('security_score', 0)
|
||||
if security_score < 80:
|
||||
summary += f"• Security headers need improvement (current score: {security_score:.0f}%)\n"
|
||||
|
||||
summary += f"\n\nDetailed technical audit completed by ALwrity Technical SEO Crawler\nGenerated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}"
|
||||
|
||||
return summary
|
||||
|
||||
# Render function for integration with main dashboard
|
||||
def render_technical_seo_crawler():
|
||||
"""Render the Technical SEO Crawler UI."""
|
||||
ui = TechnicalSEOCrawlerUI()
|
||||
ui.render()
|
||||
58
ToBeMigrated/ai_seo_tools/textstaty.py
Normal file
58
ToBeMigrated/ai_seo_tools/textstaty.py
Normal file
@@ -0,0 +1,58 @@
|
||||
"""Text analysis tools using textstat."""
|
||||
|
||||
import streamlit as st
|
||||
from textstat import textstat
|
||||
|
||||
def analyze_text(text):
|
||||
"""Analyze text using textstat metrics."""
|
||||
if not text:
|
||||
st.warning("Please enter some text to analyze.")
|
||||
return
|
||||
|
||||
# Calculate various metrics
|
||||
metrics = {
|
||||
"Flesch Reading Ease": textstat.flesch_reading_ease(text),
|
||||
"Flesch-Kincaid Grade Level": textstat.flesch_kincaid_grade(text),
|
||||
"Gunning Fog Index": textstat.gunning_fog(text),
|
||||
"SMOG Index": textstat.smog_index(text),
|
||||
"Automated Readability Index": textstat.automated_readability_index(text),
|
||||
"Coleman-Liau Index": textstat.coleman_liau_index(text),
|
||||
"Linsear Write Formula": textstat.linsear_write_formula(text),
|
||||
"Dale-Chall Readability Score": textstat.dale_chall_readability_score(text),
|
||||
"Readability Consensus": textstat.readability_consensus(text)
|
||||
}
|
||||
|
||||
# Display metrics in a clean format
|
||||
st.subheader("Text Analysis Results")
|
||||
for metric, value in metrics.items():
|
||||
st.metric(metric, f"{value:.2f}")
|
||||
|
||||
# Add visualizations
|
||||
st.subheader("Visualization")
|
||||
st.bar_chart(metrics)
|
||||
|
||||
st.title("📖 Text Readability Analyzer: Making Your Content Easy to Read")
|
||||
|
||||
st.write("""
|
||||
This tool is your guide to writing content that's easy for your audience to understand.
|
||||
Just paste in a sample of your text, and we'll break down the readability scores and offer actionable tips!
|
||||
""")
|
||||
|
||||
text_input = st.text_area("Paste your text here:", height=200)
|
||||
|
||||
if st.button("Analyze!"):
|
||||
with st.spinner("Analyzing your text..."):
|
||||
test_data = text_input
|
||||
if not test_data.strip():
|
||||
st.error("Please enter text to analyze.")
|
||||
else:
|
||||
analyze_text(test_data)
|
||||
|
||||
st.subheader("Key Takeaways:")
|
||||
st.write("---")
|
||||
st.markdown("""
|
||||
* **Don't Be Afraid to Simplify!** Often, simpler language makes content more impactful and easier to digest.
|
||||
* **Aim for a Reading Level Appropriate for Your Audience:** Consider the education level, background, and familiarity of your readers.
|
||||
* **Use Short Sentences:** This makes your content more scannable and easier to read.
|
||||
* **Write for Everyone:** Accessibility should always be a priority. When in doubt, aim for clear, concise language!
|
||||
""")
|
||||
Reference in New Issue
Block a user