ALwrity Version 0.5.1 (Fastapi + React)
This commit is contained in:
192
ToBeMigrated/ai_writers/ai_blog_faqs_writer/README.md
Normal file
192
ToBeMigrated/ai_writers/ai_blog_faqs_writer/README.md
Normal file
@@ -0,0 +1,192 @@
|
||||
# AI-Powered FAQ Generator
|
||||
|
||||
A sophisticated FAQ generation system that creates comprehensive, well-researched FAQs from various content sources. This tool leverages AI to analyze content, conduct web research, and generate detailed FAQs with customizable options.
|
||||
|
||||
## Features
|
||||
|
||||
### Content Processing
|
||||
- **Multiple Input Sources**
|
||||
- Direct text input
|
||||
- File uploads (DOCX, TXT)
|
||||
- URL content extraction
|
||||
- Support for any content type (general, technical, educational, etc.)
|
||||
|
||||
### Research Capabilities
|
||||
- **Multi-level Search Depth**
|
||||
- **Basic**: Google Search for quick, general information
|
||||
- **Comprehensive**: Tavily AI for detailed, in-depth research
|
||||
- **Expert**: Metaphor AI for specialized, expert-level content
|
||||
|
||||
### Customization Options
|
||||
- **Target Audience**
|
||||
- Beginner
|
||||
- Intermediate
|
||||
- Expert
|
||||
|
||||
- **FAQ Style**
|
||||
- Technical
|
||||
- Conversational
|
||||
- Professional
|
||||
|
||||
- **Advanced Features**
|
||||
- Emoji inclusion
|
||||
- Code example generation
|
||||
- Reference integration
|
||||
- Customizable time range for research
|
||||
- Multi-language support
|
||||
|
||||
### Output Formats
|
||||
- Interactive preview
|
||||
- Markdown
|
||||
- HTML
|
||||
- JSON
|
||||
|
||||
## Installation
|
||||
|
||||
1. Clone the repository
|
||||
2. Install dependencies:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Usage
|
||||
```python
|
||||
from lib.ai_writers.ai_blog_faqs_writer.faqs_generator_blog import FAQGenerator, FAQConfig
|
||||
|
||||
# Initialize with default configuration
|
||||
generator = FAQGenerator()
|
||||
|
||||
# Generate FAQs from content
|
||||
faqs = await generator.generate_faqs("Your content here")
|
||||
```
|
||||
|
||||
### Advanced Configuration
|
||||
```python
|
||||
from lib.ai_writers.ai_blog_faqs_writer.faqs_generator_blog import (
|
||||
FAQGenerator, FAQConfig, TargetAudience, FAQStyle, SearchDepth
|
||||
)
|
||||
|
||||
# Custom configuration
|
||||
config = FAQConfig(
|
||||
num_faqs=10,
|
||||
target_audience=TargetAudience.INTERMEDIATE,
|
||||
faq_style=FAQStyle.TECHNICAL,
|
||||
include_emojis=True,
|
||||
include_code_examples=True,
|
||||
include_references=True,
|
||||
search_depth=SearchDepth.COMPREHENSIVE,
|
||||
time_range="last_6_months",
|
||||
language="English"
|
||||
)
|
||||
|
||||
generator = FAQGenerator(config)
|
||||
```
|
||||
|
||||
### Web Interface
|
||||
Run the Streamlit interface:
|
||||
```bash
|
||||
streamlit run lib/ai_writers/ai_blog_faqs_writer/faqs_ui.py
|
||||
```
|
||||
|
||||
## Research Process
|
||||
|
||||
1. **Content Analysis**
|
||||
- Identifies key topics and concepts
|
||||
- Extracts potential questions
|
||||
- Determines research requirements
|
||||
|
||||
2. **Web Research**
|
||||
- Selects appropriate search function based on depth
|
||||
- Gathers relevant information
|
||||
- Validates and cross-references data
|
||||
|
||||
3. **FAQ Generation**
|
||||
- Creates comprehensive questions
|
||||
- Provides detailed answers
|
||||
- Includes code examples (if applicable)
|
||||
- Adds references and citations
|
||||
|
||||
## Output Structure
|
||||
|
||||
Each FAQ item includes:
|
||||
- Question
|
||||
- Detailed answer
|
||||
- Category
|
||||
- Code example (if applicable)
|
||||
- References
|
||||
- Confidence score
|
||||
- Last updated timestamp
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### FAQConfig Parameters
|
||||
- `num_faqs`: Number of FAQs to generate (default: 5)
|
||||
- `target_audience`: Target audience level (default: INTERMEDIATE)
|
||||
- `faq_style`: Writing style (default: PROFESSIONAL)
|
||||
- `include_emojis`: Whether to include emojis (default: True)
|
||||
- `include_code_examples`: Whether to include code examples (default: True)
|
||||
- `include_references`: Whether to include references (default: True)
|
||||
- `search_depth`: Research depth level (default: COMPREHENSIVE)
|
||||
- `time_range`: Time range for research (default: "last_6_months")
|
||||
- `language`: Output language (default: "English")
|
||||
|
||||
## Research Depth Options
|
||||
|
||||
### Basic (Google Search)
|
||||
- Quick, general information
|
||||
- Broad coverage
|
||||
- Suitable for basic topics
|
||||
|
||||
### Comprehensive (Tavily AI)
|
||||
- Detailed, in-depth research
|
||||
- Multiple source integration
|
||||
- Best for most use cases
|
||||
|
||||
### Expert (Metaphor AI)
|
||||
- Specialized, expert-level content
|
||||
- Advanced topic coverage
|
||||
- Technical and academic focus
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Content Preparation**
|
||||
- Provide clear, well-structured content
|
||||
- Include key terms and concepts
|
||||
- Specify target audience and style
|
||||
|
||||
2. **Research Selection**
|
||||
- Use Basic for general topics
|
||||
- Choose Comprehensive for detailed analysis
|
||||
- Select Expert for technical subjects
|
||||
|
||||
3. **Output Review**
|
||||
- Verify accuracy of information
|
||||
- Check code examples
|
||||
- Validate references
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch
|
||||
3. Commit your changes
|
||||
4. Push to the branch
|
||||
5. Create a Pull Request
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License - see the LICENSE file for details.
|
||||
|
||||
## Support
|
||||
|
||||
For support, please open an issue in the repository or contact the maintainers.
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
- OpenAI for GPT integration
|
||||
- Google Search API
|
||||
- Tavily AI
|
||||
- Metaphor AI
|
||||
- BeautifulSoup for web scraping
|
||||
- Streamlit for UI
|
||||
@@ -0,0 +1,444 @@
|
||||
"""
|
||||
Enhanced FAQ Generator
|
||||
|
||||
This module provides a comprehensive FAQ generation system that can create detailed,
|
||||
well-researched FAQs from various content sources with customizable options.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import re
|
||||
from typing import Dict, List, Optional, Union
|
||||
from pathlib import Path
|
||||
from enum import Enum
|
||||
from dataclasses import dataclass
|
||||
from loguru import logger
|
||||
|
||||
from lib.gpt_providers.text_generation.main_text_generation import llm_text_gen
|
||||
from lib.ai_web_researcher.google_serp_search import google_search
|
||||
from lib.ai_web_researcher.tavily_ai_search import do_tavily_ai_search
|
||||
from lib.ai_web_researcher.metaphor_basic_neural_web_search import metaphor_search_articles
|
||||
|
||||
logger.remove()
|
||||
logger.add(sys.stdout,
|
||||
colorize=True,
|
||||
format="<level>{level}</level>|<green>{file}:{line}:{function}</green>| {message}")
|
||||
|
||||
class TargetAudience(Enum):
|
||||
BEGINNER = "beginner"
|
||||
INTERMEDIATE = "intermediate"
|
||||
EXPERT = "expert"
|
||||
|
||||
class FAQStyle(Enum):
|
||||
TECHNICAL = "technical"
|
||||
CONVERSATIONAL = "conversational"
|
||||
PROFESSIONAL = "professional"
|
||||
|
||||
class SearchDepth(Enum):
|
||||
BASIC = "basic"
|
||||
COMPREHENSIVE = "comprehensive"
|
||||
EXPERT = "expert"
|
||||
|
||||
@dataclass
|
||||
class FAQConfig:
|
||||
"""Configuration for FAQ generation."""
|
||||
num_faqs: int = 5
|
||||
target_audience: TargetAudience = TargetAudience.INTERMEDIATE
|
||||
faq_style: FAQStyle = FAQStyle.PROFESSIONAL
|
||||
include_emojis: bool = True
|
||||
include_code_examples: bool = True
|
||||
include_references: bool = True
|
||||
search_depth: SearchDepth = SearchDepth.COMPREHENSIVE
|
||||
time_range: str = "last_6_months"
|
||||
exclude_domains: List[str] = None
|
||||
language: str = "English"
|
||||
selected_search_queries: List[str] = None
|
||||
|
||||
@dataclass
|
||||
class FAQItem:
|
||||
"""Individual FAQ item with metadata."""
|
||||
question: str
|
||||
answer: str
|
||||
category: str
|
||||
code_example: Optional[str] = None
|
||||
references: List[Dict[str, str]] = None
|
||||
confidence_score: float = 0.0
|
||||
last_updated: str = None
|
||||
|
||||
class FAQGenerator:
|
||||
"""Enhanced FAQ Generator with research capabilities."""
|
||||
|
||||
def __init__(self, config: Optional[FAQConfig] = None):
|
||||
"""Initialize the FAQ generator with optional configuration."""
|
||||
self.config = config or FAQConfig()
|
||||
self.faqs: List[FAQItem] = []
|
||||
self.research_results = {}
|
||||
self.search_queries = []
|
||||
|
||||
def generate_search_queries(self, content: str) -> List[str]:
|
||||
"""Generate search queries based on the content."""
|
||||
try:
|
||||
prompt = f"""Based on the following content, generate 5 specific search queries that would help create comprehensive FAQs.
|
||||
Content: {content}
|
||||
|
||||
Guidelines for search queries:
|
||||
1. Focus on key concepts and terms
|
||||
2. Include common questions users might have
|
||||
3. Cover technical aspects that need clarification
|
||||
4. Include best practices and recommendations
|
||||
5. Make queries specific and focused
|
||||
|
||||
Please provide exactly 5 search queries, one per line.
|
||||
Do not include numbers or bullet points in the queries.
|
||||
"""
|
||||
|
||||
response = llm_text_gen(prompt)
|
||||
# Clean up the queries by removing numbers and extra spaces
|
||||
queries = []
|
||||
for line in response.split('\n'):
|
||||
# Remove any leading numbers, dots, or spaces
|
||||
cleaned = re.sub(r'^\d+\.\s*', '', line.strip())
|
||||
if cleaned:
|
||||
queries.append(cleaned)
|
||||
|
||||
self.search_queries = queries[:5] # Ensure we only get 5 queries
|
||||
return self.search_queries
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to generate search queries: {err}")
|
||||
return []
|
||||
|
||||
def _clean_search_query(self, query: str) -> str:
|
||||
"""Clean up a search query by removing numbers and extra formatting."""
|
||||
# Remove any leading numbers, dots, or spaces
|
||||
cleaned = re.sub(r'^\d+\.\s*', '', query.strip())
|
||||
# Remove any quotes
|
||||
cleaned = cleaned.replace('"', '').replace("'", '')
|
||||
# Remove any extra spaces
|
||||
cleaned = ' '.join(cleaned.split())
|
||||
return cleaned
|
||||
|
||||
def generate_faqs(self, content: str, content_type: str = "general") -> List[FAQItem]:
|
||||
"""Generate FAQs from the given content with research integration."""
|
||||
try:
|
||||
if not self.config.selected_search_queries:
|
||||
raise ValueError("No search queries selected. Please select queries to proceed.")
|
||||
|
||||
# Clean up selected queries
|
||||
cleaned_queries = [self._clean_search_query(q) for q in self.config.selected_search_queries]
|
||||
self.config.selected_search_queries = cleaned_queries
|
||||
|
||||
# Step 1: Research the topic using selected queries
|
||||
research_results = self._conduct_research(content)
|
||||
|
||||
# Step 2: Generate initial FAQs
|
||||
initial_faqs = self._generate_initial_faqs(content, research_results)
|
||||
|
||||
# Step 3: Enhance FAQs with research
|
||||
enhanced_faqs = self._enhance_faqs_with_research(initial_faqs, research_results)
|
||||
|
||||
# Step 4: Add code examples if requested
|
||||
if self.config.include_code_examples:
|
||||
enhanced_faqs = self._add_code_examples(enhanced_faqs)
|
||||
|
||||
# Step 5: Add references if requested
|
||||
if self.config.include_references:
|
||||
enhanced_faqs = self._add_references(enhanced_faqs, research_results)
|
||||
|
||||
self.faqs = enhanced_faqs
|
||||
return enhanced_faqs
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to generate FAQs: {err}")
|
||||
raise
|
||||
|
||||
def _conduct_research(self, content: str) -> Dict:
|
||||
"""Conduct online research based on the selected search queries."""
|
||||
try:
|
||||
research_results = {}
|
||||
|
||||
for query in self.config.selected_search_queries:
|
||||
try:
|
||||
# Clean the query before searching
|
||||
cleaned_query = self._clean_search_query(query)
|
||||
logger.info(f"Researching query: {cleaned_query}")
|
||||
|
||||
# Select search function based on search depth
|
||||
if self.config.search_depth == SearchDepth.BASIC:
|
||||
results = google_search(cleaned_query)
|
||||
elif self.config.search_depth == SearchDepth.COMPREHENSIVE:
|
||||
results = do_tavily_ai_search(cleaned_query)
|
||||
elif self.config.search_depth == SearchDepth.EXPERT:
|
||||
results = metaphor_search_articles(cleaned_query)
|
||||
else:
|
||||
logger.warning(f"Unknown search depth: {self.config.search_depth}, defaulting to Google search")
|
||||
results = google_search(cleaned_query)
|
||||
|
||||
research_results[query] = results
|
||||
logger.info(f"Research completed for query: {query}")
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to research query '{query}': {err}")
|
||||
continue
|
||||
|
||||
return research_results
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to conduct research: {err}")
|
||||
return {}
|
||||
|
||||
def _generate_initial_faqs(self, content: str, research_results: Dict) -> List[FAQItem]:
|
||||
"""Generate initial FAQs using LLM."""
|
||||
try:
|
||||
system_prompt = f"""You are an expert FAQ generator with deep knowledge in content creation and technical writing.
|
||||
Your task is to create comprehensive FAQs based on the given content and research.
|
||||
|
||||
Guidelines:
|
||||
1. Target Audience: {self.config.target_audience.value}
|
||||
2. Style: {self.config.faq_style.value}
|
||||
3. Include emojis: {self.config.include_emojis}
|
||||
4. Language: {self.config.language}
|
||||
5. Number of FAQs: {self.config.num_faqs}
|
||||
|
||||
Create FAQs that are:
|
||||
- Clear and concise
|
||||
- Well-structured
|
||||
- Technically accurate
|
||||
- Engaging and informative
|
||||
- Based on the provided research
|
||||
- Relevant to the target audience
|
||||
- Written in the specified style
|
||||
|
||||
Format each FAQ exactly as follows:
|
||||
Q: [Your question here]
|
||||
A: [Your detailed answer here]
|
||||
Category: [Category name]
|
||||
Confidence: [Score between 0 and 1]
|
||||
---
|
||||
"""
|
||||
|
||||
prompt = f"""Content to generate FAQs from:
|
||||
{content}
|
||||
|
||||
Research Results:
|
||||
{json.dumps(research_results, indent=2)}
|
||||
|
||||
Please generate {self.config.num_faqs} FAQs following the guidelines above.
|
||||
Each FAQ must be separated by '---' and include all required fields.
|
||||
"""
|
||||
|
||||
response = llm_text_gen(prompt, system_prompt=system_prompt)
|
||||
logger.info(f"LLM Response: {response}")
|
||||
|
||||
# Parse the response into FAQItem objects
|
||||
faqs = []
|
||||
current_faq = None
|
||||
|
||||
for line in response.split('\n'):
|
||||
line = line.strip()
|
||||
if not line or line == '---':
|
||||
if current_faq and current_faq.question and current_faq.answer:
|
||||
faqs.append(current_faq)
|
||||
current_faq = None
|
||||
continue
|
||||
|
||||
if line.startswith('Q:'):
|
||||
if current_faq and current_faq.question and current_faq.answer:
|
||||
faqs.append(current_faq)
|
||||
current_faq = FAQItem(question=line[2:].strip(), answer="", category="")
|
||||
elif line.startswith('A:'):
|
||||
if current_faq:
|
||||
current_faq.answer = line[2:].strip()
|
||||
elif line.startswith('Category:'):
|
||||
if current_faq:
|
||||
current_faq.category = line[9:].strip()
|
||||
elif line.startswith('Confidence:'):
|
||||
if current_faq:
|
||||
try:
|
||||
current_faq.confidence_score = float(line[11:].strip())
|
||||
except ValueError:
|
||||
current_faq.confidence_score = 0.5
|
||||
|
||||
# Add the last FAQ if it exists and is complete
|
||||
if current_faq and current_faq.question and current_faq.answer:
|
||||
faqs.append(current_faq)
|
||||
|
||||
logger.info(f"Generated {len(faqs)} FAQs")
|
||||
return faqs
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to generate initial FAQs: {err}")
|
||||
raise
|
||||
|
||||
def _enhance_faqs_with_research(self, faqs: List[FAQItem], research_results: Dict) -> List[FAQItem]:
|
||||
"""Enhance FAQs with research findings."""
|
||||
try:
|
||||
enhanced_faqs = []
|
||||
|
||||
for faq in faqs:
|
||||
# Find relevant research for this FAQ
|
||||
relevant_research = self._find_relevant_research(faq, research_results)
|
||||
|
||||
if relevant_research:
|
||||
# Enhance the answer with research findings
|
||||
enhancement_prompt = f"""Enhance the following FAQ answer with the provided research:
|
||||
|
||||
Question: {faq.question}
|
||||
Current Answer: {faq.answer}
|
||||
|
||||
Research:
|
||||
{json.dumps(relevant_research, indent=2)}
|
||||
|
||||
Please enhance the answer while:
|
||||
1. Maintaining the original style and tone
|
||||
2. Adding relevant information from the research
|
||||
3. Ensuring technical accuracy
|
||||
4. Keeping the answer concise and clear
|
||||
"""
|
||||
|
||||
enhanced_answer = llm_text_gen(enhancement_prompt)
|
||||
faq.answer = enhanced_answer
|
||||
|
||||
enhanced_faqs.append(faq)
|
||||
|
||||
return enhanced_faqs
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to enhance FAQs with research: {err}")
|
||||
return faqs
|
||||
|
||||
def _add_code_examples(self, faqs: List[FAQItem]) -> List[FAQItem]:
|
||||
"""Add code examples to FAQs where applicable."""
|
||||
try:
|
||||
for faq in faqs:
|
||||
if self._is_technical_question(faq.question):
|
||||
code_prompt = f"""Generate a code example for the following FAQ:
|
||||
Question: {faq.question}
|
||||
Answer: {faq.answer}
|
||||
|
||||
Please provide a relevant code example that demonstrates the concept.
|
||||
Include comments and explanations where necessary.
|
||||
"""
|
||||
|
||||
code_example = llm_text_gen(code_prompt)
|
||||
faq.code_example = code_example
|
||||
|
||||
return faqs
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to add code examples: {err}")
|
||||
return faqs
|
||||
|
||||
def _add_references(self, faqs: List[FAQItem], research_results: Dict) -> List[FAQItem]:
|
||||
"""Add references to FAQs based on research results."""
|
||||
try:
|
||||
for faq in faqs:
|
||||
relevant_research = self._find_relevant_research(faq, research_results)
|
||||
if relevant_research:
|
||||
references = []
|
||||
for source, content in relevant_research.items():
|
||||
references.append({
|
||||
"source": source,
|
||||
"content": content
|
||||
})
|
||||
faq.references = references
|
||||
|
||||
return faqs
|
||||
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to add references: {err}")
|
||||
return faqs
|
||||
|
||||
def _find_relevant_research(self, faq: FAQItem, research_results: Dict) -> Dict:
|
||||
"""Find research results relevant to a specific FAQ."""
|
||||
relevant_research = {}
|
||||
for topic, results in research_results.items():
|
||||
if any(keyword in faq.question.lower() for keyword in topic.lower().split()):
|
||||
relevant_research[topic] = results
|
||||
return relevant_research
|
||||
|
||||
def _is_technical_question(self, question: str) -> bool:
|
||||
"""Determine if a question is technical and might benefit from a code example."""
|
||||
technical_keywords = ["code", "program", "function", "method", "class", "api", "syntax", "error", "debug"]
|
||||
return any(keyword in question.lower() for keyword in technical_keywords)
|
||||
|
||||
def to_markdown(self) -> str:
|
||||
"""Convert FAQs to markdown format."""
|
||||
markdown = "# Frequently Asked Questions\n\n"
|
||||
|
||||
for faq in self.faqs:
|
||||
markdown += f"## {faq.question}\n\n"
|
||||
markdown += f"{faq.answer}\n\n"
|
||||
|
||||
if faq.code_example:
|
||||
markdown += "```\n"
|
||||
markdown += f"{faq.code_example}\n"
|
||||
markdown += "```\n\n"
|
||||
|
||||
if faq.references:
|
||||
markdown += "### References\n"
|
||||
for ref in faq.references:
|
||||
markdown += f"- {ref['source']}\n"
|
||||
markdown += "\n"
|
||||
|
||||
return markdown
|
||||
|
||||
def to_html(self) -> str:
|
||||
"""Convert FAQs to HTML format."""
|
||||
html = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Frequently Asked Questions</title>
|
||||
<style>
|
||||
body { font-family: Arial, sans-serif; max-width: 800px; margin: 0 auto; padding: 20px; }
|
||||
.faq { margin-bottom: 30px; }
|
||||
.question { font-weight: bold; font-size: 1.2em; color: #2c3e50; }
|
||||
.answer { margin: 10px 0; }
|
||||
.code-example { background: #f8f9fa; padding: 15px; border-radius: 4px; }
|
||||
.references { margin-top: 15px; font-size: 0.9em; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Frequently Asked Questions</h1>
|
||||
"""
|
||||
|
||||
for faq in self.faqs:
|
||||
html += f"""
|
||||
<div class="faq">
|
||||
<div class="question">{faq.question}</div>
|
||||
<div class="answer">{faq.answer}</div>
|
||||
"""
|
||||
|
||||
if faq.code_example:
|
||||
html += f"""
|
||||
<div class="code-example">
|
||||
<pre><code>{faq.code_example}</code></pre>
|
||||
</div>
|
||||
"""
|
||||
|
||||
if faq.references:
|
||||
html += """
|
||||
<div class="references">
|
||||
<h3>References</h3>
|
||||
<ul>
|
||||
"""
|
||||
for ref in faq.references:
|
||||
html += f"""
|
||||
<li>{ref['source']}</li>
|
||||
"""
|
||||
html += """
|
||||
</ul>
|
||||
</div>
|
||||
"""
|
||||
|
||||
html += """
|
||||
</div>
|
||||
"""
|
||||
|
||||
html += """
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
return html
|
||||
312
ToBeMigrated/ai_writers/ai_blog_faqs_writer/faqs_ui.py
Normal file
312
ToBeMigrated/ai_writers/ai_blog_faqs_writer/faqs_ui.py
Normal file
@@ -0,0 +1,312 @@
|
||||
"""
|
||||
Streamlit UI for FAQ Generator
|
||||
|
||||
This module provides a user-friendly interface for generating FAQs from various content sources.
|
||||
"""
|
||||
|
||||
import streamlit as st
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
import json
|
||||
import requests
|
||||
from bs4 import BeautifulSoup
|
||||
import logging
|
||||
import pyperclip
|
||||
|
||||
from .faqs_generator_blog import FAQGenerator, FAQConfig, TargetAudience, FAQStyle, SearchDepth
|
||||
|
||||
# Set up logging
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def copy_to_clipboard(text: str) -> None:
|
||||
"""Copy text to clipboard and show success message."""
|
||||
try:
|
||||
pyperclip.copy(text)
|
||||
st.success("Copied to clipboard!")
|
||||
except Exception as e:
|
||||
st.error(f"Failed to copy to clipboard: {str(e)}")
|
||||
|
||||
def fetch_url_content(url):
|
||||
"""Fetch and extract content from a URL."""
|
||||
try:
|
||||
response = requests.get(url)
|
||||
response.raise_for_status()
|
||||
soup = BeautifulSoup(response.text, 'html.parser')
|
||||
|
||||
# Remove script and style elements
|
||||
for script in soup(["script", "style"]):
|
||||
script.decompose()
|
||||
|
||||
# Get text
|
||||
text = soup.get_text()
|
||||
|
||||
# Break into lines and remove leading and trailing space
|
||||
lines = (line.strip() for line in text.splitlines())
|
||||
# Break multi-headlines into a line each
|
||||
chunks = (phrase.strip() for line in lines for phrase in line.split(" "))
|
||||
# Drop blank lines
|
||||
text = '\n'.join(chunk for chunk in chunks if chunk)
|
||||
|
||||
return text
|
||||
except Exception as e:
|
||||
st.error(f"Error fetching URL content: {str(e)}")
|
||||
return None
|
||||
|
||||
def main():
|
||||
st.title("FAQ Generator")
|
||||
st.markdown("Generate comprehensive FAQs from your content with research integration.")
|
||||
|
||||
# Initialize session state variables if they don't exist
|
||||
if 'search_queries' not in st.session_state:
|
||||
st.session_state.search_queries = []
|
||||
if 'selected_queries' not in st.session_state:
|
||||
st.session_state.selected_queries = []
|
||||
if 'research_completed' not in st.session_state:
|
||||
st.session_state.research_completed = False
|
||||
if 'research_results' not in st.session_state:
|
||||
st.session_state.research_results = {}
|
||||
if 'faq_config' not in st.session_state:
|
||||
st.session_state.faq_config = None
|
||||
if 'generator' not in st.session_state:
|
||||
st.session_state.generator = FAQGenerator()
|
||||
if 'generated_faqs' not in st.session_state:
|
||||
st.session_state.generated_faqs = None
|
||||
if 'output_format' not in st.session_state:
|
||||
st.session_state.output_format = "Preview"
|
||||
|
||||
# Sidebar for configuration
|
||||
with st.sidebar:
|
||||
st.header("Configuration")
|
||||
|
||||
# Basic settings
|
||||
num_faqs = st.slider("Number of FAQs", 1, 20, 5)
|
||||
target_audience = st.selectbox(
|
||||
"Target Audience",
|
||||
[audience.value for audience in TargetAudience]
|
||||
)
|
||||
faq_style = st.selectbox(
|
||||
"FAQ Style",
|
||||
[style.value for style in FAQStyle]
|
||||
)
|
||||
|
||||
# Advanced settings
|
||||
with st.expander("Advanced Settings"):
|
||||
include_emojis = st.checkbox("Include Emojis", value=True)
|
||||
include_code_examples = st.checkbox("Include Code Examples", value=True)
|
||||
include_references = st.checkbox("Include References", value=True)
|
||||
|
||||
search_depth = st.selectbox(
|
||||
"Search Depth",
|
||||
[depth.value for depth in SearchDepth]
|
||||
)
|
||||
time_range = st.selectbox(
|
||||
"Time Range",
|
||||
["last_month", "last_6_months", "last_year", "all_time"]
|
||||
)
|
||||
language = st.text_input("Language", value="English")
|
||||
|
||||
# Main content area
|
||||
content_type = st.radio(
|
||||
"Content Source",
|
||||
["Direct Input", "File Upload", "URL"]
|
||||
)
|
||||
|
||||
content = ""
|
||||
if content_type == "Direct Input":
|
||||
content = st.text_area("Enter your content", height=300)
|
||||
|
||||
elif content_type == "URL":
|
||||
url = st.text_input("Enter URL")
|
||||
if url:
|
||||
content = fetch_url_content(url)
|
||||
if content:
|
||||
st.text_area("Extracted Content", content, height=300)
|
||||
|
||||
# Step 1: Generate search queries
|
||||
if content and not st.session_state.search_queries:
|
||||
if st.button("Generate Search Queries"):
|
||||
with st.spinner("Generating search queries..."):
|
||||
search_queries = st.session_state.generator.generate_search_queries(content)
|
||||
if search_queries:
|
||||
st.session_state.search_queries = search_queries
|
||||
st.session_state.selected_queries = [] # Reset selected queries
|
||||
st.session_state.research_completed = False # Reset research status
|
||||
st.session_state.research_results = {} # Reset research results
|
||||
st.session_state.faq_config = None # Reset config
|
||||
st.session_state.generated_faqs = None # Reset generated FAQs
|
||||
st.success("Search queries generated successfully!")
|
||||
|
||||
# Step 2: Display and select search queries
|
||||
if st.session_state.search_queries:
|
||||
st.subheader("Select Search Queries")
|
||||
st.info("Select the queries you want to use for web research. You can select multiple queries.")
|
||||
|
||||
# Create checkboxes for each search query
|
||||
selected_queries = []
|
||||
for query in st.session_state.search_queries:
|
||||
if st.checkbox(query, key=f"query_{query}", value=query in st.session_state.selected_queries):
|
||||
selected_queries.append(query)
|
||||
|
||||
# Update selected queries in session state
|
||||
st.session_state.selected_queries = selected_queries
|
||||
|
||||
# Step 3: Do web research
|
||||
if st.session_state.selected_queries and not st.session_state.research_completed:
|
||||
if st.button("Do Web Research"):
|
||||
try:
|
||||
# Create config with selected queries
|
||||
config = FAQConfig(
|
||||
num_faqs=num_faqs,
|
||||
target_audience=TargetAudience(target_audience),
|
||||
faq_style=FAQStyle(faq_style),
|
||||
include_emojis=include_emojis,
|
||||
include_code_examples=include_code_examples,
|
||||
include_references=include_references,
|
||||
search_depth=SearchDepth(search_depth),
|
||||
time_range=time_range,
|
||||
language=language,
|
||||
selected_search_queries=selected_queries
|
||||
)
|
||||
|
||||
# Store config in session state
|
||||
st.session_state.faq_config = config
|
||||
|
||||
# Update generator with config
|
||||
st.session_state.generator.config = config
|
||||
|
||||
# Do research
|
||||
with st.spinner("Conducting web research..."):
|
||||
research_results = st.session_state.generator._conduct_research(content)
|
||||
st.session_state.research_completed = True
|
||||
st.session_state.research_results = research_results
|
||||
st.success("Web research completed successfully!")
|
||||
|
||||
# Display research results
|
||||
st.subheader("Research Results")
|
||||
for query, results in research_results.items():
|
||||
with st.expander(f"Results for: {query}"):
|
||||
if isinstance(results, dict):
|
||||
st.json(results)
|
||||
else:
|
||||
st.text(results)
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error during web research: {str(e)}")
|
||||
st.error("Please try again with different search queries or adjust the search depth.")
|
||||
|
||||
# Step 4: Generate FAQs
|
||||
if st.session_state.research_completed and st.session_state.research_results and st.session_state.faq_config:
|
||||
if st.button("Generate FAQs"):
|
||||
try:
|
||||
# Update generator with stored config
|
||||
st.session_state.generator.config = st.session_state.faq_config
|
||||
|
||||
# Generate FAQs
|
||||
with st.spinner("Generating FAQs..."):
|
||||
logger.info("Starting FAQ generation...")
|
||||
faqs = st.session_state.generator.generate_faqs(content)
|
||||
logger.info(f"Generated {len(faqs) if faqs else 0} FAQs")
|
||||
|
||||
if not faqs:
|
||||
st.error("No FAQs were generated. Please try again.")
|
||||
return
|
||||
|
||||
st.session_state.generated_faqs = faqs
|
||||
st.success("FAQs generated successfully!")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error generating FAQs: {str(e)}")
|
||||
st.error(f"Error generating FAQs: {str(e)}")
|
||||
st.error("Please try again or adjust your settings.")
|
||||
|
||||
# Display generated FAQs if they exist
|
||||
if st.session_state.generated_faqs:
|
||||
st.subheader("Generated FAQs")
|
||||
|
||||
# Output format selection
|
||||
output_format = st.radio(
|
||||
"Output Format",
|
||||
["Preview", "Markdown", "HTML", "JSON"],
|
||||
key="output_format"
|
||||
)
|
||||
|
||||
# Create columns for copy and download buttons
|
||||
col1, col2 = st.columns(2)
|
||||
|
||||
if output_format == "Preview":
|
||||
# Create a formatted text for copying
|
||||
preview_text = ""
|
||||
for i, faq in enumerate(st.session_state.generated_faqs, 1):
|
||||
preview_text += f"{i}. {faq.question}\n"
|
||||
preview_text += f"{faq.answer}\n\n"
|
||||
if faq.code_example:
|
||||
preview_text += f"Code Example:\n{faq.code_example}\n\n"
|
||||
if faq.references:
|
||||
preview_text += "References:\n"
|
||||
for ref in faq.references:
|
||||
preview_text += f"- {ref['source']}\n"
|
||||
preview_text += "\n"
|
||||
|
||||
with col1:
|
||||
if st.button("Copy to Clipboard", key="copy_preview"):
|
||||
copy_to_clipboard(preview_text)
|
||||
|
||||
# Display the FAQs
|
||||
for i, faq in enumerate(st.session_state.generated_faqs, 1):
|
||||
with st.expander(f"{i}. {faq.question}"):
|
||||
st.markdown(faq.answer)
|
||||
if faq.code_example:
|
||||
st.code(faq.code_example)
|
||||
if faq.references:
|
||||
st.markdown("**References:**")
|
||||
for ref in faq.references:
|
||||
st.markdown(f"- {ref['source']}")
|
||||
|
||||
elif output_format == "Markdown":
|
||||
markdown_output = st.session_state.generator.to_markdown()
|
||||
st.code(markdown_output, language="markdown")
|
||||
|
||||
with col1:
|
||||
if st.button("Copy to Clipboard", key="copy_markdown"):
|
||||
copy_to_clipboard(markdown_output)
|
||||
with col2:
|
||||
st.download_button(
|
||||
"Download Markdown",
|
||||
markdown_output,
|
||||
file_name="faqs.md",
|
||||
mime="text/markdown"
|
||||
)
|
||||
|
||||
elif output_format == "HTML":
|
||||
html_output = st.session_state.generator.to_html()
|
||||
st.code(html_output, language="html")
|
||||
|
||||
with col1:
|
||||
if st.button("Copy to Clipboard", key="copy_html"):
|
||||
copy_to_clipboard(html_output)
|
||||
with col2:
|
||||
st.download_button(
|
||||
"Download HTML",
|
||||
html_output,
|
||||
file_name="faqs.html",
|
||||
mime="text/html"
|
||||
)
|
||||
|
||||
elif output_format == "JSON":
|
||||
json_output = json.dumps([faq.__dict__ for faq in st.session_state.generated_faqs], indent=2)
|
||||
st.code(json_output, language="json")
|
||||
|
||||
with col1:
|
||||
if st.button("Copy to Clipboard", key="copy_json"):
|
||||
copy_to_clipboard(json_output)
|
||||
with col2:
|
||||
st.download_button(
|
||||
"Download JSON",
|
||||
json_output,
|
||||
file_name="faqs.json",
|
||||
mime="application/json"
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user