From bb2904039e64c28e395b792455e073fbd63bd03e Mon Sep 17 00:00:00 2001 From: ajaysi Date: Mon, 29 Apr 2024 23:48:36 +0530 Subject: [PATCH] Agentic content team, web researched --- README.md | 110 +++++--------------- alwrity.py | 67 +++++++----- lib/ai_web_researcher/google_serp_search.py | 4 +- lib/ai_writers/blog_from_google_serp.py | 18 ++-- lib/ai_writers/combine_blog_and_keywords.py | 5 + lib/ai_writers/combine_research_and_blog.py | 14 ++- lib/ai_writers/keywords_to_blog.py | 8 +- 7 files changed, 89 insertions(+), 137 deletions(-) diff --git a/README.md b/README.md index 4a7ac811..e7f7dd0c 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,20 @@ # How to Use AI Content Generation Toolkit - Alwrity -1). Visit alwrity.com, Under tools section, you will find some AI content writing tools, which are Free & No-Signup. +1). [Visit alwrity.com](https://www.alwrity.com/ai-writing-tools), You will find AI content writing tools, which are Free & No-Signup. **Note:** Although, this is limited, as is our wallet & Resources. -2). For complete AI content toolkit, alwrity offers a commandline App. Its a BYOK model(Bring Your Own Key). +2). **For complete AI content creation toolkit**, alwrity offers a commandline App. Its a BYOK model(Bring Your Own Key). **Note:** (๐Ÿ—ฏ๏ธ commandline, byok and shit) ... Now, before you run away ๐Ÿƒ๐Ÿ’จ If you have ๐Ÿ’ป Laptop + ๐Ÿ›œ Internet + 20 minutes, you will be generating blogs, articles etc with just few words. -**[Getting Started for Absolute Begginers]**(https://www.alwrity.com/post/getting-started-with-alwrity-ai-writer) +### [Step-By-Step: Getting Started for Absolute Begginers](https://www.alwrity.com/post/getting-started-with-alwrity-ai-writer) -Getting started for pretentious Developers : Continue Reading..... +### [Getting started for pretentious Developers](https://github.com/AJaySi/AI-Writer/wiki/Alwrity--%E2%80%90-Get-started) **If you still Get stuck, Open a issue here & say pretty please** :: https://github.com/AJaySi/AI-Writer/issues +--- ## Introduction @@ -25,74 +26,9 @@ Leveraging AI technologies, it assists content creators and digital marketers in --- -## Getting Started ๐Ÿš€ ๐Ÿคž๐Ÿคž๐Ÿคž +## Under the hood: -To start using this tool, simply follow one of the options below: - ---- -### Option 1: FOLLOW-ME Local Laptop Install ๐Ÿ’ป **(WINDOWS)** - -#### Step 0 **Pre-requisites:** Git, Python3 ---- - -**Installing Python on Windows:๐Ÿ๐ŸชŸ** -- Open PowerShell as admin: Press `Windows Key + X`, then select "Windows PowerShell (Admin)". - -- Type `python`. If Python is not installed, Windows will prompt you to 'Get Python'. -- If Python is installed, you should see '>>>>>'. - ---- -**Installing Git on Windows:๐Ÿ›บ** -- Open PowerShell or Windows Terminal: Press `Windows Key + X`, then select "Windows Terminal". - -- Paste or type and press enter:โŽ.โŽ.
-winget install --id Git.Git -e --source winget -- Wait for download bars to finish - ---- - -### Step 1: Clone this repository to your local machine. - -To clone the repository to your local machine, perform the following steps: - -1. **Open Windows PowerShell as Administrator:** Open "Windows PowerShell (Admin)" from start menu. - -2. **Navigate to the Desired Directory:** Use the 'cd' command to move to the directory where you want to clone the repository. - -3. **Clone the Repository:** Run the following command in PowerShell to clone the repository: -`git clone https://github.com/AJaySi/AI-Writer.git` -This command will download all the files from the repository to your local machine. - -4. **Verify the Clone:** After the cloning process is complete, navigate into the newly created directory using: -`cd AI-Writer` - -Congratulations: Once you've cloned the repository, you can proceed with the next steps for installation and setup. - ---- - -### Step 2: Install required dependencies: - -- Open command prompt on your local machine. -- Navigate to the folder from Step 1 (AI-Writer) -- Run in powershell: -`python -m pip install -r requirements.txt` - ---- - -### Step 3: Run the script: - -- Run in powershell: -`python alwrity.py` - ---- - -### Step 4: The tool will guide you through setting up your APIs. ---- - - -## Packages, Tools, and APIs Used - -### Standing on the shoulders of Giants - Credits: +### Tech-Stack on the shoulders of Giants - (Credits): - **APIs**: - [Exa API](https://exa.ai/): Provides semantic search capabilities for finding similar topics and technologies. - [Tavily API](https://tavily.com/): Offers AI-powered web search functionality for conducting in-depth keyword research. @@ -108,7 +44,20 @@ Congratulations: Once you've cloned the repository, you can proceed with the nex ## Features -- **Online Research Integration**: Enhances blog content by integrating insights and information gathered from online research, ensuring the content is informative and up-to-date. This gives context for generating content. Tavily AI, Google search, serp and Vision AI is used to scrape web data for context augumentation. TBD: Include CrewAI for web research agents. +- **Online Research Integration**: Enhances blog content by integrating insights and information gathered from online research(SERP, Tavily, Metaphor), ensuring the content is informative and up-to-date. +This gives context for generating content. Tavily AI, Google search, serp and Vision AI is used to scrape web data for context augumentation. CrewAI for web research agents. + +- **Long Form Content Generation**: Write Essay, Story, Long form Blogs with web researched context. + +- **AI Content planning & Calender**: Writer's block, Alwrity will provide you with months worth of blog titles. + +- **Multilingual**: Write Content in your language, web research in your language and country(main_config). + +- **Prevents AI Hallucinations**: Web researched context generates factual content. + +- **Text-To-Text, Speech-To-Text, Text-To-Image, Image-To-Text**: Multimodal content generatiom suite. + +- **Agentic Content Team**: Crewai content team for you company. Define persona, roles, goals, task for your AI content team(Beta). - **Image Generation and Processing**: Utilizes AI models like DALL-E 3, stable difffusion to create relevant images based on blog content. Offers features to process and optimize images for web usage. FIXME: Need more work with stable diffusion. @@ -118,6 +67,7 @@ Congratulations: Once you've cloned the repository, you can proceed with the nex ### AI-Driven Content Creation + - **Text Generation**: Leverages OpenAI's ChatGPT, Google Gemini Pro for generating text for blogs. - [**Customizable AI Parameters**](https://github.com/AJaySi/AI-Writer/blob/main/main_config): Offers flexibility in adjusting AI parameters like model selection, temperature, and token limits to suit different content needs. @@ -125,7 +75,7 @@ Congratulations: Once you've cloned the repository, you can proceed with the nex - **Analyzing and Extracting Image Details**: Uses OpenAI's Vision API, Google Gemini vision to analyze images and extract details such as alt text, descriptions, titles, and captions, enhancing the SEO of image content. --- -### **Note**: This toolkit is designed for automated blog management and requires appropriate API keys and access credentials for full functionality. +**Note**: This toolkit is designed for automated blog management and requires appropriate API keys and access credentials for full functionality. --- ### Web Research @@ -156,22 +106,10 @@ Congratulations: Once you've cloned the repository, you can proceed with the nex - **Menu-Driven Interaction**: Choose between various options, tasks, and tools using intuitive menus and prompts. - **Task Guidance**: Receive guidance and instructions for each task, facilitating user interaction and decision-making. -## Packages, Tools, and APIs Used - -### Standing on the shoulders of Giants - Credits: -- **APIs**: - - Metaphor API: Provides semantic search capabilities for finding similar topics and technologies. - - Tavily API: Offers AI-powered web search functionality for conducting in-depth keyword research. - - SerperDev API: Enables access to search engine results and competitor analysis data. - - YOU.com: You.com enhances web search, writing, coding, digital art creation, and solving complex problems. - - Stability AI: Activating humanity's potential through generative AI. Open models in every modality, for everyone, everywhere. - - OpenAI API: Powers the Large Language Models (LLMs) for generating blog content and conducting research. - - Gemini API: Another LLM provider for natural language processing tasks. - - Ollama API (Work In Progress): An upcoming LLM provider for additional research and content generation capabilities. --- -Notes: +Notes from underground: 1). Focus is on writing/generating highly unique, SEO optimized blog content. 2). Models: Openai, gemini, ollama are interesting. Minstral API is also worth exploring. Cohere API is purpose made. diff --git a/alwrity.py b/alwrity.py index 8813e7d5..41ba9bd6 100644 --- a/alwrity.py +++ b/alwrity.py @@ -38,9 +38,11 @@ def write_blog_options(): choices = [ ("Keywords", "Keywords - Provide main blog keywords Or Title"), ("Audio To Blog", "Audio To Blog - Transcribe Audio files into blog content"), + ("AI Story Writer", "AI Story Writer"), + ("AI Essay Writer", "AI Essay writer"), + ("AI News Articles", "News - AI News article writer, factual trusted sources"), ("Programming", "Programming - Write technical blogs on latest topics"), ("Scholar", "Scholar - Research Reports from google scholar, arxiv articles."), - ("AI News Articles", "News - AI News article writer, factual trusted sources"), ("Finance/TBD", "Finance/TBD"), ("Quit", "Quit") ] @@ -53,20 +55,15 @@ def start_interactive_mode(): os.system("clear" if os.name == "posix" else "cls") text = "_______________________________________________________________________\n" text += "\nโš ๏ธ Alert! ๐Ÿ’ฅโ“๐Ÿ’ฅ\n" - text += "If you know what to write, choose 'Write Blog'\n" - text += "If unsure, let's 'do web research' to write on\n" - text += "If Testing-it-out/getting-started, choose 'Blog Tools\n" + text += "Interactive tool, needs your attention/inputs, get off your mobile..'\n" text += "_______________________________________________________________________\n" print(text) choices = [ - ("AI Writer", "AI Writer - Blog, YT2Blog, Research Report, News, Finance long-form articles"), - ("AI Story Writer", "AI Story Writer"), - ("AI Essay Writer", "AI Essay writer"), + ("AI Writers", "Choose AI Writers - (I Know What To Write)"), + ("Content Planning", "Content Planning Tools - (I Don't Know, What to write)"), ("AI Image to Text Writer", "AI Image to Text Writer"), ("Online Blog Tools/Apps", "Online AI Apps - Content & Digital marketing"), - ("Do keyword Research", "Keywords web research - Basic, AI and semantic web research"), - ("Competitor Analysis", "Competitor Analysis"), ("Create Blog Images", "Create Images - Stability, Dalle3"), ("AI Social Media(TBD)", "AI Social Media(TBD)"), ("AI Code Writer(TBD)", "AI Code Writer(TBD)"), @@ -74,31 +71,18 @@ def start_interactive_mode(): ] mode = radiolist_dialog(title="Choose an option:", values=choices).run() if mode: - if mode == 'AI Writer': + if mode == 'AI Writers': write_blog() - elif mode == 'AI Story Writer': - write_story() - elif mode == 'AI Essay Writer': - essay_writer() elif mode == 'AI Image to Text Writer': image_to_text_writer() - elif mode == 'Do keyword Research': - if check_search_apis(): - do_web_research() elif mode == 'Create Blog Images': image_generator() - elif mode == 'Competitor Analysis': - competitor_analysis() + elif mode == 'Content Planning': + content_planning_tools() elif mode == 'Online Blog Tools/Apps': blog_tools() elif mode == 'AI Social Media(TBD)': - print(""" - #whatsapp - #instagram - #youtube - #twitter/X - #Linked-in posts - """) + print(""" #whatsapp #instagram #youtube #twitter/X #Linked-in posts """) raise typer.Exit() elif mode == 'AI Code Writer(TBD)': print("Coming soon, TBD") @@ -108,6 +92,31 @@ def start_interactive_mode(): raise typer.Exit() +def content_planning_tools(): + """ """ + os.system("clear" if os.name == "posix" else "cls") + text = "_______________________________________________________________________\n" + text += "\nโš ๏ธ Alert! ๐Ÿ’ฅโ“๐Ÿ’ฅ\n" + text += "ไฝœๅฎถ็š„้šœ็ข - Writer's block - Bloqueo de escritor - Schreibblockade\n" + text += "Use Google Keyword planner, google ads instead. Better tools than below.\n" + text += "Note: Who Cares, just give some titles, keywords to get started.. To Err is Human & AI..\n" + text += "_______________________________________________________________________\n" + print(text) + + choices = [ + ("Do keyword Research", "Keywords web research - ๐Ÿค“ Will read & earn my bread.."), + ("Competitor Analysis", "Competitor Analysis - ๐ŸงWhat's my neighbour doing.."), + ("Blog Titles", "๐Ÿฅน๐Ÿฅน Just give me Topic titles") + ] + mode = radiolist_dialog(title="Choose an option:", values=choices).run() + + if mode == 'Do keyword Research': + if check_search_apis(): + do_web_research() + elif mode == 'Competitor Analysis': + competitor_analysis() + + def check_search_apis(): """ Check if necessary environment variables are present. @@ -163,10 +172,12 @@ def write_blog(): if blog_type: if blog_type == 'Keywords': blog_from_keyword() - + elif mode == 'AI Story Writer': + write_story() + elif mode == 'AI Essay Writer': + essay_writer() elif blog_type == 'Audio To Blog': blog_from_audio() - elif blog_type == 'AI News Articles': ai_news_writer() elif blog_type == 'GitHub': diff --git a/lib/ai_web_researcher/google_serp_search.py b/lib/ai_web_researcher/google_serp_search.py index c3efd5fa..5e672a2d 100644 --- a/lib/ai_web_researcher/google_serp_search.py +++ b/lib/ai_web_researcher/google_serp_search.py @@ -181,7 +181,6 @@ def perform_serperdev_google_search(query): # Check if the request was successful if response.status_code == 200: # Parse and return the JSON response - process_search_results(response) return response.json() else: # Print an error message if the request fails @@ -218,7 +217,6 @@ def perform_serper_news_search(news_keywords, news_country, news_language): if response.status_code == 200: # Parse and return the JSON response #process_search_results(response, "news") - #google_news(news_keywords) return response.json() else: # Print an error message if the request fails @@ -255,7 +253,7 @@ def process_search_results(search_results, search_type="general"): pd.DataFrame: Pandas DataFrame containing the search results. """ data = [] - #logger.info(f"Google Search Parameters: {search_results.get('searchParameters', {})}") + logger.info(f"Google Search Parameters: {search_results.get('searchParameters', {})}") if 'general' in search_type: organic_results = search_results.get("organic", []) if 'news' in search_type: diff --git a/lib/ai_writers/blog_from_google_serp.py b/lib/ai_writers/blog_from_google_serp.py index 31eb0f3a..7785a101 100644 --- a/lib/ai_writers/blog_from_google_serp.py +++ b/lib/ai_writers/blog_from_google_serp.py @@ -16,19 +16,21 @@ from ..gpt_providers.text_generation.main_text_generation import llm_text_gen def write_blog_google_serp(search_keyword, search_results): """Combine the given online research and gpt blog content""" prompt = f""" + You are Alwrity, the Creative Content writer, writing up fresh ideas and crafts them with care and uniqueness. + She makes complex topics simple to understand and writes in a friendly, conversational tone that connects with everyone. + She excels at creating a introductions that attract users, to read more. + As a SEO expert and content writer, I will provide you with my 'web research keywords' and its 'google search result'. - Your goal is to create SEO-optimized content and also include 5 FAQs. + Your goal is to create detailed SEO-optimized content and also include 5 FAQs. + Follow below guidelines: 1). Your blog content should compete against all blogs from search results. 2). Your FAQ should be based on 'People also ask' and 'Related Queries' from given search result. - Always include answers for each FAQ, use your knowledge and confirm with snippets given in search result. - 3). Your blog should be highly detailed, unique and written in human-like personality & tone. - 4). Adopt an engaging, helpful voice, providing actionable and concrete insights, and avoiding buzzwords. - 5). Act as subject matter expert for given research keywords and include statistics and facts. - 6). Do not explain, describe your response. - 7). Your blog should be highly formatted in markdown style and highly readable. - 8). Important: Please read the entire prompt before writing anything. Follow the prompt exactly as I instructed. + Always include answers for each FAQ, use your knowledge and confirm with snippets given in search result. + 3). Act as subject matter expert for given research keywords: {search_keyword}. + 4). Your blog should be highly formatted in markdown style and highly readable. + 5). Always write in the first person. \n\nWeb Research Keyword: "{search_keyword}" Google search Result: "{search_results}" diff --git a/lib/ai_writers/combine_blog_and_keywords.py b/lib/ai_writers/combine_blog_and_keywords.py index 3542c0fb..8c92ccda 100644 --- a/lib/ai_writers/combine_blog_and_keywords.py +++ b/lib/ai_writers/combine_blog_and_keywords.py @@ -20,6 +20,11 @@ def blog_with_keywords(blog, keywords): Your task is to write an original blog, utilizing given keywords and blog content. Your blog should be highly detailed and well formatted. + You are Sarah, the Creative Content writer, writing up fresh ideas and crafts them with care. + She makes complex topics easy to understand and writes in a friendly tone that connects with everyone. + She excels at simplifying complex topics and communicates with charisma, making technical jargon come alive for her audience. + + Blog content: '{blog}' list of keywords: '{keywords}' """ diff --git a/lib/ai_writers/combine_research_and_blog.py b/lib/ai_writers/combine_research_and_blog.py index 839044ab..cf84089d 100644 --- a/lib/ai_writers/combine_research_and_blog.py +++ b/lib/ai_writers/combine_research_and_blog.py @@ -14,23 +14,21 @@ from ..gpt_providers.text_generation.main_text_generation import llm_text_gen def blog_with_research(report, blog): """Combine the given online research and gpt blog content""" prompt = f""" - You are an expert content editor specializing in SEO content optimization for blogs. + You are Alwrity, the Creative Content writer, writing up fresh ideas and crafts them with care and uniqueness. + She makes complex topics simple to understand and writes in a friendly, conversational tone that connects with everyone. + She excels at simplifying complex topics and writes in the first person, for her audience. + I will provide you with a 'research report' and a 'blog content' on the same topic. - Your task is to follow below given guidelines to write a new, unique, and informative blog article + Your task is to follow below given guidelines to write a detailed blog article, that will rank well in search engine results and engage readers effectively. Follow below given guidelines: 1. Master the report and blog content: Understand main ideas, key points, and the core message. 2. Sentence Structure: Rephrase while preserving logical flow and conversational tone. 3. Identify Main Keywords: Determine the primary topic and combine the articles on that main topic. - 4. Use Creative and Human-like Style: Incorporate contractions, idioms, transitional phrases, - interjections, and colloquialisms. 5. Blog Structuring: Include an Introduction, subtopics and use bullet points or numbered lists if appropriate. Important to include FAQs, Conclusion and Referances. - 6. Ensure Uniqueness: Guarantee the article is plagiarism-free. Write in human-like and informative style. - 7. Act as subject matter expert and include statistics and facts in your combined article. - 8. Do not provide explanations for your response. - Important: Please read the entire prompt before writing anything. Follow the prompt exactly as I instructed.\n\n + 7. .\n\n Research report: '{report}' Blog content: '{blog}' diff --git a/lib/ai_writers/keywords_to_blog.py b/lib/ai_writers/keywords_to_blog.py index 2723df74..57376abf 100644 --- a/lib/ai_writers/keywords_to_blog.py +++ b/lib/ai_writers/keywords_to_blog.py @@ -81,13 +81,13 @@ def write_blog_from_keywords(search_keywords, url=None): #blog_markdown_str = blog_with_research(blog_markdown_str, you_search_result) #logger.info(f"Final blog content: {blog_markdown_str}") - logger.info("Pass Final blog for blog-proof reading and *improvements.") + #logger.info("Pass Final blog for blog-proof reading and *improvements.") # Pass the final content for proofreading. - blog_markdown_str = blog_proof_editor(blog_markdown_str) + #blog_markdown_str = blog_proof_editor(blog_markdown_str) # Pass the content to remove obivious words used by AI. - logger.info("Pass Final blog for Humanizing it further, Doesn't matter, Really?") - blog_markdown_str = blog_humanize(blog_markdown_str) + #logger.info("Pass Final blog for Humanizing it further, Doesn't matter, Really?") + #blog_markdown_str = blog_humanize(blog_markdown_str) blog_title, blog_meta_desc, blog_tags, blog_categories = blog_metadata(blog_markdown_str, search_keywords, example_blog_titles)