HTML tags. Convert code blocks (```) to HTML tags
- for preserving formatting.
- - Blockquotes: Convert blockquotes (>) to HTML tags.
- Convert the following Markdown text to HTML: {md_content}
- """
- try:
- # TBD: Add logic for which_provider and which_model
- response = openai_chatgpt(prompt)
- return response
- except Exception as err:
- SystemError(f"Error in convert to HTML")
-
-
-def blog_with_research(report, blog):
- """Combine the given online research and gpt blog content"""
-
- prompt = f"""
- You are an expert copywriter specializing in content optimization for SEO.
- I will provide you with a research report and a blog content on the same topic.
- Treat the research report as the context for the blog and better it accordingly.
- Your task is to transform and combine the given research and blog content into a well-structured, unique
- and engaging blog article.
- Your objectives include:
- 1. Master the report and blog content: Understand main ideas, key points, and the core message.
- 2. Sentence Structure: Rephrase while preserving logical flow and coherence.
- 3. Identify Main Keyword: Determine the primary topic and combine the articles on the main topic.
- 4. Keyword Integration: Naturally integrate keywords in headings, subheadings, and body text, avoiding overuse.
- 5. Write Unique Content: Avoid direct copying from given report and blog; rewrite in your own words and style.
- 6. Optimize for SEO: Generate high quality informative content.
- Implement SEO best practises with appropriate keyword density.
- 7. Craft Engaging and Informative Article: Provide value and insight to readers.
- 8. Proofread: Important to Check for grammar, spelling, and punctuation errors.
- 9. Use Creative and Human-like Style: Incorporate contractions, idioms, transitional phrases,
- interjections, and colloquialisms. Avoid repetitive phrases and unnatural sentence structures.
- 10. Structuring: Include an Introduction, subtopics and use bullet points or
- numbered lists if appropriate. Important to include FAQs, and Conclusion.
- 11. Ensure Uniqueness: Guarantee the article is plagiarism-free. Write in unique, informative style.
- 12. Punctuation: Use appropriate question marks at the end of questions.
- 13. Pass AI Detection Tools: Create content that easily passes AI plagiarism detection tools.
- 14. REMEMBER to give final response as complete HTML.
- Follow these guidelines to create a well-optimized, unique, and informative article
- that will rank well in search engine results and engage readers effectively.
-
- Create a blog post from the given research report and blog content below.
- Research report: {report}
- Blog content: {blog}
- """
- try:
- # TBD: Add logic for which_provider and which_model
- response = openai_chatgpt(prompt)
- return response
- except Exception as err:
- SystemError(f"Error in getting related keywords.")
-
-
def blog_proof_editor(blog_content, blog_keywords):
"""
Helper for blog proof reading.
@@ -659,12 +422,12 @@ def blog_proof_editor(blog_content, blog_keywords):
exit(1)
prompt = f"""I am looking for detailed editing and enhancement of the given blog post,
- with a particular focus on maintaining originality.
- The topic of the content is [{blog_keywords}]. Please go through the blog and make direct edits to improve it,
+ with a particular focus on originality. I will provide you with a blog content and its keywords.
+ The keywords for the blog are [{blog_keywords}]. Please go through the blog and make direct edits to improve it,
ensuring the final output is both high-quality and original.
Note: There are duplicates headings and corresponding paragraphs, rewrite into one subheading.
- Here are the specific areas to focus on:
+ Here are the specific guidelines to focus on:
1). Ensure Originality: Edit any sections that lack originality, replacing them with unique and creative content.
2). Eliminate Repetitive Language: Rewrite repetitive phrases with varied and engaging language.
@@ -677,8 +440,11 @@ def blog_proof_editor(blog_content, blog_keywords):
7). Remove Redundancies: Important, Cut out any redundant information or overly complex jargon.
8). Refine Overall Structure: Make structural changes to improve the overall impact of the content.
9). Remember, rewrite all content that repeated, while maintaining the formatting of the given blog text.
+ 10). Remember Not to include SEO meta description and Title in your final response.
+ 11). REMEMBER to maintain the formatting style of the provided blog.
+ 12). Judge if the given blog is about technology then provide code snippets and examples for it.
- Please apply these changes directly to the following blog text and provide the edited version:
+ Please make direct changes as per above guideline to the provided blog text below:
[{blog_content}]. """
try:
diff --git a/lib/gpt_online_researcher.py b/lib/gpt_online_researcher.py
index 02a14041..15c357df 100644
--- a/lib/gpt_online_researcher.py
+++ b/lib/gpt_online_researcher.py
@@ -68,11 +68,11 @@ def do_online_research(query):
f'Please use MLA format and markdown syntax.')
}]
# Run GPT-4
- logging.info("Generating report with GPT-4...")
+ logging.info("Generating Research report with GPT-4...")
lc_messages = convert_openai_messages(prompt)
try:
report = ChatOpenAI(model='gpt-4', openai_api_key=openai_api_key).invoke(lc_messages).content
- logging.info(f"\n Below is the online research report for given keywords/title: \n\n{report}")
+ #logging.info(f"\n Below is the online research report for given keywords/title: \n\n{report}")
return report
except Exception as err:
logging.error("Failed to generate do_online_research with ChatOpenAI")
diff --git a/lib/gpt_providers/stt_audio_blog.py b/lib/gpt_providers/stt_audio_blog.py
index 892a05b4..c3012ec7 100644
--- a/lib/gpt_providers/stt_audio_blog.py
+++ b/lib/gpt_providers/stt_audio_blog.py
@@ -71,7 +71,7 @@ def speech_to_text(video_url, output_path='.'):
file=open(audio_file, "rb"),
response_format="text"
)
- logger.info("\nYouTube video transcription:\n\n{transcript}\n")
+ logger.info(f"\nYouTube video transcription:\n\n{transcript}\n")
return transcript, yt.title
except Exception as e:
diff --git a/lib/online_research_agent.py b/lib/online_research_agent.py
new file mode 100644
index 00000000..72512480
--- /dev/null
+++ b/lib/online_research_agent.py
@@ -0,0 +1,201 @@
+import os
+import requests
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+from langchain import PromptTemplate
+from langchain.chains.summarize import load_summarize_chain
+from bs4 import BeautifulSoup
+from langchain.chat_models import ChatOpenAI
+from dotenv import load_dotenv
+import json
+from autogen import config_list_from_json
+from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent
+from autogen import UserProxyAgent
+import autogen
+
+
+load_dotenv()
+brwoserless_api_key = os.getenv("BROWSERLESS_API_KEY")
+serper_api_key = os.getenv("SERP_API_KEY")
+airtable_api_key = os.getenv("AIRTABLE_API_KEY")
+config_list = config_list_from_json("OAI_CONFIG_LIST")
+
+
+# ------------------ Create functions ------------------ #
+
+# Function for google search
+def google_search(search_keyword):
+ url = "https://google.serper.dev/search"
+
+ payload = json.dumps({
+ "q": search_keyword
+ })
+
+ headers = {
+ 'X-API-KEY': serper_api_key,
+ 'Content-Type': 'application/json'
+ }
+
+ response = requests.request("POST", url, headers=headers, data=payload)
+ print("RESPONSE:", response.text)
+ return response.text
+
+# Function for scraping
+def summary(objective, content):
+ llm = ChatOpenAI(temperature = 0, model = "gpt-3.5-turbo-16k-0613")
+
+ text_splitter = RecursiveCharacterTextSplitter(separators=["\n\n", "\n"], chunk_size = 10000, chunk_overlap=500)
+ docs = text_splitter.create_documents([content])
+
+ map_prompt = """
+ Write a summary of the following text for {objective}:
+ "{text}"
+ SUMMARY:
+ """
+ map_prompt_template = PromptTemplate(template=map_prompt, input_variables=["text", "objective"])
+
+ summary_chain = load_summarize_chain(
+ llm=llm,
+ chain_type='map_reduce',
+ map_prompt = map_prompt_template,
+ combine_prompt = map_prompt_template,
+ verbose = False
+ )
+
+ output = summary_chain.run(input_documents=docs, objective=objective)
+
+ return output
+
+def web_scraping(objective: str, url: str):
+ #scrape website, and also will summarize the content based on objective if the content is too large
+ #objective is the original objective & task that user give to the agent, url is the url of the website to be scraped
+
+ print("Scraping website...")
+ # Define the headers for the request
+ headers = {
+ 'Cache-Control': 'no-cache',
+ 'Content-Type': 'application/json',
+ }
+
+ # Define the data to be sent in the request
+ data = {
+ "url": url
+ }
+
+ # Convert Python object to JSON string
+ data_json = json.dumps(data)
+
+ # Send the POST request
+ response = requests.post(f"https://chrome.browserless.io/content?token={brwoserless_api_key}", headers=headers, data=data_json)
+
+ # Check the response status code
+ if response.status_code == 200:
+ soup = BeautifulSoup(response.content, "html.parser")
+ text = soup.get_text()
+ print("CONTENTTTTTT:", text)
+ if len(text) > 10000:
+ output = summary(objective,text)
+ return output
+ else:
+ return text
+ else:
+ print(f"HTTP request failed with status code {response.status_code}")
+
+
+# Function for get airtable records
+def get_airtable_records(base_id, table_id):
+ url = f"https://api.airtable.com/v0/{base_id}/{table_id}"
+
+ headers = {
+ 'Authorization': f'Bearer {airtable_api_key}',
+ }
+
+ response = requests.request("GET", url, headers=headers)
+ data = response.json()
+ print(data)
+ return data
+
+
+# Function for update airtable records
+
+def update_single_airtable_record(base_id, table_id, id, fields):
+ url = f"https://api.airtable.com/v0/{base_id}/{table_id}"
+
+ headers = {
+ 'Authorization': f'Bearer {airtable_api_key}',
+ "Content-Type": "application/json"
+ }
+
+ data = {
+ "records": [{
+ "id": id,
+ "fields": fields
+ }]
+ }
+
+ response = requests.patch(url, headers=headers, data=json.dumps(data))
+ data = response.json()
+ return data
+
+
+# ------------------ Create agent ------------------ #
+
+# Create user proxy agent
+user_proxy = UserProxyAgent(name="user_proxy",
+ is_termination_msg=lambda msg: "TERMINATE" in msg["content"],
+ human_input_mode="ALWAYS",
+ max_consecutive_auto_reply=1
+ )
+
+# Create researcher agent
+researcher = GPTAssistantAgent(
+ name = "researcher",
+ llm_config = {
+ "config_list": config_list,
+ "assistant_id": "asst_qyvioid5My8K3SdFClaEnwmB"
+ }
+)
+
+researcher.register_function(
+ function_map={
+ "web_scraping": web_scraping,
+ "google_search": google_search
+ }
+)
+
+# Create research manager agent
+research_manager = GPTAssistantAgent(
+ name="research_manager",
+ llm_config = {
+ "config_list": config_list,
+ "assistant_id": "asst_C1Ta5XmmEcYD6vnOSVflnwG9"
+ }
+)
+
+
+# Create director agent
+director = GPTAssistantAgent(
+ name = "director",
+ llm_config = {
+ "config_list": config_list,
+ "assistant_id": "asst_zVBJGch5mOyCYl9H1J3L9Ime",
+ }
+)
+
+director.register_function(
+ function_map={
+ "get_airtable_records": get_airtable_records,
+ "update_single_airtable_record": update_single_airtable_record
+ }
+)
+
+
+# Create group chat
+groupchat = autogen.GroupChat(agents=[user_proxy, researcher, research_manager, director], messages=[], max_round=15)
+group_chat_manager = autogen.GroupChatManager(groupchat=groupchat, llm_config={"config_list": config_list})
+
+
+# ------------------ start conversation ------------------ #
+message = """
+Research the funding stage/amount & pricing for each company in the list: https://airtable.com/appj0J4gFpvLrQWjI/tblF4OmG6oLjYtgZl/viwmFx2ttAVrJm0E3?blocks=hide
+"""
+user_proxy.initiate_chat(group_chat_manager, message=message)
diff --git a/lib/optimize_images_for_upload.py b/lib/optimize_images_for_upload.py
new file mode 100644
index 00000000..16b938f4
--- /dev/null
+++ b/lib/optimize_images_for_upload.py
@@ -0,0 +1,112 @@
+import sys
+import os
+import tinify
+from PIL import Image
+from loguru import logger
+from tqdm import tqdm
+from dotenv import load_dotenv
+
+#default directory for .env file is the current directory
+#if you set .env in different directory, put the directory address load_dotenv("directory_of_.env)
+load_dotenv()
+
+# Retrieve Tinyfy API key from environment variable
+tinify.key = os.getenv('TINIFY_API_KEY')
+
+# Configure logger
+logger.remove()
+logger.add(sys.stdout, colorize=True, format="{level}|{file}:{line}:{function}| {message}")
+
+def compress_image(image_path, quality=45, resize=None, preserve_exif=False):
+ """
+ Compress and optionally resize an image, and overwrite the original image.
+
+ Args:
+ image_path (str): Path to the original image.
+ quality (int): Quality of the output image (1-100).
+ resize (tuple): Tuple (width, height) to resize image.
+ preserve_exif (bool): Preserve EXIF data if True.
+ """
+ if not os.path.exists(image_path):
+ logger.error(f"Image path does not exist: {image_path}")
+ return
+
+ original_size = os.path.getsize(image_path)
+ try:
+ with Image.open(image_path) as img:
+ img_format = img.format
+ exif = img.info['exif'] if preserve_exif and 'exif' in img.info else None
+
+ if resize:
+ img = img.resize(resize, Image.ANTIALIAS)
+
+ img.save(image_path, format=img_format, quality=quality, optimize=True, exif=exif)
+
+ compressed_size = os.path.getsize(image_path)
+ reduction = (1 - (compressed_size / original_size)) * 100
+ logger.info(f"Compressed {image_path}, Reduction: {reduction:.2f}%")
+ except Exception as e:
+ logger.error(f"Error compressing image {image_path}: {e}")
+
+def is_image_file(filename):
+ """
+ Check if a file is an image based on its extension.
+
+ Args:
+ filename (str): Name of the file to check.
+
+ Returns:
+ bool: True if the file is an image, False otherwise.
+ """
+ valid_extensions = ['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.webp']
+ return any(filename.lower().endswith(ext) for ext in valid_extensions)
+
+
+def convert_to_webp(image_path):
+ """
+ Convert an image to WebP format.
+
+ Args:
+ image_path (str): Path to the original image.
+
+ Returns:
+ str: Path to the WebP image.
+ """
+ if not os.path.exists(image_path):
+ logger.error(f"Image path does not exist: {image_path}")
+ return
+
+ try:
+ with Image.open(image_path) as img:
+ webp_path = os.path.splitext(image_path)[0] + '.webp'
+ img.save(webp_path, 'WEBP')
+ logger.info(f"Converted {image_path} to WebP")
+ return webp_path
+ except Exception as e:
+ logger.error(f"Error converting image to WebP: {e}")
+
+
+def compress_image_tinyfy(image_path):
+ """
+ Compress the image using Tinyfy API.
+
+ Args:
+ image_path (str): Path to the original image.
+ """
+ if not os.path.exists(image_path):
+ logger.error(f"Image path does not exist: {image_path}")
+ return
+
+ try:
+ source = tinify.from_file(image_path)
+ source.to_file(image_path)
+ logger.info(f"Compressed {image_path} using Tinyfy API")
+ except tinify.Error as e:
+ logger.error(f"Tinyfy API error: {e}")
+
+
+def optimize_image(image_path):
+ image_path = convert_to_webp(image_path)
+ compress_image_tinyfy(image_path)
+ compress_image(image_path)
+ return image_path
diff --git a/lib/save_blog_to_file.py b/lib/save_blog_to_file.py
index 9ebe93fc..4de3c648 100644
--- a/lib/save_blog_to_file.py
+++ b/lib/save_blog_to_file.py
@@ -1,76 +1,101 @@
-def save_blog_to_file(blog_content, blog_title,
- blog_meta_desc, blog_tags, blog_categories, main_img_path, file_type="md"):
- """ Common function to save the generated blog to a file.
- arg: file_type can be md or html
+import sys
+import os
+import re
+import datetime
+from textwrap import dedent
+import logging
+from zoneinfo import ZoneInfo
+import nltk
+from nltk.corpus import stopwords
+from loguru import logger
+logger.remove()
+logger.add(sys.stdout,
+ colorize=True,
+ format="{level}|{file}:{line}:{function}| {message}"
+ )
+
+
+def save_blog_to_file(blog_content, blog_title, blog_meta_desc, blog_tags, blog_categories, main_img_path, output_path, file_type="md"):
"""
- # Convert the spaces in blog_title with dash
- logger.info(f"The blog will be saved at: {output_path}")
- logger.debug(f"Blog Title is: {blog_title}")
- blog_title_md = blog_title
- regex = re.compile('[^a-zA-Z0-9- ]')
- blog_title_md = regex.sub('', blog_title_md)
- blog_title= blog_title.replace(":", "")
- blog_title_md = re.sub('--+', '-', blog_title_md)
- blog_title_md = blog_title_md.replace(' ', '-')
+ Saves the provided blog content to a file in the specified format.
+
+ Args:
+ blog_content (str): The main content of the blog.
+ blog_title (str): Title of the blog.
+ blog_meta_desc (str): Meta description of the blog.
+ blog_tags (list): List of tags associated with the blog.
+ blog_categories (list): List of categories associated with the blog.
+ main_img_path (str): Path to the main image of the blog.
+ output_path (str): Path to the directory where the blog will be saved.
+ file_type (str, optional): The file format for saving the blog ('md' for Markdown or 'html' for HTML). Defaults to 'md'.
+
+ Raises:
+ FileNotFoundError: If the output_path does not exist.
+ Exception: If the blog content cannot be written to the file.
+ """
+ # Sanitize and prepare the blog title
+ # Remove colon and ampersand
+ blog_title_md = blog_title.replace(":", "").replace("&", "")
+ # Replace spaces with hyphens
+ blog_title_md = blog_title_md.replace(" ", "-")
+ blog_title_md = re.sub('[^A-Za-z0-9-]', '', blog_title_md)
+ # Replace multiple consecutive dashes with a single dash
+ blog_title_md = re.sub('-+', '-', blog_title_md)
blog_title_md = remove_stop_words(blog_title_md)
+ logger.debug(f"Blog Title is: {blog_title_md}")
- if ':' in blog_meta_desc:
- blog_meta_desc = blog_meta_desc.split(':')[1].strip()
-
+ # Check if output path exists
if not os.path.exists(output_path):
- logger.error("Error: Blog output directory is set to {output_path}, which Does Not Exist.")
+ logger.error(f"Error: Blog output directory is set to {output_path}, which does not exist.")
+ raise FileNotFoundError(f"Output directory does not exist: {output_path}")
- # Different output formats are plaintext, html and markdown.
- if file_type in "md":
- logger.info(f"Writing/Saving the resultant blog content in Markdown format.")
- # fill the Front Matter as below at the top of the post: https://jekyllrb.com/docs/front-matter/
- # date: YYYY-MM-DD HH:MM:SS +/-TTTT
- from zoneinfo import ZoneInfo
- tz=ZoneInfo('Asia/Kolkata')
- dtobj = datetime.datetime.now(tz=ZoneInfo('Asia/Kolkata'))
- formatted_date = f"{dtobj.strftime('%Y-%m-%d %H:%M:%S %z')}"
-
- blog_frontmatter = f"""\
+ # Handle Markdown file type
+ if file_type == "md":
+ logger.info("Writing/Saving the resultant blog content in Markdown format.")
+ dtobj = datetime.datetime.now(ZoneInfo('Asia/Kolkata'))
+ formatted_date = dtobj.strftime('%Y-%m-%d %H:%M:%S %z')
+ blog_title = blog_title.replace(":", "-").replace('"', '')
+ blog_frontmatter = dedent(f"""\
---
title: {blog_title}
date: {formatted_date}
categories: [{blog_categories}]
tags: [{blog_tags}]
- description: {blog_meta_desc}
+ description: {blog_meta_desc.replace(":", "-")}
img_path: '/assets/'
image:
path: {os.path.basename(main_img_path)}
alt: {blog_title}
- ---\n\n"""
+ ---\n\n""")
- # Create a new file named YYYY-MM-DD-TITLE.EXTENSION and put it in the _posts of the root directory.
- # Please note that the EXTENSION must be one of md or markdown
blog_output_path = os.path.join(
- output_path,
- f"{datetime.date.today().strftime('%Y-%m-%d')}-{blog_title_md}.md"
- )
- # Save the generated blog content to a file.
+ output_path,
+ f"{datetime.date.today().strftime('%Y-%m-%d')}-{blog_title_md}.md"
+ )
+
+ # Write to the file
try:
with open(blog_output_path, "w") as f:
- f.write(dedent(blog_frontmatter))
+ f.write(blog_frontmatter)
f.write(blog_content)
except Exception as e:
raise Exception(f"Failed to write blog content: {e}")
- logger.info(f"\nSuccessfully saved and Posted blog at: {blog_output_path,}\n")
+
+ logger.info(f"Successfully saved and posted blog at: {blog_output_path}")
# Helper function
def remove_stop_words(sentence):
- # Tokenize the sentence into words
+ """
+ Removes stop words from a given sentence.
+
+ Args:
+ sentence (str): The sentence from which to remove stop words.
+
+ Returns:
+ str: The sentence after removing stop words.
+ """
words = nltk.word_tokenize(sentence)
-
- # Get the list of English stop words
stop_words = set(stopwords.words('english'))
-
- # Remove stop words from the sentence
filtered_words = [word for word in words if word.lower() not in stop_words]
-
- # Join the filtered words back into a sentence
- filtered_sentence = ' '.join(filtered_words)
-
- return filtered_sentence
+ return ' '.join(filtered_words)
diff --git a/lib/is_content_ai_generated.py b/lib/seo_module/is_content_ai_generated.py
similarity index 100%
rename from lib/is_content_ai_generated.py
rename to lib/seo_module/is_content_ai_generated.py
diff --git a/lib/plagiarism_checker_from_known_sources.py b/lib/seo_module/plagiarism_checker_from_known_sources.py
similarity index 100%
rename from lib/plagiarism_checker_from_known_sources.py
rename to lib/seo_module/plagiarism_checker_from_known_sources.py
diff --git a/lib/toc_example.py b/lib/toc_example.py
deleted file mode 100644
index df52680b..00000000
--- a/lib/toc_example.py
+++ /dev/null
@@ -1,44 +0,0 @@
-from bs4 import BeautifulSoup
-import re
-
-def create_table_of_contents(html_content):
- """
- Create a table of contents for a given HTML content.
-
- Args:
- html_content (str): HTML content of the blog post.
-
- Returns:
- str: HTML content with a table of contents.
- """
- # Use BeautifulSoup to parse the HTML
- soup = BeautifulSoup(html_content, 'html.parser')
-
- # Find all header tags (h1, h2, h3, h4, h5, h6)
- headers = soup.find_all(re.compile('^h[1-6]$'))
-
- # Create a table of contents
- toc = BeautifulSoup('', 'html.parser')
- toc_ul = toc.find('ul')
-
- # Loop through headers and add them to the table of contents
- for i, header in enumerate(headers, start=1):
- header_id = f"toc_{i}"
- header['id'] = header_id
-
- toc_entry = soup.new_tag('li')
- toc_link = soup.new_tag('a', href=f"#{header_id}")
- toc_link.string = header.get_text()
- toc_entry.append(toc_link)
- toc_ul.append(toc_entry)
-
- # Insert the table of contents at the beginning of the content
- soup.insert(0, toc)
-
- return str(soup)
-
-# Example usage
-html_content = "Title
Some text
Subtitle 1
Text under subtitle 1
Subtitle 2
Text under subtitle 2
"
-html_with_toc = create_table_of_contents(html_content)
-print(html_with_toc)
-
diff --git a/lib/write_blogs_from_youtube_videos.py b/lib/write_blogs_from_youtube_videos.py
index 521e70f5..dc359273 100644
--- a/lib/write_blogs_from_youtube_videos.py
+++ b/lib/write_blogs_from_youtube_videos.py
@@ -7,6 +7,8 @@ import tempfile
import openai
from html2image import Html2Image
from tqdm import tqdm, trange
+import google.generativeai as genai
+
from loguru import logger
logger.remove()
logger.add(sys.stdout,
@@ -36,7 +38,8 @@ def youtube_to_blog(video_url):
try:
# Summarizing the content of the YouTube video
- audio_blog_content = summarize_youtube_video(audio_text)
+ audio_blog_content = summarize_youtube_video_openai(audio_text, "gemini")
+ logger.info("Successfully converted given URL to blog article.")
return audio_blog_content, audio_title
except Exception as e:
logger.error(f"Error in summarize_youtube_video: {e}")
@@ -44,41 +47,51 @@ def youtube_to_blog(video_url):
return audio_blog_content
-def summarize_youtube_video(user_content):
- """Generates a summary of a YouTube video using OpenAI GPT-3 and displays a progress bar.
-
+def summarize_youtube_video(user_contenti, gpt_providers):
+ """Generates a summary of a YouTube video using OpenAI GPT-3 and displays a progress bar.
Args:
video_link: The URL of the YouTube video to summarize.
-
Returns:
A string containing the summary of the video.
"""
+
logger.info("Start summarize_youtube_video..")
prompt = f"""
- You are an expert copywriter specializing in content optimization for SEO.
- Your task is to transform a given transcript into a well-structured and engaging blog article. Your objectives include:
+ You are an expert copywriter specializing in digital content writing. I will provide you with a transcript.
+ Your task is to transform a given transcript into a well-structured and informative blog article.
+ Please follow the below objectives:
1. Master the Transcript: Understand main ideas, key points, and the core message.
2. Sentence Structure: Rephrase while preserving logical flow and coherence. Dont quote anyone from video.
- 3. Identify Main Keywords: Determine the primary video topic.
- 4. Keyword Integration: Naturally integrate keywords in headings, subheadings, and body text, avoiding overuse.
- 5. Write Unique Content: Avoid direct copying; rewrite in your own words.
- REMEMBER to avoid direct quoting and maintain uniqueness.
- 6. Optimize for SEO: Implement meta tags, header tags, and appropriate keyword density.
- 7. Craft Engaging and Informative Article: Provide value and insight to readers.
- 8. Proofread: Check for grammar, spelling, and punctuation errors.
- 9. Use Creative and Human-like Style: Incorporate contractions, idioms, transitional phrases, interjections, and colloquialisms. Avoid repetitive phrases and unnatural sentence structures.
- 10. Structuring: Include a Creative Title, SEO Meta-description, ## Introduction ##, bullet points or numbered lists if appropriate, FAQs, and Conclusion.
- 11. Ensure Uniqueness: Guarantee the article is plagiarism-free.
- 12. Punctuation: Use appropriate question marks at the end of questions.
- 13. Pass AI Detection Tools: Create content that easily passes AI plagiarism detection tools.
- 14. Rephrase words like 'video, youtube, channel' with 'article, blog' and such suitable words.
+ 3. Note: Check if the transcript is about programming, then include code examples and snippets in your article.
+ 4. Write Unique Content: Avoid direct copying; rewrite in your own words.
+ 5. REMEMBER to avoid direct quoting and maintain uniqueness.
+ 6. Proofread: Check for grammar, spelling, and punctuation errors.
+ 7. Use Creative and Human-like Style: Incorporate contractions, idioms, transitional phrases, interjections, and colloquialisms. 8. Avoid repetitive phrases and unnatural sentence structures.
+ 9. Ensure Uniqueness: Guarantee the article is plagiarism-free.
+ 10. Punctuation: Use appropriate question marks at the end of questions.
+ 11. Pass AI Detection Tools: Create content that easily passes AI plagiarism detection tools.
+ 12. Rephrase words like 'video, youtube, channel' with 'article, blog' and such suitable words.
- Follow the above guidelines to create a well-optimized, unique, and informative article that will rank well in search engine results and engage readers effectively.
- Craft a blog content from the following transcript:\n{user_content}
+ Follow the above guidelines to create a well-optimized, unique, and informative article,
+ that will rank well in search engine results and engage readers effectively.
+ Follow above guidelines to craft a blog content from the following transcript:\n{user_content}
"""
- try:
- response = openai_chatgpt(prompt)
- return response
- except Exception as err:
- SystemError(f"Error in generating blog summary: {err}")
+ if 'gemini' in gpt_providers:
+ try:
+ genai.configure(api_key=os.getenv('GEMINI_API_KEY'))
+ except Exception as err:
+ logger.error("Failed in getting GEMINI_API_KEY")
+ # Use gemini-pro model for text and image.
+ model = genai.GenerativeModel('gemini-pro')
+ try:
+ response = model.generate_content(prompt)
+ return response.text
+ except Exception as err:
+ logger.error("Failed to get response from gemini.")
+ elif 'openai' in gpt_providers:
+ try:
+ response = openai_chatgpt(prompt)
+ return response
+ except Exception as err:
+ SystemError(f"Error in generating blog summary: {err}")
diff --git a/requirements.txt b/requirements.txt
index e69de29b..f21fcd77 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -0,0 +1,133 @@
+aiofiles
+aiohttp
+aiosignal
+annotated-types
+anyio
+args
+async-timeout
+asyncio
+attrs
+beautifulsoup4
+blinker
+blis
+Brotli
+catalogue
+certifi
+cffi
+charset-normalizer
+chromedriver-autoinstaller
+click
+clint
+cloudpathlib
+colorama
+confection
+cssselect2
+cymem
+dataclasses-json
+decorator
+distro
+docopt
+duckduckgo-search
+exceptiongroup
+fastapi
+Flask
+fonttools
+frozenlist
+greenlet
+grpcio
+grpcio-tools
+h11
+h2
+hpack
+html2image
+html5lib
+httpcore
+httpx
+hyperframe
+idna
+imageio
+imageio-ffmpeg
+itsdangerous
+Jinja2
+joblib
+jsonpatch
+jsonpointer
+langchain
+langchain-core
+langcodes
+langsmith
+loguru
+lxml
+Markdown
+markdown2
+MarkupSafe
+marshmallow
+md2pdf
+moviepy
+multidict
+murmurhash
+mypy-extensions
+nltk
+numpy
+openai
+outcome
+packaging
+param
+permchain
+Pillow
+playwright
+preshed
+proglog
+protobuf
+pycparser
+pydantic
+pydantic_core
+pydub
+pydyf
+pyee
+pyphen
+PySocks
+python-dotenv
+python-multipart
+pytube
+PyYAML
+regex
+requests
+selenium
+serpapi
+six
+smart-open
+sniffio
+socksio
+sortedcontainers
+soupsieve
+spacy-legacy
+spacy-loggers
+SQLAlchemy
+srsly
+stability-sdk
+starlette
+tavily-python
+tenacity
+thinc
+tiktoken
+tinycss2
+tqdm
+trio
+trio-websocket
+typer
+typing-inspect
+typing_extensions
+urllib3
+uvicorn
+wasabi
+weasel
+weasyprint
+webdriver-manager
+webencodings
+websocket-client
+Werkzeug
+wsproto
+yarl
+youtube-transcript-api
+zopfli