kunthawat/ALwrity
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Blogen-V0.1 AI blog writer. Video, Image, Research and write blogs.

Browse Source
This commit is contained in:
AjaySi
2023-11-09 13:22:30 +05:30
parent eb87847124
commit d91688055b
118 changed files with 61599 additions and 3142 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
lib/amzn_affliate_links/README.md Normal file
View File

@@ -0,0 +1 @@
I want you to pretend that you are an E-commerce SEO expert who writes compelling product descriptions for users looking to buy online. I am going to provide the title of one e-commerce product and I want you to come up with a minimum of three distinct content sections for the product description, each section about a unique subset of keywords relating to the product I provide you. Make sure that each of the unique content sections are labeled with an informative and eye-catching subheading describing the main focus of the content section. The main point of these commands is for you to developing a new keyword-rich, informative, and captivating product summary/description that is less than 1000 words. The purpose of product description is marketing the products to users looking to buy. Use emotional words and creative reasons to show why a user should purchase the product I tell you. After you generate the new product summary, please generate a bulleted list of 5 possible H1 headings for this product page, and make each H1 less than 7 words each. Please also include bulleted list of broad match keywords that were used to accomplish writing the product summary. Write a persuasive and professional sounding Meta Title and Description that integrates similar language present in the new product summary text. Make sure to include a numerical aspect in the Meta Title. Do not echo my prompt. Do not remind me what I asked you for. Do not apologize. Do not self-reference. Please use the following products:{Enter descriptive product name - 1 only}

30
lib/amzn_affliate_links/amzn_product_description_writer.py Normal file
View File

@@ -0,0 +1,30 @@
import openai
from dotenv import load_dotenv
load_dotenv()
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY")
def gen_ecomm_product_desc(product_name):
"""Given a product name, generate relevant content for blogging.
"""
product_desc = openai.ChatCompletion.create(
model="gpt-3.5-turbo-16k",
messages=[
{"role": "system",
"content": """Act as an expert E-commerce copywriter specializing in content optimization for SEO. As an E-commerce SEO expert who writes compelling product descriptions for users looking to buy online. I am going to provide the title of one e-commerce product and I want you to come up with a minimum of three distinct content sections for the product description, each section about a unique subset of keywords relating to the product I provide you. Make sure that each of the unique content sections are labeled with an informative and eye-catching subheading describing the main focus of the content section. The main point of these commands is for you to developing a new keyword-rich, informative, and captivating product summary/description that is less than 1000 words. The purpose of product description is marketing the products to users looking to buy. Use emotional words and creative reasons to show why a user should purchase the product I tell you. After you generate the new product summary, please generate a bulleted list of 5 possible H1 headings for this product page, and make each H1 less than 7 words each. Please also include bulleted list of broad match keywords that were used to accomplish writing the product summary. Write a persuasive and professional sounding Meta Title and Description that integrates similar language present in the new product summary text. Make sure to include a numerical aspect in the Meta Title. Do not echo my prompt. Do not remind me what I asked you for. Do not apologize. Do not self-reference."""
},
{
"role": "user",
"content": f"""Craft blog content for following e commerce product.
Product: {product_name}""",
},
],
max_tokens=4096,
temperature=1,
)
if "choices" in product_desc and len(product_desc["choices"]) > 0:
return product_desc["choices"][0]["message"]["content"]
else:
return None

BIN
lib/blog_images/generated_image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.0 MiB

6
lib/blog_proof_reader.py
View File

@@ -1,6 +0,0 @@
"""
I want you act as a proofreader. I will provide you texts and I would like you to review them for any spelling, grammar, or punctuation errors. Once you have finished reviewing the text, provide me with any necessary corrections or suggestions for improve the text.
I want you to act as SEO editor and copywriter. I need you to proofread and analyze the following text and optimize it for the focus keyword. I also need you to correct any grammar mistakes you find in the article. Ask me to provide you with the article text and focus keyword.
"""

181
lib/generate_image_from_prompt.py
View File

@@ -8,27 +8,23 @@
#########################################################
# imports
import sys
import datetime
import openai # OpenAI Python library to make API calls
import requests # used to download images
import os # used to access filepaths
from PIL import Image # used to logger.info and edit images
from loguru import logger
logger.remove()
logger.add(sys.stdout,
colorize=True,
format="<level>{level}</level>|<green>{file}:{line}:{function}</green>| {message}"
)
# set API key
# Taking from env is safer than hardcoding here. But, not all have shell to export.
# Better to take it from a config file and pass it as a parameter.
# variable OPENAI_API_KEY=<API-KEY>
openai.api_key = os.environ.get("OPENAI_API_KEY")
# set a directory to save DALL·E images to
image_dir_name = "blog_images"
image_dir = os.path.join(os.curdir, image_dir_name)
# create the directory if it doesn't yet exist
if not os.path.isdir(image_dir):
os.mkdir(image_dir)
from .gpt_providers.openai_gpt_provider import generate_dalle2_images, generate_dalle3_images, openai_chatgpt
from .stabl_diff_img2html import generate_stable_diffusion_image
def generate_image(logger, num_images=1, img_size="1024x1024", response_format="url"):
def generate_image(user_prompt, image_dir, image_engine="dalle3"):
"""
The generation API endpoint creates an image based on a text prompt.
@@ -36,6 +32,7 @@ def generate_image(logger, num_images=1, img_size="1024x1024", response_format="
prompt (str): A text description of the desired image(s). The maximum length is 1000 characters.
Optional inputs:
--> image_engine: dalle2, dalle3, stable diffusion are supported.
--> num_images (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.
--> size (str): The size of the generated images. Must be one of "256x256", "512x512", or "1024x1024".
Smaller images are faster. Defaults to "1024x1024".
@@ -43,137 +40,37 @@ def generate_image(logger, num_images=1, img_size="1024x1024", response_format="
Must be one of "url" or "b64_json". Defaults to "url".
--> user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse.
"""
# logger.info the directory to save to. TBD: Need to log these.
logger.info(f"Generated blog images will be stored at: {image_dir=}")
# TBD: Ask gpt for prompt for AI generated images as:
img_prompt = generate_img_prompt(user_prompt)
# call the OpenAI API to generate image from prompt.
logger.info(f"Calling openai.image.generate with prompt: {img_prompt}")
if 'dalle2' in image_engine:
image_stored_at = generate_dalle2_images(img_prompt, image_dir)
elif 'dalle3' in image_engine:
image_stored_at = generate_dalle3_images(img_prompt, image_dir)
elif 'stable_diffusion' in image_engine:
image_stored_at = generate_stable_diffusion_image(img_prompt, image_dir)
return image_stored_at
def generate_img_prompt(user_prompt):
"""
Given prompt, this functions generated a prompt for image generation.
"""
# I want you to act as an artist advisor providing advice on various art styles such tips on utilizing
# light & shadow effects effectively in painting, shading techniques while sculpting etc.
# Develop prompts for an AI-generated art piece inspired by [concept], using [symbolism] and [metaphor].
# Provide prompts for an AI-generated art piece inspired by [era] art, incorporating [medium] and [subject matter].
# Develop a set of prompts that could be used to generate AI-generated art focused on the theme of “urban decay.”
# I want you to act as a prompt generator for Science Fiction Art and
# give me five prompts that transport me to a futuristic world.
# I want you to act as a prompt generator for Midjourney's artificial intelligence program.
# Your job is to provide detailed and creative descriptions that will inspire unique and interesting images from the AI.
# Keep in mind that the AI is capable of understanding a wide range of language and can interpret abstract concepts,
# so feel free to be as imaginative and descriptive as possible. For example,
# you could describe a scene from a futuristic city, or a surreal landscape filled with strange creatures.
# The more detailed and imaginative your description, the more interesting the resulting image will be.
# Here is your first prompt: ""
prompt = "An illustration of AI teaching human to speak"
# call the OpenAI API to generate image from prompt.
logger.info(f"Calling openai.image.generate with prompt: {prompt}")
try:
img_generation_response = openai.Image.create(
prompt=prompt,
n=1,
size="1024x1024",
response_format="url",
)
except AttributeError as aerr:
logger.info(f"Failed to generate Image, Try: pip install openai --upgrade in your terminal.Error: {aerr}")
else:
# logger.info response/result. dbg.
print(f"{img_generation_response}")
save_generated_image(logger, img_generation_response)
def save_generated_image(logger, img_generation_response):
"""
"""
# save the image
# We need to change the image name to unique, overwrite and for SEO considerations.
# Note: filetype should be *.png
generated_image_name = "generated_image.png"
generated_image_filepath = os.path.join(image_dir, generated_image_name)
# extract image URL from response
generated_image_url = img_generation_response["data"][0]["url"]
print(f"Extracted URL: {generated_image_url}")
# We use the requests library to fetch the image from URL
response = requests.get(generated_image_url, stream=True)
# We use the Image Class from PIL library to open the image
Image.open(response.raw)
# Download the image.
try:
generated_image = requests.get(generated_image_url).content
except requests.exceptions.RequestException as e:
raise SystemExit(f"Failed to get generted image content: {e}")
else:
with open(generated_image_filepath, "wb") as image_file:
# Write the image to a file and store.
image_file.write(generated_image)
# Optional, dbg.
# logger.info the image
#logger.info(generated_image_filepath)
print("Display the generated image.")
img = Image.open(generated_image_filepath)
img.show()
# Close image window.
#for proc in psutil.process_iter():
# if proc.name() == "Image Viewer":
# proc.kill()
# WIP
# The idea is to download images from other blogs and recreate from it.
# This helps us generate images very close to the topic and also not worry about prompt message.
def gen_new_from_given_img(logger, num_img=1, img_size="1024x1024", response_format="url"):
"""
This function will take an image and produce a variant of it.
Required inputs:
image (str): The image to use as the basis for the variation(s). Must be a valid PNG file, less than 4MB, and square.
Optional inputs:
n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.
size (str): The size of the generated images. Must be one of "256x256", "512x512", or "1024x1024".
Smaller images are faster. Defaults to "1024x1024".
response_format (str): The format in which the generated images are returned. Must be one of "url" or "b64_json". Defaults to "url".
user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse.
"""
img_path = "/home/ajsingh/pseo_experiments_V0.0.1/blog_images/variation_example.png"
try:
png = Image.open(img_path).convert('RGBA')
background = Image.new('RGBA', png.size, (255, 255, 255))
alpha_composite = Image.alpha_composite(background, png)
alpha_composite.save('foo.png', 'PNG', quality=80)
variation_response = openai.Image.create_variation(
image=open('foo.jpg', "rb"),
n=num_img,
size=img_size,
response_format=response_format,
)
except Exception as err:
logger.error(f"An error occured in Image.create_variation::: {err}")
SystemExit(1)
# logger.info response
logger.info(variation_response)
# save the images
variation_urls = [datum["url"] for datum in variation_response["data"]] # extract URLs
variation_images = [requests.get(url).content for url in variation_urls] # download images
variation_image_names = [f"variation_image_{i}.png" for i in range(len(variation_images))] # create names
variation_image_filepaths = [os.path.join(image_dir, name) for name in variation_image_names] # create filepaths
for image, filepath in zip(variation_images, variation_image_filepaths): # loop through the variations
with open(filepath, "wb") as image_file: # open the file
image_file.write(image) # write the image to the file
# logger.info the original image
logger.info(generated_image_filepath)
orig_img = Image.open(generated_image_filepath)
orig_img.show()
# logger.info the new variations
for variation_image_filepaths in variation_image_filepaths:
logger.info(variation_image_filepaths)
var_img = Image.open(variation_image_filepaths)
var_img.show()
logger.info(f"Generate image prompt for : {user_prompt}")
prompt = f"""As an educationist and expert infographic artist, your tasked to create prompts that will be used for image generation.
Craft prompt for Openai Dall-e image generation program. Clearly describe the given text to represent it as image.
Make sure to avoid common image generation mistakes.
Advice for creating prompt for image from the given text(no more than 150 words).
Reply with only one answer and no descrition. Generate image prompt for the below text.
Text: {user_prompt}"""
response = openai_chatgpt(prompt)
return response

514
lib/get_text_response.py
View File

@@ -10,14 +10,93 @@
import json
import os
import datetime #I wish
import sys
import openai
from tqdm import tqdm, trange
import time
import re
from textwrap import dedent
import nltk
nltk.download('punkt')
from nltk.corpus import stopwords
nltk.download('stopwords')
from .gpt_providers.openai_gpt_provider import openai_chatgpt
from .gpt_providers.openai_gpt_provider import openai_chatgpt, gen_new_from_given_img
from .generate_image_from_prompt import generate_image
from .write_blogs_from_youtube_videos import youtube_to_blog
from loguru import logger
logger.remove()
logger.add(sys.stdout,
colorize=True,
format="<level>{level}</level>|<green>{file}:{line}:{function}</green>| {message}"
)
# fixme: Remove the hardcoding, need add another option OR in config ?
image_dir = "pseo_website/assets/"
image_dir = os.path.join(os.getcwd(), image_dir)
# TBD: This can come from config file.
output_path = "pseo_website/_posts/"
output_path = os.path.join(os.getcwd(), output_path)
def generate_youtube_blog(yt_url_list):
"""Takes a list of youtube videos and generates blog for each one of them.
"""
# Use to store the blog in a string, to save in a *.md file.
blog_markdown_str = ""
for a_yt_url in yt_url_list:
try:
yt_img_path, yt_blog = youtube_to_blog(a_yt_url)
# Get the title and meta description of the blog.
title = generate_blog_title(yt_blog)
blog_meta_desc = generate_blog_description(yt_blog)
logger.info(f"Title is {title} and description is {blog_meta_desc}")
#blog_markdown_str = "# " + title.replace('"', '') + "\n\n"
# Generate an introduction for the blog
blog_intro = get_blog_intro(title, yt_blog)
logger.info(f"The Blog intro is:\n {blog_intro}")
blog_markdown_str = blog_markdown_str + "\n\n" + f"{blog_intro}" + "\n\n"
# Generate an image based on meta description
logger.info(f"Calling Image generation with prompt: {blog_meta_desc}")
main_img_path = generate_image(blog_meta_desc, image_dir, "dalle3")
# Get a variation of the yt url screenshot to use in the blog.
#varied_img_path = gen_new_from_given_img(yt_img_path, image_dir)
#logger.info(f"Image path: {main_img_path} and varied path: {varied_img_path}")
#blog_markdown_str = blog_markdown_str + f'![img-description]({os.path.basename(varied_img_path)})' + '_Image Caption_'
stbdiff_img_path = generate_image(yt_img_path, image_dir, "stable_diffusion")
logger.info(f"Image path: {main_img_path} from stable diffusion: {stbdiff_img_path}")
blog_markdown_str = blog_markdown_str + f'![img-description]({os.path.basename(stbdiff_img_path)})' + f'_{title}_'
# Add the body of the blog content.
blog_markdown_str = blog_markdown_str + "\n\n" + f'{yt_blog}' + "\n\n"
# Get the Conclusion of the blog, by passing the generated blog.
blog_conclusion = get_blog_conclusion(blog_markdown_str)
# TBD: Add another image.
blog_markdown_str = blog_markdown_str + "### Conclusion" + "\n\n" + f"{blog_conclusion}" + "\n"
print(f"Conclusion: {blog_markdown_str}")
# Get blog tags and categories.
blog_tags = get_blog_tags(yt_blog)
logger.info(f"Blog tags are: {blog_tags}")
blog_categories = get_blog_categories(yt_blog)
logger.info(f"Blog categories are: {blog_categories}")
save_blog_to_file(blog_markdown_str, title, blog_meta_desc, blog_tags, blog_categories, main_img_path)
#html_blog = convert_markdown_to_html(blog_markdown_str)
#print(html_blog)
except Exception as e:
# raise assertionerror
logger.info(f"Error: Failed to generate_youtube_blog: {e}")
exit(1)
def generate_detailed_blog(num_blogs, blog_keywords, niche, num_subtopics):
@@ -25,22 +104,12 @@ def generate_detailed_blog(num_blogs, blog_keywords, niche, num_subtopics):
This function will take a blog Topic to first generate sections for it
and then generate content for each section.
"""
# I want you to act as a blogger and you want to write a blog post about [topic],
# with a friendly and approachable tone that engages readers.
# Your target audience is [define your target audience].
# Write in a personal style using singular first-person pronouns only.
# I want you to include these keywords: [keyword 1], [keyword 2], [keyword 3] throughout the article.
# Format your response using markdown.
# Use headings, subheadings, bullet points, and bold to organize the information.
# Answer the most commonly asked questions about the topic at the end of the article.
# Create a list of the most popular tools used by the [Field of Interest] professionals with the pros and cons of each tool.
# Use to store the blog in a string, to save in a *.md file.
blog_markdown_str = ""
# TBD: Check if the generated topics are equal to what user asked.
blog_topic_arr = generate_blog_topics(blog_keywords, num_blogs, niche)
print(f"Generated Blog Topics:---- {blog_topic_arr}\n")
logger.info(f"Generated Blog Topics:---- {blog_topic_arr}\n")
# For each of blog topic, generate content.
for a_blog_topic in blog_topic_arr:
@@ -48,13 +117,15 @@ def generate_detailed_blog(num_blogs, blog_keywords, niche, num_subtopics):
blog_markdown_str = "# " + a_blog_topic.replace('"', '') + "\n\n"
# Get the introduction specific to blog title and sub topics.
tpc_outlines = generate_topic_outline(a_blog_topic, num_subtopics)
blog_intro = get_blog_intro(a_blog_topic, tpc_outlines)
print(f"The intro is:\n {blog_intro}")
logger.info(f"The intro is:\n {blog_intro}")
blog_markdown_str = blog_markdown_str + "### Introduction" + "\n\n" + f"{blog_intro}" + "\n\n"
# Now, for each blog we have sub topic. Generate content for each of the sub topic.
for a_outline in tpc_outlines:
sub_topic_content = generate_topic_content(blog_keywords, a_outline)
print(f"Generating content for sub-topic: {a_outline}")
logger.info(f"Generating content for sub-topic: {a_outline}")
# a_outline is sub topic heading, hence part ToC also.
blog_markdown_str = blog_markdown_str + "\n\n" + f"### {a_outline}" + "\n\n"
blog_markdown_str = blog_markdown_str + "\n" + f"\n {sub_topic_content}" + "\n\n"
@@ -64,20 +135,21 @@ def generate_detailed_blog(num_blogs, blog_keywords, niche, num_subtopics):
blog_conclusion = get_blog_conclusion(blog_markdown_str)
blog_markdown_str = blog_markdown_str + "### Conclusion" + "\n" + f"{blog_conclusion}" + "\n"
# print/check the final blog content.
print(f"Final blog content: {blog_markdown_str}")
# logger.info/check the final blog content.
logger.info(f"Final blog content: {blog_markdown_str}")
blog_meta_desc = generate_blog_description(blog_markdown_str)
print(f"\nGet the blog meta description:{blog_meta_desc}")
logger.info(f"\nGet the blog meta description:{blog_meta_desc}")
blog_tags = get_blog_tags(blog_markdown_str)
print(f"\nBlog tags for generated content: {blog_tags}")
logger.info(f"\nBlog tags for generated content: {blog_tags}")
blog_categories = get_blog_categories(blog_markdown_str)
print(f"Generated blog categories: {blog_categories}")
logger.info(f"Generated blog categories: {blog_categories}")
# TBD: Save the blog content as a .md file. Markdown or HTML ?
save_blog_to_file(blog_markdown_str, a_blog_topic, blog_meta_desc, blog_tags, blog_categories)
exit(1)
# Use chatgpt to convert the text into HTML or markdown.
# Now, we need perform some *basic checks on the blog content, such as:
@@ -104,24 +176,36 @@ def generate_blog_topics(blog_keywords, num_blogs, niche):
more_keywords = get_related_keywords(num_blogs, blog_keywords, niche)
prompt = prompt + """Use the following keywords wisely, without keyword stuffing: {more_keywords}"""
print(f"prompt used for blog titles: {prompt}\n")
logger.info(f"prompt used for blog topics: {prompt}\n")
# Calculate the max tokens based on the number of blogs
max_tokens = min(1000, num_blogs * 100)
try:
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.1,
max_tokens=max_tokens,
top_p=1,
n=1
)
topic_list = extract_key_text(response)
return(topic_list)
response = openai_chatgpt(prompt)
return response
except Exception as err:
SystemError(f"Error in generating blog topics: {err}")
def generate_blog_title(blog_meta_desc):
"""
Given a blog title generate an outline for it
"""
# TBD: Remove hardcoding, make dynamic
prompt = f"""As a SEO expert and content writer, I will provide you with blog. Your task is write title for it.
Follows SEO best practises to suggest the blog title.
Please keep the titles concise, not exceeding 60 words, and ensure to maintain their meaning.
Respond with only one title and no description, for this given blog content: {blog_meta_desc}
"""
# The suggested {num_subtopics} outline should include few long-tailed keywords and most popular questions.
# TBD: Include --niche
logger.debug(f"Prompt used for blog title :{prompt}")
try:
response = openai_chatgpt(prompt)
except Exception as err:
SystemError(f"Error in generating Blog Title: {err}")
return response
def generate_topic_outline(blog_title, num_subtopics):
"""
Given a blog title generate an outline for it
@@ -132,20 +216,13 @@ def generate_topic_outline(blog_title, num_subtopics):
"""
# The suggested {num_subtopics} outline should include few long-tailed keywords and most popular questions.
# TBD: Include --niche
print(f"prompt used for blog title Outline :{prompt}")
logger.info(f"\nPrompt used for blog title Outline :{prompt}\n\n")
# TBD: Add logic for which_provider and which_model
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.1,
max_tokens=100,
top_p=0.9,
n=1
)
text_values = []
for choice in response["choices"]:
text_values.extend(choice["text"].split("\n"))
return ([element for element in text_values if element])
try:
response = openai_chatgpt(prompt)
except Exception as err:
SystemError(f"Error in generating Blog Title: {err}")
return response
def generate_topic_content(blog_keywords, sub_topic):
@@ -162,11 +239,10 @@ def generate_topic_content(blog_keywords, sub_topic):
"Format your response using markdown. Use headings, subheadings, bullet points, and bold to organize the information."
)
try:
response = openai_chatgpt(prompt)
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.7,
model="gpt-3.5-turbo",
temperature=0.2,
max_tokens=1000,
top_p=0.9,
n=1
@@ -185,26 +261,18 @@ def get_blog_intro(blog_title, blog_topics):
"""
Generate blog introduction as per title and sub topics
"""
prompt = f"""As a professional writer, craft a captivating, inviting, and concise (no more than 550 characters).
The introduction should compel readers to delve deeper into the blog post,
titled: {blog_title} with the following sub-topics: {blog_topics}
"""
prompt = f"""As a skilled wordsmith, I'll equip you with a blog title and relevant topics, tasking you with crafting an engaging introduction. Your challenge: Create a brief, compelling entry that entices readers to explore the entire post. This introduction must be concise (under 250 characters) yet powerful, clearly stating the blog's purpose and what readers stand to gain.
Intrigue your audience from the start with vibrant language, employing strong verbs and vivid descriptions. Address a common challenge your readers face, demonstrating empathy and positioning yourself as their go-to expert. Pose thought-provoking questions that prompt reader engagement and contemplation.
Remember, your words matter. This introduction serves as the cornerstone of the blog post. It should not only captivate attention but also encourage deeper exploration. Additionally, strategically integrate relevant keywords to enhance visibility on search engine results pages (SERPs). Your mission: Craft an introduction that resonates, leaving readers eager to delve further into the titled piece: '{blog_title}', covering these intriguing sub-topics: {blog_topics}."""
try:
# TBD: Add logic for which_provider and which_model
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.1,
max_tokens=1000,
top_p=0.9,
n=1
)
text_values = []
for choice in response["choices"]:
text_values.extend(choice["text"].split("\n"))
return (' '.join([element for element in text_values if element]))
response = openai_chatgpt(prompt)
except Exception as err:
SystemError(f"Error in generating topic content: {err}")
SystemError(f"Error in generating Blog Introduction: {err}")
return response
def get_blog_conclusion(blog_content):
@@ -214,54 +282,31 @@ def get_blog_conclusion(blog_content):
prompt = f"""As an expert SEO and blog writer, please conclude the given blog providing vital take aways,
summarise key points (no more than 300 characters) in bullet points. The blog content: {blog_content}
"""
print(f"Generating blog conclusion iwth prompt: {prompt}")
logger.info(f"Generating blog conclusion iwth prompt: {prompt}")
try:
# TBD: Add logic for which_provider and which_model
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.1,
max_tokens=450,
top_p=0.7,
n=1
)
text_values = []
for choice in response["choices"]:
text_values.extend(choice["text"].split("\n"))
return (' '.join([element for element in text_values if element]))
response = openai_chatgpt(prompt)
except Exception as err:
SystemError(f"Error in generating blog conclusion: {err}")
else:
return response
def generate_blog_description(blog_content):
"""
Prompt designed to give SEO optimized blog descripton
"""
# Suggest keywords that I should include in my meta description for my blog post on [topic]
# I want to generate high CTR meta and keyword rich meta title and meta descriptions in text format.
# My keywords are – [keyword 1], [keyword 2], [keyword 3]
# Suggest a meta description for the content above, make it user-friendly
# and with a call to action, include the keyword [keyword].
prompt = f"""As an expert SEO and blog writer, write meta description for given blog content.
The description should be between 150 and 160 characters long, uses strong, active verbs,
avoids using all caps or excessive punctuation, and is relevant to the blog content.
It should be engaging and encourages users to click on the link.
The blog content: {blog_content}"""
prompt = f"""As an expert SEO and blog writer, Compose a compelling meta description for the given blog content,
adhering to SEO best practices. Keep it between 150-160 characters, incorporating active verbs,
avoiding all caps and excessive punctuation. Ensure relevance, engage users, and encourage clicks.
Use keywords naturally and provide a glimpse of the content's value to entice readers.
Respond with only one of your best effort and do not include your explanations.
Blog Content: {blog_content}"""
try:
# TBD: Add logic for which_provider and which_model
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.1,
max_tokens=450,
top_p=0.7,
n=1
)
text_values = []
for choice in response["choices"]:
text_values.extend(choice["text"].split("\n"))
return (' '.join([element for element in text_values if element]))
response = openai_chatgpt(prompt)
return response
except Exception as err:
SystemError(f"Error in generating blog description: {err}")
@@ -271,92 +316,87 @@ def get_blog_tags(blog_article):
Function to suggest tags for the given blog content
"""
# Suggest at least 5 tags for the following blog post [Enter your blog post text here].
prompt = f"""As an expert SEO and blog writer, suggest 3 to 5 relevant, specific,
and popular tags that are unique and consistent to improve the visibility
and discoverability of following blog content: {blog_article}"
"""
prompt = f"""As an expert SEO and blog writer, suggest only 2 relevant and specific blog tags
for the given blog content. Only reply with comma separated values.
Blog content: {blog_article}."""
try:
# TBD: Add logic for which_provider and which_model
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.1,
max_tokens=450,
top_p=0.7,
n=1
)
text_values = []
for choice in response["choices"]:
text_values.extend(choice["text"].split("\n"))
return (' '.join([element for element in text_values if element]))
response = openai_chatgpt(prompt)
except Exception as err:
SystemError(f"Error in generating blog tags: {err}")
else:
return response
def get_blog_categories(blog_article):
"""
Function to generate blog categories for given blog content.
"""
prompt = f"""As an expert SEO and content writer, Suggest 2-3 blog categories by identifying
the main topic, most relevant categories, considering the target
audience and the blog's category taxonomy for the following blog content: {blog_article}"
prompt = f"""As an expert SEO and content writer, I will provide you with blog content.
Suggest only 2 blog categories which are most relevant to provided blog content,
by identifying the main topic. Also consider the target audience and the
blog's category taxonomy. Only reply with comma separated values. The blog content is: {blog_article}"
"""
try:
# TBD: Add logic for which_provider and which_model
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.1,
max_tokens=100,
top_p=0.7,
n=1
)
text_values = []
for choice in response["choices"]:
text_values.extend(choice["text"].split("\n"))
return (' '.join([element for element in text_values if element]))
response = openai_chatgpt(prompt)
except Exception as err:
SystemError(f"Error in generating blog categories: {err}")
else:
return response
def save_blog_to_file(blog_content, blog_title,
blog_meta_desc, blog_tags, blog_categories, file_type="md"
):
blog_meta_desc, blog_tags, blog_categories, main_img_path, file_type="md"):
""" Common function to save the generated blog to a file.
arg: file_type can be md or html
"""
# TBD: This can come from config file.
output_path = "pseo_website/_posts/"
output_path = os.path.join(os.getcwd(), output_path)
# Convert the spaces in blog_title with dash
logger.info(f"The blog will be saved at: {output_path}")
logger.debug(f"Blog Title is: {blog_title}")
blog_title_md = blog_title
regex = re.compile('[^a-zA-Z0-9- ]')
blog_title = regex.sub('', blog_title)
blog_title = re.sub('--+', '-', blog_title)
blog_title = blog_title.replace(' ', '-')
blog_title_md = regex.sub('', blog_title_md)
blog_title= blog_title.replace(":", "")
blog_title_md = re.sub('--+', '-', blog_title_md)
blog_title_md = blog_title_md.replace(' ', '-')
blog_title_md = remove_stop_words(blog_title_md)
if ':' in blog_meta_desc:
blog_meta_desc = blog_meta_desc.split(':')[1].strip()
if not os.path.exists(output_path):
# If the directory does not exist, create it
#os.makedirs(output_path)
print("Error: Blog output directory is set to {output_path}, which Does Not Exist.")
logger.error("Error: Blog output directory is set to {output_path}, which Does Not Exist.")
# Different output formats are plaintext, html and markdown.
if file_type in "md":
logger.info(f"Writing/Saving the resultant blog content in Markdown format.")
# fill the Front Matter as below at the top of the post: https://jekyllrb.com/docs/front-matter/
# date: YYYY-MM-DD HH:MM:SS +/-TTTT
formatted_date = f"{datetime.datetime.now():%Y-%m-%d %H:%M:%S %z}"
blog_frontmatter = """---
title: {blog_title}
date: {formatted_date}
categories: [{blog_categories}]
tags: [{blog_tags}]
description: {blog_meta_desc}
img_path: '/posts/20180809'
---\n\n"""
from zoneinfo import ZoneInfo
tz=ZoneInfo('Asia/Kolkata')
dtobj = datetime.datetime.now(tz=ZoneInfo('Asia/Kolkata'))
formatted_date = f"{dtobj.strftime('%Y-%m-%d %H:%M:%S %z')}"
blog_frontmatter = f"""\
---
title: {blog_title}
date: {formatted_date}
categories: [{blog_categories}]
tags: [{blog_tags}]
description: {blog_meta_desc}
img_path: '/assets/'
image:
path: {os.path.basename(main_img_path)}
alt: {blog_title}
---\n\n"""
# Create a new file named YYYY-MM-DD-TITLE.EXTENSION and put it in the _posts of the root directory.
# Please note that the EXTENSION must be one of md or markdown
blog_output_path = os.path.join(
output_path,
f"{datetime.date.today().strftime('%Y-%m-%d')}-{blog_title}.md"
f"{datetime.date.today().strftime('%Y-%m-%d')}-{blog_title_md}.md"
)
# Save the generated blog content to a file.
try:
@@ -365,44 +405,7 @@ def save_blog_to_file(blog_content, blog_title,
f.write(blog_content)
except Exception as e:
raise Exception(f"Failed to write blog content: {e}")
print(f"\nSuccessfully saved and Posted blog at: {blog_output_path,}\n")
def extract_key_text(json_data):
"""Extracts key text from a given JSON object.
Args:json_data: A JSON object.
Returns: A list of strings containing the key text.
Raises: ValueError: If the JSON object is not valid.
"""
try:
# Extract the "choices" key from the JSON object.
choices = json_data["choices"]
# Iterate over the "choices" list and extract the "text" key from each item.
key_text = []
for choice in choices:
text = choice["text"]
# Split the text into a list of sentences.
sentences = text.split("\n")
# Iterate over the list of sentences and extract the first sentence.
for sentence in sentences:
# The generated topics usually have 1) or ^\W*\D* . Remove them from prompt.
new_str = sentence.replace("'", '')
new_str = re.sub(r'^(\d*\.)', '', new_str)
key_text.append(new_str)
# Remove duplicate key text.
key_text = list(set(key_text))
# Remove empty values.
key_text = [i for i in key_text if i]
return key_text
except KeyError as e:
raise ValueError(f"Missing key in JSON object: {e.args[0]}")
except TypeError as e:
raise ValueError(f"Invalid JSON object: {e.args[0]}")
logger.info(f"\nSuccessfully saved and Posted blog at: {blog_output_path,}\n")
def get_related_keywords(num_blogs, keywords, niche):
@@ -422,37 +425,104 @@ def get_related_keywords(num_blogs, keywords, niche):
)
try:
# TBD: Add logic for which_provider and which_model
response = openai_chatgpt(
prompt,
model="text-davinci-003",
temperature=0.7,
max_tokens=100,
top_p=0.9,
n=10
)
# Extract the keywords from the response
keywords = []
for choice in response.choices:
# Split the response into words
words = choice.text.split(" ")
# Add the words to the list of keywords
for text in words:
# Remove digits
text = re.sub(r'\d', '', text)
# Remove special characters
text = re.sub(r'[^\w\s]', '', text)
# Remove newline characters
text = text.replace('\n', '')
keywords.append(text)
# Remove any duplicate keywords
keywords = set(keywords)
# Return the list of keywords
return (' '.join(keywords))
response = openai_chatgpt(prompt)
return response
except Exception as err:
SystemError(f"Error in getting related keywords.")
def convert_markdown_to_html(md_content):
""" Helper function to convert given text to HTML
"""
html_response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-16k",
messages=[
{"role": "system", "content": """
Convert Markdown to HTML:
You are a skilled developer tasked with converting a Markdown-formatted text to HTML. You will be given text in markdown format. Follow these steps to perform the conversion:
1. Parse User's Markdown Input: You will receive a Markdown-formatted text as input from the user. Carefully analyze the provided Markdown text, paying attention to different elements such as headings (#), lists (unordered and ordered), bold and italic text, links, images, and code blocks.
2. Generate and Validate HTML: Generate corresponding HTML code for each Markdown element following the conversion guidelines below. Ensure the generated HTML is well-structured and syntactically correct.
3. Preserve Line Breaks: Markdown line breaks (soft breaks) represented by two spaces at the end of a line should be converted to <br> tags in HTML to preserve the line breaks.
4. REMEMBER to generate complete, valid HTML response only.
Follow below Conversion Guidelines:
- Headers: Convert Markdown headers (#, ##, ###, etc.) to corresponding HTML header tags (<h1>, <h2>, <h3>, etc.).
- Lists: Convert unordered lists (*) and ordered lists (1., 2., 3., etc.) to <ul> and <ol> HTML tags, respectively. List items should be enclosed in <li> tags.
- Emphasis: Convert bold (**) and italic (*) text to <strong> and <em> HTML tags, respectively.
- Links: Convert Markdown links ([text](url)) to HTML anchor (<a>) tags. Ensure the href attribute contains the correct URL.
- Images: Convert Markdown image tags (![alt text](image_url)) to HTML image (<img>) tags. Include the alt attribute for accessibility.
- Code: Convert inline code (`code`) to <code> HTML tags. Convert code blocks (```) to <pre> HTML tags for preserving formatting.
- Blockquotes: Convert blockquotes (>) to <blockquote> HTML tags.
"""
},
{"role": "user", "content": f"Convert the following Markdown text to HTML:\n\n{md_content}"}
],
max_tokens=8192,
temperature=1,
n=1,
stream=True
)
for chunk in response:
print(chunk)
logger.info("Finished converting markdown to html.")
if "choices" in html_response and len(html_response["choices"]) > 0:
return html_response["choices"][0]["message"]["content"]
else:
return None
# Helper function
def remove_stop_words(sentence):
# Tokenize the sentence into words
words = nltk.word_tokenize(sentence)
# Get the list of English stop words
stop_words = set(stopwords.words('english'))
# Remove stop words from the sentence
filtered_words = [word for word in words if word.lower() not in stop_words]
# Join the filtered words back into a sentence
filtered_sentence = ' '.join(filtered_words)
return filtered_sentence
def convert_markdown_to_html(md_content):
""" Helper function to convert given text to HTML
"""
html_response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-16k",
messages=[
{"role": "system", "content": """
Convert Markdown to HTML:
You are a skilled developer tasked with converting a Markdown-formatted text to HTML. You will be given text in markdown format. Follow these steps to perform the conversion:
1. Parse User's Markdown Input: You will receive a Markdown-formatted text as input from the user. Carefully analyze the provided Markdown text, paying attention to different elements such as headings (#), lists (unordered and ordered), bold and italic text, links, images, and code blocks.
2. Generate and Validate HTML: Generate corresponding HTML code for each Markdown element following the conversion guidelines below. Ensure the generated HTML is well-structured and syntactically correct.
3. Preserve Line Breaks: Markdown line breaks (soft breaks) represented by two spaces at the end of a line should be converted to <br> tags in HTML to preserve the line breaks.
4. REMEMBER to generate complete, valid HTML response only.
Follow below Conversion Guidelines:
- Headers: Convert Markdown headers (#, ##, ###, etc.) to corresponding HTML header tags (<h1>, <h2>, <h3>, etc.).
- Lists: Convert unordered lists (*) and ordered lists (1., 2., 3., etc.) to <ul> and <ol> HTML tags, respectively. List items should be enclosed in <li> tags.
- Emphasis: Convert bold (**) and italic (*) text to <strong> and <em> HTML tags, respectively.
- Links: Convert Markdown links ([text](url)) to HTML anchor (<a>) tags. Ensure the href attribute contains the correct URL.
- Images: Convert Markdown image tags (![alt text](image_url)) to HTML image (<img>) tags. Include the alt attribute for accessibility.
- Code: Convert inline code (`code`) to <code> HTML tags. Convert code blocks (```) to <pre> HTML tags for preserving formatting.
- Blockquotes: Convert blockquotes (>) to <blockquote> HTML tags.
"""
},
{"role": "user", "content": f"Convert the following Markdown text to HTML:\n\n{md_content}"}
],
max_tokens=8192,
temperature=1,
n=1,
)
logger.info("Finished converting markdown to html.")
if "choices" in html_response and len(html_response["choices"]) > 0:
return html_response["choices"][0]["message"]["content"]
else:
return None

1
lib/gpt-researcher Submodule

Submodule lib/gpt-researcher added at 6ada6e23d0

0
lib/gpt_providers/__init__.py Normal file
View File

276
lib/gpt_providers/openai_gpt_provider.py
View File

@@ -5,26 +5,46 @@
#
########################################################
import os
import sys
from tqdm import tqdm, trange
import openai
import time # I wish
import openai
from openai import OpenAI
from pytube import YouTube
import tempfile
from html2image import Html2Image
import datetime
from PIL import Image
import requests
from loguru import logger
logger.remove()
logger.add(sys.stdout,
colorize=True,
format="<level>{level}</level>|<green>{file}:{line}:{function}</green>| {message}"
)
def openai_chatgpt(prompt, model="text-davinci-003", temperature=0.5, max_tokens=2048, top_p=0.9, n=10):
def openai_chatgpt(prompt, model="gpt-3.5-turbo-16k", temperature=0.2, max_tokens=8192, top_p=0.9, n=1):
"""
Wrapper function for openai chat Completion
"""
# Error in generating topic content: Rate limit reached for default-global-with-image-limits
# in free account on requests per min. Limit: 3 / min. Please try again in 20s.
for i in trange(10):
time.sleep(1)
try:
# Error in generating topic content: Rate limit reached for default-global-with-image-limits
# in free account on requests per min. Limit: 3 / min. Please try again in 20s.
for i in trange(21):
time.sleep(1)
# using OpenAI's Completion module that helps execute
# any tasks involving text
response = openai.Completion.create(
# model name used here is text-davinci-003
# there are many other models available under the
# umbrella of GPT-3
model="text-davinci-003",
client = OpenAI()
# using OpenAI's Completion module that helps execute any tasks involving text
response = client.chat.completions.create(
# model name used, there are many other models available under the umbrella of GPT-3
model=model,
# passing the user input
prompt=prompt,
messages=[{"role": "user", "content": prompt}],
# generated output can have "max_tokens" number of tokens
max_tokens=max_tokens,
# number of outputs generated in one call
@@ -32,26 +52,210 @@ def openai_chatgpt(prompt, model="text-davinci-003", temperature=0.5, max_tokens
top_p=top_p,
#frequency_penalty=0,
#presence_penalty=0
)
except openai.APIError as e:
#Handle API error here, e.g. retry or log
SystemError(f"OpenAI API returned an API Error: {e}")
except openai.APIConnectionError as e:
#Handle connection error here
SystemError(f"Failed to connect to OpenAI API: {e}")
except openai.RateLimitError as e:
#Handle rate limit error (we recommend using exponential backoff)
SystemError(f"OpenAI API request exceeded rate limit: {e}")
return response.choices[0].message.content
def openai_chatgpt_streaming_text(user_prompt):
"""
Function to use stream=True for real time output from openai
"""
client = OpenAI()
response = client.chat.completions.create(
model="gpt-3.5-turbo-16k",
messages=[{"role": "user", "content": f"{user_prompt}"}],
max_tokens = 8192,
temperature = 0.9,
n=1,
stream=True
)
return(response)
except openai.error.Timeout as e:
#Handle timeout error, e.g. retry or log
SystemError(f"OpenAI API request timed out: {e}")
except openai.error.APIError as e:
#Handle API error, e.g. retry or log
SystemError(f"OpenAI API returned an API Error: {e}")
except openai.error.APIConnectionError as e:
#Handle connection error, e.g. check network or log
SystemError(f"OpenAI API request failed to connect: {e}")
except openai.error.InvalidRequestError as e:
#Handle invalid request error, e.g. validate parameters or log
SystemError(f"OpenAI API request was invalid: {e}")
except openai.error.AuthenticationError as e:
#Handle authentication error, e.g. check credentials or log
SystemError(f"OpenAI API request was not authorized: {e}")
except openai.error.PermissionError as e:
#Handle permission error, e.g. check scope or log
SystemError(f"OpenAI API request was not permitted: {e}")
except openai.error.RateLimitError as e:
#Handle rate limit error, e.g. wait or log
SystemError(f"OpenAI API request exceeded rate limit: {e}")
# Create variables to collect the stream of events, iterate through the stream of events
collected_events = []
completion_text = ''
print("\n\n.....COME ONE...\n\n")
for chunk in response:
collected_events.append(chunk) # save the event response
event_text = chunk.choices[0].delta.content # extract the text
completion_text += event_text # append the text
sys.stdout.write(chunk.choices[0].delta.content)
sys.stdout.flush()
print(f"COMLETION: {completion_text}")
return completion_text
def generate_dalle2_images(user_prompt, image_dir, num_images=1, img_size="512x512", response_format="url"):
"""
The generation API endpoint creates an image based on a text prompt.
Required inputs:
prompt (str): A text description of the desired image(s). The maximum length is 1000 characters.
Optional inputs:
--> num_images (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.
--> size (str): The size of the generated images. Must be one of "256x256", "512x512", or "1024x1024".
Smaller images are faster. Defaults to "1024x1024".
-->response_format (str): The format in which the generated images are returned.
Must be one of "url" or "b64_json". Defaults to "url".
--> user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse.
"""
logger.info(f"Generated Dall-e-2 blog images will be stored at: {image_dir=}")
try:
client = OpenAI()
img_generation_response = client.images.generate(
model="dall-e-2",
prompt=user_prompt,
n=num_images,
size=img_size
)
except openai.OpenAIError as e:
logger.error(f"Dalle-2 image generate error: {e.http_status}")
logger.error(f"{e.error}")
except Exception as aerr:
logger.info(f"Failed to generate Image with Dalle2, Error: {aerr}")
else:
img_path = save_generated_image(img_generation_response, image_dir)
return img_path
def generate_dalle3_images(img_prompt, image_dir, size="1024x1024", quality="hd", n=1):
""" Function to create images using Dalle3 """
client = OpenAI()
logger.info("Generating Dall-e-3 image for the blog.")
try:
img_generation_response = client.images.generate(
model="dall-e-3",
prompt=f"{img_prompt}",
size=size,
quality=quality,
n=1,
)
except openai.OpenAIError as e:
logger.error(f"Dalle-3 image generate error: {e.http_status}")
logger.error(f"{e.error}")
except Exception as e:
SystemError("Failed to Generate images with Dalle3.")
else:
#image_url = response.data[0].url
img_path = save_generated_image(img_generation_response, image_dir)
return img_path
def speech_to_text(video_url):
""" Common openai function for speech to text. """
client = OpenAI()
try:
# Download YouTube video
logger.info(f"Download YouTube video: {video_url}")
yt = YouTube(video_url)
stream = yt.streams.filter(only_audio=True).first()
# Save the video in a temporary file
logger.info(f"Finished Downloading, Saving video for transcription.")
with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as temp_file:
temp_file_name = temp_file.name
stream.download(output_path=os.path.dirname(temp_file_name), filename=os.path.basename(temp_file_name))
try:
# Transcribe the video using OpenAI's Whisper API
logger.info(f"Transcribe the video using OpenAI's Whisper API")
with open(temp_file_name, "rb") as audio_file:
transcript = client.audio.transcriptions.create(
model="whisper-1",
file=audio_file
)
except Exception as err:
logger.error(f"Failed to transcribe using whisper model: {err}")
logger.info("Finished Transcribing. Creating a blog from the transcript.")
# Remove the temporary file after transcription
os.remove(temp_file_name)
return(transcript)
except Exception as e:
logger.error(f"Error: speech-to-text, Failed to transcribe url: {video_url} with error: {e}")
# The idea is to download images from other blogs and recreate from it.
# This helps us generate images very close to the topic and also not worry about prompt message.
def gen_new_from_given_img(img_path, image_dir, num_img=1, img_size="1024x1024", response_format="url"):
"""
This function will take an image and produce a variant of it.
Required inputs:
image (str): The image to use as the basis for the variation(s). Must be a valid PNG file, less than 4MB, and square.
Optional inputs:
n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.
size (str): The size of the generated images. Must be one of "256x256", "512x512", or "1024x1024".
Smaller images are faster. Defaults to "1024x1024".
response_format (str): The format in which the generated images are returned. Must be one of "url" or "b64_json". Defaults to "url".
user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse.
"""
logger.info(f"Generating a variation of the image at: {img_path}")
try:
client = OpenAI()
png = Image.open(img_path).convert('RGBA')
background = Image.new('RGBA', png.size, (255, 255, 255))
alpha_composite = Image.alpha_composite(background, png)
alpha_composite.save(img_path, 'PNG', quality=80)
variation_response = client.images.create_variation(
image=open(img_path, "rb"),
n=num_img,
size=img_size,
response_format=response_format,
)
except Exception as err:
logger.error(f"An error occured in Image.create_variation::: {err}")
SystemExit(1)
try:
img_path = save_generated_image(variation_response, image_dir)
except Exception as err:
logger.error(f"An error in Saving Image.create_variation::: {err}")
SystemExit(1)
else:
return img_path
def save_generated_image(img_generation_response, image_dir):
"""
Common util function to save the generated images for blog.
"""
# save the image
# We need to change the image name to unique, overwrite and for SEO considerations.
# Note: filetype should be *.png
generated_image_name = f"generated_image_{datetime.datetime.now():%Y-%m-%d-%H-%M-%S}.png"
generated_image_filepath = os.path.join(image_dir, generated_image_name)
# extract image URL from response
generated_image_url = img_generation_response.data[0].url
# We use the requests library to fetch the image from URL
logger.info(f"Fetch the image from url: {generated_image_url}")
response = requests.get(generated_image_url, stream=True)
# We use the Image Class from PIL library to open the image
Image.open(response.raw)
# Download the image.
try:
generated_image = requests.get(generated_image_url).content
except requests.exceptions.RequestException as e:
raise SystemExit(f"Failed to get generted image content: {e}")
else:
logger.info(f"Saving image at path: {generated_image_filepath}")
with open(generated_image_filepath, "wb") as image_file:
# Write the image to a file and store.
image_file.write(generated_image)
#logger.info(generated_image_filepath)
logger.info("Display the generated image.")
img = Image.open(generated_image_filepath)
img.show()
return generated_image_filepath

65
lib/stabl_diff_img2html.py Normal file
View File

@@ -0,0 +1,65 @@
import os
import io
import warnings
from PIL import Image
from stability_sdk import client
import stability_sdk.interfaces.gooseai.generation.generation_pb2 as generation
# Our Host URL should not be prepended with "https" nor should it have a trailing slash.
os.environ['STABILITY_HOST'] = 'grpc.stability.ai:443'
# Sign up for an account at the following link to get an API Key.
# https://platform.stability.ai/
# Click on the following link once you have created an account to be taken to your API Key.
# https://platform.stability.ai/account/keys
# Paste your API Key below.
os.environ['STABILITY_KEY'] = 'sk-KGCeQFf4iQYogzAe6WEISIOij12g4Ztvnkw92dJTJZ7vsL0j'
def generate_stable_diffusion_image(prompt):
# Set up our connection to the API.
# Check out the following link for a list of available engines:
# https://platform.stability.ai/docs/features/api-parameters#engine
stability_api = client.StabilityInference(
key=os.environ['STABILITY_KEY'], # API Key reference.
verbose=True, # Print debug messages.
engine="stable-diffusion-xl-1024-v1-0", # Set the engine to use for generation.
)
# Set up our initial generation parameters.
answers = stability_api.generate(
prompt=prompt,
seed=4253978046, # If a seed is provided, the resulting generated image will be deterministic.
# What this means is that as long as all generation parameters remain the same,
# you can always recall the same image simply by generating it again.
# Note: This isn't quite the case for Clip Guided generations, which we'll tackle in a future example notebook.
steps=50, # Amount of inference steps performed on image generation. Defaults to 30.
cfg_scale=7.0,
# Influences how strongly your generation is guided to match your prompt.
# Setting this value higher increases the strength in which it tries to match your prompt.
# Defaults to 7.0 if not specified.
width=1024, # Generation width, defaults to 512 if not included.
height=1024, # Generation height, defaults to 512 if not included.
samples=1, # Number of images to generate, defaults to 1 if not included.
sampler=generation.SAMPLER_K_DPMPP_2M
# Choose which sampler we want to denoise our generation with.
# Defaults to k_dpmpp_2m if not specified. Clip Guidance only supports ancestral samplers.
# (Available Samplers: ddim, plms, k_euler, k_euler_ancestral, k_heun, k_dpm_2, k_dpm_2_ancestral,
# k_dpmpp_2s_ancestral, k_lms, k_dpmpp_2m, k_dpmpp_sde)
)
# Set up our warning to print to the console if the adult content classifier is tripped.
# If adult content classifier is not tripped, save generated images.
for resp in answers:
for artifact in resp.artifacts:
if artifact.finish_reason == generation.FILTER:
warnings.warn(
"Your request activated the API's safety filters and could not be processed."
"Please modify the prompt and try again.")
if artifact.type == generation.ARTIFACT_IMAGE:
img = Image.open(io.BytesIO(artifact.binary))
img_name = image_dir + str(artifact.seed) + ".png"
img.show()
img.save(img_name)
# Save our generated images with their seed number as the filename.

70
lib/write_blogs_from_youtube_videos.py Normal file
View File

@@ -0,0 +1,70 @@
import os
import time
import sys
from pytube import YouTube
import tempfile
import openai
from html2image import Html2Image
from tqdm import tqdm, trange
from loguru import logger
logger.remove()
logger.add(sys.stdout,
colorize=True,
format="<level>{level}</level>|<green>{file}:{line}:{function}</green>| {message}"
)
from .gpt_providers.openai_gpt_provider import openai_chatgpt, openai_chatgpt_streaming_text, speech_to_text
def youtube_to_blog(video_url):
"""Function to transcribe a given youtube url """
# fixme: Doesnt work all types of yt urls.
vid_id = video_url.split("=")[1]
hti = Html2Image(output_path="../blog_images")
hti.screenshot(url=video_url, save_as=f"yt-img-{vid_id}.png")
yt_img_path = os.path.join("../blog_images", f"yt-img-{vid_id}.png")
try:
audio_text = speech_to_text(video_url)
audio_blog_content = summarize_youtube_video(audio_text)
return(yt_img_path, audio_blog_content)
except Exception as e:
logger.error(f"Error: Failed to transcribe YouTube video_url: {video_url} with error: {e}")
def summarize_youtube_video(user_content):
"""Generates a summary of a YouTube video using OpenAI GPT-3 and displays a progress bar.
Args:
video_link: The URL of the YouTube video to summarize.
Returns:
A string containing the summary of the video.
"""
prompt = f"""
You are an expert copywriter specializing in content optimization for SEO.
Your task is to transform a given transcript into a well-structured and engaging blog article. Your objectives include:
1. Master the Transcript: Understand main ideas, key points, and the core message.
2. Sentence Structure: Rephrase while preserving logical flow and coherence. Dont quote anyone from video.
3. Identify Main Keywords: Determine the primary video topic.
4. Keyword Integration: Naturally integrate keywords in headings, subheadings, and body text, avoiding overuse.
5. Write Unique Content: Avoid direct copying; rewrite in your own words.
REMEMBER to avoid direct quoting and maintain uniqueness.
6. Optimize for SEO: Implement meta tags, header tags, and appropriate keyword density.
7. Craft Engaging and Informative Article: Provide value and insight to readers.
8. Proofread: Check for grammar, spelling, and punctuation errors.
9. Use Creative and Human-like Style: Incorporate contractions, idioms, transitional phrases, interjections, and colloquialisms. Avoid repetitive phrases and unnatural sentence structures.
10. Structuring: Include a Creative Title, SEO Meta-description, ## Introduction ##, bullet points or numbered lists if appropriate, FAQs, and Conclusion.
11. Ensure Uniqueness: Guarantee the article is plagiarism-free.
12. Punctuation: Use appropriate question marks at the end of questions.
13. Pass AI Detection Tools: Create content that easily passes AI plagiarism detection tools.
14. Rephrase words like 'video, youtube, channel' with 'article, blog' and such suitable words.
Follow the above guidelines to create a well-optimized, unique, and informative article that will rank well in search engine results and engage readers effectively.
Craft a blog content from the following transcript:\n{user_content}
"""
#completion_text = openai_chatgpt_streaming_text(prompt)
completion_text = openai_chatgpt(prompt)
return completion_text