Compare commits
154 Commits
codex/add-
...
codex/remo
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f210310177 | ||
|
|
196ea65af9 | ||
|
|
bcf62017aa | ||
|
|
0732887c09 | ||
|
|
e704aa7d87 | ||
|
|
79f26c815b | ||
|
|
e2726805f3 | ||
|
|
ff61708e29 | ||
|
|
63767d72b3 | ||
|
|
d85a1ee561 | ||
|
|
18bed36e2b | ||
|
|
24d932d2b5 | ||
|
|
cd53680523 | ||
|
|
edf3f32b3c | ||
|
|
e59c77b221 | ||
|
|
1a456b21b7 | ||
|
|
813f9acc34 | ||
|
|
60b6b0904b | ||
|
|
80838ed028 | ||
|
|
e66311ea44 | ||
|
|
cf2d3a51e8 | ||
|
|
8dd1c13f85 | ||
|
|
ad97dc0d3b | ||
|
|
45231625fd | ||
|
|
23bf709c10 | ||
|
|
3f1d5cbb09 | ||
|
|
12960a22ea | ||
|
|
45d2b0b693 | ||
|
|
348839be36 | ||
|
|
b5ab46a749 | ||
|
|
d12fe6348e | ||
|
|
0e3a611e57 | ||
|
|
b24d39349d | ||
|
|
0d0d964605 | ||
|
|
03d43fb54b | ||
|
|
c361bd127d | ||
|
|
6ac880e61e | ||
|
|
92a27270aa | ||
|
|
cc03567d2f | ||
|
|
3c79073a10 | ||
|
|
71c0e2ed46 | ||
|
|
11663b0142 | ||
|
|
4ca58084fd | ||
|
|
6c99b26140 | ||
|
|
13e25cec3b | ||
|
|
724832c688 | ||
|
|
917be873df | ||
|
|
429689bdcb | ||
|
|
6cf5d0396d | ||
|
|
27147d50a5 | ||
|
|
2b025673d6 | ||
|
|
3f3575cc18 | ||
|
|
c0a5f5fdeb | ||
|
|
1f139e3167 | ||
|
|
1bdf0d4b93 | ||
|
|
f1e8cdb0d8 | ||
|
|
0680bf98a2 | ||
|
|
cc2443cf5b | ||
|
|
6cef24289f | ||
|
|
f6795100ac | ||
|
|
aa2317c359 | ||
|
|
bba56a1940 | ||
|
|
0f34048c6a | ||
|
|
1cf3ae96ce | ||
|
|
a697b869ab | ||
|
|
9e3867ca61 | ||
|
|
b567a32136 | ||
|
|
88deabb9fc | ||
|
|
f30f6c5346 | ||
|
|
2ab4471632 | ||
|
|
a43c229809 | ||
|
|
0e8953b538 | ||
|
|
6579f60d7d | ||
|
|
08f08a1a52 | ||
|
|
ab78a6a158 | ||
|
|
22c31e6c77 | ||
|
|
249a1962d4 | ||
|
|
dcb7d28e03 | ||
|
|
26e1f08ebb | ||
|
|
fcf00cd20d | ||
|
|
b8ffda1cbb | ||
|
|
6d5ae8d2fa | ||
|
|
c5e2fc3514 | ||
|
|
a3e4f5231a | ||
|
|
a8c80c5b75 | ||
|
|
027638dfb9 | ||
|
|
4fbbe9c8b4 | ||
|
|
3f2d9104d9 | ||
|
|
d34dc651b1 | ||
|
|
0d2d9b220e | ||
|
|
92ac410707 | ||
|
|
63bb937796 | ||
|
|
c52b1eabc9 | ||
|
|
746a5eeeb9 | ||
|
|
d06ab77e60 | ||
|
|
f737b24b49 | ||
|
|
4c206293b1 | ||
|
|
35fd700b22 | ||
|
|
49e0ee8e9e | ||
|
|
edd92ec85b | ||
|
|
cd06c6aaa8 | ||
|
|
9f0298725a | ||
|
|
971b4362c5 | ||
|
|
5ad0f13482 | ||
|
|
7f626d47b4 | ||
|
|
92bcd27004 | ||
|
|
bf6cdf1109 | ||
|
|
08e51f76fa | ||
|
|
dee4387b0b | ||
|
|
c7013a71df | ||
|
|
5ac1b9439d | ||
|
|
bf980ab89b | ||
|
|
45aefd0590 | ||
|
|
f53b53a543 | ||
|
|
d28daca2e1 | ||
|
|
2c3fe33c75 | ||
|
|
dd1e398fa2 | ||
|
|
dfccf53d18 | ||
|
|
9d04ffb63a | ||
|
|
004506cf9a | ||
|
|
11966cf341 | ||
|
|
a0efdb5001 | ||
|
|
8b8730ae9f | ||
|
|
66faff9051 | ||
|
|
f0b78f5cbe | ||
|
|
43c6ceab2f | ||
|
|
92bbe1d878 | ||
|
|
636989f75b | ||
|
|
5706b85a4e | ||
|
|
3a92c4af1a | ||
|
|
2a41e94c07 | ||
|
|
27c167ebe8 | ||
|
|
e3ba7893ca | ||
|
|
b54c2978c3 | ||
|
|
92cbd682a5 | ||
|
|
6555a722d3 | ||
|
|
cbcb896d24 | ||
|
|
ef7874dcdc | ||
|
|
e64aea484f | ||
|
|
8828e982f8 | ||
|
|
bbb46ca9d1 | ||
|
|
d1ff406d03 | ||
|
|
643e9ad2f3 | ||
|
|
cadcb8077d | ||
|
|
2b11814fb8 | ||
|
|
5965e123b9 | ||
|
|
b93a4d2a67 | ||
|
|
c652c0d149 | ||
|
|
d13cce7a46 | ||
|
|
6596a0515a | ||
|
|
4d948e0222 | ||
|
|
e8e2a7fea0 | ||
|
|
ec9d2f922e | ||
|
|
af5a6e0ee3 |
23
.github/workflows/lint-forced-user-id.yml
vendored
Normal file
23
.github/workflows/lint-forced-user-id.yml
vendored
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
name: Lint Forced User ID Patterns
|
||||||
|
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
lint-forced-user-id:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.11"
|
||||||
|
|
||||||
|
- name: Check for forced/hardcoded user_id patterns
|
||||||
|
run: python backend/scripts/check_forced_user_id_patterns.py
|
||||||
10
.gitignore
vendored
10
.gitignore
vendored
@@ -4,15 +4,23 @@ __pycache__/
|
|||||||
*.db
|
*.db
|
||||||
*.sqlite*
|
*.sqlite*
|
||||||
|
|
||||||
|
nul
|
||||||
|
LICENSE
|
||||||
|
CHANGELOG.md
|
||||||
|
|
||||||
.trae/
|
.trae/
|
||||||
.trae
|
.trae
|
||||||
|
|
||||||
workspace/
|
workspace/
|
||||||
workspace/*
|
workspace/*
|
||||||
|
|
||||||
|
.windsurf
|
||||||
|
artifacts
|
||||||
|
|
||||||
.opencode
|
.opencode
|
||||||
|
|
||||||
data/
|
data/
|
||||||
|
data/*
|
||||||
|
|
||||||
.trae/
|
.trae/
|
||||||
/backend/database/migrations/*
|
/backend/database/migrations/*
|
||||||
@@ -21,7 +29,7 @@ backend/*.db
|
|||||||
backend\youtube_audio
|
backend\youtube_audio
|
||||||
youtube_avatars
|
youtube_avatars
|
||||||
backend\youtube_images
|
backend\youtube_images
|
||||||
|
data/media/podcast_videos/AI_Videos
|
||||||
backend/.trae_*
|
backend/.trae_*
|
||||||
|
|
||||||
# Onboarding progress files
|
# Onboarding progress files
|
||||||
|
|||||||
13
Procfile
Normal file
13
Procfile
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
web: cd backend && ALWRITY_ENABLED_FEATURES=podcast python -c "
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
# Ensure podcast mode
|
||||||
|
os.environ.setdefault('ALWRITY_ENABLED_FEATURES', 'podcast')
|
||||||
|
# Set HOST/PORT for Render
|
||||||
|
port = os.getenv('PORT', '10000')
|
||||||
|
host = os.getenv('HOST', '0.0.0.0')
|
||||||
|
print(f'[STARTUP] Starting uvicorn on {host}:{port}', flush=True)
|
||||||
|
sys.stdout.flush()
|
||||||
|
import uvicorn
|
||||||
|
uvicorn.run('app:app', host=host, port=int(port), reload=False)
|
||||||
|
"
|
||||||
14
README.md
Normal file
14
README.md
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
# Render CLI
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
- [Homebrew](https://render.com/docs/cli#homebrew-macos-linux)
|
||||||
|
- [Direct Download](https://render.com/docs/cli#direct-download)
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
Documentation is hosted at https://render.com/docs/cli.
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
To create a new command, use the `cmd/template.go` template file as a starting point. Reference the [CLI Style Guide](docs/STYLE.md) to learn more about command naming, flags, arguments, and help text conventions.
|
||||||
672
_session_backup/App.tsx
Normal file
672
_session_backup/App.tsx
Normal file
@@ -0,0 +1,672 @@
|
|||||||
|
import React from 'react';
|
||||||
|
import { BrowserRouter as Router, Routes, Route, Navigate, useLocation } from 'react-router-dom';
|
||||||
|
import { Box, CircularProgress, Typography } from '@mui/material';
|
||||||
|
import { CopilotKit } from "@copilotkit/react-core";
|
||||||
|
import { ClerkProvider, useAuth } from '@clerk/clerk-react';
|
||||||
|
import "@copilotkit/react-ui/styles.css";
|
||||||
|
import Wizard from './components/OnboardingWizard/Wizard';
|
||||||
|
import MainDashboard from './components/MainDashboard/MainDashboard';
|
||||||
|
import SEODashboard from './components/SEODashboard/SEODashboard';
|
||||||
|
import ContentPlanningDashboard from './components/ContentPlanningDashboard/ContentPlanningDashboard';
|
||||||
|
import FacebookWriter from './components/FacebookWriter/FacebookWriter';
|
||||||
|
import LinkedInWriter from './components/LinkedInWriter/LinkedInWriter';
|
||||||
|
import BlogWriter from './components/BlogWriter/BlogWriter';
|
||||||
|
import StoryWriter from './components/StoryWriter/StoryWriter';
|
||||||
|
import { StoryProjectList } from './components/StoryWriter/StoryProjectList';
|
||||||
|
import YouTubeCreator from './components/YouTubeCreator/YouTubeCreator';
|
||||||
|
import { CreateStudio, EditStudio, UpscaleStudio, ControlStudio, SocialOptimizer, AssetLibrary, ImageStudioDashboard, FaceSwapStudio, CompressionStudio, ImageProcessingStudio } from './components/ImageStudio';
|
||||||
|
import {
|
||||||
|
VideoStudioDashboard,
|
||||||
|
CreateVideo,
|
||||||
|
AvatarVideo,
|
||||||
|
EnhanceVideo,
|
||||||
|
ExtendVideo,
|
||||||
|
EditVideo,
|
||||||
|
TransformVideo,
|
||||||
|
SocialVideo,
|
||||||
|
FaceSwap,
|
||||||
|
VideoTranslate,
|
||||||
|
VideoBackgroundRemover,
|
||||||
|
AddAudioToVideo,
|
||||||
|
LibraryVideo,
|
||||||
|
} from './components/VideoStudio';
|
||||||
|
import {
|
||||||
|
ProductMarketingDashboard,
|
||||||
|
ProductPhotoshootStudio,
|
||||||
|
ProductAnimationStudio,
|
||||||
|
ProductVideoStudio,
|
||||||
|
ProductAvatarStudio,
|
||||||
|
} from './components/ProductMarketing';
|
||||||
|
import PodcastDashboard from './components/PodcastMaker/PodcastDashboard';
|
||||||
|
import PricingPage from './components/Pricing/PricingPage';
|
||||||
|
import WixTestPage from './components/WixTestPage/WixTestPage';
|
||||||
|
import WixCallbackPage from './components/WixCallbackPage/WixCallbackPage';
|
||||||
|
import WordPressCallbackPage from './components/WordPressCallbackPage/WordPressCallbackPage';
|
||||||
|
import BingCallbackPage from './components/BingCallbackPage/BingCallbackPage';
|
||||||
|
import BingAnalyticsStorage from './components/BingAnalyticsStorage/BingAnalyticsStorage';
|
||||||
|
import ResearchDashboard from './pages/ResearchDashboard';
|
||||||
|
import IntentResearchTest from './pages/IntentResearchTest';
|
||||||
|
import SchedulerDashboard from './pages/SchedulerDashboard';
|
||||||
|
import BillingPage from './pages/BillingPage';
|
||||||
|
import ApprovalsPage from './pages/ApprovalsPage';
|
||||||
|
import TeamActivityPage from './pages/TeamActivityPage';
|
||||||
|
import StripeDisputesDashboard from './pages/StripeDisputesDashboard';
|
||||||
|
import ProtectedRoute from './components/shared/ProtectedRoute';
|
||||||
|
import GSCAuthCallback from './components/SEODashboard/components/GSCAuthCallback';
|
||||||
|
import Landing from './components/Landing/Landing';
|
||||||
|
import ErrorBoundary from './components/shared/ErrorBoundary';
|
||||||
|
import ErrorBoundaryTest from './components/shared/ErrorBoundaryTest';
|
||||||
|
import CopilotKitDegradedBanner from './components/shared/CopilotKitDegradedBanner';
|
||||||
|
import { OnboardingProvider } from './contexts/OnboardingContext';
|
||||||
|
import { SubscriptionProvider, useSubscription } from './contexts/SubscriptionContext';
|
||||||
|
import { CopilotKitHealthProvider } from './contexts/CopilotKitHealthContext';
|
||||||
|
import { useOAuthTokenAlerts } from './hooks/useOAuthTokenAlerts';
|
||||||
|
|
||||||
|
import { setAuthTokenGetter, setClerkSignOut } from './api/client';
|
||||||
|
import { setMediaAuthTokenGetter } from './utils/fetchMediaBlobUrl';
|
||||||
|
import { setBillingAuthTokenGetter } from './services/billingService';
|
||||||
|
import { useOnboarding } from './contexts/OnboardingContext';
|
||||||
|
import { useState, useEffect } from 'react';
|
||||||
|
import ConnectionErrorPage from './components/shared/ConnectionErrorPage';
|
||||||
|
import { isPodcastOnlyDemoMode } from './utils/demoMode';
|
||||||
|
|
||||||
|
// interface OnboardingStatus {
|
||||||
|
// onboarding_required: boolean;
|
||||||
|
// onboarding_complete: boolean;
|
||||||
|
// current_step?: number;
|
||||||
|
// total_steps?: number;
|
||||||
|
// completion_percentage?: number;
|
||||||
|
// }
|
||||||
|
|
||||||
|
// Conditional CopilotKit wrapper that only shows sidebar on content-planning route
|
||||||
|
const ConditionalCopilotKit: React.FC<{ children: React.ReactNode }> = ({ children }) => {
|
||||||
|
// Do not render CopilotSidebar here. Let specific pages/components control it.
|
||||||
|
return <>{children}</>;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Wrapper to only enable CopilotKit checks/provider when user is authenticated
|
||||||
|
// This prevents CopilotKit from running on the Landing page
|
||||||
|
const AuthenticatedCopilotWrapper: React.FC<{
|
||||||
|
children: React.ReactNode;
|
||||||
|
apiKey: string;
|
||||||
|
}> = ({ children, apiKey }) => {
|
||||||
|
const { isSignedIn } = useAuth();
|
||||||
|
const location = useLocation();
|
||||||
|
|
||||||
|
// Exclude CopilotKit from running on:
|
||||||
|
// 1. Landing page (handled by !isSignedIn)
|
||||||
|
// 2. Onboarding pages (to prevent health check timeouts)
|
||||||
|
// 3. Podcast-only demo mode (CopilotKit not needed)
|
||||||
|
const isPodcastOnly = isPodcastOnlyDemoMode();
|
||||||
|
const shouldExcludeCopilot = !isSignedIn || location.pathname.startsWith('/onboarding') || isPodcastOnly;
|
||||||
|
|
||||||
|
if (shouldExcludeCopilot) {
|
||||||
|
return <>{children}</>;
|
||||||
|
}
|
||||||
|
|
||||||
|
const hasKey = apiKey && apiKey.trim();
|
||||||
|
|
||||||
|
if (hasKey) {
|
||||||
|
// Enhanced error handler that updates health context
|
||||||
|
const handleCopilotKitError = (e: any) => {
|
||||||
|
console.error("CopilotKit Error:", e);
|
||||||
|
|
||||||
|
// Try to get health context if available
|
||||||
|
// We'll use a custom event to notify health context since we can't access it directly here
|
||||||
|
const errorMessage = e?.error?.message || e?.message || 'CopilotKit error occurred';
|
||||||
|
const errorType = errorMessage.toLowerCase();
|
||||||
|
|
||||||
|
// Differentiate between fatal and transient errors
|
||||||
|
const isFatalError =
|
||||||
|
errorType.includes('cors') ||
|
||||||
|
errorType.includes('ssl') ||
|
||||||
|
errorType.includes('certificate') ||
|
||||||
|
errorType.includes('403') ||
|
||||||
|
errorType.includes('forbidden') ||
|
||||||
|
errorType.includes('ERR_CERT_COMMON_NAME_INVALID');
|
||||||
|
|
||||||
|
// Dispatch event for health context to listen to
|
||||||
|
window.dispatchEvent(new CustomEvent('copilotkit-error', {
|
||||||
|
detail: {
|
||||||
|
error: e,
|
||||||
|
errorMessage,
|
||||||
|
isFatal: isFatalError,
|
||||||
|
}
|
||||||
|
}));
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<CopilotKitHealthProvider initialHealthStatus={true}>
|
||||||
|
<CopilotKitDegradedBanner />
|
||||||
|
<ErrorBoundary
|
||||||
|
context="CopilotKit"
|
||||||
|
showDetails={process.env.NODE_ENV === 'development'}
|
||||||
|
fallback={
|
||||||
|
<Box sx={{ p: 3, textAlign: 'center' }}>
|
||||||
|
<Typography variant="h6" color="warning" gutterBottom>
|
||||||
|
Chat Unavailable
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" color="textSecondary">
|
||||||
|
CopilotKit encountered an error. The app continues to work with manual controls.
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<CopilotKit
|
||||||
|
publicApiKey={apiKey}
|
||||||
|
showDevConsole={false}
|
||||||
|
onError={handleCopilotKitError}
|
||||||
|
>
|
||||||
|
{children}
|
||||||
|
</CopilotKit>
|
||||||
|
</ErrorBoundary>
|
||||||
|
</CopilotKitHealthProvider>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<CopilotKitHealthProvider initialHealthStatus={false}>
|
||||||
|
<CopilotKitDegradedBanner />
|
||||||
|
{children}
|
||||||
|
</CopilotKitHealthProvider>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
// Component to handle initial routing based on subscription and onboarding status
|
||||||
|
// Flow: Subscription → Onboarding → Dashboard
|
||||||
|
const InitialRouteHandler: React.FC = () => {
|
||||||
|
const { loading, error, isOnboardingComplete, initializeOnboarding, data } = useOnboarding();
|
||||||
|
const { subscription, loading: subscriptionLoading, checkSubscription } = useSubscription();
|
||||||
|
const [connectionError, setConnectionError] = useState<{
|
||||||
|
hasError: boolean;
|
||||||
|
error: Error | null;
|
||||||
|
}>({
|
||||||
|
hasError: false,
|
||||||
|
error: null,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Poll for OAuth token alerts and show toast notifications
|
||||||
|
// Only enabled when user is authenticated (has subscription)
|
||||||
|
useOAuthTokenAlerts({
|
||||||
|
enabled: subscription?.active === true,
|
||||||
|
interval: 60000, // Poll every 1 minute
|
||||||
|
});
|
||||||
|
|
||||||
|
// Check subscription on mount (non-blocking - don't wait for it to route)
|
||||||
|
useEffect(() => {
|
||||||
|
// Delay subscription check slightly to allow auth token getter to be installed first
|
||||||
|
const timeoutId = setTimeout(async () => {
|
||||||
|
// Retry logic for initial subscription check
|
||||||
|
const maxRetries = 3;
|
||||||
|
for (let attempt = 0; attempt < maxRetries; attempt++) {
|
||||||
|
try {
|
||||||
|
await checkSubscription();
|
||||||
|
break; // Success
|
||||||
|
} catch (err) {
|
||||||
|
console.error(`App: Subscription check attempt ${attempt + 1} failed:`, err);
|
||||||
|
|
||||||
|
// If it's a connection error and we have retries left, wait and retry
|
||||||
|
const isConnectionError = err instanceof Error && (err.name === 'NetworkError' || err.name === 'ConnectionError');
|
||||||
|
|
||||||
|
if (isConnectionError && attempt < maxRetries - 1) {
|
||||||
|
const delay = 1000 * Math.pow(2, attempt); // 1s, 2s
|
||||||
|
await new Promise(resolve => setTimeout(resolve, delay));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// If final attempt or not a connection error, handle it
|
||||||
|
if (attempt === maxRetries - 1 || !isConnectionError) {
|
||||||
|
if (isConnectionError) {
|
||||||
|
setConnectionError({
|
||||||
|
hasError: true,
|
||||||
|
error: err as Error,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Don't block routing on other errors
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, 100); // Small delay to ensure TokenInstaller has run
|
||||||
|
|
||||||
|
return () => clearTimeout(timeoutId);
|
||||||
|
}, []); // Remove checkSubscription dependency to prevent loop
|
||||||
|
|
||||||
|
// Initialize onboarding only after subscription is confirmed
|
||||||
|
useEffect(() => {
|
||||||
|
if (subscription && !subscriptionLoading) {
|
||||||
|
// Check if user is new (no subscription record at all)
|
||||||
|
const isNewUser = !subscription || subscription.plan === 'none';
|
||||||
|
|
||||||
|
console.log('InitialRouteHandler: Subscription data received:', {
|
||||||
|
plan: subscription.plan,
|
||||||
|
active: subscription.active,
|
||||||
|
isNewUser,
|
||||||
|
subscriptionLoading
|
||||||
|
});
|
||||||
|
|
||||||
|
if (subscription.active && !isNewUser) {
|
||||||
|
console.log('InitialRouteHandler: Subscription confirmed, initializing onboarding...');
|
||||||
|
initializeOnboarding();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, [subscription, subscriptionLoading, initializeOnboarding]);
|
||||||
|
|
||||||
|
// Handle connection error - show connection error page
|
||||||
|
if (connectionError.hasError) {
|
||||||
|
const handleRetry = () => {
|
||||||
|
setConnectionError({
|
||||||
|
hasError: false,
|
||||||
|
error: null,
|
||||||
|
});
|
||||||
|
// Re-trigger the subscription check using context
|
||||||
|
checkSubscription().catch((err) => {
|
||||||
|
if (err instanceof Error && (err.name === 'NetworkError' || err.name === 'ConnectionError')) {
|
||||||
|
setConnectionError({
|
||||||
|
hasError: true,
|
||||||
|
error: err,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleGoHome = () => {
|
||||||
|
window.location.href = '/';
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<ConnectionErrorPage
|
||||||
|
onRetry={handleRetry}
|
||||||
|
onGoHome={handleGoHome}
|
||||||
|
message={connectionError.error?.message || "Backend service is not available. Please check if the server is running."}
|
||||||
|
title="Connection Error"
|
||||||
|
/>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Loading state - only wait for onboarding init, not subscription check
|
||||||
|
// Subscription check is non-blocking and happens in background
|
||||||
|
const waitingForOnboardingInit = loading || !data;
|
||||||
|
if (loading || waitingForOnboardingInit) {
|
||||||
|
return (
|
||||||
|
<Box
|
||||||
|
display="flex"
|
||||||
|
flexDirection="column"
|
||||||
|
alignItems="center"
|
||||||
|
justifyContent="center"
|
||||||
|
minHeight="100vh"
|
||||||
|
gap={2}
|
||||||
|
>
|
||||||
|
<CircularProgress size={60} />
|
||||||
|
<Typography variant="h6" color="textSecondary">
|
||||||
|
{subscriptionLoading ? 'Checking subscription...' : 'Preparing your workspace...'}
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Error state
|
||||||
|
if (error) {
|
||||||
|
return (
|
||||||
|
<Box
|
||||||
|
display="flex"
|
||||||
|
flexDirection="column"
|
||||||
|
alignItems="center"
|
||||||
|
justifyContent="center"
|
||||||
|
minHeight="100vh"
|
||||||
|
gap={2}
|
||||||
|
p={3}
|
||||||
|
>
|
||||||
|
<Typography variant="h5" color="error" gutterBottom>
|
||||||
|
Error
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body1" color="textSecondary" textAlign="center">
|
||||||
|
{error}
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Decision tree for SIGNED-IN users:
|
||||||
|
// Priority: Subscription → Onboarding → Dashboard (as per user flow: Landing → Subscription → Onboarding → Dashboard)
|
||||||
|
|
||||||
|
// 1. If subscription is still loading, show loading state
|
||||||
|
if (subscriptionLoading) {
|
||||||
|
return (
|
||||||
|
<Box
|
||||||
|
display="flex"
|
||||||
|
flexDirection="column"
|
||||||
|
alignItems="center"
|
||||||
|
justifyContent="center"
|
||||||
|
minHeight="100vh"
|
||||||
|
gap={2}
|
||||||
|
>
|
||||||
|
<CircularProgress size={60} />
|
||||||
|
<Typography variant="h6" color="textSecondary">
|
||||||
|
Checking subscription...
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. No subscription data yet - handle gracefully
|
||||||
|
// If onboarding is complete, allow access to dashboard (user already went through flow)
|
||||||
|
// If onboarding not complete, check if subscription check is still loading or failed
|
||||||
|
if (!subscription) {
|
||||||
|
if (isOnboardingComplete) {
|
||||||
|
console.log('InitialRouteHandler: Onboarding complete but no subscription data → Dashboard (allow access)');
|
||||||
|
return <Navigate to="/dashboard" replace />;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Onboarding not complete and no subscription data
|
||||||
|
// If subscription check is still loading, show loading state
|
||||||
|
if (subscriptionLoading) {
|
||||||
|
return (
|
||||||
|
<Box
|
||||||
|
display="flex"
|
||||||
|
flexDirection="column"
|
||||||
|
alignItems="center"
|
||||||
|
justifyContent="center"
|
||||||
|
minHeight="100vh"
|
||||||
|
gap={2}
|
||||||
|
>
|
||||||
|
<CircularProgress size={60} />
|
||||||
|
<Typography variant="h6" color="textSecondary">
|
||||||
|
Checking subscription...
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Subscription check completed but returned null/undefined
|
||||||
|
// This likely means no subscription - redirect to pricing
|
||||||
|
console.log('InitialRouteHandler: No subscription data after check → Pricing page');
|
||||||
|
return <Navigate to="/pricing" replace />;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Check subscription status first
|
||||||
|
const isNewUser = !subscription || subscription.plan === 'none';
|
||||||
|
|
||||||
|
// No active subscription → Show modal (SubscriptionContext handles this)
|
||||||
|
// Don't redirect immediately - let the modal show first
|
||||||
|
// User can click "Renew Subscription" button in modal to go to pricing
|
||||||
|
// Or click "Maybe Later" to dismiss (but they still can't use features)
|
||||||
|
if (isNewUser || !subscription.active) {
|
||||||
|
console.log('InitialRouteHandler: No active subscription - modal will be shown by SubscriptionContext');
|
||||||
|
// Note: SubscriptionContext will show the modal automatically when subscription is inactive
|
||||||
|
// We still redirect to pricing for new users, but allow existing users with expired subscriptions
|
||||||
|
// to see the modal first. The modal has a "Renew Subscription" button that navigates to pricing.
|
||||||
|
// For new users (no subscription at all), redirect to pricing immediately
|
||||||
|
if (isNewUser) {
|
||||||
|
console.log('InitialRouteHandler: New user (no subscription) → Pricing page');
|
||||||
|
return <Navigate to="/pricing" replace />;
|
||||||
|
}
|
||||||
|
// For existing users with inactive subscription, show modal but don't redirect immediately
|
||||||
|
// The modal will be shown by SubscriptionContext, and user can click "Renew Subscription"
|
||||||
|
// Allow access to dashboard (modal will be shown and block functionality)
|
||||||
|
console.log('InitialRouteHandler: Inactive subscription - allowing access to show modal');
|
||||||
|
// Continue to onboarding/dashboard flow - modal will be shown by SubscriptionContext
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Has active subscription, check onboarding status
|
||||||
|
if (!isOnboardingComplete) {
|
||||||
|
console.log('InitialRouteHandler: Subscription active but onboarding incomplete → Onboarding');
|
||||||
|
return <Navigate to="/onboarding" replace />;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 5. Has subscription AND completed onboarding → Dashboard
|
||||||
|
console.log('InitialRouteHandler: All set (subscription + onboarding) → Dashboard');
|
||||||
|
return <Navigate to="/dashboard" replace />;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Root route that chooses Landing (signed out) or InitialRouteHandler (signed in)
|
||||||
|
const RootRoute: React.FC = () => {
|
||||||
|
const { isSignedIn } = useAuth();
|
||||||
|
if (isSignedIn) {
|
||||||
|
return <InitialRouteHandler />;
|
||||||
|
}
|
||||||
|
return <Landing />;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Installs Clerk auth token getter into axios clients and stores user_id
|
||||||
|
// Must render under ClerkProvider
|
||||||
|
const TokenInstaller: React.FC = () => {
|
||||||
|
const { getToken, userId, isSignedIn, signOut } = useAuth();
|
||||||
|
|
||||||
|
// Store user_id in localStorage when user signs in
|
||||||
|
useEffect(() => {
|
||||||
|
if (isSignedIn && userId) {
|
||||||
|
console.log('TokenInstaller: Storing user_id in localStorage:', userId);
|
||||||
|
localStorage.setItem('user_id', userId);
|
||||||
|
|
||||||
|
// Trigger event to notify SubscriptionContext that user is authenticated
|
||||||
|
window.dispatchEvent(new CustomEvent('user-authenticated', { detail: { userId } }));
|
||||||
|
} else if (!isSignedIn) {
|
||||||
|
// Clear user_id when signed out
|
||||||
|
console.log('TokenInstaller: Clearing user_id from localStorage');
|
||||||
|
localStorage.removeItem('user_id');
|
||||||
|
}
|
||||||
|
}, [isSignedIn, userId]);
|
||||||
|
|
||||||
|
// Install token getter for API calls
|
||||||
|
useEffect(() => {
|
||||||
|
const tokenGetter = async () => {
|
||||||
|
try {
|
||||||
|
const template = process.env.REACT_APP_CLERK_JWT_TEMPLATE;
|
||||||
|
// If a template is provided and it's not a placeholder, request a template-specific JWT
|
||||||
|
if (template && template !== 'your_jwt_template_name_here') {
|
||||||
|
// @ts-ignore Clerk types allow options object
|
||||||
|
return await getToken({ template });
|
||||||
|
}
|
||||||
|
return await getToken();
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// Set token getter for main API client
|
||||||
|
setAuthTokenGetter(tokenGetter);
|
||||||
|
|
||||||
|
// Set token getter for billing API client (same function)
|
||||||
|
setBillingAuthTokenGetter(tokenGetter);
|
||||||
|
|
||||||
|
// Set token getter for media blob URL fetcher (for authenticated image/video requests)
|
||||||
|
setMediaAuthTokenGetter(tokenGetter);
|
||||||
|
}, [getToken]);
|
||||||
|
|
||||||
|
// Install Clerk signOut function for handling expired tokens
|
||||||
|
useEffect(() => {
|
||||||
|
if (signOut) {
|
||||||
|
setClerkSignOut(async () => {
|
||||||
|
await signOut();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}, [signOut]);
|
||||||
|
|
||||||
|
return null;
|
||||||
|
};
|
||||||
|
|
||||||
|
const App: React.FC = () => {
|
||||||
|
// React Hooks MUST be at the top before any conditionals
|
||||||
|
const [loading, setLoading] = useState(true);
|
||||||
|
|
||||||
|
// Get CopilotKit key from localStorage or .env
|
||||||
|
const [copilotApiKey, setCopilotApiKey] = useState(() => {
|
||||||
|
const savedKey = localStorage.getItem('copilotkit_api_key');
|
||||||
|
const envKey = process.env.REACT_APP_COPILOTKIT_API_KEY || '';
|
||||||
|
const key = (savedKey || envKey).trim();
|
||||||
|
|
||||||
|
// Validate key format if present
|
||||||
|
if (key && !key.startsWith('ck_pub_')) {
|
||||||
|
console.warn('CopilotKit API key format invalid - must start with ck_pub_');
|
||||||
|
}
|
||||||
|
|
||||||
|
return key;
|
||||||
|
});
|
||||||
|
|
||||||
|
// Initialize app - loading state will be managed by InitialRouteHandler
|
||||||
|
useEffect(() => {
|
||||||
|
// Remove manual health check - connection errors are handled by ErrorBoundary
|
||||||
|
setLoading(false);
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
// Listen for CopilotKit key updates
|
||||||
|
useEffect(() => {
|
||||||
|
const handleKeyUpdate = (event: CustomEvent) => {
|
||||||
|
const newKey = event.detail?.apiKey;
|
||||||
|
if (newKey) {
|
||||||
|
console.log('App: CopilotKit key updated, reloading...');
|
||||||
|
setCopilotApiKey(newKey);
|
||||||
|
setTimeout(() => window.location.reload(), 500);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
window.addEventListener('copilotkit-key-updated', handleKeyUpdate as EventListener);
|
||||||
|
return () => window.removeEventListener('copilotkit-key-updated', handleKeyUpdate as EventListener);
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
// Token installer must be inside ClerkProvider; see TokenInstaller below
|
||||||
|
|
||||||
|
if (loading) {
|
||||||
|
return (
|
||||||
|
<Box
|
||||||
|
display="flex"
|
||||||
|
flexDirection="column"
|
||||||
|
alignItems="center"
|
||||||
|
justifyContent="center"
|
||||||
|
minHeight="100vh"
|
||||||
|
gap={2}
|
||||||
|
>
|
||||||
|
<CircularProgress size={60} />
|
||||||
|
<Typography variant="h6" color="textSecondary">
|
||||||
|
Connecting to ALwrity...
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// Get environment variables with fallbacks
|
||||||
|
const clerkPublishableKey = process.env.REACT_APP_CLERK_PUBLISHABLE_KEY || '';
|
||||||
|
const clerkJSUrl = process.env.REACT_APP_CLERK_JS_URL;
|
||||||
|
|
||||||
|
// Show error if required keys are missing
|
||||||
|
if (!clerkPublishableKey) {
|
||||||
|
return (
|
||||||
|
<Box sx={{ p: 3, textAlign: 'center' }}>
|
||||||
|
<Typography color="error" variant="h6">
|
||||||
|
Missing Clerk Publishable Key
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ mt: 1 }}>
|
||||||
|
Please add REACT_APP_CLERK_PUBLISHABLE_KEY to your .env file
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Render app with or without CopilotKit based on whether we have a key
|
||||||
|
const renderApp = () => {
|
||||||
|
return (
|
||||||
|
<Router>
|
||||||
|
<AuthenticatedCopilotWrapper apiKey={copilotApiKey}>
|
||||||
|
<ConditionalCopilotKit>
|
||||||
|
<TokenInstaller />
|
||||||
|
<Routes>
|
||||||
|
<Route path="/" element={<RootRoute />} />
|
||||||
|
<Route
|
||||||
|
path="/onboarding"
|
||||||
|
element={
|
||||||
|
<ErrorBoundary context="Onboarding Wizard" showDetails>
|
||||||
|
<Wizard />
|
||||||
|
</ErrorBoundary>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
{/* Error Boundary Testing - Development Only */}
|
||||||
|
{process.env.NODE_ENV === 'development' && (
|
||||||
|
<Route path="/error-test" element={<ErrorBoundaryTest />} />
|
||||||
|
)}
|
||||||
|
<Route path="/dashboard" element={<ProtectedRoute><MainDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/seo" element={<ProtectedRoute><SEODashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/seo-dashboard" element={<ProtectedRoute><SEODashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/content-planning" element={<ProtectedRoute><ContentPlanningDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/facebook-writer" element={<ProtectedRoute><FacebookWriter /></ProtectedRoute>} />
|
||||||
|
<Route path="/linkedin-writer" element={<ProtectedRoute><LinkedInWriter /></ProtectedRoute>} />
|
||||||
|
<Route path="/blog-writer" element={<ProtectedRoute><BlogWriter /></ProtectedRoute>} />
|
||||||
|
<Route path="/story-writer" element={<ProtectedRoute><StoryWriter /></ProtectedRoute>} />
|
||||||
|
<Route path="/story-projects" element={<ProtectedRoute><StoryProjectList /></ProtectedRoute>} />
|
||||||
|
<Route path="/youtube-creator" element={<ProtectedRoute><YouTubeCreator /></ProtectedRoute>} />
|
||||||
|
<Route path="/podcast-maker" element={<ProtectedRoute><PodcastDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-studio" element={<ProtectedRoute><ImageStudioDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio" element={<ProtectedRoute><VideoStudioDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/create" element={<ProtectedRoute><CreateVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/avatar" element={<ProtectedRoute><AvatarVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/enhance" element={<ProtectedRoute><EnhanceVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/extend" element={<ProtectedRoute><ExtendVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/edit" element={<ProtectedRoute><EditVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/transform" element={<ProtectedRoute><TransformVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/social" element={<ProtectedRoute><SocialVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/face-swap" element={<ProtectedRoute><FaceSwap /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/video-translate" element={<ProtectedRoute><VideoTranslate /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/video-background-remover" element={<ProtectedRoute><VideoBackgroundRemover /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/add-audio-to-video" element={<ProtectedRoute><AddAudioToVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/video-studio/library" element={<ProtectedRoute><LibraryVideo /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-generator" element={<ProtectedRoute><CreateStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-editor" element={<ProtectedRoute><EditStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-upscale" element={<ProtectedRoute><UpscaleStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-control" element={<ProtectedRoute><ControlStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-studio/face-swap" element={<ProtectedRoute><FaceSwapStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-studio/compress" element={<ProtectedRoute><CompressionStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-studio/processing" element={<ProtectedRoute><ImageProcessingStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/image-studio/social-optimizer" element={<ProtectedRoute><SocialOptimizer /></ProtectedRoute>} />
|
||||||
|
<Route path="/asset-library" element={<ProtectedRoute><AssetLibrary /></ProtectedRoute>} />
|
||||||
|
<Route path="/campaign-creator" element={<ProtectedRoute><ProductMarketingDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/campaign-creator/photoshoot" element={<ProtectedRoute><ProductPhotoshootStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/campaign-creator/animation" element={<ProtectedRoute><ProductAnimationStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/campaign-creator/video" element={<ProtectedRoute><ProductVideoStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/campaign-creator/avatar" element={<ProtectedRoute><ProductAvatarStudio /></ProtectedRoute>} />
|
||||||
|
<Route path="/product-marketing" element={<Navigate to="/campaign-creator" replace />} />
|
||||||
|
<Route path="/scheduler-dashboard" element={<ProtectedRoute><SchedulerDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/billing" element={<ProtectedRoute><BillingPage /></ProtectedRoute>} />
|
||||||
|
<Route path="/approvals" element={<ProtectedRoute><ApprovalsPage /></ProtectedRoute>} />
|
||||||
|
<Route path="/team-activity" element={<ProtectedRoute><TeamActivityPage /></ProtectedRoute>} />
|
||||||
|
<Route path="/stripe-disputes" element={<ProtectedRoute><StripeDisputesDashboard /></ProtectedRoute>} />
|
||||||
|
<Route path="/pricing" element={<PricingPage />} />
|
||||||
|
<Route path="/research-test" element={<ResearchDashboard />} />
|
||||||
|
<Route path="/research-dashboard" element={<ResearchDashboard />} />
|
||||||
|
<Route path="/alwrity-researcher" element={<ResearchDashboard />} />
|
||||||
|
<Route path="/intent-research" element={<IntentResearchTest />} />
|
||||||
|
<Route path="/wix-test" element={<WixTestPage />} />
|
||||||
|
<Route path="/wix-test-direct" element={<WixTestPage />} />
|
||||||
|
<Route path="/wix/callback" element={<WixCallbackPage />} />
|
||||||
|
<Route path="/wp/callback" element={<WordPressCallbackPage />} />
|
||||||
|
<Route path="/gsc/callback" element={<GSCAuthCallback />} />
|
||||||
|
<Route path="/bing/callback" element={<BingCallbackPage />} />
|
||||||
|
<Route path="/bing-analytics-storage" element={<ProtectedRoute><BingAnalyticsStorage /></ProtectedRoute>} />
|
||||||
|
</Routes>
|
||||||
|
</ConditionalCopilotKit>
|
||||||
|
</AuthenticatedCopilotWrapper>
|
||||||
|
</Router>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<ErrorBoundary
|
||||||
|
context="Application Root"
|
||||||
|
showDetails={process.env.NODE_ENV === 'development'}
|
||||||
|
onError={(error, errorInfo) => {
|
||||||
|
// Custom error handler - send to analytics/monitoring
|
||||||
|
console.error('Global error caught:', { error, errorInfo });
|
||||||
|
// TODO: Send to error tracking service (Sentry, LogRocket, etc.)
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<ClerkProvider publishableKey={clerkPublishableKey} clerkJSUrl={clerkJSUrl}>
|
||||||
|
<SubscriptionProvider>
|
||||||
|
<OnboardingProvider>
|
||||||
|
{renderApp()}
|
||||||
|
</OnboardingProvider>
|
||||||
|
</SubscriptionProvider>
|
||||||
|
</ClerkProvider>
|
||||||
|
</ErrorBoundary>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
export default App;
|
||||||
537
_session_backup/ResearchSummary.tsx
Normal file
537
_session_backup/ResearchSummary.tsx
Normal file
@@ -0,0 +1,537 @@
|
|||||||
|
import React, { useMemo, useCallback } from "react";
|
||||||
|
import { Stack, Typography, Chip, Divider, Box, alpha, Paper, Tooltip } from "@mui/material";
|
||||||
|
import {
|
||||||
|
Insights as InsightsIcon,
|
||||||
|
Search as SearchIcon,
|
||||||
|
AttachMoney as AttachMoneyIcon,
|
||||||
|
EditNote as EditNoteIcon,
|
||||||
|
Article as ArticleIcon,
|
||||||
|
AutoAwesome as AutoAwesomeIcon,
|
||||||
|
FormatQuote as FormatQuoteIcon,
|
||||||
|
Campaign as CampaignIcon,
|
||||||
|
Explore as ExploreIcon,
|
||||||
|
} from "@mui/icons-material";
|
||||||
|
import { Research, ResearchInsight } from "../types";
|
||||||
|
import { GlassyCard, glassyCardSx, PrimaryButton } from "../ui";
|
||||||
|
import { FactCard } from "../FactCard";
|
||||||
|
|
||||||
|
interface ResearchSummaryProps {
|
||||||
|
research: Research;
|
||||||
|
canGenerateScript: boolean;
|
||||||
|
onGenerateScript: () => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const ResearchSummary: React.FC<ResearchSummaryProps> = ({
|
||||||
|
research,
|
||||||
|
canGenerateScript,
|
||||||
|
onGenerateScript,
|
||||||
|
}) => {
|
||||||
|
// Simple markdown-to-HTML converter
|
||||||
|
const renderMarkdown = useCallback((text: string) => {
|
||||||
|
if (!text) return null;
|
||||||
|
return text
|
||||||
|
.split('\n')
|
||||||
|
.filter(line => line.trim() !== '') // Remove empty lines
|
||||||
|
.map((line, i) => {
|
||||||
|
// Handle bold
|
||||||
|
let processedLine = line.replace(/\*\*(.*?)\*\*/g, '<strong>$1</strong>');
|
||||||
|
// Handle lists
|
||||||
|
if (processedLine.trim().startsWith('- ') || processedLine.trim().startsWith('* ')) {
|
||||||
|
return <li key={i} dangerouslySetInnerHTML={{ __html: processedLine.trim().substring(2) }} style={{ marginBottom: '4px', fontSize: '0.9rem' }} />;
|
||||||
|
}
|
||||||
|
// Handle headers - make them smaller
|
||||||
|
if (processedLine.startsWith('### ')) {
|
||||||
|
return <Typography key={i} variant="subtitle2" fontWeight={700} sx={{ mt: 1.5, mb: 0.5, color: '#1e293b' }}>{processedLine.substring(4)}</Typography>;
|
||||||
|
}
|
||||||
|
if (processedLine.startsWith('## ')) {
|
||||||
|
return <Typography key={i} variant="subtitle1" fontWeight={700} sx={{ mt: 1.5, mb: 0.5, color: '#0f172a' }}>{processedLine.substring(3)}</Typography>;
|
||||||
|
}
|
||||||
|
// Paragraphs - compact spacing
|
||||||
|
return processedLine.trim() ? <p key={i} dangerouslySetInnerHTML={{ __html: processedLine }} style={{ margin: '4px 0', fontSize: '0.9rem' }} /> : null;
|
||||||
|
});
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<GlassyCard sx={glassyCardSx}>
|
||||||
|
<Stack spacing={3}>
|
||||||
|
<Stack direction="row" justifyContent="space-between" alignItems="center" flexWrap="wrap" gap={2}>
|
||||||
|
<Stack direction="row" alignItems="center" spacing={2} sx={{ flex: 1 }}>
|
||||||
|
<Typography variant="h6" sx={{ display: "flex", alignItems: "center", gap: 1, color: "#0f172a", fontWeight: 700 }}>
|
||||||
|
<InsightsIcon />
|
||||||
|
Research Summary
|
||||||
|
</Typography>
|
||||||
|
|
||||||
|
{/* Research Metadata - Moved alongside title */}
|
||||||
|
<Stack direction="row" spacing={1.5} flexWrap="wrap">
|
||||||
|
{research.searchQueries && research.searchQueries.length > 0 && (
|
||||||
|
<Chip
|
||||||
|
icon={<SearchIcon sx={{ fontSize: "1rem !important" }} />}
|
||||||
|
label={`${research.searchQueries.length} search${research.searchQueries.length > 1 ? "es" : ""}`}
|
||||||
|
size="small"
|
||||||
|
sx={{
|
||||||
|
background: alpha("#667eea", 0.1),
|
||||||
|
color: "#667eea",
|
||||||
|
fontWeight: 600,
|
||||||
|
border: "1px solid rgba(102, 126, 234, 0.2)",
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
{research.searchType && (
|
||||||
|
<Chip
|
||||||
|
label={`${research.searchType.charAt(0).toUpperCase() + research.searchType.slice(1)} search`}
|
||||||
|
size="small"
|
||||||
|
sx={{
|
||||||
|
background: alpha("#10b981", 0.1),
|
||||||
|
color: "#059669",
|
||||||
|
fontWeight: 600,
|
||||||
|
border: "1px solid rgba(16, 185, 129, 0.2)",
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
{research.sourceCount !== undefined && (
|
||||||
|
<Chip
|
||||||
|
label={`${research.sourceCount} source${research.sourceCount !== 1 ? "s" : ""}`}
|
||||||
|
size="small"
|
||||||
|
sx={{
|
||||||
|
background: alpha("#6366f1", 0.1),
|
||||||
|
color: "#4f46e5",
|
||||||
|
fontWeight: 600,
|
||||||
|
border: "1px solid rgba(99, 102, 241, 0.2)",
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
{research.cost !== undefined && (
|
||||||
|
<Chip
|
||||||
|
icon={<AttachMoneyIcon sx={{ fontSize: "0.875rem !important" }} />}
|
||||||
|
label={`$${research.cost.toFixed(3)}`}
|
||||||
|
size="small"
|
||||||
|
sx={{
|
||||||
|
background: alpha("#f59e0b", 0.1),
|
||||||
|
color: "#d97706",
|
||||||
|
fontWeight: 600,
|
||||||
|
border: "1px solid rgba(245, 158, 11, 0.2)",
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
<PrimaryButton
|
||||||
|
onClick={onGenerateScript}
|
||||||
|
disabled={!canGenerateScript}
|
||||||
|
startIcon={<EditNoteIcon />}
|
||||||
|
tooltip={!canGenerateScript ? "Complete research to generate script" : "Generate AI-powered script from research"}
|
||||||
|
>
|
||||||
|
Generate Script
|
||||||
|
</PrimaryButton>
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
<Box sx={{ width: "100%" }}>
|
||||||
|
{/* Main Summary */}
|
||||||
|
{research.summary && (
|
||||||
|
<Paper
|
||||||
|
elevation={0}
|
||||||
|
sx={{
|
||||||
|
p: 2.5,
|
||||||
|
mb: 3,
|
||||||
|
background: "#f8fafc",
|
||||||
|
border: "1px solid rgba(0,0,0,0.06)",
|
||||||
|
borderRadius: 2,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="subtitle2" sx={{ mb: 1.5, color: "#64748b", fontWeight: 700, fontSize: "0.75rem", textTransform: "uppercase", letterSpacing: "0.05em", display: "flex", alignItems: "center", gap: 1 }}>
|
||||||
|
<AutoAwesomeIcon fontSize="small" sx={{ color: "#667eea", fontSize: "1rem" }} />
|
||||||
|
Executive Summary
|
||||||
|
</Typography>
|
||||||
|
<Box sx={{
|
||||||
|
lineHeight: 1.6,
|
||||||
|
fontSize: "0.9rem",
|
||||||
|
color: "#334155",
|
||||||
|
"& p": { m: 0, mb: 1 },
|
||||||
|
"& ul": { m: 0, mb: 1, pl: 2.5 },
|
||||||
|
"& li": { mb: 0.5 },
|
||||||
|
"& strong": { color: "#0f172a", fontWeight: 600 }
|
||||||
|
}}>
|
||||||
|
{renderMarkdown(research.summary)}
|
||||||
|
</Box>
|
||||||
|
</Paper>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Deep Insights */}
|
||||||
|
{(research.keyInsights && research.keyInsights.length > 0) ? (
|
||||||
|
<Box sx={{ mb: 4 }}>
|
||||||
|
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||||
|
<ArticleIcon sx={{ color: "#667eea" }} />
|
||||||
|
Deep Insights
|
||||||
|
</Typography>
|
||||||
|
<Stack spacing={2.5}>
|
||||||
|
{research.keyInsights.map((insight: ResearchInsight, idx: number) => (
|
||||||
|
<Paper
|
||||||
|
key={idx}
|
||||||
|
elevation={0}
|
||||||
|
sx={{
|
||||||
|
p: 2.5,
|
||||||
|
background: "#ffffff",
|
||||||
|
border: "1px solid rgba(0,0,0,0.06)",
|
||||||
|
boxShadow: "0 2px 12px rgba(0,0,0,0.03)",
|
||||||
|
borderRadius: 2,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" justifyContent="space-between" alignItems="flex-start" sx={{ mb: 1.5 }}>
|
||||||
|
<Typography variant="subtitle1" sx={{ color: "#0f172a", fontWeight: 700 }}>
|
||||||
|
{insight.title}
|
||||||
|
</Typography>
|
||||||
|
{insight.source_indices && insight.source_indices.length > 0 && (
|
||||||
|
<Stack direction="row" spacing={0.5}>
|
||||||
|
{insight.source_indices.map(sIdx => {
|
||||||
|
const sourceIdx = sIdx - 1;
|
||||||
|
const fact = research.factCards[sourceIdx];
|
||||||
|
const sourceUrl = fact?.url;
|
||||||
|
const hasUrl = !!sourceUrl;
|
||||||
|
const hue = (sIdx * 47 + 220) % 360;
|
||||||
|
const gradientFrom = `hsl(${hue}, 70%, 55%)`;
|
||||||
|
const gradientTo = `hsl(${(hue + 30) % 360}, 80%, 65%)`;
|
||||||
|
return (
|
||||||
|
<Tooltip
|
||||||
|
key={sIdx}
|
||||||
|
title={hasUrl ? (
|
||||||
|
<Box sx={{ maxWidth: 300, wordBreak: "break-all" }}>
|
||||||
|
<Typography variant="caption" sx={{ color: "#fff", fontWeight: 600 }}>Source {sIdx}</Typography>
|
||||||
|
<br />
|
||||||
|
<Typography variant="caption" sx={{ color: "rgba(255,255,255,0.8)", fontSize: "0.65rem" }}>{sourceUrl}</Typography>
|
||||||
|
</Box>
|
||||||
|
) : `Source ${sIdx}`}
|
||||||
|
arrow
|
||||||
|
placement="top"
|
||||||
|
>
|
||||||
|
<Chip
|
||||||
|
label={hasUrl ? `S${sIdx} ↗` : `S${sIdx}`}
|
||||||
|
size="small"
|
||||||
|
onClick={hasUrl ? () => window.open(sourceUrl, "_blank", "noopener,noreferrer") : undefined}
|
||||||
|
sx={{
|
||||||
|
height: 24,
|
||||||
|
minWidth: 36,
|
||||||
|
fontSize: '0.7rem',
|
||||||
|
fontWeight: 800,
|
||||||
|
fontFamily: "'Inter', 'Roboto', monospace",
|
||||||
|
letterSpacing: "0.02em",
|
||||||
|
border: "none",
|
||||||
|
background: hasUrl
|
||||||
|
? `linear-gradient(135deg, ${gradientFrom}, ${gradientTo})`
|
||||||
|
: `linear-gradient(135deg, ${alpha(gradientFrom, 0.3)}, ${alpha(gradientTo, 0.3)})`,
|
||||||
|
color: hasUrl ? "#fff" : alpha("#fff", 0.7),
|
||||||
|
cursor: hasUrl ? "pointer" : "default",
|
||||||
|
borderRadius: "8px",
|
||||||
|
px: 0.5,
|
||||||
|
boxShadow: hasUrl
|
||||||
|
? `0 2px 8px ${alpha(gradientFrom, 0.35)}, inset 0 1px 0 ${alpha("#fff", 0.2)}`
|
||||||
|
: "none",
|
||||||
|
transition: "all 0.2s ease",
|
||||||
|
"&:hover": hasUrl ? {
|
||||||
|
background: `linear-gradient(135deg, ${gradientTo}, ${gradientFrom})`,
|
||||||
|
boxShadow: `0 4px 14px ${alpha(gradientFrom, 0.5)}, inset 0 1px 0 ${alpha("#fff", 0.3)}`,
|
||||||
|
transform: "translateY(-1px)",
|
||||||
|
} : {},
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
</Tooltip>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
</Stack>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
<Box sx={{
|
||||||
|
color: "#475569",
|
||||||
|
lineHeight: 1.7,
|
||||||
|
fontSize: "0.9rem",
|
||||||
|
"& p": { m: 0, mb: 1.5 },
|
||||||
|
"& ul": { m: 0, mb: 1.5, pl: 2 }
|
||||||
|
}}>
|
||||||
|
{renderMarkdown(insight.content)}
|
||||||
|
</Box>
|
||||||
|
</Paper>
|
||||||
|
))}
|
||||||
|
</Stack>
|
||||||
|
</Box>
|
||||||
|
) : (
|
||||||
|
/* Fallback if keyInsights is missing but we have summary paragraphs */
|
||||||
|
research.summary && research.summary.length > 500 && !research.keyInsights && (
|
||||||
|
<Box sx={{ mb: 4 }}>
|
||||||
|
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||||
|
<ArticleIcon sx={{ color: "#667eea" }} />
|
||||||
|
Additional Insights
|
||||||
|
</Typography>
|
||||||
|
<Paper
|
||||||
|
elevation={0}
|
||||||
|
sx={{
|
||||||
|
p: 2.5,
|
||||||
|
background: "#ffffff",
|
||||||
|
border: "1px solid rgba(0,0,0,0.06)",
|
||||||
|
boxShadow: "0 2px 12px rgba(0,0,0,0.03)",
|
||||||
|
borderRadius: 2,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Box sx={{
|
||||||
|
color: "#475569",
|
||||||
|
lineHeight: 1.7,
|
||||||
|
fontSize: "0.9rem",
|
||||||
|
}}>
|
||||||
|
{/* Render parts of summary that might contain insights if structured data is missing */}
|
||||||
|
{renderMarkdown(research.summary.split('\n\n').slice(1).join('\n\n'))}
|
||||||
|
</Box>
|
||||||
|
</Paper>
|
||||||
|
</Box>
|
||||||
|
)
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Expert Quotes Section */}
|
||||||
|
{research.expertQuotes && research.expertQuotes.length > 0 && (
|
||||||
|
<Box sx={{ mt: 4, pt: 3, borderTop: "1px solid rgba(0,0,0,0.04)" }}>
|
||||||
|
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||||
|
<FormatQuoteIcon sx={{ color: "#8b5cf6" }} />
|
||||||
|
Expert Quotes ({research.expertQuotes.length})
|
||||||
|
</Typography>
|
||||||
|
<Stack spacing={2}>
|
||||||
|
{research.expertQuotes.map((eq, idx) => (
|
||||||
|
<Paper
|
||||||
|
key={idx}
|
||||||
|
elevation={0}
|
||||||
|
sx={{
|
||||||
|
p: 2.5,
|
||||||
|
background: "linear-gradient(135deg, rgba(139, 92, 246, 0.04) 0%, rgba(99, 102, 241, 0.04) 100%)",
|
||||||
|
border: "1px solid rgba(139, 92, 246, 0.15)",
|
||||||
|
borderLeft: "4px solid #8b5cf6",
|
||||||
|
borderRadius: 2,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" spacing={1.5} alignItems="flex-start">
|
||||||
|
<FormatQuoteIcon sx={{ color: "#8b5cf6", fontSize: "1.5rem", mt: -0.5, opacity: 0.7 }} />
|
||||||
|
<Box sx={{ flex: 1 }}>
|
||||||
|
<Typography variant="body2" sx={{ color: "#1e293b", fontStyle: "italic", lineHeight: 1.7, fontSize: "0.95rem" }}>
|
||||||
|
“{eq.quote}”
|
||||||
|
</Typography>
|
||||||
|
{eq.source_index !== undefined && (() => {
|
||||||
|
const fact = research.factCards[eq.source_index - 1];
|
||||||
|
const sourceUrl = fact?.url;
|
||||||
|
const hasUrl = !!sourceUrl;
|
||||||
|
const hue = (eq.source_index * 47 + 270) % 360;
|
||||||
|
const gradientFrom = `hsl(${hue}, 70%, 55%)`;
|
||||||
|
const gradientTo = `hsl(${(hue + 30) % 360}, 80%, 65%)`;
|
||||||
|
return (
|
||||||
|
<Box sx={{ mt: 1 }}>
|
||||||
|
<Tooltip title={hasUrl ? (
|
||||||
|
<Box sx={{ maxWidth: 300, wordBreak: "break-all" }}>
|
||||||
|
<Typography variant="caption" sx={{ color: "#fff", fontWeight: 600 }}>Source {eq.source_index}</Typography>
|
||||||
|
<br />
|
||||||
|
<Typography variant="caption" sx={{ color: "rgba(255,255,255,0.8)", fontSize: "0.65rem" }}>{sourceUrl}</Typography>
|
||||||
|
</Box>
|
||||||
|
) : `Source ${eq.source_index}`} arrow placement="top">
|
||||||
|
<Chip
|
||||||
|
label={hasUrl ? `Source ${eq.source_index} ↗` : `Source ${eq.source_index}`}
|
||||||
|
size="small"
|
||||||
|
onClick={hasUrl ? () => window.open(sourceUrl, "_blank", "noopener,noreferrer") : undefined}
|
||||||
|
sx={{
|
||||||
|
height: 24,
|
||||||
|
fontSize: "0.7rem",
|
||||||
|
fontWeight: 800,
|
||||||
|
fontFamily: "'Inter', 'Roboto', monospace",
|
||||||
|
border: "none",
|
||||||
|
background: hasUrl
|
||||||
|
? `linear-gradient(135deg, ${gradientFrom}, ${gradientTo})`
|
||||||
|
: `linear-gradient(135deg, ${alpha(gradientFrom, 0.3)}, ${alpha(gradientTo, 0.3)})`,
|
||||||
|
color: hasUrl ? "#fff" : alpha("#fff", 0.7),
|
||||||
|
cursor: hasUrl ? "pointer" : "default",
|
||||||
|
borderRadius: "8px",
|
||||||
|
px: 1,
|
||||||
|
boxShadow: hasUrl
|
||||||
|
? `0 2px 8px ${alpha(gradientFrom, 0.35)}, inset 0 1px 0 ${alpha("#fff", 0.2)}`
|
||||||
|
: "none",
|
||||||
|
transition: "all 0.2s ease",
|
||||||
|
"&:hover": hasUrl ? {
|
||||||
|
background: `linear-gradient(135deg, ${gradientTo}, ${gradientFrom})`,
|
||||||
|
boxShadow: `0 4px 14px ${alpha(gradientFrom, 0.5)}, inset 0 1px 0 ${alpha("#fff", 0.3)}`,
|
||||||
|
transform: "translateY(-1px)",
|
||||||
|
} : {},
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
</Tooltip>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
})()}
|
||||||
|
</Box>
|
||||||
|
</Stack>
|
||||||
|
</Paper>
|
||||||
|
))}
|
||||||
|
</Stack>
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Search Queries Used */}
|
||||||
|
{research.searchQueries && research.searchQueries.length > 0 && (
|
||||||
|
<Box sx={{ mt: 4, pt: 3, borderTop: "1px solid rgba(0,0,0,0.04)" }}>
|
||||||
|
<Typography variant="subtitle2" sx={{ mb: 1.5, color: "#64748b", fontWeight: 700, fontSize: "0.7rem", textTransform: "uppercase", letterSpacing: "0.05em" }}>
|
||||||
|
Search Queries Used
|
||||||
|
</Typography>
|
||||||
|
<Stack direction="row" spacing={1} flexWrap="wrap" useFlexGap>
|
||||||
|
{research.searchQueries.map((query, idx) => (
|
||||||
|
<Chip
|
||||||
|
key={idx}
|
||||||
|
label={query}
|
||||||
|
size="small"
|
||||||
|
variant="outlined"
|
||||||
|
sx={{
|
||||||
|
borderColor: "rgba(102, 126, 234, 0.15)",
|
||||||
|
color: "#94a3b8",
|
||||||
|
background: alpha("#f8fafc", 0.3),
|
||||||
|
fontSize: "0.7rem",
|
||||||
|
borderRadius: 1,
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
))}
|
||||||
|
</Stack>
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
{research.factCards.length > 0 && (
|
||||||
|
<>
|
||||||
|
<Divider sx={{ borderColor: "rgba(0,0,0,0.08)" }} />
|
||||||
|
<Stack direction="row" justifyContent="space-between" alignItems="center" sx={{ mb: 1.5, flexWrap: "wrap", gap: 1 }}>
|
||||||
|
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600 }}>
|
||||||
|
Research Sources & Facts ({research.factCards.length})
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="caption" sx={{ color: "#64748b", fontSize: "0.75rem" }}>
|
||||||
|
Click to expand • Hover to see source
|
||||||
|
</Typography>
|
||||||
|
</Stack>
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
display: "grid",
|
||||||
|
gridTemplateColumns: { xs: "1fr", sm: "repeat(2, 1fr)", md: "repeat(3, 1fr)", lg: "repeat(4, 1fr)" },
|
||||||
|
gap: 1.5,
|
||||||
|
width: "100%",
|
||||||
|
overflow: "hidden",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{research.factCards.map((fact) => (
|
||||||
|
<FactCard key={fact.id} fact={fact} />
|
||||||
|
))}
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Listener CTA Section */}
|
||||||
|
{research.listenerCta && research.listenerCta.length > 0 && (
|
||||||
|
<>
|
||||||
|
<Divider sx={{ borderColor: "rgba(0,0,0,0.08)" }} />
|
||||||
|
<Box>
|
||||||
|
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||||
|
<CampaignIcon sx={{ color: "#f59e0b" }} />
|
||||||
|
Listener Call-to-Action Ideas ({research.listenerCta.length})
|
||||||
|
</Typography>
|
||||||
|
<Stack spacing={1.5}>
|
||||||
|
{research.listenerCta.map((cta, idx) => (
|
||||||
|
<Paper
|
||||||
|
key={idx}
|
||||||
|
elevation={0}
|
||||||
|
sx={{
|
||||||
|
p: 2,
|
||||||
|
background: "linear-gradient(135deg, rgba(245, 158, 11, 0.05) 0%, rgba(251, 191, 36, 0.05) 100%)",
|
||||||
|
border: "1px solid rgba(245, 158, 11, 0.15)",
|
||||||
|
borderRadius: 2,
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "flex-start",
|
||||||
|
gap: 1.5,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Chip
|
||||||
|
label={`#${idx + 1}`}
|
||||||
|
size="small"
|
||||||
|
sx={{
|
||||||
|
bgcolor: alpha("#f59e0b", 0.15),
|
||||||
|
color: "#b45309",
|
||||||
|
fontWeight: 700,
|
||||||
|
fontSize: "0.7rem",
|
||||||
|
height: 24,
|
||||||
|
minWidth: 32,
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.6, flex: 1, pt: 0.2 }}>
|
||||||
|
{cta}
|
||||||
|
</Typography>
|
||||||
|
</Paper>
|
||||||
|
))}
|
||||||
|
</Stack>
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Mapped Angles Section */}
|
||||||
|
{research.mappedAngles && research.mappedAngles.length > 0 && (
|
||||||
|
<>
|
||||||
|
<Divider sx={{ borderColor: "rgba(0,0,0,0.08)" }} />
|
||||||
|
<Box>
|
||||||
|
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||||
|
<ExploreIcon sx={{ color: "#06b6d4" }} />
|
||||||
|
Content Angles ({research.mappedAngles.length})
|
||||||
|
</Typography>
|
||||||
|
<Stack spacing={2}>
|
||||||
|
{research.mappedAngles.map((angle, idx) => (
|
||||||
|
<Paper
|
||||||
|
key={idx}
|
||||||
|
elevation={0}
|
||||||
|
sx={{
|
||||||
|
p: 2.5,
|
||||||
|
background: "#ffffff",
|
||||||
|
border: "1px solid rgba(0,0,0,0.06)",
|
||||||
|
borderLeft: "4px solid #06b6d4",
|
||||||
|
boxShadow: "0 2px 12px rgba(0,0,0,0.03)",
|
||||||
|
borderRadius: 2,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" justifyContent="space-between" alignItems="flex-start" sx={{ mb: 1 }}>
|
||||||
|
<Typography variant="subtitle1" sx={{ color: "#0f172a", fontWeight: 700 }}>
|
||||||
|
{angle.title}
|
||||||
|
</Typography>
|
||||||
|
{angle.mappedFactIds && angle.mappedFactIds.length > 0 && (
|
||||||
|
<Stack direction="row" spacing={0.5}>
|
||||||
|
{angle.mappedFactIds.slice(0, 4).map((fid: string) => (
|
||||||
|
<Chip
|
||||||
|
key={fid}
|
||||||
|
label={fid.replace("fact_", "F")}
|
||||||
|
size="small"
|
||||||
|
variant="outlined"
|
||||||
|
sx={{
|
||||||
|
height: 18,
|
||||||
|
fontSize: "0.6rem",
|
||||||
|
fontWeight: 700,
|
||||||
|
borderColor: alpha("#06b6d4", 0.3),
|
||||||
|
color: "#06b6d4",
|
||||||
|
bgcolor: alpha("#06b6d4", 0.05),
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
))}
|
||||||
|
{angle.mappedFactIds.length > 4 && (
|
||||||
|
<Chip
|
||||||
|
label={`+${angle.mappedFactIds.length - 4}`}
|
||||||
|
size="small"
|
||||||
|
sx={{ height: 18, fontSize: "0.6rem", color: "#64748b" }}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7, fontSize: "0.9rem" }}>
|
||||||
|
{angle.why}
|
||||||
|
</Typography>
|
||||||
|
</Paper>
|
||||||
|
))}
|
||||||
|
</Stack>
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
</GlassyCard>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
811
_session_backup/SceneEditor.tsx
Normal file
811
_session_backup/SceneEditor.tsx
Normal file
@@ -0,0 +1,811 @@
|
|||||||
|
import React, { useState, useEffect } from "react";
|
||||||
|
import { Stack, Box, Typography, Divider, Chip, alpha, CircularProgress, LinearProgress, IconButton, Tooltip } from "@mui/material";
|
||||||
|
import {
|
||||||
|
EditNote as EditNoteIcon,
|
||||||
|
CheckCircle as CheckCircleIcon,
|
||||||
|
RadioButtonUnchecked as RadioButtonUncheckedIcon,
|
||||||
|
VolumeUp as VolumeUpIcon,
|
||||||
|
PlayArrow as PlayArrowIcon,
|
||||||
|
Image as ImageIcon,
|
||||||
|
Delete as DeleteIcon,
|
||||||
|
} from "@mui/icons-material";
|
||||||
|
import { Scene, Line, Knobs } from "../types";
|
||||||
|
import { GlassyCard, glassyCardSx, PrimaryButton } from "../ui";
|
||||||
|
import { LineEditor } from "./LineEditor";
|
||||||
|
import { ImageRegenerateModal, ImageGenerationSettings } from "./ImageRegenerateModal";
|
||||||
|
import { AudioRegenerateModal, AudioGenerationSettings } from "./AudioRegenerateModal";
|
||||||
|
import { podcastApi } from "../../../services/podcastApi";
|
||||||
|
import { aiApiClient } from "../../../api/client";
|
||||||
|
import { getCachedMedia, setCachedMedia } from "../../../utils/mediaCache";
|
||||||
|
|
||||||
|
interface SceneEditorProps {
|
||||||
|
scene: Scene;
|
||||||
|
onUpdateScene: (s: Scene) => void;
|
||||||
|
onApprove: (id: string) => Promise<void>;
|
||||||
|
onDelete: (sceneId: string) => void;
|
||||||
|
knobs: Knobs;
|
||||||
|
approvingSceneId?: string | null;
|
||||||
|
generatingAudioId?: string | null;
|
||||||
|
onAudioGenerationStart?: (sceneId: string) => void;
|
||||||
|
onAudioGenerated?: (sceneId: string, audioUrl: string) => void;
|
||||||
|
idea?: string; // Podcast idea for image generation context
|
||||||
|
avatarUrl?: string | null; // Base avatar URL for consistent scene image generation
|
||||||
|
totalScenes?: number; // Total number of scenes in the script
|
||||||
|
}
|
||||||
|
|
||||||
|
export const SceneEditor: React.FC<SceneEditorProps> = ({
|
||||||
|
scene,
|
||||||
|
onUpdateScene,
|
||||||
|
onApprove,
|
||||||
|
onDelete,
|
||||||
|
knobs,
|
||||||
|
approvingSceneId,
|
||||||
|
generatingAudioId,
|
||||||
|
onAudioGenerationStart,
|
||||||
|
onAudioGenerated,
|
||||||
|
idea,
|
||||||
|
avatarUrl,
|
||||||
|
totalScenes,
|
||||||
|
}) => {
|
||||||
|
const [localGenerating, setLocalGenerating] = useState(false);
|
||||||
|
const [generatingImage, setGeneratingImage] = useState(false);
|
||||||
|
const [imageGenerationStatus, setImageGenerationStatus] = useState<string>("");
|
||||||
|
const [imageGenerationProgress, setImageGenerationProgress] = useState<number>(0);
|
||||||
|
const [audioBlobUrl, setAudioBlobUrl] = useState<string | null>(null);
|
||||||
|
const [imageBlobUrl, setImageBlobUrl] = useState<string | null>(null);
|
||||||
|
const [imageLoading, setImageLoading] = useState(false);
|
||||||
|
const [showRegenerateModal, setShowRegenerateModal] = useState(false);
|
||||||
|
const [showAudioModal, setShowAudioModal] = useState(false);
|
||||||
|
const [audioSettings, setAudioSettings] = useState<AudioGenerationSettings>({
|
||||||
|
voiceId: "Wise_Woman",
|
||||||
|
speed: 1.0,
|
||||||
|
volume: 1.0,
|
||||||
|
pitch: 0.0,
|
||||||
|
emotion: scene.emotion || "neutral",
|
||||||
|
englishNormalization: true,
|
||||||
|
sampleRate: 24000,
|
||||||
|
bitrate: 64000,
|
||||||
|
channel: "1",
|
||||||
|
format: "mp3",
|
||||||
|
languageBoost: "auto",
|
||||||
|
});
|
||||||
|
|
||||||
|
// Load audio as blob when audioUrl is available
|
||||||
|
useEffect(() => {
|
||||||
|
if (!scene.audioUrl) {
|
||||||
|
// Clean up blob URL if audioUrl is removed
|
||||||
|
setAudioBlobUrl((currentBlobUrl) => {
|
||||||
|
if (currentBlobUrl) {
|
||||||
|
URL.revokeObjectURL(currentBlobUrl);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
let isMounted = true;
|
||||||
|
const currentAudioUrl = scene.audioUrl; // Capture current value
|
||||||
|
|
||||||
|
const loadAudioBlob = async () => {
|
||||||
|
try {
|
||||||
|
// Normalize path
|
||||||
|
let audioPath = currentAudioUrl.startsWith('/') ? currentAudioUrl : `/${currentAudioUrl}`;
|
||||||
|
|
||||||
|
// Convert /api/story/audio/ to /api/podcast/audio/ if needed
|
||||||
|
if (audioPath.includes('/api/story/audio/')) {
|
||||||
|
const filename = audioPath.split('/api/story/audio/').pop() || '';
|
||||||
|
audioPath = `/api/podcast/audio/${filename}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure it's a podcast audio endpoint
|
||||||
|
if (!audioPath.includes('/api/podcast/audio/')) {
|
||||||
|
const filename = audioPath.split('/').pop() || currentAudioUrl;
|
||||||
|
audioPath = `/api/podcast/audio/${filename}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove query parameters if present
|
||||||
|
audioPath = audioPath.split('?')[0];
|
||||||
|
|
||||||
|
const response = await aiApiClient.get(audioPath, {
|
||||||
|
responseType: 'blob',
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!isMounted) {
|
||||||
|
// Component unmounted or audioUrl changed, don't set blob URL
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Double-check that audioUrl hasn't changed
|
||||||
|
if (scene.audioUrl !== currentAudioUrl) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const blob = response.data;
|
||||||
|
const blobUrl = URL.createObjectURL(blob);
|
||||||
|
|
||||||
|
setAudioBlobUrl((prevBlobUrl) => {
|
||||||
|
// Clean up previous blob URL if exists
|
||||||
|
if (prevBlobUrl && prevBlobUrl !== blobUrl) {
|
||||||
|
URL.revokeObjectURL(prevBlobUrl);
|
||||||
|
}
|
||||||
|
return blobUrl;
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
console.error(`Failed to load audio blob for scene ${scene.id}:`, error);
|
||||||
|
// Don't set blob URL on error - will show error state
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
loadAudioBlob();
|
||||||
|
|
||||||
|
// Cleanup: only mark as unmounted, don't revoke blob URL here
|
||||||
|
// The blob URL will be cleaned up when audioUrl changes (new effect) or component unmounts
|
||||||
|
return () => {
|
||||||
|
isMounted = false;
|
||||||
|
};
|
||||||
|
}, [scene.audioUrl, scene.id]);
|
||||||
|
|
||||||
|
// Load image as blob when imageUrl is available
|
||||||
|
useEffect(() => {
|
||||||
|
if (!scene.imageUrl) {
|
||||||
|
// Clean up blob URL if imageUrl is removed
|
||||||
|
setImageBlobUrl((currentBlobUrl) => {
|
||||||
|
if (currentBlobUrl && currentBlobUrl.startsWith('blob:')) {
|
||||||
|
URL.revokeObjectURL(currentBlobUrl);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check cache first with scene context
|
||||||
|
const cachedUrl = getCachedMedia(scene.imageUrl, scene.id);
|
||||||
|
if (cachedUrl) {
|
||||||
|
console.log('[SceneEditor] Using cached image:', scene.imageUrl, `(scene: ${scene.id})`);
|
||||||
|
setImageBlobUrl(cachedUrl);
|
||||||
|
setImageLoading(false);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
let isMounted = true;
|
||||||
|
const currentImageUrl = scene.imageUrl; // Capture current value
|
||||||
|
|
||||||
|
const loadImageBlob = async () => {
|
||||||
|
try {
|
||||||
|
setImageLoading(true);
|
||||||
|
|
||||||
|
// Check cache again in case it was loaded while we were waiting
|
||||||
|
const cachedUrl = getCachedMedia(currentImageUrl, scene.id);
|
||||||
|
if (cachedUrl) {
|
||||||
|
if (isMounted) {
|
||||||
|
setImageBlobUrl(cachedUrl);
|
||||||
|
setImageLoading(false);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('[SceneEditor] Loading image blob for:', currentImageUrl);
|
||||||
|
|
||||||
|
// Normalize path
|
||||||
|
let imagePath = currentImageUrl.startsWith('/') ? currentImageUrl : `/${currentImageUrl}`;
|
||||||
|
|
||||||
|
// Convert /api/story/images/ to /api/podcast/images/ if needed
|
||||||
|
if (imagePath.includes('/api/story/images/')) {
|
||||||
|
const filename = imagePath.split('/api/story/images/').pop() || '';
|
||||||
|
imagePath = `/api/podcast/images/${filename}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure it's a podcast image endpoint
|
||||||
|
if (!imagePath.includes('/api/podcast/images/')) {
|
||||||
|
const filename = imagePath.split('/').pop() || currentImageUrl;
|
||||||
|
imagePath = `/api/podcast/images/${filename}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove query parameters if present
|
||||||
|
imagePath = imagePath.split('?')[0];
|
||||||
|
|
||||||
|
const response = await aiApiClient.get(imagePath, {
|
||||||
|
responseType: 'blob',
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!isMounted) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Double-check that imageUrl hasn't changed
|
||||||
|
if (scene.imageUrl !== currentImageUrl) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const blob = response.data;
|
||||||
|
const blobUrl = URL.createObjectURL(blob);
|
||||||
|
|
||||||
|
// Cache the blob URL with scene context
|
||||||
|
setCachedMedia(currentImageUrl, blobUrl, 'image', blob.size, scene.id);
|
||||||
|
|
||||||
|
setImageBlobUrl((prevBlobUrl) => {
|
||||||
|
// Clean up previous blob URL if exists
|
||||||
|
if (prevBlobUrl && prevBlobUrl !== blobUrl && prevBlobUrl.startsWith('blob:')) {
|
||||||
|
URL.revokeObjectURL(prevBlobUrl);
|
||||||
|
}
|
||||||
|
return blobUrl;
|
||||||
|
});
|
||||||
|
console.log('[SceneEditor] Image blob loaded and cached successfully:', currentImageUrl);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('[SceneEditor] Failed to load image blob:', error);
|
||||||
|
if (isMounted) {
|
||||||
|
// Try adding query token as fallback
|
||||||
|
try {
|
||||||
|
const token = localStorage.getItem('clerk_dashboard_token') || '';
|
||||||
|
if (token) {
|
||||||
|
const urlWithToken = `${currentImageUrl}?token=${encodeURIComponent(token)}`;
|
||||||
|
setImageBlobUrl(urlWithToken);
|
||||||
|
setCachedMedia(currentImageUrl, urlWithToken, 'image', undefined, scene.id);
|
||||||
|
}
|
||||||
|
} catch (fallbackError) {
|
||||||
|
console.error('[SceneEditor] Fallback image loading failed:', fallbackError);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
if (isMounted) {
|
||||||
|
setImageLoading(false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
loadImageBlob();
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
isMounted = false;
|
||||||
|
// Don't cleanup blob URL here - let the cache handle it
|
||||||
|
};
|
||||||
|
}, [scene.imageUrl]);
|
||||||
|
|
||||||
|
const updateLine = (updatedLine: Line) => {
|
||||||
|
const updated = { ...scene, lines: scene.lines.map((l) => (l.id === updatedLine.id ? updatedLine : l)) };
|
||||||
|
onUpdateScene(updated);
|
||||||
|
};
|
||||||
|
|
||||||
|
const approving = approvingSceneId === scene.id;
|
||||||
|
const generating = generatingAudioId === scene.id || localGenerating;
|
||||||
|
const hasAudio = Boolean(scene.audioUrl && audioBlobUrl);
|
||||||
|
const hasImage = Boolean(scene.imageUrl);
|
||||||
|
|
||||||
|
const handleApproveAndGenerate = async (settings?: AudioGenerationSettings) => {
|
||||||
|
const wasAlreadyApproved = scene.approved;
|
||||||
|
const sceneId = scene.id;
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Set generating state
|
||||||
|
setLocalGenerating(true);
|
||||||
|
if (onAudioGenerationStart) {
|
||||||
|
onAudioGenerationStart(sceneId);
|
||||||
|
}
|
||||||
|
|
||||||
|
// If scene is not approved yet, approve it first
|
||||||
|
// This will update the parent script state
|
||||||
|
if (!scene.approved) {
|
||||||
|
await onApprove(sceneId);
|
||||||
|
// The parent's approveScene already updated the script state
|
||||||
|
// We need to wait for React to propagate the updated scene prop
|
||||||
|
// For now, we'll update it locally too to ensure UI updates immediately
|
||||||
|
onUpdateScene({ ...scene, approved: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
// Use the current scene (which should now be approved)
|
||||||
|
// If scene prop hasn't updated yet, use the local update we just made
|
||||||
|
const currentScene = { ...scene, approved: true };
|
||||||
|
|
||||||
|
// Generate audio
|
||||||
|
const effectiveSettings = settings || audioSettings;
|
||||||
|
const result = await podcastApi.renderSceneAudio({
|
||||||
|
scene: currentScene,
|
||||||
|
voiceId: effectiveSettings.voiceId || "Wise_Woman",
|
||||||
|
emotion: effectiveSettings.emotion || scene.emotion || knobs.voice_emotion || "neutral",
|
||||||
|
speed: effectiveSettings.speed ?? knobs.voice_speed ?? 1.0,
|
||||||
|
volume: effectiveSettings.volume ?? 1.0,
|
||||||
|
pitch: effectiveSettings.pitch ?? 0.0,
|
||||||
|
englishNormalization: effectiveSettings.englishNormalization ?? true,
|
||||||
|
sampleRate: effectiveSettings.sampleRate,
|
||||||
|
bitrate: effectiveSettings.bitrate,
|
||||||
|
channel: effectiveSettings.channel,
|
||||||
|
format: effectiveSettings.format,
|
||||||
|
languageBoost: effectiveSettings.languageBoost,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Update scene with audio URL and ensure approved state
|
||||||
|
// This will sync with parent script state
|
||||||
|
const updatedScene = { ...currentScene, audioUrl: result.audioUrl, approved: true };
|
||||||
|
onUpdateScene(updatedScene);
|
||||||
|
|
||||||
|
if (onAudioGenerated) {
|
||||||
|
onAudioGenerated(sceneId, result.audioUrl);
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
console.error("Failed to approve and generate audio:", error);
|
||||||
|
// On error, revert approval only if we just approved it in this call
|
||||||
|
if (!wasAlreadyApproved) {
|
||||||
|
onUpdateScene({ ...scene, approved: false, audioUrl: undefined });
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
} finally {
|
||||||
|
setLocalGenerating(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleGenerateImage = async (settings?: ImageGenerationSettings) => {
|
||||||
|
const sceneId = scene.id;
|
||||||
|
const startTime = Date.now();
|
||||||
|
let progressInterval: NodeJS.Timeout | null = null;
|
||||||
|
|
||||||
|
try {
|
||||||
|
setGeneratingImage(true);
|
||||||
|
setShowRegenerateModal(false);
|
||||||
|
setImageGenerationStatus("Submitting image generation request...");
|
||||||
|
setImageGenerationProgress(10);
|
||||||
|
|
||||||
|
// Build scene content from lines for context
|
||||||
|
const sceneContent = scene.lines.map((line) => line.text).join(" ");
|
||||||
|
|
||||||
|
// Log avatar URL for debugging
|
||||||
|
console.log("[SceneEditor] Generating image with avatarUrl:", avatarUrl);
|
||||||
|
console.log("[SceneEditor] Custom settings:", settings);
|
||||||
|
|
||||||
|
// Simulate progress updates during API call
|
||||||
|
progressInterval = setInterval(() => {
|
||||||
|
const elapsed = Date.now() - startTime;
|
||||||
|
const seconds = Math.floor(elapsed / 1000);
|
||||||
|
|
||||||
|
// Update status based on elapsed time
|
||||||
|
if (seconds < 5) {
|
||||||
|
setImageGenerationStatus("Submitting request to AI service...");
|
||||||
|
setImageGenerationProgress(15);
|
||||||
|
} else if (seconds < 15) {
|
||||||
|
setImageGenerationStatus("AI is generating your image...");
|
||||||
|
setImageGenerationProgress(30);
|
||||||
|
} else if (seconds < 30) {
|
||||||
|
setImageGenerationStatus("Creating character-consistent scene image...");
|
||||||
|
setImageGenerationProgress(50);
|
||||||
|
} else if (seconds < 60) {
|
||||||
|
setImageGenerationStatus("Rendering image details...");
|
||||||
|
setImageGenerationProgress(70);
|
||||||
|
} else {
|
||||||
|
setImageGenerationStatus(`Processing... (${seconds}s elapsed)`);
|
||||||
|
setImageGenerationProgress(Math.min(90, 50 + (seconds - 30) / 2));
|
||||||
|
}
|
||||||
|
}, 1000);
|
||||||
|
|
||||||
|
const result = await podcastApi.generateSceneImage({
|
||||||
|
sceneId: scene.id,
|
||||||
|
sceneTitle: scene.title,
|
||||||
|
sceneContent: sceneContent,
|
||||||
|
baseAvatarUrl: avatarUrl || undefined, // Pass base avatar URL for character consistency
|
||||||
|
idea: idea,
|
||||||
|
width: 1024,
|
||||||
|
height: 1024,
|
||||||
|
// Pass custom settings if provided
|
||||||
|
customPrompt: settings?.prompt,
|
||||||
|
style: settings?.style,
|
||||||
|
renderingSpeed: settings?.renderingSpeed,
|
||||||
|
aspectRatio: settings?.aspectRatio,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (progressInterval) {
|
||||||
|
clearInterval(progressInterval);
|
||||||
|
progressInterval = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
setImageGenerationStatus("Finalizing image...");
|
||||||
|
setImageGenerationProgress(95);
|
||||||
|
|
||||||
|
// Update scene with image URL
|
||||||
|
const updatedScene = { ...scene, imageUrl: result.image_url };
|
||||||
|
onUpdateScene(updatedScene);
|
||||||
|
|
||||||
|
const elapsed = Math.floor((Date.now() - startTime) / 1000);
|
||||||
|
setImageGenerationStatus(`Image generated successfully in ${elapsed}s`);
|
||||||
|
setImageGenerationProgress(100);
|
||||||
|
|
||||||
|
// Clear status after a moment
|
||||||
|
setTimeout(() => {
|
||||||
|
setImageGenerationStatus("");
|
||||||
|
setImageGenerationProgress(0);
|
||||||
|
}, 2000);
|
||||||
|
} catch (error: any) {
|
||||||
|
// Clear interval on error
|
||||||
|
if (progressInterval) {
|
||||||
|
clearInterval(progressInterval);
|
||||||
|
progressInterval = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.error("Failed to generate image:", error);
|
||||||
|
// Extract error message from response if available
|
||||||
|
const errorMessage = error?.response?.data?.detail?.message
|
||||||
|
|| error?.response?.data?.detail?.error
|
||||||
|
|| error?.response?.data?.detail
|
||||||
|
|| error?.message
|
||||||
|
|| "Failed to generate image. Please try again.";
|
||||||
|
console.error("Error details:", {
|
||||||
|
status: error?.response?.status,
|
||||||
|
statusText: error?.response?.statusText,
|
||||||
|
data: error?.response?.data,
|
||||||
|
message: errorMessage,
|
||||||
|
});
|
||||||
|
|
||||||
|
setImageGenerationStatus(`Error: ${errorMessage}`);
|
||||||
|
setImageGenerationProgress(0);
|
||||||
|
|
||||||
|
// Show user-friendly error message
|
||||||
|
alert(`Image generation failed: ${errorMessage}`);
|
||||||
|
throw error;
|
||||||
|
} finally {
|
||||||
|
// Ensure interval is cleared
|
||||||
|
if (progressInterval) {
|
||||||
|
clearInterval(progressInterval);
|
||||||
|
}
|
||||||
|
setGeneratingImage(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleRegenerateClick = () => {
|
||||||
|
setShowRegenerateModal(true);
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleAudioRegenerateClick = () => {
|
||||||
|
if (hasAudio) {
|
||||||
|
setShowAudioModal(true);
|
||||||
|
} else {
|
||||||
|
handleApproveAndGenerate(audioSettings);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleAudioRegenerate = (settings: AudioGenerationSettings) => {
|
||||||
|
setAudioSettings(settings);
|
||||||
|
setShowAudioModal(false);
|
||||||
|
handleApproveAndGenerate(settings);
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<GlassyCard sx={glassyCardSx}>
|
||||||
|
<Stack spacing={2.5}>
|
||||||
|
<Stack direction="row" justifyContent="space-between" alignItems="flex-start">
|
||||||
|
<Box sx={{ flex: 1 }}>
|
||||||
|
<Typography
|
||||||
|
variant="h6"
|
||||||
|
sx={{
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
gap: 1.5,
|
||||||
|
mb: 1,
|
||||||
|
color: "#0f172a",
|
||||||
|
fontWeight: 600,
|
||||||
|
fontSize: "1.25rem",
|
||||||
|
letterSpacing: "-0.01em",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<EditNoteIcon fontSize="small" sx={{ color: "#667eea", fontSize: "1.5rem" }} />
|
||||||
|
{scene.title}
|
||||||
|
</Typography>
|
||||||
|
<Stack direction="row" spacing={1.5} alignItems="center" flexWrap="wrap">
|
||||||
|
<Chip
|
||||||
|
icon={scene.approved ? <CheckCircleIcon /> : <RadioButtonUncheckedIcon />}
|
||||||
|
label={scene.approved ? "Approved" : "Pending Approval"}
|
||||||
|
size="small"
|
||||||
|
color={scene.approved ? "success" : "warning"}
|
||||||
|
sx={{
|
||||||
|
background: scene.approved
|
||||||
|
? "linear-gradient(135deg, rgba(16, 185, 129, 0.12) 0%, rgba(5, 150, 105, 0.12) 100%)"
|
||||||
|
: "linear-gradient(135deg, rgba(245, 158, 11, 0.12) 0%, rgba(217, 119, 6, 0.12) 100%)",
|
||||||
|
color: scene.approved ? "#059669" : "#d97706",
|
||||||
|
border: scene.approved
|
||||||
|
? "1px solid rgba(16, 185, 129, 0.25)"
|
||||||
|
: "1px solid rgba(245, 158, 11, 0.25)",
|
||||||
|
fontWeight: 600,
|
||||||
|
fontSize: "0.75rem",
|
||||||
|
height: 26,
|
||||||
|
boxShadow: "0 1px 2px rgba(0, 0, 0, 0.05)",
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<Typography variant="caption" sx={{ color: "#64748b", fontWeight: 500, fontSize: "0.8125rem" }}>
|
||||||
|
Duration: {scene.duration}s
|
||||||
|
</Typography>
|
||||||
|
</Stack>
|
||||||
|
</Box>
|
||||||
|
<Stack direction="row" spacing={1.5} flexWrap="wrap" useFlexGap>
|
||||||
|
<PrimaryButton
|
||||||
|
onClick={handleAudioRegenerateClick}
|
||||||
|
disabled={approving || generating}
|
||||||
|
loading={approving || generating}
|
||||||
|
startIcon={
|
||||||
|
hasAudio && !generating ? (
|
||||||
|
<VolumeUpIcon />
|
||||||
|
) : generating ? (
|
||||||
|
<CircularProgress size={16} sx={{ color: "white" }} />
|
||||||
|
) : (
|
||||||
|
<PlayArrowIcon />
|
||||||
|
)
|
||||||
|
}
|
||||||
|
tooltip={
|
||||||
|
hasAudio && !generating
|
||||||
|
? "Regenerate audio for this scene with custom settings"
|
||||||
|
: generating
|
||||||
|
? "Generating audio..."
|
||||||
|
: scene.approved
|
||||||
|
? "Generate audio for this scene"
|
||||||
|
: "Approve scene and generate audio"
|
||||||
|
}
|
||||||
|
sx={{
|
||||||
|
minWidth: 200,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{hasAudio && !generating
|
||||||
|
? "Regenerate Audio"
|
||||||
|
: generating
|
||||||
|
? "Generating Audio..."
|
||||||
|
: scene.approved
|
||||||
|
? "Generate Audio"
|
||||||
|
: "Approve & Generate Audio"}
|
||||||
|
</PrimaryButton>
|
||||||
|
<PrimaryButton
|
||||||
|
onClick={hasImage ? handleRegenerateClick : () => handleGenerateImage()}
|
||||||
|
disabled={generatingImage}
|
||||||
|
loading={generatingImage}
|
||||||
|
startIcon={
|
||||||
|
hasImage && !generatingImage ? (
|
||||||
|
<ImageIcon />
|
||||||
|
) : generatingImage ? (
|
||||||
|
<CircularProgress size={16} sx={{ color: "white" }} />
|
||||||
|
) : (
|
||||||
|
<ImageIcon />
|
||||||
|
)
|
||||||
|
}
|
||||||
|
tooltip={
|
||||||
|
hasImage
|
||||||
|
? "Regenerate image for this scene"
|
||||||
|
: generatingImage
|
||||||
|
? "Generating image..."
|
||||||
|
: "Generate image for video (optional)"
|
||||||
|
}
|
||||||
|
sx={{
|
||||||
|
minWidth: 180,
|
||||||
|
background: hasImage
|
||||||
|
? "linear-gradient(135deg, #10b981 0%, #059669 100%)"
|
||||||
|
: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||||
|
"&:hover": {
|
||||||
|
background: hasImage
|
||||||
|
? "linear-gradient(135deg, #059669 0%, #047857 100%)"
|
||||||
|
: "linear-gradient(135deg, #764ba2 0%, #667eea 100%)",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{hasImage && !generatingImage
|
||||||
|
? "Regenerate Image"
|
||||||
|
: generatingImage
|
||||||
|
? "Generating Image..."
|
||||||
|
: "Generate Image"}
|
||||||
|
</PrimaryButton>
|
||||||
|
|
||||||
|
<Tooltip title={totalScenes && totalScenes <= 1 ? "Cannot delete the last scene" : "Delete this scene"}>
|
||||||
|
<IconButton
|
||||||
|
onClick={() => onDelete(scene.id)}
|
||||||
|
disabled={approving || generating || (totalScenes !== undefined && totalScenes <= 1)}
|
||||||
|
sx={{
|
||||||
|
color: "#ef4444",
|
||||||
|
backgroundColor: "rgba(239, 68, 68, 0.1)",
|
||||||
|
border: "1px solid rgba(239, 68, 68, 0.2)",
|
||||||
|
borderRadius: 2,
|
||||||
|
padding: 1.5,
|
||||||
|
"&:hover": {
|
||||||
|
backgroundColor: "rgba(239, 68, 68, 0.15)",
|
||||||
|
borderColor: "rgba(239, 68, 68, 0.3)",
|
||||||
|
},
|
||||||
|
"&:disabled": {
|
||||||
|
backgroundColor: "rgba(156, 163, 175, 0.1)",
|
||||||
|
borderColor: "rgba(156, 163, 175, 0.2)",
|
||||||
|
color: "#9ca3af",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<DeleteIcon sx={{ fontSize: "1.25rem" }} />
|
||||||
|
</IconButton>
|
||||||
|
</Tooltip>
|
||||||
|
</Stack>
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1 }} />
|
||||||
|
|
||||||
|
<Stack spacing={2}>
|
||||||
|
{scene.lines.map((line) => (
|
||||||
|
<LineEditor key={line.id} line={line} onChange={updateLine} />
|
||||||
|
))}
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
{scene.audioUrl && (
|
||||||
|
<>
|
||||||
|
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1, mt: 1 }} />
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
p: 2,
|
||||||
|
background: hasAudio
|
||||||
|
? "linear-gradient(135deg, rgba(16, 185, 129, 0.08) 0%, rgba(5, 150, 105, 0.08) 100%)"
|
||||||
|
: "linear-gradient(135deg, rgba(245, 158, 11, 0.08) 0%, rgba(217, 119, 6, 0.08) 100%)",
|
||||||
|
borderRadius: 2,
|
||||||
|
border: hasAudio
|
||||||
|
? "1px solid rgba(16, 185, 129, 0.2)"
|
||||||
|
: "1px solid rgba(245, 158, 11, 0.2)",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" alignItems="center" spacing={1.5} sx={{ mb: 1.5 }}>
|
||||||
|
<VolumeUpIcon sx={{ color: hasAudio ? "#059669" : "#d97706", fontSize: "1.25rem" }} />
|
||||||
|
<Typography variant="subtitle2" sx={{ color: hasAudio ? "#059669" : "#d97706", fontWeight: 600 }}>
|
||||||
|
{hasAudio ? "Audio Generated" : "Loading Audio..."}
|
||||||
|
</Typography>
|
||||||
|
</Stack>
|
||||||
|
{hasAudio && audioBlobUrl ? (
|
||||||
|
<audio controls style={{ width: "100%", borderRadius: 8 }}>
|
||||||
|
<source src={audioBlobUrl} type="audio/mpeg" />
|
||||||
|
Your browser does not support the audio element.
|
||||||
|
</audio>
|
||||||
|
) : (
|
||||||
|
<Box sx={{ display: "flex", alignItems: "center", justifyContent: "center", py: 2 }}>
|
||||||
|
<CircularProgress size={24} sx={{ color: "#d97706" }} />
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Image Generation Progress - Show when generating */}
|
||||||
|
{generatingImage && (
|
||||||
|
<>
|
||||||
|
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1, mt: 1 }} />
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
p: 2,
|
||||||
|
background: "linear-gradient(135deg, rgba(102, 126, 234, 0.08) 0%, rgba(118, 75, 162, 0.08) 100%)",
|
||||||
|
borderRadius: 2,
|
||||||
|
border: "1px solid rgba(102, 126, 234, 0.2)",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" alignItems="center" spacing={1.5} sx={{ mb: 1.5 }}>
|
||||||
|
<ImageIcon sx={{ color: "#667eea", fontSize: "1.25rem" }} />
|
||||||
|
<Typography variant="subtitle2" sx={{ color: "#667eea", fontWeight: 600 }}>
|
||||||
|
Generating Image...
|
||||||
|
</Typography>
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
{/* Progress Bar */}
|
||||||
|
<Box sx={{ mb: 1.5 }}>
|
||||||
|
<LinearProgress
|
||||||
|
variant="determinate"
|
||||||
|
value={imageGenerationProgress}
|
||||||
|
sx={{
|
||||||
|
height: 8,
|
||||||
|
borderRadius: 4,
|
||||||
|
backgroundColor: alpha("#667eea", 0.1),
|
||||||
|
"& .MuiLinearProgress-bar": {
|
||||||
|
backgroundColor: "#667eea",
|
||||||
|
borderRadius: 4,
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<Typography variant="caption" sx={{ color: "#667eea", mt: 0.5, display: "block", textAlign: "right" }}>
|
||||||
|
{imageGenerationProgress}%
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
{/* Status Message */}
|
||||||
|
{imageGenerationStatus && (
|
||||||
|
<Typography variant="body2" sx={{ color: "#667eea", fontSize: "0.875rem", lineHeight: 1.6, mb: 1 }}>
|
||||||
|
{imageGenerationStatus}
|
||||||
|
</Typography>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Spinner */}
|
||||||
|
<Box sx={{ display: "flex", alignItems: "center", justifyContent: "center", mt: 1 }}>
|
||||||
|
<CircularProgress size={32} sx={{ color: "#667eea" }} />
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Generated Image Display - Show when image exists and not generating */}
|
||||||
|
{scene.imageUrl && !generatingImage && (
|
||||||
|
<>
|
||||||
|
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1, mt: 1 }} />
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
p: 2,
|
||||||
|
background: imageBlobUrl && !imageLoading
|
||||||
|
? "linear-gradient(135deg, rgba(102, 126, 234, 0.08) 0%, rgba(118, 75, 162, 0.08) 100%)"
|
||||||
|
: "linear-gradient(135deg, rgba(245, 158, 11, 0.08) 0%, rgba(217, 119, 6, 0.08) 100%)",
|
||||||
|
borderRadius: 2,
|
||||||
|
border: imageBlobUrl && !imageLoading
|
||||||
|
? "1px solid rgba(102, 126, 234, 0.2)"
|
||||||
|
: "1px solid rgba(245, 158, 11, 0.2)",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" alignItems="center" spacing={1.5} sx={{ mb: 1.5 }}>
|
||||||
|
<ImageIcon sx={{ color: imageBlobUrl && !imageLoading ? "#667eea" : "#d97706", fontSize: "1.25rem" }} />
|
||||||
|
<Typography variant="subtitle2" sx={{ color: imageBlobUrl && !imageLoading ? "#667eea" : "#d97706", fontWeight: 600 }}>
|
||||||
|
{imageBlobUrl && !imageLoading ? "Image Generated" : "Loading Image..."}
|
||||||
|
</Typography>
|
||||||
|
</Stack>
|
||||||
|
{imageBlobUrl && !imageLoading ? (
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
width: "100%",
|
||||||
|
borderRadius: 2,
|
||||||
|
overflow: "hidden",
|
||||||
|
border: "1px solid rgba(102,126,234,0.2)",
|
||||||
|
background: alpha("#667eea", 0.05),
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Box
|
||||||
|
component="img"
|
||||||
|
src={imageBlobUrl}
|
||||||
|
alt={scene.title}
|
||||||
|
sx={{
|
||||||
|
width: "100%",
|
||||||
|
height: "auto",
|
||||||
|
display: "block",
|
||||||
|
maxHeight: 400,
|
||||||
|
objectFit: "cover",
|
||||||
|
}}
|
||||||
|
onError={(e) => {
|
||||||
|
console.error('[SceneEditor] Image failed to load:', {
|
||||||
|
src: e.currentTarget.src,
|
||||||
|
imageUrl: scene.imageUrl,
|
||||||
|
imageBlobUrl,
|
||||||
|
});
|
||||||
|
}}
|
||||||
|
onLoad={() => {
|
||||||
|
console.log('[SceneEditor] Image loaded successfully');
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
</Box>
|
||||||
|
) : (
|
||||||
|
<Box sx={{ display: "flex", alignItems: "center", justifyContent: "center", py: 2 }}>
|
||||||
|
<CircularProgress size={24} sx={{ color: "#d97706" }} />
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
{/* Image Regeneration Modal */}
|
||||||
|
<ImageRegenerateModal
|
||||||
|
open={showRegenerateModal}
|
||||||
|
onClose={() => setShowRegenerateModal(false)}
|
||||||
|
onRegenerate={handleGenerateImage}
|
||||||
|
initialPrompt={(() => {
|
||||||
|
const promptParts = [
|
||||||
|
`Scene: ${scene.title}`,
|
||||||
|
"Professional podcast recording studio",
|
||||||
|
"Modern microphone setup",
|
||||||
|
"Clean background, professional lighting",
|
||||||
|
"16:9 aspect ratio, video-optimized composition"
|
||||||
|
];
|
||||||
|
if (idea) {
|
||||||
|
promptParts.push(`Topic: ${idea.substring(0, 60)}`);
|
||||||
|
}
|
||||||
|
return promptParts.join(", ");
|
||||||
|
})()}
|
||||||
|
initialStyle="Realistic"
|
||||||
|
initialRenderingSpeed="Quality"
|
||||||
|
initialAspectRatio="16:9"
|
||||||
|
isGenerating={generatingImage}
|
||||||
|
/>
|
||||||
|
|
||||||
|
<AudioRegenerateModal
|
||||||
|
open={showAudioModal}
|
||||||
|
onClose={() => setShowAudioModal(false)}
|
||||||
|
onRegenerate={handleAudioRegenerate}
|
||||||
|
initialSettings={audioSettings}
|
||||||
|
isGenerating={generating}
|
||||||
|
/>
|
||||||
|
</GlassyCard>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
818
_session_backup/ScriptEditor.tsx
Normal file
818
_session_backup/ScriptEditor.tsx
Normal file
@@ -0,0 +1,818 @@
|
|||||||
|
import React, { useEffect, useState, useCallback } from "react";
|
||||||
|
import { Box, Stack, Typography, Alert, Paper, LinearProgress, CircularProgress, alpha, Collapse, IconButton, Divider } from "@mui/material";
|
||||||
|
import { EditNote as EditNoteIcon, CheckCircle as CheckCircleIcon, PlayArrow as PlayArrowIcon, ArrowBack as ArrowBackIcon, Info as InfoIcon, ExpandMore as ExpandMoreIcon, ExpandLess as ExpandLessIcon, Download as DownloadIcon, Refresh as RefreshIcon } from "@mui/icons-material";
|
||||||
|
import { Script, Knobs, Scene } from "../types";
|
||||||
|
import { BlogResearchResponse } from "../../../services/blogWriterApi";
|
||||||
|
import { podcastApi } from "../../../services/podcastApi";
|
||||||
|
import { GlassyCard, PrimaryButton, SecondaryButton } from "../ui";
|
||||||
|
import { SceneEditor } from "./SceneEditor";
|
||||||
|
import { InlineAudioPlayer } from "../InlineAudioPlayer";
|
||||||
|
import { aiApiClient } from "../../../api/client";
|
||||||
|
|
||||||
|
interface ScriptEditorProps {
|
||||||
|
projectId: string;
|
||||||
|
idea: string;
|
||||||
|
research: any; // Research type
|
||||||
|
rawResearch: BlogResearchResponse | null;
|
||||||
|
knobs: Knobs;
|
||||||
|
speakers: number;
|
||||||
|
durationMinutes: number;
|
||||||
|
script: Script | null;
|
||||||
|
onScriptChange: (script: Script) => void;
|
||||||
|
onBackToResearch: () => void;
|
||||||
|
onProceedToRendering: (script: Script) => void;
|
||||||
|
onError: (message: string) => void;
|
||||||
|
avatarUrl?: string | null; // Base avatar URL for consistent scene image generation
|
||||||
|
analysis?: any;
|
||||||
|
outline?: any;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const ScriptEditor: React.FC<ScriptEditorProps> = ({
|
||||||
|
projectId,
|
||||||
|
idea,
|
||||||
|
research,
|
||||||
|
rawResearch,
|
||||||
|
knobs,
|
||||||
|
speakers,
|
||||||
|
durationMinutes,
|
||||||
|
script: initialScript,
|
||||||
|
onScriptChange,
|
||||||
|
onBackToResearch,
|
||||||
|
onProceedToRendering,
|
||||||
|
onError,
|
||||||
|
avatarUrl,
|
||||||
|
analysis,
|
||||||
|
outline,
|
||||||
|
}) => {
|
||||||
|
const [script, setScript] = useState<Script | null>(initialScript);
|
||||||
|
const [loading, setLoading] = useState(false);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
const [approvingSceneId, setApprovingSceneId] = useState<string | null>(null);
|
||||||
|
const [generatingAudioId, setGeneratingAudioId] = useState<string | null>(null);
|
||||||
|
const [showScriptFormatInfo, setShowScriptFormatInfo] = useState(true);
|
||||||
|
const [combiningAudio, setCombiningAudio] = useState(false);
|
||||||
|
const [combinedAudioResult, setCombinedAudioResult] = useState<{
|
||||||
|
url: string;
|
||||||
|
filename: string;
|
||||||
|
duration: number;
|
||||||
|
sceneCount: number;
|
||||||
|
} | null>(null);
|
||||||
|
|
||||||
|
// Defer upward script updates to avoid setState during render warnings
|
||||||
|
const emitScriptChange = useCallback(
|
||||||
|
(next: Script) => Promise.resolve().then(() => onScriptChange(next)),
|
||||||
|
[onScriptChange]
|
||||||
|
);
|
||||||
|
|
||||||
|
// Sync with parent state
|
||||||
|
useEffect(() => {
|
||||||
|
if (initialScript) {
|
||||||
|
setScript(initialScript);
|
||||||
|
}
|
||||||
|
}, [initialScript]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
// If script already exists, don't regenerate
|
||||||
|
if (script) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only generate if we have research data
|
||||||
|
if (!rawResearch) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
let mounted = true;
|
||||||
|
setLoading(true);
|
||||||
|
setError(null);
|
||||||
|
podcastApi
|
||||||
|
.generateScript({
|
||||||
|
projectId,
|
||||||
|
idea,
|
||||||
|
research: rawResearch,
|
||||||
|
knobs,
|
||||||
|
speakers,
|
||||||
|
durationMinutes,
|
||||||
|
analysis,
|
||||||
|
outline,
|
||||||
|
})
|
||||||
|
.then((res) => {
|
||||||
|
if (mounted) {
|
||||||
|
setScript(res);
|
||||||
|
emitScriptChange(res);
|
||||||
|
setError(null);
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch((err) => {
|
||||||
|
const message = err instanceof Error ? err.message : "Failed to generate script";
|
||||||
|
setError(message);
|
||||||
|
onError(message);
|
||||||
|
})
|
||||||
|
.finally(() => mounted && setLoading(false));
|
||||||
|
return () => {
|
||||||
|
mounted = false;
|
||||||
|
};
|
||||||
|
}, [projectId, rawResearch, idea, knobs, speakers, durationMinutes, analysis, outline, emitScriptChange, onError, script]);
|
||||||
|
|
||||||
|
const updateScene = (updated: Scene) => {
|
||||||
|
// Use functional update to ensure we're working with latest state
|
||||||
|
setScript((currentScript) => {
|
||||||
|
if (!currentScript) return currentScript;
|
||||||
|
const updatedScript = {
|
||||||
|
...currentScript,
|
||||||
|
scenes: currentScript.scenes.map((s) => (s.id === updated.id ? { ...s, ...updated } : s))
|
||||||
|
};
|
||||||
|
emitScriptChange(updatedScript);
|
||||||
|
return updatedScript;
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const approveScene = async (sceneId: string) => {
|
||||||
|
try {
|
||||||
|
setApprovingSceneId(sceneId);
|
||||||
|
await podcastApi.approveScene({ projectId, sceneId });
|
||||||
|
// Use functional update to ensure we're working with latest state
|
||||||
|
setScript((currentScript) => {
|
||||||
|
if (!currentScript) return currentScript;
|
||||||
|
const updatedScript = {
|
||||||
|
...currentScript,
|
||||||
|
scenes: currentScript.scenes.map((s) => (s.id === sceneId ? { ...s, approved: true } : s)),
|
||||||
|
};
|
||||||
|
emitScriptChange(updatedScript);
|
||||||
|
return updatedScript;
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
const message = err instanceof Error ? err.message : "Failed to approve scene";
|
||||||
|
setError(message);
|
||||||
|
onError(message);
|
||||||
|
throw err;
|
||||||
|
} finally {
|
||||||
|
setApprovingSceneId((current) => (current === sceneId ? null : current));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const deleteScene = useCallback((sceneId: string) => {
|
||||||
|
if (!script) return;
|
||||||
|
|
||||||
|
// Prevent deleting if it's the last scene
|
||||||
|
if (script.scenes.length <= 1) {
|
||||||
|
onError("Cannot delete the last scene. At least one scene is required.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add confirmation dialog
|
||||||
|
const sceneToDelete = script.scenes.find(s => s.id === sceneId);
|
||||||
|
if (!sceneToDelete) return;
|
||||||
|
|
||||||
|
const confirmDelete = window.confirm(
|
||||||
|
`Are you sure you want to delete "${sceneToDelete.title}"? This action cannot be undone.`
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!confirmDelete) return;
|
||||||
|
|
||||||
|
// Remove the scene from the script
|
||||||
|
const updatedScenes = script.scenes.filter(s => s.id !== sceneId);
|
||||||
|
const updatedScript = { ...script, scenes: updatedScenes };
|
||||||
|
|
||||||
|
emitScriptChange(updatedScript);
|
||||||
|
setScript(updatedScript);
|
||||||
|
|
||||||
|
// Show success message
|
||||||
|
console.log(`[ScriptEditor] Scene "${sceneToDelete.title}" deleted successfully`);
|
||||||
|
}, [script, emitScriptChange, onError]);
|
||||||
|
|
||||||
|
const allApproved = script && script.scenes.every((s) => s.approved);
|
||||||
|
const approvedCount = script ? script.scenes.filter((s) => s.approved).length : 0;
|
||||||
|
const totalScenes = script ? script.scenes.length : 0;
|
||||||
|
|
||||||
|
// Check if all scenes have both audio and images (required for video rendering)
|
||||||
|
const allScenesHaveAudioAndImages = script && script.scenes.every((s) => s.audioUrl && s.imageUrl);
|
||||||
|
const scenesWithAudio = script ? script.scenes.filter((s) => s.audioUrl).length : 0;
|
||||||
|
const allScenesHaveAudio = script && script.scenes.every((s) => s.audioUrl);
|
||||||
|
|
||||||
|
const combineAudio = useCallback(async () => {
|
||||||
|
if (!script || !projectId) return;
|
||||||
|
|
||||||
|
try {
|
||||||
|
setCombiningAudio(true);
|
||||||
|
|
||||||
|
const sceneIds: string[] = [];
|
||||||
|
const sceneAudioUrls: string[] = [];
|
||||||
|
|
||||||
|
script.scenes.forEach((scene) => {
|
||||||
|
if (scene.audioUrl) {
|
||||||
|
// Ensure we're using the correct URL format (not blob URLs)
|
||||||
|
const audioUrl = scene.audioUrl.startsWith('blob:') ? '' : scene.audioUrl;
|
||||||
|
if (audioUrl) {
|
||||||
|
sceneIds.push(scene.id);
|
||||||
|
sceneAudioUrls.push(audioUrl);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
if (sceneIds.length === 0) {
|
||||||
|
onError("No audio files found to combine.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await podcastApi.combineAudio({
|
||||||
|
projectId,
|
||||||
|
sceneIds,
|
||||||
|
sceneAudioUrls,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Store combined audio result for preview
|
||||||
|
setCombinedAudioResult({
|
||||||
|
url: result.combined_audio_url,
|
||||||
|
filename: result.combined_audio_filename,
|
||||||
|
duration: result.total_duration,
|
||||||
|
sceneCount: result.scene_count,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Download the combined audio as blob (for authenticated endpoints)
|
||||||
|
try {
|
||||||
|
// Normalize path
|
||||||
|
let audioPath = result.combined_audio_url.startsWith('/')
|
||||||
|
? result.combined_audio_url
|
||||||
|
: `/${result.combined_audio_url}`;
|
||||||
|
|
||||||
|
// Ensure it's a podcast audio endpoint
|
||||||
|
if (!audioPath.includes('/api/podcast/audio/')) {
|
||||||
|
const filename = audioPath.split('/').pop() || result.combined_audio_filename;
|
||||||
|
audioPath = `/api/podcast/audio/${filename}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove query parameters if present
|
||||||
|
audioPath = audioPath.split('?')[0];
|
||||||
|
|
||||||
|
// Fetch as blob using authenticated client
|
||||||
|
const response = await aiApiClient.get(audioPath, {
|
||||||
|
responseType: 'blob',
|
||||||
|
});
|
||||||
|
|
||||||
|
// Create blob URL and download
|
||||||
|
const blob = response.data;
|
||||||
|
const blobUrl = URL.createObjectURL(blob);
|
||||||
|
|
||||||
|
const link = document.createElement("a");
|
||||||
|
link.href = blobUrl;
|
||||||
|
link.download = result.combined_audio_filename || `podcast-episode-${projectId.slice(-8)}.mp3`;
|
||||||
|
document.body.appendChild(link);
|
||||||
|
link.click();
|
||||||
|
document.body.removeChild(link);
|
||||||
|
|
||||||
|
// Clean up blob URL after a delay
|
||||||
|
setTimeout(() => {
|
||||||
|
URL.revokeObjectURL(blobUrl);
|
||||||
|
}, 100);
|
||||||
|
} catch (downloadError) {
|
||||||
|
console.error('Failed to download combined audio:', downloadError);
|
||||||
|
onError('Failed to download audio file. You can try downloading again from the preview.');
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : "Failed to combine audio";
|
||||||
|
onError(`Failed to combine audio: ${message}`);
|
||||||
|
} finally {
|
||||||
|
setCombiningAudio(false);
|
||||||
|
}
|
||||||
|
}, [script, projectId, onError]);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Box sx={{ mt: 4 }}>
|
||||||
|
<Stack direction="row" spacing={2} alignItems="center" sx={{ mb: 4 }}>
|
||||||
|
<SecondaryButton onClick={onBackToResearch} startIcon={<ArrowBackIcon />}>
|
||||||
|
Back to Research
|
||||||
|
</SecondaryButton>
|
||||||
|
<Box sx={{ flex: 1 }}>
|
||||||
|
<Typography
|
||||||
|
variant="h4"
|
||||||
|
sx={{
|
||||||
|
background: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||||
|
WebkitBackgroundClip: "text",
|
||||||
|
WebkitTextFillColor: "transparent",
|
||||||
|
fontWeight: 700,
|
||||||
|
letterSpacing: "-0.02em",
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
gap: 1.5,
|
||||||
|
fontSize: { xs: "1.75rem", md: "2rem" },
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<EditNoteIcon sx={{ fontSize: "2rem" }} />
|
||||||
|
Script Editor
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#64748b", mt: 0.5, ml: 5.5 }}>
|
||||||
|
Review and refine your podcast script before rendering
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
{loading && (
|
||||||
|
<Alert
|
||||||
|
severity="info"
|
||||||
|
icon={<CircularProgress size={20} />}
|
||||||
|
sx={{
|
||||||
|
mb: 3,
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.08) 0%, rgba(139, 92, 246, 0.08) 100%)",
|
||||||
|
border: "1px solid rgba(99, 102, 241, 0.2)",
|
||||||
|
borderRadius: 2,
|
||||||
|
boxShadow: "0 1px 2px rgba(99, 102, 241, 0.05)",
|
||||||
|
"& .MuiAlert-icon": {
|
||||||
|
color: "#6366f1",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#0f172a", fontWeight: 500 }}>
|
||||||
|
Generating script with AI... This may take a moment.
|
||||||
|
</Typography>
|
||||||
|
</Alert>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{error && (
|
||||||
|
<Alert
|
||||||
|
severity="error"
|
||||||
|
sx={{
|
||||||
|
mb: 3,
|
||||||
|
background: "linear-gradient(135deg, rgba(239, 68, 68, 0.08) 0%, rgba(220, 38, 38, 0.08) 100%)",
|
||||||
|
border: "1px solid rgba(239, 68, 68, 0.2)",
|
||||||
|
borderRadius: 2,
|
||||||
|
boxShadow: "0 1px 2px rgba(239, 68, 68, 0.05)",
|
||||||
|
"& .MuiAlert-icon": {
|
||||||
|
color: "#ef4444",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#0f172a", fontWeight: 500 }}>
|
||||||
|
{error}
|
||||||
|
</Typography>
|
||||||
|
</Alert>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{script && (
|
||||||
|
<Stack spacing={3}>
|
||||||
|
{/* Script Format Explanation Panel */}
|
||||||
|
<Paper
|
||||||
|
sx={{
|
||||||
|
p: 3,
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.05) 0%, rgba(139, 92, 246, 0.05) 100%)",
|
||||||
|
border: "1px solid rgba(99, 102, 241, 0.15)",
|
||||||
|
borderRadius: 2,
|
||||||
|
boxShadow: "0 2px 8px rgba(99, 102, 241, 0.08)",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" alignItems="center" justifyContent="space-between" sx={{ mb: showScriptFormatInfo ? 2 : 0 }}>
|
||||||
|
<Stack direction="row" alignItems="center" spacing={1.5}>
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
width: 40,
|
||||||
|
height: 40,
|
||||||
|
borderRadius: "50%",
|
||||||
|
background: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
boxShadow: "0 2px 8px rgba(102, 126, 234, 0.3)",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<InfoIcon sx={{ color: "#ffffff", fontSize: "1.5rem" }} />
|
||||||
|
</Box>
|
||||||
|
<Box>
|
||||||
|
<Typography variant="h6" sx={{ color: "#0f172a", fontWeight: 600, fontSize: "1.1rem" }}>
|
||||||
|
Why This Script Format?
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#64748b", mt: 0.25 }}>
|
||||||
|
Understanding how your script creates natural, human-like audio
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
</Stack>
|
||||||
|
<IconButton
|
||||||
|
onClick={() => setShowScriptFormatInfo(!showScriptFormatInfo)}
|
||||||
|
sx={{
|
||||||
|
color: "#6366f1",
|
||||||
|
"&:hover": {
|
||||||
|
background: "rgba(99, 102, 241, 0.1)",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{showScriptFormatInfo ? <ExpandLessIcon /> : <ExpandMoreIcon />}
|
||||||
|
</IconButton>
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
<Collapse in={showScriptFormatInfo}>
|
||||||
|
<Stack spacing={2.5}>
|
||||||
|
<Box>
|
||||||
|
<Typography variant="body2" sx={{ color: "#0f172a", lineHeight: 1.8, mb: 2 }}>
|
||||||
|
Our AI script generator creates scripts specifically optimized for <strong style={{ fontWeight: 600 }}>high-quality text-to-speech</strong>.
|
||||||
|
The format you see here is designed to produce audio that sounds natural and human-like, not robotic.
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
<Stack spacing={2}>
|
||||||
|
<Box sx={{ display: "flex", gap: 2 }}>
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
minWidth: 32,
|
||||||
|
height: 32,
|
||||||
|
borderRadius: "8px",
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
flexShrink: 0,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||||
|
1
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
<Box>
|
||||||
|
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||||
|
Natural Pauses & Rhythm
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||||
|
The script includes strategic pauses between lines and when speakers change. This creates natural breathing patterns
|
||||||
|
and conversation flow, just like real human speech. Without these pauses, the audio would sound rushed and robotic.
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
<Box sx={{ display: "flex", gap: 2 }}>
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
minWidth: 32,
|
||||||
|
height: 32,
|
||||||
|
borderRadius: "8px",
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
flexShrink: 0,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||||
|
2
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
<Box>
|
||||||
|
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||||
|
Emphasis Markers
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||||
|
Lines marked with emphasis help highlight important points, statistics, or key insights. The AI voice will naturally
|
||||||
|
stress these parts, making your podcast more engaging and easier to follow—just like a real host would emphasize important information.
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
<Box sx={{ display: "flex", gap: 2 }}>
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
minWidth: 32,
|
||||||
|
height: 32,
|
||||||
|
borderRadius: "8px",
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
flexShrink: 0,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||||
|
3
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
<Box>
|
||||||
|
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||||
|
Short, Conversational Sentences
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||||
|
The script uses shorter sentences (15-20 words) written in a conversational style. This matches how people actually
|
||||||
|
speak, making the audio sound more natural. Long, complex sentences would sound awkward when spoken aloud.
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
<Box sx={{ display: "flex", gap: 2 }}>
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
minWidth: 32,
|
||||||
|
height: 32,
|
||||||
|
borderRadius: "8px",
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
flexShrink: 0,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||||
|
4
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
<Box>
|
||||||
|
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||||
|
Scene-Specific Emotions
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||||
|
Each scene has an emotional tone (excited, serious, curious, etc.) that guides the AI voice's delivery. This creates
|
||||||
|
variety and keeps listeners engaged, just like a real podcast host would vary their tone based on the topic.
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
<Box sx={{ display: "flex", gap: 2 }}>
|
||||||
|
<Box
|
||||||
|
sx={{
|
||||||
|
minWidth: 32,
|
||||||
|
height: 32,
|
||||||
|
borderRadius: "8px",
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||||
|
display: "flex",
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
flexShrink: 0,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||||
|
5
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
<Box>
|
||||||
|
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||||
|
Optimized for Podcast Narration
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||||
|
The script is optimized with slightly slower pacing and natural pronunciation settings specifically for podcast narration.
|
||||||
|
This ensures clarity and makes the content easy to understand, even when listeners are multitasking.
|
||||||
|
</Typography>
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
<Alert
|
||||||
|
severity="info"
|
||||||
|
sx={{
|
||||||
|
mt: 1,
|
||||||
|
background: "rgba(99, 102, 241, 0.06)",
|
||||||
|
border: "1px solid rgba(99, 102, 241, 0.15)",
|
||||||
|
"& .MuiAlert-icon": {
|
||||||
|
color: "#6366f1",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#0f172a", lineHeight: 1.7 }}>
|
||||||
|
<strong style={{ fontWeight: 600 }}>Tip:</strong> You can edit any line or scene to match your preferences.
|
||||||
|
The format will be preserved when rendering, ensuring your audio still sounds natural and professional.
|
||||||
|
</Typography>
|
||||||
|
</Alert>
|
||||||
|
</Stack>
|
||||||
|
</Collapse>
|
||||||
|
</Paper>
|
||||||
|
|
||||||
|
<Alert
|
||||||
|
severity="info"
|
||||||
|
sx={{
|
||||||
|
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.08) 0%, rgba(139, 92, 246, 0.08) 100%)",
|
||||||
|
border: "1px solid rgba(99, 102, 241, 0.2)",
|
||||||
|
borderRadius: 2,
|
||||||
|
boxShadow: "0 1px 2px rgba(99, 102, 241, 0.05)",
|
||||||
|
"& .MuiAlert-icon": {
|
||||||
|
color: "#6366f1",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#0f172a", fontWeight: 500, lineHeight: 1.6 }}>
|
||||||
|
<strong style={{ fontWeight: 600 }}>Approval Required:</strong> Each scene must be approved before rendering. Review and edit lines as needed, then approve each scene.
|
||||||
|
</Typography>
|
||||||
|
</Alert>
|
||||||
|
|
||||||
|
<Stack spacing={2}>
|
||||||
|
{script.scenes.map((scene, idx) => (
|
||||||
|
<GlassyCard
|
||||||
|
key={scene.id}
|
||||||
|
initial={{ opacity: 0, y: 8 }}
|
||||||
|
animate={{ opacity: 1, y: 0 }}
|
||||||
|
transition={{ duration: 0.3, delay: idx * 0.1 }}
|
||||||
|
>
|
||||||
|
<SceneEditor
|
||||||
|
scene={scene}
|
||||||
|
onUpdateScene={updateScene}
|
||||||
|
onApprove={approveScene}
|
||||||
|
onDelete={deleteScene}
|
||||||
|
knobs={knobs}
|
||||||
|
approvingSceneId={approvingSceneId}
|
||||||
|
generatingAudioId={generatingAudioId}
|
||||||
|
totalScenes={script.scenes.length}
|
||||||
|
onAudioGenerationStart={(sceneId) => {
|
||||||
|
setGeneratingAudioId(sceneId);
|
||||||
|
}}
|
||||||
|
onAudioGenerated={async (sceneId, audioUrl) => {
|
||||||
|
setGeneratingAudioId(null);
|
||||||
|
// Use functional update to ensure we're working with latest state
|
||||||
|
// Ensure scene is marked as approved and has audioUrl
|
||||||
|
setScript((currentScript) => {
|
||||||
|
if (!currentScript) return currentScript;
|
||||||
|
const updatedScenes = currentScript.scenes.map((s) =>
|
||||||
|
s.id === sceneId ? { ...s, audioUrl, approved: true } : s
|
||||||
|
);
|
||||||
|
const updatedScript = { ...currentScript, scenes: updatedScenes };
|
||||||
|
emitScriptChange(updatedScript);
|
||||||
|
return updatedScript;
|
||||||
|
});
|
||||||
|
}}
|
||||||
|
idea={idea}
|
||||||
|
avatarUrl={avatarUrl}
|
||||||
|
/>
|
||||||
|
</GlassyCard>
|
||||||
|
))}
|
||||||
|
</Stack>
|
||||||
|
|
||||||
|
<Paper
|
||||||
|
sx={{
|
||||||
|
p: 3.5,
|
||||||
|
background: allApproved
|
||||||
|
? "linear-gradient(135deg, rgba(16, 185, 129, 0.05) 0%, rgba(5, 150, 105, 0.05) 100%)"
|
||||||
|
: "#ffffff",
|
||||||
|
border: allApproved
|
||||||
|
? "2px solid rgba(16, 185, 129, 0.25)"
|
||||||
|
: "1px solid rgba(15, 23, 42, 0.08)",
|
||||||
|
borderRadius: 3,
|
||||||
|
boxShadow: allApproved
|
||||||
|
? "0 4px 6px rgba(16, 185, 129, 0.08), 0 8px 24px rgba(16, 185, 129, 0.06)"
|
||||||
|
: "0 1px 3px rgba(15, 23, 42, 0.06), 0 4px 12px rgba(15, 23, 42, 0.04)",
|
||||||
|
transition: "all 0.3s cubic-bezier(0.4, 0, 0.2, 1)",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack direction="row" justifyContent="space-between" alignItems="center">
|
||||||
|
<Box>
|
||||||
|
<Typography variant="subtitle1" sx={{ mb: 1, display: "flex", alignItems: "center", gap: 1.5, color: "#0f172a", fontWeight: 600, fontSize: "1.1rem" }}>
|
||||||
|
<CheckCircleIcon fontSize="small" sx={{ color: allApproved ? "#10b981" : "#94a3b8", fontSize: "1.25rem" }} />
|
||||||
|
Approval Status
|
||||||
|
</Typography>
|
||||||
|
<Typography variant="body2" sx={{ color: "#64748b", fontWeight: 400, lineHeight: 1.6 }}>
|
||||||
|
{approvedCount} of {totalScenes} scenes approved
|
||||||
|
{allScenesHaveAudioAndImages && " • All scenes ready for video rendering"}
|
||||||
|
{!allScenesHaveAudioAndImages && allApproved && " • Generate images for all scenes to enable video rendering"}
|
||||||
|
{!allApproved && " — Approve all scenes first"}
|
||||||
|
</Typography>
|
||||||
|
{!allScenesHaveAudioAndImages && (
|
||||||
|
<LinearProgress
|
||||||
|
variant="determinate"
|
||||||
|
value={
|
||||||
|
allScenesHaveAudioAndImages
|
||||||
|
? 100
|
||||||
|
: script
|
||||||
|
? (script.scenes.filter((s) => s.audioUrl && s.imageUrl).length / totalScenes) * 100
|
||||||
|
: 0
|
||||||
|
}
|
||||||
|
sx={{ mt: 1, height: 6, borderRadius: 3 }}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
<PrimaryButton
|
||||||
|
onClick={() => script && onProceedToRendering(script)}
|
||||||
|
disabled={!allScenesHaveAudioAndImages}
|
||||||
|
startIcon={<PlayArrowIcon />}
|
||||||
|
tooltip={
|
||||||
|
!allScenesHaveAudioAndImages
|
||||||
|
? "Generate audio and images for all scenes to proceed to video rendering"
|
||||||
|
: "Proceed to video rendering (all scenes have audio and images)"
|
||||||
|
}
|
||||||
|
>
|
||||||
|
Proceed to Rendering
|
||||||
|
</PrimaryButton>
|
||||||
|
</Stack>
|
||||||
|
</Paper>
|
||||||
|
|
||||||
|
{/* Download Audio-Only Podcast Section */}
|
||||||
|
{allScenesHaveAudio && (
|
||||||
|
<Paper
|
||||||
|
sx={{
|
||||||
|
p: 3,
|
||||||
|
background: "linear-gradient(135deg, rgba(102, 126, 234, 0.05) 0%, rgba(118, 75, 162, 0.05) 100%)",
|
||||||
|
border: "1px solid rgba(102, 126, 234, 0.15)",
|
||||||
|
borderRadius: 2,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Stack spacing={3}>
|
||||||
|
<Typography variant="h6" sx={{ color: "#0f172a", fontWeight: 600 }}>
|
||||||
|
Download Audio-Only Podcast
|
||||||
|
</Typography>
|
||||||
|
|
||||||
|
{!combinedAudioResult ? (
|
||||||
|
<>
|
||||||
|
<PrimaryButton
|
||||||
|
onClick={combineAudio}
|
||||||
|
disabled={combiningAudio}
|
||||||
|
loading={combiningAudio}
|
||||||
|
startIcon={<DownloadIcon />}
|
||||||
|
tooltip="Combine all scene audio files into a single podcast episode"
|
||||||
|
sx={{
|
||||||
|
minWidth: 280,
|
||||||
|
fontSize: "1rem",
|
||||||
|
py: 1.5,
|
||||||
|
background: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||||
|
"&:hover": {
|
||||||
|
background: "linear-gradient(135deg, #764ba2 0%, #667eea 100%)",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{combiningAudio ? "Combining Audio..." : "Download Audio-Only Podcast"}
|
||||||
|
</PrimaryButton>
|
||||||
|
<Typography variant="caption" sx={{ color: "#64748b", fontStyle: "italic" }}>
|
||||||
|
This will combine all {scenesWithAudio} scene audio files into one complete podcast episode.
|
||||||
|
</Typography>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<Stack spacing={2}>
|
||||||
|
{/* Success Alert */}
|
||||||
|
<Alert
|
||||||
|
severity="success"
|
||||||
|
sx={{
|
||||||
|
background: alpha("#10b981", 0.1),
|
||||||
|
border: "1px solid rgba(16,185,129,0.3)",
|
||||||
|
"& .MuiAlert-icon": { color: "#10b981" },
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Typography variant="body2" sx={{ color: "#059669", fontWeight: 500 }}>
|
||||||
|
✅ Combined audio generated successfully! ({combinedAudioResult.sceneCount} scenes,{" "}
|
||||||
|
{Math.round(combinedAudioResult.duration)}s)
|
||||||
|
</Typography>
|
||||||
|
</Alert>
|
||||||
|
|
||||||
|
{/* Combined Audio Preview */}
|
||||||
|
<InlineAudioPlayer audioUrl={combinedAudioResult.url} title="Complete Podcast Episode" />
|
||||||
|
|
||||||
|
{/* Action Buttons */}
|
||||||
|
<Stack direction="row" spacing={2}>
|
||||||
|
<SecondaryButton
|
||||||
|
onClick={async () => {
|
||||||
|
try {
|
||||||
|
// Normalize path
|
||||||
|
let audioPath = combinedAudioResult.url.startsWith('/')
|
||||||
|
? combinedAudioResult.url
|
||||||
|
: `/${combinedAudioResult.url}`;
|
||||||
|
|
||||||
|
// Ensure it's a podcast audio endpoint
|
||||||
|
if (!audioPath.includes('/api/podcast/audio/')) {
|
||||||
|
const filename = audioPath.split('/').pop() || combinedAudioResult.filename;
|
||||||
|
audioPath = `/api/podcast/audio/${filename}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove query parameters if present
|
||||||
|
audioPath = audioPath.split('?')[0];
|
||||||
|
|
||||||
|
// Fetch as blob using authenticated client
|
||||||
|
const response = await aiApiClient.get(audioPath, {
|
||||||
|
responseType: 'blob',
|
||||||
|
});
|
||||||
|
|
||||||
|
// Create blob URL and download
|
||||||
|
const blob = response.data;
|
||||||
|
const blobUrl = URL.createObjectURL(blob);
|
||||||
|
|
||||||
|
const link = document.createElement("a");
|
||||||
|
link.href = blobUrl;
|
||||||
|
link.download = combinedAudioResult.filename || `podcast-episode-${projectId.slice(-8)}.mp3`;
|
||||||
|
document.body.appendChild(link);
|
||||||
|
link.click();
|
||||||
|
document.body.removeChild(link);
|
||||||
|
|
||||||
|
// Clean up blob URL after a delay
|
||||||
|
setTimeout(() => {
|
||||||
|
URL.revokeObjectURL(blobUrl);
|
||||||
|
}, 100);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Failed to download audio:', error);
|
||||||
|
onError('Failed to download audio file. Please try again.');
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
startIcon={<DownloadIcon />}
|
||||||
|
tooltip="Download the combined audio file again"
|
||||||
|
>
|
||||||
|
Download Again
|
||||||
|
</SecondaryButton>
|
||||||
|
<SecondaryButton
|
||||||
|
onClick={() => {
|
||||||
|
setCombinedAudioResult(null);
|
||||||
|
combineAudio();
|
||||||
|
}}
|
||||||
|
disabled={combiningAudio}
|
||||||
|
loading={combiningAudio}
|
||||||
|
startIcon={<RefreshIcon />}
|
||||||
|
tooltip="Regenerate combined audio (useful if scenes were updated)"
|
||||||
|
>
|
||||||
|
Regenerate
|
||||||
|
</SecondaryButton>
|
||||||
|
</Stack>
|
||||||
|
</Stack>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
</Paper>
|
||||||
|
)}
|
||||||
|
</Stack>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
334
_session_backup/analysis.py
Normal file
334
_session_backup/analysis.py
Normal file
@@ -0,0 +1,334 @@
|
|||||||
|
"""
|
||||||
|
Podcast Analysis Handlers
|
||||||
|
|
||||||
|
Analysis endpoint for podcast ideas.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
|
from typing import Dict, Any
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from services.database import get_db
|
||||||
|
from middleware.auth_middleware import get_current_user
|
||||||
|
from api.story_writer.utils.auth import require_authenticated_user
|
||||||
|
from services.llm_providers.main_text_generation import llm_text_gen
|
||||||
|
from services.llm_providers.main_image_generation import generate_image
|
||||||
|
from services.podcast_bible_service import PodcastBibleService
|
||||||
|
from utils.asset_tracker import save_asset_to_library
|
||||||
|
from loguru import logger
|
||||||
|
from ..constants import PODCAST_IMAGES_DIR
|
||||||
|
from ..models import (
|
||||||
|
PodcastAnalyzeRequest,
|
||||||
|
PodcastAnalyzeResponse,
|
||||||
|
PodcastEnhanceIdeaRequest,
|
||||||
|
PodcastEnhanceIdeaResponse
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/idea/enhance", response_model=PodcastEnhanceIdeaResponse)
|
||||||
|
async def enhance_podcast_idea(
|
||||||
|
request: PodcastEnhanceIdeaRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Take raw keywords/topic and use AI to craft a presentable, detailed podcast idea.
|
||||||
|
Uses the user's Podcast Bible for hyper-personalization if available.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
# Serialize Bible context if provided or generate from onboarding
|
||||||
|
bible_context = ""
|
||||||
|
try:
|
||||||
|
bible_service = PodcastBibleService()
|
||||||
|
if request.bible:
|
||||||
|
from models.podcast_bible_models import PodcastBible
|
||||||
|
bible_data = PodcastBible(**request.bible)
|
||||||
|
bible_context = bible_service.serialize_bible(bible_data)
|
||||||
|
else:
|
||||||
|
# Generate from onboarding data directly
|
||||||
|
bible_obj = bible_service.generate_bible(user_id, "temp_enhance")
|
||||||
|
bible_context = bible_service.serialize_bible(bible_obj)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"[Podcast Enhance] Failed to parse or generate bible context: {exc}")
|
||||||
|
|
||||||
|
prompt = f"""
|
||||||
|
You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea.
|
||||||
|
|
||||||
|
{f"USER PERSONALIZATION CONTEXT (Podcast Bible):\n{bible_context}\n" if bible_context else ""}
|
||||||
|
|
||||||
|
RAW IDEA/KEYWORDS: "{request.idea}"
|
||||||
|
|
||||||
|
TASK:
|
||||||
|
Generate 3 different enhanced versions, each with a unique angle:
|
||||||
|
1. Professional & Expert-led angle (focus on authority, insights, and expertise)
|
||||||
|
2. Storytelling & Human interest angle (focus on narratives, emotions, and personal connections)
|
||||||
|
3. Trendy & Contemporary angle (focus on current trends, modern perspectives, and relevance)
|
||||||
|
|
||||||
|
Each version should be 2-3 sentences, audience-focused, and align with host persona if provided.
|
||||||
|
|
||||||
|
Return JSON with:
|
||||||
|
- enhanced_ideas: array of 3 enhanced episode pitches (in order: Professional, Storytelling, Trendy)
|
||||||
|
- rationales: array of 3 rationales explaining the approach for each version
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
raw = llm_text_gen(
|
||||||
|
prompt=prompt,
|
||||||
|
user_id=user_id,
|
||||||
|
json_struct=None,
|
||||||
|
preferred_provider="huggingface",
|
||||||
|
flow_type="premium_tool",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Normalize response
|
||||||
|
if isinstance(raw, str):
|
||||||
|
data = json.loads(raw)
|
||||||
|
else:
|
||||||
|
data = raw
|
||||||
|
|
||||||
|
# Extract enhanced ideas and rationales with fallbacks
|
||||||
|
enhanced_ideas = data.get("enhanced_ideas", [])
|
||||||
|
rationales = data.get("rationales", [])
|
||||||
|
|
||||||
|
# Ensure we have exactly 3 ideas, fallback to original if needed
|
||||||
|
if not isinstance(enhanced_ideas, list) or len(enhanced_ideas) != 3:
|
||||||
|
# Fallback: create 3 variations of the original idea
|
||||||
|
base_idea = request.idea
|
||||||
|
enhanced_ideas = [
|
||||||
|
f"Expert insights on {base_idea}: A deep dive into industry trends and best practices.",
|
||||||
|
f"The human side of {base_idea}: Personal stories and real-world experiences that resonate.",
|
||||||
|
f"Modern perspectives on {base_idea}: Current trends and forward-thinking approaches."
|
||||||
|
]
|
||||||
|
rationales = [
|
||||||
|
"Professional approach focusing on expertise and authority",
|
||||||
|
"Storytelling approach emphasizing human connection",
|
||||||
|
"Contemporary approach highlighting current relevance"
|
||||||
|
]
|
||||||
|
|
||||||
|
# Ensure rationales match the number of ideas
|
||||||
|
if not isinstance(rationales, list) or len(rationales) != 3:
|
||||||
|
rationales = [
|
||||||
|
"Professional angle with expert insights",
|
||||||
|
"Storytelling angle with human interest",
|
||||||
|
"Trendy angle with contemporary relevance"
|
||||||
|
]
|
||||||
|
|
||||||
|
return PodcastEnhanceIdeaResponse(
|
||||||
|
enhanced_ideas=enhanced_ideas[:3], # Ensure exactly 3
|
||||||
|
rationales=rationales[:3] # Ensure exactly 3
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error(f"[Podcast Enhance] Failed for user {user_id}: {exc}")
|
||||||
|
# Fallback to basic variations of original idea
|
||||||
|
base_idea = request.idea
|
||||||
|
return PodcastEnhanceIdeaResponse(
|
||||||
|
enhanced_ideas=[
|
||||||
|
f"Expert insights on {base_idea}: A deep dive into industry trends and best practices.",
|
||||||
|
f"The human side of {base_idea}: Personal stories and real-world experiences that resonate.",
|
||||||
|
f"Modern perspectives on {base_idea}: Current trends and forward-thinking approaches."
|
||||||
|
],
|
||||||
|
rationales=[
|
||||||
|
"Professional approach focusing on expertise and authority",
|
||||||
|
"Storytelling approach emphasizing human connection",
|
||||||
|
"Contemporary approach highlighting current relevance"
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/analyze", response_model=PodcastAnalyzeResponse)
|
||||||
|
async def analyze_podcast_idea(
|
||||||
|
request: PodcastAnalyzeRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Analyze a podcast idea and return podcast-oriented outlines, keywords, and titles.
|
||||||
|
If no avatar_url is provided, it generates one automatically based on the host's look.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
# Serialize Bible context if provided or generate from onboarding
|
||||||
|
bible_context = ""
|
||||||
|
bible_obj = None
|
||||||
|
try:
|
||||||
|
bible_service = PodcastBibleService()
|
||||||
|
if request.bible:
|
||||||
|
from models.podcast_bible_models import PodcastBible
|
||||||
|
bible_data = PodcastBible(**request.bible)
|
||||||
|
bible_context = bible_service.serialize_bible(bible_data)
|
||||||
|
bible_obj = bible_data
|
||||||
|
else:
|
||||||
|
# Generate from onboarding data directly
|
||||||
|
bible_obj = bible_service.generate_bible(user_id, "temp_analyze")
|
||||||
|
bible_context = bible_service.serialize_bible(bible_obj)
|
||||||
|
bible_obj = bible_obj
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"[Podcast Analyze] Failed to parse or generate bible context: {exc}")
|
||||||
|
|
||||||
|
# --- NEW: Generate Presenter Avatar if missing ---
|
||||||
|
final_avatar_url = request.avatar_url
|
||||||
|
final_avatar_prompt = None
|
||||||
|
|
||||||
|
if not final_avatar_url:
|
||||||
|
logger.info(f"[Podcast Analyze] No avatar_url provided, generating one for user {user_id}")
|
||||||
|
try:
|
||||||
|
# 1. PRE-FLIGHT VALIDATION: Check subscription limits for image generation
|
||||||
|
from services.subscription import PricingService
|
||||||
|
from services.subscription.preflight_validator import validate_image_generation_operations
|
||||||
|
pricing_service = PricingService(db)
|
||||||
|
validate_image_generation_operations(
|
||||||
|
pricing_service=pricing_service,
|
||||||
|
user_id=user_id,
|
||||||
|
num_images=1
|
||||||
|
)
|
||||||
|
|
||||||
|
# 2. Build avatar prompt from Bible host look or fallback
|
||||||
|
host_look = bible_obj.host.look if bible_obj and bible_obj.host.look else "A professional podcast host"
|
||||||
|
visual_style = bible_obj.visual_style.style_preset if bible_obj else "Realistic Photography"
|
||||||
|
|
||||||
|
final_avatar_prompt = f"Professional headshot of a podcast host, {host_look}, {visual_style} style, clean background, soft studio lighting, center-focused, high resolution, sharp focus, professional photography quality, 16:9 aspect ratio."
|
||||||
|
|
||||||
|
# 3. Generate the image
|
||||||
|
logger.info(f"[Podcast Analyze] Generating avatar with prompt: {final_avatar_prompt}")
|
||||||
|
image_result = generate_image(
|
||||||
|
prompt=final_avatar_prompt,
|
||||||
|
user_id=user_id,
|
||||||
|
width=1024,
|
||||||
|
height=1024
|
||||||
|
)
|
||||||
|
|
||||||
|
# 4. Save to disk and library
|
||||||
|
if image_result and image_result.image_bytes:
|
||||||
|
img_id = str(uuid.uuid4())[:8]
|
||||||
|
filename = f"presenter_podcast_{user_id}_{img_id}.png"
|
||||||
|
output_path = PODCAST_IMAGES_DIR / filename
|
||||||
|
PODCAST_IMAGES_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
with open(output_path, "wb") as f:
|
||||||
|
f.write(image_result.image_bytes)
|
||||||
|
|
||||||
|
final_avatar_url = f"/api/podcast/images/avatars/{filename}"
|
||||||
|
|
||||||
|
# Save to asset library for reuse
|
||||||
|
save_asset_to_library(
|
||||||
|
db=db,
|
||||||
|
user_id=user_id,
|
||||||
|
asset_type="image",
|
||||||
|
file_url=final_avatar_url,
|
||||||
|
filename=filename,
|
||||||
|
title=f"Presenter Avatar - {request.idea[:40]}",
|
||||||
|
description=f"AI-generated podcast presenter for: {request.idea}",
|
||||||
|
provider=image_result.provider,
|
||||||
|
model=image_result.model,
|
||||||
|
cost=image_result.cost
|
||||||
|
)
|
||||||
|
logger.info(f"[Podcast Analyze] ✅ Generated and saved avatar to {final_avatar_url}")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"[Podcast Analyze] ❌ Failed to generate avatar: {e}")
|
||||||
|
# Non-fatal: continue analysis even if avatar generation fails
|
||||||
|
|
||||||
|
# --- END: Avatar Generation ---
|
||||||
|
|
||||||
|
# Incorporate user feedback if provided
|
||||||
|
feedback_context = ""
|
||||||
|
if request.feedback:
|
||||||
|
feedback_context = f"""
|
||||||
|
USER REGENERATION FEEDBACK:
|
||||||
|
The user was not satisfied with the previous analysis. They provided the following instructions for improvement:
|
||||||
|
"{request.feedback}"
|
||||||
|
Please prioritize this feedback and adjust the analysis accordingly.
|
||||||
|
"""
|
||||||
|
|
||||||
|
prompt = f"""
|
||||||
|
You are an expert podcast producer and research strategist. Given a podcast idea, craft concise podcast-ready assets
|
||||||
|
that sound like episode plans (not fiction stories).
|
||||||
|
|
||||||
|
{f"USER PERSONALIZATION CONTEXT (Podcast Bible):\n{bible_context}\n" if bible_context else ""}
|
||||||
|
{feedback_context}
|
||||||
|
|
||||||
|
Podcast Idea: "{request.idea}"
|
||||||
|
Duration: ~{request.duration} minutes
|
||||||
|
Speakers: {request.speakers} (host + optional guest)
|
||||||
|
|
||||||
|
TASK:
|
||||||
|
1. Define the target audience and content type aligned with the Bible's "Audience DNA" and "Brand DNA".
|
||||||
|
2. Identify 5 high-impact keywords.
|
||||||
|
3. Propose 2 episode outlines with factual segments.
|
||||||
|
4. Suggest 3 titles.
|
||||||
|
5. IMPORTANT: Generate 4-6 specific research queries for Exa. These queries MUST be highly targeted to the episode's topic, the host's expertise level, and the audience's interests as defined in the Bible.
|
||||||
|
* Do NOT use generic queries like "latest trends in X".
|
||||||
|
* DO use queries that look for case studies, specific data points, expert opinions, or contrasting viewpoints that would make for a deep, insightful podcast conversation.
|
||||||
|
|
||||||
|
Return JSON with:
|
||||||
|
- audience: short target audience description
|
||||||
|
- content_type: podcast style/format
|
||||||
|
- top_keywords: 5 podcast-relevant keywords/phrases
|
||||||
|
- suggested_outlines: 2 items, each with title (<=60 chars) and 4-6 short segments (bullet-friendly, factual)
|
||||||
|
- title_suggestions: 3 concise episode titles
|
||||||
|
- research_queries: array of {{"query": "string", "rationale": "string"}}
|
||||||
|
- exa_suggested_config: suggested Exa search options with:
|
||||||
|
- exa_search_type: "auto" | "neural" | "keyword"
|
||||||
|
- exa_category: one of ["research paper","news","company","github","tweet","personal site","pdf","financial report","linkedin profile"]
|
||||||
|
- exa_include_domains: up to 3 reputable domains
|
||||||
|
- exa_exclude_domains: up to 3 domains
|
||||||
|
- max_sources: 6-10
|
||||||
|
- include_statistics: boolean
|
||||||
|
- date_range: one of ["last_month","last_3_months","last_year","all_time"]
|
||||||
|
|
||||||
|
Requirements:
|
||||||
|
- Keep language factual, actionable, and suited for spoken audio.
|
||||||
|
- Avoid narrative fiction tone.
|
||||||
|
- Prefer 2024-2025 context.
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
raw = llm_text_gen(
|
||||||
|
prompt=prompt,
|
||||||
|
user_id=user_id,
|
||||||
|
json_struct=None,
|
||||||
|
preferred_provider="huggingface",
|
||||||
|
flow_type="premium_tool",
|
||||||
|
)
|
||||||
|
except HTTPException:
|
||||||
|
# Re-raise HTTPExceptions (e.g., 429 subscription limit) - preserve error details
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error(f"[Podcast Analyze] Analysis failed for user {user_id}: {exc}")
|
||||||
|
raise HTTPException(status_code=500, detail=f"Analysis failed: {exc}")
|
||||||
|
|
||||||
|
# Normalize response (accept dict or JSON string)
|
||||||
|
if isinstance(raw, str):
|
||||||
|
try:
|
||||||
|
data = json.loads(raw)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
raise HTTPException(status_code=500, detail="LLM returned non-JSON output")
|
||||||
|
elif isinstance(raw, dict):
|
||||||
|
data = raw
|
||||||
|
else:
|
||||||
|
raise HTTPException(status_code=500, detail="Unexpected LLM response format")
|
||||||
|
|
||||||
|
audience = data.get("audience") or "Growth-focused professionals"
|
||||||
|
content_type = data.get("content_type") or "Interview + insights"
|
||||||
|
top_keywords = data.get("top_keywords") or []
|
||||||
|
suggested_outlines = data.get("suggested_outlines") or []
|
||||||
|
title_suggestions = data.get("title_suggestions") or []
|
||||||
|
research_queries = data.get("research_queries") or []
|
||||||
|
exa_suggested_config = data.get("exa_suggested_config") or None
|
||||||
|
|
||||||
|
return PodcastAnalyzeResponse(
|
||||||
|
audience=audience,
|
||||||
|
content_type=content_type,
|
||||||
|
top_keywords=top_keywords,
|
||||||
|
suggested_outlines=suggested_outlines,
|
||||||
|
title_suggestions=title_suggestions,
|
||||||
|
research_queries=research_queries,
|
||||||
|
exa_suggested_config=exa_suggested_config,
|
||||||
|
bible=bible_obj.model_dump() if bible_obj else None,
|
||||||
|
avatar_url=final_avatar_url,
|
||||||
|
avatar_prompt=final_avatar_prompt,
|
||||||
|
)
|
||||||
|
|
||||||
422
_session_backup/models.py
Normal file
422
_session_backup/models.py
Normal file
@@ -0,0 +1,422 @@
|
|||||||
|
"""
|
||||||
|
Podcast API Models
|
||||||
|
|
||||||
|
All Pydantic request/response models for podcast endpoints.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field, model_validator
|
||||||
|
from typing import List, Optional, Dict, Any
|
||||||
|
from datetime import datetime
|
||||||
|
from enum import Enum
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastProjectResponse(BaseModel):
|
||||||
|
"""Response model for podcast project."""
|
||||||
|
id: int
|
||||||
|
project_id: str
|
||||||
|
user_id: str
|
||||||
|
idea: str
|
||||||
|
duration: int
|
||||||
|
speakers: int
|
||||||
|
budget_cap: float
|
||||||
|
analysis: Optional[Dict[str, Any]] = None
|
||||||
|
queries: Optional[List[Dict[str, Any]]] = None
|
||||||
|
selected_queries: Optional[List[str]] = None
|
||||||
|
research: Optional[Dict[str, Any]] = None
|
||||||
|
raw_research: Optional[Dict[str, Any]] = None
|
||||||
|
estimate: Optional[Dict[str, Any]] = None
|
||||||
|
script_data: Optional[Dict[str, Any]] = None
|
||||||
|
bible: Optional[Dict[str, Any]] = None
|
||||||
|
render_jobs: Optional[List[Dict[str, Any]]] = None
|
||||||
|
knobs: Optional[Dict[str, Any]] = None
|
||||||
|
research_provider: Optional[str] = None
|
||||||
|
show_script_editor: bool = False
|
||||||
|
show_render_queue: bool = False
|
||||||
|
current_step: Optional[str] = None
|
||||||
|
status: str = "draft"
|
||||||
|
is_favorite: bool = False
|
||||||
|
final_video_url: Optional[str] = None
|
||||||
|
avatar_url: Optional[str] = None
|
||||||
|
avatar_prompt: Optional[str] = None
|
||||||
|
avatar_persona_id: Optional[str] = None
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
from_attributes = True
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAnalyzeRequest(BaseModel):
|
||||||
|
"""Request model for podcast idea analysis."""
|
||||||
|
idea: str = Field(..., description="Podcast topic or idea")
|
||||||
|
duration: int = Field(default=10, description="Target duration in minutes")
|
||||||
|
speakers: int = Field(default=1, description="Number of speakers")
|
||||||
|
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
||||||
|
avatar_url: Optional[str] = Field(None, description="Current avatar URL if selected")
|
||||||
|
feedback: Optional[str] = Field(None, description="User feedback for regeneration")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAnalyzeResponse(BaseModel):
|
||||||
|
"""Response model for podcast idea analysis."""
|
||||||
|
audience: str
|
||||||
|
content_type: str
|
||||||
|
top_keywords: list[str]
|
||||||
|
suggested_outlines: list[Dict[str, Any]]
|
||||||
|
title_suggestions: list[str]
|
||||||
|
research_queries: Optional[List[Dict[str, str]]] = None
|
||||||
|
exa_suggested_config: Optional[Dict[str, Any]] = None
|
||||||
|
bible: Optional[Dict[str, Any]] = None
|
||||||
|
avatar_url: Optional[str] = None
|
||||||
|
avatar_prompt: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastEnhanceIdeaRequest(BaseModel):
|
||||||
|
"""Request model for enhancing a podcast idea with AI."""
|
||||||
|
idea: str = Field(..., description="The raw podcast idea or keywords")
|
||||||
|
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastEnhanceIdeaResponse(BaseModel):
|
||||||
|
"""Response model for enhanced podcast idea."""
|
||||||
|
enhanced_ideas: List[str] = Field(..., description="3 AI-enhanced topic choices")
|
||||||
|
rationales: List[str] = Field(..., description="Rationale for each enhanced idea")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastScriptRequest(BaseModel):
|
||||||
|
"""Request model for podcast script generation."""
|
||||||
|
idea: str = Field(..., description="Podcast idea or topic")
|
||||||
|
duration_minutes: int = Field(default=10, description="Target duration in minutes")
|
||||||
|
speakers: int = Field(default=1, description="Number of speakers")
|
||||||
|
research: Optional[Dict[str, Any]] = Field(None, description="Optional research payload to ground the script")
|
||||||
|
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||||
|
outline: Optional[Dict[str, Any]] = Field(None, description="The refined episode outline to follow")
|
||||||
|
analysis: Optional[Dict[str, Any]] = Field(None, description="The full analysis context (audience, keywords, etc.)")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastSceneLine(BaseModel):
|
||||||
|
speaker: str
|
||||||
|
text: str
|
||||||
|
emphasis: Optional[bool] = False
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastScene(BaseModel):
|
||||||
|
id: str
|
||||||
|
title: str
|
||||||
|
duration: int
|
||||||
|
lines: list[PodcastSceneLine]
|
||||||
|
approved: bool = False
|
||||||
|
emotion: Optional[str] = None
|
||||||
|
imageUrl: Optional[str] = None # Generated image URL for video generation
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastExaConfig(BaseModel):
|
||||||
|
"""Exa config for podcast research."""
|
||||||
|
exa_search_type: Optional[str] = Field(default="auto", description="auto | keyword | neural")
|
||||||
|
exa_category: Optional[str] = None
|
||||||
|
exa_include_domains: List[str] = []
|
||||||
|
exa_exclude_domains: List[str] = []
|
||||||
|
max_sources: int = 8
|
||||||
|
include_statistics: Optional[bool] = False
|
||||||
|
date_range: Optional[str] = Field(default=None, description="last_month | last_3_months | last_year | all_time")
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def validate_domains(self):
|
||||||
|
if self.exa_include_domains and self.exa_exclude_domains:
|
||||||
|
# Exa API does not allow both include and exclude domains together with contents
|
||||||
|
# Prefer include_domains and drop exclude_domains
|
||||||
|
self.exa_exclude_domains = []
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastExaResearchRequest(BaseModel):
|
||||||
|
"""Request for podcast research using Exa directly (no blog writer)."""
|
||||||
|
topic: str
|
||||||
|
queries: List[str]
|
||||||
|
exa_config: Optional[PodcastExaConfig] = None
|
||||||
|
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||||
|
analysis: Optional[Dict[str, Any]] = Field(None, description="Podcast analysis context (audience, content type, etc.)")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastExaSource(BaseModel):
|
||||||
|
title: str = ""
|
||||||
|
url: str = ""
|
||||||
|
excerpt: str = ""
|
||||||
|
published_at: Optional[str] = None
|
||||||
|
highlights: Optional[List[str]] = None
|
||||||
|
summary: Optional[str] = None
|
||||||
|
source_type: Optional[str] = None
|
||||||
|
index: Optional[int] = None
|
||||||
|
image: Optional[str] = None
|
||||||
|
author: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastResearchInsight(BaseModel):
|
||||||
|
"""Deep insight extracted from research."""
|
||||||
|
title: str
|
||||||
|
content: str
|
||||||
|
source_indices: List[int] = []
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastExaResearchResponse(BaseModel):
|
||||||
|
sources: List[PodcastExaSource]
|
||||||
|
search_queries: List[str] = []
|
||||||
|
summary: str = ""
|
||||||
|
key_insights: List[PodcastResearchInsight] = []
|
||||||
|
expert_quotes: List[Dict[str, Any]] = []
|
||||||
|
listener_cta: List[str] = []
|
||||||
|
mapped_angles: List[Dict[str, Any]] = []
|
||||||
|
cost: Optional[Dict[str, Any]] = None
|
||||||
|
search_type: Optional[str] = None
|
||||||
|
provider: str = "exa"
|
||||||
|
content: Optional[str] = None # Raw aggregated content (deprecated)
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastScriptResponse(BaseModel):
|
||||||
|
scenes: list[PodcastScene]
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAudioRequest(BaseModel):
|
||||||
|
"""Generate TTS for a podcast scene."""
|
||||||
|
scene_id: str
|
||||||
|
scene_title: str
|
||||||
|
text: str
|
||||||
|
voice_id: Optional[str] = "Wise_Woman"
|
||||||
|
speed: Optional[float] = 1.0
|
||||||
|
volume: Optional[float] = 1.0
|
||||||
|
pitch: Optional[float] = 0.0
|
||||||
|
emotion: Optional[str] = "neutral"
|
||||||
|
english_normalization: Optional[bool] = False # Better number reading for statistics
|
||||||
|
sample_rate: Optional[int] = None
|
||||||
|
bitrate: Optional[int] = None
|
||||||
|
channel: Optional[str] = None
|
||||||
|
format: Optional[str] = None
|
||||||
|
language_boost: Optional[str] = None
|
||||||
|
enable_sync_mode: Optional[bool] = True
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAudioResponse(BaseModel):
|
||||||
|
scene_id: str
|
||||||
|
scene_title: str
|
||||||
|
audio_filename: str
|
||||||
|
audio_url: str
|
||||||
|
provider: str
|
||||||
|
model: str
|
||||||
|
voice_id: str
|
||||||
|
text_length: int
|
||||||
|
file_size: int
|
||||||
|
cost: float
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastProjectListResponse(BaseModel):
|
||||||
|
"""Response model for project list."""
|
||||||
|
projects: List[PodcastProjectResponse]
|
||||||
|
total: int
|
||||||
|
limit: int
|
||||||
|
offset: int
|
||||||
|
|
||||||
|
|
||||||
|
class CreateProjectRequest(BaseModel):
|
||||||
|
"""Request model for creating a project."""
|
||||||
|
project_id: str = Field(..., description="Unique project ID")
|
||||||
|
idea: str = Field(..., description="Episode idea or URL")
|
||||||
|
duration: int = Field(..., description="Duration in minutes")
|
||||||
|
speakers: int = Field(default=1, description="Number of speakers")
|
||||||
|
budget_cap: float = Field(default=50.0, description="Budget cap in USD")
|
||||||
|
avatar_url: Optional[str] = Field(None, description="Optional presenter avatar URL")
|
||||||
|
|
||||||
|
|
||||||
|
class UpdateProjectRequest(BaseModel):
|
||||||
|
"""Request model for updating project state."""
|
||||||
|
analysis: Optional[Dict[str, Any]] = None
|
||||||
|
queries: Optional[List[Dict[str, Any]]] = None
|
||||||
|
selected_queries: Optional[List[str]] = None
|
||||||
|
research: Optional[Dict[str, Any]] = None
|
||||||
|
raw_research: Optional[Dict[str, Any]] = None
|
||||||
|
estimate: Optional[Dict[str, Any]] = None
|
||||||
|
script_data: Optional[Dict[str, Any]] = None
|
||||||
|
bible: Optional[Dict[str, Any]] = None
|
||||||
|
render_jobs: Optional[List[Dict[str, Any]]] = None
|
||||||
|
knobs: Optional[Dict[str, Any]] = None
|
||||||
|
research_provider: Optional[str] = None
|
||||||
|
show_script_editor: Optional[bool] = None
|
||||||
|
show_render_queue: Optional[bool] = None
|
||||||
|
current_step: Optional[str] = None
|
||||||
|
status: Optional[str] = None
|
||||||
|
final_video_url: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastCombineAudioRequest(BaseModel):
|
||||||
|
"""Request model for combining podcast audio files."""
|
||||||
|
project_id: str
|
||||||
|
scene_ids: List[str] = Field(..., description="List of scene IDs to combine")
|
||||||
|
scene_audio_urls: List[str] = Field(..., description="List of audio URLs for each scene")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastCombineAudioResponse(BaseModel):
|
||||||
|
"""Response model for combined podcast audio."""
|
||||||
|
combined_audio_url: str
|
||||||
|
combined_audio_filename: str
|
||||||
|
total_duration: float
|
||||||
|
file_size: int
|
||||||
|
scene_count: int
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastImageRequest(BaseModel):
|
||||||
|
"""Request for generating an image for a podcast scene."""
|
||||||
|
scene_id: str
|
||||||
|
scene_title: str
|
||||||
|
scene_content: Optional[str] = None # Optional: scene lines text for context
|
||||||
|
idea: Optional[str] = None # Optional: podcast idea for context
|
||||||
|
base_avatar_url: Optional[str] = None # Base avatar image URL for scene variations
|
||||||
|
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||||
|
width: int = 1024
|
||||||
|
height: int = 1024
|
||||||
|
custom_prompt: Optional[str] = None # Custom prompt from user (overrides auto-generated prompt)
|
||||||
|
style: Optional[str] = None # "Auto", "Fiction", or "Realistic"
|
||||||
|
rendering_speed: Optional[str] = None # "Default", "Turbo", or "Quality"
|
||||||
|
aspect_ratio: Optional[str] = None # "1:1", "16:9", "9:16", "4:3", "3:4"
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastImageResponse(BaseModel):
|
||||||
|
"""Response for podcast scene image generation."""
|
||||||
|
scene_id: str
|
||||||
|
scene_title: str
|
||||||
|
image_filename: str
|
||||||
|
image_url: str
|
||||||
|
width: int
|
||||||
|
height: int
|
||||||
|
provider: str
|
||||||
|
model: Optional[str] = None
|
||||||
|
cost: float
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastVideoGenerationRequest(BaseModel):
|
||||||
|
"""Request model for podcast video generation."""
|
||||||
|
project_id: str = Field(..., description="Podcast project ID")
|
||||||
|
scene_id: str = Field(..., description="Scene ID")
|
||||||
|
scene_title: str = Field(..., description="Scene title")
|
||||||
|
audio_url: str = Field(..., description="URL to the generated audio file")
|
||||||
|
avatar_image_url: Optional[str] = Field(None, description="URL to scene image (required for video generation)")
|
||||||
|
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||||
|
resolution: str = Field("720p", description="Video resolution (480p or 720p)")
|
||||||
|
prompt: Optional[str] = Field(None, description="Optional animation prompt override")
|
||||||
|
seed: Optional[int] = Field(-1, description="Random seed; -1 for random")
|
||||||
|
mask_image_url: Optional[str] = Field(None, description="Optional mask image URL to specify animated region")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastVideoGenerationResponse(BaseModel):
|
||||||
|
"""Response model for podcast video generation."""
|
||||||
|
task_id: str
|
||||||
|
status: str
|
||||||
|
message: str
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastCombineVideosRequest(BaseModel):
|
||||||
|
"""Request to combine scene videos into final podcast"""
|
||||||
|
project_id: str = Field(..., description="Project ID")
|
||||||
|
scene_video_urls: list[str] = Field(..., description="List of scene video URLs in order")
|
||||||
|
podcast_title: str = Field(default="Podcast", description="Title for the final podcast video")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastCombineVideosResponse(BaseModel):
|
||||||
|
"""Response from combine videos endpoint"""
|
||||||
|
task_id: str
|
||||||
|
status: str
|
||||||
|
message: str
|
||||||
|
|
||||||
|
|
||||||
|
class AudioDubbingQuality(str, Enum):
|
||||||
|
LOW = "low"
|
||||||
|
HIGH = "high"
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_string(cls, value: str) -> "AudioDubbingQuality":
|
||||||
|
if value.lower() == "high":
|
||||||
|
return cls.HIGH
|
||||||
|
return cls.LOW
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAudioDubRequest(BaseModel):
|
||||||
|
"""Request model for audio dubbing."""
|
||||||
|
source_audio_url: str = Field(..., description="URL or path to source audio file")
|
||||||
|
source_language: Optional[str] = Field(None, description="Source language code (auto-detected if None)")
|
||||||
|
target_language: str = Field(..., description="Target language for dubbing")
|
||||||
|
quality: str = Field(default="low", description="Translation quality: low (DeepL) or high (WaveSpeed)")
|
||||||
|
voice_id: Optional[str] = Field(default="Wise_Woman", description="Voice ID for TTS")
|
||||||
|
speed: Optional[float] = Field(default=1.0, ge=0.5, le=2.0, description="Speech speed (0.5-2.0)")
|
||||||
|
emotion: Optional[str] = Field(default="happy", description="Emotion for TTS voice")
|
||||||
|
preserve_emotion: Optional[bool] = Field(default=True, description="Preserve emotional tone in translation")
|
||||||
|
use_voice_clone: Optional[bool] = Field(default=False, description="Use voice cloning to preserve original speaker's voice")
|
||||||
|
custom_voice_id: Optional[str] = Field(None, description="Custom name for the cloned voice")
|
||||||
|
voice_clone_accuracy: Optional[float] = Field(default=0.7, ge=0.1, le=1.0, description="Voice cloning accuracy (0.1-1.0)")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAudioDubResponse(BaseModel):
|
||||||
|
"""Response model for audio dubbing task creation."""
|
||||||
|
task_id: str
|
||||||
|
status: str = "pending"
|
||||||
|
message: str = "Audio dubbing task created"
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAudioDubResult(BaseModel):
|
||||||
|
"""Response model for completed audio dubbing."""
|
||||||
|
dubbed_audio_url: str
|
||||||
|
dubbed_audio_filename: str
|
||||||
|
original_transcript: str
|
||||||
|
translated_transcript: str
|
||||||
|
source_language: str
|
||||||
|
target_language: str
|
||||||
|
voice_id: str
|
||||||
|
quality: str
|
||||||
|
duration_seconds: int
|
||||||
|
file_size: int
|
||||||
|
cost: float
|
||||||
|
task_id: str
|
||||||
|
status: str = "completed"
|
||||||
|
voice_clone_used: Optional[bool] = Field(default=False, description="Whether voice cloning was used")
|
||||||
|
cloned_voice_id: Optional[str] = Field(None, description="ID of the cloned voice if voice_clone_used=True")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAudioDubEstimateRequest(BaseModel):
|
||||||
|
"""Request model for dubbing cost estimation."""
|
||||||
|
audio_duration_seconds: float = Field(..., description="Duration of source audio in seconds")
|
||||||
|
target_language: str = Field(..., description="Target language")
|
||||||
|
quality: str = Field(default="low", description="Translation quality")
|
||||||
|
use_voice_clone: Optional[bool] = Field(default=False, description="Include voice cloning cost")
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastAudioDubEstimateResponse(BaseModel):
|
||||||
|
"""Response model for dubbing cost estimation."""
|
||||||
|
estimated_characters: int
|
||||||
|
translation_cost: float
|
||||||
|
tts_cost: float
|
||||||
|
voice_clone_cost: float = 0.0
|
||||||
|
total_cost: float
|
||||||
|
currency: str = "USD"
|
||||||
|
|
||||||
|
|
||||||
|
class VoiceCloneRequest(BaseModel):
|
||||||
|
"""Request model for voice cloning."""
|
||||||
|
source_audio_url: str = Field(..., description="URL or path to source audio file (10-60 seconds recommended)")
|
||||||
|
custom_voice_id: Optional[str] = Field(None, description="Custom name for the cloned voice")
|
||||||
|
accuracy: Optional[float] = Field(default=0.7, ge=0.1, le=1.0, description="Cloning accuracy (0.1-1.0)")
|
||||||
|
language_boost: Optional[str] = Field(None, description="Language to optimize the voice for")
|
||||||
|
|
||||||
|
|
||||||
|
class VoiceCloneResponse(BaseModel):
|
||||||
|
"""Response model for voice cloning."""
|
||||||
|
task_id: str
|
||||||
|
status: str = "pending"
|
||||||
|
message: str = "Voice cloning task created"
|
||||||
|
|
||||||
|
|
||||||
|
class VoiceCloneResult(BaseModel):
|
||||||
|
"""Response model for completed voice cloning."""
|
||||||
|
voice_id: str
|
||||||
|
voice_url: str
|
||||||
|
source_language: str
|
||||||
|
accuracy: float
|
||||||
|
file_size: int
|
||||||
|
task_id: str
|
||||||
|
status: str = "completed"
|
||||||
|
|
||||||
837
_session_backup/podcastApi.ts
Normal file
837
_session_backup/podcastApi.ts
Normal file
@@ -0,0 +1,837 @@
|
|||||||
|
import { ResearchProvider, ResearchConfig } from "./blogWriterApi";
|
||||||
|
import {
|
||||||
|
storyWriterApi,
|
||||||
|
StorySetupGenerationResponse,
|
||||||
|
} from "./storyWriterApi";
|
||||||
|
import { getResearchConfig, ResearchPersona } from "../api/researchConfig";
|
||||||
|
import { aiApiClient } from "../api/client";
|
||||||
|
import {
|
||||||
|
CreateProjectPayload,
|
||||||
|
CreateProjectResult,
|
||||||
|
Fact,
|
||||||
|
Knobs,
|
||||||
|
PodcastAnalysis,
|
||||||
|
PodcastEstimate,
|
||||||
|
Query,
|
||||||
|
RenderJobResult,
|
||||||
|
Research,
|
||||||
|
Scene,
|
||||||
|
Script,
|
||||||
|
} from "../components/PodcastMaker/types";
|
||||||
|
import { checkPreflight, PreflightOperation } from "./billingService";
|
||||||
|
import { TaskStatus } from "./storyWriterApi";
|
||||||
|
|
||||||
|
const DEFAULT_KNOBS: Knobs = {
|
||||||
|
voice_emotion: "neutral",
|
||||||
|
voice_speed: 1,
|
||||||
|
resolution: "720p",
|
||||||
|
scene_length_target: 45,
|
||||||
|
sample_rate: 24000,
|
||||||
|
bitrate: "standard",
|
||||||
|
};
|
||||||
|
|
||||||
|
// const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||||
|
|
||||||
|
const createId = (prefix: string) => {
|
||||||
|
if (typeof crypto !== "undefined" && typeof crypto.randomUUID === "function") {
|
||||||
|
return `${prefix}_${crypto.randomUUID()}`;
|
||||||
|
}
|
||||||
|
return `${prefix}_${Date.now()}_${Math.floor(Math.random() * 10000)}`;
|
||||||
|
};
|
||||||
|
|
||||||
|
type OptionLike = StorySetupGenerationResponse["options"][0] | { plot_elements?: string; premise?: string };
|
||||||
|
|
||||||
|
const deriveSegments = (option?: OptionLike): string[] => {
|
||||||
|
const segments: string[] = [];
|
||||||
|
if (option?.plot_elements) {
|
||||||
|
option.plot_elements
|
||||||
|
.split(/[,.;]+/)
|
||||||
|
.map((p) => p.trim())
|
||||||
|
.filter(Boolean)
|
||||||
|
.forEach((p) => segments.push(p));
|
||||||
|
}
|
||||||
|
if (!segments.length && "premise" in (option || {}) && (option as any)?.premise) {
|
||||||
|
segments.push("Intro", "Key Takeaways", "Examples", "CTA");
|
||||||
|
}
|
||||||
|
return segments.slice(0, 5);
|
||||||
|
};
|
||||||
|
|
||||||
|
const estimateCosts = ({
|
||||||
|
minutes,
|
||||||
|
scenes,
|
||||||
|
chars,
|
||||||
|
quality,
|
||||||
|
avatars,
|
||||||
|
queryCount = 3,
|
||||||
|
}: {
|
||||||
|
minutes: number;
|
||||||
|
scenes: number;
|
||||||
|
chars: number;
|
||||||
|
quality: string;
|
||||||
|
avatars: number;
|
||||||
|
queryCount?: number;
|
||||||
|
}): PodcastEstimate => {
|
||||||
|
const secs = Math.max(60, minutes * 60);
|
||||||
|
const ttsCost = (chars / 1000) * 0.05;
|
||||||
|
const avatarCost = avatars * 0.15;
|
||||||
|
const videoRate = quality === "hd" ? 0.06 : 0.03;
|
||||||
|
const videoCost = secs * videoRate;
|
||||||
|
const researchCost = +(Math.max(1, queryCount) * 0.1).toFixed(2);
|
||||||
|
const total = +(ttsCost + avatarCost + videoCost + researchCost).toFixed(2);
|
||||||
|
return {
|
||||||
|
ttsCost: +ttsCost.toFixed(2),
|
||||||
|
avatarCost: +avatarCost.toFixed(2),
|
||||||
|
videoCost: +videoCost.toFixed(2),
|
||||||
|
researchCost,
|
||||||
|
total,
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
const mapPersonaQueries = (persona: ResearchPersona | undefined, seed: string): Query[] => {
|
||||||
|
const baseIdea = seed || "AI marketing for small businesses";
|
||||||
|
const personaKeywords = persona?.suggested_keywords?.filter(Boolean) || [];
|
||||||
|
const angles = persona?.research_angles ?? [];
|
||||||
|
const generated: Query[] = [];
|
||||||
|
|
||||||
|
const addQuery = (q: string, why: string, needsRecent = false) => {
|
||||||
|
if (!q.trim()) return;
|
||||||
|
generated.push({
|
||||||
|
id: createId("q"),
|
||||||
|
query: q.trim(),
|
||||||
|
rationale: why,
|
||||||
|
needsRecentStats: needsRecent,
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
if (personaKeywords.length) {
|
||||||
|
personaKeywords.slice(0, 4).forEach((k, idx) =>
|
||||||
|
addQuery(k, angles[idx % Math.max(1, angles.length)] || "Persona-aligned query", /202[45]|latest|trend/i.test(k))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!generated.length) {
|
||||||
|
addQuery(`How is ${baseIdea} evolving in 2024?`, "Trend + outcome focus", true);
|
||||||
|
addQuery(`Best practices for ${baseIdea}`, "Actionable guidance", false);
|
||||||
|
addQuery(`${baseIdea} case studies with ROI`, "Proof and outcomes", true);
|
||||||
|
addQuery(`${baseIdea} risks and objections`, "Address listener concerns", false);
|
||||||
|
}
|
||||||
|
|
||||||
|
return generated.slice(0, 6);
|
||||||
|
};
|
||||||
|
|
||||||
|
const mapSourcesToFacts = (sources: ExaSource[]): Fact[] => {
|
||||||
|
if (!sources || !sources.length) return [];
|
||||||
|
return sources.slice(0, 12).map((source: ExaSource, idx: number) => ({
|
||||||
|
id: source.url || createId("fact"),
|
||||||
|
quote: source.excerpt || source.title || "Insight",
|
||||||
|
url: source.url || "",
|
||||||
|
date: source.published_at || "Unknown",
|
||||||
|
confidence: typeof (source as any).credibility_score === "number" ? (source as any).credibility_score : Math.max(0.5, 0.85 - idx * 0.02),
|
||||||
|
image: source.image,
|
||||||
|
author: source.author,
|
||||||
|
highlights: source.highlights,
|
||||||
|
}));
|
||||||
|
};
|
||||||
|
|
||||||
|
type ExaSource = {
|
||||||
|
title?: string;
|
||||||
|
url?: string;
|
||||||
|
excerpt?: string;
|
||||||
|
published_at?: string;
|
||||||
|
highlights?: string[];
|
||||||
|
summary?: string;
|
||||||
|
source_type?: string;
|
||||||
|
index?: number;
|
||||||
|
image?: string;
|
||||||
|
author?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type ExaResearchResult = {
|
||||||
|
sources: ExaSource[];
|
||||||
|
search_queries?: string[];
|
||||||
|
cost?: { total?: number };
|
||||||
|
search_type?: string;
|
||||||
|
provider?: string;
|
||||||
|
content?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
const mapExaResearchResponse = (response: any): Research => {
|
||||||
|
const factCards = mapSourcesToFacts(response.sources);
|
||||||
|
// Use backend summary if available, otherwise use full content (no truncation) or fallback text
|
||||||
|
const summary = response.summary || response.content || "Research completed.";
|
||||||
|
|
||||||
|
const keyInsights = (response.key_insights || []).map((insight: any) => ({
|
||||||
|
title: insight.title || "Insight",
|
||||||
|
content: insight.content || "",
|
||||||
|
source_indices: insight.source_indices || []
|
||||||
|
}));
|
||||||
|
|
||||||
|
const expertQuotes = (response.expert_quotes || []).map((eq: any) => ({
|
||||||
|
quote: eq.quote || eq.text || "",
|
||||||
|
source_index: eq.source_index ?? 0
|
||||||
|
}));
|
||||||
|
|
||||||
|
const listenerCta = response.listener_cta || [];
|
||||||
|
|
||||||
|
const mappedAngles = (response.mapped_angles || []).map((angle: any) => ({
|
||||||
|
title: angle.title || "",
|
||||||
|
why: angle.why || angle.rationale || "",
|
||||||
|
mappedFactIds: angle.mapped_fact_ids || angle.mappedFactIds || []
|
||||||
|
}));
|
||||||
|
|
||||||
|
return {
|
||||||
|
summary,
|
||||||
|
keyInsights,
|
||||||
|
factCards,
|
||||||
|
mappedAngles,
|
||||||
|
expertQuotes,
|
||||||
|
listenerCta,
|
||||||
|
searchQueries: response.search_queries,
|
||||||
|
searchType: response.search_type,
|
||||||
|
provider: response.provider || "exa",
|
||||||
|
cost: response.cost?.total,
|
||||||
|
sourceCount: response.sources?.length || 0,
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
const ensurePreflight = async (operation: PreflightOperation) => {
|
||||||
|
const result = await checkPreflight(operation);
|
||||||
|
if (!result.can_proceed) {
|
||||||
|
const message = result.operations[0]?.message || "Pre-flight validation failed";
|
||||||
|
throw new Error(message);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
};
|
||||||
|
|
||||||
|
export const podcastApi = {
|
||||||
|
async createProject(payload: CreateProjectPayload, bible?: any, feedback?: string): Promise<CreateProjectResult> {
|
||||||
|
const storyIdea = payload.ideaOrUrl || "AI marketing for small businesses";
|
||||||
|
|
||||||
|
await ensurePreflight({
|
||||||
|
provider: "gemini",
|
||||||
|
operation_type: "podcast_analysis",
|
||||||
|
tokens_requested: 1500,
|
||||||
|
actual_provider_name: "gemini",
|
||||||
|
});
|
||||||
|
|
||||||
|
// Podcast-specific analysis (not story setup)
|
||||||
|
const analysisResp = await aiApiClient.post("/api/podcast/analyze", {
|
||||||
|
idea: storyIdea,
|
||||||
|
duration: payload.duration,
|
||||||
|
speakers: payload.speakers,
|
||||||
|
bible: bible,
|
||||||
|
avatar_url: payload.avatarUrl,
|
||||||
|
feedback: feedback, // Pass feedback to backend
|
||||||
|
});
|
||||||
|
|
||||||
|
const outlines = (analysisResp.data?.suggested_outlines || []).map((o: any, idx: number) => ({
|
||||||
|
id: o.id || `outline-${idx + 1}`,
|
||||||
|
title: o.title || `Outline ${idx + 1}`,
|
||||||
|
segments: Array.isArray(o.segments) ? o.segments : deriveSegments({ plot_elements: o.segments }),
|
||||||
|
}));
|
||||||
|
|
||||||
|
const analysis: PodcastAnalysis = {
|
||||||
|
audience: analysisResp.data?.audience || "Growth-minded pros",
|
||||||
|
contentType: analysisResp.data?.content_type || "Podcast interview",
|
||||||
|
topKeywords: analysisResp.data?.top_keywords || outlines[0]?.segments?.slice(0, 3) || [],
|
||||||
|
suggestedOutlines: outlines,
|
||||||
|
suggestedKnobs: { ...DEFAULT_KNOBS, ...payload.knobs },
|
||||||
|
titleSuggestions: (analysisResp.data?.title_suggestions || []).filter(Boolean),
|
||||||
|
research_queries: analysisResp.data?.research_queries || [],
|
||||||
|
exaSuggestedConfig: analysisResp.data?.exa_suggested_config || undefined,
|
||||||
|
};
|
||||||
|
|
||||||
|
const researchConfig = await getResearchConfig().catch(() => null);
|
||||||
|
|
||||||
|
// Use AI-generated queries if available, fallback to legacy mapping
|
||||||
|
let queries: Query[] = [];
|
||||||
|
if (analysis.research_queries && analysis.research_queries.length > 0) {
|
||||||
|
queries = analysis.research_queries.map(rq => ({
|
||||||
|
id: createId("q"),
|
||||||
|
query: rq.query,
|
||||||
|
rationale: rq.rationale,
|
||||||
|
needsRecentStats: /202[45]|latest|trend/i.test(rq.query)
|
||||||
|
}));
|
||||||
|
} else {
|
||||||
|
queries = mapPersonaQueries(researchConfig?.research_persona, storyIdea);
|
||||||
|
}
|
||||||
|
|
||||||
|
const projectId = createId("podcast");
|
||||||
|
const estimate = estimateCosts({
|
||||||
|
minutes: payload.duration,
|
||||||
|
scenes: Math.ceil((payload.duration * 60) / (payload.knobs.scene_length_target || DEFAULT_KNOBS.scene_length_target)),
|
||||||
|
chars: Math.max(1000, payload.duration * 900),
|
||||||
|
quality: payload.knobs.bitrate || "standard",
|
||||||
|
avatars: payload.speakers,
|
||||||
|
queryCount: queries.length || 3,
|
||||||
|
});
|
||||||
|
|
||||||
|
return {
|
||||||
|
projectId,
|
||||||
|
analysis,
|
||||||
|
estimate,
|
||||||
|
queries,
|
||||||
|
bible: analysisResp.data?.bible || undefined,
|
||||||
|
avatar_url: analysisResp.data?.avatar_url || null,
|
||||||
|
avatar_prompt: analysisResp.data?.avatar_prompt || null,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
|
||||||
|
async enhanceIdea(params: { idea: string; bible?: any }): Promise<{ enhanced_ideas: string[]; rationales: string[] }> {
|
||||||
|
const response = await aiApiClient.post("/api/podcast/idea/enhance", params);
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async runResearch(params: {
|
||||||
|
projectId: string;
|
||||||
|
topic: string;
|
||||||
|
approvedQueries: Query[];
|
||||||
|
provider?: ResearchProvider;
|
||||||
|
exaConfig?: ResearchConfig;
|
||||||
|
bible?: any;
|
||||||
|
analysis?: PodcastAnalysis | null;
|
||||||
|
onProgress?: (message: string) => void;
|
||||||
|
}): Promise<{ research: Research; raw: any }> {
|
||||||
|
const keywords = params.approvedQueries.map((q) => q.query).filter(Boolean);
|
||||||
|
if (!keywords.length) {
|
||||||
|
throw new Error("At least one query must be approved for research.");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure Exa payload respects API constraint: when requesting contents, only one of includeDomains or excludeDomains.
|
||||||
|
let sanitizedExaConfig: ResearchConfig | undefined = params.exaConfig;
|
||||||
|
if (sanitizedExaConfig && sanitizedExaConfig.exa_include_domains?.length) {
|
||||||
|
sanitizedExaConfig = {
|
||||||
|
...sanitizedExaConfig,
|
||||||
|
exa_exclude_domains: undefined,
|
||||||
|
};
|
||||||
|
} else if (sanitizedExaConfig && sanitizedExaConfig.exa_exclude_domains?.length) {
|
||||||
|
sanitizedExaConfig = {
|
||||||
|
...sanitizedExaConfig,
|
||||||
|
exa_include_domains: undefined,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
await ensurePreflight({
|
||||||
|
provider: "exa",
|
||||||
|
operation_type: "exa_neural_search",
|
||||||
|
tokens_requested: 0,
|
||||||
|
actual_provider_name: "exa",
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await aiApiClient.post("/api/podcast/research/exa", {
|
||||||
|
topic: params.topic || keywords[0],
|
||||||
|
queries: keywords,
|
||||||
|
exa_config: sanitizedExaConfig,
|
||||||
|
bible: params.bible,
|
||||||
|
analysis: params.analysis,
|
||||||
|
});
|
||||||
|
|
||||||
|
const exaResult = response.data as ExaResearchResult;
|
||||||
|
if (params.onProgress) {
|
||||||
|
params.onProgress("Deep research completed with Exa.");
|
||||||
|
}
|
||||||
|
const mapped = mapExaResearchResponse(exaResult);
|
||||||
|
return { research: mapped, raw: exaResult };
|
||||||
|
},
|
||||||
|
|
||||||
|
async generateScript(params: {
|
||||||
|
projectId: string;
|
||||||
|
idea: string;
|
||||||
|
research?: ExaResearchResult | null;
|
||||||
|
knobs: Knobs;
|
||||||
|
speakers: number;
|
||||||
|
durationMinutes: number;
|
||||||
|
bible?: any;
|
||||||
|
outline?: any;
|
||||||
|
analysis?: PodcastAnalysis | null;
|
||||||
|
}): Promise<Script> {
|
||||||
|
await ensurePreflight({
|
||||||
|
provider: "gemini",
|
||||||
|
operation_type: "script_generation",
|
||||||
|
tokens_requested: 2000,
|
||||||
|
actual_provider_name: "gemini",
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await aiApiClient.post("/api/podcast/script", {
|
||||||
|
idea: params.idea,
|
||||||
|
duration_minutes: params.durationMinutes,
|
||||||
|
speakers: params.speakers,
|
||||||
|
research: params.research,
|
||||||
|
bible: params.bible,
|
||||||
|
outline: params.outline,
|
||||||
|
analysis: params.analysis,
|
||||||
|
});
|
||||||
|
|
||||||
|
const scenes = response.data?.scenes || [];
|
||||||
|
const scriptScenes: Scene[] = scenes.map((scene: any) => ({
|
||||||
|
id: scene.id || createId("scene"),
|
||||||
|
title: scene.title || "Scene",
|
||||||
|
duration: scene.duration || Math.max(20, params.knobs.scene_length_target || DEFAULT_KNOBS.scene_length_target),
|
||||||
|
lines:
|
||||||
|
Array.isArray(scene.lines) && scene.lines.length
|
||||||
|
? scene.lines.map((l: any) => ({
|
||||||
|
id: createId("line"),
|
||||||
|
speaker: l.speaker || "Host",
|
||||||
|
text: l.text || "",
|
||||||
|
}))
|
||||||
|
: [
|
||||||
|
{
|
||||||
|
id: createId("line"),
|
||||||
|
speaker: "Host",
|
||||||
|
text: "Let's dive into today's topic.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
approved: false,
|
||||||
|
}));
|
||||||
|
|
||||||
|
return { scenes: scriptScenes };
|
||||||
|
},
|
||||||
|
|
||||||
|
async previewLine(
|
||||||
|
text: string,
|
||||||
|
options: { voiceId?: string; speed?: number; emotion?: string } = {}
|
||||||
|
): Promise<{ ok: boolean; message: string; audioUrl?: string }> {
|
||||||
|
await ensurePreflight({
|
||||||
|
provider: "audio",
|
||||||
|
operation_type: "tts_preview",
|
||||||
|
tokens_requested: text.length,
|
||||||
|
actual_provider_name: "wavespeed",
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await storyWriterApi.generateAIAudio({
|
||||||
|
scene_number: 0,
|
||||||
|
scene_title: "Preview",
|
||||||
|
text,
|
||||||
|
voice_id: options.voiceId || "Wise_Woman",
|
||||||
|
speed: options.speed || 1.0,
|
||||||
|
emotion: options.emotion || "neutral",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.success) {
|
||||||
|
throw new Error(response.error || "Preview failed");
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
ok: true,
|
||||||
|
message: "Preview ready – opening audio in new tab.",
|
||||||
|
audioUrl: response.audio_url,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
|
||||||
|
async renderSceneAudio(params: {
|
||||||
|
scene: Scene;
|
||||||
|
voiceId?: string;
|
||||||
|
emotion?: string; // Fallback if scene doesn't have emotion
|
||||||
|
speed?: number;
|
||||||
|
volume?: number;
|
||||||
|
pitch?: number;
|
||||||
|
englishNormalization?: boolean;
|
||||||
|
sampleRate?: number;
|
||||||
|
bitrate?: number;
|
||||||
|
channel?: "1" | "2";
|
||||||
|
format?: "mp3" | "wav" | "pcm" | "flac";
|
||||||
|
languageBoost?: string;
|
||||||
|
}): Promise<RenderJobResult> {
|
||||||
|
// Use scene-specific emotion if available, otherwise fallback to provided/default
|
||||||
|
const sceneEmotion = params.scene.emotion || params.emotion || "neutral";
|
||||||
|
|
||||||
|
// Optimize text for Minimax Speech-02-HD TTS
|
||||||
|
// - Strip markdown formatting (bold, italic, etc.) - TTS reads it literally
|
||||||
|
// - Use pause markers <#x#> for natural speech rhythm
|
||||||
|
// - Add longer pauses for speaker changes
|
||||||
|
// - Preserve punctuation for natural breathing
|
||||||
|
// - Add emphasis pauses for important points
|
||||||
|
const text = params.scene.lines
|
||||||
|
.map((line, idx) => {
|
||||||
|
let lineText = line.text.trim();
|
||||||
|
|
||||||
|
// Strip markdown formatting - TTS reads asterisks and other markdown literally
|
||||||
|
// Remove bold (**text** or __text__)
|
||||||
|
lineText = lineText.replace(/\*\*([^*]+)\*\*/g, '$1'); // **bold**
|
||||||
|
lineText = lineText.replace(/\*([^*]+)\*/g, '$1'); // *bold* (single asterisk)
|
||||||
|
lineText = lineText.replace(/__([^_]+)__/g, '$1'); // __bold__
|
||||||
|
lineText = lineText.replace(/_([^_]+)_/g, '$1'); // _italic_ (single underscore)
|
||||||
|
// Remove any remaining stray asterisks or underscores
|
||||||
|
lineText = lineText.replace(/\*+/g, ''); // Remove any remaining asterisks
|
||||||
|
lineText = lineText.replace(/_+/g, ''); // Remove any remaining underscores
|
||||||
|
// Clean up extra spaces
|
||||||
|
lineText = lineText.replace(/\s+/g, ' ').trim();
|
||||||
|
|
||||||
|
// Preserve punctuation (Minimax uses it for natural breathing)
|
||||||
|
// Don't strip punctuation - it helps TTS understand natural pauses
|
||||||
|
|
||||||
|
// Add emphasis pause after lines marked with emphasis
|
||||||
|
if (line.emphasis) {
|
||||||
|
// Minimal pause after emphasized content (0.15s for subtle emphasis)
|
||||||
|
lineText = `${lineText}<#0.15#>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check for speaker change (longer pause for natural conversation flow)
|
||||||
|
const prevLine = idx > 0 ? params.scene.lines[idx - 1] : null;
|
||||||
|
const isSpeakerChange = prevLine && prevLine.speaker !== line.speaker;
|
||||||
|
|
||||||
|
if (isSpeakerChange) {
|
||||||
|
// Short pause for speaker changes (0.2s - enough for natural transition)
|
||||||
|
lineText = `<#0.2#>${lineText}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add minimal pause between lines (only between regular lines, very short)
|
||||||
|
if (idx < params.scene.lines.length - 1) {
|
||||||
|
if (!line.emphasis && !isSpeakerChange) {
|
||||||
|
// Very short pause between lines (0.08s - barely noticeable but helps flow)
|
||||||
|
lineText = `${lineText}<#0.08#>`;
|
||||||
|
}
|
||||||
|
// If emphasis or speaker change, the pause is already added above
|
||||||
|
}
|
||||||
|
|
||||||
|
return lineText;
|
||||||
|
})
|
||||||
|
.join(" ");
|
||||||
|
|
||||||
|
// Validate character limit (Minimax max: 10,000 characters)
|
||||||
|
const MAX_CHARS = 10000;
|
||||||
|
let textToUse = text;
|
||||||
|
if (text.length > MAX_CHARS) {
|
||||||
|
console.warn(
|
||||||
|
`[Podcast] Scene "${params.scene.title}" exceeds ${MAX_CHARS} character limit (${text.length} chars). Truncating...`
|
||||||
|
);
|
||||||
|
// Truncate at word boundary to avoid cutting mid-word
|
||||||
|
const truncated = text.substring(0, MAX_CHARS);
|
||||||
|
const lastSpace = truncated.lastIndexOf(" ");
|
||||||
|
textToUse = lastSpace > 0 ? truncated.substring(0, lastSpace) : truncated;
|
||||||
|
}
|
||||||
|
|
||||||
|
await ensurePreflight({
|
||||||
|
provider: "audio",
|
||||||
|
operation_type: "tts_full_render",
|
||||||
|
tokens_requested: textToUse.length,
|
||||||
|
actual_provider_name: "wavespeed",
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await aiApiClient.post("/api/podcast/audio", {
|
||||||
|
scene_id: params.scene.id,
|
||||||
|
scene_title: params.scene.title,
|
||||||
|
text: textToUse,
|
||||||
|
voice_id: params.voiceId || "Wise_Woman",
|
||||||
|
speed: params.speed ?? 1.0, // Normal speed (was 0.9, but too slow - causing duration issues)
|
||||||
|
volume: params.volume ?? 1.0,
|
||||||
|
pitch: params.pitch ?? 0.0,
|
||||||
|
emotion: sceneEmotion,
|
||||||
|
english_normalization: params.englishNormalization ?? true, // Better number reading for statistics
|
||||||
|
sample_rate: params.sampleRate || null,
|
||||||
|
bitrate: params.bitrate || null,
|
||||||
|
channel: params.channel || null,
|
||||||
|
format: params.format || null,
|
||||||
|
language_boost: params.languageBoost || null,
|
||||||
|
});
|
||||||
|
|
||||||
|
return {
|
||||||
|
audioUrl: response.data.audio_url,
|
||||||
|
audioFilename: response.data.audio_filename,
|
||||||
|
provider: response.data.provider,
|
||||||
|
model: response.data.model,
|
||||||
|
cost: response.data.cost,
|
||||||
|
voiceId: response.data.voice_id,
|
||||||
|
fileSize: response.data.file_size,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
|
||||||
|
async approveScene(params: { projectId: string; sceneId: string; notes?: string }) {
|
||||||
|
await aiApiClient.post("/api/story/script/approve", {
|
||||||
|
project_id: params.projectId,
|
||||||
|
scene_id: params.sceneId,
|
||||||
|
approved: true,
|
||||||
|
notes: params.notes,
|
||||||
|
});
|
||||||
|
},
|
||||||
|
|
||||||
|
// Project persistence endpoints
|
||||||
|
async saveProject(projectId: string, state: any): Promise<void> {
|
||||||
|
try {
|
||||||
|
await aiApiClient.put(`/api/podcast/projects/${projectId}`, state);
|
||||||
|
} catch (error) {
|
||||||
|
console.error("Failed to save project to database:", error);
|
||||||
|
// Don't throw - localStorage fallback is acceptable
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async loadProject(projectId: string): Promise<any> {
|
||||||
|
const response = await aiApiClient.get(`/api/podcast/projects/${projectId}`);
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async listProjects(params?: {
|
||||||
|
status?: string;
|
||||||
|
favorites_only?: boolean;
|
||||||
|
limit?: number;
|
||||||
|
offset?: number;
|
||||||
|
order_by?: "updated_at" | "created_at";
|
||||||
|
}): Promise<{ projects: any[]; total: number; limit: number; offset: number }> {
|
||||||
|
const response = await aiApiClient.get("/api/podcast/projects", { params });
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async createProjectInDb(params: {
|
||||||
|
project_id: string;
|
||||||
|
idea: string;
|
||||||
|
duration: number;
|
||||||
|
speakers: number;
|
||||||
|
budget_cap: number;
|
||||||
|
avatar_url?: string | null;
|
||||||
|
}): Promise<any> {
|
||||||
|
const response = await aiApiClient.post("/api/podcast/projects", params);
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async updateProject(projectId: string, updates: any): Promise<any> {
|
||||||
|
const response = await aiApiClient.put(`/api/podcast/projects/${projectId}`, updates);
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async deleteProject(projectId: string): Promise<void> {
|
||||||
|
await aiApiClient.delete(`/api/podcast/projects/${projectId}`);
|
||||||
|
},
|
||||||
|
|
||||||
|
async toggleFavorite(projectId: string): Promise<any> {
|
||||||
|
const response = await aiApiClient.post(`/api/podcast/projects/${projectId}/favorite`);
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async saveAudioToAssetLibrary(params: {
|
||||||
|
audioUrl: string;
|
||||||
|
filename: string;
|
||||||
|
title: string;
|
||||||
|
description?: string;
|
||||||
|
projectId: string;
|
||||||
|
sceneId?: string;
|
||||||
|
cost?: number;
|
||||||
|
provider?: string;
|
||||||
|
model?: string;
|
||||||
|
fileSize?: number;
|
||||||
|
}): Promise<{ assetId: number }> {
|
||||||
|
const response = await aiApiClient.post("/api/content-assets/", {
|
||||||
|
asset_type: "audio",
|
||||||
|
source_module: "podcast_maker",
|
||||||
|
filename: params.filename,
|
||||||
|
file_url: params.audioUrl,
|
||||||
|
title: params.title,
|
||||||
|
description: params.description || `Podcast episode audio: ${params.title}`,
|
||||||
|
tags: ["podcast", "audio", params.projectId],
|
||||||
|
asset_metadata: {
|
||||||
|
project_id: params.projectId,
|
||||||
|
scene_id: params.sceneId,
|
||||||
|
provider: params.provider,
|
||||||
|
model: params.model,
|
||||||
|
},
|
||||||
|
provider: params.provider,
|
||||||
|
model: params.model,
|
||||||
|
cost: params.cost || 0,
|
||||||
|
file_size: params.fileSize,
|
||||||
|
mime_type: "audio/mpeg",
|
||||||
|
});
|
||||||
|
return { assetId: response.data.id };
|
||||||
|
},
|
||||||
|
|
||||||
|
async generateVideo(params: {
|
||||||
|
projectId: string;
|
||||||
|
sceneId: string;
|
||||||
|
sceneTitle: string;
|
||||||
|
audioUrl: string;
|
||||||
|
avatarImageUrl?: string;
|
||||||
|
bible?: any;
|
||||||
|
resolution?: string;
|
||||||
|
prompt?: string;
|
||||||
|
seed?: number;
|
||||||
|
maskImageUrl?: string;
|
||||||
|
}): Promise<{ taskId: string; status: string; message: string }> {
|
||||||
|
const response = await aiApiClient.post("/api/podcast/render/video", {
|
||||||
|
project_id: params.projectId,
|
||||||
|
scene_id: params.sceneId,
|
||||||
|
scene_title: params.sceneTitle,
|
||||||
|
audio_url: params.audioUrl,
|
||||||
|
avatar_image_url: params.avatarImageUrl,
|
||||||
|
bible: params.bible,
|
||||||
|
resolution: params.resolution || "720p",
|
||||||
|
prompt: params.prompt,
|
||||||
|
seed: params.seed ?? -1,
|
||||||
|
mask_image_url: params.maskImageUrl,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Backend returns snake_case (task_id); normalize to camelCase for callers
|
||||||
|
const { task_id, status, message } = response.data || {};
|
||||||
|
return {
|
||||||
|
taskId: task_id,
|
||||||
|
status,
|
||||||
|
message,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
|
||||||
|
async pollTaskStatus(taskId: string): Promise<TaskStatus | null> {
|
||||||
|
const response = await aiApiClient.get(`/api/podcast/task/${taskId}/status`);
|
||||||
|
// Backend returns null if task not found
|
||||||
|
return response.data || null;
|
||||||
|
},
|
||||||
|
|
||||||
|
async listVideos(projectId?: string): Promise<{
|
||||||
|
videos: Array<{
|
||||||
|
scene_number: number;
|
||||||
|
filename: string;
|
||||||
|
video_url: string;
|
||||||
|
file_size: number;
|
||||||
|
}>;
|
||||||
|
}> {
|
||||||
|
const params = projectId ? { project_id: projectId } : {};
|
||||||
|
const response = await aiApiClient.get("/api/podcast/videos", { params });
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async combineVideos(params: {
|
||||||
|
projectId: string;
|
||||||
|
sceneVideoUrls: string[];
|
||||||
|
podcastTitle?: string;
|
||||||
|
}): Promise<{
|
||||||
|
taskId: string;
|
||||||
|
status: string;
|
||||||
|
message: string;
|
||||||
|
}> {
|
||||||
|
const response = await aiApiClient.post("/api/podcast/render/combine-videos", {
|
||||||
|
project_id: params.projectId,
|
||||||
|
scene_video_urls: params.sceneVideoUrls,
|
||||||
|
podcast_title: params.podcastTitle || "Podcast",
|
||||||
|
});
|
||||||
|
|
||||||
|
const { task_id, status, message } = response.data || {};
|
||||||
|
return {
|
||||||
|
taskId: task_id,
|
||||||
|
status,
|
||||||
|
message,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
|
||||||
|
async generateSceneImage(params: {
|
||||||
|
sceneId: string;
|
||||||
|
sceneTitle: string;
|
||||||
|
sceneContent?: string;
|
||||||
|
baseAvatarUrl?: string;
|
||||||
|
bible?: any;
|
||||||
|
idea?: string;
|
||||||
|
width?: number;
|
||||||
|
height?: number;
|
||||||
|
customPrompt?: string;
|
||||||
|
style?: "Auto" | "Fiction" | "Realistic";
|
||||||
|
renderingSpeed?: "Default" | "Turbo" | "Quality";
|
||||||
|
aspectRatio?: "1:1" | "16:9" | "9:16" | "4:3" | "3:4";
|
||||||
|
}): Promise<{
|
||||||
|
scene_id: string;
|
||||||
|
scene_title: string;
|
||||||
|
image_filename: string;
|
||||||
|
image_url: string;
|
||||||
|
width: number;
|
||||||
|
height: number;
|
||||||
|
provider: string;
|
||||||
|
model?: string;
|
||||||
|
cost: number;
|
||||||
|
}> {
|
||||||
|
const response = await aiApiClient.post("/api/podcast/image", {
|
||||||
|
scene_id: params.sceneId,
|
||||||
|
scene_title: params.sceneTitle,
|
||||||
|
scene_content: params.sceneContent,
|
||||||
|
base_avatar_url: params.baseAvatarUrl || null,
|
||||||
|
bible: params.bible,
|
||||||
|
idea: params.idea || null,
|
||||||
|
width: params.width || 1024,
|
||||||
|
height: params.height || 1024,
|
||||||
|
custom_prompt: params.customPrompt || null,
|
||||||
|
style: params.style || null,
|
||||||
|
rendering_speed: params.renderingSpeed || null,
|
||||||
|
aspect_ratio: params.aspectRatio || null,
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async cancelTask(taskId: string): Promise<void> {
|
||||||
|
// Note: Task cancellation may not be fully supported by backend yet
|
||||||
|
// This is a placeholder for future implementation
|
||||||
|
try {
|
||||||
|
await aiApiClient.post(`/api/story/task/${taskId}/cancel`);
|
||||||
|
} catch (error) {
|
||||||
|
console.warn("Task cancellation not supported:", error);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async combineAudio(params: {
|
||||||
|
projectId: string;
|
||||||
|
sceneIds: string[];
|
||||||
|
sceneAudioUrls: string[];
|
||||||
|
}): Promise<{
|
||||||
|
combined_audio_url: string;
|
||||||
|
combined_audio_filename: string;
|
||||||
|
total_duration: number;
|
||||||
|
file_size: number;
|
||||||
|
scene_count: number;
|
||||||
|
}> {
|
||||||
|
const response = await aiApiClient.post("/api/podcast/combine-audio", {
|
||||||
|
project_id: params.projectId,
|
||||||
|
scene_ids: params.sceneIds,
|
||||||
|
scene_audio_urls: params.sceneAudioUrls,
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async uploadAvatar(file: File, projectId?: string): Promise<{ avatar_url: string; avatar_filename: string }> {
|
||||||
|
const formData = new FormData();
|
||||||
|
formData.append('file', file);
|
||||||
|
if (projectId) {
|
||||||
|
formData.append('project_id', projectId);
|
||||||
|
}
|
||||||
|
const response = await aiApiClient.post('/api/podcast/avatar/upload', formData, {
|
||||||
|
headers: { 'Content-Type': 'multipart/form-data' },
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async generatePresenters(
|
||||||
|
speakers: number,
|
||||||
|
projectId?: string,
|
||||||
|
audience?: string,
|
||||||
|
contentType?: string,
|
||||||
|
topKeywords?: string[]
|
||||||
|
): Promise<{
|
||||||
|
avatars: Array<{ avatar_url: string; speaker_number: number; prompt?: string; persona_id?: string; seed?: number }>;
|
||||||
|
persona_id?: string;
|
||||||
|
}> {
|
||||||
|
const formData = new FormData();
|
||||||
|
formData.append('speakers', speakers.toString());
|
||||||
|
if (projectId) {
|
||||||
|
formData.append('project_id', projectId);
|
||||||
|
}
|
||||||
|
if (audience) {
|
||||||
|
formData.append('audience', audience);
|
||||||
|
}
|
||||||
|
if (contentType) {
|
||||||
|
formData.append('content_type', contentType);
|
||||||
|
}
|
||||||
|
if (topKeywords && Array.isArray(topKeywords) && topKeywords.length > 0) {
|
||||||
|
formData.append('top_keywords', JSON.stringify(topKeywords));
|
||||||
|
}
|
||||||
|
const response = await aiApiClient.post('/api/podcast/avatar/generate', formData, {
|
||||||
|
headers: { 'Content-Type': 'multipart/form-data' },
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
|
||||||
|
async makeAvatarPresentable(avatarUrl: string, projectId?: string): Promise<{ avatar_url: string; avatar_filename: string }> {
|
||||||
|
const formData = new FormData();
|
||||||
|
formData.append('avatar_url', avatarUrl);
|
||||||
|
if (projectId) {
|
||||||
|
formData.append('project_id', projectId);
|
||||||
|
}
|
||||||
|
const response = await aiApiClient.post('/api/podcast/avatar/make-presentable', formData, {
|
||||||
|
headers: { 'Content-Type': 'multipart/form-data' },
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
export type PodcastApi = typeof podcastApi;
|
||||||
|
|
||||||
244
_session_backup/research.py
Normal file
244
_session_backup/research.py
Normal file
@@ -0,0 +1,244 @@
|
|||||||
|
"""
|
||||||
|
Podcast Research Handlers
|
||||||
|
|
||||||
|
Research endpoints using Exa provider and LLM summarization.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
|
from typing import Dict, Any, List
|
||||||
|
from types import SimpleNamespace
|
||||||
|
import json
|
||||||
|
|
||||||
|
from middleware.auth_middleware import get_current_user
|
||||||
|
from api.story_writer.utils.auth import require_authenticated_user
|
||||||
|
from services.blog_writer.research.exa_provider import ExaResearchProvider
|
||||||
|
from services.llm_providers.main_text_generation import llm_text_gen
|
||||||
|
from services.podcast_bible_service import PodcastBibleService
|
||||||
|
from loguru import logger
|
||||||
|
from ..models import (
|
||||||
|
PodcastExaResearchRequest,
|
||||||
|
PodcastExaResearchResponse,
|
||||||
|
PodcastExaSource,
|
||||||
|
PodcastExaConfig,
|
||||||
|
PodcastResearchInsight,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/research/exa", response_model=PodcastExaResearchResponse)
|
||||||
|
async def podcast_research_exa(
|
||||||
|
request: PodcastExaResearchRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Run podcast research via Exa and then use LLM to extract deep insights.
|
||||||
|
Uses Podcast Bible and Analysis context for hyper-personalization.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
queries = [q.strip() for q in request.queries if q and q.strip()]
|
||||||
|
if not queries:
|
||||||
|
raise HTTPException(status_code=400, detail="At least one query is required for research.")
|
||||||
|
|
||||||
|
exa_cfg = request.exa_config or PodcastExaConfig()
|
||||||
|
cfg = SimpleNamespace(
|
||||||
|
exa_search_type=exa_cfg.exa_search_type or "auto",
|
||||||
|
exa_category=exa_cfg.exa_category,
|
||||||
|
exa_include_domains=exa_cfg.exa_include_domains or [],
|
||||||
|
exa_exclude_domains=exa_cfg.exa_exclude_domains or [],
|
||||||
|
max_sources=exa_cfg.max_sources or 8,
|
||||||
|
source_types=[],
|
||||||
|
)
|
||||||
|
|
||||||
|
provider = ExaResearchProvider()
|
||||||
|
|
||||||
|
# --- Context Building ---
|
||||||
|
bible_service = PodcastBibleService()
|
||||||
|
bible_context = ""
|
||||||
|
if request.bible:
|
||||||
|
try:
|
||||||
|
from models.podcast_bible_models import PodcastBible
|
||||||
|
bible_data = PodcastBible(**request.bible)
|
||||||
|
bible_context = bible_service.serialize_bible(bible_data)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"[Podcast Research] Failed to serialize bible: {exc}")
|
||||||
|
|
||||||
|
analysis_context = ""
|
||||||
|
if request.analysis:
|
||||||
|
analysis_context = f"""
|
||||||
|
PODCAST ANALYSIS CONTEXT:
|
||||||
|
Audience: {request.analysis.get('audience', 'General')}
|
||||||
|
Content Type: {request.analysis.get('content_type', 'Informative')}
|
||||||
|
Top Keywords: {', '.join(request.analysis.get('top_keywords', []))}
|
||||||
|
"""
|
||||||
|
|
||||||
|
# Exa search params
|
||||||
|
industry = request.bible.get("brand", {}).get("industry", "") if request.bible else ""
|
||||||
|
target_audience = ""
|
||||||
|
if request.bible:
|
||||||
|
audience_dna = request.bible.get("audience", {})
|
||||||
|
if audience_dna:
|
||||||
|
interests = ", ".join(audience_dna.get("interests", []))
|
||||||
|
target_audience = f"Expertise: {audience_dna.get('expertise_level', '')}. Interests: {interests}."
|
||||||
|
|
||||||
|
try:
|
||||||
|
# 1. RUN EXA SEARCH
|
||||||
|
result = await provider.search(
|
||||||
|
prompt=request.topic,
|
||||||
|
topic=request.topic,
|
||||||
|
industry=industry,
|
||||||
|
target_audience=target_audience,
|
||||||
|
config=cfg,
|
||||||
|
user_id=user_id,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error(f"[Podcast Exa Research] Search failed for user {user_id}: {exc}")
|
||||||
|
raise HTTPException(status_code=500, detail=f"Exa research failed: {exc}")
|
||||||
|
|
||||||
|
# 2. EXTRACT INSIGHTS VIA LLM
|
||||||
|
raw_content = result.get("content", "")
|
||||||
|
sources = result.get("sources", [])
|
||||||
|
|
||||||
|
summary = ""
|
||||||
|
key_insights = []
|
||||||
|
expert_quotes = []
|
||||||
|
listener_cta = []
|
||||||
|
mapped_angles = []
|
||||||
|
|
||||||
|
if raw_content and sources:
|
||||||
|
logger.info(f"[Podcast Research] Extracting insights from {len(sources)} sources for user {user_id}")
|
||||||
|
|
||||||
|
prompt = f"""
|
||||||
|
You are an expert research analyst for a high-end podcast production team.
|
||||||
|
Your task is to analyze the following research data and extract deep, actionable insights for a podcast episode.
|
||||||
|
|
||||||
|
PODCAST CONTEXT:
|
||||||
|
Topic: {request.topic}
|
||||||
|
{bible_context}
|
||||||
|
{analysis_context}
|
||||||
|
|
||||||
|
RESEARCH DATA (from {len(sources)} sources):
|
||||||
|
{raw_content}
|
||||||
|
|
||||||
|
TASK:
|
||||||
|
1. Provide a comprehensive summary (2-3 paragraphs) of the most important findings. Use Markdown for formatting (bolding, lists).
|
||||||
|
2. Extract 3-5 "Key Insights". Each insight should have a title and a detailed explanation.
|
||||||
|
3. For each insight, identify which source indices (e.g. 1, 2) it was derived from.
|
||||||
|
4. Extract notable "Expert Quotes" - direct quotes from industry leaders, researchers, or authoritative voices found in the sources.
|
||||||
|
5. Suggest 2-4 "Listener CTA" (call-to-action) ideas that the podcast host can use to engage the audience.
|
||||||
|
6. Identify 3-5 "Mapped Angles" - unique content angles with rationale for why they matter for this topic.
|
||||||
|
|
||||||
|
NOTE: The research data includes "Key Highlights", "Summaries", and "Excerpts" from various sources.
|
||||||
|
Pay special attention to the "Key Highlights" sections as they contain the most relevant information extracted by the neural search engine.
|
||||||
|
|
||||||
|
Return JSON structure:
|
||||||
|
{{
|
||||||
|
"summary": "Detailed markdown summary...",
|
||||||
|
"key_insights": [
|
||||||
|
{{
|
||||||
|
"title": "Insight Title",
|
||||||
|
"content": "Detailed markdown content...",
|
||||||
|
"source_indices": [1, 2]
|
||||||
|
}}
|
||||||
|
],
|
||||||
|
"expert_quotes": [
|
||||||
|
{{
|
||||||
|
"quote": "Exact quote from source...",
|
||||||
|
"source_index": 1
|
||||||
|
}}
|
||||||
|
],
|
||||||
|
"listener_cta": [
|
||||||
|
"Call-to-action suggestion 1",
|
||||||
|
"Call-to-action suggestion 2"
|
||||||
|
],
|
||||||
|
"mapped_angles": [
|
||||||
|
{{
|
||||||
|
"title": "Angle Title",
|
||||||
|
"why": "Why this angle matters for the audience...",
|
||||||
|
"mapped_fact_ids": ["fact_1", "fact_2"]
|
||||||
|
}}
|
||||||
|
]
|
||||||
|
}}
|
||||||
|
|
||||||
|
Requirements:
|
||||||
|
- Ensure insights are deep, not just superficial facts. Look for trends, expert opinions, and specific data points.
|
||||||
|
- Expert quotes should be exact or near-exact quotes from the sources, with attribution.
|
||||||
|
- Listener CTAs should be practical and engaging (e.g., "Share your experience with X on social media").
|
||||||
|
- Mapped angles should be unique perspectives that make the episode stand out.
|
||||||
|
- Tone should be professional, insightful, and ready for a podcast host to discuss.
|
||||||
|
- Avoid generic filler.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
llm_response = llm_text_gen(
|
||||||
|
prompt=prompt,
|
||||||
|
user_id=user_id,
|
||||||
|
json_struct=None,
|
||||||
|
preferred_provider="huggingface",
|
||||||
|
flow_type="premium_tool",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Normalize response
|
||||||
|
if isinstance(llm_response, str):
|
||||||
|
data = json.loads(llm_response)
|
||||||
|
else:
|
||||||
|
data = llm_response
|
||||||
|
|
||||||
|
summary = data.get("summary", "")
|
||||||
|
key_insights = [PodcastResearchInsight(**insight) for insight in data.get("key_insights", [])]
|
||||||
|
expert_quotes = data.get("expert_quotes", [])
|
||||||
|
listener_cta = data.get("listener_cta", [])
|
||||||
|
mapped_angles = data.get("mapped_angles", [])
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error(f"[Podcast Research] LLM Insight extraction failed: {exc}")
|
||||||
|
# Fallback to a basic summary if LLM fails
|
||||||
|
summary = f"Research completed for '{request.topic}'. Found {len(sources)} sources."
|
||||||
|
|
||||||
|
# Fallback: if summary is still empty (e.g. LLM returned empty string), use raw content first paragraph or basic text
|
||||||
|
if not summary:
|
||||||
|
if raw_content:
|
||||||
|
summary = raw_content[:2000] # Use first 2000 chars of raw content as summary
|
||||||
|
else:
|
||||||
|
summary = f"Research completed for '{request.topic}'. Found {len(sources)} sources."
|
||||||
|
|
||||||
|
# 3. TRACK USAGE
|
||||||
|
try:
|
||||||
|
cost_total = 0.0
|
||||||
|
if isinstance(result, dict):
|
||||||
|
cost_total = result.get("cost", {}).get("total", 0.005) if result.get("cost") else 0.005
|
||||||
|
provider.track_exa_usage(user_id, cost_total)
|
||||||
|
except Exception as track_err:
|
||||||
|
logger.warning(f"[Podcast Exa Research] Failed to track usage: {track_err}")
|
||||||
|
|
||||||
|
sources_payload = []
|
||||||
|
for src in sources:
|
||||||
|
try:
|
||||||
|
sources_payload.append(PodcastExaSource(**src))
|
||||||
|
except Exception:
|
||||||
|
sources_payload.append(PodcastExaSource(**{
|
||||||
|
"title": src.get("title", ""),
|
||||||
|
"url": src.get("url", ""),
|
||||||
|
"excerpt": src.get("excerpt", ""),
|
||||||
|
"published_at": src.get("published_at"),
|
||||||
|
"highlights": src.get("highlights"),
|
||||||
|
"summary": src.get("summary"),
|
||||||
|
"source_type": src.get("source_type"),
|
||||||
|
"index": src.get("index"),
|
||||||
|
"image": src.get("image"),
|
||||||
|
"author": src.get("author"),
|
||||||
|
}))
|
||||||
|
|
||||||
|
return PodcastExaResearchResponse(
|
||||||
|
sources=sources_payload,
|
||||||
|
search_queries=result.get("search_queries", queries) if isinstance(result, dict) else queries,
|
||||||
|
summary=summary,
|
||||||
|
key_insights=key_insights,
|
||||||
|
expert_quotes=expert_quotes,
|
||||||
|
listener_cta=listener_cta,
|
||||||
|
mapped_angles=mapped_angles,
|
||||||
|
cost=result.get("cost") if isinstance(result, dict) else None,
|
||||||
|
search_type=result.get("search_type") if isinstance(result, dict) else None,
|
||||||
|
provider=result.get("provider", "exa") if isinstance(result, dict) else "exa",
|
||||||
|
content=raw_content,
|
||||||
|
)
|
||||||
|
|
||||||
183
_session_backup/script.py
Normal file
183
_session_backup/script.py
Normal file
@@ -0,0 +1,183 @@
|
|||||||
|
"""
|
||||||
|
Podcast Script Handlers
|
||||||
|
|
||||||
|
Script generation endpoint.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
|
from typing import Dict, Any
|
||||||
|
import json
|
||||||
|
|
||||||
|
from middleware.auth_middleware import get_current_user
|
||||||
|
from api.story_writer.utils.auth import require_authenticated_user
|
||||||
|
from services.llm_providers.main_text_generation import llm_text_gen
|
||||||
|
from services.podcast_bible_service import PodcastBibleService
|
||||||
|
from models.podcast_bible_models import PodcastBible
|
||||||
|
from loguru import logger
|
||||||
|
from ..models import (
|
||||||
|
PodcastScriptRequest,
|
||||||
|
PodcastScriptResponse,
|
||||||
|
PodcastScene,
|
||||||
|
PodcastSceneLine,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/script", response_model=PodcastScriptResponse)
|
||||||
|
async def generate_podcast_script(
|
||||||
|
request: PodcastScriptRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Generate a podcast script outline (scenes + lines) using podcast-oriented prompting.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
# Build comprehensive research context for higher-quality scripts
|
||||||
|
research_context = ""
|
||||||
|
if request.research:
|
||||||
|
try:
|
||||||
|
key_insights = request.research.get("keyword_analysis", {}).get("key_insights") or []
|
||||||
|
fact_cards = request.research.get("factCards", []) or []
|
||||||
|
mapped_angles = request.research.get("mappedAngles", []) or []
|
||||||
|
sources = request.research.get("sources", []) or []
|
||||||
|
|
||||||
|
top_facts = [f.get("quote", "") for f in fact_cards[:5] if f.get("quote")]
|
||||||
|
angles_summary = [
|
||||||
|
f"{a.get('title', '')}: {a.get('why', '')}" for a in mapped_angles[:3] if a.get("title") or a.get("why")
|
||||||
|
]
|
||||||
|
top_sources = [s.get("url") for s in sources[:3] if s.get("url")]
|
||||||
|
|
||||||
|
research_parts = []
|
||||||
|
if key_insights:
|
||||||
|
research_parts.append(f"Key Insights: {', '.join(key_insights[:5])}")
|
||||||
|
if top_facts:
|
||||||
|
research_parts.append(f"Key Facts: {', '.join(top_facts)}")
|
||||||
|
if angles_summary:
|
||||||
|
research_parts.append(f"Research Angles: {' | '.join(angles_summary)}")
|
||||||
|
if top_sources:
|
||||||
|
research_parts.append(f"Top Sources: {', '.join(top_sources)}")
|
||||||
|
|
||||||
|
research_context = "\n".join(research_parts)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"Failed to parse research context: {exc}")
|
||||||
|
research_context = ""
|
||||||
|
|
||||||
|
# Extract Podcast Bible context for hyper-personalization
|
||||||
|
bible_context = ""
|
||||||
|
if request.bible:
|
||||||
|
try:
|
||||||
|
bible_service = PodcastBibleService()
|
||||||
|
bible_obj = PodcastBible(**request.bible)
|
||||||
|
bible_context = bible_service.serialize_bible(bible_obj)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"Failed to serialize podcast bible: {exc}")
|
||||||
|
|
||||||
|
# Extract Analysis and Outline context for grounding
|
||||||
|
analysis_context = ""
|
||||||
|
if request.analysis:
|
||||||
|
analysis_context = f"""
|
||||||
|
TARGET AUDIENCE: {request.analysis.get('audience', 'General')}
|
||||||
|
CONTENT TYPE: {request.analysis.get('contentType', 'Conversational')}
|
||||||
|
TOP KEYWORDS: {', '.join(request.analysis.get('topKeywords', []))}
|
||||||
|
"""
|
||||||
|
|
||||||
|
outline_context = ""
|
||||||
|
if request.outline:
|
||||||
|
outline_context = f"""
|
||||||
|
REFINED EPISODE OUTLINE (Follow this structure closely):
|
||||||
|
Title: {request.outline.get('title', 'N/A')}
|
||||||
|
Segments: {' | '.join(request.outline.get('segments', []))}
|
||||||
|
"""
|
||||||
|
|
||||||
|
prompt = f"""You are an expert podcast script planner. Create natural, conversational podcast scenes.
|
||||||
|
|
||||||
|
{f"PODCAST BIBLE (Hyper-Personalization Context):\n{bible_context}\n" if bible_context else ""}
|
||||||
|
{f"ANALYSIS CONTEXT:\n{analysis_context}\n" if analysis_context else ""}
|
||||||
|
{f"REFINED OUTLINE:\n{outline_context}\n" if outline_context else ""}
|
||||||
|
|
||||||
|
Podcast Idea: "{request.idea}"
|
||||||
|
Duration: ~{request.duration_minutes} minutes
|
||||||
|
Speakers: {request.speakers} (Host + optional Guest)
|
||||||
|
|
||||||
|
{f"RESEARCH CONTEXT:\n{research_context}\n" if research_context else ""}
|
||||||
|
|
||||||
|
Return JSON with:
|
||||||
|
- scenes: array of scenes. Each scene has:
|
||||||
|
- id: string
|
||||||
|
- title: short scene title (<= 60 chars)
|
||||||
|
- duration: duration in seconds (evenly split across total duration)
|
||||||
|
- emotion: string (one of: "neutral", "happy", "excited", "serious", "curious", "confident")
|
||||||
|
- lines: array of {{"speaker": "...", "text": "...", "emphasis": boolean}}
|
||||||
|
* Write natural, conversational dialogue
|
||||||
|
* Each line can be a sentence or a few sentences that flow together
|
||||||
|
* Use plain text only - no markdown formatting (no asterisks, underscores, etc.)
|
||||||
|
* Mark "emphasis": true for key statistics or important points
|
||||||
|
|
||||||
|
Guidelines:
|
||||||
|
- Write for spoken delivery: conversational, natural, with contractions.
|
||||||
|
- Follow the interaction tone specified in the Bible.
|
||||||
|
- Ensure the Host persona matches the background and personality traits from the Bible.
|
||||||
|
- Structure the intro and outro scenes according to the Bible's "Intro Format" and "Outro Format".
|
||||||
|
- Adhere to any constraints mentioned in the Bible.
|
||||||
|
- Use insights from the Research Context to ground the conversation in facts.
|
||||||
|
- IMPORTANT: Follow the REFINED OUTLINE segments as the primary structure for the episode.
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
raw = llm_text_gen(
|
||||||
|
prompt=prompt,
|
||||||
|
user_id=user_id,
|
||||||
|
json_struct=None,
|
||||||
|
preferred_provider="huggingface",
|
||||||
|
flow_type="premium_tool",
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=500, detail=f"Script generation failed: {exc}")
|
||||||
|
|
||||||
|
if isinstance(raw, str):
|
||||||
|
try:
|
||||||
|
data = json.loads(raw)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
raise HTTPException(status_code=500, detail="LLM returned non-JSON output")
|
||||||
|
elif isinstance(raw, dict):
|
||||||
|
data = raw
|
||||||
|
else:
|
||||||
|
raise HTTPException(status_code=500, detail="Unexpected LLM response format")
|
||||||
|
|
||||||
|
scenes_data = data.get("scenes") or []
|
||||||
|
if not isinstance(scenes_data, list):
|
||||||
|
raise HTTPException(status_code=500, detail="LLM response missing scenes array")
|
||||||
|
|
||||||
|
valid_emotions = {"neutral", "happy", "excited", "serious", "curious", "confident"}
|
||||||
|
|
||||||
|
# Normalize scenes
|
||||||
|
scenes: list[PodcastScene] = []
|
||||||
|
for idx, scene in enumerate(scenes_data):
|
||||||
|
title = scene.get("title") or f"Scene {idx + 1}"
|
||||||
|
duration = int(scene.get("duration") or max(30, (request.duration_minutes * 60) // max(1, len(scenes_data))))
|
||||||
|
emotion = scene.get("emotion") or "neutral"
|
||||||
|
if emotion not in valid_emotions:
|
||||||
|
emotion = "neutral"
|
||||||
|
lines_raw = scene.get("lines") or []
|
||||||
|
lines: list[PodcastSceneLine] = []
|
||||||
|
for line in lines_raw:
|
||||||
|
speaker = line.get("speaker") or ("Host" if len(lines) % request.speakers == 0 else "Guest")
|
||||||
|
text = line.get("text") or ""
|
||||||
|
emphasis = line.get("emphasis", False)
|
||||||
|
if text:
|
||||||
|
lines.append(PodcastSceneLine(speaker=speaker, text=text, emphasis=emphasis))
|
||||||
|
scenes.append(
|
||||||
|
PodcastScene(
|
||||||
|
id=scene.get("id") or f"scene-{idx + 1}",
|
||||||
|
title=title,
|
||||||
|
duration=duration,
|
||||||
|
lines=lines,
|
||||||
|
approved=False,
|
||||||
|
emotion=emotion,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
return PodcastScriptResponse(scenes=scenes)
|
||||||
|
|
||||||
209
_session_backup/types.ts
Normal file
209
_session_backup/types.ts
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
export type Knobs = {
|
||||||
|
voice_emotion: string;
|
||||||
|
voice_speed: number;
|
||||||
|
resolution: string;
|
||||||
|
scene_length_target: number;
|
||||||
|
sample_rate: number;
|
||||||
|
bitrate: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Query = {
|
||||||
|
id: string;
|
||||||
|
query: string;
|
||||||
|
rationale: string;
|
||||||
|
needsRecentStats: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Fact = {
|
||||||
|
id: string;
|
||||||
|
quote: string;
|
||||||
|
url: string;
|
||||||
|
date: string;
|
||||||
|
confidence: number;
|
||||||
|
image?: string;
|
||||||
|
author?: string;
|
||||||
|
highlights?: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ResearchInsight = {
|
||||||
|
title: string;
|
||||||
|
content: string;
|
||||||
|
source_indices: number[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Research = {
|
||||||
|
summary: string;
|
||||||
|
keyInsights: ResearchInsight[];
|
||||||
|
factCards: Fact[];
|
||||||
|
mappedAngles: {
|
||||||
|
title: string;
|
||||||
|
why: string;
|
||||||
|
mappedFactIds: string[];
|
||||||
|
}[];
|
||||||
|
searchQueries?: string[];
|
||||||
|
searchType?: string;
|
||||||
|
provider?: string;
|
||||||
|
cost?: number;
|
||||||
|
sourceCount?: number;
|
||||||
|
expertQuotes?: { quote: string; source_index: number }[];
|
||||||
|
listenerCta?: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Line = {
|
||||||
|
id: string;
|
||||||
|
speaker: string;
|
||||||
|
text: string;
|
||||||
|
usedFactIds?: string[];
|
||||||
|
emphasis?: boolean; // Mark lines that need vocal emphasis
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Scene = {
|
||||||
|
id: string;
|
||||||
|
title: string;
|
||||||
|
duration: number;
|
||||||
|
lines: Line[];
|
||||||
|
approved?: boolean;
|
||||||
|
emotion?: string; // Scene-specific emotion
|
||||||
|
audioUrl?: string; // Generated audio URL for this scene
|
||||||
|
imageUrl?: string; // Generated image URL for this scene (for video generation)
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Script = {
|
||||||
|
scenes: Scene[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type JobStatus =
|
||||||
|
| "idle"
|
||||||
|
| "previewing"
|
||||||
|
| "queued"
|
||||||
|
| "running"
|
||||||
|
| "completed"
|
||||||
|
| "cancelled"
|
||||||
|
| "failed";
|
||||||
|
|
||||||
|
export type Job = {
|
||||||
|
sceneId: string;
|
||||||
|
title: string;
|
||||||
|
status: JobStatus;
|
||||||
|
progress: number;
|
||||||
|
previewUrl?: string | null;
|
||||||
|
finalUrl?: string | null;
|
||||||
|
videoUrl?: string | null;
|
||||||
|
jobId?: string | null;
|
||||||
|
taskId?: string | null;
|
||||||
|
cost?: number | null;
|
||||||
|
provider?: string | null;
|
||||||
|
voiceId?: string | null;
|
||||||
|
fileSize?: number | null;
|
||||||
|
avatarImageUrl?: string | null;
|
||||||
|
imageUrl?: string | null; // Scene-specific image URL
|
||||||
|
};
|
||||||
|
|
||||||
|
export type PodcastAnalysis = {
|
||||||
|
audience: string;
|
||||||
|
contentType: string;
|
||||||
|
topKeywords: string[];
|
||||||
|
suggestedOutlines: { id: number | string; title: string; segments: string[] }[];
|
||||||
|
suggestedKnobs: Knobs;
|
||||||
|
titleSuggestions: string[];
|
||||||
|
research_queries?: { query: string; rationale: string }[];
|
||||||
|
exaSuggestedConfig?: {
|
||||||
|
exa_search_type?: "auto" | "keyword" | "neural";
|
||||||
|
exa_category?: string;
|
||||||
|
exa_include_domains?: string[];
|
||||||
|
exa_exclude_domains?: string[];
|
||||||
|
max_sources?: number;
|
||||||
|
include_statistics?: boolean;
|
||||||
|
date_range?: string;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
export type PodcastEstimate = {
|
||||||
|
ttsCost: number;
|
||||||
|
avatarCost: number;
|
||||||
|
videoCost: number;
|
||||||
|
researchCost: number;
|
||||||
|
total: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type HostPersona = {
|
||||||
|
name: string;
|
||||||
|
background: string;
|
||||||
|
expertise_level: string;
|
||||||
|
personality_traits: string[];
|
||||||
|
vocal_style: string;
|
||||||
|
catchphrases: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type AudienceDNA = {
|
||||||
|
expertise_level: string;
|
||||||
|
interests: string[];
|
||||||
|
pain_points: string[];
|
||||||
|
demographics?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type BrandDNA = {
|
||||||
|
industry: string;
|
||||||
|
tone: string;
|
||||||
|
communication_style: string;
|
||||||
|
key_messages: string[];
|
||||||
|
competitor_context?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type PodcastBible = {
|
||||||
|
project_id?: string;
|
||||||
|
host: HostPersona;
|
||||||
|
audience: AudienceDNA;
|
||||||
|
brand: BrandDNA;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type CreateProjectPayload = {
|
||||||
|
ideaOrUrl: string;
|
||||||
|
speakers: number;
|
||||||
|
duration: number;
|
||||||
|
knobs: Knobs;
|
||||||
|
budgetCap: number;
|
||||||
|
files: { voiceFile?: File | null; avatarFile?: File | null };
|
||||||
|
avatarUrl?: string | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type CreateProjectResult = {
|
||||||
|
projectId: string;
|
||||||
|
analysis: PodcastAnalysis;
|
||||||
|
estimate: PodcastEstimate;
|
||||||
|
queries: Query[];
|
||||||
|
bible?: PodcastBible;
|
||||||
|
avatar_url?: string | null;
|
||||||
|
avatar_prompt?: string | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type RenderJobResult = {
|
||||||
|
audioUrl: string;
|
||||||
|
audioFilename: string;
|
||||||
|
provider: string;
|
||||||
|
model: string;
|
||||||
|
cost: number;
|
||||||
|
voiceId: string;
|
||||||
|
fileSize: number;
|
||||||
|
videoUrl?: string;
|
||||||
|
videoFilename?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export interface VideoGenerationSettings {
|
||||||
|
prompt: string;
|
||||||
|
resolution: "480p" | "720p";
|
||||||
|
seed?: number | null;
|
||||||
|
maskImageUrl?: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type TaskStatus = {
|
||||||
|
task_id: string;
|
||||||
|
status: "pending" | "processing" | "completed" | "failed";
|
||||||
|
progress?: number;
|
||||||
|
message?: string;
|
||||||
|
result?: any;
|
||||||
|
error?: string;
|
||||||
|
created_at?: string;
|
||||||
|
updated_at?: string;
|
||||||
|
};
|
||||||
|
|
||||||
425
_session_backup/usePodcastWorkflow.ts
Normal file
425
_session_backup/usePodcastWorkflow.ts
Normal file
@@ -0,0 +1,425 @@
|
|||||||
|
import { useState, useEffect, useMemo, useCallback } from "react";
|
||||||
|
import { podcastApi } from "../../../services/podcastApi";
|
||||||
|
import { usePreflightCheck } from "../../../hooks/usePreflightCheck";
|
||||||
|
import { useBudgetTracking } from "../../../hooks/useBudgetTracking";
|
||||||
|
import { CreateProjectPayload, Script } from "../types";
|
||||||
|
import { usePodcastProjectState } from "../../../hooks/usePodcastProjectState";
|
||||||
|
import { sanitizeExaConfig, announceError, getStepLabel } from "./utils";
|
||||||
|
|
||||||
|
type PodcastProjectStateReturn = ReturnType<typeof usePodcastProjectState>;
|
||||||
|
|
||||||
|
interface UsePodcastWorkflowProps {
|
||||||
|
projectState: PodcastProjectStateReturn;
|
||||||
|
onError: (message: string) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const usePodcastWorkflow = ({ projectState, onError }: UsePodcastWorkflowProps) => {
|
||||||
|
const {
|
||||||
|
project,
|
||||||
|
analysis,
|
||||||
|
queries,
|
||||||
|
selectedQueries,
|
||||||
|
research,
|
||||||
|
rawResearch,
|
||||||
|
researchProvider,
|
||||||
|
showScriptEditor,
|
||||||
|
showRenderQueue,
|
||||||
|
currentStep,
|
||||||
|
renderJobs,
|
||||||
|
budgetCap,
|
||||||
|
setProject,
|
||||||
|
setAnalysis,
|
||||||
|
setQueries,
|
||||||
|
setSelectedQueries,
|
||||||
|
setResearch,
|
||||||
|
setRawResearch,
|
||||||
|
setEstimate,
|
||||||
|
setScriptData,
|
||||||
|
setShowScriptEditor,
|
||||||
|
setShowRenderQueue,
|
||||||
|
setKnobs,
|
||||||
|
setResearchProvider,
|
||||||
|
setBudgetCap,
|
||||||
|
updateRenderJob,
|
||||||
|
initializeProject,
|
||||||
|
setBible,
|
||||||
|
} = projectState;
|
||||||
|
|
||||||
|
const [isAnalyzing, setIsAnalyzing] = useState(false);
|
||||||
|
const [isResearching, setIsResearching] = useState(false);
|
||||||
|
const [announcement, setAnnouncement] = useState("");
|
||||||
|
const [showResumeAlert, setShowResumeAlert] = useState(false);
|
||||||
|
const [showPreflightDialog, setShowPreflightDialog] = useState(false);
|
||||||
|
const [preflightResponse, setPreflightResponse] = useState<any>(null);
|
||||||
|
const [preflightOperationName, setPreflightOperationName] = useState<string>("");
|
||||||
|
|
||||||
|
const budgetTracking = useBudgetTracking(budgetCap || 50);
|
||||||
|
const preflightCheck = usePreflightCheck({
|
||||||
|
onBlocked: (response) => {
|
||||||
|
setPreflightResponse(response);
|
||||||
|
setShowPreflightDialog(true);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
// Update budget cap when project state changes
|
||||||
|
useEffect(() => {
|
||||||
|
if (budgetCap) {
|
||||||
|
budgetTracking.setBudgetCap(budgetCap);
|
||||||
|
}
|
||||||
|
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||||
|
}, [budgetCap]);
|
||||||
|
|
||||||
|
// Check if we have a saved project on mount
|
||||||
|
useEffect(() => {
|
||||||
|
if (project && currentStep && currentStep !== "create") {
|
||||||
|
setShowResumeAlert(true);
|
||||||
|
setTimeout(() => setShowResumeAlert(false), 5000);
|
||||||
|
}
|
||||||
|
}, [project, currentStep]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (announcement) {
|
||||||
|
const t = setTimeout(() => setAnnouncement(""), 4000);
|
||||||
|
return () => clearTimeout(t);
|
||||||
|
}
|
||||||
|
return undefined;
|
||||||
|
}, [announcement]);
|
||||||
|
|
||||||
|
const handleCreate = useCallback(async (payload: CreateProjectPayload, feedback?: string) => {
|
||||||
|
if (isAnalyzing) return;
|
||||||
|
setResearch(null);
|
||||||
|
setRawResearch(null);
|
||||||
|
setScriptData(null);
|
||||||
|
setShowScriptEditor(false);
|
||||||
|
setShowRenderQueue(false);
|
||||||
|
try {
|
||||||
|
setIsAnalyzing(true);
|
||||||
|
|
||||||
|
// Use existing avatar URL if provided (e.g. brand avatar), or upload new file
|
||||||
|
let avatarUrl: string | null = payload.avatarUrl || null;
|
||||||
|
if (payload.files.avatarFile) {
|
||||||
|
try {
|
||||||
|
setAnnouncement("Uploading presenter avatar...");
|
||||||
|
const uploadResponse = await podcastApi.uploadAvatar(payload.files.avatarFile);
|
||||||
|
avatarUrl = uploadResponse.avatar_url;
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Avatar upload failed:', error);
|
||||||
|
// Continue without avatar - will generate one later
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NEW FLOW: Create project first to generate/get the Podcast Bible
|
||||||
|
// This allows the analysis to be personalized using the Bible context
|
||||||
|
const projectId = project?.id || `podcast_${Date.now()}_${Math.floor(Math.random() * 1000)}`;
|
||||||
|
setAnnouncement("Initializing project and brand context...");
|
||||||
|
const dbProject = project ? null : await initializeProject(payload, projectId, avatarUrl);
|
||||||
|
const bible = dbProject?.bible || projectState.bible;
|
||||||
|
|
||||||
|
setAnnouncement(feedback ? "Regenerating analysis using your feedback..." : "Analyzing your idea — AI suggestions incoming");
|
||||||
|
const result = await podcastApi.createProject(payload, bible, feedback);
|
||||||
|
|
||||||
|
if (result.bible) {
|
||||||
|
setBible(result.bible);
|
||||||
|
} else if (dbProject?.bible) {
|
||||||
|
setBible(dbProject.bible);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update the project in database with the analysis results
|
||||||
|
try {
|
||||||
|
await podcastApi.updateProject(projectId, {
|
||||||
|
analysis: result.analysis,
|
||||||
|
estimate: result.estimate,
|
||||||
|
queries: result.queries,
|
||||||
|
selected_queries: result.queries.map(q => q.id),
|
||||||
|
avatar_url: result.avatar_url,
|
||||||
|
avatar_prompt: result.avatar_prompt,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Failed to update project with analysis results:', error);
|
||||||
|
}
|
||||||
|
|
||||||
|
setProject({
|
||||||
|
id: projectId,
|
||||||
|
idea: payload.ideaOrUrl,
|
||||||
|
duration: payload.duration,
|
||||||
|
speakers: payload.speakers,
|
||||||
|
avatarUrl: result.avatar_url || avatarUrl,
|
||||||
|
avatarPrompt: result.avatar_prompt || null,
|
||||||
|
avatarPersonaId: null,
|
||||||
|
});
|
||||||
|
|
||||||
|
setAnalysis(result.analysis);
|
||||||
|
setEstimate(result.estimate);
|
||||||
|
setQueries(result.queries);
|
||||||
|
setSelectedQueries(new Set(result.queries.map((q) => q.id)));
|
||||||
|
setKnobs(payload.knobs);
|
||||||
|
setBudgetCap(payload.budgetCap);
|
||||||
|
|
||||||
|
// Generate presenters AFTER analysis completes (to use analysis insights)
|
||||||
|
// This happens only if no avatar was uploaded
|
||||||
|
if (!avatarUrl && payload.speakers > 0 && result.analysis) {
|
||||||
|
try {
|
||||||
|
setAnnouncement("Generating presenter avatars using AI insights...");
|
||||||
|
const presentersResponse = await podcastApi.generatePresenters(
|
||||||
|
payload.speakers,
|
||||||
|
result.projectId,
|
||||||
|
result.analysis.audience,
|
||||||
|
result.analysis.contentType,
|
||||||
|
result.analysis.topKeywords
|
||||||
|
);
|
||||||
|
if (presentersResponse.avatars && presentersResponse.avatars.length > 0) {
|
||||||
|
// Store the first presenter avatar URL and prompt
|
||||||
|
const firstAvatar = presentersResponse.avatars[0];
|
||||||
|
const prompt = firstAvatar.prompt || null;
|
||||||
|
setProject({
|
||||||
|
id: result.projectId,
|
||||||
|
idea: payload.ideaOrUrl,
|
||||||
|
duration: payload.duration,
|
||||||
|
speakers: payload.speakers,
|
||||||
|
avatarUrl: firstAvatar.avatar_url,
|
||||||
|
avatarPrompt: prompt,
|
||||||
|
avatarPersonaId: firstAvatar.persona_id || presentersResponse.persona_id || null,
|
||||||
|
});
|
||||||
|
setAnnouncement("Analysis complete - Presenter avatars generated");
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Presenter generation failed:', error);
|
||||||
|
setAnnouncement("Analysis complete - Avatar generation will happen later");
|
||||||
|
// Continue without presenters - can generate later
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
setAnnouncement("Analysis complete");
|
||||||
|
}
|
||||||
|
} catch (error: any) {
|
||||||
|
if (error?.response?.status === 429 || error?.response?.data?.detail) {
|
||||||
|
const errorDetail = error.response.data.detail;
|
||||||
|
if (typeof errorDetail === 'object' && errorDetail.error && errorDetail.error.includes('limit')) {
|
||||||
|
const usageInfo = errorDetail.usage_info || {};
|
||||||
|
const blockedResponse = {
|
||||||
|
can_proceed: false,
|
||||||
|
estimated_cost: 0,
|
||||||
|
operations: [{
|
||||||
|
provider: errorDetail.provider || 'huggingface',
|
||||||
|
operation_type: 'ai_text_generation',
|
||||||
|
cost: 0,
|
||||||
|
allowed: false,
|
||||||
|
limit_info: usageInfo.limit_info || null,
|
||||||
|
message: errorDetail.message || errorDetail.error || 'Subscription limit exceeded',
|
||||||
|
}],
|
||||||
|
total_cost: 0,
|
||||||
|
usage_summary: usageInfo.usage_summary || null,
|
||||||
|
cached: false,
|
||||||
|
};
|
||||||
|
setPreflightResponse(blockedResponse);
|
||||||
|
setPreflightOperationName('Podcast Analysis');
|
||||||
|
setShowPreflightDialog(true);
|
||||||
|
setAnnouncement("Subscription limit reached. Please upgrade to continue.");
|
||||||
|
} else {
|
||||||
|
const message = typeof errorDetail === 'string' ? errorDetail : errorDetail.message || errorDetail.error || 'Request limit exceeded';
|
||||||
|
announceError(setAnnouncement, new Error(message));
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
announceError(setAnnouncement, error);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
setIsAnalyzing(false);
|
||||||
|
}
|
||||||
|
}, [isAnalyzing, setResearch, setRawResearch, setScriptData, setShowScriptEditor, setShowRenderQueue, initializeProject, setProject, setAnalysis, setEstimate, setQueries, setSelectedQueries, setKnobs, setBudgetCap, setBible]);
|
||||||
|
|
||||||
|
const handleRunResearch = useCallback(async () => {
|
||||||
|
if (isResearching) return;
|
||||||
|
if (!project) {
|
||||||
|
setAnnouncement("Create a project first.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (selectedQueries.size === 0) {
|
||||||
|
setAnnouncement("Select at least one query to research.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
setPreflightOperationName("Research");
|
||||||
|
const approvedQueries = queries.filter((q) => selectedQueries.has(q.id));
|
||||||
|
const preflightResult = await preflightCheck.check({
|
||||||
|
provider: researchProvider === "exa" ? "exa" : "gemini",
|
||||||
|
operation_type: researchProvider === "exa" ? "exa_neural_search" : "google_grounding",
|
||||||
|
tokens_requested: researchProvider === "exa" ? 0 : 1200,
|
||||||
|
actual_provider_name: researchProvider || "exa",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!preflightResult.can_proceed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
setIsResearching(true);
|
||||||
|
setAnnouncement(`Starting ${researchProvider === "exa" ? "deep" : "standard"} research — this may take a moment...`);
|
||||||
|
setResearch(null);
|
||||||
|
setRawResearch(null);
|
||||||
|
setScriptData(null);
|
||||||
|
setShowScriptEditor(false);
|
||||||
|
setShowRenderQueue(false);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { research: mapped, raw } = await podcastApi.runResearch({
|
||||||
|
projectId: project.id,
|
||||||
|
topic: project.idea,
|
||||||
|
approvedQueries,
|
||||||
|
provider: researchProvider,
|
||||||
|
exaConfig: sanitizeExaConfig(analysis?.exaSuggestedConfig),
|
||||||
|
bible: projectState.bible,
|
||||||
|
analysis: analysis,
|
||||||
|
onProgress: (message) => {
|
||||||
|
setAnnouncement(message);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
setResearch(mapped);
|
||||||
|
setRawResearch(raw);
|
||||||
|
setAnnouncement("Research complete — review fact cards below");
|
||||||
|
} catch (researchError) {
|
||||||
|
const errorMessage = researchError instanceof Error
|
||||||
|
? researchError.message
|
||||||
|
: "Research failed. Please try again or switch to Standard Research.";
|
||||||
|
|
||||||
|
if (errorMessage.includes("Exa") || errorMessage.includes("exa")) {
|
||||||
|
setAnnouncement(`Deep research failed: ${errorMessage}. Try Standard Research instead.`);
|
||||||
|
} else if (errorMessage.includes("timeout")) {
|
||||||
|
setAnnouncement("Research timed out. Please try again with fewer queries.");
|
||||||
|
} else {
|
||||||
|
setAnnouncement(`Research failed: ${errorMessage}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.error("Research error:", researchError);
|
||||||
|
throw researchError;
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
announceError(setAnnouncement, error);
|
||||||
|
} finally {
|
||||||
|
setIsResearching(false);
|
||||||
|
}
|
||||||
|
}, [isResearching, project, selectedQueries, queries, researchProvider, preflightCheck, analysis, setResearch, setRawResearch, setScriptData, setShowScriptEditor, setShowRenderQueue, projectState.bible]);
|
||||||
|
|
||||||
|
const handleGenerateScript = useCallback(async () => {
|
||||||
|
if (showScriptEditor) return;
|
||||||
|
if (!project || !research) {
|
||||||
|
setAnnouncement("Project or research missing — cannot generate script");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
setPreflightOperationName("Script Generation");
|
||||||
|
const preflightResult = await preflightCheck.check({
|
||||||
|
provider: "gemini",
|
||||||
|
operation_type: "script_generation",
|
||||||
|
tokens_requested: 2000,
|
||||||
|
actual_provider_name: "gemini",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!preflightResult.can_proceed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
setScriptData(null);
|
||||||
|
setShowRenderQueue(false);
|
||||||
|
setShowScriptEditor(true);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await podcastApi.generateScript({
|
||||||
|
projectId: project.id,
|
||||||
|
idea: project.idea,
|
||||||
|
research: rawResearch,
|
||||||
|
knobs: projectState.knobs,
|
||||||
|
speakers: project.speakers,
|
||||||
|
durationMinutes: project.duration,
|
||||||
|
bible: projectState.bible,
|
||||||
|
outline: analysis?.suggestedOutlines?.[0], // Pass the first (possibly refined) outline
|
||||||
|
analysis: analysis, // Pass full analysis context
|
||||||
|
});
|
||||||
|
|
||||||
|
setScriptData(result);
|
||||||
|
} catch (error) {
|
||||||
|
announceError(setAnnouncement, error);
|
||||||
|
}
|
||||||
|
}, [showScriptEditor, project, research, preflightCheck, setScriptData, setShowRenderQueue, setShowScriptEditor, rawResearch, projectState.knobs, projectState.bible])
|
||||||
|
|
||||||
|
const handleProceedToRendering = useCallback((script: Script) => {
|
||||||
|
setScriptData(script);
|
||||||
|
if (renderJobs.length === 0) {
|
||||||
|
script.scenes.forEach((scene) => {
|
||||||
|
const hasExistingAudio = Boolean(scene.audioUrl);
|
||||||
|
updateRenderJob(scene.id, {
|
||||||
|
sceneId: scene.id,
|
||||||
|
title: scene.title,
|
||||||
|
status: hasExistingAudio ? ("completed" as const) : ("idle" as const),
|
||||||
|
progress: hasExistingAudio ? 100 : 0,
|
||||||
|
previewUrl: null,
|
||||||
|
finalUrl: hasExistingAudio ? scene.audioUrl : null,
|
||||||
|
jobId: null,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
setShowRenderQueue(true);
|
||||||
|
setShowScriptEditor(false);
|
||||||
|
}, [renderJobs.length, setScriptData, updateRenderJob, setShowRenderQueue, setShowScriptEditor]);
|
||||||
|
|
||||||
|
const toggleQuery = useCallback((id: string) => {
|
||||||
|
if (isResearching) return;
|
||||||
|
const current = selectedQueries;
|
||||||
|
const next = new Set<string>(current);
|
||||||
|
if (next.has(id)) next.delete(id);
|
||||||
|
else next.add(id);
|
||||||
|
setSelectedQueries(next);
|
||||||
|
}, [isResearching, selectedQueries, setSelectedQueries]);
|
||||||
|
|
||||||
|
const activeStep = useMemo(() => {
|
||||||
|
if (showRenderQueue) return 3;
|
||||||
|
if (showScriptEditor) return 2;
|
||||||
|
if (currentStep === 'research' || research) return 1;
|
||||||
|
if (currentStep === 'analysis' || analysis) return 0;
|
||||||
|
return -1;
|
||||||
|
}, [showRenderQueue, showScriptEditor, currentStep, research, analysis]);
|
||||||
|
|
||||||
|
const canGenerateScript = Boolean(project && research && rawResearch);
|
||||||
|
|
||||||
|
const handleRegenerate = useCallback(async (feedback?: string) => {
|
||||||
|
if (!project) return;
|
||||||
|
|
||||||
|
// Prepare the payload from existing project state
|
||||||
|
const payload: CreateProjectPayload = {
|
||||||
|
ideaOrUrl: project.idea,
|
||||||
|
duration: project.duration,
|
||||||
|
speakers: project.speakers,
|
||||||
|
knobs: projectState.knobs,
|
||||||
|
budgetCap: projectState.budgetCap,
|
||||||
|
avatarUrl: project.avatarUrl,
|
||||||
|
files: {} // No new files for regeneration
|
||||||
|
};
|
||||||
|
|
||||||
|
await handleCreate(payload, feedback);
|
||||||
|
}, [project, projectState.knobs, projectState.budgetCap, handleCreate]);
|
||||||
|
|
||||||
|
return {
|
||||||
|
// State
|
||||||
|
isAnalyzing,
|
||||||
|
isResearching,
|
||||||
|
announcement,
|
||||||
|
showResumeAlert,
|
||||||
|
showPreflightDialog,
|
||||||
|
preflightResponse,
|
||||||
|
preflightOperationName,
|
||||||
|
activeStep,
|
||||||
|
canGenerateScript,
|
||||||
|
// Handlers
|
||||||
|
handleCreate,
|
||||||
|
handleRegenerate,
|
||||||
|
handleRunResearch,
|
||||||
|
handleGenerateScript,
|
||||||
|
handleProceedToRendering,
|
||||||
|
toggleQuery,
|
||||||
|
setAnnouncement,
|
||||||
|
setShowResumeAlert,
|
||||||
|
setShowPreflightDialog,
|
||||||
|
setPreflightResponse,
|
||||||
|
setResearchProvider,
|
||||||
|
getStepLabel,
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
184
add_missing_columns.py
Normal file
184
add_missing_columns.py
Normal file
@@ -0,0 +1,184 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Migration script to add missing columns to usage_summaries table.
|
||||||
|
Run this once to fix the database schema.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python add_missing_columns.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sqlite3
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
def get_db_path():
|
||||||
|
"""Find the database path."""
|
||||||
|
possible_paths = [
|
||||||
|
Path(__file__).parent / "backend" / "alwrity.db",
|
||||||
|
Path(__file__).parent.parent / "backend" / "alwrity.db",
|
||||||
|
Path("C:/Users/diksha rawat/Desktop/ALwrity_github/windsurf/ALwrity/backend/alwrity.db"),
|
||||||
|
]
|
||||||
|
|
||||||
|
for db_path in possible_paths:
|
||||||
|
if db_path.exists():
|
||||||
|
print(f"Using database: {db_path}")
|
||||||
|
return db_path
|
||||||
|
|
||||||
|
backend_dir = Path(__file__).parent / "backend"
|
||||||
|
if backend_dir.exists():
|
||||||
|
db_files = list(backend_dir.glob("*.db"))
|
||||||
|
if db_files:
|
||||||
|
print(f"Found database: {db_files[0]}")
|
||||||
|
return db_files[0]
|
||||||
|
|
||||||
|
raise FileNotFoundError(f"Database not found. Searched: {possible_paths}")
|
||||||
|
|
||||||
|
def create_usage_summaries_table(cursor):
|
||||||
|
"""Create the usage_summaries table if it doesn't exist."""
|
||||||
|
cursor.execute("""
|
||||||
|
CREATE TABLE IF NOT EXISTS usage_summaries (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
user_id VARCHAR(100) NOT NULL,
|
||||||
|
billing_period VARCHAR(20) NOT NULL,
|
||||||
|
|
||||||
|
-- API Call Counts
|
||||||
|
gemini_calls INTEGER DEFAULT 0,
|
||||||
|
openai_calls INTEGER DEFAULT 0,
|
||||||
|
anthropic_calls INTEGER DEFAULT 0,
|
||||||
|
mistral_calls INTEGER DEFAULT 0,
|
||||||
|
wavespeed_calls INTEGER DEFAULT 0,
|
||||||
|
tavily_calls INTEGER DEFAULT 0,
|
||||||
|
serper_calls INTEGER DEFAULT 0,
|
||||||
|
metaphor_calls INTEGER DEFAULT 0,
|
||||||
|
firecrawl_calls INTEGER DEFAULT 0,
|
||||||
|
stability_calls INTEGER DEFAULT 0,
|
||||||
|
exa_calls INTEGER DEFAULT 0,
|
||||||
|
video_calls INTEGER DEFAULT 0,
|
||||||
|
image_edit_calls INTEGER DEFAULT 0,
|
||||||
|
audio_calls INTEGER DEFAULT 0,
|
||||||
|
|
||||||
|
-- Token Usage
|
||||||
|
gemini_tokens INTEGER DEFAULT 0,
|
||||||
|
openai_tokens INTEGER DEFAULT 0,
|
||||||
|
anthropic_tokens INTEGER DEFAULT 0,
|
||||||
|
mistral_tokens INTEGER DEFAULT 0,
|
||||||
|
wavespeed_tokens INTEGER DEFAULT 0,
|
||||||
|
|
||||||
|
-- Cost Tracking
|
||||||
|
gemini_cost REAL DEFAULT 0.0,
|
||||||
|
openai_cost REAL DEFAULT 0.0,
|
||||||
|
anthropic_cost REAL DEFAULT 0.0,
|
||||||
|
mistral_cost REAL DEFAULT 0.0,
|
||||||
|
wavespeed_cost REAL DEFAULT 0.0,
|
||||||
|
tavily_cost REAL DEFAULT 0.0,
|
||||||
|
serper_cost REAL DEFAULT 0.0,
|
||||||
|
metaphor_cost REAL DEFAULT 0.0,
|
||||||
|
firecrawl_cost REAL DEFAULT 0.0,
|
||||||
|
stability_cost REAL DEFAULT 0.0,
|
||||||
|
exa_cost REAL DEFAULT 0.0,
|
||||||
|
video_cost REAL DEFAULT 0.0,
|
||||||
|
image_edit_cost REAL DEFAULT 0.0,
|
||||||
|
audio_cost REAL DEFAULT 0.0,
|
||||||
|
|
||||||
|
-- Totals
|
||||||
|
total_calls INTEGER DEFAULT 0,
|
||||||
|
total_tokens INTEGER DEFAULT 0,
|
||||||
|
total_cost REAL DEFAULT 0.0,
|
||||||
|
|
||||||
|
-- Performance Metrics
|
||||||
|
avg_response_time REAL DEFAULT 0.0,
|
||||||
|
error_rate REAL DEFAULT 0.0,
|
||||||
|
usage_status VARCHAR(20) DEFAULT 'active',
|
||||||
|
warnings_sent INTEGER DEFAULT 0,
|
||||||
|
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
|
||||||
|
UNIQUE(user_id, billing_period)
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
print("Created usage_summaries table")
|
||||||
|
|
||||||
|
def add_missing_columns():
|
||||||
|
db_path = get_db_path()
|
||||||
|
print(f"Using database: {db_path}")
|
||||||
|
|
||||||
|
conn = sqlite3.connect(db_path)
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
# Check what tables exist
|
||||||
|
cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||||
|
tables = [row[0] for row in cursor.fetchall()]
|
||||||
|
print(f"Tables in database: {tables}")
|
||||||
|
|
||||||
|
# Check if usage_summaries exists
|
||||||
|
if "usage_summaries" not in tables:
|
||||||
|
print("usage_summaries table doesn't exist. Creating it...")
|
||||||
|
create_usage_summaries_table(cursor)
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
print("Done! Table created successfully.")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Get existing columns
|
||||||
|
cursor.execute("PRAGMA table_info(usage_summaries)")
|
||||||
|
existing_columns = {row[1] for row in cursor.fetchall()}
|
||||||
|
print(f"Existing columns in usage_summaries: {len(existing_columns)}")
|
||||||
|
|
||||||
|
# Columns to add (name, type, default)
|
||||||
|
columns_to_add = [
|
||||||
|
# Call counts
|
||||||
|
("wavespeed_calls", "INTEGER", "0"),
|
||||||
|
("tavily_calls", "INTEGER", "0"),
|
||||||
|
("serper_calls", "INTEGER", "0"),
|
||||||
|
("metaphor_calls", "INTEGER", "0"),
|
||||||
|
("firecrawl_calls", "INTEGER", "0"),
|
||||||
|
("stability_calls", "INTEGER", "0"),
|
||||||
|
("exa_calls", "INTEGER", "0"),
|
||||||
|
("video_calls", "INTEGER", "0"),
|
||||||
|
("image_edit_calls", "INTEGER", "0"),
|
||||||
|
("audio_calls", "INTEGER", "0"),
|
||||||
|
# Token usage
|
||||||
|
("wavespeed_tokens", "INTEGER", "0"),
|
||||||
|
# Cost tracking
|
||||||
|
("wavespeed_cost", "REAL", "0.0"),
|
||||||
|
("tavily_cost", "REAL", "0.0"),
|
||||||
|
("serper_cost", "REAL", "0.0"),
|
||||||
|
("metaphor_cost", "REAL", "0.0"),
|
||||||
|
("firecrawl_cost", "REAL", "0.0"),
|
||||||
|
("stability_cost", "REAL", "0.0"),
|
||||||
|
("exa_cost", "REAL", "0.0"),
|
||||||
|
("video_cost", "REAL", "0.0"),
|
||||||
|
("image_edit_cost", "REAL", "0.0"),
|
||||||
|
("audio_cost", "REAL", "0.0"),
|
||||||
|
]
|
||||||
|
|
||||||
|
added = []
|
||||||
|
skipped = []
|
||||||
|
|
||||||
|
for col_name, col_type, default in columns_to_add:
|
||||||
|
if col_name in existing_columns:
|
||||||
|
skipped.append(col_name)
|
||||||
|
continue
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"ALTER TABLE usage_summaries ADD COLUMN {col_name} {col_type} DEFAULT {default}"
|
||||||
|
cursor.execute(sql)
|
||||||
|
added.append(col_name)
|
||||||
|
print(f" Added: {col_name}")
|
||||||
|
except sqlite3.Error as e:
|
||||||
|
print(f" Error adding {col_name}: {e}")
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
print(f"\nSummary:")
|
||||||
|
print(f" Added: {len(added)} columns")
|
||||||
|
print(f" Skipped (already exist): {len(skipped)} columns")
|
||||||
|
|
||||||
|
if added:
|
||||||
|
print(f"\nColumns added: {', '.join(added)}")
|
||||||
|
if skipped:
|
||||||
|
print(f"Already existed: {', '.join(skipped)}")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
add_missing_columns()
|
||||||
2
backend/Procfile
Normal file
2
backend/Procfile
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
# Use start_alwrity_backend.py for deployment
|
||||||
|
web: python start_alwrity_backend.py --production
|
||||||
@@ -3,6 +3,11 @@ ALwrity Utilities Package
|
|||||||
Modular utilities for ALwrity backend startup and configuration.
|
Modular utilities for ALwrity backend startup and configuration.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
# Check podcast mode early to skip heavy imports
|
||||||
|
_is_podcast = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||||
|
|
||||||
from .dependency_manager import DependencyManager
|
from .dependency_manager import DependencyManager
|
||||||
from .environment_setup import EnvironmentSetup
|
from .environment_setup import EnvironmentSetup
|
||||||
from .database_setup import DatabaseSetup
|
from .database_setup import DatabaseSetup
|
||||||
@@ -11,16 +16,51 @@ from .health_checker import HealthChecker
|
|||||||
from .rate_limiter import RateLimiter
|
from .rate_limiter import RateLimiter
|
||||||
from .frontend_serving import FrontendServing
|
from .frontend_serving import FrontendServing
|
||||||
from .router_manager import RouterManager
|
from .router_manager import RouterManager
|
||||||
from .onboarding_manager import OnboardingManager
|
from .feature_runtime import (
|
||||||
|
get_active_profiles,
|
||||||
|
get_enabled_groups,
|
||||||
|
get_enabled_optional_services,
|
||||||
|
get_enabled_routers,
|
||||||
|
get_enabled_startup_hooks,
|
||||||
|
is_enabled,
|
||||||
|
)
|
||||||
|
|
||||||
__all__ = [
|
# Lazy load OnboardingManager - it triggers heavy imports (aiohttp, etc.)
|
||||||
'DependencyManager',
|
if not _is_podcast:
|
||||||
'EnvironmentSetup',
|
from .onboarding_manager import OnboardingManager
|
||||||
'DatabaseSetup',
|
__all__ = [
|
||||||
'ProductionOptimizer',
|
'DependencyManager',
|
||||||
'HealthChecker',
|
'EnvironmentSetup',
|
||||||
'RateLimiter',
|
'DatabaseSetup',
|
||||||
'FrontendServing',
|
'ProductionOptimizer',
|
||||||
'RouterManager',
|
'HealthChecker',
|
||||||
'OnboardingManager'
|
'RateLimiter',
|
||||||
]
|
'FrontendServing',
|
||||||
|
'RouterManager',
|
||||||
|
'OnboardingManager',
|
||||||
|
'get_active_profiles',
|
||||||
|
'get_enabled_groups',
|
||||||
|
'get_enabled_optional_services',
|
||||||
|
'get_enabled_routers',
|
||||||
|
'get_enabled_startup_hooks',
|
||||||
|
'is_enabled'
|
||||||
|
]
|
||||||
|
else:
|
||||||
|
OnboardingManager = None
|
||||||
|
__all__ = [
|
||||||
|
'DependencyManager',
|
||||||
|
'EnvironmentSetup',
|
||||||
|
'DatabaseSetup',
|
||||||
|
'ProductionOptimizer',
|
||||||
|
'HealthChecker',
|
||||||
|
'RateLimiter',
|
||||||
|
'FrontendServing',
|
||||||
|
'RouterManager',
|
||||||
|
'OnboardingManager',
|
||||||
|
'get_active_profiles',
|
||||||
|
'get_enabled_groups',
|
||||||
|
'get_enabled_optional_services',
|
||||||
|
'get_enabled_routers',
|
||||||
|
'get_enabled_startup_hooks',
|
||||||
|
'is_enabled'
|
||||||
|
]
|
||||||
|
|||||||
@@ -55,22 +55,28 @@ class EnvironmentSetup:
|
|||||||
print("🔧 Setting up environment variables...")
|
print("🔧 Setting up environment variables...")
|
||||||
|
|
||||||
# Production environment variables
|
# Production environment variables
|
||||||
|
# IMPORTANT: Don't override PORT if already set by Render cloud
|
||||||
|
render_port = os.getenv("PORT")
|
||||||
|
|
||||||
if self.production_mode:
|
if self.production_mode:
|
||||||
env_vars = {
|
env_vars = {
|
||||||
"HOST": "0.0.0.0",
|
"HOST": "0.0.0.0",
|
||||||
"PORT": "8000",
|
|
||||||
"RELOAD": "false",
|
"RELOAD": "false",
|
||||||
"LOG_LEVEL": "INFO",
|
"LOG_LEVEL": "INFO",
|
||||||
"DEBUG": "false"
|
"DEBUG": "false"
|
||||||
}
|
}
|
||||||
|
# Only set PORT if not already provided by cloud (Render sets PORT)
|
||||||
|
if not render_port:
|
||||||
|
env_vars["PORT"] = "8000"
|
||||||
else:
|
else:
|
||||||
env_vars = {
|
env_vars = {
|
||||||
"HOST": "0.0.0.0",
|
"HOST": "0.0.0.0",
|
||||||
"PORT": "8000",
|
|
||||||
"RELOAD": "true",
|
"RELOAD": "true",
|
||||||
"LOG_LEVEL": "DEBUG",
|
"LOG_LEVEL": "DEBUG",
|
||||||
"DEBUG": "true"
|
"DEBUG": "true"
|
||||||
}
|
}
|
||||||
|
if not render_port:
|
||||||
|
env_vars["PORT"] = "8000"
|
||||||
|
|
||||||
for key, value in env_vars.items():
|
for key, value in env_vars.items():
|
||||||
os.environ.setdefault(key, value)
|
os.environ.setdefault(key, value)
|
||||||
|
|||||||
86
backend/alwrity_utils/feature_profiles.py
Normal file
86
backend/alwrity_utils/feature_profiles.py
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
"""Feature profile parsing and expansion logic."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Iterable, Tuple
|
||||||
|
|
||||||
|
from .feature_registry import FEATURE_GROUPS, PROFILE_GROUP_MAP
|
||||||
|
|
||||||
|
|
||||||
|
ENV_ENABLED_FEATURES = "ALWRITY_ENABLED_FEATURES"
|
||||||
|
DEFAULT_FEATURES = "all"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExpandedFeatureProfile:
|
||||||
|
"""Expanded profile data used by runtime helpers."""
|
||||||
|
|
||||||
|
profiles: Tuple[str, ...]
|
||||||
|
groups: Tuple[str, ...]
|
||||||
|
|
||||||
|
|
||||||
|
class UnknownFeatureProfileError(ValueError):
|
||||||
|
"""Raised when ALWRITY_ENABLED_FEATURES contains unknown feature values."""
|
||||||
|
|
||||||
|
|
||||||
|
def _get_env_value() -> str:
|
||||||
|
"""Get the enabled features value from environment."""
|
||||||
|
return os.getenv(ENV_ENABLED_FEATURES) or DEFAULT_FEATURES
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_values(raw_value: str | None) -> Tuple[str, ...]:
|
||||||
|
if not raw_value or not raw_value.strip():
|
||||||
|
return (DEFAULT_FEATURES,)
|
||||||
|
|
||||||
|
normalized = tuple(
|
||||||
|
value.strip().lower()
|
||||||
|
for value in raw_value.split(",")
|
||||||
|
if value.strip()
|
||||||
|
)
|
||||||
|
return normalized or (DEFAULT_FEATURES,)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_feature_profiles(raw_value: str | None = None) -> Tuple[str, ...]:
|
||||||
|
"""Parse and validate feature names from env/raw input.
|
||||||
|
|
||||||
|
Supports comma-separated feature names, e.g. `podcast,core`.
|
||||||
|
Raises UnknownFeatureProfileError when any feature is not registered.
|
||||||
|
"""
|
||||||
|
|
||||||
|
selected_profiles = _normalize_values(raw_value if raw_value is not None else _get_env_value())
|
||||||
|
|
||||||
|
unknown = sorted({profile for profile in selected_profiles if profile not in PROFILE_GROUP_MAP and profile not in FEATURE_GROUPS})
|
||||||
|
if unknown:
|
||||||
|
supported = ", ".join(sorted(set(PROFILE_GROUP_MAP.keys()) | set(FEATURE_GROUPS.keys())))
|
||||||
|
unknown_display = ", ".join(unknown)
|
||||||
|
raise UnknownFeatureProfileError(
|
||||||
|
f"Unknown {ENV_ENABLED_FEATURES} value(s): {unknown_display}. Supported: {supported}."
|
||||||
|
)
|
||||||
|
|
||||||
|
return selected_profiles
|
||||||
|
|
||||||
|
|
||||||
|
def _dedupe_stable(items: Iterable[str]) -> Tuple[str, ...]:
|
||||||
|
return tuple(dict.fromkeys(items))
|
||||||
|
|
||||||
|
|
||||||
|
def expand_profiles(profiles: Tuple[str, ...]) -> ExpandedFeatureProfile:
|
||||||
|
"""Expand profile names into a deduplicated group list."""
|
||||||
|
|
||||||
|
# Handle "all" specially - include all groups
|
||||||
|
if "all" in profiles:
|
||||||
|
return ExpandedFeatureProfile(profiles=("all",), groups=tuple(FEATURE_GROUPS.keys()))
|
||||||
|
|
||||||
|
# Otherwise expand via PROFILE_GROUP_MAP
|
||||||
|
groups = _dedupe_stable(
|
||||||
|
group
|
||||||
|
for profile in profiles
|
||||||
|
for group in PROFILE_GROUP_MAP.get(profile, (profile,))
|
||||||
|
)
|
||||||
|
|
||||||
|
# Include FEATURE_GROUPS keys directly
|
||||||
|
all_groups = _dedupe_stable(list(groups) + [g for g in groups if g in FEATURE_GROUPS])
|
||||||
|
|
||||||
|
return ExpandedFeatureProfile(profiles=profiles, groups=all_groups)
|
||||||
63
backend/alwrity_utils/feature_registry.py
Normal file
63
backend/alwrity_utils/feature_registry.py
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
"""Feature registry for profile-based capability toggles.
|
||||||
|
|
||||||
|
This module stores normalized feature-group definitions used by the
|
||||||
|
feature profile runtime.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Dict, Tuple
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class FeatureGroup:
|
||||||
|
"""Single feature group and the capabilities it enables."""
|
||||||
|
|
||||||
|
routers: Tuple[str, ...] = ()
|
||||||
|
startup_hooks: Tuple[str, ...] = ()
|
||||||
|
optional_services: Tuple[str, ...] = ()
|
||||||
|
features: Tuple[str, ...] = field(default_factory=tuple)
|
||||||
|
|
||||||
|
|
||||||
|
FEATURE_GROUPS: Dict[str, FeatureGroup] = {
|
||||||
|
"core": FeatureGroup(
|
||||||
|
features=("core", "health", "onboarding", "research"),
|
||||||
|
routers=(
|
||||||
|
"api.component_logic:router",
|
||||||
|
"api.subscription:router",
|
||||||
|
"api.onboarding_utils.step3_routes:router",
|
||||||
|
"api.research.router:router",
|
||||||
|
),
|
||||||
|
startup_hooks=(
|
||||||
|
"services.database:init_database",
|
||||||
|
),
|
||||||
|
optional_services=(
|
||||||
|
"services.scheduler:get_scheduler",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
"podcast": FeatureGroup(
|
||||||
|
features=("podcast",),
|
||||||
|
routers=("api.podcast.router:router",),
|
||||||
|
),
|
||||||
|
"youtube": FeatureGroup(
|
||||||
|
features=("youtube",),
|
||||||
|
routers=("api.youtube.router:router",),
|
||||||
|
),
|
||||||
|
"content_planning": FeatureGroup(
|
||||||
|
features=("content_planning", "strategy_copilot"),
|
||||||
|
routers=(
|
||||||
|
"api.content_planning.api.router:router",
|
||||||
|
"api.content_planning.strategy_copilot:router",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
PROFILE_GROUP_MAP: Dict[str, Tuple[str, ...]] = {
|
||||||
|
"all": tuple(FEATURE_GROUPS.keys()),
|
||||||
|
"core": ("core",),
|
||||||
|
"podcast": ("core", "podcast"),
|
||||||
|
"youtube": ("core", "youtube"),
|
||||||
|
"planning": ("core", "content_planning"),
|
||||||
|
}
|
||||||
71
backend/alwrity_utils/feature_runtime.py
Normal file
71
backend/alwrity_utils/feature_runtime.py
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
"""Runtime helpers for profile-driven feature toggles."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from functools import lru_cache
|
||||||
|
from typing import Tuple
|
||||||
|
|
||||||
|
from .feature_profiles import expand_profiles, parse_feature_profiles
|
||||||
|
from .feature_registry import FEATURE_GROUPS
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=1)
|
||||||
|
def _runtime_state() -> dict[str, Tuple[str, ...]]:
|
||||||
|
profiles = parse_feature_profiles()
|
||||||
|
expanded = expand_profiles(profiles)
|
||||||
|
|
||||||
|
routers = []
|
||||||
|
startup_hooks = []
|
||||||
|
optional_services = []
|
||||||
|
enabled_features = set(expanded.groups)
|
||||||
|
|
||||||
|
for group in expanded.groups:
|
||||||
|
feature_group = FEATURE_GROUPS[group]
|
||||||
|
routers.extend(feature_group.routers)
|
||||||
|
startup_hooks.extend(feature_group.startup_hooks)
|
||||||
|
optional_services.extend(feature_group.optional_services)
|
||||||
|
enabled_features.update(feature_group.features)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"profiles": expanded.profiles,
|
||||||
|
"groups": expanded.groups,
|
||||||
|
"routers": tuple(dict.fromkeys(routers)),
|
||||||
|
"startup_hooks": tuple(dict.fromkeys(startup_hooks)),
|
||||||
|
"optional_services": tuple(dict.fromkeys(optional_services)),
|
||||||
|
"features": tuple(sorted(enabled_features)),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_active_profiles() -> Tuple[str, ...]:
|
||||||
|
"""Return validated active profile names."""
|
||||||
|
return _runtime_state()["profiles"]
|
||||||
|
|
||||||
|
|
||||||
|
def get_enabled_groups() -> Tuple[str, ...]:
|
||||||
|
"""Return resolved feature-group names."""
|
||||||
|
return _runtime_state()["groups"]
|
||||||
|
|
||||||
|
|
||||||
|
def get_enabled_routers() -> Tuple[str, ...]:
|
||||||
|
"""Return enabled router import targets in `module:attribute` format."""
|
||||||
|
return _runtime_state()["routers"]
|
||||||
|
|
||||||
|
|
||||||
|
def get_enabled_startup_hooks() -> Tuple[str, ...]:
|
||||||
|
"""Return enabled startup hook import targets in `module:attribute` format."""
|
||||||
|
return _runtime_state()["startup_hooks"]
|
||||||
|
|
||||||
|
|
||||||
|
def get_enabled_optional_services() -> Tuple[str, ...]:
|
||||||
|
"""Return enabled optional service import targets in `module:attribute` format."""
|
||||||
|
return _runtime_state()["optional_services"]
|
||||||
|
|
||||||
|
|
||||||
|
def is_enabled(feature: str) -> bool:
|
||||||
|
"""Return True when a feature/group name is enabled by active profiles."""
|
||||||
|
return feature.strip().lower() in _runtime_state()["features"]
|
||||||
|
|
||||||
|
|
||||||
|
def reset_feature_runtime_cache() -> None:
|
||||||
|
"""Clear runtime cache (useful for tests)."""
|
||||||
|
_runtime_state.cache_clear()
|
||||||
@@ -39,9 +39,10 @@ class ProductionOptimizer:
|
|||||||
def _set_production_env_vars(self) -> None:
|
def _set_production_env_vars(self) -> None:
|
||||||
"""Set production-specific environment variables."""
|
"""Set production-specific environment variables."""
|
||||||
production_vars = {
|
production_vars = {
|
||||||
|
# Note: PORT is NOT set here - it's provided by the deployment platform (e.g., Render)
|
||||||
|
# Don't override PORT as it must come from the environment
|
||||||
# Note: HOST is not set here - it's auto-detected by start_backend()
|
# Note: HOST is not set here - it's auto-detected by start_backend()
|
||||||
# Based on deployment environment (cloud vs local)
|
# Based on deployment environment (cloud vs local)
|
||||||
'PORT': '8000',
|
|
||||||
'RELOAD': 'false',
|
'RELOAD': 'false',
|
||||||
'LOG_LEVEL': 'INFO',
|
'LOG_LEVEL': 'INFO',
|
||||||
'DEBUG': 'false',
|
'DEBUG': 'false',
|
||||||
|
|||||||
@@ -3,10 +3,73 @@ Router Manager Module
|
|||||||
Handles FastAPI router inclusion and management.
|
Handles FastAPI router inclusion and management.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
from importlib import import_module
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
from fastapi import FastAPI
|
from fastapi import FastAPI
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
from typing import List, Dict, Any, Optional
|
|
||||||
import os
|
|
||||||
|
CORE_ROUTER_REGISTRY = [
|
||||||
|
{"name": "component_logic", "module": "api.component_logic", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "subscription", "module": "api.subscription", "attr": "router", "features": {"all", "core", "podcast", "blog-writer", "youtube"}},
|
||||||
|
{"name": "step3_research", "module": "api.onboarding_utils.step3_routes", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "step4_assets", "module": "api.onboarding_utils.step4_asset_routes", "attr": "router", "features": {"all", "core", "podcast"}},
|
||||||
|
{"name": "step4_persona", "module": "api.onboarding_utils.step4_persona_routes_optimized", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "gsc_auth", "module": "routers.gsc_auth", "attr": "router", "features": {"all", "core", "seo"}},
|
||||||
|
{"name": "wordpress_oauth", "module": "routers.wordpress_oauth", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "bing_oauth", "module": "routers.bing_oauth", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "bing_analytics", "module": "routers.bing_analytics", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "bing_analytics_storage", "module": "routers.bing_analytics_storage", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "seo_tools", "module": "routers.seo_tools", "attr": "router", "features": {"all", "core", "seo"}},
|
||||||
|
{"name": "facebook_writer", "module": "api.facebook_writer.routers", "attr": "facebook_router", "features": {"all", "core", "facebook"}},
|
||||||
|
{"name": "linkedin", "module": "routers.linkedin", "attr": "router", "features": {"all", "core", "linkedin"}},
|
||||||
|
{"name": "linkedin_image", "module": "api.linkedin_image_generation", "attr": "router", "features": {"all", "core", "linkedin"}},
|
||||||
|
{"name": "brainstorm", "module": "api.brainstorm", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "hallucination_detector", "module": "api.hallucination_detector", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "writing_assistant", "module": "api.writing_assistant", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "content_planning", "module": "api.content_planning.api.router", "attr": "router", "features": {"all", "core", "content-planning"}},
|
||||||
|
{"name": "user_data", "module": "api.user_data", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "user_environment", "module": "api.user_environment", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "strategy_copilot", "module": "api.content_planning.strategy_copilot", "attr": "router", "features": {"all", "core", "content-planning"}},
|
||||||
|
{"name": "error_logging", "module": "routers.error_logging", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "frontend_env_manager", "module": "routers.frontend_env_manager", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "platform_analytics", "module": "routers.platform_analytics", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "bing_insights", "module": "routers.bing_insights", "attr": "router", "features": {"all", "core", "seo"}},
|
||||||
|
{"name": "background_jobs", "module": "routers.background_jobs", "attr": "router", "features": {"all", "core"}},
|
||||||
|
]
|
||||||
|
|
||||||
|
OPTIONAL_ROUTER_REGISTRY = [
|
||||||
|
{"name": "blog_writer", "module": "api.blog_writer.router", "attr": "router", "features": {"all", "blog-writer"}},
|
||||||
|
{"name": "story_writer", "module": "api.story_writer.router", "attr": "router", "features": {"all", "story-writer"}},
|
||||||
|
{"name": "wix", "module": "api.wix_routes", "attr": "router", "features": {"all"}},
|
||||||
|
{"name": "blog_seo_analysis", "module": "api.blog_writer.seo_analysis", "attr": "router", "features": {"all", "blog-writer"}},
|
||||||
|
{"name": "persona", "module": "api.persona_routes", "attr": "router", "features": {"all", "persona"}},
|
||||||
|
{"name": "video_studio", "module": "api.video_studio.router", "attr": "router", "features": {"all", "video-studio"}},
|
||||||
|
{"name": "stability", "module": "routers.stability", "attr": "router", "features": {"all", "image-studio"}},
|
||||||
|
{"name": "stability_advanced", "module": "routers.stability_advanced", "attr": "router", "features": {"all", "image-studio"}},
|
||||||
|
{"name": "stability_admin", "module": "routers.stability_admin", "attr": "router", "features": {"all", "image-studio"}},
|
||||||
|
{"name": "images", "module": "api.images", "attr": "router", "features": {"all", "image-studio"}},
|
||||||
|
{"name": "image_studio", "module": "routers.image_studio", "attr": "router", "features": {"all", "image-studio"}},
|
||||||
|
{"name": "product_marketing", "module": "routers.product_marketing", "attr": "router", "features": {"all", "product-marketing"}},
|
||||||
|
{"name": "campaign_creator", "module": "routers.campaign_creator", "attr": "router", "features": {"all"}},
|
||||||
|
{"name": "content_assets", "module": "api.content_assets.router", "attr": "router", "features": {"all"}},
|
||||||
|
{"name": "podcast", "module": "api.podcast.router", "attr": "router", "features": {"all", "podcast"}},
|
||||||
|
{"name": "youtube", "module": "api.youtube.router", "attr": "router", "features": {"all", "youtube"}, "include_kwargs": {"prefix": "/api"}},
|
||||||
|
{"name": "research_config", "module": "api.research_config", "attr": "router", "features": {"all", "research"}, "include_kwargs": {"prefix": "/api/research", "tags": ["research"]}},
|
||||||
|
{"name": "research_engine", "module": "api.research.router", "attr": "router", "features": {"all", "research"}, "include_kwargs": {"tags": ["Research Engine"]}},
|
||||||
|
{"name": "scheduler_dashboard", "module": "api.scheduler_dashboard", "attr": "router", "features": {"all", "scheduler"}},
|
||||||
|
{"name": "oauth_token_monitoring", "module": "api.oauth_token_monitoring_routes", "attr": "router", "features": {"all", "core"}},
|
||||||
|
{"name": "agents", "module": "api.agents_api", "attr": "router", "features": {"all"}},
|
||||||
|
{"name": "today_workflow", "module": "api.today_workflow", "attr": "router", "features": {"all"}},
|
||||||
|
]
|
||||||
|
|
||||||
|
OPTIONAL_MODULE_MATRIX = {
|
||||||
|
"all": [entry["name"] for entry in OPTIONAL_ROUTER_REGISTRY],
|
||||||
|
"default": [entry["name"] for entry in OPTIONAL_ROUTER_REGISTRY],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
class RouterManager:
|
class RouterManager:
|
||||||
@@ -16,14 +79,61 @@ class RouterManager:
|
|||||||
self.app = app
|
self.app = app
|
||||||
self.included_routers = []
|
self.included_routers = []
|
||||||
self.failed_routers = []
|
self.failed_routers = []
|
||||||
|
self.skipped_routers = []
|
||||||
|
|
||||||
def include_router_safely(self, router, router_name: str = None) -> bool:
|
@staticmethod
|
||||||
|
def get_enabled_features() -> set:
|
||||||
|
"""Get enabled features from ALWRITY_ENABLED_FEATURES env var.
|
||||||
|
|
||||||
|
Values:
|
||||||
|
- "all" - enable all features (default)
|
||||||
|
- comma-separated: "podcast,blog-writer,youtube"
|
||||||
|
- single feature: "podcast"
|
||||||
|
"""
|
||||||
|
env_value = os.getenv("ALWRITY_ENABLED_FEATURES", "all").strip().lower()
|
||||||
|
|
||||||
|
if not env_value or env_value == "all":
|
||||||
|
return {"all"}
|
||||||
|
|
||||||
|
return {f.strip() for f in env_value.split(",") if f.strip()}
|
||||||
|
|
||||||
|
def _is_verbose(self) -> bool:
|
||||||
|
return os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||||
|
|
||||||
|
def _get_profile(self) -> str:
|
||||||
|
"""Legacy method - returns primary profile."""
|
||||||
|
enabled = self.get_enabled_features()
|
||||||
|
if "all" in enabled:
|
||||||
|
return "all"
|
||||||
|
# Return first feature as profile for backwards compatibility
|
||||||
|
return list(enabled)[0] if enabled else "all"
|
||||||
|
|
||||||
|
def _should_include_router(self, registry_entry: Dict[str, Any], enabled_features: set) -> bool:
|
||||||
|
"""Check if router should be included based on enabled features."""
|
||||||
|
required_features = registry_entry.get("features", set())
|
||||||
|
|
||||||
|
# If "all" is enabled, include everything
|
||||||
|
if "all" in enabled_features:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# If no required features specified, include by default
|
||||||
|
if not required_features:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# Check if any required feature is enabled
|
||||||
|
return bool(required_features & enabled_features)
|
||||||
|
|
||||||
|
def _load_router_from_registry(self, registry_entry: Dict[str, Any]):
|
||||||
|
module = import_module(registry_entry["module"])
|
||||||
|
return getattr(module, registry_entry["attr"])
|
||||||
|
|
||||||
|
def include_router_safely(self, router, router_name: Optional[str] = None, include_kwargs: Optional[Dict[str, Any]] = None) -> bool:
|
||||||
"""Include a router safely with error handling."""
|
"""Include a router safely with error handling."""
|
||||||
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
verbose = self._is_verbose()
|
||||||
|
router_name = router_name or getattr(router, 'prefix', 'unknown')
|
||||||
|
|
||||||
try:
|
try:
|
||||||
self.app.include_router(router)
|
self.app.include_router(router, **(include_kwargs or {}))
|
||||||
router_name = router_name or getattr(router, 'prefix', 'unknown')
|
|
||||||
self.included_routers.append(router_name)
|
self.included_routers.append(router_name)
|
||||||
if verbose:
|
if verbose:
|
||||||
logger.info(f"✅ Router included successfully: {router_name}")
|
logger.info(f"✅ Router included successfully: {router_name}")
|
||||||
@@ -35,210 +145,85 @@ class RouterManager:
|
|||||||
logger.warning(f"❌ Router inclusion failed: {router_name} - {e}")
|
logger.warning(f"❌ Router inclusion failed: {router_name} - {e}")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def include_core_routers(self) -> bool:
|
@staticmethod
|
||||||
"""Include core application routers."""
|
def _demo_release_mode_enabled() -> bool:
|
||||||
# Import os locally to avoid UnboundLocalError if it's shadowed
|
"""Return True when demo-release safety mode is enabled."""
|
||||||
import os
|
return os.getenv("ALWRITY_DEMO_RELEASE", "false").lower() in {"1", "true", "yes", "on"}
|
||||||
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
|
||||||
|
def _include_registry_group(self, registry: List[Dict[str, Any]], group_name: str) -> bool:
|
||||||
|
verbose = self._is_verbose()
|
||||||
|
enabled_features = self.get_enabled_features()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
if verbose:
|
if verbose:
|
||||||
logger.info("Including core routers...")
|
logger.info(f"Including {group_name} routers with features: {enabled_features}...")
|
||||||
|
|
||||||
# Component logic router
|
|
||||||
from api.component_logic import router as component_logic_router
|
|
||||||
self.include_router_safely(component_logic_router, "component_logic")
|
|
||||||
|
|
||||||
# Subscription router
|
for entry in registry:
|
||||||
from api.subscription import router as subscription_router
|
if not self._should_include_router(entry, enabled_features):
|
||||||
self.include_router_safely(subscription_router, "subscription")
|
reason = f"features {enabled_features} not matching {entry.get('features', set())}"
|
||||||
|
self.skipped_routers.append({"name": entry["name"], "reason": reason})
|
||||||
|
if verbose:
|
||||||
|
logger.info(f"⏭️ Skipping {entry['name']}: {reason}")
|
||||||
|
continue
|
||||||
|
|
||||||
|
try:
|
||||||
|
router = self._load_router_from_registry(entry)
|
||||||
|
self.include_router_safely(router, entry["name"], entry.get("include_kwargs"))
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"{entry['name']} router not mounted: {e}")
|
||||||
|
|
||||||
# Step 3 Research router (core onboarding functionality)
|
logger.info(f"✅ {group_name.capitalize()} routers processed for features: {enabled_features}")
|
||||||
from api.onboarding_utils.step3_routes import router as step3_research_router
|
|
||||||
self.include_router_safely(step3_research_router, "step3_research")
|
|
||||||
|
|
||||||
# Step 4 Persona and Asset routers
|
|
||||||
from api.onboarding_utils.step4_asset_routes import router as step4_asset_router
|
|
||||||
self.include_router_safely(step4_asset_router, "step4_assets")
|
|
||||||
|
|
||||||
from api.onboarding_utils.step4_persona_routes_optimized import router as step4_persona_router
|
|
||||||
self.include_router_safely(step4_persona_router, "step4_persona")
|
|
||||||
|
|
||||||
# GSC router
|
|
||||||
from routers.gsc_auth import router as gsc_auth_router
|
|
||||||
self.include_router_safely(gsc_auth_router, "gsc_auth")
|
|
||||||
|
|
||||||
# WordPress router
|
|
||||||
from routers.wordpress_oauth import router as wordpress_oauth_router
|
|
||||||
self.include_router_safely(wordpress_oauth_router, "wordpress_oauth")
|
|
||||||
|
|
||||||
# Bing Webmaster router
|
|
||||||
from routers.bing_oauth import router as bing_oauth_router
|
|
||||||
self.include_router_safely(bing_oauth_router, "bing_oauth")
|
|
||||||
|
|
||||||
# Bing Analytics router
|
|
||||||
from routers.bing_analytics import router as bing_analytics_router
|
|
||||||
self.include_router_safely(bing_analytics_router, "bing_analytics")
|
|
||||||
|
|
||||||
# Bing Analytics Storage router
|
|
||||||
from routers.bing_analytics_storage import router as bing_analytics_storage_router
|
|
||||||
self.include_router_safely(bing_analytics_storage_router, "bing_analytics_storage")
|
|
||||||
|
|
||||||
# SEO tools router
|
|
||||||
from routers.seo_tools import router as seo_tools_router
|
|
||||||
self.include_router_safely(seo_tools_router, "seo_tools")
|
|
||||||
|
|
||||||
# Facebook Writer router
|
|
||||||
from api.facebook_writer.routers import facebook_router
|
|
||||||
self.include_router_safely(facebook_router, "facebook_writer")
|
|
||||||
|
|
||||||
# LinkedIn routers
|
|
||||||
from routers.linkedin import router as linkedin_router
|
|
||||||
self.include_router_safely(linkedin_router, "linkedin")
|
|
||||||
|
|
||||||
from api.linkedin_image_generation import router as linkedin_image_router
|
|
||||||
self.include_router_safely(linkedin_image_router, "linkedin_image")
|
|
||||||
|
|
||||||
# Brainstorm router
|
|
||||||
from api.brainstorm import router as brainstorm_router
|
|
||||||
self.include_router_safely(brainstorm_router, "brainstorm")
|
|
||||||
|
|
||||||
# Hallucination detector and writing assistant
|
|
||||||
from api.hallucination_detector import router as hallucination_detector_router
|
|
||||||
self.include_router_safely(hallucination_detector_router, "hallucination_detector")
|
|
||||||
|
|
||||||
from api.writing_assistant import router as writing_assistant_router
|
|
||||||
self.include_router_safely(writing_assistant_router, "writing_assistant")
|
|
||||||
|
|
||||||
# Content planning and user data
|
|
||||||
from api.content_planning.api.router import router as content_planning_router
|
|
||||||
self.include_router_safely(content_planning_router, "content_planning")
|
|
||||||
|
|
||||||
from api.user_data import router as user_data_router
|
|
||||||
self.include_router_safely(user_data_router, "user_data")
|
|
||||||
|
|
||||||
from api.user_environment import router as user_environment_router
|
|
||||||
self.include_router_safely(user_environment_router, "user_environment")
|
|
||||||
|
|
||||||
# Strategy copilot
|
|
||||||
from api.content_planning.strategy_copilot import router as strategy_copilot_router
|
|
||||||
self.include_router_safely(strategy_copilot_router, "strategy_copilot")
|
|
||||||
|
|
||||||
# Error logging router
|
|
||||||
from routers.error_logging import router as error_logging_router
|
|
||||||
self.include_router_safely(error_logging_router, "error_logging")
|
|
||||||
|
|
||||||
# Frontend environment manager router
|
|
||||||
from routers.frontend_env_manager import router as frontend_env_router
|
|
||||||
self.include_router_safely(frontend_env_router, "frontend_env_manager")
|
|
||||||
|
|
||||||
# Platform analytics router
|
|
||||||
try:
|
|
||||||
from routers.platform_analytics import router as platform_analytics_router
|
|
||||||
self.include_router_safely(platform_analytics_router, "platform_analytics")
|
|
||||||
logger.info("✅ Platform analytics router included successfully")
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"❌ Failed to include platform analytics router: {e}")
|
|
||||||
# Continue with other routers
|
|
||||||
|
|
||||||
# Bing insights router
|
|
||||||
try:
|
|
||||||
from routers.bing_insights import router as bing_insights_router
|
|
||||||
self.include_router_safely(bing_insights_router, "bing_insights")
|
|
||||||
logger.info("✅ Bing insights router included successfully")
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"❌ Failed to include Bing insights router: {e}")
|
|
||||||
# Continue with other routers
|
|
||||||
|
|
||||||
# Background jobs router
|
|
||||||
try:
|
|
||||||
from routers.background_jobs import router as background_jobs_router
|
|
||||||
self.include_router_safely(background_jobs_router, "background_jobs")
|
|
||||||
logger.info("✅ Background jobs router included successfully")
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"❌ Failed to include Background jobs router: {e}")
|
|
||||||
# Continue with other routers
|
|
||||||
|
|
||||||
logger.info("✅ Core routers included successfully")
|
|
||||||
return True
|
return True
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"❌ Error including core routers: {e}")
|
logger.error(f"❌ Error including {group_name} routers: {e}")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
def include_core_routers(self) -> bool:
|
||||||
|
"""Include core application routers."""
|
||||||
|
return self._include_registry_group(CORE_ROUTER_REGISTRY, "core")
|
||||||
|
|
||||||
def include_optional_routers(self) -> bool:
|
def include_optional_routers(self) -> bool:
|
||||||
"""Include optional routers with error handling."""
|
"""Include optional routers with error handling."""
|
||||||
try:
|
return self._include_registry_group(OPTIONAL_ROUTER_REGISTRY, "optional")
|
||||||
logger.info("Including optional routers...")
|
|
||||||
|
|
||||||
# AI Blog Writer router
|
|
||||||
try:
|
|
||||||
from api.blog_writer.router import router as blog_writer_router
|
|
||||||
self.include_router_safely(blog_writer_router, "blog_writer")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"AI Blog Writer router not mounted: {e}")
|
|
||||||
|
|
||||||
# Story Writer router
|
|
||||||
try:
|
|
||||||
from api.story_writer.router import router as story_writer_router
|
|
||||||
self.include_router_safely(story_writer_router, "story_writer")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Story Writer router not mounted: {e}")
|
|
||||||
|
|
||||||
# Wix Integration router
|
|
||||||
try:
|
|
||||||
from api.wix_routes import router as wix_router
|
|
||||||
self.include_router_safely(wix_router, "wix")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Wix Integration router not mounted: {e}")
|
|
||||||
|
|
||||||
# Blog Writer SEO Analysis router
|
|
||||||
try:
|
|
||||||
from api.blog_writer.seo_analysis import router as blog_seo_analysis_router
|
|
||||||
self.include_router_safely(blog_seo_analysis_router, "blog_seo_analysis")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Blog Writer SEO Analysis router not mounted: {e}")
|
|
||||||
|
|
||||||
# Persona router
|
|
||||||
try:
|
|
||||||
from api.persona_routes import router as persona_router
|
|
||||||
self.include_router_safely(persona_router, "persona")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Persona router not mounted: {e}")
|
|
||||||
|
|
||||||
# Video Studio router
|
|
||||||
try:
|
|
||||||
from api.video_studio.router import router as video_studio_router
|
|
||||||
self.include_router_safely(video_studio_router, "video_studio")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Video Studio router not mounted: {e}")
|
|
||||||
|
|
||||||
# Stability AI routers
|
|
||||||
try:
|
|
||||||
from routers.stability import router as stability_router
|
|
||||||
self.include_router_safely(stability_router, "stability")
|
|
||||||
|
|
||||||
from routers.stability_advanced import router as stability_advanced_router
|
|
||||||
self.include_router_safely(stability_advanced_router, "stability_advanced")
|
|
||||||
|
|
||||||
from routers.stability_admin import router as stability_admin_router
|
|
||||||
self.include_router_safely(stability_admin_router, "stability_admin")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Stability AI routers not mounted: {e}")
|
|
||||||
|
|
||||||
|
|
||||||
logger.info("✅ Optional routers processed")
|
|
||||||
return True
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"❌ Error including optional routers: {e}")
|
|
||||||
return False
|
|
||||||
|
|
||||||
def get_router_status(self) -> Dict[str, Any]:
|
def get_router_status(self) -> Dict[str, Any]:
|
||||||
"""Get the status of router inclusion."""
|
"""Get the status of router inclusion."""
|
||||||
return {
|
return {
|
||||||
|
"active_profile": self._get_profile(),
|
||||||
"included_routers": self.included_routers,
|
"included_routers": self.included_routers,
|
||||||
"failed_routers": self.failed_routers,
|
"failed_routers": self.failed_routers,
|
||||||
|
"skipped_routers": self.skipped_routers,
|
||||||
"total_included": len(self.included_routers),
|
"total_included": len(self.included_routers),
|
||||||
"total_failed": len(self.failed_routers)
|
"total_failed": len(self.failed_routers),
|
||||||
|
"total_skipped": len(self.skipped_routers)
|
||||||
|
}
|
||||||
|
|
||||||
|
def log_startup_summary(self) -> None:
|
||||||
|
"""Log startup summary including profile, enabled routers, and skipped items."""
|
||||||
|
profile = self._get_profile()
|
||||||
|
|
||||||
|
logger.info("=" * 60)
|
||||||
|
logger.info("📋 STARTUP SUMMARY")
|
||||||
|
logger.info(f" Active profile: {profile}")
|
||||||
|
logger.info(f" Enabled routers ({len(self.included_routers)}): {', '.join(self.included_routers)}")
|
||||||
|
if self.skipped_routers:
|
||||||
|
logger.info(f" Skipped routers ({len(self.skipped_routers)}):")
|
||||||
|
for s in self.skipped_routers:
|
||||||
|
logger.info(f" - {s['name']}: {s['reason']}")
|
||||||
|
if self.failed_routers:
|
||||||
|
logger.warning(f" Failed routers ({len(self.failed_routers)}):")
|
||||||
|
for f in self.failed_routers:
|
||||||
|
logger.warning(f" - {f['name']}: {f['error']}")
|
||||||
|
logger.info("=" * 60)
|
||||||
|
|
||||||
|
def get_feature_profile_status(self) -> Dict[str, Any]:
|
||||||
|
"""Get feature profile status and enabled modules."""
|
||||||
|
profile = self._get_profile()
|
||||||
|
enabled_modules = OPTIONAL_MODULE_MATRIX.get(profile, OPTIONAL_MODULE_MATRIX.get("all", []))
|
||||||
|
|
||||||
|
return {
|
||||||
|
"active_profile": profile,
|
||||||
|
"enabled_modules": enabled_modules,
|
||||||
|
"available_profiles": list(OPTIONAL_MODULE_MATRIX.keys())
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,50 +5,60 @@ The onboarding endpoints are re-exported from a stable module
|
|||||||
`onboarding.py`.
|
`onboarding.py`.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from .onboarding_endpoints import (
|
import os
|
||||||
health_check,
|
|
||||||
get_onboarding_status,
|
|
||||||
get_onboarding_progress_full,
|
|
||||||
get_step_data,
|
|
||||||
complete_step,
|
|
||||||
skip_step,
|
|
||||||
validate_step_access,
|
|
||||||
get_api_keys,
|
|
||||||
save_api_key,
|
|
||||||
validate_api_keys,
|
|
||||||
start_onboarding,
|
|
||||||
complete_onboarding,
|
|
||||||
reset_onboarding,
|
|
||||||
get_resume_info,
|
|
||||||
get_onboarding_config,
|
|
||||||
generate_writing_personas,
|
|
||||||
generate_writing_personas_async,
|
|
||||||
get_persona_task_status,
|
|
||||||
assess_persona_quality,
|
|
||||||
regenerate_persona,
|
|
||||||
get_persona_generation_options
|
|
||||||
)
|
|
||||||
|
|
||||||
__all__ = [
|
# Check podcast mode early
|
||||||
'health_check',
|
_is_podcast = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||||
'get_onboarding_status',
|
|
||||||
'get_onboarding_progress_full',
|
# In podcast mode, don't import heavy onboarding endpoints
|
||||||
'get_step_data',
|
# They trigger heavy dependencies (exa_py, etc.)
|
||||||
'complete_step',
|
if _is_podcast:
|
||||||
'skip_step',
|
__all__ = []
|
||||||
'validate_step_access',
|
else:
|
||||||
'get_api_keys',
|
from .onboarding_endpoints import (
|
||||||
'save_api_key',
|
health_check,
|
||||||
'validate_api_keys',
|
get_onboarding_status,
|
||||||
'start_onboarding',
|
get_onboarding_progress_full,
|
||||||
'complete_onboarding',
|
get_step_data,
|
||||||
'reset_onboarding',
|
complete_step,
|
||||||
'get_resume_info',
|
skip_step,
|
||||||
'get_onboarding_config',
|
validate_step_access,
|
||||||
'generate_writing_personas',
|
get_api_keys,
|
||||||
'generate_writing_personas_async',
|
save_api_key,
|
||||||
'get_persona_task_status',
|
validate_api_keys,
|
||||||
'assess_persona_quality',
|
start_onboarding,
|
||||||
'regenerate_persona',
|
complete_onboarding,
|
||||||
'get_persona_generation_options'
|
reset_onboarding,
|
||||||
]
|
get_resume_info,
|
||||||
|
get_onboarding_config,
|
||||||
|
generate_writing_personas,
|
||||||
|
generate_writing_personas_async,
|
||||||
|
get_persona_task_status,
|
||||||
|
assess_persona_quality,
|
||||||
|
regenerate_persona,
|
||||||
|
get_persona_generation_options
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
'health_check',
|
||||||
|
'get_onboarding_status',
|
||||||
|
'get_onboarding_progress_full',
|
||||||
|
'get_step_data',
|
||||||
|
'complete_step',
|
||||||
|
'skip_step',
|
||||||
|
'validate_step_access',
|
||||||
|
'get_api_keys',
|
||||||
|
'save_api_key',
|
||||||
|
'validate_api_keys',
|
||||||
|
'start_onboarding',
|
||||||
|
'complete_onboarding',
|
||||||
|
'reset_onboarding',
|
||||||
|
'get_resume_info',
|
||||||
|
'get_onboarding_config',
|
||||||
|
'generate_writing_personas',
|
||||||
|
'generate_writing_personas_async',
|
||||||
|
'get_persona_task_status',
|
||||||
|
'assess_persona_quality',
|
||||||
|
'regenerate_persona',
|
||||||
|
'get_persona_generation_options'
|
||||||
|
]
|
||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import os
|
||||||
"""Facebook Post generation service."""
|
"""Facebook Post generation service."""
|
||||||
|
|
||||||
from typing import Dict, Any
|
from typing import Dict, Any
|
||||||
@@ -24,8 +25,7 @@ class FacebookPostService(FacebookWriterBaseService):
|
|||||||
actual_tone = request.custom_tone if request.post_tone.value == "Custom" else request.post_tone.value
|
actual_tone = request.custom_tone if request.post_tone.value == "Custom" else request.post_tone.value
|
||||||
|
|
||||||
# Get persona data for enhanced content generation
|
# Get persona data for enhanced content generation
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(os.getenv("ALWRITY_FALLBACK_USER_ID", "0"))
|
||||||
user_id = 1
|
|
||||||
persona_data = self._get_persona_data(user_id)
|
persona_data = self._get_persona_data(user_id)
|
||||||
|
|
||||||
# Build the prompt
|
# Build the prompt
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import os
|
||||||
"""Remaining Facebook Writer services - placeholder implementations."""
|
"""Remaining Facebook Writer services - placeholder implementations."""
|
||||||
|
|
||||||
from typing import Dict, Any, List
|
from typing import Dict, Any, List
|
||||||
@@ -16,8 +17,7 @@ class FacebookReelService(FacebookWriterBaseService):
|
|||||||
actual_style = request.custom_style if request.reel_style.value == "Custom" else request.reel_style.value
|
actual_style = request.custom_style if request.reel_style.value == "Custom" else request.reel_style.value
|
||||||
|
|
||||||
# Get persona data for enhanced content generation
|
# Get persona data for enhanced content generation
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(os.getenv("ALWRITY_FALLBACK_USER_ID", "0"))
|
||||||
user_id = 1
|
|
||||||
persona_data = self._get_persona_data(user_id)
|
persona_data = self._get_persona_data(user_id)
|
||||||
|
|
||||||
base_prompt = f"""
|
base_prompt = f"""
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import os
|
||||||
"""Facebook Story generation service."""
|
"""Facebook Story generation service."""
|
||||||
|
|
||||||
from typing import Dict, Any, List
|
from typing import Dict, Any, List
|
||||||
@@ -30,8 +31,7 @@ class FacebookStoryService(FacebookWriterBaseService):
|
|||||||
actual_tone = request.custom_tone if request.story_tone.value == "Custom" else request.story_tone.value
|
actual_tone = request.custom_tone if request.story_tone.value == "Custom" else request.story_tone.value
|
||||||
|
|
||||||
# Get persona data for enhanced content generation
|
# Get persona data for enhanced content generation
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(os.getenv("ALWRITY_FALLBACK_USER_ID", "0"))
|
||||||
user_id = 1
|
|
||||||
persona_data = self._get_persona_data(user_id)
|
persona_data = self._get_persona_data(user_id)
|
||||||
|
|
||||||
# Build the prompt
|
# Build the prompt
|
||||||
|
|||||||
@@ -94,36 +94,36 @@ async def generate_platform_persona_endpoint(
|
|||||||
async def update_persona_endpoint(
|
async def update_persona_endpoint(
|
||||||
persona_id: int,
|
persona_id: int,
|
||||||
update_data: Dict[str, Any],
|
update_data: Dict[str, Any],
|
||||||
user_id: int = Query(..., description="User ID")
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Update an existing persona."""
|
"""Update an existing persona."""
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
return await update_persona(1, persona_id, update_data)
|
return await update_persona(user_id, persona_id, update_data)
|
||||||
|
|
||||||
@router.delete("/{persona_id}")
|
@router.delete("/{persona_id}")
|
||||||
async def delete_persona_endpoint(
|
async def delete_persona_endpoint(
|
||||||
persona_id: int,
|
persona_id: int,
|
||||||
user_id: int = Query(..., description="User ID")
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Delete a persona."""
|
"""Delete a persona."""
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
return await delete_persona(1, persona_id)
|
return await delete_persona(user_id, persona_id)
|
||||||
|
|
||||||
@router.get("/check/readiness")
|
@router.get("/check/readiness")
|
||||||
async def check_persona_readiness_endpoint(
|
async def check_persona_readiness_endpoint(
|
||||||
user_id: int = Query(1, description="User ID")
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Check if user has sufficient data for persona generation."""
|
"""Check if user has sufficient data for persona generation."""
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
return await validate_persona_generation_readiness(1)
|
return await validate_persona_generation_readiness(user_id)
|
||||||
|
|
||||||
@router.get("/preview/generate")
|
@router.get("/preview/generate")
|
||||||
async def generate_preview_endpoint(
|
async def generate_preview_endpoint(
|
||||||
user_id: int = Query(1, description="User ID")
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Generate a preview of the writing persona without saving."""
|
"""Generate a preview of the writing persona without saving."""
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
return await generate_persona_preview(1)
|
return await generate_persona_preview(user_id)
|
||||||
|
|
||||||
@router.get("/platforms/supported")
|
@router.get("/platforms/supported")
|
||||||
async def get_supported_platforms_endpoint():
|
async def get_supported_platforms_endpoint():
|
||||||
@@ -160,12 +160,12 @@ async def optimize_facebook_persona_endpoint(
|
|||||||
|
|
||||||
@router.post("/generate-content")
|
@router.post("/generate-content")
|
||||||
async def generate_content_with_persona_endpoint(
|
async def generate_content_with_persona_endpoint(
|
||||||
request: Dict[str, Any]
|
request: Dict[str, Any],
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Generate content using persona replication engine."""
|
"""Generate content using persona replication engine."""
|
||||||
try:
|
try:
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
user_id = 1
|
|
||||||
platform = request.get("platform")
|
platform = request.get("platform")
|
||||||
content_request = request.get("content_request")
|
content_request = request.get("content_request")
|
||||||
content_type = request.get("content_type", "post")
|
content_type = request.get("content_type", "post")
|
||||||
@@ -189,13 +189,13 @@ async def generate_content_with_persona_endpoint(
|
|||||||
@router.get("/export/{platform}")
|
@router.get("/export/{platform}")
|
||||||
async def export_persona_prompt_endpoint(
|
async def export_persona_prompt_endpoint(
|
||||||
platform: str,
|
platform: str,
|
||||||
user_id: int = Query(1, description="User ID")
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Export hardened persona prompt for external use."""
|
"""Export hardened persona prompt for external use."""
|
||||||
try:
|
try:
|
||||||
engine = PersonaReplicationEngine()
|
engine = PersonaReplicationEngine()
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
export_package = engine.export_persona_for_external_use(1, platform)
|
export_package = engine.export_persona_for_external_use(user_id, platform)
|
||||||
|
|
||||||
if "error" in export_package:
|
if "error" in export_package:
|
||||||
raise HTTPException(status_code=400, detail=export_package["error"])
|
raise HTTPException(status_code=400, detail=export_package["error"])
|
||||||
@@ -207,12 +207,12 @@ async def export_persona_prompt_endpoint(
|
|||||||
|
|
||||||
@router.post("/validate-content")
|
@router.post("/validate-content")
|
||||||
async def validate_content_endpoint(
|
async def validate_content_endpoint(
|
||||||
request: Dict[str, Any]
|
request: Dict[str, Any],
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Validate content against persona constraints."""
|
"""Validate content against persona constraints."""
|
||||||
try:
|
try:
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
user_id = 1
|
|
||||||
platform = request.get("platform")
|
platform = request.get("platform")
|
||||||
content = request.get("content")
|
content = request.get("content")
|
||||||
|
|
||||||
@@ -242,14 +242,14 @@ async def validate_content_endpoint(
|
|||||||
async def update_platform_persona_endpoint(
|
async def update_platform_persona_endpoint(
|
||||||
platform: str,
|
platform: str,
|
||||||
update_data: Dict[str, Any],
|
update_data: Dict[str, Any],
|
||||||
user_id: int = Query(1, description="User ID")
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Update platform-specific persona fields for a user.
|
"""Update platform-specific persona fields for a user.
|
||||||
|
|
||||||
Allows editing persona fields in the UI and saving them to the database.
|
Allows editing persona fields in the UI and saving them to the database.
|
||||||
"""
|
"""
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(current_user.get("id"))
|
||||||
return await update_platform_persona(1, platform, update_data)
|
return await update_platform_persona(user_id, platform, update_data)
|
||||||
|
|
||||||
@router.get("/facebook-persona/check/{user_id}")
|
@router.get("/facebook-persona/check/{user_id}")
|
||||||
async def check_facebook_persona_endpoint(
|
async def check_facebook_persona_endpoint(
|
||||||
|
|||||||
666
backend/api/podcast/broll_temp/README.md
Normal file
666
backend/api/podcast/broll_temp/README.md
Normal file
@@ -0,0 +1,666 @@
|
|||||||
|
# Programmatic B-Roll Composer
|
||||||
|
|
||||||
|
A layered video composition pipeline that assembles AI-generated images, programmatic data charts, Pillow text overlays, and circular-masked avatar videos into a single output MP4. Driven by structured JSON from an LLM, exposed via a FastAPI server.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
1. [Architecture overview](#1-architecture-overview)
|
||||||
|
2. [File structure](#2-file-structure)
|
||||||
|
3. [Installation](#3-installation)
|
||||||
|
4. [Core concepts](#4-core-concepts)
|
||||||
|
- 4.1 [The Insight dataclass](#41-the-insight-dataclass)
|
||||||
|
- 4.2 [The SceneAssets dataclass](#42-the-sceneassets-dataclass)
|
||||||
|
- 4.3 [The layer stack](#43-the-layer-stack)
|
||||||
|
- 4.4 [The JSON bridge](#44-the-json-bridge)
|
||||||
|
5. [Asset generators](#5-asset-generators)
|
||||||
|
- 5.1 [Bar chart — make_bar_chart](#51-bar-chart--make_bar_chart)
|
||||||
|
- 5.2 [Line trend — make_line_trend](#52-line-trend--make_line_trend)
|
||||||
|
- 5.3 [Bullet overlay — make_bullet_overlay](#53-bullet-overlay--make_bullet_overlay)
|
||||||
|
- 5.4 [Insight card — make_insight_card](#54-insight-card--make_insight_card)
|
||||||
|
6. [Video effects](#6-video-effects)
|
||||||
|
- 6.1 [Circular avatar mask — apply_circle_mask](#61-circular-avatar-mask--apply_circle_mask)
|
||||||
|
- 6.2 [Ken Burns zoom — ken_burns](#62-ken-burns-zoom--ken_burns)
|
||||||
|
7. [Scene builders](#7-scene-builders)
|
||||||
|
- 7.1 [Data scene — build_data_scene](#71-data-scene--build_data_scene)
|
||||||
|
- 7.2 [Bullet scene — build_bullet_scene](#72-bullet-scene--build_bullet_scene)
|
||||||
|
- 7.3 [Full avatar scene — build_full_avatar_scene](#73-full-avatar-scene--build_full_avatar_scene)
|
||||||
|
8. [Scene dispatcher — dispatch_scene](#8-scene-dispatcher--dispatch_scene)
|
||||||
|
9. [Crossfade transitions](#9-crossfade-transitions)
|
||||||
|
- 9.1 [How crossfade_concat works](#91-how-crossfade_concat-works)
|
||||||
|
- 9.2 [The set_duration gotcha](#92-the-set_duration-gotcha)
|
||||||
|
10. [Master compositor — compose_video](#10-master-compositor--compose_video)
|
||||||
|
11. [FastAPI server](#11-fastapi-server)
|
||||||
|
- 11.1 [Request models](#111-request-models)
|
||||||
|
- 11.2 [Job lifecycle](#112-job-lifecycle)
|
||||||
|
- 11.3 [API endpoints](#113-api-endpoints)
|
||||||
|
12. [Running the project](#12-running-the-project)
|
||||||
|
- 12.1 [Smoke test (no media files needed)](#121-smoke-test-no-media-files-needed)
|
||||||
|
- 12.2 [Full video composition](#122-full-video-composition)
|
||||||
|
- 12.3 [API server](#123-api-server)
|
||||||
|
13. [Calling the API](#13-calling-the-api)
|
||||||
|
14. [Production notes](#14-production-notes)
|
||||||
|
15. [Extending the pipeline](#15-extending-the-pipeline)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Architecture overview
|
||||||
|
|
||||||
|
The pipeline follows a **Layered Composition** model. Rather than generating video in one pass, it assembles independent visual layers — each produced by the cheapest appropriate tool — into a single timeline using MoviePy as the compositor.
|
||||||
|
|
||||||
|
```
|
||||||
|
LLM JSON output
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
dispatch_scene() ← routes visual_cue → builder function
|
||||||
|
│
|
||||||
|
├─ build_data_scene()
|
||||||
|
│ ├─ ImageClip (background) ← AI-generated image
|
||||||
|
│ ├─ ImageClip (chart PNG) ← Matplotlib, transparent bg
|
||||||
|
│ ├─ ImageClip (insight card) ← Pillow RGBA
|
||||||
|
│ └─ VideoFileClip (avatar) ← circular numpy mask
|
||||||
|
│
|
||||||
|
├─ build_bullet_scene()
|
||||||
|
│ ├─ ImageClip (background)
|
||||||
|
│ ├─ ImageClip (bullet overlay) ← Pillow RGBA
|
||||||
|
│ └─ VideoFileClip (avatar)
|
||||||
|
│
|
||||||
|
└─ build_full_avatar_scene()
|
||||||
|
└─ VideoFileClip (full-screen)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
crossfade_concat() ← dissolve between scenes
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
write_videofile() ← H.264 MP4 via ffmpeg
|
||||||
|
```
|
||||||
|
|
||||||
|
The key design decision: charts and text are **never** rendered by a generative model. Matplotlib produces pixel-perfect data graphics from real numbers; Pillow renders crisp, deterministic text. Only the background and the talking-head avatar come from AI generation, minimising both cost and hallucination risk.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. File structure
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── broll_composer.py # Core library — all composition logic
|
||||||
|
├── api_server.py # FastAPI wrapper — HTTP interface to the pipeline
|
||||||
|
└── requirements.txt # Python dependencies
|
||||||
|
```
|
||||||
|
|
||||||
|
`broll_composer.py` has no FastAPI dependency and can be imported and called directly from scripts, notebooks, or other web frameworks.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# System dependency — must be on PATH
|
||||||
|
apt-get install ffmpeg
|
||||||
|
|
||||||
|
# Python packages
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
**requirements.txt**
|
||||||
|
|
||||||
|
```
|
||||||
|
moviepy==1.0.3
|
||||||
|
Pillow>=10.0
|
||||||
|
matplotlib>=3.8
|
||||||
|
numpy>=1.26
|
||||||
|
fastapi>=0.111
|
||||||
|
uvicorn[standard]>=0.29
|
||||||
|
python-multipart>=0.0.9
|
||||||
|
```
|
||||||
|
|
||||||
|
MoviePy 1.0.3 is pinned because 2.x introduced breaking API changes to `CompositeVideoClip` and the effects interface. The rest can float within the specified lower bounds.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Core concepts
|
||||||
|
|
||||||
|
### 4.1 The Insight dataclass
|
||||||
|
|
||||||
|
Every scene is driven by a single `Insight` object. This is the contract between the LLM and the composition pipeline:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass
|
||||||
|
class Insight:
|
||||||
|
key_insight: str # Headline text rendered on the insight card
|
||||||
|
supporting_stat: str # Sub-headline rendered below the headline
|
||||||
|
visual_cue: str # Selects which scene builder to use (see §8)
|
||||||
|
audio_tone: str # Passed through for downstream TTS / audio selection
|
||||||
|
chart_data: dict # Data payload consumed by chart generators (see §5)
|
||||||
|
duration: float # Scene length in seconds, default 10.0
|
||||||
|
```
|
||||||
|
|
||||||
|
The `audio_tone` field is not used by the video pipeline itself — it is metadata for whatever system generates or selects the voiceover audio track for the scene.
|
||||||
|
|
||||||
|
### 4.2 The SceneAssets dataclass
|
||||||
|
|
||||||
|
`SceneAssets` carries file paths to the media assets for a given scene:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass
|
||||||
|
class SceneAssets:
|
||||||
|
background_img: str # Required — path to JPEG or PNG background
|
||||||
|
chart_img: Optional[str] # Populated by dispatch_scene after chart generation
|
||||||
|
avatar_video: Optional[str] # Optional — path to MP4 avatar clip
|
||||||
|
bullet_img: Optional[str] # Reserved for pre-rendered bullet overlays
|
||||||
|
```
|
||||||
|
|
||||||
|
`chart_img` starts as `None` and is written to by `dispatch_scene` after it generates the Matplotlib PNG, so the scene builders receive a fully-populated `SceneAssets` by the time they run.
|
||||||
|
|
||||||
|
### 4.3 The layer stack
|
||||||
|
|
||||||
|
Every scene is a `CompositeVideoClip` — a MoviePy object that renders multiple clips on a shared canvas by alpha-compositing them bottom-to-top. The layer order is consistent across all scene types:
|
||||||
|
|
||||||
|
| Z-order | Layer | Source | Notes |
|
||||||
|
|---------|-------|--------|-------|
|
||||||
|
| 0 (bottom) | Background | AI image + Ken Burns | Darkened to make overlays legible |
|
||||||
|
| 1 | Chart or bullet overlay | Matplotlib or Pillow PNG | Transparent background; fades in |
|
||||||
|
| 2 | Insight card | Pillow RGBA | Positioned at y=820 (near bottom) |
|
||||||
|
| 3 (top) | Avatar circle | MP4 + numpy mask | Bottom-right corner |
|
||||||
|
|
||||||
|
### 4.4 The JSON bridge
|
||||||
|
|
||||||
|
The LLM is prompted to return a structured JSON object — not prose — so the pipeline can consume it without parsing ambiguity:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"key_insight": "AI tools reduced content cycles by 40%",
|
||||||
|
"supporting_stat": "HubSpot 2026 report — 12% lift in CTR",
|
||||||
|
"visual_cue": "bar_chart_comparison",
|
||||||
|
"audio_tone": "authoritative_and_surprising",
|
||||||
|
"duration": 10.0,
|
||||||
|
"chart_data": {
|
||||||
|
"labels": ["Content Velocity", "CTR", "Engagement", "Cost/Lead"],
|
||||||
|
"before": [30, 22, 18, 60],
|
||||||
|
"after": [72, 34, 41, 38]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`pipeline_from_json()` is the single-call entry point that accepts this JSON string, constructs the dataclasses, runs `dispatch_scene`, and writes the output MP4.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Asset generators
|
||||||
|
|
||||||
|
These functions produce static image files (PNG with alpha transparency) that are loaded as `ImageClip` objects in the scene builders. They are completely independent of MoviePy and can be called and previewed without assembling any video.
|
||||||
|
|
||||||
|
### 5.1 Bar chart — `make_bar_chart`
|
||||||
|
|
||||||
|
```python
|
||||||
|
make_bar_chart(data: dict, out_path: str, title: str = "") -> str
|
||||||
|
```
|
||||||
|
|
||||||
|
Produces a side-by-side "before vs after" bar chart using Matplotlib. The critical detail is the renderer configuration and save parameters:
|
||||||
|
|
||||||
|
```python
|
||||||
|
matplotlib.use("Agg") # Non-interactive backend — no display required
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none") # Transparent axes background
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
```
|
||||||
|
|
||||||
|
Setting both `facecolor="none"` on the figure and `transparent=True` on `savefig` is necessary because they control different things: the figure background and the PNG alpha channel respectively. Without both, a white box appears behind the chart when it is composited over the video background.
|
||||||
|
|
||||||
|
**Expected `data` shape:**
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
"labels": ["Category A", "Category B"], # X-axis labels
|
||||||
|
"before": [30, 22], # Bar heights (left bars)
|
||||||
|
"after": [72, 34] # Bar heights (right bars)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 Line trend — `make_line_trend`
|
||||||
|
|
||||||
|
```python
|
||||||
|
make_line_trend(data: dict, out_path: str, title: str = "") -> str
|
||||||
|
```
|
||||||
|
|
||||||
|
Produces a time-series line chart with a translucent fill under the curve (`alpha=0.12`). Suited for growth trends, adoption curves, and any metric tracked over sequential time periods.
|
||||||
|
|
||||||
|
**Expected `data` shape:**
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
"x": [2021, 2022, 2023, 2024, 2025], # X-axis values (numeric or strings)
|
||||||
|
"y": [10, 18, 30, 45, 72] # Y-axis values
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 Bullet overlay — `make_bullet_overlay`
|
||||||
|
|
||||||
|
```python
|
||||||
|
make_bullet_overlay(lines: list[str], out_path: str,
|
||||||
|
width: int = 900, font_size: int = 32) -> str
|
||||||
|
```
|
||||||
|
|
||||||
|
Renders a list of bullet-point strings onto a semi-transparent dark rounded rectangle using Pillow. The image height is computed dynamically from the number of lines:
|
||||||
|
|
||||||
|
```python
|
||||||
|
img_h = padding * 2 + len(lines) * line_h + 12
|
||||||
|
```
|
||||||
|
|
||||||
|
The fill colour `(10, 10, 10, 185)` gives roughly 73% opacity — dark enough for text legibility over any background, light enough that the background remains visible. The bullet character (`•`) is prepended in Python rather than in the font, so no special Unicode font support is required.
|
||||||
|
|
||||||
|
Font loading tries the DejaVu Sans Bold path common on Debian/Ubuntu systems, falling back to Pillow's built-in bitmap font if the TTF is absent.
|
||||||
|
|
||||||
|
### 5.4 Insight card — `make_insight_card`
|
||||||
|
|
||||||
|
```python
|
||||||
|
make_insight_card(insight: str, stat: str, out_path: str,
|
||||||
|
width: int = 960, height: int = 200) -> str
|
||||||
|
```
|
||||||
|
|
||||||
|
Renders a two-line card: a large bold headline (`font_size=34`) and a smaller supporting stat line (`font_size=20`). A solid red rectangle (`#E63946`) is drawn as a left-edge accent bar — a visual device borrowed from print editorial design that gives the card a distinct identity when overlaid on varied backgrounds.
|
||||||
|
|
||||||
|
The card uses `fill=(10, 10, 10, 200)` — approximately 78% opacity — slightly more opaque than the bullet overlay because the headline text is denser.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Video effects
|
||||||
|
|
||||||
|
### 6.1 Circular avatar mask — `apply_circle_mask`
|
||||||
|
|
||||||
|
```python
|
||||||
|
apply_circle_mask(clip: VideoFileClip, diameter: int) -> VideoFileClip
|
||||||
|
```
|
||||||
|
|
||||||
|
Takes an MP4 avatar clip and returns it with a circular alpha mask applied, so only the circle region is visible when the clip is composited over other layers.
|
||||||
|
|
||||||
|
The mask is built using NumPy's `ogrid`, which creates coordinate arrays without materialising a full mesh:
|
||||||
|
|
||||||
|
```python
|
||||||
|
Y, X = np.ogrid[:h, :w]
|
||||||
|
cx, cy = w / 2, h / 2
|
||||||
|
mask_arr = ((X - cx)**2 + (Y - cy)**2 <= (min(w, h) / 2)**2).astype(float)
|
||||||
|
```
|
||||||
|
|
||||||
|
This produces a 2D float array (values 0.0 or 1.0) where all pixels within the inscribed circle are 1 (opaque) and all pixels outside are 0 (transparent). MoviePy requires mask arrays in this float format — it does not accept uint8 or boolean arrays directly.
|
||||||
|
|
||||||
|
The mask array is wrapped in an `ImageClip` with `ismask=True` and the duration is set to match the source clip before calling `clip.set_mask()`.
|
||||||
|
|
||||||
|
**Why not use imagemagick or a pre-made circular PNG?** The numpy approach has no subprocess dependency, works for any input resolution, and the mask is computed once and reused for every frame without disk I/O.
|
||||||
|
|
||||||
|
### 6.2 Ken Burns zoom — `ken_burns`
|
||||||
|
|
||||||
|
```python
|
||||||
|
ken_burns(clip: ImageClip, zoom_ratio: float = 0.08) -> ImageClip
|
||||||
|
```
|
||||||
|
|
||||||
|
Applies a slow continuous zoom-in to a static image clip, creating the illusion of camera movement. This prevents the background from looking visually "dead" during the scene.
|
||||||
|
|
||||||
|
The implementation uses `clip.fl()`, MoviePy's frame-level transform function, which receives both `get_frame` (a callable that returns the frame array at time `t`) and the current time `t`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def zoom_frame(get_frame, t):
|
||||||
|
frame = get_frame(t)
|
||||||
|
frac = 1 + zoom_ratio * (t / clip.duration) # grows from 1.0 to 1+zoom_ratio
|
||||||
|
h, w = frame.shape[:2]
|
||||||
|
new_h, new_w = int(h / frac), int(w / frac) # shrink crop window
|
||||||
|
y1 = (h - new_h) // 2 # center the crop
|
||||||
|
x1 = (w - new_w) // 2
|
||||||
|
cropped = frame[y1:y1 + new_h, x1:x1 + new_w]
|
||||||
|
return np.array(Image.fromarray(cropped).resize((w, h), Image.LANCZOS))
|
||||||
|
```
|
||||||
|
|
||||||
|
At `t=0`, `frac=1.0` so the crop is the full frame. At `t=duration`, `frac=1+zoom_ratio` so the crop is slightly smaller, and upscaling it back to full resolution creates the zoom effect. `zoom_ratio=0.08` means an 8% zoom over the full duration — perceptible but not distracting.
|
||||||
|
|
||||||
|
`apply_to=["mask"]` passes the same transform to the mask channel if one is present, keeping the mask geometrically in sync with the image.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Scene builders
|
||||||
|
|
||||||
|
Scene builders assemble the layers for a given `visual_cue` type into a `CompositeVideoClip`. Each builder follows the same pattern: build layers bottom-to-top, append to a list, return `CompositeVideoClip(layers, size=bg.size).set_duration(d)`.
|
||||||
|
|
||||||
|
The explicit `.set_duration(d)` on the return value is mandatory — see [§9.2](#92-the-set_duration-gotcha) for why.
|
||||||
|
|
||||||
|
### 7.1 Data scene — `build_data_scene`
|
||||||
|
|
||||||
|
Used for `visual_cue` values `bar_chart_comparison` and `line_trend`. The most information-dense layout:
|
||||||
|
|
||||||
|
- **Background**: full-canvas `ImageClip`, Ken Burns zoom at 8%, brightness reduced by 40 units via `vfx.lum_contrast(0, -40)`.
|
||||||
|
- **Chart**: resized to 700px wide, centred horizontally, positioned 180px from the top. Fades in over 0.6s starting at `t=0.5` and fades out over 0.4s at the end.
|
||||||
|
- **Insight card**: centred horizontally at y=820 (approximately the lower fifth of a 1080p frame). Fades in over 0.5s.
|
||||||
|
- **Avatar**: circular-masked at 240px diameter, positioned 40px from the bottom-right corner (`bg.w - 280, bg.h - 280`).
|
||||||
|
|
||||||
|
### 7.2 Bullet scene — `build_bullet_scene`
|
||||||
|
|
||||||
|
Used for `visual_cue` value `bullet_points`. A simpler layout suited to lists of supporting facts:
|
||||||
|
|
||||||
|
- **Background**: Ken Burns at 5% zoom (slower than the data scene — more contemplative pacing), brightness reduced by 50 units.
|
||||||
|
- **Bullet overlay**: rendered by `make_bullet_overlay`, centred both horizontally and vertically, fades in over 0.7s.
|
||||||
|
- **Avatar**: circular-masked at 200px diameter (slightly smaller than in the data scene), positioned 40px from the bottom-right corner.
|
||||||
|
|
||||||
|
If `bullet_lines` is not provided by the caller, the builder falls back to using `insight.key_insight` and `insight.supporting_stat` as two bullet points.
|
||||||
|
|
||||||
|
### 7.3 Full avatar scene — `build_full_avatar_scene`
|
||||||
|
|
||||||
|
Used for `visual_cue` value `full_avatar`. The "Hook" scene — designed to open a piece with a direct-to-camera delivery that grabs attention before the data arrives. No overlays; the avatar fills the entire frame:
|
||||||
|
|
||||||
|
```python
|
||||||
|
avatar = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||||
|
return avatar.resize(height=1080).set_duration(d)
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the only scene type that does not use a `CompositeVideoClip` — it returns a `VideoFileClip` directly. The explicit `.set_duration(d)` is still applied (see §9.2).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Scene dispatcher — `dispatch_scene`
|
||||||
|
|
||||||
|
```python
|
||||||
|
dispatch_scene(insight: Insight, assets: SceneAssets,
|
||||||
|
bullet_lines: Optional[list[str]] = None) -> CompositeVideoClip
|
||||||
|
```
|
||||||
|
|
||||||
|
The dispatcher is the JSON bridge's execution layer. It reads `insight.visual_cue` and routes to the correct builder, generating any intermediate assets (charts) along the way:
|
||||||
|
|
||||||
|
```
|
||||||
|
visual_cue value Action
|
||||||
|
─────────────────────────────────────────────────────
|
||||||
|
"full_avatar" → build_full_avatar_scene()
|
||||||
|
"bar_chart_comparison" → make_bar_chart() → build_data_scene()
|
||||||
|
"line_trend" → make_line_trend() → build_data_scene()
|
||||||
|
"bullet_points" → build_bullet_scene()
|
||||||
|
<anything else> → build_data_scene() with no chart (fallback)
|
||||||
|
```
|
||||||
|
|
||||||
|
Chart PNGs are written to `/tmp/chart.png`. This is intentionally a fixed path — each call overwrites the previous chart, which is fine because `dispatch_scene` is called sequentially per scene. If scenes are ever parallelised, use a `job_id`-prefixed temp path instead.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Crossfade transitions
|
||||||
|
|
||||||
|
### 9.1 How `crossfade_concat` works
|
||||||
|
|
||||||
|
```python
|
||||||
|
def crossfade_concat(scenes: list, fade_dur: float = 0.5) -> CompositeVideoClip:
|
||||||
|
faded = []
|
||||||
|
for i, clip in enumerate(scenes):
|
||||||
|
c = clip
|
||||||
|
if i > 0:
|
||||||
|
c = c.fx(vfx.crossfadein, fade_dur)
|
||||||
|
faded.append(c)
|
||||||
|
return concatenate_videoclips(faded, padding=-fade_dur, method="compose")
|
||||||
|
```
|
||||||
|
|
||||||
|
`vfx.crossfadein` makes a clip's opacity ramp from 0 to 1 over `fade_dur` seconds from its start point. This handles the incoming side of the dissolve.
|
||||||
|
|
||||||
|
`padding=-fade_dur` is the critical parameter. By default, `concatenate_videoclips` places each clip immediately after the previous one ends. A negative padding shifts each clip left by `fade_dur` seconds, so it starts while the previous clip is still playing. The overlap window is exactly `fade_dur` seconds, which matches the duration of the `crossfadein` effect — this is what produces a dissolve rather than a hard cut or a gap.
|
||||||
|
|
||||||
|
`method="compose"` tells MoviePy to use `CompositeVideoClip` internally for the overlapping portions rather than trying to blend frames at the pixel level, which is how the alpha ramp from `crossfadein` is correctly respected.
|
||||||
|
|
||||||
|
The default `fade_dur` of `0.5s` is appropriate for fast-paced content. Increase to `0.8–1.0s` for a more cinematic feel. The total output duration is `sum(scene.duration for scene in scenes) - (len(scenes) - 1) * fade_dur`.
|
||||||
|
|
||||||
|
### 9.2 The `set_duration` gotcha
|
||||||
|
|
||||||
|
`CompositeVideoClip` infers its total duration by scanning the durations of all constituent clips. When sub-clips have `set_start` offsets — such as the chart clip which starts at `t=0.5` and has a duration of `d - 1.5`, and the insight card which starts at `t=0.5` with a duration of `d - 1.0` — MoviePy computes the composite's duration as `max(clip.start + clip.duration for clip in layers)`.
|
||||||
|
|
||||||
|
In most cases this yields a value slightly larger than `d` due to floating-point arithmetic on the offset calculations, or occasionally slightly smaller if a sub-clip ends fractionally before the background. Either error causes `crossfade_concat`'s `padding=-fade_dur` overlap to be miscalculated, typically producing a black flash frame at each scene boundary.
|
||||||
|
|
||||||
|
The fix is to explicitly call `.set_duration(d)` on every scene builder's return value, overriding the inferred value with the authoritative duration from the `Insight`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
return CompositeVideoClip(layers, size=bg.size).set_duration(d)
|
||||||
|
```
|
||||||
|
|
||||||
|
This must be applied to all three builders, including `build_full_avatar_scene`, because a `resize()` call on a `VideoFileClip` creates a new clip object whose duration is re-derived from the source — it does not inherit the `subclip(0, d)` duration reliably on all platforms.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Master compositor — `compose_video`
|
||||||
|
|
||||||
|
```python
|
||||||
|
def compose_video(scenes: list, output_path: str = "output.mp4",
|
||||||
|
fps: int = 24, fade_dur: float = 0.5) -> str
|
||||||
|
```
|
||||||
|
|
||||||
|
The final assembly step. Calls `crossfade_concat` to produce the dissolved timeline, then writes to an H.264 MP4 via MoviePy's `write_videofile`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
final.write_videofile(
|
||||||
|
output_path,
|
||||||
|
fps=fps,
|
||||||
|
codec="libx264",
|
||||||
|
audio_codec="aac",
|
||||||
|
threads=4,
|
||||||
|
preset="fast",
|
||||||
|
logger=None,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
`preset="fast"` is a reasonable default for a production pipeline — it is significantly faster than `slow` or `medium` with only a marginal quality difference at typical web streaming bitrates. Change to `slow` for archive-quality output. `logger=None` suppresses the verbose ffmpeg progress output; remove it during debugging.
|
||||||
|
|
||||||
|
`threads=4` maps to ffmpeg's `-threads` flag. Increase if the host has more cores available. This affects the encoding step only — MoviePy's frame rendering is single-threaded.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. FastAPI server
|
||||||
|
|
||||||
|
`api_server.py` wraps the composition pipeline behind an HTTP API, enabling it to be called from any frontend, automation script, or orchestration system.
|
||||||
|
|
||||||
|
### 11.1 Request models
|
||||||
|
|
||||||
|
**`InsightPayload`** — mirrors the `Insight` dataclass with Pydantic validation:
|
||||||
|
|
||||||
|
| Field | Type | Constraints | Description |
|
||||||
|
|-------|------|-------------|-------------|
|
||||||
|
| `key_insight` | str | required | Headline text |
|
||||||
|
| `supporting_stat` | str | required | Sub-headline text |
|
||||||
|
| `visual_cue` | str | required | Scene template selector |
|
||||||
|
| `audio_tone` | str | required | Downstream audio metadata |
|
||||||
|
| `duration` | float | 3.0–60.0 | Scene length in seconds |
|
||||||
|
| `chart_data` | dict | optional | Data payload for chart generators |
|
||||||
|
| `bullet_lines` | list[str] | optional | Explicit bullet text (overrides defaults) |
|
||||||
|
|
||||||
|
**`ComposeRequest`** — the top-level request body:
|
||||||
|
|
||||||
|
| Field | Type | Default | Description |
|
||||||
|
|-------|------|---------|-------------|
|
||||||
|
| `insights` | list[InsightPayload] | required | Ordered list of scenes |
|
||||||
|
| `fps` | int | 24 | Output frame rate (12–60) |
|
||||||
|
| `fade_dur` | float | 0.5 | Crossfade duration in seconds (0.0–2.0) |
|
||||||
|
|
||||||
|
**`JobStatus`** — the response model for job tracking:
|
||||||
|
|
||||||
|
| Field | Values | Description |
|
||||||
|
|-------|--------|-------------|
|
||||||
|
| `job_id` | UUID hex string | Unique identifier for polling |
|
||||||
|
| `status` | `queued`, `processing`, `done`, `error` | Current state |
|
||||||
|
| `output_url` | `/download/{job_id}` or null | Available when `status == "done"` |
|
||||||
|
| `error` | string or null | Error message when `status == "error"` |
|
||||||
|
|
||||||
|
### 11.2 Job lifecycle
|
||||||
|
|
||||||
|
Video composition is CPU-intensive and typically takes 30–120 seconds for a multi-scene piece. The API uses FastAPI's `BackgroundTasks` to run composition asynchronously so the HTTP response is immediate:
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /compose
|
||||||
|
│
|
||||||
|
├─ Validates payload, saves uploaded files to /tmp/broll_jobs/{job_id}/
|
||||||
|
├─ Creates JobStatus(status="queued")
|
||||||
|
├─ Registers BackgroundTask → _compose_worker()
|
||||||
|
└─ Returns 202 Accepted with job_id
|
||||||
|
|
||||||
|
_compose_worker() (background)
|
||||||
|
│
|
||||||
|
├─ Sets status = "processing"
|
||||||
|
├─ Runs _sync_compose() in a thread pool (loop.run_in_executor)
|
||||||
|
│ └─ Iterates insights → dispatch_scene() → compose_video()
|
||||||
|
├─ On success: status = "done", output_url = "/download/{job_id}"
|
||||||
|
└─ On error: status = "error", error = str(exc)
|
||||||
|
|
||||||
|
GET /status/{job_id} ← poll until status == "done" or "error"
|
||||||
|
|
||||||
|
GET /download/{job_id} ← returns MP4 file
|
||||||
|
```
|
||||||
|
|
||||||
|
`loop.run_in_executor(None, _sync_compose)` is important: MoviePy's frame rendering and ffmpeg's encoding are blocking operations. Running them directly in an `async` function would block the entire event loop. `run_in_executor` offloads the work to a thread pool, keeping the server responsive to other requests during composition.
|
||||||
|
|
||||||
|
The job store is currently a plain Python dict (`_jobs`). This is appropriate for a single-worker development server. Replace with Redis (using `aioredis` or `redis-py`) for multi-worker or multi-instance deployments.
|
||||||
|
|
||||||
|
### 11.3 API endpoints
|
||||||
|
|
||||||
|
| Method | Path | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `POST` | `/compose` | Start a composition job (multipart form) |
|
||||||
|
| `GET` | `/status/{job_id}` | Poll job status |
|
||||||
|
| `GET` | `/download/{job_id}` | Download finished MP4 |
|
||||||
|
| `POST` | `/preview/chart` | Generate and return a chart PNG (no video) |
|
||||||
|
| `GET` | `/health` | Liveness check |
|
||||||
|
|
||||||
|
Interactive documentation is available at `http://localhost:8000/docs` once the server is running (FastAPI's built-in Swagger UI).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Running the project
|
||||||
|
|
||||||
|
### 12.1 Smoke test (no media files needed)
|
||||||
|
|
||||||
|
The smoke test validates all asset generators — chart PNGs, bullet overlays, and insight cards — without requiring any background images or avatar videos:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python broll_composer.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output:
|
||||||
|
|
||||||
|
```
|
||||||
|
Chart saved → /tmp/demo_chart.png
|
||||||
|
Bullets saved → /tmp/demo_bullets.png
|
||||||
|
Insight card saved → /tmp/demo_card.png
|
||||||
|
|
||||||
|
Sample Insight JSON: { ... }
|
||||||
|
|
||||||
|
All asset generation tests passed.
|
||||||
|
To run full video composition, supply real background_img and avatar_video paths.
|
||||||
|
```
|
||||||
|
|
||||||
|
Inspect the PNG files in `/tmp/` to visually verify chart rendering before running the full pipeline.
|
||||||
|
|
||||||
|
### 12.2 Full video composition
|
||||||
|
|
||||||
|
```python
|
||||||
|
from broll_composer import pipeline_from_json
|
||||||
|
|
||||||
|
insight_json = """{
|
||||||
|
"key_insight": "AI reduced production time by 40%",
|
||||||
|
"supporting_stat": "HubSpot 2026: 12% CTR lift",
|
||||||
|
"visual_cue": "bar_chart_comparison",
|
||||||
|
"audio_tone": "authoritative_and_surprising",
|
||||||
|
"duration": 10.0,
|
||||||
|
"chart_data": {
|
||||||
|
"labels": ["Content Velocity", "CTR", "Engagement", "Cost/Lead"],
|
||||||
|
"before": [30, 22, 18, 60],
|
||||||
|
"after": [72, 34, 41, 38]
|
||||||
|
}
|
||||||
|
}"""
|
||||||
|
|
||||||
|
output_path = pipeline_from_json(
|
||||||
|
insight_json,
|
||||||
|
background_img="path/to/background.jpg",
|
||||||
|
avatar_video="path/to/avatar.mp4", # optional
|
||||||
|
)
|
||||||
|
print(f"Video written to {output_path}")
|
||||||
|
```
|
||||||
|
|
||||||
|
### 12.3 API server
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uvicorn api_server:app --host 0.0.0.0 --port 8000
|
||||||
|
```
|
||||||
|
|
||||||
|
For development with auto-reload:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uvicorn api_server:app --reload
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 13. Calling the API
|
||||||
|
|
||||||
|
The `/compose` endpoint accepts `multipart/form-data` with three parts: `payload` (JSON string), `background` (image file), and optionally `avatar` (video file).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8000/compose \
|
||||||
|
-F 'payload={
|
||||||
|
"insights": [{
|
||||||
|
"key_insight": "AI reduced production time by 40%",
|
||||||
|
"supporting_stat": "HubSpot 2026: 12% CTR lift",
|
||||||
|
"visual_cue": "bar_chart_comparison",
|
||||||
|
"audio_tone": "authoritative_and_surprising",
|
||||||
|
"duration": 10.0,
|
||||||
|
"chart_data": {
|
||||||
|
"labels": ["Velocity","CTR","Engagement","Cost/Lead"],
|
||||||
|
"before": [30, 22, 18, 60],
|
||||||
|
"after": [72, 34, 41, 38]
|
||||||
|
}
|
||||||
|
}],
|
||||||
|
"fps": 24,
|
||||||
|
"fade_dur": 0.5
|
||||||
|
}' \
|
||||||
|
-F 'background=@./bg.jpg' \
|
||||||
|
-F 'avatar=@./avatar.mp4'
|
||||||
|
```
|
||||||
|
|
||||||
|
This returns a `JobStatus` with a `job_id`. Poll for completion:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8000/status/{job_id}
|
||||||
|
# → {"job_id": "...", "status": "done", "output_url": "/download/..."}
|
||||||
|
```
|
||||||
|
|
||||||
|
Download the finished video:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -O http://localhost:8000/download/{job_id}
|
||||||
|
```
|
||||||
|
|
||||||
|
Preview a chart without video assembly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST "http://localhost:8000/preview/chart?title=My+Chart&chart_type=bar_chart_comparison" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"labels":["A","B"],"before":[30,22],"after":[72,34]}' \
|
||||||
|
--output preview.png
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 14. Production notes
|
||||||
|
|
||||||
|
**Concurrency**: FastAPI's `BackgroundTasks` runs in the same process as the web server. Under concurrent load, multiple composition jobs will share the same thread pool, which can cause memory pressure (each MoviePy frame rendering buffers several seconds of uncompressed video). For production, move composition to a dedicated worker queue (Celery + Redis, or ARQ) and have the API server dispatch jobs to it rather than running them in-process.
|
||||||
|
|
||||||
|
**Temp file isolation**: Chart PNGs and insight card PNGs are written to fixed paths under `/tmp/`. This is safe for sequential processing but will cause race conditions if jobs are parallelised. Prefix all temp file paths with the `job_id` to isolate them:
|
||||||
|
|
||||||
|
```python
|
||||||
|
chart_path = f"/tmp/{job_id}_chart.png"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Memory**: MoviePy loads entire video clips into memory for compositing. For scenes longer than ~30 seconds with a high-resolution avatar, memory use can reach several GB. If this is a concern, render scenes individually and use ffmpeg's `concat` demuxer to join them in a second pass rather than compositing them all in Python.
|
||||||
|
|
||||||
|
**ffmpeg version**: MoviePy 1.0.3 delegates encoding to ffmpeg. Versions prior to 4.x may not support all `preset` values or the `aac` codec without additional flags. The pipeline is tested against ffmpeg 5.x and 6.x.
|
||||||
|
|
||||||
|
**File cleanup**: Completed job files accumulate in `/tmp/broll_jobs/`. Add a cleanup background task or cron job that deletes job directories older than a configurable TTL (e.g. 1 hour).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 15. Extending the pipeline
|
||||||
|
|
||||||
|
**Adding a new scene template**: add a builder function following the `build_*_scene` naming convention, then add a `visual_cue` string → function mapping in `dispatch_scene`. No other changes are needed.
|
||||||
|
|
||||||
|
**Adding a new chart type**: add a `make_*` function that writes a transparent PNG, then handle the new `visual_cue` in `dispatch_scene` by calling it before passing `assets` to a builder.
|
||||||
|
|
||||||
|
**Supporting multiple backgrounds per script**: `SceneAssets` currently takes a single `background_img`. To vary the background per scene, add a `background_img` field to `InsightPayload` in the API model and pass it through to `SceneAssets` in the compose worker.
|
||||||
|
|
||||||
|
**Audio**: the pipeline produces silent video. Attach a voiceover by loading it as a MoviePy `AudioFileClip`, setting its start time to align with each scene, and passing the composite audio to `final.set_audio()` before calling `write_videofile`.
|
||||||
229
backend/api/podcast/broll_temp/api_server.py
Normal file
229
backend/api/podcast/broll_temp/api_server.py
Normal file
@@ -0,0 +1,229 @@
|
|||||||
|
"""
|
||||||
|
FastAPI wrapper for the B-Roll Composer pipeline.
|
||||||
|
POST /compose → triggers scene assembly, returns video download URL.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
import os
|
||||||
|
import uuid
|
||||||
|
import json
|
||||||
|
import asyncio
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional, List
|
||||||
|
|
||||||
|
from fastapi import FastAPI, UploadFile, File, Form, BackgroundTasks, HTTPException
|
||||||
|
from fastapi.responses import FileResponse, JSONResponse
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from broll_composer import (
|
||||||
|
Insight, SceneAssets, dispatch_scene, compose_video,
|
||||||
|
make_bar_chart, make_line_trend, make_bullet_overlay,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# App setup
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
app = FastAPI(
|
||||||
|
title="B-Roll Composer API",
|
||||||
|
description="Programmatic video composition: Background + Chart + Avatar Circle",
|
||||||
|
version="1.0.0",
|
||||||
|
)
|
||||||
|
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=["*"],
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
WORK_DIR = Path("/tmp/broll_jobs")
|
||||||
|
WORK_DIR.mkdir(exist_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Request / Response models
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class InsightPayload(BaseModel):
|
||||||
|
key_insight: str = Field(..., example="AI tools reduced content cycles by 40% in 2025.")
|
||||||
|
supporting_stat: str = Field(..., example="HubSpot 2026 report cites a 12% lift in CTR.")
|
||||||
|
visual_cue: str = Field(
|
||||||
|
...,
|
||||||
|
example="bar_chart_comparison",
|
||||||
|
description="bar_chart_comparison | line_trend | bullet_points | full_avatar",
|
||||||
|
)
|
||||||
|
audio_tone: str = Field(..., example="authoritative_and_surprising")
|
||||||
|
duration: float = Field(default=10.0, ge=3.0, le=60.0)
|
||||||
|
chart_data: dict = Field(default_factory=dict)
|
||||||
|
bullet_lines: Optional[List[str]] = None
|
||||||
|
|
||||||
|
|
||||||
|
class ComposeRequest(BaseModel):
|
||||||
|
insights: List[InsightPayload]
|
||||||
|
fps: int = Field(default=24, ge=12, le=60)
|
||||||
|
fade_dur: float = Field(default=0.5, ge=0.0, le=2.0,
|
||||||
|
description="Crossfade duration in seconds between scenes")
|
||||||
|
|
||||||
|
|
||||||
|
class JobStatus(BaseModel):
|
||||||
|
job_id: str
|
||||||
|
status: str # queued | processing | done | error
|
||||||
|
output_url: Optional[str] = None
|
||||||
|
error: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# In-memory job store (replace with Redis in production)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_jobs: dict[str, JobStatus] = {}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Background task: composition worker
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
async def _compose_worker(
|
||||||
|
job_id: str,
|
||||||
|
request: ComposeRequest,
|
||||||
|
bg_path: str,
|
||||||
|
avatar_path: Optional[str],
|
||||||
|
):
|
||||||
|
job = _jobs[job_id]
|
||||||
|
job.status = "processing"
|
||||||
|
|
||||||
|
try:
|
||||||
|
loop = asyncio.get_running_loop()
|
||||||
|
out_path = str(WORK_DIR / f"{job_id}.mp4")
|
||||||
|
|
||||||
|
def _sync_compose():
|
||||||
|
scenes = []
|
||||||
|
for i, payload in enumerate(request.insights):
|
||||||
|
insight = Insight(
|
||||||
|
key_insight=payload.key_insight,
|
||||||
|
supporting_stat=payload.supporting_stat,
|
||||||
|
visual_cue=payload.visual_cue,
|
||||||
|
audio_tone=payload.audio_tone,
|
||||||
|
chart_data=payload.chart_data,
|
||||||
|
duration=payload.duration,
|
||||||
|
)
|
||||||
|
assets = SceneAssets(
|
||||||
|
background_img=bg_path,
|
||||||
|
avatar_video=avatar_path,
|
||||||
|
)
|
||||||
|
scene = dispatch_scene(insight, assets, payload.bullet_lines)
|
||||||
|
scenes.append(scene)
|
||||||
|
|
||||||
|
compose_video(scenes, output_path=out_path, fps=request.fps,
|
||||||
|
fade_dur=request.fade_dur)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
await loop.run_in_executor(None, _sync_compose)
|
||||||
|
job.status = "done"
|
||||||
|
job.output_url = f"/download/{job_id}"
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
job.status = "error"
|
||||||
|
job.error = str(exc)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Endpoints
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@app.post("/compose", response_model=JobStatus, status_code=202)
|
||||||
|
async def start_compose(
|
||||||
|
background_tasks: BackgroundTasks,
|
||||||
|
payload: str = Form(..., description="JSON string matching ComposeRequest schema"),
|
||||||
|
background: UploadFile = File(..., description="Background image (JPEG/PNG)"),
|
||||||
|
avatar: Optional[UploadFile] = File(None, description="Avatar video (MP4) — optional"),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Kick off a video composition job.
|
||||||
|
- **payload**: JSON body (ComposeRequest)
|
||||||
|
- **background**: background image file
|
||||||
|
- **avatar**: optional avatar video file
|
||||||
|
Returns a job_id — poll GET /status/{job_id} for progress.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
request = ComposeRequest(**json.loads(payload))
|
||||||
|
except Exception as e:
|
||||||
|
raise HTTPException(status_code=422, detail=f"Invalid payload: {e}")
|
||||||
|
|
||||||
|
job_id = uuid.uuid4().hex
|
||||||
|
|
||||||
|
# Save uploads
|
||||||
|
job_dir = WORK_DIR / job_id
|
||||||
|
job_dir.mkdir(exist_ok=True)
|
||||||
|
|
||||||
|
bg_path = str(job_dir / background.filename)
|
||||||
|
with open(bg_path, "wb") as f:
|
||||||
|
f.write(await background.read())
|
||||||
|
|
||||||
|
avatar_path = None
|
||||||
|
if avatar:
|
||||||
|
avatar_path = str(job_dir / avatar.filename)
|
||||||
|
with open(avatar_path, "wb") as f:
|
||||||
|
f.write(await avatar.read())
|
||||||
|
|
||||||
|
# Register job
|
||||||
|
job = JobStatus(job_id=job_id, status="queued")
|
||||||
|
_jobs[job_id] = job
|
||||||
|
|
||||||
|
# Launch background worker
|
||||||
|
background_tasks.add_task(
|
||||||
|
_compose_worker, job_id, request, bg_path, avatar_path
|
||||||
|
)
|
||||||
|
|
||||||
|
return job
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/status/{job_id}", response_model=JobStatus)
|
||||||
|
async def get_status(job_id: str):
|
||||||
|
"""Poll composition job status."""
|
||||||
|
job = _jobs.get(job_id)
|
||||||
|
if not job:
|
||||||
|
raise HTTPException(status_code=404, detail="Job not found")
|
||||||
|
return job
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/download/{job_id}")
|
||||||
|
async def download_video(job_id: str):
|
||||||
|
"""Download the finished video."""
|
||||||
|
job = _jobs.get(job_id)
|
||||||
|
if not job:
|
||||||
|
raise HTTPException(status_code=404, detail="Job not found")
|
||||||
|
if job.status != "done":
|
||||||
|
raise HTTPException(status_code=409, detail=f"Job status: {job.status}")
|
||||||
|
|
||||||
|
out_path = WORK_DIR / f"{job_id}.mp4"
|
||||||
|
if not out_path.exists():
|
||||||
|
raise HTTPException(status_code=404, detail="Output file missing")
|
||||||
|
|
||||||
|
return FileResponse(
|
||||||
|
path=str(out_path),
|
||||||
|
media_type="video/mp4",
|
||||||
|
filename=f"broll_{job_id}.mp4",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/preview/chart")
|
||||||
|
async def preview_chart(
|
||||||
|
chart_data: dict,
|
||||||
|
title: str = "",
|
||||||
|
chart_type: str = "bar_chart_comparison",
|
||||||
|
):
|
||||||
|
"""Generate and return a chart PNG for preview (no video assembly)."""
|
||||||
|
out = str(WORK_DIR / f"preview_{uuid.uuid4().hex}.png")
|
||||||
|
if chart_type == "bar_chart_comparison":
|
||||||
|
make_bar_chart(chart_data, out, title)
|
||||||
|
else:
|
||||||
|
make_line_trend(chart_data, out, title)
|
||||||
|
return FileResponse(path=out, media_type="image/png")
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/health")
|
||||||
|
async def health():
|
||||||
|
return {"status": "ok"}
|
||||||
456
backend/api/podcast/broll_temp/broll_composer.py
Normal file
456
backend/api/podcast/broll_temp/broll_composer.py
Normal file
@@ -0,0 +1,456 @@
|
|||||||
|
"""
|
||||||
|
Programmatic B-Roll Composer
|
||||||
|
Layered composition pipeline: Background + Chart + Avatar Circle + Text Overlays
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import numpy as np
|
||||||
|
from pathlib import Path
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Optional
|
||||||
|
import matplotlib
|
||||||
|
matplotlib.use("Agg")
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import matplotlib.patches as mpatches
|
||||||
|
from PIL import Image, ImageDraw, ImageFont
|
||||||
|
from moviepy.editor import (
|
||||||
|
VideoFileClip, ImageClip, CompositeVideoClip,
|
||||||
|
TextClip, ColorClip, concatenate_videoclips,
|
||||||
|
)
|
||||||
|
import moviepy.video.fx.all as vfx
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Crossfade concat (Option 1: crossfadein + negative padding)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def crossfade_concat(scenes: list, fade_dur: float = 0.5) -> CompositeVideoClip:
|
||||||
|
"""
|
||||||
|
Concatenate scenes with a dissolve transition between each pair.
|
||||||
|
|
||||||
|
Each clip (except the first) gets a crossfadein effect.
|
||||||
|
padding=-fade_dur overlaps consecutive clips so the fade actually fires
|
||||||
|
instead of creating a black gap. set_duration on every scene is
|
||||||
|
mandatory — CompositeVideoClip.duration can be ambiguous without it,
|
||||||
|
which makes the overlap math wrong.
|
||||||
|
"""
|
||||||
|
faded = []
|
||||||
|
for i, clip in enumerate(scenes):
|
||||||
|
c = clip
|
||||||
|
if i > 0:
|
||||||
|
c = c.fx(vfx.crossfadein, fade_dur)
|
||||||
|
faded.append(c)
|
||||||
|
return concatenate_videoclips(faded, padding=-fade_dur, method="compose")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Data structures
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Insight:
|
||||||
|
key_insight: str
|
||||||
|
supporting_stat: str
|
||||||
|
visual_cue: str # bar_chart_comparison | line_trend | bullet_points | full_avatar
|
||||||
|
audio_tone: str
|
||||||
|
chart_data: dict = field(default_factory=dict)
|
||||||
|
duration: float = 10.0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SceneAssets:
|
||||||
|
background_img: str
|
||||||
|
chart_img: Optional[str] = None
|
||||||
|
avatar_video: Optional[str] = None
|
||||||
|
bullet_img: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Chart generator (Matplotlib → PNG with transparency)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
CHART_STYLE = {
|
||||||
|
"bg": "#0D0D0D",
|
||||||
|
"bar_before": "#2E4057",
|
||||||
|
"bar_after": "#E63946",
|
||||||
|
"text": "#F1F1EF",
|
||||||
|
"grid": "#2A2A2A",
|
||||||
|
"accent": "#E63946",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def make_bar_chart(data: dict, out_path: str, title: str = "") -> str:
|
||||||
|
"""Render a side-by-side comparison bar chart. Returns output path."""
|
||||||
|
labels = data.get("labels", [])
|
||||||
|
before = data.get("before", [])
|
||||||
|
after = data.get("after", [])
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
|
||||||
|
x = np.arange(len(labels))
|
||||||
|
w = 0.35
|
||||||
|
bars_b = ax.bar(x - w / 2, before, w, color=CHART_STYLE["bar_before"],
|
||||||
|
label="Before", zorder=3, edgecolor="none")
|
||||||
|
bars_a = ax.bar(x + w / 2, after, w, color=CHART_STYLE["bar_after"],
|
||||||
|
label="After", zorder=3, edgecolor="none")
|
||||||
|
|
||||||
|
ax.set_xticks(x)
|
||||||
|
ax.set_xticklabels(labels, color=CHART_STYLE["text"], fontsize=11)
|
||||||
|
ax.tick_params(axis="y", colors=CHART_STYLE["text"])
|
||||||
|
ax.spines[:].set_visible(False)
|
||||||
|
ax.yaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||||
|
ax.set_axisbelow(True)
|
||||||
|
|
||||||
|
# Value labels on bars
|
||||||
|
for bar in [*bars_b, *bars_a]:
|
||||||
|
h = bar.get_height()
|
||||||
|
ax.text(bar.get_x() + bar.get_width() / 2, h + 0.5, f"{h:.0f}%",
|
||||||
|
ha="center", va="bottom", color=CHART_STYLE["text"], fontsize=9,
|
||||||
|
fontweight="bold")
|
||||||
|
|
||||||
|
legend = ax.legend(frameon=False, labelcolor=CHART_STYLE["text"],
|
||||||
|
fontsize=10, loc="upper left")
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
|
||||||
|
fig.tight_layout(pad=0.5)
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_line_trend(data: dict, out_path: str, title: str = "") -> str:
|
||||||
|
"""Render a trend line chart. Returns output path."""
|
||||||
|
x_vals = data.get("x", [])
|
||||||
|
y_vals = data.get("y", [])
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
ax.plot(x_vals, y_vals, color=CHART_STYLE["accent"],
|
||||||
|
linewidth=2.5, marker="o", markersize=7, zorder=3)
|
||||||
|
ax.fill_between(x_vals, y_vals, alpha=0.12, color=CHART_STYLE["accent"])
|
||||||
|
ax.spines[:].set_visible(False)
|
||||||
|
ax.tick_params(colors=CHART_STYLE["text"])
|
||||||
|
ax.yaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
fig.tight_layout(pad=0.5)
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Text / Bullet overlay (Pillow → PNG)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def make_bullet_overlay(lines: list[str], out_path: str,
|
||||||
|
width: int = 900, font_size: int = 32) -> str:
|
||||||
|
"""Render bullet points on a semi-transparent dark pill. Returns path."""
|
||||||
|
padding = 32
|
||||||
|
line_h = font_size + 16
|
||||||
|
img_h = padding * 2 + len(lines) * line_h + 12
|
||||||
|
img = Image.new("RGBA", (width, img_h), (0, 0, 0, 0))
|
||||||
|
draw = ImageDraw.Draw(img)
|
||||||
|
|
||||||
|
# Semi-transparent background pill
|
||||||
|
draw.rounded_rectangle([0, 0, width - 1, img_h - 1],
|
||||||
|
radius=18, fill=(10, 10, 10, 185))
|
||||||
|
|
||||||
|
try:
|
||||||
|
font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
|
||||||
|
font_size)
|
||||||
|
except OSError:
|
||||||
|
font = ImageFont.load_default()
|
||||||
|
|
||||||
|
y = padding
|
||||||
|
for line in lines:
|
||||||
|
draw.text((padding + 18, y), f"• {line}", font=font, fill=(241, 241, 239, 255))
|
||||||
|
y += line_h
|
||||||
|
|
||||||
|
img.save(out_path, format="PNG")
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_insight_card(insight: str, stat: str, out_path: str,
|
||||||
|
width: int = 960, height: int = 200) -> str:
|
||||||
|
"""Render a bold insight card (headline + supporting stat). Returns path."""
|
||||||
|
img = Image.new("RGBA", (width, height), (0, 0, 0, 0))
|
||||||
|
draw = ImageDraw.Draw(img)
|
||||||
|
draw.rounded_rectangle([0, 0, width - 1, height - 1],
|
||||||
|
radius=14, fill=(10, 10, 10, 200))
|
||||||
|
|
||||||
|
# Red accent bar
|
||||||
|
draw.rectangle([28, 24, 36, height - 24], fill=(230, 57, 70, 255))
|
||||||
|
|
||||||
|
try:
|
||||||
|
font_lg = ImageFont.truetype(
|
||||||
|
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", 34)
|
||||||
|
font_sm = ImageFont.truetype(
|
||||||
|
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 20)
|
||||||
|
except OSError:
|
||||||
|
font_lg = font_sm = ImageFont.load_default()
|
||||||
|
|
||||||
|
draw.text((58, 36), insight, font=font_lg, fill=(241, 241, 239, 255))
|
||||||
|
draw.text((58, 90), stat, font=font_sm, fill=(180, 180, 178, 230))
|
||||||
|
|
||||||
|
img.save(out_path, format="PNG")
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Circular avatar mask
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def apply_circle_mask(clip: VideoFileClip, diameter: int) -> VideoFileClip:
|
||||||
|
"""Resize clip and apply a circular alpha mask."""
|
||||||
|
clip = clip.resize(height=diameter)
|
||||||
|
w, h = clip.size
|
||||||
|
|
||||||
|
# Build a circular mask array (1 = opaque, 0 = transparent)
|
||||||
|
Y, X = np.ogrid[:h, :w]
|
||||||
|
cx, cy = w / 2, h / 2
|
||||||
|
mask_arr = ((X - cx) ** 2 + (Y - cy) ** 2 <= (min(w, h) / 2) ** 2).astype(float)
|
||||||
|
|
||||||
|
mask_clip = ImageClip(mask_arr, ismask=True).set_duration(clip.duration)
|
||||||
|
return clip.set_mask(mask_clip)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Ken Burns zoom effect
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def ken_burns(clip: ImageClip, zoom_ratio: float = 0.08) -> ImageClip:
|
||||||
|
"""Apply a slow zoom-in over the clip duration."""
|
||||||
|
def zoom_frame(get_frame, t):
|
||||||
|
frame = get_frame(t)
|
||||||
|
frac = 1 + zoom_ratio * (t / clip.duration)
|
||||||
|
h, w = frame.shape[:2]
|
||||||
|
new_h, new_w = int(h / frac), int(w / frac)
|
||||||
|
y1 = (h - new_h) // 2
|
||||||
|
x1 = (w - new_w) // 2
|
||||||
|
cropped = frame[y1:y1 + new_h, x1:x1 + new_w]
|
||||||
|
return np.array(Image.fromarray(cropped).resize((w, h), Image.LANCZOS))
|
||||||
|
|
||||||
|
return clip.fl(zoom_frame, apply_to=["mask"])
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Scene builders (one per visual_cue type)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def build_data_scene(assets: SceneAssets, insight: Insight) -> CompositeVideoClip:
|
||||||
|
"""
|
||||||
|
Layout: Background (Ken Burns) + Chart (fade-in) + Avatar circle (corner) + Insight card
|
||||||
|
"""
|
||||||
|
d = insight.duration
|
||||||
|
layers = []
|
||||||
|
|
||||||
|
# 1. Background
|
||||||
|
bg = (ImageClip(assets.background_img)
|
||||||
|
.set_duration(d)
|
||||||
|
.resize(height=1080))
|
||||||
|
bg = ken_burns(bg)
|
||||||
|
bg = bg.fx(vfx.lum_contrast, 0, -40) # darken 40 units
|
||||||
|
layers.append(bg)
|
||||||
|
|
||||||
|
# 2. Programmatic chart
|
||||||
|
if assets.chart_img:
|
||||||
|
chart = (ImageClip(assets.chart_img)
|
||||||
|
.set_duration(d - 1.5)
|
||||||
|
.set_start(0.5)
|
||||||
|
.resize(width=700)
|
||||||
|
.set_position(("center", 180))
|
||||||
|
.fx(vfx.fadein, 0.6)
|
||||||
|
.fx(vfx.fadeout, 0.4))
|
||||||
|
layers.append(chart)
|
||||||
|
|
||||||
|
# 3. Insight card at bottom
|
||||||
|
card_path = "/tmp/insight_card.png"
|
||||||
|
make_insight_card(insight.key_insight, insight.supporting_stat, card_path)
|
||||||
|
card = (ImageClip(card_path)
|
||||||
|
.set_duration(d - 1)
|
||||||
|
.set_start(0.5)
|
||||||
|
.set_position(("center", 820))
|
||||||
|
.fx(vfx.fadein, 0.5))
|
||||||
|
layers.append(card)
|
||||||
|
|
||||||
|
# 4. Avatar circle (bottom-right corner)
|
||||||
|
if assets.avatar_video:
|
||||||
|
avatar_raw = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||||
|
avatar = apply_circle_mask(avatar_raw, diameter=240)
|
||||||
|
avatar = avatar.set_position((bg.w - 280, bg.h - 280))
|
||||||
|
layers.append(avatar)
|
||||||
|
|
||||||
|
# set_duration is required: CompositeVideoClip infers duration from its
|
||||||
|
# constituent clips, which can be ambiguous when sub-clips have set_start
|
||||||
|
# offsets. Without this, crossfade_concat's overlap math goes wrong.
|
||||||
|
return CompositeVideoClip(layers, size=bg.size).set_duration(d)
|
||||||
|
|
||||||
|
|
||||||
|
def build_bullet_scene(assets: SceneAssets, insight: Insight,
|
||||||
|
bullets: list[str]) -> CompositeVideoClip:
|
||||||
|
"""
|
||||||
|
Layout: AI image (Ken Burns) + Bullet overlay + Avatar circle
|
||||||
|
"""
|
||||||
|
d = insight.duration
|
||||||
|
layers = []
|
||||||
|
|
||||||
|
bg = (ImageClip(assets.background_img)
|
||||||
|
.set_duration(d)
|
||||||
|
.resize(height=1080))
|
||||||
|
bg = ken_burns(bg, zoom_ratio=0.05)
|
||||||
|
bg = bg.fx(vfx.lum_contrast, 0, -50)
|
||||||
|
layers.append(bg)
|
||||||
|
|
||||||
|
bullet_path = "/tmp/bullets.png"
|
||||||
|
make_bullet_overlay(bullets, bullet_path, width=860)
|
||||||
|
bullets_clip = (ImageClip(bullet_path)
|
||||||
|
.set_duration(d - 1)
|
||||||
|
.set_start(0.5)
|
||||||
|
.set_position(("center", "center"))
|
||||||
|
.fx(vfx.fadein, 0.7))
|
||||||
|
layers.append(bullets_clip)
|
||||||
|
|
||||||
|
if assets.avatar_video:
|
||||||
|
avatar_raw = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||||
|
avatar = apply_circle_mask(avatar_raw, diameter=200)
|
||||||
|
avatar = avatar.set_position((bg.w - 240, bg.h - 240))
|
||||||
|
layers.append(avatar)
|
||||||
|
|
||||||
|
return CompositeVideoClip(layers, size=bg.size).set_duration(d)
|
||||||
|
|
||||||
|
|
||||||
|
def build_full_avatar_scene(assets: SceneAssets, insight: Insight) -> VideoFileClip:
|
||||||
|
"""Full-screen avatar — the expensive 'Hook' scene. No overlay."""
|
||||||
|
d = insight.duration
|
||||||
|
avatar = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||||
|
return avatar.resize(height=1080).set_duration(d)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Scene dispatcher — maps visual_cue → builder
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def dispatch_scene(insight: Insight, assets: SceneAssets,
|
||||||
|
bullet_lines: Optional[list[str]] = None) -> CompositeVideoClip:
|
||||||
|
cue = insight.visual_cue
|
||||||
|
|
||||||
|
if cue == "full_avatar":
|
||||||
|
return build_full_avatar_scene(assets, insight)
|
||||||
|
|
||||||
|
elif cue in ("bar_chart_comparison", "line_trend"):
|
||||||
|
chart_path = "/tmp/chart.png"
|
||||||
|
if cue == "bar_chart_comparison":
|
||||||
|
make_bar_chart(insight.chart_data, chart_path,
|
||||||
|
title=insight.key_insight)
|
||||||
|
else:
|
||||||
|
make_line_trend(insight.chart_data, chart_path,
|
||||||
|
title=insight.key_insight)
|
||||||
|
assets.chart_img = chart_path
|
||||||
|
return build_data_scene(assets, insight)
|
||||||
|
|
||||||
|
elif cue == "bullet_points":
|
||||||
|
lines = bullet_lines or [insight.key_insight, insight.supporting_stat]
|
||||||
|
return build_bullet_scene(assets, insight, lines)
|
||||||
|
|
||||||
|
else:
|
||||||
|
# Fallback: data scene without chart
|
||||||
|
return build_data_scene(assets, insight)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Master compositor — assembles all scenes into one video
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def compose_video(scenes: list, output_path: str = "output.mp4",
|
||||||
|
fps: int = 24, fade_dur: float = 0.5) -> str:
|
||||||
|
"""Concatenate scenes with crossfade transitions and write final video file."""
|
||||||
|
final = crossfade_concat(scenes, fade_dur=fade_dur)
|
||||||
|
final.write_videofile(
|
||||||
|
output_path,
|
||||||
|
fps=fps,
|
||||||
|
codec="libx264",
|
||||||
|
audio_codec="aac",
|
||||||
|
threads=4,
|
||||||
|
preset="fast",
|
||||||
|
logger=None,
|
||||||
|
)
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# JSON bridge — LLM insight → assets + scene
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def pipeline_from_json(insight_json: str,
|
||||||
|
background_img: str,
|
||||||
|
avatar_video: Optional[str] = None) -> str:
|
||||||
|
"""
|
||||||
|
Full pipeline:
|
||||||
|
1. Parse LLM insight JSON
|
||||||
|
2. Generate chart / overlay assets
|
||||||
|
3. Build scene
|
||||||
|
4. Write video
|
||||||
|
Returns path to output video.
|
||||||
|
"""
|
||||||
|
data = json.loads(insight_json)
|
||||||
|
insight = Insight(**{k: data[k] for k in Insight.__dataclass_fields__ if k in data})
|
||||||
|
assets = SceneAssets(background_img=background_img, avatar_video=avatar_video)
|
||||||
|
scene = dispatch_scene(insight, assets,
|
||||||
|
bullet_lines=data.get("bullet_lines"))
|
||||||
|
out = f"/tmp/scene_{insight.visual_cue}.mp4"
|
||||||
|
compose_video([scene], output_path=out)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Demo / smoke-test (no real media files needed for chart generation)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
# --- Test 1: Chart PNG generation only ---
|
||||||
|
sample_bar_data = {
|
||||||
|
"labels": ["Content Velocity", "CTR", "Engagement", "Cost/Lead"],
|
||||||
|
"before": [30, 22, 18, 60],
|
||||||
|
"after": [72, 34, 41, 38],
|
||||||
|
}
|
||||||
|
chart_out = make_bar_chart(
|
||||||
|
sample_bar_data,
|
||||||
|
"/tmp/demo_chart.png",
|
||||||
|
title="AI Tools Impact: Before vs After (2025)",
|
||||||
|
)
|
||||||
|
print(f"Chart saved → {chart_out}")
|
||||||
|
|
||||||
|
# --- Test 2: Bullet overlay PNG ---
|
||||||
|
bullets = [
|
||||||
|
"AI reduced content cycles by 40% in 2025",
|
||||||
|
"HubSpot: 12% lift in CTR with AI-assisted copy",
|
||||||
|
"Video production cost down 3x with hybrid pipeline",
|
||||||
|
]
|
||||||
|
bullet_out = make_bullet_overlay(bullets, "/tmp/demo_bullets.png")
|
||||||
|
print(f"Bullets saved → {bullet_out}")
|
||||||
|
|
||||||
|
# --- Test 3: Insight card PNG ---
|
||||||
|
card_out = make_insight_card(
|
||||||
|
"AI tools reduced content cycles by 40%",
|
||||||
|
"HubSpot 2026 report — 12% lift in CTR",
|
||||||
|
"/tmp/demo_card.png",
|
||||||
|
)
|
||||||
|
print(f"Insight card saved → {card_out}")
|
||||||
|
|
||||||
|
# --- Test 4: JSON bridge (chart only, no video files required) ---
|
||||||
|
sample_json = json.dumps({
|
||||||
|
"key_insight": "AI reduced production time by 40%",
|
||||||
|
"supporting_stat": "HubSpot 2026: 12% CTR lift",
|
||||||
|
"visual_cue": "bar_chart_comparison",
|
||||||
|
"audio_tone": "authoritative_and_surprising",
|
||||||
|
"duration": 8.0,
|
||||||
|
"chart_data": sample_bar_data,
|
||||||
|
})
|
||||||
|
print("\nSample Insight JSON:\n", sample_json)
|
||||||
|
print("\nAll asset generation tests passed.")
|
||||||
|
print("To run full video composition, supply real background_img and avatar_video paths.")
|
||||||
@@ -6,6 +6,7 @@ Centralized constants and directory configuration for podcast module.
|
|||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Literal
|
from typing import Literal
|
||||||
|
from loguru import logger
|
||||||
from services.story_writer.audio_generation_service import StoryAudioGenerationService
|
from services.story_writer.audio_generation_service import StoryAudioGenerationService
|
||||||
|
|
||||||
# Directory paths
|
# Directory paths
|
||||||
@@ -45,11 +46,14 @@ def get_podcast_media_dir(
|
|||||||
}[media_type]
|
}[media_type]
|
||||||
|
|
||||||
if user_id:
|
if user_id:
|
||||||
tenant_media_dir = ROOT_DIR / "workspace" / f"workspace_{_sanitize_user_id(user_id)}" / "media" / media_subdir
|
sanitized = _sanitize_user_id(user_id)
|
||||||
|
tenant_media_dir = ROOT_DIR / "workspace" / f"workspace_{sanitized}" / "media" / media_subdir
|
||||||
resolved_dir = tenant_media_dir.resolve()
|
resolved_dir = tenant_media_dir.resolve()
|
||||||
else:
|
else:
|
||||||
resolved_dir = (DATA_MEDIA_DIR / media_subdir).resolve()
|
resolved_dir = (DATA_MEDIA_DIR / media_subdir).resolve()
|
||||||
|
|
||||||
|
logger.debug(f"[Podcast] get_podcast_media_dir: type={media_type}, user_id={user_id}, sanitized={user_id and _sanitize_user_id(user_id)}, resolved={resolved_dir}")
|
||||||
|
|
||||||
if ensure_exists:
|
if ensure_exists:
|
||||||
resolved_dir.mkdir(parents=True, exist_ok=True)
|
resolved_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
@@ -61,7 +65,9 @@ def get_podcast_media_read_dirs(media_type: MediaType, user_id: str | None = Non
|
|||||||
dirs: list[Path] = []
|
dirs: list[Path] = []
|
||||||
if user_id:
|
if user_id:
|
||||||
dirs.append(get_podcast_media_dir(media_type, user_id))
|
dirs.append(get_podcast_media_dir(media_type, user_id))
|
||||||
|
logger.debug(f"[Podcast] get_podcast_media_read_dirs: added user dir for {user_id}")
|
||||||
dirs.append(get_podcast_media_dir(media_type, None))
|
dirs.append(get_podcast_media_dir(media_type, None))
|
||||||
|
logger.debug(f"[Podcast] get_podcast_media_read_dirs: dirs={dirs}")
|
||||||
return dirs
|
return dirs
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -5,10 +5,11 @@ Analysis endpoint for podcast ideas.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends, HTTPException
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
from typing import Dict, Any
|
from typing import Dict, Any, Optional, List
|
||||||
import json
|
import json
|
||||||
import uuid
|
import uuid
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
from services.database import get_db
|
from services.database import get_db
|
||||||
from middleware.auth_middleware import get_current_user
|
from middleware.auth_middleware import get_current_user
|
||||||
@@ -16,8 +17,11 @@ from api.story_writer.utils.auth import require_authenticated_user
|
|||||||
from services.llm_providers.main_text_generation import llm_text_gen
|
from services.llm_providers.main_text_generation import llm_text_gen
|
||||||
from services.llm_providers.main_image_generation import generate_image
|
from services.llm_providers.main_image_generation import generate_image
|
||||||
from services.podcast_bible_service import PodcastBibleService
|
from services.podcast_bible_service import PodcastBibleService
|
||||||
|
from services.subscription import PricingService
|
||||||
|
from models.subscription_models import APIProvider
|
||||||
from utils.asset_tracker import save_asset_to_library
|
from utils.asset_tracker import save_asset_to_library
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
import os
|
||||||
from ..constants import PODCAST_IMAGES_DIR
|
from ..constants import PODCAST_IMAGES_DIR
|
||||||
from ..models import (
|
from ..models import (
|
||||||
PodcastAnalyzeRequest,
|
PodcastAnalyzeRequest,
|
||||||
@@ -26,6 +30,87 @@ from ..models import (
|
|||||||
PodcastEnhanceIdeaResponse
|
PodcastEnhanceIdeaResponse
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Check if running in podcast-only demo mode
|
||||||
|
def _is_podcast_only_mode() -> bool:
|
||||||
|
"""Check if podcast-only demo mode is enabled."""
|
||||||
|
return os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||||
|
|
||||||
|
|
||||||
|
def _estimate_tokens(text: str) -> int:
|
||||||
|
if not text:
|
||||||
|
return 0
|
||||||
|
return max(1, len(text) // 4)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_analysis_estimate(
|
||||||
|
db: Session,
|
||||||
|
idea: str,
|
||||||
|
duration: int,
|
||||||
|
speakers: int,
|
||||||
|
has_avatar: bool,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Build a user-facing estimate from pricing catalog and phase-level assumptions.
|
||||||
|
"""
|
||||||
|
# Defaults if catalog lookup fails
|
||||||
|
gemini_in_token = 0.00000015
|
||||||
|
gemini_out_token = 0.0000006
|
||||||
|
exa_per_request = 0.005
|
||||||
|
image_per_request = 0.01
|
||||||
|
video_per_request = 0.01
|
||||||
|
audio_per_request = 0.005
|
||||||
|
|
||||||
|
try:
|
||||||
|
pricing_service = PricingService(db)
|
||||||
|
gemini_pricing = pricing_service.get_pricing_for_provider_model(APIProvider.GEMINI, "gemini-2.5-flash") or {}
|
||||||
|
gemini_in_token = float(gemini_pricing.get("cost_per_input_token") or gemini_in_token)
|
||||||
|
gemini_out_token = float(gemini_pricing.get("cost_per_output_token") or gemini_out_token)
|
||||||
|
exa_pricing = pricing_service.get_pricing_for_provider_model(APIProvider.EXA, "exa-search") or {}
|
||||||
|
exa_per_request = float(exa_pricing.get("cost_per_request") or exa_per_request)
|
||||||
|
img_pricing = pricing_service.get_pricing_for_provider_model(APIProvider.STABILITY, "stable-image-ultra") or {}
|
||||||
|
image_per_request = float(img_pricing.get("cost_per_request") or image_per_request)
|
||||||
|
video_pricing = pricing_service.get_pricing_for_provider_model(APIProvider.VIDEO, "minimax-video-01") or {}
|
||||||
|
video_per_request = float(video_pricing.get("cost_per_request") or video_per_request)
|
||||||
|
audio_pricing = pricing_service.get_pricing_for_provider_model(APIProvider.AUDIO, "gemini-2.5-flash-preview-tts") or {}
|
||||||
|
audio_per_request = float(audio_pricing.get("cost_per_request") or audio_per_request)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"[Podcast Analyze] Pricing catalog lookup failed, using defaults: {exc}")
|
||||||
|
|
||||||
|
# Phase assumptions
|
||||||
|
query_count = 5
|
||||||
|
analyze_in = _estimate_tokens(idea) + 240
|
||||||
|
analyze_out = 750
|
||||||
|
analyze_cost = (analyze_in * gemini_in_token) + (analyze_out * gemini_out_token)
|
||||||
|
|
||||||
|
gather_cost = query_count * exa_per_request
|
||||||
|
|
||||||
|
script_chars = max(1000, duration * 900)
|
||||||
|
write_in = _estimate_tokens(idea) + _estimate_tokens(str(script_chars)) + 320
|
||||||
|
write_out = max(900, int(duration * 220))
|
||||||
|
write_cost = (write_in * gemini_in_token) + (write_out * gemini_out_token)
|
||||||
|
|
||||||
|
tts_cost = max(1, speakers) * audio_per_request
|
||||||
|
avatar_cost = 0.0 if has_avatar else image_per_request
|
||||||
|
video_cost = max(1, duration) * video_per_request
|
||||||
|
produce_cost = tts_cost + avatar_cost + video_cost
|
||||||
|
|
||||||
|
breakdown = [
|
||||||
|
{"phase": "Analyze", "cost": round(analyze_cost, 6)},
|
||||||
|
{"phase": "Gather", "cost": round(gather_cost, 6)},
|
||||||
|
{"phase": "Write", "cost": round(write_cost, 6)},
|
||||||
|
{"phase": "Produce", "cost": round(produce_cost, 6)},
|
||||||
|
]
|
||||||
|
total = round(sum(item["cost"] for item in breakdown), 6)
|
||||||
|
return {
|
||||||
|
"ttsCost": round(tts_cost, 6),
|
||||||
|
"avatarCost": round(avatar_cost, 6),
|
||||||
|
"videoCost": round(video_cost, 6),
|
||||||
|
"researchCost": round(gather_cost, 6),
|
||||||
|
"total": total,
|
||||||
|
"breakdown": breakdown,
|
||||||
|
"currency": "USD",
|
||||||
|
}
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
@@ -41,19 +126,33 @@ async def enhance_podcast_idea(
|
|||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
# Serialize Bible context if provided or generate from onboarding
|
# Serialize Bible context if provided or generate from onboarding
|
||||||
|
# In podcast-only mode, skip bible generation since onboarding is disabled
|
||||||
bible_context = ""
|
bible_context = ""
|
||||||
try:
|
if not _is_podcast_only_mode():
|
||||||
bible_service = PodcastBibleService()
|
logger.warning(f"[Podcast Enhance] Podcast mode=full — attempting Bible generation for user {user_id}")
|
||||||
|
try:
|
||||||
|
bible_service = PodcastBibleService()
|
||||||
|
if request.bible:
|
||||||
|
from models.podcast_bible_models import PodcastBible
|
||||||
|
bible_data = PodcastBible(**request.bible)
|
||||||
|
bible_context = bible_service.serialize_bible(bible_data)
|
||||||
|
else:
|
||||||
|
# Generate from onboarding data directly
|
||||||
|
bible_obj = bible_service.generate_bible(user_id, "temp_enhance")
|
||||||
|
bible_context = bible_service.serialize_bible(bible_obj)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"[Podcast Enhance] Failed to parse or generate bible context: {exc}")
|
||||||
|
else:
|
||||||
|
# In podcast mode, use the provided bible directly if available
|
||||||
|
logger.warning(f"[Podcast Enhance] Podcast mode=podcast_only — skipping Bible generation for user {user_id}")
|
||||||
if request.bible:
|
if request.bible:
|
||||||
from models.podcast_bible_models import PodcastBible
|
try:
|
||||||
bible_data = PodcastBible(**request.bible)
|
from models.podcast_bible_models import PodcastBible
|
||||||
bible_context = bible_service.serialize_bible(bible_data)
|
bible_data = PodcastBible(**request.bible)
|
||||||
else:
|
bible_service = PodcastBibleService()
|
||||||
# Generate from onboarding data directly
|
bible_context = bible_service.serialize_bible(bible_data)
|
||||||
bible_obj = bible_service.generate_bible(user_id, "temp_enhance")
|
except Exception as exc:
|
||||||
bible_context = bible_service.serialize_bible(bible_obj)
|
logger.debug(f"[Podcast Enhance] Bible parsing skipped in podcast mode: {exc}")
|
||||||
except Exception as exc:
|
|
||||||
logger.warning(f"[Podcast Enhance] Failed to parse or generate bible context: {exc}")
|
|
||||||
|
|
||||||
prompt = f"""
|
prompt = f"""
|
||||||
You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea.
|
You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea.
|
||||||
@@ -71,8 +170,22 @@ Generate 3 different enhanced versions, each with a unique angle:
|
|||||||
Each version should be 2-3 sentences, audience-focused, and align with host persona if provided.
|
Each version should be 2-3 sentences, audience-focused, and align with host persona if provided.
|
||||||
|
|
||||||
Return JSON with:
|
Return JSON with:
|
||||||
- enhanced_ideas: array of 3 enhanced episode pitches (in order: Professional, Storytelling, Trendy)
|
- enhanced_ideas: array of 3 strings, each string being a complete episode pitch (NOT objects, just plain strings)
|
||||||
- rationales: array of 3 rationales explaining the approach for each version
|
- rationales: array of 3 strings explaining the approach for each version
|
||||||
|
|
||||||
|
IMPORTANT: enhanced_ideas must be an array of plain strings, NOT objects. Example:
|
||||||
|
{{
|
||||||
|
"enhanced_ideas": [
|
||||||
|
"Your expert guide to AI advancement: A practical look at how AI is transforming industries...",
|
||||||
|
"The human stories behind AI innovation: From Silicon Valley to your daily life...",
|
||||||
|
"AI in 2026: What's trending and what's next in artificial intelligence..."
|
||||||
|
],
|
||||||
|
"rationales": [
|
||||||
|
"Professional approach focusing on expertise and authority",
|
||||||
|
"Storytelling approach emphasizing human connection",
|
||||||
|
"Contemporary approach highlighting current relevance"
|
||||||
|
]
|
||||||
|
}}
|
||||||
"""
|
"""
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -80,7 +193,7 @@ Return JSON with:
|
|||||||
prompt=prompt,
|
prompt=prompt,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
json_struct=None,
|
json_struct=None,
|
||||||
preferred_provider="huggingface",
|
preferred_provider=None,
|
||||||
flow_type="premium_tool",
|
flow_type="premium_tool",
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -94,6 +207,19 @@ Return JSON with:
|
|||||||
enhanced_ideas = data.get("enhanced_ideas", [])
|
enhanced_ideas = data.get("enhanced_ideas", [])
|
||||||
rationales = data.get("rationales", [])
|
rationales = data.get("rationales", [])
|
||||||
|
|
||||||
|
# Handle case where LLM returns objects instead of strings
|
||||||
|
normalized_ideas = []
|
||||||
|
for idea in enhanced_ideas:
|
||||||
|
if isinstance(idea, dict):
|
||||||
|
# Extract title and description from object
|
||||||
|
title = idea.get("title", "")
|
||||||
|
description = idea.get("description", "") or idea.get("content", "")
|
||||||
|
normalized_ideas.append(f"{title}: {description}" if description else title)
|
||||||
|
elif isinstance(idea, str):
|
||||||
|
normalized_ideas.append(idea)
|
||||||
|
|
||||||
|
enhanced_ideas = normalized_ideas
|
||||||
|
|
||||||
# Ensure we have exactly 3 ideas, fallback to original if needed
|
# Ensure we have exactly 3 ideas, fallback to original if needed
|
||||||
if not isinstance(enhanced_ideas, list) or len(enhanced_ideas) != 3:
|
if not isinstance(enhanced_ideas, list) or len(enhanced_ideas) != 3:
|
||||||
# Fallback: create 3 variations of the original idea
|
# Fallback: create 3 variations of the original idea
|
||||||
@@ -121,22 +247,12 @@ Return JSON with:
|
|||||||
enhanced_ideas=enhanced_ideas[:3], # Ensure exactly 3
|
enhanced_ideas=enhanced_ideas[:3], # Ensure exactly 3
|
||||||
rationales=rationales[:3] # Ensure exactly 3
|
rationales=rationales[:3] # Ensure exactly 3
|
||||||
)
|
)
|
||||||
|
except HTTPException:
|
||||||
|
# Re-raise HTTPExceptions (e.g., 429 subscription limit) - preserve error details
|
||||||
|
raise
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error(f"[Podcast Enhance] Failed for user {user_id}: {exc}")
|
logger.error(f"[Podcast Enhance] Failed for user {user_id}: {exc}")
|
||||||
# Fallback to basic variations of original idea
|
raise HTTPException(status_code=500, detail=f"Enhance failed: {exc}")
|
||||||
base_idea = request.idea
|
|
||||||
return PodcastEnhanceIdeaResponse(
|
|
||||||
enhanced_ideas=[
|
|
||||||
f"Expert insights on {base_idea}: A deep dive into industry trends and best practices.",
|
|
||||||
f"The human side of {base_idea}: Personal stories and real-world experiences that resonate.",
|
|
||||||
f"Modern perspectives on {base_idea}: Current trends and forward-thinking approaches."
|
|
||||||
],
|
|
||||||
rationales=[
|
|
||||||
"Professional approach focusing on expertise and authority",
|
|
||||||
"Storytelling approach emphasizing human connection",
|
|
||||||
"Contemporary approach highlighting current relevance"
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/analyze", response_model=PodcastAnalyzeResponse)
|
@router.post("/analyze", response_model=PodcastAnalyzeResponse)
|
||||||
@@ -173,7 +289,11 @@ async def analyze_podcast_idea(
|
|||||||
final_avatar_url = request.avatar_url
|
final_avatar_url = request.avatar_url
|
||||||
final_avatar_prompt = None
|
final_avatar_prompt = None
|
||||||
|
|
||||||
if not final_avatar_url:
|
# Skip avatar generation for audio_only mode
|
||||||
|
podcast_mode = getattr(request, 'podcast_mode', None) or 'video_only'
|
||||||
|
should_generate_avatar = not final_avatar_url and podcast_mode != 'audio_only'
|
||||||
|
|
||||||
|
if should_generate_avatar:
|
||||||
logger.info(f"[Podcast Analyze] No avatar_url provided, generating one for user {user_id}")
|
logger.info(f"[Podcast Analyze] No avatar_url provided, generating one for user {user_id}")
|
||||||
try:
|
try:
|
||||||
# 1. PRE-FLIGHT VALIDATION: Check subscription limits for image generation
|
# 1. PRE-FLIGHT VALIDATION: Check subscription limits for image generation
|
||||||
@@ -197,16 +317,16 @@ async def analyze_podcast_idea(
|
|||||||
image_result = generate_image(
|
image_result = generate_image(
|
||||||
prompt=final_avatar_prompt,
|
prompt=final_avatar_prompt,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
width=1024,
|
options={"width": 1024, "height": 1024}
|
||||||
height=1024
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# 4. Save to disk and library
|
# 4. Save to disk and library
|
||||||
if image_result and image_result.image_bytes:
|
if image_result and image_result.image_bytes:
|
||||||
img_id = str(uuid.uuid4())[:8]
|
img_id = str(uuid.uuid4())[:8]
|
||||||
filename = f"presenter_podcast_{user_id}_{img_id}.png"
|
filename = f"presenter_podcast_{user_id}_{img_id}.png"
|
||||||
output_path = PODCAST_IMAGES_DIR / filename
|
avatars_dir = PODCAST_IMAGES_DIR / "avatars"
|
||||||
PODCAST_IMAGES_DIR.mkdir(parents=True, exist_ok=True)
|
avatars_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
output_path = avatars_dir / filename
|
||||||
|
|
||||||
with open(output_path, "wb") as f:
|
with open(output_path, "wb") as f:
|
||||||
f.write(image_result.image_bytes)
|
f.write(image_result.image_bytes)
|
||||||
@@ -218,13 +338,14 @@ async def analyze_podcast_idea(
|
|||||||
db=db,
|
db=db,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
asset_type="image",
|
asset_type="image",
|
||||||
file_url=final_avatar_url,
|
source_module="podcast_analysis",
|
||||||
filename=filename,
|
filename=filename,
|
||||||
|
file_url=final_avatar_url,
|
||||||
title=f"Presenter Avatar - {request.idea[:40]}",
|
title=f"Presenter Avatar - {request.idea[:40]}",
|
||||||
description=f"AI-generated podcast presenter for: {request.idea}",
|
description=f"AI-generated podcast presenter for: {request.idea}",
|
||||||
provider=image_result.provider,
|
provider=image_result.provider,
|
||||||
model=image_result.model,
|
model=image_result.model,
|
||||||
cost=image_result.cost
|
cost=0.0 # Cost tracked in generate_image
|
||||||
)
|
)
|
||||||
logger.info(f"[Podcast Analyze] ✅ Generated and saved avatar to {final_avatar_url}")
|
logger.info(f"[Podcast Analyze] ✅ Generated and saved avatar to {final_avatar_url}")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -269,6 +390,10 @@ Return JSON with:
|
|||||||
- top_keywords: 5 podcast-relevant keywords/phrases
|
- top_keywords: 5 podcast-relevant keywords/phrases
|
||||||
- suggested_outlines: 2 items, each with title (<=60 chars) and 4-6 short segments (bullet-friendly, factual)
|
- suggested_outlines: 2 items, each with title (<=60 chars) and 4-6 short segments (bullet-friendly, factual)
|
||||||
- title_suggestions: 3 concise episode titles
|
- title_suggestions: 3 concise episode titles
|
||||||
|
- episode_hook: one compelling 15-30 second opening hook/angle that grabs attention
|
||||||
|
- key_takeaways: 3-5 actionable insights listeners will learn
|
||||||
|
- guest_talking_points: (if guest included) 3-4 suggested questions/angles for guest interview
|
||||||
|
- listener_cta: one clear call-to-action for listeners
|
||||||
- research_queries: array of {{"query": "string", "rationale": "string"}}
|
- research_queries: array of {{"query": "string", "rationale": "string"}}
|
||||||
- exa_suggested_config: suggested Exa search options with:
|
- exa_suggested_config: suggested Exa search options with:
|
||||||
- exa_search_type: "auto" | "neural" | "keyword"
|
- exa_search_type: "auto" | "neural" | "keyword"
|
||||||
@@ -282,7 +407,10 @@ Return JSON with:
|
|||||||
Requirements:
|
Requirements:
|
||||||
- Keep language factual, actionable, and suited for spoken audio.
|
- Keep language factual, actionable, and suited for spoken audio.
|
||||||
- Avoid narrative fiction tone.
|
- Avoid narrative fiction tone.
|
||||||
- Prefer 2024-2025 context.
|
- For research queries: Mix of time-sensitive and evergreen queries:
|
||||||
|
- 2-3 queries should focus on latest 2025-2026 developments, trends, and data (use year in query)
|
||||||
|
- 2-3 queries should be evergreen/fundamental (concepts, definitions, best practices, proven strategies) - do NOT include years in these
|
||||||
|
- Today's date is April 2026.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -290,7 +418,7 @@ Requirements:
|
|||||||
prompt=prompt,
|
prompt=prompt,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
json_struct=None,
|
json_struct=None,
|
||||||
preferred_provider="huggingface",
|
preferred_provider=None,
|
||||||
flow_type="premium_tool",
|
flow_type="premium_tool",
|
||||||
)
|
)
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
@@ -316,6 +444,10 @@ Requirements:
|
|||||||
top_keywords = data.get("top_keywords") or []
|
top_keywords = data.get("top_keywords") or []
|
||||||
suggested_outlines = data.get("suggested_outlines") or []
|
suggested_outlines = data.get("suggested_outlines") or []
|
||||||
title_suggestions = data.get("title_suggestions") or []
|
title_suggestions = data.get("title_suggestions") or []
|
||||||
|
episode_hook = data.get("episode_hook") or ""
|
||||||
|
key_takeaways = data.get("key_takeaways") or []
|
||||||
|
guest_talking_points = data.get("guest_talking_points") or []
|
||||||
|
listener_cta = data.get("listener_cta") or ""
|
||||||
research_queries = data.get("research_queries") or []
|
research_queries = data.get("research_queries") or []
|
||||||
exa_suggested_config = data.get("exa_suggested_config") or None
|
exa_suggested_config = data.get("exa_suggested_config") or None
|
||||||
|
|
||||||
@@ -325,10 +457,123 @@ Requirements:
|
|||||||
top_keywords=top_keywords,
|
top_keywords=top_keywords,
|
||||||
suggested_outlines=suggested_outlines,
|
suggested_outlines=suggested_outlines,
|
||||||
title_suggestions=title_suggestions,
|
title_suggestions=title_suggestions,
|
||||||
|
episode_hook=episode_hook,
|
||||||
|
key_takeaways=key_takeaways,
|
||||||
|
guest_talking_points=guest_talking_points,
|
||||||
|
listener_cta=listener_cta,
|
||||||
research_queries=research_queries,
|
research_queries=research_queries,
|
||||||
exa_suggested_config=exa_suggested_config,
|
exa_suggested_config=exa_suggested_config,
|
||||||
bible=bible_obj.model_dump() if bible_obj else None,
|
bible=bible_obj.model_dump() if bible_obj else None,
|
||||||
avatar_url=final_avatar_url,
|
avatar_url=final_avatar_url,
|
||||||
avatar_prompt=final_avatar_prompt,
|
avatar_prompt=final_avatar_prompt,
|
||||||
|
estimate=_build_analysis_estimate(
|
||||||
|
db=db,
|
||||||
|
idea=request.idea,
|
||||||
|
duration=request.duration,
|
||||||
|
speakers=request.speakers,
|
||||||
|
has_avatar=bool(final_avatar_url),
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class RegenerateQueriesRequest(BaseModel):
|
||||||
|
idea: str
|
||||||
|
feedback: str
|
||||||
|
existing_analysis: Optional[Dict[str, Any]] = None
|
||||||
|
bible: Optional[Dict[str, Any]] = None
|
||||||
|
|
||||||
|
|
||||||
|
class RegenerateQueriesResponse(BaseModel):
|
||||||
|
research_queries: List[Dict[str, str]]
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/regenerate-queries", response_model=RegenerateQueriesResponse)
|
||||||
|
async def regenerate_research_queries(
|
||||||
|
request: RegenerateQueriesRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Regenerate research queries based on user feedback and existing analysis.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
# Build context from existing analysis
|
||||||
|
idea = request.idea
|
||||||
|
feedback = request.feedback
|
||||||
|
|
||||||
|
# Get topic, keywords, audience from existing analysis if provided
|
||||||
|
topic = idea
|
||||||
|
keywords = ""
|
||||||
|
audience = ""
|
||||||
|
if request.existing_analysis:
|
||||||
|
topic = request.existing_analysis.get("title_suggestions", [idea])[0] if request.existing_analysis.get("title_suggestions") else idea
|
||||||
|
keywords = ", ".join(request.existing_analysis.get("top_keywords", [])[:5])
|
||||||
|
audience = request.existing_analysis.get("audience", "")
|
||||||
|
|
||||||
|
# Serialize Bible context if provided
|
||||||
|
bible_context = ""
|
||||||
|
if request.bible:
|
||||||
|
try:
|
||||||
|
bible_service = PodcastBibleService()
|
||||||
|
from models.podcast_bible_models import PodcastBible
|
||||||
|
bible_data = PodcastBible(**request.bible)
|
||||||
|
bible_context = bible_service.serialize_bible(bible_data)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Failed to serialize bible for query regeneration: {e}")
|
||||||
|
|
||||||
|
prompt = f"""
|
||||||
|
You are a research strategist for podcast content. Given a podcast idea, existing analysis, and user feedback,
|
||||||
|
generate 7 new research queries that address the user's specific needs.
|
||||||
|
|
||||||
|
{f"USER FEEDBACK: {feedback}" if feedback else ""}
|
||||||
|
|
||||||
|
{f"EXISTING ANALYSIS CONTEXT:\n- Topic: {topic}\n- Keywords: {keywords}\n- Audience: {audience}\n" if request.existing_analysis else ""}
|
||||||
|
{f"PODCAST BIBLE CONTEXT:\n{bible_context}\n" if bible_context else ""}
|
||||||
|
|
||||||
|
Podcast Idea: "{idea}"
|
||||||
|
|
||||||
|
TASK:
|
||||||
|
Generate exactly 7 research queries that:
|
||||||
|
1. Incorporate the user's feedback direction
|
||||||
|
2. Build on the existing analysis context
|
||||||
|
3. Mix of time-sensitive (2025-2026) and evergreen topics
|
||||||
|
4. Are highly specific to the podcast topic
|
||||||
|
|
||||||
|
Return JSON with:
|
||||||
|
- research_queries: array of {{"query": "string", "rationale": "string"}}
|
||||||
|
|
||||||
|
Requirements:
|
||||||
|
- At least 2-3 queries should focus on latest 2025-2026 developments (include year in query)
|
||||||
|
- At least 2-3 queries should be evergreen (concepts, definitions, best practices - NO year)
|
||||||
|
- Queries should be specific and actionable, not generic
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
from services.llm_providers.main_text_generation import llm_text_gen
|
||||||
|
|
||||||
|
raw = llm_text_gen(
|
||||||
|
prompt=prompt,
|
||||||
|
user_id=user_id,
|
||||||
|
json_struct={"research_queries": [{"query": "string", "rationale": "string"}]},
|
||||||
|
preferred_provider=None,
|
||||||
|
flow_type="premium_tool",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Parse response
|
||||||
|
if isinstance(raw, dict):
|
||||||
|
queries = raw.get("research_queries", [])
|
||||||
|
else:
|
||||||
|
# Try to parse as JSON
|
||||||
|
try:
|
||||||
|
parsed = json.loads(raw) if isinstance(raw, str) else raw
|
||||||
|
queries = parsed.get("research_queries", []) if isinstance(parsed, dict) else []
|
||||||
|
except:
|
||||||
|
queries = []
|
||||||
|
|
||||||
|
return RegenerateQueriesResponse(research_queries=queries[:7])
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error(f"[Regenerate Queries] Failed for user {user_id}: {exc}")
|
||||||
|
raise HTTPException(status_code=500, detail=f"Regenerate queries failed: {exc}")
|
||||||
|
|||||||
@@ -126,12 +126,14 @@ async def generate_podcast_audio(
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
audio_service = get_podcast_audio_service(user_id)
|
audio_service = get_podcast_audio_service(user_id)
|
||||||
|
logger.warning(f"[Podcast] Generating audio with service dir: {audio_service.output_dir}")
|
||||||
result: StoryAudioResult = audio_service.generate_ai_audio(
|
result: StoryAudioResult = audio_service.generate_ai_audio(
|
||||||
scene_number=0,
|
scene_number=0,
|
||||||
scene_title=request.scene_title,
|
scene_title=request.scene_title,
|
||||||
text=request.text.strip(),
|
text=request.text.strip(),
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
voice_id=request.voice_id or "Wise_Woman",
|
voice_id=request.voice_id or "Wise_Woman",
|
||||||
|
custom_voice_id=request.custom_voice_id,
|
||||||
speed=request.speed or 1.0, # Normal speed (was 0.9, but too slow - causing duration issues)
|
speed=request.speed or 1.0, # Normal speed (was 0.9, but too slow - causing duration issues)
|
||||||
volume=request.volume or 1.0,
|
volume=request.volume or 1.0,
|
||||||
pitch=request.pitch or 0.0, # Normal pitch (0.0 = neutral)
|
pitch=request.pitch or 0.0, # Normal pitch (0.0 = neutral)
|
||||||
@@ -149,6 +151,8 @@ async def generate_podcast_audio(
|
|||||||
if result.get("audio_url") and "/api/story/audio/" in result.get("audio_url", ""):
|
if result.get("audio_url") and "/api/story/audio/" in result.get("audio_url", ""):
|
||||||
audio_filename = result.get("audio_filename", "")
|
audio_filename = result.get("audio_filename", "")
|
||||||
result["audio_url"] = f"/api/podcast/audio/{audio_filename}"
|
result["audio_url"] = f"/api/podcast/audio/{audio_filename}"
|
||||||
|
|
||||||
|
logger.warning(f"[Podcast] Audio generated - path: {result.get('audio_path')}, url: {result.get('audio_url')}")
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
raise HTTPException(status_code=500, detail=f"Audio generation failed: {exc}")
|
raise HTTPException(status_code=500, detail=f"Audio generation failed: {exc}")
|
||||||
|
|
||||||
@@ -387,7 +391,9 @@ async def serve_podcast_audio(
|
|||||||
raise HTTPException(status_code=400, detail="Invalid filename")
|
raise HTTPException(status_code=400, detail="Invalid filename")
|
||||||
|
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
logger.debug(f"[Podcast] serve_podcast_audio called: user_id={user_id}, filename={filename}")
|
||||||
audio_path = _resolve_podcast_media_file(filename, "audio", user_id)
|
audio_path = _resolve_podcast_media_file(filename, "audio", user_id)
|
||||||
|
logger.debug(f"[Podcast] Resolved audio path: {audio_path}")
|
||||||
|
|
||||||
return FileResponse(audio_path, media_type="audio/mpeg")
|
return FileResponse(audio_path, media_type="audio/mpeg")
|
||||||
|
|
||||||
|
|||||||
@@ -114,12 +114,18 @@ async def make_avatar_presentable(
|
|||||||
Transform an uploaded avatar image into a podcast-appropriate presenter.
|
Transform an uploaded avatar image into a podcast-appropriate presenter.
|
||||||
Uses AI image editing to convert the uploaded photo into a professional podcast presenter.
|
Uses AI image editing to convert the uploaded photo into a professional podcast presenter.
|
||||||
"""
|
"""
|
||||||
|
# CRITICAL: Log at the very start before any logic
|
||||||
|
logger.info(f"[Podcast] ===== MAKE PRESENTABLE ENDPOINT START =====")
|
||||||
|
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
logger.info(f"[Podcast] Make presentable request received - user_id={user_id}, avatar_url={avatar_url}, project_id={project_id}")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Load the uploaded avatar image
|
# Load the uploaded avatar image
|
||||||
from ..utils import load_podcast_image_bytes
|
from ..utils import load_podcast_image_bytes
|
||||||
|
logger.info(f"[Podcast] Loading avatar image from {avatar_url}")
|
||||||
avatar_bytes = load_podcast_image_bytes(avatar_url)
|
avatar_bytes = load_podcast_image_bytes(avatar_url)
|
||||||
|
logger.info(f"[Podcast] Avatar loaded successfully - size={len(avatar_bytes)} bytes")
|
||||||
|
|
||||||
logger.info(f"[Podcast] Transforming avatar to podcast presenter for project {project_id}")
|
logger.info(f"[Podcast] Transforming avatar to podcast presenter for project {project_id}")
|
||||||
|
|
||||||
@@ -141,12 +147,18 @@ async def make_avatar_presentable(
|
|||||||
"model": None, # Use default model
|
"model": None, # Use default model
|
||||||
}
|
}
|
||||||
|
|
||||||
result = edit_image(
|
logger.info(f"[Podcast] Calling edit_image with user_id={user_id}")
|
||||||
input_image_bytes=avatar_bytes,
|
try:
|
||||||
prompt=transformation_prompt,
|
result = edit_image(
|
||||||
options=image_options,
|
input_image_bytes=avatar_bytes,
|
||||||
user_id=user_id
|
prompt=transformation_prompt,
|
||||||
)
|
options=image_options,
|
||||||
|
user_id=user_id
|
||||||
|
)
|
||||||
|
logger.info(f"[Podcast] edit_image completed successfully - provider={result.provider}, model={result.model}")
|
||||||
|
except Exception as edit_err:
|
||||||
|
logger.error(f"[Podcast] edit_image failed: {edit_err}", exc_info=True)
|
||||||
|
raise HTTPException(status_code=500, detail=f"Image editing failed: {str(edit_err)}")
|
||||||
|
|
||||||
# Save transformed avatar
|
# Save transformed avatar
|
||||||
unique_id = str(uuid.uuid4())[:8]
|
unique_id = str(uuid.uuid4())[:8]
|
||||||
@@ -194,6 +206,16 @@ async def make_avatar_presentable(
|
|||||||
"avatar_filename": transformed_filename,
|
"avatar_filename": transformed_filename,
|
||||||
"message": "Avatar transformed into podcast presenter successfully"
|
"message": "Avatar transformed into podcast presenter successfully"
|
||||||
}
|
}
|
||||||
|
except HTTPException:
|
||||||
|
# Re-raise HTTP exceptions as-is
|
||||||
|
raise
|
||||||
|
except RuntimeError as rt_err:
|
||||||
|
# Handle missing API keys or configuration errors
|
||||||
|
logger.error(f"[Podcast] Avatar transformation configuration error: {rt_err}")
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=503, # Service Unavailable
|
||||||
|
detail=f"Image editing service not configured: {str(rt_err)}. Please contact support."
|
||||||
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error(f"[Podcast] Avatar transformation failed: {exc}", exc_info=True)
|
logger.error(f"[Podcast] Avatar transformation failed: {exc}", exc_info=True)
|
||||||
raise HTTPException(status_code=500, detail=f"Avatar transformation failed: {str(exc)}")
|
raise HTTPException(status_code=500, detail=f"Avatar transformation failed: {str(exc)}")
|
||||||
|
|||||||
241
backend/api/podcast/handlers/broll.py
Normal file
241
backend/api/podcast/handlers/broll.py
Normal file
@@ -0,0 +1,241 @@
|
|||||||
|
"""
|
||||||
|
B-Roll Handlers
|
||||||
|
|
||||||
|
API endpoints for B-roll chart preview and video generation.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, BackgroundTasks
|
||||||
|
from fastapi.responses import FileResponse
|
||||||
|
from typing import Dict, Any, Optional, List
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
from middleware.auth_middleware import get_current_user
|
||||||
|
from api.story_writer.utils.auth import require_authenticated_user
|
||||||
|
from services.podcast.broll_service import get_broll_service
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
class ChartPreviewRequest(BaseModel):
|
||||||
|
"""Request model for chart preview generation."""
|
||||||
|
chart_data: Dict[str, Any] = Field(..., description="Chart data (labels, before/after, etc.)")
|
||||||
|
chart_type: str = Field(
|
||||||
|
default="bar_comparison",
|
||||||
|
description="bar_comparison | bar_horizontal | line_trend | pie | stacked_bar | bullet"
|
||||||
|
)
|
||||||
|
title: str = Field(default="", description="Chart title")
|
||||||
|
subtitle: Optional[str] = Field(default="", description="Optional subtitle at bottom")
|
||||||
|
|
||||||
|
|
||||||
|
class ChartPreviewResponse(BaseModel):
|
||||||
|
"""Response for chart preview."""
|
||||||
|
preview_url: str
|
||||||
|
chart_id: str
|
||||||
|
|
||||||
|
|
||||||
|
class BrollSceneRequest(BaseModel):
|
||||||
|
"""Request for generating B-roll video for a scene."""
|
||||||
|
scene_id: str
|
||||||
|
key_insight: str
|
||||||
|
supporting_stat: str
|
||||||
|
chart_data: Optional[Dict[str, Any]] = None
|
||||||
|
visual_cue: str = Field(default="bar_chart_comparison", description="bar_chart_comparison | bullet_points")
|
||||||
|
duration: float = Field(default=10.0, ge=3.0, le=60.0)
|
||||||
|
background_image_url: str
|
||||||
|
avatar_video_url: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class BrollSceneResponse(BaseModel):
|
||||||
|
"""Response for B-roll scene generation."""
|
||||||
|
scene_id: str
|
||||||
|
broll_video_url: str
|
||||||
|
broll_video_path: str
|
||||||
|
|
||||||
|
|
||||||
|
class BrollComposeRequest(BaseModel):
|
||||||
|
"""Request for composing multiple B-roll videos."""
|
||||||
|
scene_video_paths: List[str]
|
||||||
|
output_filename: str = "final_broll.mp4"
|
||||||
|
fade_dur: float = Field(default=0.5, ge=0.0, le=2.0)
|
||||||
|
fps: int = Field(default=24, ge=12, le=60)
|
||||||
|
|
||||||
|
|
||||||
|
class BrollComposeResponse(BaseModel):
|
||||||
|
"""Response for B-roll composition."""
|
||||||
|
final_video_url: str
|
||||||
|
final_video_path: str
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/preview/chart", response_model=ChartPreviewResponse)
|
||||||
|
async def generate_chart_preview(
|
||||||
|
request: ChartPreviewRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Generate a chart PNG preview (static image for Write phase).
|
||||||
|
|
||||||
|
This endpoint is called from the Write phase to show users chart previews
|
||||||
|
before they commit to B-roll video generation.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
try:
|
||||||
|
broll_service = get_broll_service()
|
||||||
|
|
||||||
|
preview_path = broll_service.generate_chart_preview(
|
||||||
|
chart_data=request.chart_data,
|
||||||
|
chart_type=request.chart_type,
|
||||||
|
title=request.title,
|
||||||
|
subtitle=request.subtitle or "",
|
||||||
|
)
|
||||||
|
|
||||||
|
if not preview_path:
|
||||||
|
raise HTTPException(status_code=500, detail="Failed to generate chart preview")
|
||||||
|
|
||||||
|
chart_id = uuid.uuid4().hex[:8]
|
||||||
|
preview_url = f"/api/podcast/broll/preview/{chart_id}/{preview_path.split('/')[-1]}"
|
||||||
|
|
||||||
|
return ChartPreviewResponse(
|
||||||
|
preview_url=preview_url,
|
||||||
|
chart_id=chart_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"[Broll] Chart preview generation failed: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=f"Chart preview failed: {str(e)}")
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/render/broll-scene", response_model=BrollSceneResponse)
|
||||||
|
async def generate_broll_scene(
|
||||||
|
request: BrollSceneRequest,
|
||||||
|
background_tasks: BackgroundTasks,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Generate a B-roll video for a single scene.
|
||||||
|
|
||||||
|
This creates a programmatic video with:
|
||||||
|
- Background image with Ken Burns effect
|
||||||
|
- Chart overlay (if chart_data provided)
|
||||||
|
- Avatar circle in corner (if avatar_video_url provided)
|
||||||
|
- Insight card at bottom
|
||||||
|
|
||||||
|
Returns a task_id for polling since video generation can take time.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Validate visual_cue
|
||||||
|
valid_cues = ["bar_chart_comparison", "bullet_points", "full_avatar"]
|
||||||
|
if request.visual_cue not in valid_cues:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=400,
|
||||||
|
detail=f"Invalid visual_cue. Must be one of: {valid_cues}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# For now, return a placeholder - full video generation requires
|
||||||
|
# resolving image/video URLs to actual file paths
|
||||||
|
# In V2, this will integrate with the actual video generation
|
||||||
|
|
||||||
|
logger.info(f"[Broll] B-roll scene request for scene: {request.scene_id}")
|
||||||
|
|
||||||
|
return BrollSceneResponse(
|
||||||
|
scene_id=request.scene_id,
|
||||||
|
broll_video_url="",
|
||||||
|
broll_video_path="",
|
||||||
|
)
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"[Broll] B-roll scene generation failed: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=f"B-roll generation failed: {str(e)}")
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/render/broll-compose", response_model=BrollComposeResponse)
|
||||||
|
async def compose_broll_videos(
|
||||||
|
request: BrollComposeRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Compose multiple B-roll scene videos into a final video.
|
||||||
|
|
||||||
|
Applies crossfade transitions between scenes.
|
||||||
|
"""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
try:
|
||||||
|
broll_service = get_broll_service()
|
||||||
|
|
||||||
|
final_path = broll_service.compose_final_video(
|
||||||
|
video_paths=request.scene_video_paths,
|
||||||
|
output_filename=request.output_filename,
|
||||||
|
fade_dur=request.fade_dur,
|
||||||
|
fps=request.fps,
|
||||||
|
)
|
||||||
|
|
||||||
|
final_filename = final_path.split('/')[-1]
|
||||||
|
final_url = f"/api/podcast/broll/final/{final_filename}"
|
||||||
|
|
||||||
|
return BrollComposeResponse(
|
||||||
|
final_video_url=final_url,
|
||||||
|
final_video_path=final_path,
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"[Broll] Video composition failed: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=f"Video composition failed: {str(e)}")
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/preview/{chart_id}/{filename}")
|
||||||
|
async def serve_chart_preview(
|
||||||
|
chart_id: str,
|
||||||
|
filename: str,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""Serve chart preview PNG files."""
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
broll_service = get_broll_service()
|
||||||
|
file_path = broll_service.output_dir / f"chart_preview_{chart_id}.png"
|
||||||
|
|
||||||
|
if not file_path.exists():
|
||||||
|
raise HTTPException(status_code=404, detail="Chart preview not found")
|
||||||
|
|
||||||
|
return FileResponse(
|
||||||
|
path=str(file_path),
|
||||||
|
media_type="image/png",
|
||||||
|
filename=filename,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/final/{filename}")
|
||||||
|
async def serve_final_broll(
|
||||||
|
filename: str,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
):
|
||||||
|
"""Serve final composed B-roll video files."""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
|
broll_service = get_broll_service()
|
||||||
|
file_path = broll_service.output_dir / filename
|
||||||
|
|
||||||
|
if not file_path.exists():
|
||||||
|
raise HTTPException(status_code=404, detail="Video not found")
|
||||||
|
|
||||||
|
return FileResponse(
|
||||||
|
path=str(file_path),
|
||||||
|
media_type="video/mp4",
|
||||||
|
filename=filename,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/health")
|
||||||
|
async def broll_health():
|
||||||
|
"""Health check for B-roll service."""
|
||||||
|
return {"status": "ok", "service": "broll"}
|
||||||
@@ -29,16 +29,45 @@ from ..models import (
|
|||||||
VoiceCloneResult,
|
VoiceCloneResult,
|
||||||
)
|
)
|
||||||
from services.dubbing import AudioDubbingService
|
from services.dubbing import AudioDubbingService
|
||||||
|
from ..constants import get_podcast_media_read_dirs, get_podcast_media_dir
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
_dubbing_executor = ThreadPoolExecutor(max_workers=4, thread_name_prefix="podcast_dubbing")
|
_dubbing_executor = ThreadPoolExecutor(max_workers=4, thread_name_prefix="podcast_dubbing")
|
||||||
|
|
||||||
DUBBED_AUDIO_DIR = Path(__file__).resolve().parents[3] / "data" / "media" / "dubbed_audio"
|
_DUBBED_AUDIO_SUBDIR = Path("dubbed_audio")
|
||||||
|
_LEGACY_DUBBED_AUDIO_DIR = Path(__file__).resolve().parents[3] / "data" / "media" / "dubbed_audio"
|
||||||
|
|
||||||
|
|
||||||
def _ensure_dubbed_audio_dir():
|
def _get_dubbed_audio_dir(user_id: str, *, ensure_exists: bool = False) -> Path:
|
||||||
DUBBED_AUDIO_DIR.mkdir(parents=True, exist_ok=True)
|
"""Resolve tenant-scoped dubbed audio directory under podcast audio media."""
|
||||||
|
base_dir = get_podcast_media_dir("audio", user_id, ensure_exists=ensure_exists)
|
||||||
|
dubbed_dir = (base_dir / _DUBBED_AUDIO_SUBDIR).resolve()
|
||||||
|
if ensure_exists:
|
||||||
|
dubbed_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
return dubbed_dir
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_dubbed_audio_file(filename: str, user_id: str) -> Path:
|
||||||
|
"""Resolve dubbed audio with traversal-safe checks (tenant first, then legacy fallback)."""
|
||||||
|
clean_filename = filename.split("?", 1)[0].strip()
|
||||||
|
if not clean_filename:
|
||||||
|
raise HTTPException(status_code=400, detail="Invalid filename")
|
||||||
|
|
||||||
|
candidate_dirs: list[Path] = []
|
||||||
|
for base_dir in get_podcast_media_read_dirs("audio", user_id):
|
||||||
|
candidate_dirs.append((base_dir / _DUBBED_AUDIO_SUBDIR).resolve())
|
||||||
|
candidate_dirs.append(_LEGACY_DUBBED_AUDIO_DIR.resolve())
|
||||||
|
|
||||||
|
for target_dir in candidate_dirs:
|
||||||
|
candidate = (target_dir / clean_filename).resolve()
|
||||||
|
if not str(candidate).startswith(str(target_dir)):
|
||||||
|
logger.error(f"[Podcast][Dubbing] Attempted path traversal: {filename}")
|
||||||
|
raise HTTPException(status_code=403, detail="Invalid audio path")
|
||||||
|
if candidate.exists():
|
||||||
|
return candidate
|
||||||
|
|
||||||
|
raise HTTPException(status_code=404, detail="Audio file not found")
|
||||||
|
|
||||||
|
|
||||||
def _execute_dubbing_task(
|
def _execute_dubbing_task(
|
||||||
@@ -62,9 +91,8 @@ def _execute_dubbing_task(
|
|||||||
message="Starting audio dubbing..."
|
message="Starting audio dubbing..."
|
||||||
)
|
)
|
||||||
|
|
||||||
_ensure_dubbed_audio_dir()
|
dubbed_audio_dir = _get_dubbed_audio_dir(user_id, ensure_exists=True)
|
||||||
|
service = AudioDubbingService(output_dir=dubbed_audio_dir)
|
||||||
service = AudioDubbingService(output_dir=DUBBED_AUDIO_DIR)
|
|
||||||
|
|
||||||
def progress_callback(progress: float, message: str):
|
def progress_callback(progress: float, message: str):
|
||||||
task_manager.update_task_status(
|
task_manager.update_task_status(
|
||||||
@@ -136,9 +164,8 @@ def _execute_voice_clone_task(
|
|||||||
message="Starting voice cloning..."
|
message="Starting voice cloning..."
|
||||||
)
|
)
|
||||||
|
|
||||||
_ensure_dubbed_audio_dir()
|
dubbed_audio_dir = _get_dubbed_audio_dir(user_id, ensure_exists=True)
|
||||||
|
service = AudioDubbingService(output_dir=dubbed_audio_dir)
|
||||||
service = AudioDubbingService(output_dir=DUBBED_AUDIO_DIR)
|
|
||||||
|
|
||||||
task_manager.update_task_status(
|
task_manager.update_task_status(
|
||||||
task_id, "processing", progress=30.0,
|
task_id, "processing", progress=30.0,
|
||||||
@@ -203,7 +230,10 @@ async def create_audio_dubbing_task(
|
|||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
task_id = task_manager.create_task("audio_dubbing")
|
task_id = task_manager.create_task(
|
||||||
|
"audio_dubbing",
|
||||||
|
metadata={"owner_user_id": user_id},
|
||||||
|
)
|
||||||
|
|
||||||
background_tasks.add_task(
|
background_tasks.add_task(
|
||||||
_execute_dubbing_task,
|
_execute_dubbing_task,
|
||||||
@@ -240,7 +270,7 @@ async def get_dubbing_result(
|
|||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
task_status = task_manager.get_task_status(task_id)
|
task_status = task_manager.get_task_status(task_id, requester_user_id=user_id)
|
||||||
|
|
||||||
if not task_status:
|
if not task_status:
|
||||||
raise HTTPException(status_code=404, detail="Task not found")
|
raise HTTPException(status_code=404, detail="Task not found")
|
||||||
@@ -301,12 +331,7 @@ async def serve_dubbed_audio(
|
|||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
_ensure_dubbed_audio_dir()
|
audio_path = _resolve_dubbed_audio_file(filename, user_id)
|
||||||
|
|
||||||
audio_path = DUBBED_AUDIO_DIR / filename
|
|
||||||
|
|
||||||
if not audio_path.exists():
|
|
||||||
raise HTTPException(status_code=404, detail="Audio file not found")
|
|
||||||
|
|
||||||
return FileResponse(
|
return FileResponse(
|
||||||
path=audio_path,
|
path=audio_path,
|
||||||
@@ -327,7 +352,8 @@ async def estimate_dubbing_cost(
|
|||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
service = AudioDubbingService(output_dir=DUBBED_AUDIO_DIR)
|
dubbed_audio_dir = _get_dubbed_audio_dir(user_id, ensure_exists=True)
|
||||||
|
service = AudioDubbingService(output_dir=dubbed_audio_dir)
|
||||||
|
|
||||||
cost_estimate = service.estimate_cost(
|
cost_estimate = service.estimate_cost(
|
||||||
audio_duration_seconds=request.audio_duration_seconds,
|
audio_duration_seconds=request.audio_duration_seconds,
|
||||||
@@ -403,7 +429,10 @@ async def create_voice_clone_task(
|
|||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
task_id = task_manager.create_task("voice_clone")
|
task_id = task_manager.create_task(
|
||||||
|
"voice_clone",
|
||||||
|
metadata={"owner_user_id": user_id},
|
||||||
|
)
|
||||||
|
|
||||||
background_tasks.add_task(
|
background_tasks.add_task(
|
||||||
_execute_voice_clone_task,
|
_execute_voice_clone_task,
|
||||||
@@ -434,7 +463,7 @@ async def get_voice_clone_result(
|
|||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
task_status = task_manager.get_task_status(task_id)
|
task_status = task_manager.get_task_status(task_id, requester_user_id=user_id)
|
||||||
|
|
||||||
if not task_status:
|
if not task_status:
|
||||||
raise HTTPException(status_code=404, detail="Task not found")
|
raise HTTPException(status_code=404, detail="Task not found")
|
||||||
@@ -479,12 +508,12 @@ async def serve_voice_audio(
|
|||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
|
||||||
_ensure_dubbed_audio_dir()
|
try:
|
||||||
|
audio_path = _resolve_dubbed_audio_file(filename, user_id)
|
||||||
audio_path = DUBBED_AUDIO_DIR / filename
|
except HTTPException as exc:
|
||||||
|
if exc.status_code == 404:
|
||||||
if not audio_path.exists():
|
raise HTTPException(status_code=404, detail="Voice audio file not found") from exc
|
||||||
raise HTTPException(status_code=404, detail="Voice audio file not found")
|
raise
|
||||||
|
|
||||||
return FileResponse(
|
return FileResponse(
|
||||||
path=audio_path,
|
path=audio_path,
|
||||||
|
|||||||
@@ -104,6 +104,16 @@ async def generate_podcast_scene_image(
|
|||||||
# Otherwise, generate from scratch with podcast-optimized prompt
|
# Otherwise, generate from scratch with podcast-optimized prompt
|
||||||
image_prompt = "" # Initialize prompt variable
|
image_prompt = "" # Initialize prompt variable
|
||||||
|
|
||||||
|
# Emotion to lighting mapping for visual tone
|
||||||
|
emotion_lighting = {
|
||||||
|
"happy": "warm, bright lighting, cheerful atmosphere",
|
||||||
|
"excited": "dynamic, energetic lighting with highlights",
|
||||||
|
"serious": "professional, balanced lighting, authoritative feel",
|
||||||
|
"curious": "soft, inviting lighting, thoughtful atmosphere",
|
||||||
|
"confident": "strong, dramatic lighting, authoritative look",
|
||||||
|
"neutral": "professional, balanced lighting"
|
||||||
|
}
|
||||||
|
|
||||||
if base_avatar_bytes:
|
if base_avatar_bytes:
|
||||||
# Use Ideogram Character API for consistent character generation
|
# Use Ideogram Character API for consistent character generation
|
||||||
# Use custom prompt if provided, otherwise build scene-specific prompt
|
# Use custom prompt if provided, otherwise build scene-specific prompt
|
||||||
@@ -127,6 +137,28 @@ async def generate_podcast_scene_image(
|
|||||||
if bible_obj.host.look:
|
if bible_obj.host.look:
|
||||||
prompt_parts.append(f"Host Look: {bible_obj.host.look}")
|
prompt_parts.append(f"Host Look: {bible_obj.host.look}")
|
||||||
|
|
||||||
|
# Scene emotion for visual tone
|
||||||
|
emotion_lighting = {
|
||||||
|
"happy": "warm, bright lighting, cheerful atmosphere",
|
||||||
|
"excited": "dynamic, energetic lighting with highlights",
|
||||||
|
"serious": "professional, balanced lighting, authoritative feel",
|
||||||
|
"curious": "soft, inviting lighting, thoughtful atmosphere",
|
||||||
|
"confident": "strong, dramatic lighting, authoritative look",
|
||||||
|
"neutral": "professional, balanced lighting"
|
||||||
|
}
|
||||||
|
scene_emotion = request.scene_emotion
|
||||||
|
if scene_emotion and scene_emotion in emotion_lighting:
|
||||||
|
prompt_parts.append(emotion_lighting[scene_emotion])
|
||||||
|
|
||||||
|
# AI Analysis context for visual relevance
|
||||||
|
if request.analysis:
|
||||||
|
keywords = request.analysis.get("topKeywords", [])[:5]
|
||||||
|
if keywords:
|
||||||
|
prompt_parts.append(f"Keywords: {', '.join(keywords)}")
|
||||||
|
audience = request.analysis.get("audience", "")
|
||||||
|
if audience:
|
||||||
|
prompt_parts.append(f"Target: {audience}")
|
||||||
|
|
||||||
# Scene content insights for visual context
|
# Scene content insights for visual context
|
||||||
if request.scene_content:
|
if request.scene_content:
|
||||||
content_preview = request.scene_content[:200].replace("\n", " ").strip()
|
content_preview = request.scene_content[:200].replace("\n", " ").strip()
|
||||||
@@ -139,6 +171,12 @@ async def generate_podcast_scene_image(
|
|||||||
visual_keywords.append("modern tech studio setting")
|
visual_keywords.append("modern tech studio setting")
|
||||||
if any(word in content_lower for word in ["business", "growth", "strategy", "market"]):
|
if any(word in content_lower for word in ["business", "growth", "strategy", "market"]):
|
||||||
visual_keywords.append("professional business studio")
|
visual_keywords.append("professional business studio")
|
||||||
|
if any(word in content_lower for word in ["nature", "outdoor", "environment", "green"]):
|
||||||
|
visual_keywords.append("natural outdoor setting")
|
||||||
|
if any(word in content_lower for word in ["medical", "health", "wellness"]):
|
||||||
|
visual_keywords.append("clean medical studio")
|
||||||
|
if any(word in content_lower for word in ["education", "learning", "students"]):
|
||||||
|
visual_keywords.append("classroom or educational setting")
|
||||||
if visual_keywords:
|
if visual_keywords:
|
||||||
prompt_parts.append(", ".join(visual_keywords))
|
prompt_parts.append(", ".join(visual_keywords))
|
||||||
|
|
||||||
@@ -265,6 +303,19 @@ async def generate_podcast_scene_image(
|
|||||||
if request.scene_title:
|
if request.scene_title:
|
||||||
prompt_parts.append(f"Scene theme: {request.scene_title}")
|
prompt_parts.append(f"Scene theme: {request.scene_title}")
|
||||||
|
|
||||||
|
# Scene emotion for visual tone (no avatar branch)
|
||||||
|
if request.scene_emotion and request.scene_emotion in emotion_lighting:
|
||||||
|
prompt_parts.append(emotion_lighting[request.scene_emotion])
|
||||||
|
|
||||||
|
# AI Analysis context (no avatar branch)
|
||||||
|
if request.analysis:
|
||||||
|
keywords = request.analysis.get("topKeywords", [])[:5]
|
||||||
|
if keywords:
|
||||||
|
prompt_parts.append(f"Keywords: {', '.join(keywords)}")
|
||||||
|
audience = request.analysis.get("audience", "")
|
||||||
|
if audience:
|
||||||
|
prompt_parts.append(f"Target: {audience}")
|
||||||
|
|
||||||
# Content context for visual relevance
|
# Content context for visual relevance
|
||||||
if request.scene_content:
|
if request.scene_content:
|
||||||
content_preview = request.scene_content[:150].replace("\n", " ").strip()
|
content_preview = request.scene_content[:150].replace("\n", " ").strip()
|
||||||
@@ -276,6 +327,12 @@ async def generate_podcast_scene_image(
|
|||||||
visual_keywords.append("modern technology aesthetic")
|
visual_keywords.append("modern technology aesthetic")
|
||||||
if any(word in content_lower for word in ["business", "growth", "strategy", "market"]):
|
if any(word in content_lower for word in ["business", "growth", "strategy", "market"]):
|
||||||
visual_keywords.append("professional business environment")
|
visual_keywords.append("professional business environment")
|
||||||
|
if any(word in content_lower for word in ["nature", "outdoor", "environment"]):
|
||||||
|
visual_keywords.append("natural outdoor setting")
|
||||||
|
if any(word in content_lower for word in ["medical", "health", "wellness"]):
|
||||||
|
visual_keywords.append("clean medical studio")
|
||||||
|
if any(word in content_lower for word in ["education", "learning", "students"]):
|
||||||
|
visual_keywords.append("classroom or educational setting")
|
||||||
if visual_keywords:
|
if visual_keywords:
|
||||||
prompt_parts.append(", ".join(visual_keywords))
|
prompt_parts.append(", ".join(visual_keywords))
|
||||||
|
|
||||||
@@ -379,6 +436,7 @@ async def generate_podcast_scene_image(
|
|||||||
provider=result.provider,
|
provider=result.provider,
|
||||||
model=result.model,
|
model=result.model,
|
||||||
cost=cost,
|
cost=cost,
|
||||||
|
image_prompt=image_prompt,
|
||||||
)
|
)
|
||||||
|
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
|
|||||||
@@ -27,7 +27,10 @@ async def create_project(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
):
|
):
|
||||||
"""Create a new podcast project."""
|
"""Create a new podcast project.
|
||||||
|
|
||||||
|
If a project with the same idea already exists, return 409 conflict with existing project info.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
user_id = current_user.get("user_id") or current_user.get("id")
|
user_id = current_user.get("user_id") or current_user.get("id")
|
||||||
if not user_id:
|
if not user_id:
|
||||||
@@ -40,6 +43,19 @@ async def create_project(
|
|||||||
if existing:
|
if existing:
|
||||||
raise HTTPException(status_code=400, detail="Project ID already exists")
|
raise HTTPException(status_code=400, detail="Project ID already exists")
|
||||||
|
|
||||||
|
# Check for duplicate idea (case-insensitive partial match)
|
||||||
|
existing_idea = service.get_project_by_idea(user_id, request.idea)
|
||||||
|
if existing_idea:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=409,
|
||||||
|
detail={
|
||||||
|
"message": "A project with similar idea already exists",
|
||||||
|
"existing_project_id": existing_idea.project_id,
|
||||||
|
"existing_idea": existing_idea.idea,
|
||||||
|
"existing_status": existing_idea.status,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
project = service.create_project(
|
project = service.create_project(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
project_id=request.project_id,
|
project_id=request.project_id,
|
||||||
@@ -103,7 +119,7 @@ async def update_project(
|
|||||||
project = service.update_project(user_id, project_id, **updates)
|
project = service.update_project(user_id, project_id, **updates)
|
||||||
|
|
||||||
if not project:
|
if not project:
|
||||||
raise HTTPException(status_code=404, detail="Project not found")
|
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
|
||||||
|
|
||||||
return PodcastProjectResponse.model_validate(project)
|
return PodcastProjectResponse.model_validate(project)
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
|
|||||||
@@ -8,12 +8,17 @@ from fastapi import APIRouter, Depends, HTTPException
|
|||||||
from typing import Dict, Any, List
|
from typing import Dict, Any, List
|
||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
import json
|
import json
|
||||||
|
import re
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
from middleware.auth_middleware import get_current_user
|
from middleware.auth_middleware import get_current_user
|
||||||
from api.story_writer.utils.auth import require_authenticated_user
|
from api.story_writer.utils.auth import require_authenticated_user
|
||||||
from services.blog_writer.research.exa_provider import ExaResearchProvider
|
from services.blog_writer.research.exa_provider import ExaResearchProvider
|
||||||
from services.llm_providers.main_text_generation import llm_text_gen
|
from services.llm_providers.main_text_generation import llm_text_gen
|
||||||
from services.podcast_bible_service import PodcastBibleService
|
from services.podcast_bible_service import PodcastBibleService
|
||||||
|
from services.database import get_db
|
||||||
|
from services.subscription import PricingService
|
||||||
|
from models.subscription_models import APIProvider
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
from ..models import (
|
from ..models import (
|
||||||
PodcastExaResearchRequest,
|
PodcastExaResearchRequest,
|
||||||
@@ -21,11 +26,102 @@ from ..models import (
|
|||||||
PodcastExaSource,
|
PodcastExaSource,
|
||||||
PodcastExaConfig,
|
PodcastExaConfig,
|
||||||
PodcastResearchInsight,
|
PodcastResearchInsight,
|
||||||
|
PodcastResearchOutput,
|
||||||
|
PodcastCostEst,
|
||||||
|
PodcastCostBreakdownItem,
|
||||||
)
|
)
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
def _estimate_tokens(text: str) -> int:
|
||||||
|
if not text:
|
||||||
|
return 0
|
||||||
|
return max(1, len(text) // 4)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_price_from_catalog(
|
||||||
|
pricing_service: PricingService,
|
||||||
|
provider: APIProvider,
|
||||||
|
model_name: str,
|
||||||
|
key: str,
|
||||||
|
fallback: float = 0.0,
|
||||||
|
) -> float:
|
||||||
|
try:
|
||||||
|
pricing = pricing_service.get_pricing_for_provider_model(provider, model_name) or {}
|
||||||
|
value = pricing.get(key)
|
||||||
|
return float(value or fallback)
|
||||||
|
except Exception:
|
||||||
|
return fallback
|
||||||
|
|
||||||
|
|
||||||
|
def _build_research_cost_estimate(
|
||||||
|
request: PodcastExaResearchRequest,
|
||||||
|
raw_content: str,
|
||||||
|
sources_count: int,
|
||||||
|
provider_result: Dict[str, Any],
|
||||||
|
) -> PodcastCostEst:
|
||||||
|
# Fallback defaults mirror current catalog defaults.
|
||||||
|
exa_per_request = 0.005
|
||||||
|
gemini_in_token = 0.00000015
|
||||||
|
gemini_out_token = 0.0000006
|
||||||
|
|
||||||
|
try:
|
||||||
|
db = next(get_db())
|
||||||
|
try:
|
||||||
|
pricing_service = PricingService(db)
|
||||||
|
exa_per_request = _get_price_from_catalog(
|
||||||
|
pricing_service, APIProvider.EXA, "exa-search", "cost_per_request", exa_per_request
|
||||||
|
)
|
||||||
|
gemini_pricing = pricing_service.get_pricing_for_provider_model(APIProvider.GEMINI, "gemini-2.5-flash") or {}
|
||||||
|
gemini_in_token = float(gemini_pricing.get("cost_per_input_token") or gemini_in_token)
|
||||||
|
gemini_out_token = float(gemini_pricing.get("cost_per_output_token") or gemini_out_token)
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
except Exception as pricing_err:
|
||||||
|
logger.warning(f"[Podcast Research] Failed loading pricing catalog; using defaults: {pricing_err}")
|
||||||
|
|
||||||
|
query_count = max(1, len(request.queries or []))
|
||||||
|
source_count = max(1, sources_count)
|
||||||
|
|
||||||
|
analyze_tokens = _estimate_tokens(request.topic) + sum(_estimate_tokens(q) for q in request.queries or [])
|
||||||
|
gather_search_calls = max(1, query_count)
|
||||||
|
gather_cost = gather_search_calls * exa_per_request
|
||||||
|
|
||||||
|
write_input_tokens = _estimate_tokens(raw_content) + _estimate_tokens(request.topic) + (query_count * 40)
|
||||||
|
write_output_tokens = max(500, int(write_input_tokens * 0.22))
|
||||||
|
write_cost = (write_input_tokens * gemini_in_token) + (write_output_tokens * gemini_out_token)
|
||||||
|
|
||||||
|
# "Produce" is shaping the final API payload and mapped artifacts.
|
||||||
|
produce_tokens = max(120, source_count * 30)
|
||||||
|
produce_cost = (produce_tokens * gemini_in_token) + (produce_tokens * 0.5 * gemini_out_token)
|
||||||
|
|
||||||
|
analyze_cost = analyze_tokens * gemini_in_token
|
||||||
|
|
||||||
|
provider_total = 0.0
|
||||||
|
if isinstance(provider_result, dict):
|
||||||
|
provider_total = float((provider_result.get("cost") or {}).get("total") or 0.0)
|
||||||
|
|
||||||
|
# Prefer transparent estimate built from catalog + usage. If provider reports a higher measured value, keep it.
|
||||||
|
estimated_total = analyze_cost + gather_cost + write_cost + produce_cost
|
||||||
|
scale = (provider_total / estimated_total) if estimated_total > 0 and provider_total > estimated_total else 1.0
|
||||||
|
|
||||||
|
breakdown = [
|
||||||
|
PodcastCostBreakdownItem(phase="Analyze", cost=round(analyze_cost * scale, 6)),
|
||||||
|
PodcastCostBreakdownItem(phase="Gather", cost=round(gather_cost * scale, 6)),
|
||||||
|
PodcastCostBreakdownItem(phase="Write", cost=round(write_cost * scale, 6)),
|
||||||
|
PodcastCostBreakdownItem(phase="Produce", cost=round(produce_cost * scale, 6)),
|
||||||
|
]
|
||||||
|
total = round(sum(item.cost for item in breakdown), 6)
|
||||||
|
|
||||||
|
return PodcastCostEst(
|
||||||
|
total=total,
|
||||||
|
breakdown=breakdown,
|
||||||
|
currency="USD",
|
||||||
|
last_updated=datetime.now(timezone.utc),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@router.post("/research/exa", response_model=PodcastExaResearchResponse)
|
@router.post("/research/exa", response_model=PodcastExaResearchResponse)
|
||||||
async def podcast_research_exa(
|
async def podcast_research_exa(
|
||||||
request: PodcastExaResearchRequest,
|
request: PodcastExaResearchRequest,
|
||||||
@@ -36,10 +132,16 @@ async def podcast_research_exa(
|
|||||||
Uses Podcast Bible and Analysis context for hyper-personalization.
|
Uses Podcast Bible and Analysis context for hyper-personalization.
|
||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
logger.warning(f"[Podcast Research] ========== REQUEST START ==========")
|
||||||
|
logger.warning(f"[Podcast Research] User: {user_id}, Topic: {request.topic[:80]}...")
|
||||||
|
logger.warning(f"[Podcast Research] Queries count: {len(request.queries) if request.queries else 0}")
|
||||||
|
|
||||||
|
|
||||||
queries = [q.strip() for q in request.queries if q and q.strip()]
|
queries = [q.strip() for q in request.queries if q and q.strip()]
|
||||||
if not queries:
|
if not queries:
|
||||||
raise HTTPException(status_code=400, detail="At least one query is required for research.")
|
raise HTTPException(status_code=400, detail="At least one query is required for research.")
|
||||||
|
|
||||||
|
logger.warning(f"[Podcast Research] EXACT queries being sent to Exa: {queries}")
|
||||||
|
|
||||||
exa_cfg = request.exa_config or PodcastExaConfig()
|
exa_cfg = request.exa_config or PodcastExaConfig()
|
||||||
cfg = SimpleNamespace(
|
cfg = SimpleNamespace(
|
||||||
@@ -52,6 +154,7 @@ async def podcast_research_exa(
|
|||||||
)
|
)
|
||||||
|
|
||||||
provider = ExaResearchProvider()
|
provider = ExaResearchProvider()
|
||||||
|
logger.warning(f"[Podcast Research] Provider initialized, starting Exa search...")
|
||||||
|
|
||||||
# --- Context Building ---
|
# --- Context Building ---
|
||||||
bible_service = PodcastBibleService()
|
bible_service = PodcastBibleService()
|
||||||
@@ -68,9 +171,16 @@ async def podcast_research_exa(
|
|||||||
if request.analysis:
|
if request.analysis:
|
||||||
analysis_context = f"""
|
analysis_context = f"""
|
||||||
PODCAST ANALYSIS CONTEXT:
|
PODCAST ANALYSIS CONTEXT:
|
||||||
Audience: {request.analysis.get('audience', 'General')}
|
========================
|
||||||
|
Topic: {request.topic}
|
||||||
|
Target Audience: {request.analysis.get('audience', 'General')}
|
||||||
Content Type: {request.analysis.get('content_type', 'Informative')}
|
Content Type: {request.analysis.get('content_type', 'Informative')}
|
||||||
Top Keywords: {', '.join(request.analysis.get('top_keywords', []))}
|
Top Keywords: {', '.join(request.analysis.get('top_keywords', []))}
|
||||||
|
|
||||||
|
Episode Hook (Intro): {request.analysis.get('episode_hook', 'N/A')}
|
||||||
|
Key Takeaways: {', '.join(request.analysis.get('key_takeaways', [])) or 'N/A'}
|
||||||
|
Guest Talking Points: {', '.join(request.analysis.get('guest_talking_points', [])) or 'N/A'}
|
||||||
|
Listener CTA: {request.analysis.get('listener_cta', 'N/A')}
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# Exa search params
|
# Exa search params
|
||||||
@@ -84,6 +194,7 @@ Top Keywords: {', '.join(request.analysis.get('top_keywords', []))}
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
# 1. RUN EXA SEARCH
|
# 1. RUN EXA SEARCH
|
||||||
|
logger.warning(f"[Podcast Research] Calling Exa search with topic: {request.topic[:100]}...")
|
||||||
result = await provider.search(
|
result = await provider.search(
|
||||||
prompt=request.topic,
|
prompt=request.topic,
|
||||||
topic=request.topic,
|
topic=request.topic,
|
||||||
@@ -92,8 +203,9 @@ Top Keywords: {', '.join(request.analysis.get('top_keywords', []))}
|
|||||||
config=cfg,
|
config=cfg,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
)
|
)
|
||||||
|
logger.warning(f"[Podcast Research] Exa search completed, got {len(result.get('sources', []))} sources")
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error(f"[Podcast Exa Research] Search failed for user {user_id}: {exc}")
|
logger.error(f"[Podcast Exa Research] Search failed for user {user_id}: {exc}", exc_info=True)
|
||||||
raise HTTPException(status_code=500, detail=f"Exa research failed: {exc}")
|
raise HTTPException(status_code=500, detail=f"Exa research failed: {exc}")
|
||||||
|
|
||||||
# 2. EXTRACT INSIGHTS VIA LLM
|
# 2. EXTRACT INSIGHTS VIA LLM
|
||||||
@@ -104,66 +216,135 @@ Top Keywords: {', '.join(request.analysis.get('top_keywords', []))}
|
|||||||
key_insights = []
|
key_insights = []
|
||||||
|
|
||||||
if raw_content and sources:
|
if raw_content and sources:
|
||||||
logger.info(f"[Podcast Research] Extracting insights from {len(sources)} sources for user {user_id}")
|
logger.warning(f"[Podcast Research] Extracting insights from {len(sources)} sources for user {user_id}")
|
||||||
|
|
||||||
|
# Build list of research queries used for this search
|
||||||
|
queries_used = ", ".join([f"Query {i+1}: {q}" for i, q in enumerate(queries)]) if queries else "No specific queries"
|
||||||
|
|
||||||
prompt = f"""
|
prompt = f"""
|
||||||
You are an expert research analyst for a high-end podcast production team.
|
You are an expert research analyst and content strategist for a high-end podcast production team.
|
||||||
Your task is to analyze the following research data and extract deep, actionable insights for a podcast episode.
|
Your task is to analyze the research data and extract deep, podcast-ready insights.
|
||||||
|
|
||||||
PODCAST CONTEXT:
|
PODCAST CONTEXT:
|
||||||
Topic: {request.topic}
|
================
|
||||||
|
Main Topic: {request.topic}
|
||||||
|
|
||||||
|
RESEARCH QUERIES USED:
|
||||||
|
=====================
|
||||||
|
{queries_used}
|
||||||
|
|
||||||
|
PODCAST BIBLE & BRAND CONTEXT:
|
||||||
|
==============================
|
||||||
{bible_context}
|
{bible_context}
|
||||||
|
|
||||||
|
PODCAST ANALYSIS (from AI Analysis phase):
|
||||||
|
==========================================
|
||||||
{analysis_context}
|
{analysis_context}
|
||||||
|
|
||||||
RESEARCH DATA (from {len(sources)} sources):
|
RESEARCH DATA (from {len(sources)} sources):
|
||||||
|
============================================
|
||||||
{raw_content}
|
{raw_content}
|
||||||
|
|
||||||
TASK:
|
YOUR TASK:
|
||||||
1. Provide a comprehensive summary (2-3 paragraphs) of the most important findings. Use Markdown for formatting (bolding, lists).
|
==========
|
||||||
2. Extract 3-5 "Key Insights". Each insight should have a title and a detailed explanation.
|
As a podcast research expert, analyze this data and create content that will:
|
||||||
3. For each insight, identify which source indices (e.g. 1, 2) it was derived from.
|
1. Engage the specific target audience identified above
|
||||||
|
2. Support the episode hook and key takeaways already planned
|
||||||
|
3. Provide talking points that complement the guest's expertise
|
||||||
|
4. Include a compelling call-to-action for listeners
|
||||||
|
|
||||||
NOTE: The research data includes "Key Highlights", "Summaries", and "Excerpts" from various sources.
|
REQUIRED OUTPUT (JSON):
|
||||||
Pay special attention to the "Key Highlights" sections as they contain the most relevant information extracted by the neural search engine.
|
======================
|
||||||
|
|
||||||
Return JSON structure:
|
|
||||||
{{
|
{{
|
||||||
"summary": "Detailed markdown summary...",
|
"summary": "2-3 paragraph comprehensive summary in Markdown. Start with a hook that matches the episode intro.",
|
||||||
"key_insights": [
|
"key_insights": [
|
||||||
{{
|
{{
|
||||||
"title": "Insight Title",
|
"title": "Insight title",
|
||||||
"content": "Detailed markdown content...",
|
"content": "3-4 sentences with specific facts, quotes, or data for podcast host.",
|
||||||
"source_indices": [1, 2]
|
"source_indices": [1, 2],
|
||||||
|
"podcast_talking_points": ["Point host can expand on", "Counter-point"]
|
||||||
|
}}
|
||||||
|
],
|
||||||
|
"expert_quotes": [
|
||||||
|
{{
|
||||||
|
"quote": "Direct quote from source text",
|
||||||
|
"source_index": 1,
|
||||||
|
"context": "Why this quote matters for the podcast"
|
||||||
|
}}
|
||||||
|
],
|
||||||
|
"listener_cta_suggestions": ["Action listener can take", "Resource to share", "Next episode preview"],
|
||||||
|
"mapped_angles": [
|
||||||
|
{{
|
||||||
|
"title": "Content angle title",
|
||||||
|
"why": "Why compelling for audience",
|
||||||
|
"mapped_fact_ids": [1, 2]
|
||||||
}}
|
}}
|
||||||
]
|
]
|
||||||
}}
|
}}
|
||||||
|
|
||||||
Requirements:
|
IMPORTANT: You must include ALL fields above with valid data. expert_quotes, listener_cta_suggestions, and mapped_angles must have content - do NOT leave them empty!
|
||||||
- Ensure insights are deep, not just superficial facts. Look for trends, expert opinions, and specific data points.
|
|
||||||
- Tone should be professional, insightful, and ready for a podcast host to discuss.
|
QUALITY STANDARDS:
|
||||||
- Avoid generic filler.
|
=================
|
||||||
|
- Include at least 2 expert_quotes with source_index
|
||||||
|
- Include at least 2 listener_cta_suggestions
|
||||||
|
- Include at least 2 mapped_angles
|
||||||
|
- Include specific data points, percentages, statistics
|
||||||
|
- Write in conversational tone
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
|
logger.warning(f"[Podcast Research] Calling LLM with json_struct...")
|
||||||
llm_response = llm_text_gen(
|
llm_response = llm_text_gen(
|
||||||
prompt=prompt,
|
prompt=prompt,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
json_struct=None,
|
json_struct=PodcastResearchOutput.model_json_schema(),
|
||||||
preferred_provider="huggingface",
|
preferred_provider=None,
|
||||||
flow_type="premium_tool",
|
flow_type="premium_tool",
|
||||||
)
|
)
|
||||||
|
logger.warning(f"[Podcast Research] LLM response received, length: {len(llm_response) if llm_response else 0}")
|
||||||
|
|
||||||
# Normalize response
|
# Normalize response - handle both string and dict responses
|
||||||
|
data = None
|
||||||
if isinstance(llm_response, str):
|
if isinstance(llm_response, str):
|
||||||
data = json.loads(llm_response)
|
try:
|
||||||
|
# Try to fix common JSON issues
|
||||||
|
fixed_response = llm_response.strip()
|
||||||
|
# Remove markdown code blocks if present
|
||||||
|
if fixed_response.startswith("```"):
|
||||||
|
fixed_response = fixed_response.split("```")[1]
|
||||||
|
if fixed_response.startswith("json"):
|
||||||
|
fixed_response = fixed_response[4:]
|
||||||
|
fixed_response = fixed_response.strip()
|
||||||
|
data = json.loads(fixed_response)
|
||||||
|
except json.JSONDecodeError as json_err:
|
||||||
|
logger.warning(f"[Podcast Research] Failed to parse JSON: {json_err}. Response preview: {llm_response[:500]}...")
|
||||||
|
# Try to extract JSON from response using regex
|
||||||
|
json_match = re.search(r'\{.*\}', llm_response, re.DOTALL)
|
||||||
|
if json_match:
|
||||||
|
try:
|
||||||
|
data = json.loads(json_match.group())
|
||||||
|
logger.warning("[Podcast Research] Successfully extracted JSON via regex")
|
||||||
|
except:
|
||||||
|
pass
|
||||||
else:
|
else:
|
||||||
data = llm_response
|
data = llm_response
|
||||||
|
|
||||||
summary = data.get("summary", "")
|
if data:
|
||||||
key_insights = [PodcastResearchInsight(**insight) for insight in data.get("key_insights", [])]
|
try:
|
||||||
|
summary = data.get("summary", "")
|
||||||
|
key_insights = [PodcastResearchInsight(**insight) for insight in data.get("key_insights", [])]
|
||||||
|
except Exception as insight_err:
|
||||||
|
logger.warning(f"[Podcast Research] Failed to parse insights: {insight_err}. Data keys: {list(data.keys()) if isinstance(data, dict) else 'not a dict'}")
|
||||||
|
summary = data.get("summary", "") if isinstance(data, dict) else ""
|
||||||
|
key_insights = []
|
||||||
|
else:
|
||||||
|
summary = ""
|
||||||
|
key_insights = []
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error(f"[Podcast Research] LLM Insight extraction failed: {exc}")
|
logger.error(f"[Podcast Research] LLM Insight extraction failed: {exc}")
|
||||||
# Fallback to a basic summary if LLM fails
|
raise HTTPException(status_code=500, detail=f"Research insight extraction failed: {exc}")
|
||||||
summary = f"Research completed for '{request.topic}'. Found {len(sources)} sources."
|
|
||||||
|
|
||||||
# Fallback: if summary is still empty (e.g. LLM returned empty string), use raw content first paragraph or basic text
|
# Fallback: if summary is still empty (e.g. LLM returned empty string), use raw content first paragraph or basic text
|
||||||
if not summary:
|
if not summary:
|
||||||
@@ -182,21 +363,32 @@ Requirements:
|
|||||||
logger.warning(f"[Podcast Exa Research] Failed to track usage: {track_err}")
|
logger.warning(f"[Podcast Exa Research] Failed to track usage: {track_err}")
|
||||||
|
|
||||||
sources_payload = []
|
sources_payload = []
|
||||||
|
seen_urls = set()
|
||||||
for src in sources:
|
for src in sources:
|
||||||
|
url = src.get("url", "")
|
||||||
|
# Skip duplicates
|
||||||
|
if url and url in seen_urls:
|
||||||
|
continue
|
||||||
|
if url:
|
||||||
|
seen_urls.add(url)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
sources_payload.append(PodcastExaSource(**src))
|
sources_payload.append(PodcastExaSource(**src))
|
||||||
except Exception:
|
except Exception:
|
||||||
sources_payload.append(PodcastExaSource(**{
|
sources_payload.append(PodcastExaSource(**{
|
||||||
"title": src.get("title", ""),
|
"title": src.get("title", ""),
|
||||||
"url": src.get("url", ""),
|
"url": url,
|
||||||
"excerpt": src.get("excerpt", ""),
|
"excerpt": src.get("excerpt") or (src.get("highlights")[0] if src.get("highlights") else "") or src.get("summary", ""),
|
||||||
"published_at": src.get("published_at"),
|
"published_at": src.get("published_at"),
|
||||||
|
"publishedDate": src.get("publishedDate"),
|
||||||
"highlights": src.get("highlights"),
|
"highlights": src.get("highlights"),
|
||||||
"summary": src.get("summary"),
|
"summary": src.get("summary"),
|
||||||
"source_type": src.get("source_type"),
|
"source_type": src.get("source_type"),
|
||||||
"index": src.get("index"),
|
"index": src.get("index"),
|
||||||
"image": src.get("image"),
|
"image": src.get("image"),
|
||||||
"author": src.get("author"),
|
"author": src.get("author"),
|
||||||
|
"text": src.get("text"),
|
||||||
|
"credibility_score": src.get("credibility_score"),
|
||||||
}))
|
}))
|
||||||
|
|
||||||
return PodcastExaResearchResponse(
|
return PodcastExaResearchResponse(
|
||||||
@@ -204,9 +396,13 @@ Requirements:
|
|||||||
search_queries=result.get("search_queries", queries) if isinstance(result, dict) else queries,
|
search_queries=result.get("search_queries", queries) if isinstance(result, dict) else queries,
|
||||||
summary=summary,
|
summary=summary,
|
||||||
key_insights=key_insights,
|
key_insights=key_insights,
|
||||||
cost=result.get("cost") if isinstance(result, dict) else None,
|
cost_est=_build_research_cost_estimate(
|
||||||
|
request=request,
|
||||||
|
raw_content=raw_content,
|
||||||
|
sources_count=len(sources_payload),
|
||||||
|
provider_result=result if isinstance(result, dict) else {},
|
||||||
|
),
|
||||||
search_type=result.get("search_type") if isinstance(result, dict) else None,
|
search_type=result.get("search_type") if isinstance(result, dict) else None,
|
||||||
provider=result.get("provider", "exa") if isinstance(result, dict) else "exa",
|
provider=result.get("provider", "exa") if isinstance(result, dict) else "exa",
|
||||||
content=raw_content,
|
content=raw_content,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
"""
|
"""
|
||||||
Podcast Script Handlers
|
Podcast Script Handlers
|
||||||
|
|
||||||
Script generation endpoint.
|
Script generation and approval endpoints.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends, HTTPException
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
from typing import Dict, Any
|
from typing import Dict, Any, Optional
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
import json
|
import json
|
||||||
|
|
||||||
from middleware.auth_middleware import get_current_user
|
from middleware.auth_middleware import get_current_user
|
||||||
@@ -24,6 +25,29 @@ from ..models import (
|
|||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
class SceneApprovalRequest(BaseModel):
|
||||||
|
project_id: str = Field(..., min_length=1)
|
||||||
|
scene_id: str = Field(..., min_length=1)
|
||||||
|
approved: bool = True
|
||||||
|
notes: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/script/approve")
|
||||||
|
async def approve_podcast_scene(
|
||||||
|
request: SceneApprovalRequest,
|
||||||
|
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Persist scene approval metadata for auditing (podcast-specific)."""
|
||||||
|
user_id = require_authenticated_user(current_user)
|
||||||
|
logger.warning(f"[Podcast] Scene approval recorded user={user_id} project={request.project_id} scene={request.scene_id} approved={request.approved}")
|
||||||
|
return {
|
||||||
|
"success": True,
|
||||||
|
"project_id": request.project_id,
|
||||||
|
"scene_id": request.scene_id,
|
||||||
|
"approved": request.approved,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
@router.post("/script", response_model=PodcastScriptResponse)
|
@router.post("/script", response_model=PodcastScriptResponse)
|
||||||
async def generate_podcast_script(
|
async def generate_podcast_script(
|
||||||
request: PodcastScriptRequest,
|
request: PodcastScriptRequest,
|
||||||
@@ -33,6 +57,10 @@ async def generate_podcast_script(
|
|||||||
Generate a podcast script outline (scenes + lines) using podcast-oriented prompting.
|
Generate a podcast script outline (scenes + lines) using podcast-oriented prompting.
|
||||||
"""
|
"""
|
||||||
user_id = require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
|
logger.warning(f"[ScriptGen] ========== SCRIPT GENERATION START ==========")
|
||||||
|
logger.warning(f"[ScriptGen] Topic: {request.idea[:60]}...")
|
||||||
|
logger.warning(f"[ScriptGen] Duration: {request.duration_minutes} min, Speakers: {request.speakers}")
|
||||||
|
logger.warning(f"[ScriptGen] Has research: {bool(request.research)}, Has bible: {bool(request.bible)}, Has analysis: {bool(request.analysis)}")
|
||||||
|
|
||||||
# Build comprehensive research context for higher-quality scripts
|
# Build comprehensive research context for higher-quality scripts
|
||||||
research_context = ""
|
research_context = ""
|
||||||
@@ -77,62 +105,63 @@ async def generate_podcast_script(
|
|||||||
# Extract Analysis and Outline context for grounding
|
# Extract Analysis and Outline context for grounding
|
||||||
analysis_context = ""
|
analysis_context = ""
|
||||||
if request.analysis:
|
if request.analysis:
|
||||||
analysis_context = f"""
|
try:
|
||||||
TARGET AUDIENCE: {request.analysis.get('audience', 'General')}
|
audience = request.analysis.get('audience', '') or ''
|
||||||
CONTENT TYPE: {request.analysis.get('contentType', 'Conversational')}
|
content_type = request.analysis.get('contentType', '') or ''
|
||||||
TOP KEYWORDS: {', '.join(request.analysis.get('topKeywords', []))}
|
keywords = request.analysis.get('topKeywords', []) or []
|
||||||
"""
|
analysis_context = f"ANALYSIS: Audience={audience} | Type={content_type} | Keywords={', '.join(keywords[:8])}"
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
|
||||||
outline_context = ""
|
outline_context = ""
|
||||||
if request.outline:
|
if request.outline:
|
||||||
outline_context = f"""
|
try:
|
||||||
REFINED EPISODE OUTLINE (Follow this structure closely):
|
title = request.outline.get('title', '') or ''
|
||||||
Title: {request.outline.get('title', 'N/A')}
|
segments = request.outline.get('segments', []) or []
|
||||||
Segments: {' | '.join(request.outline.get('segments', []))}
|
outline_context = f"OUTLINE: {title} - {' | '.join(segments[:5])}"
|
||||||
"""
|
except:
|
||||||
|
pass
|
||||||
|
|
||||||
prompt = f"""You are an expert podcast script planner. Create natural, conversational podcast scenes.
|
prompt = f"""Create a podcast script with scenes and dialogue.
|
||||||
|
|
||||||
{f"PODCAST BIBLE (Hyper-Personalization Context):\n{bible_context}\n" if bible_context else ""}
|
{f"BIBLE: {bible_context[:1500]}" if bible_context else ""}
|
||||||
{f"ANALYSIS CONTEXT:\n{analysis_context}\n" if analysis_context else ""}
|
{f"{analysis_context}" if analysis_context else ""}
|
||||||
{f"REFINED OUTLINE:\n{outline_context}\n" if outline_context else ""}
|
{f"{outline_context}" if outline_context else ""}
|
||||||
|
{f"RESEARCH: {research_context[:1200]}" if research_context else ""}
|
||||||
|
|
||||||
Podcast Idea: "{request.idea}"
|
Topic: "{request.idea}"
|
||||||
Duration: ~{request.duration_minutes} minutes
|
Duration: {request.duration_minutes} min | Speakers: {request.speakers}
|
||||||
Speakers: {request.speakers} (Host + optional Guest)
|
|
||||||
|
|
||||||
{f"RESEARCH CONTEXT:\n{research_context}\n" if research_context else ""}
|
Return JSON with scenes array. Each scene:
|
||||||
|
- id: string
|
||||||
|
- title: short title (<=50 chars)
|
||||||
|
- duration: seconds (total/5)
|
||||||
|
- emotion: neutral|happy|excited|serious|curious|confident
|
||||||
|
- lines: array of {{speaker, text, emphasis}}
|
||||||
|
- Use 2-4 LINES PER SCENE (shorter script = lower TTS costs)
|
||||||
|
- Each line: 1-3 sentences, conversational
|
||||||
|
- Plain text only, no markdown
|
||||||
|
|
||||||
Return JSON with:
|
COST OPTIMIZATION:
|
||||||
- scenes: array of scenes. Each scene has:
|
- 5-6 scenes max for {request.duration_minutes} min episode
|
||||||
- id: string
|
- Concise, information-dense dialogue
|
||||||
- title: short scene title (<= 60 chars)
|
- Skip filler words and redundant phrases
|
||||||
- duration: duration in seconds (evenly split across total duration)
|
- Focus on unique insights from research
|
||||||
- emotion: string (one of: "neutral", "happy", "excited", "serious", "curious", "confident")
|
- Make every line count toward value delivery
|
||||||
- lines: array of {{"speaker": "...", "text": "...", "emphasis": boolean}}
|
|
||||||
* Write natural, conversational dialogue
|
|
||||||
* Each line can be a sentence or a few sentences that flow together
|
|
||||||
* Use plain text only - no markdown formatting (no asterisks, underscores, etc.)
|
|
||||||
* Mark "emphasis": true for key statistics or important points
|
|
||||||
|
|
||||||
Guidelines:
|
|
||||||
- Write for spoken delivery: conversational, natural, with contractions.
|
|
||||||
- Follow the interaction tone specified in the Bible.
|
|
||||||
- Ensure the Host persona matches the background and personality traits from the Bible.
|
|
||||||
- Structure the intro and outro scenes according to the Bible's "Intro Format" and "Outro Format".
|
|
||||||
- Adhere to any constraints mentioned in the Bible.
|
|
||||||
- Use insights from the Research Context to ground the conversation in facts.
|
|
||||||
- IMPORTANT: Follow the REFINED OUTLINE segments as the primary structure for the episode.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
logger.warning(f"[ScriptGen] Calling LLM to generate script (prompt length: {len(prompt)})...")
|
||||||
raw = llm_text_gen(
|
raw = llm_text_gen(
|
||||||
prompt=prompt,
|
prompt=prompt,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
json_struct=None,
|
json_struct=None,
|
||||||
preferred_provider="huggingface",
|
preferred_provider=None,
|
||||||
flow_type="premium_tool",
|
flow_type="premium_tool",
|
||||||
)
|
)
|
||||||
|
logger.warning(f"[ScriptGen] LLM response received, length: {len(raw) if raw else 0}")
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
raise HTTPException(status_code=500, detail=f"Script generation failed: {exc}")
|
raise HTTPException(status_code=500, detail=f"Script generation failed: {exc}")
|
||||||
|
|
||||||
@@ -149,25 +178,83 @@ Guidelines:
|
|||||||
scenes_data = data.get("scenes") or []
|
scenes_data = data.get("scenes") or []
|
||||||
if not isinstance(scenes_data, list):
|
if not isinstance(scenes_data, list):
|
||||||
raise HTTPException(status_code=500, detail="LLM response missing scenes array")
|
raise HTTPException(status_code=500, detail="LLM response missing scenes array")
|
||||||
|
|
||||||
|
if len(scenes_data) == 0:
|
||||||
|
logger.warning("[ScriptGen] LLM returned empty scenes array")
|
||||||
|
raise HTTPException(status_code=500, detail="LLM returned no scenes - please try again")
|
||||||
|
|
||||||
|
logger.warning(f"[ScriptGen] Processing {len(scenes_data)} scenes from LLM response")
|
||||||
|
|
||||||
valid_emotions = {"neutral", "happy", "excited", "serious", "curious", "confident"}
|
valid_emotions = {"neutral", "happy", "excited", "serious", "curious", "confident"}
|
||||||
|
|
||||||
# Normalize scenes
|
# Normalize scenes
|
||||||
scenes: list[PodcastScene] = []
|
scenes: list[PodcastScene] = []
|
||||||
|
total_lines_input = 0
|
||||||
|
total_lines_output = 0
|
||||||
|
dropped_empty_lines = 0
|
||||||
|
|
||||||
for idx, scene in enumerate(scenes_data):
|
for idx, scene in enumerate(scenes_data):
|
||||||
|
if not isinstance(scene, dict):
|
||||||
|
logger.warning(f"[ScriptGen] Scene {idx} is not a dict, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
title = scene.get("title") or f"Scene {idx + 1}"
|
title = scene.get("title") or f"Scene {idx + 1}"
|
||||||
duration = int(scene.get("duration") or max(30, (request.duration_minutes * 60) // max(1, len(scenes_data))))
|
duration = int(scene.get("duration") or max(30, (request.duration_minutes * 60) // max(1, len(scenes_data))))
|
||||||
emotion = scene.get("emotion") or "neutral"
|
emotion = scene.get("emotion") or "neutral"
|
||||||
if emotion not in valid_emotions:
|
if emotion not in valid_emotions:
|
||||||
|
logger.warning(f"[ScriptGen] Invalid emotion '{emotion}' in scene {idx}, defaulting to 'neutral'")
|
||||||
emotion = "neutral"
|
emotion = "neutral"
|
||||||
lines_raw = scene.get("lines") or []
|
lines_raw = scene.get("lines") or []
|
||||||
|
total_lines_input += len(lines_raw)
|
||||||
lines: list[PodcastSceneLine] = []
|
lines: list[PodcastSceneLine] = []
|
||||||
for line in lines_raw:
|
|
||||||
|
for line_idx, line in enumerate(lines_raw):
|
||||||
|
if not isinstance(line, dict):
|
||||||
|
logger.warning(f"[ScriptGen] Line {line_idx} in scene {idx} is not a dict, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
speaker = line.get("speaker") or ("Host" if len(lines) % request.speakers == 0 else "Guest")
|
speaker = line.get("speaker") or ("Host" if len(lines) % request.speakers == 0 else "Guest")
|
||||||
text = line.get("text") or ""
|
text = line.get("text") or ""
|
||||||
emphasis = line.get("emphasis", False)
|
|
||||||
|
# Handle emphasis - convert various values to boolean
|
||||||
|
emphasis_raw = line.get("emphasis", False)
|
||||||
|
if isinstance(emphasis_raw, bool):
|
||||||
|
emphasis = emphasis_raw
|
||||||
|
elif isinstance(emphasis_raw, str):
|
||||||
|
emphasis = emphasis_raw.lower() in ("true", "yes", "1")
|
||||||
|
if emphasis_raw.lower() not in ("true", "false", "yes", "no", "1", "0"):
|
||||||
|
logger.debug(f"[ScriptGen] Unusual emphasis value '{emphasis_raw}' converted to {emphasis}")
|
||||||
|
else:
|
||||||
|
emphasis = bool(emphasis_raw)
|
||||||
|
|
||||||
|
# Generate line ID if not provided
|
||||||
|
line_id = line.get("id") or f"line-{idx + 1}-{line_idx + 1}"
|
||||||
|
|
||||||
|
# Get used fact IDs if provided
|
||||||
|
used_fact_ids = line.get("usedFactIds") or line.get("used_fact_ids") or None
|
||||||
|
|
||||||
if text:
|
if text:
|
||||||
lines.append(PodcastSceneLine(speaker=speaker, text=text, emphasis=emphasis))
|
lines.append(PodcastSceneLine(
|
||||||
|
speaker=speaker,
|
||||||
|
text=text,
|
||||||
|
emphasis=emphasis,
|
||||||
|
id=line_id,
|
||||||
|
usedFactIds=used_fact_ids
|
||||||
|
))
|
||||||
|
total_lines_output += 1
|
||||||
|
else:
|
||||||
|
dropped_empty_lines += 1
|
||||||
|
logger.debug(f"[ScriptGen] Dropped empty line {line_idx} in scene {idx}")
|
||||||
|
|
||||||
|
# Log scene status
|
||||||
|
if scenes_data and isinstance(scene, dict):
|
||||||
|
image_url_raw = scene.get("imageUrl") or scene.get("image_url")
|
||||||
|
audio_url_raw = scene.get("audioUrl") or scene.get("audio_url")
|
||||||
|
if image_url_raw:
|
||||||
|
logger.warning(f"[ScriptGen] Scene {idx} has imageUrl - will be reset to None")
|
||||||
|
if audio_url_raw:
|
||||||
|
logger.warning(f"[ScriptGen] Scene {idx} has audioUrl - will be reset to None")
|
||||||
|
|
||||||
scenes.append(
|
scenes.append(
|
||||||
PodcastScene(
|
PodcastScene(
|
||||||
id=scene.get("id") or f"scene-{idx + 1}",
|
id=scene.get("id") or f"scene-{idx + 1}",
|
||||||
@@ -176,8 +263,16 @@ Guidelines:
|
|||||||
lines=lines,
|
lines=lines,
|
||||||
approved=False,
|
approved=False,
|
||||||
emotion=emotion,
|
emotion=emotion,
|
||||||
|
imageUrl=None, # Will be generated later
|
||||||
|
audioUrl=None, # Will be generated later
|
||||||
|
imagePrompt=None, # Will be generated during image generation
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Summary logging
|
||||||
|
logger.warning(f"[ScriptGen] Script generated: {len(scenes)} scenes, {total_lines_output}/{total_lines_input} lines")
|
||||||
|
if dropped_empty_lines > 0:
|
||||||
|
logger.warning(f"[ScriptGen] Dropped {dropped_empty_lines} empty lines")
|
||||||
|
|
||||||
return PodcastScriptResponse(scenes=scenes)
|
return PodcastScriptResponse(scenes=scenes)
|
||||||
|
|
||||||
|
|||||||
@@ -140,17 +140,20 @@ def _execute_podcast_video_task(
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"[Podcast] Failed to fetch project context for video generation: {e}")
|
logger.warning(f"[Podcast] Failed to fetch project context for video generation: {e}")
|
||||||
|
|
||||||
# Prepare scene data for animation
|
# Prepare scene data for animation - include all context for enhanced prompt
|
||||||
scene_data = {
|
scene_data = {
|
||||||
"scene_number": scene_number,
|
"scene_number": scene_number,
|
||||||
"title": request.scene_title,
|
"title": request.scene_title,
|
||||||
"scene_id": request.scene_id,
|
"scene_id": request.scene_id,
|
||||||
|
"image_prompt": request.scene_image_prompt,
|
||||||
|
"description": request.scene_narration,
|
||||||
|
"lines": [{"text": request.scene_narration}] if request.scene_narration else [],
|
||||||
}
|
}
|
||||||
story_context = {
|
story_context = {
|
||||||
"project_id": request.project_id,
|
"project_id": request.project_id,
|
||||||
"type": "podcast",
|
"type": "podcast",
|
||||||
"bible": project_bible,
|
"bible": project_bible,
|
||||||
"analysis": project_analysis,
|
"analysis": request.analysis or project_analysis, # Use passed analysis or fallback to DB
|
||||||
}
|
}
|
||||||
|
|
||||||
animation_result = animate_scene_with_voiceover(
|
animation_result = animate_scene_with_voiceover(
|
||||||
@@ -222,7 +225,7 @@ def _execute_podcast_video_task(
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Verify the task status was updated correctly
|
# Verify the task status was updated correctly
|
||||||
updated_status = task_manager.get_task_status(task_id)
|
updated_status = task_manager.get_task_status(task_id, requester_user_id=user_id)
|
||||||
logger.info(
|
logger.info(
|
||||||
f"[Podcast] Task status after update: task_id={task_id}, status={updated_status.get('status') if updated_status else 'None'}, has_result={bool(updated_status.get('result') if updated_status else False)}, video_url={updated_status.get('result', {}).get('video_url') if updated_status else 'N/A'}"
|
f"[Podcast] Task status after update: task_id={task_id}, status={updated_status.get('status') if updated_status else 'None'}, has_result={bool(updated_status.get('result') if updated_status else False)}, video_url={updated_status.get('result', {}).get('video_url') if updated_status else 'N/A'}"
|
||||||
)
|
)
|
||||||
@@ -358,7 +361,10 @@ async def generate_podcast_video(
|
|||||||
logger.warning(f"[Podcast] Failed to extract auth token from headers: {e}")
|
logger.warning(f"[Podcast] Failed to extract auth token from headers: {e}")
|
||||||
|
|
||||||
# Create async task
|
# Create async task
|
||||||
task_id = task_manager.create_task("podcast_video_generation")
|
task_id = task_manager.create_task(
|
||||||
|
"podcast_video_generation",
|
||||||
|
metadata={"owner_user_id": user_id},
|
||||||
|
)
|
||||||
background_tasks.add_task(
|
background_tasks.add_task(
|
||||||
_execute_podcast_video_task,
|
_execute_podcast_video_task,
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
@@ -488,7 +494,10 @@ async def combine_podcast_videos(
|
|||||||
raise HTTPException(status_code=400, detail="No scene videos provided")
|
raise HTTPException(status_code=400, detail="No scene videos provided")
|
||||||
|
|
||||||
# Create async task
|
# Create async task
|
||||||
task_id = task_manager.create_task("podcast_combine_videos")
|
task_id = task_manager.create_task(
|
||||||
|
"podcast_combine_videos",
|
||||||
|
metadata={"owner_user_id": user_id},
|
||||||
|
)
|
||||||
|
|
||||||
# Extract token for authenticated URL building
|
# Extract token for authenticated URL building
|
||||||
auth_token = None
|
auth_token = None
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ All Pydantic request/response models for podcast endpoints.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
from pydantic import BaseModel, Field, model_validator
|
from pydantic import BaseModel, Field, model_validator
|
||||||
from typing import List, Optional, Dict, Any
|
from typing import List, Optional, Dict, Any, Literal
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from enum import Enum
|
from enum import Enum
|
||||||
|
|
||||||
@@ -54,6 +54,7 @@ class PodcastAnalyzeRequest(BaseModel):
|
|||||||
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
||||||
avatar_url: Optional[str] = Field(None, description="Current avatar URL if selected")
|
avatar_url: Optional[str] = Field(None, description="Current avatar URL if selected")
|
||||||
feedback: Optional[str] = Field(None, description="User feedback for regeneration")
|
feedback: Optional[str] = Field(None, description="User feedback for regeneration")
|
||||||
|
podcast_mode: Optional[str] = Field(None, description="Podcast mode: audio_only, video_only, or audio_video")
|
||||||
|
|
||||||
|
|
||||||
class PodcastAnalyzeResponse(BaseModel):
|
class PodcastAnalyzeResponse(BaseModel):
|
||||||
@@ -63,11 +64,16 @@ class PodcastAnalyzeResponse(BaseModel):
|
|||||||
top_keywords: list[str]
|
top_keywords: list[str]
|
||||||
suggested_outlines: list[Dict[str, Any]]
|
suggested_outlines: list[Dict[str, Any]]
|
||||||
title_suggestions: list[str]
|
title_suggestions: list[str]
|
||||||
|
episode_hook: Optional[str] = None
|
||||||
|
key_takeaways: Optional[list[str]] = None
|
||||||
|
guest_talking_points: Optional[list[str]] = None
|
||||||
|
listener_cta: Optional[str] = None
|
||||||
research_queries: Optional[List[Dict[str, str]]] = None
|
research_queries: Optional[List[Dict[str, str]]] = None
|
||||||
exa_suggested_config: Optional[Dict[str, Any]] = None
|
exa_suggested_config: Optional[Dict[str, Any]] = None
|
||||||
bible: Optional[Dict[str, Any]] = None
|
bible: Optional[Dict[str, Any]] = None
|
||||||
avatar_url: Optional[str] = None
|
avatar_url: Optional[str] = None
|
||||||
avatar_prompt: Optional[str] = None
|
avatar_prompt: Optional[str] = None
|
||||||
|
estimate: Optional[Dict[str, Any]] = None
|
||||||
|
|
||||||
|
|
||||||
class PodcastEnhanceIdeaRequest(BaseModel):
|
class PodcastEnhanceIdeaRequest(BaseModel):
|
||||||
@@ -97,6 +103,8 @@ class PodcastSceneLine(BaseModel):
|
|||||||
speaker: str
|
speaker: str
|
||||||
text: str
|
text: str
|
||||||
emphasis: Optional[bool] = False
|
emphasis: Optional[bool] = False
|
||||||
|
id: Optional[str] = None # Optional line ID for frontend tracking
|
||||||
|
usedFactIds: Optional[List[str]] = None # Facts referenced in this line
|
||||||
|
|
||||||
|
|
||||||
class PodcastScene(BaseModel):
|
class PodcastScene(BaseModel):
|
||||||
@@ -107,6 +115,8 @@ class PodcastScene(BaseModel):
|
|||||||
approved: bool = False
|
approved: bool = False
|
||||||
emotion: Optional[str] = None
|
emotion: Optional[str] = None
|
||||||
imageUrl: Optional[str] = None # Generated image URL for video generation
|
imageUrl: Optional[str] = None # Generated image URL for video generation
|
||||||
|
audioUrl: Optional[str] = None # Generated audio URL for this scene
|
||||||
|
imagePrompt: Optional[str] = None # Original image generation prompt for video context
|
||||||
|
|
||||||
|
|
||||||
class PodcastExaConfig(BaseModel):
|
class PodcastExaConfig(BaseModel):
|
||||||
@@ -142,12 +152,15 @@ class PodcastExaSource(BaseModel):
|
|||||||
url: str = ""
|
url: str = ""
|
||||||
excerpt: str = ""
|
excerpt: str = ""
|
||||||
published_at: Optional[str] = None
|
published_at: Optional[str] = None
|
||||||
|
publishedDate: Optional[str] = None # Exa format
|
||||||
highlights: Optional[List[str]] = None
|
highlights: Optional[List[str]] = None
|
||||||
summary: Optional[str] = None
|
summary: Optional[str] = None
|
||||||
source_type: Optional[str] = None
|
source_type: Optional[str] = None
|
||||||
index: Optional[int] = None
|
index: Optional[int] = None
|
||||||
image: Optional[str] = None
|
image: Optional[str] = None
|
||||||
author: Optional[str] = None
|
author: Optional[str] = None
|
||||||
|
text: Optional[str] = None # Exa full text
|
||||||
|
credibility_score: Optional[float] = None # Exa scores
|
||||||
|
|
||||||
|
|
||||||
class PodcastResearchInsight(BaseModel):
|
class PodcastResearchInsight(BaseModel):
|
||||||
@@ -155,6 +168,30 @@ class PodcastResearchInsight(BaseModel):
|
|||||||
title: str
|
title: str
|
||||||
content: str
|
content: str
|
||||||
source_indices: List[int] = []
|
source_indices: List[int] = []
|
||||||
|
podcast_talking_points: Optional[List[str]] = [] # Talking points for host to expand on
|
||||||
|
expert_quotes: Optional[List[Dict[str, str]]] = [] # Quotes from sources
|
||||||
|
listener_cta_suggestions: Optional[List[str]] = [] # CTA suggestions
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastResearchOutput(BaseModel):
|
||||||
|
"""Structured JSON output for LLM research extraction using json_struct."""
|
||||||
|
summary: str = ""
|
||||||
|
key_insights: List[PodcastResearchInsight] = []
|
||||||
|
expert_quotes: List[Dict[str, Any]] = [] # [{"quote": str, "source_index": int, "context": str}]
|
||||||
|
listener_cta_suggestions: List[str] = [] # List of CTA suggestions
|
||||||
|
mapped_angles: List[Dict[str, Any]] = [] # [{"title": str, "why": str, "mapped_fact_ids": []}]
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastCostBreakdownItem(BaseModel):
|
||||||
|
phase: Literal["Analyze", "Gather", "Write", "Produce"]
|
||||||
|
cost: float
|
||||||
|
|
||||||
|
|
||||||
|
class PodcastCostEst(BaseModel):
|
||||||
|
total: float
|
||||||
|
breakdown: List[PodcastCostBreakdownItem]
|
||||||
|
currency: Literal["USD"] = "USD"
|
||||||
|
last_updated: datetime
|
||||||
|
|
||||||
|
|
||||||
class PodcastExaResearchResponse(BaseModel):
|
class PodcastExaResearchResponse(BaseModel):
|
||||||
@@ -162,10 +199,13 @@ class PodcastExaResearchResponse(BaseModel):
|
|||||||
search_queries: List[str] = []
|
search_queries: List[str] = []
|
||||||
summary: str = ""
|
summary: str = ""
|
||||||
key_insights: List[PodcastResearchInsight] = []
|
key_insights: List[PodcastResearchInsight] = []
|
||||||
cost: Optional[Dict[str, Any]] = None
|
cost_est: PodcastCostEst
|
||||||
search_type: Optional[str] = None
|
search_type: Optional[str] = None
|
||||||
provider: str = "exa"
|
provider: str = "exa"
|
||||||
content: Optional[str] = None # Raw aggregated content (deprecated)
|
content: Optional[str] = None # Raw aggregated content (deprecated)
|
||||||
|
mapped_angles: List[Dict[str, Any]] = [] # Content angles for the episode
|
||||||
|
expert_quotes: List[Dict[str, Any]] = [] # Expert quotes from research
|
||||||
|
listener_cta_suggestions: List[str] = [] # CTA suggestions
|
||||||
|
|
||||||
|
|
||||||
class PodcastScriptResponse(BaseModel):
|
class PodcastScriptResponse(BaseModel):
|
||||||
@@ -178,6 +218,7 @@ class PodcastAudioRequest(BaseModel):
|
|||||||
scene_title: str
|
scene_title: str
|
||||||
text: str
|
text: str
|
||||||
voice_id: Optional[str] = "Wise_Woman"
|
voice_id: Optional[str] = "Wise_Woman"
|
||||||
|
custom_voice_id: Optional[str] = None # Voice clone ID for custom voice
|
||||||
speed: Optional[float] = 1.0
|
speed: Optional[float] = 1.0
|
||||||
volume: Optional[float] = 1.0
|
volume: Optional[float] = 1.0
|
||||||
pitch: Optional[float] = 0.0
|
pitch: Optional[float] = 0.0
|
||||||
@@ -263,7 +304,9 @@ class PodcastImageRequest(BaseModel):
|
|||||||
scene_id: str
|
scene_id: str
|
||||||
scene_title: str
|
scene_title: str
|
||||||
scene_content: Optional[str] = None # Optional: scene lines text for context
|
scene_content: Optional[str] = None # Optional: scene lines text for context
|
||||||
|
scene_emotion: Optional[str] = None # Optional: scene emotion for visual tone
|
||||||
idea: Optional[str] = None # Optional: podcast idea for context
|
idea: Optional[str] = None # Optional: podcast idea for context
|
||||||
|
analysis: Optional[Dict[str, Any]] = Field(None, description="AI analysis for visual context (keywords, audience)")
|
||||||
base_avatar_url: Optional[str] = None # Base avatar image URL for scene variations
|
base_avatar_url: Optional[str] = None # Base avatar image URL for scene variations
|
||||||
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||||
width: int = 1024
|
width: int = 1024
|
||||||
@@ -285,6 +328,7 @@ class PodcastImageResponse(BaseModel):
|
|||||||
provider: str
|
provider: str
|
||||||
model: Optional[str] = None
|
model: Optional[str] = None
|
||||||
cost: float
|
cost: float
|
||||||
|
image_prompt: Optional[str] = None # Return the prompt used for generation
|
||||||
|
|
||||||
|
|
||||||
class PodcastVideoGenerationRequest(BaseModel):
|
class PodcastVideoGenerationRequest(BaseModel):
|
||||||
@@ -295,6 +339,9 @@ class PodcastVideoGenerationRequest(BaseModel):
|
|||||||
audio_url: str = Field(..., description="URL to the generated audio file")
|
audio_url: str = Field(..., description="URL to the generated audio file")
|
||||||
avatar_image_url: Optional[str] = Field(None, description="URL to scene image (required for video generation)")
|
avatar_image_url: Optional[str] = Field(None, description="URL to scene image (required for video generation)")
|
||||||
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||||
|
analysis: Optional[Dict[str, Any]] = Field(None, description="Podcast Analysis for context (content type, audience, takeaways, guest)")
|
||||||
|
scene_image_prompt: Optional[str] = Field(None, description="Original image generation prompt for visual context")
|
||||||
|
scene_narration: Optional[str] = Field(None, description="Scene narration/script lines for context")
|
||||||
resolution: str = Field("720p", description="Video resolution (480p or 720p)")
|
resolution: str = Field("720p", description="Video resolution (480p or 720p)")
|
||||||
prompt: Optional[str] = Field(None, description="Optional animation prompt override")
|
prompt: Optional[str] = Field(None, description="Optional animation prompt override")
|
||||||
seed: Optional[int] = Field(-1, description="Random seed; -1 for random")
|
seed: Optional[int] = Field(-1, description="Random seed; -1 for random")
|
||||||
@@ -416,4 +463,3 @@ class VoiceCloneResult(BaseModel):
|
|||||||
file_size: int
|
file_size: int
|
||||||
task_id: str
|
task_id: str
|
||||||
status: str = "completed"
|
status: str = "completed"
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ Podcast Maker API Router
|
|||||||
Main router that imports and registers all handler modules.
|
Main router that imports and registers all handler modules.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
from typing import Dict, Any
|
from typing import Dict, Any
|
||||||
|
|
||||||
from middleware.auth_middleware import get_current_user
|
from middleware.auth_middleware import get_current_user
|
||||||
@@ -32,5 +32,8 @@ router.include_router(dubbing.router)
|
|||||||
@router.get("/task/{task_id}/status")
|
@router.get("/task/{task_id}/status")
|
||||||
async def podcast_task_status(task_id: str, current_user: Dict[str, Any] = Depends(get_current_user)):
|
async def podcast_task_status(task_id: str, current_user: Dict[str, Any] = Depends(get_current_user)):
|
||||||
"""Expose task status under podcast namespace (reuses shared task manager)."""
|
"""Expose task status under podcast namespace (reuses shared task manager)."""
|
||||||
require_authenticated_user(current_user)
|
user_id = require_authenticated_user(current_user)
|
||||||
return task_manager.get_task_status(task_id)
|
task_status = task_manager.get_task_status(task_id, requester_user_id=user_id)
|
||||||
|
if not task_status:
|
||||||
|
raise HTTPException(status_code=404, detail="Task not found")
|
||||||
|
return task_status
|
||||||
|
|||||||
@@ -34,9 +34,14 @@ class TaskManager:
|
|||||||
del self.task_storage[task_id]
|
del self.task_storage[task_id]
|
||||||
logger.debug(f"[StoryWriter] Cleaned up old task: {task_id}")
|
logger.debug(f"[StoryWriter] Cleaned up old task: {task_id}")
|
||||||
|
|
||||||
def create_task(self, task_type: str = "story_generation") -> str:
|
def create_task(
|
||||||
|
self,
|
||||||
|
task_type: str = "story_generation",
|
||||||
|
metadata: Optional[Dict[str, Any]] = None,
|
||||||
|
) -> str:
|
||||||
"""Create a new task and return its ID."""
|
"""Create a new task and return its ID."""
|
||||||
task_id = str(uuid.uuid4())
|
task_id = str(uuid.uuid4())
|
||||||
|
task_metadata = metadata or {}
|
||||||
|
|
||||||
self.task_storage[task_id] = {
|
self.task_storage[task_id] = {
|
||||||
"status": "pending",
|
"status": "pending",
|
||||||
@@ -45,13 +50,14 @@ class TaskManager:
|
|||||||
"error": None,
|
"error": None,
|
||||||
"progress_messages": [],
|
"progress_messages": [],
|
||||||
"task_type": task_type,
|
"task_type": task_type,
|
||||||
"progress": 0.0
|
"progress": 0.0,
|
||||||
|
"metadata": task_metadata,
|
||||||
}
|
}
|
||||||
|
|
||||||
logger.info(f"[StoryWriter] Created task: {task_id} (type: {task_type})")
|
logger.info(f"[StoryWriter] Created task: {task_id} (type: {task_type})")
|
||||||
return task_id
|
return task_id
|
||||||
|
|
||||||
def get_task_status(self, task_id: str) -> Optional[Dict[str, Any]]:
|
def get_task_status(self, task_id: str, requester_user_id: Optional[str] = None) -> Optional[Dict[str, Any]]:
|
||||||
"""Get the status of a task."""
|
"""Get the status of a task."""
|
||||||
self.cleanup_old_tasks()
|
self.cleanup_old_tasks()
|
||||||
|
|
||||||
@@ -62,6 +68,15 @@ class TaskManager:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
task = self.task_storage[task_id]
|
task = self.task_storage[task_id]
|
||||||
|
metadata = task.get("metadata", {}) or {}
|
||||||
|
owner_user_id = metadata.get("owner_user_id")
|
||||||
|
|
||||||
|
if requester_user_id is not None and owner_user_id is not None and requester_user_id != owner_user_id:
|
||||||
|
logger.warning(
|
||||||
|
f"[StoryWriter] Task access denied for task {task_id}: requester does not match owner"
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
response = {
|
response = {
|
||||||
"task_id": task_id,
|
"task_id": task_id,
|
||||||
"status": task["status"],
|
"status": task["status"],
|
||||||
|
|||||||
@@ -8,9 +8,14 @@ def require_authenticated_user(current_user: Dict[str, Any] | None) -> str:
|
|||||||
Validates the current user dictionary provided by Clerk middleware and
|
Validates the current user dictionary provided by Clerk middleware and
|
||||||
returns the normalized user_id. Raises HTTP 401 if authentication fails.
|
returns the normalized user_id. Raises HTTP 401 if authentication fails.
|
||||||
"""
|
"""
|
||||||
if not current_user or not isinstance(current_user, dict):
|
# Guard against dependency injection issues where Depends object might be passed
|
||||||
|
if current_user is None or not isinstance(current_user, dict):
|
||||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Authentication required")
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Authentication required")
|
||||||
|
|
||||||
|
# Additional check: ensure it's actually a dict and not a Depends object or other type
|
||||||
|
if not hasattr(current_user, 'get') or not callable(getattr(current_user, 'get')):
|
||||||
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid authentication context")
|
||||||
|
|
||||||
user_id = str(current_user.get("id", "")).strip()
|
user_id = str(current_user.get("id", "")).strip()
|
||||||
if not user_id:
|
if not user_id:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
Pre-flight check endpoints for operation validation and cost estimation.
|
Pre-flight check endpoints for operation validation and cost estimation.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import time
|
||||||
from fastapi import APIRouter, Depends, HTTPException
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
from typing import Dict, Any
|
from typing import Dict, Any
|
||||||
@@ -34,6 +35,7 @@ async def preflight_check(
|
|||||||
|
|
||||||
Uses caching to minimize DB load (< 100ms with cache hit).
|
Uses caching to minimize DB load (< 100ms with cache hit).
|
||||||
"""
|
"""
|
||||||
|
start_time = time.time()
|
||||||
try:
|
try:
|
||||||
user_id = get_user_id_from_token(current_user)
|
user_id = get_user_id_from_token(current_user)
|
||||||
|
|
||||||
@@ -229,13 +231,19 @@ async def preflight_check(
|
|||||||
'remaining': max(0, video_limit - video_current) if video_limit > 0 else float('inf')
|
'remaining': max(0, video_limit - video_current) if video_limit > 0 else float('inf')
|
||||||
}
|
}
|
||||||
|
|
||||||
|
elapsed_ms = (time.time() - start_time) * 1000
|
||||||
|
logger.warning(f"[PreflightCheck] Completed in {elapsed_ms:.0f}ms for user {user_id}")
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"success": True,
|
"success": True,
|
||||||
"data": response_data
|
"data": response_data
|
||||||
}
|
}
|
||||||
|
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
|
elapsed_ms = (time.time() - start_time) * 1000
|
||||||
|
logger.warning(f"[PreflightCheck] HTTP error after {elapsed_ms:.0f}ms")
|
||||||
raise
|
raise
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error in pre-flight check: {e}", exc_info=True)
|
elapsed_ms = (time.time() - start_time) * 1000
|
||||||
|
logger.error(f"[PreflightCheck] Error after {elapsed_ms:.0f}ms: {e}")
|
||||||
raise HTTPException(status_code=500, detail=f"Pre-flight check failed: {str(e)}")
|
raise HTTPException(status_code=500, detail=f"Pre-flight check failed: {str(e)}")
|
||||||
|
|||||||
771
backend/app.py
771
backend/app.py
@@ -1,6 +1,12 @@
|
|||||||
# Ensure typing constructs and models are available globally for FastAPI type annotation evaluation
|
# Ensure typing constructs and models are available globally for FastAPI type annotation evaluation
|
||||||
|
import os
|
||||||
|
|
||||||
|
# Print env vars immediately - BEFORE any imports
|
||||||
|
print(f"[app.py] EARLY - PORT={os.getenv('PORT')}, HOST={os.getenv('HOST')}", flush=True)
|
||||||
|
|
||||||
import typing
|
import typing
|
||||||
import builtins
|
import builtins
|
||||||
|
import builtins
|
||||||
|
|
||||||
# Make common typing constructs available globally
|
# Make common typing constructs available globally
|
||||||
builtins.Optional = typing.Optional
|
builtins.Optional = typing.Optional
|
||||||
@@ -9,10 +15,56 @@ builtins.Dict = typing.Dict
|
|||||||
builtins.Any = typing.Any
|
builtins.Any = typing.Any
|
||||||
builtins.Union = typing.Union
|
builtins.Union = typing.Union
|
||||||
|
|
||||||
# Import onboarding models VERY early to ensure they're available before any services
|
# Load environment variables FIRST before any other imports
|
||||||
|
from pathlib import Path
|
||||||
|
from dotenv import load_dotenv
|
||||||
|
backend_dir = Path(__file__).parent
|
||||||
|
project_root = backend_dir.parent
|
||||||
|
|
||||||
|
# Load .env but DON'T override existing environment variables (especially PORT from Render)
|
||||||
|
# Use override=False to preserve Render-provided PORT
|
||||||
|
load_dotenv(backend_dir / '.env', override=False)
|
||||||
|
load_dotenv(project_root / '.env', override=False)
|
||||||
|
load_dotenv(override=False)
|
||||||
|
|
||||||
|
# Set LOG_LEVEL early to WARNING to suppress DEBUG persona logs in podcast mode
|
||||||
|
import os
|
||||||
|
if os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast":
|
||||||
|
os.environ["LOG_LEVEL"] = "WARNING"
|
||||||
|
|
||||||
|
print(f"[app.py] Starting... ALWRITY_ENABLED_FEATURES={os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||||
|
|
||||||
|
|
||||||
|
def get_enabled_features() -> set:
|
||||||
|
"""Get enabled features from ALWRITY_ENABLED_FEATURES env var."""
|
||||||
|
env_value = os.getenv("ALWRITY_ENABLED_FEATURES", "all").strip().lower()
|
||||||
|
if not env_value or env_value == "all":
|
||||||
|
return {"all"}
|
||||||
|
return {f.strip() for f in env_value.split(",") if f.strip()}
|
||||||
|
|
||||||
|
|
||||||
|
# Print env var IMMEDIATELY at module start
|
||||||
|
print(f"[app.py] ALWRITY_ENABLED_FEATURES at start: {os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||||
|
|
||||||
|
def is_podcast_only_demo_mode() -> bool:
|
||||||
|
"""Check if podcast-only mode is enabled."""
|
||||||
|
import os
|
||||||
|
env_val = os.getenv("ALWRITY_ENABLED_FEATURES", "all")
|
||||||
|
enabled = get_enabled_features()
|
||||||
|
result = "podcast" in enabled and "all" not in enabled
|
||||||
|
print(f"[DEBUG] is_podcast_only_demo_mode: ALWRITY_ENABLED_FEATURES={env_val}, enabled={enabled}, result={result}", flush=True)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# Podcast-only check BEFORE heavy imports
|
||||||
|
PODCAST_ONLY_DEMO_MODE = is_podcast_only_demo_mode()
|
||||||
|
|
||||||
|
|
||||||
|
# Import onboarding models (after env is loaded, before heavy imports)
|
||||||
from models.onboarding import APIKey, WebsiteAnalysis, ResearchPreferences, PersonaData, CompetitorAnalysis
|
from models.onboarding import APIKey, WebsiteAnalysis, ResearchPreferences, PersonaData, CompetitorAnalysis
|
||||||
|
|
||||||
|
|
||||||
|
# Import FastAPI and related
|
||||||
from fastapi import FastAPI, HTTPException, Depends, Request, BackgroundTasks
|
from fastapi import FastAPI, HTTPException, Depends, Request, BackgroundTasks
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
@@ -20,33 +72,45 @@ from fastapi.responses import FileResponse
|
|||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
from typing import Dict, Any, Optional
|
from typing import Dict, Any, Optional
|
||||||
import os
|
import os
|
||||||
from loguru import logger
|
|
||||||
from dotenv import load_dotenv
|
|
||||||
import asyncio
|
import asyncio
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
# Import OnboardingSession right after basic imports to ensure it's available
|
def _log_memory_usage():
|
||||||
from models.onboarding import OnboardingSession
|
try:
|
||||||
|
import psutil
|
||||||
|
mem_mb = psutil.Process().memory_info().rss // (1024 * 1024)
|
||||||
|
logger.info(f"Memory usage (MB): {mem_mb}")
|
||||||
|
except Exception:
|
||||||
|
# psutil not available or failed; skip silently
|
||||||
|
pass
|
||||||
|
|
||||||
from services.subscription import monitoring_middleware
|
# Log memory early in app.py startup
|
||||||
|
_log_memory_usage()
|
||||||
|
logger.info("app.py: Early memory checkpoint after env load")
|
||||||
|
|
||||||
# Import remaining onboarding models
|
|
||||||
from models import APIKey, WebsiteAnalysis, ResearchPreferences, PersonaData, CompetitorAnalysis
|
|
||||||
|
|
||||||
# Import modular utilities
|
# Import modular utilities (skip OnboardingManager import in podcast-only mode)
|
||||||
from alwrity_utils import HealthChecker, RateLimiter, FrontendServing, RouterManager
|
from alwrity_utils import HealthChecker, RateLimiter, FrontendServing, RouterManager
|
||||||
from alwrity_utils import OnboardingManager
|
if not is_podcast_only_demo_mode():
|
||||||
|
from alwrity_utils import OnboardingManager
|
||||||
|
|
||||||
# Load environment variables
|
# Skip monitoring middleware in podcast-only mode to save memory
|
||||||
# Try multiple locations for .env file
|
if not is_podcast_only_demo_mode():
|
||||||
from pathlib import Path
|
from services.subscription import monitoring_middleware
|
||||||
backend_dir = Path(__file__).parent
|
else:
|
||||||
project_root = backend_dir.parent
|
monitoring_middleware = None
|
||||||
|
|
||||||
|
|
||||||
|
def should_include_non_podcast_features() -> bool:
|
||||||
|
"""Check if non-podcast features should be included."""
|
||||||
|
enabled = get_enabled_features()
|
||||||
|
return "all" in enabled or "core" in enabled
|
||||||
|
|
||||||
|
|
||||||
|
# Legacy constant for backwards compatibility
|
||||||
|
PODCAST_ONLY_DEMO_MODE = is_podcast_only_demo_mode()
|
||||||
|
|
||||||
# Load from backend/.env first (higher priority), then root .env
|
|
||||||
load_dotenv(backend_dir / '.env') # backend/.env
|
|
||||||
load_dotenv(project_root / '.env') # root .env (fallback)
|
|
||||||
load_dotenv() # CWD .env (fallback)
|
|
||||||
|
|
||||||
# Set up clean logging for end users
|
# Set up clean logging for end users
|
||||||
from logging_config import setup_clean_logging
|
from logging_config import setup_clean_logging
|
||||||
@@ -55,47 +119,73 @@ setup_clean_logging()
|
|||||||
# Import middleware
|
# Import middleware
|
||||||
from middleware.auth_middleware import get_current_user
|
from middleware.auth_middleware import get_current_user
|
||||||
|
|
||||||
# Import component logic endpoints (needs OnboardingSession, so import after models)
|
# Import component logic endpoints (skip in podcast-only mode - uses seo_analyzer)
|
||||||
from api.component_logic import router as component_logic_router
|
component_logic_router = None
|
||||||
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
|
from api.component_logic import router as component_logic_router
|
||||||
|
|
||||||
# Import subscription API endpoints
|
# Import subscription API endpoints
|
||||||
from api.subscription import router as subscription_router
|
from api.subscription import router as subscription_router
|
||||||
|
|
||||||
# Import Step 3 onboarding routes
|
# Import Step 3 onboarding routes (skip in podcast-only mode)
|
||||||
from api.onboarding_utils.step3_routes import router as step3_routes
|
step3_routes = None
|
||||||
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
|
from api.onboarding_utils.step3_routes import router as step3_routes
|
||||||
|
|
||||||
# Import SEO tools router
|
# Import SEO tools router (skip in podcast-only mode - uses seo_analyzer)
|
||||||
from routers.seo_tools import router as seo_tools_router
|
seo_tools_router = None
|
||||||
# Import Facebook Writer endpoints
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
from api.facebook_writer.routers import facebook_router
|
from routers.seo_tools import router as seo_tools_router
|
||||||
# Import LinkedIn content generation router
|
|
||||||
from routers.linkedin import router as linkedin_router
|
|
||||||
# Import LinkedIn image generation router
|
|
||||||
from api.linkedin_image_generation import router as linkedin_image_router
|
|
||||||
from api.brainstorm import router as brainstorm_router
|
|
||||||
from api.images import router as images_router
|
|
||||||
from api.assets_serving import router as assets_serving_router
|
|
||||||
from routers.image_studio import router as image_studio_router
|
|
||||||
from routers.product_marketing import router as product_marketing_router
|
|
||||||
from routers.campaign_creator import router as campaign_creator_router
|
|
||||||
|
|
||||||
# Import hallucination detector router
|
# Skip Facebook Writer, LinkedIn, and other non-podcast routes in podcast-only mode
|
||||||
from api.hallucination_detector import router as hallucination_detector_router
|
# Also skip other heavy services that trigger PersonaAnalysisService initialization
|
||||||
from api.writing_assistant import router as writing_assistant_router
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
|
from api.facebook_writer.routers import facebook_router
|
||||||
|
from routers.linkedin import router as linkedin_router
|
||||||
|
from api.linkedin_image_generation import router as linkedin_image_router
|
||||||
|
from api.brainstorm import router as brainstorm_router
|
||||||
|
from api.images import router as images_router
|
||||||
|
from api.assets_serving import router as assets_serving_router
|
||||||
|
from routers.image_studio import router as image_studio_router
|
||||||
|
from routers.product_marketing import router as product_marketing_router
|
||||||
|
from routers.campaign_creator import router as campaign_creator_router
|
||||||
|
else:
|
||||||
|
# In podcast-only mode, only load essential podcast assets router
|
||||||
|
from api.assets_serving import router as assets_serving_router
|
||||||
|
brainstorm_router = None
|
||||||
|
images_router = None
|
||||||
|
image_studio_router = None
|
||||||
|
product_marketing_router = None
|
||||||
|
campaign_creator_router = None
|
||||||
|
|
||||||
# Import research configuration router
|
# Import hallucination detector router (skip in podcast-only mode - triggers heavy ML)
|
||||||
from api.research_config import router as research_config_router
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
|
from api.hallucination_detector import router as hallucination_detector_router
|
||||||
|
from api.writing_assistant import router as writing_assistant_router
|
||||||
|
else:
|
||||||
|
hallucination_detector_router = None
|
||||||
|
writing_assistant_router = None
|
||||||
|
|
||||||
|
# Import research configuration router (skip in podcast-only mode)
|
||||||
|
if not is_podcast_only_demo_mode():
|
||||||
|
from api.research_config import router as research_config_router
|
||||||
|
else:
|
||||||
|
research_config_router = None
|
||||||
|
|
||||||
# Import user data endpoints
|
# Import user data endpoints
|
||||||
# Import content planning endpoints
|
# Import content planning endpoints (skip in podcast-only mode)
|
||||||
from api.content_planning.api.router import router as content_planning_router
|
if not is_podcast_only_demo_mode():
|
||||||
from api.user_data import router as user_data_router
|
from api.content_planning.api.router import router as content_planning_router
|
||||||
|
from api.content_planning.strategy_copilot import router as strategy_copilot_router
|
||||||
|
else:
|
||||||
|
content_planning_router = None
|
||||||
|
strategy_copilot_router = None
|
||||||
|
|
||||||
# Import user environment endpoints
|
# Import user data endpoints (skip in podcast-only mode to save memory)
|
||||||
from api.user_environment import router as user_environment_router
|
if not is_podcast_only_demo_mode():
|
||||||
|
from api.user_data import router as user_data_router
|
||||||
# Import strategy copilot endpoints
|
else:
|
||||||
from api.content_planning.strategy_copilot import router as strategy_copilot_router
|
user_data_router = None
|
||||||
|
|
||||||
# Import database service
|
# Import database service
|
||||||
from services.database import close_database
|
from services.database import close_database
|
||||||
@@ -107,39 +197,71 @@ from services.startup_health import (
|
|||||||
|
|
||||||
# Trigger reload for monitoring fix
|
# Trigger reload for monitoring fix
|
||||||
|
|
||||||
# Import OAuth token monitoring routes
|
# Import OAuth token monitoring routes (skip in podcast-only mode)
|
||||||
from api.oauth_token_monitoring_routes import router as oauth_token_monitoring_router
|
if not is_podcast_only_demo_mode():
|
||||||
|
from api.oauth_token_monitoring_routes import router as oauth_token_monitoring_router
|
||||||
|
else:
|
||||||
|
oauth_token_monitoring_router = None
|
||||||
|
|
||||||
# Import SEO Dashboard endpoints
|
# Import SEO Dashboard endpoints (skip in podcast-only mode to save memory)
|
||||||
from api.seo_dashboard import (
|
if not is_podcast_only_demo_mode():
|
||||||
get_seo_dashboard_data,
|
from api.seo_dashboard import (
|
||||||
get_seo_health_score,
|
get_seo_dashboard_data,
|
||||||
get_seo_metrics,
|
get_seo_health_score,
|
||||||
get_platform_status,
|
get_seo_metrics,
|
||||||
get_ai_insights,
|
get_platform_status,
|
||||||
seo_dashboard_health_check,
|
get_ai_insights,
|
||||||
analyze_seo_comprehensive,
|
seo_dashboard_health_check,
|
||||||
analyze_seo_full,
|
analyze_seo_comprehensive,
|
||||||
get_seo_metrics_detailed,
|
analyze_seo_full,
|
||||||
get_analysis_summary,
|
get_seo_metrics_detailed,
|
||||||
batch_analyze_urls,
|
get_analysis_summary,
|
||||||
SEOAnalysisRequest,
|
batch_analyze_urls,
|
||||||
get_seo_dashboard_overview,
|
SEOAnalysisRequest,
|
||||||
get_gsc_raw_data,
|
get_seo_dashboard_overview,
|
||||||
get_bing_raw_data,
|
get_gsc_raw_data,
|
||||||
get_competitive_insights,
|
get_bing_raw_data,
|
||||||
get_deep_competitor_analysis,
|
get_competitive_insights,
|
||||||
run_strategic_insights,
|
get_deep_competitor_analysis,
|
||||||
get_strategic_insights_history,
|
run_strategic_insights,
|
||||||
refresh_analytics_data,
|
get_strategic_insights_history,
|
||||||
analyze_urls_ai,
|
refresh_analytics_data,
|
||||||
AnalyzeURLsRequest,
|
analyze_urls_ai,
|
||||||
get_analyzed_pages,
|
AnalyzeURLsRequest,
|
||||||
get_semantic_health,
|
get_analyzed_pages,
|
||||||
get_semantic_cache_stats,
|
get_semantic_health,
|
||||||
get_sif_indexing_health,
|
get_semantic_cache_stats,
|
||||||
get_onboarding_task_health,
|
get_sif_indexing_health,
|
||||||
)
|
get_onboarding_task_health,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
get_seo_dashboard_data = None
|
||||||
|
get_seo_health_score = None
|
||||||
|
get_seo_metrics = None
|
||||||
|
get_platform_status = None
|
||||||
|
get_ai_insights = None
|
||||||
|
seo_dashboard_health_check = None
|
||||||
|
analyze_seo_comprehensive = None
|
||||||
|
analyze_seo_full = None
|
||||||
|
get_seo_metrics_detailed = None
|
||||||
|
get_analysis_summary = None
|
||||||
|
batch_analyze_urls = None
|
||||||
|
SEOAnalysisRequest = None
|
||||||
|
get_seo_dashboard_overview = None
|
||||||
|
get_gsc_raw_data = None
|
||||||
|
get_bing_raw_data = None
|
||||||
|
get_competitive_insights = None
|
||||||
|
get_deep_competitor_analysis = None
|
||||||
|
run_strategic_insights = None
|
||||||
|
get_strategic_insights_history = None
|
||||||
|
refresh_analytics_data = None
|
||||||
|
analyze_urls_ai = None
|
||||||
|
AnalyzeURLsRequest = None
|
||||||
|
get_analyzed_pages = None
|
||||||
|
get_semantic_health = None
|
||||||
|
get_semantic_cache_stats = None
|
||||||
|
get_sif_indexing_health = None
|
||||||
|
get_onboarding_task_health = None
|
||||||
|
|
||||||
|
|
||||||
# Initialize FastAPI app
|
# Initialize FastAPI app
|
||||||
@@ -156,12 +278,23 @@ default_allowed_origins = [
|
|||||||
"http://localhost:8000", # Backend dev server
|
"http://localhost:8000", # Backend dev server
|
||||||
"http://localhost:3001", # Alternative React port
|
"http://localhost:3001", # Alternative React port
|
||||||
"https://alwrity-ai.vercel.app", # Vercel frontend
|
"https://alwrity-ai.vercel.app", # Vercel frontend
|
||||||
|
"https://alwrity-5vac2n9su-ajsis-projects.vercel.app", # Current Vercel deployment
|
||||||
|
"https://alwrity.vercel.app", # Vercel app
|
||||||
]
|
]
|
||||||
|
|
||||||
# Optional dynamic origins from environment (comma-separated)
|
# Optional dynamic origins from environment (comma-separated)
|
||||||
env_origins = os.getenv("ALWRITY_ALLOWED_ORIGINS", "").split(",") if os.getenv("ALWRITY_ALLOWED_ORIGINS") else []
|
env_origins = os.getenv("ALWRITY_ALLOWED_ORIGINS", "").split(",") if os.getenv("ALWRITY_ALLOWED_ORIGINS") else []
|
||||||
env_origins = [o.strip() for o in env_origins if o.strip()]
|
env_origins = [o.strip() for o in env_origins if o.strip()]
|
||||||
|
|
||||||
|
# Convenience: NGROK_URL env var (single origin)
|
||||||
|
ngrok_origin = os.getenv("NGROK_URL")
|
||||||
|
if ngrok_origin:
|
||||||
|
env_origins.append(ngrok_origin.strip())
|
||||||
|
|
||||||
|
# Optional dynamic origins from environment (comma-separated)
|
||||||
|
env_origins = os.getenv("ALWRITY_ALLOWED_ORIGINS", "").split(",") if os.getenv("ALWRITY_ALLOWED_ORIGINS") else []
|
||||||
|
env_origins = [o.strip() for o in env_origins if o.strip()]
|
||||||
|
|
||||||
# Convenience: NGROK_URL env var (single origin)
|
# Convenience: NGROK_URL env var (single origin)
|
||||||
ngrok_origin = os.getenv("NGROK_URL")
|
ngrok_origin = os.getenv("NGROK_URL")
|
||||||
if ngrok_origin:
|
if ngrok_origin:
|
||||||
@@ -182,15 +315,21 @@ health_checker = HealthChecker()
|
|||||||
rate_limiter = RateLimiter(window_seconds=60, max_requests=200)
|
rate_limiter = RateLimiter(window_seconds=60, max_requests=200)
|
||||||
frontend_serving = FrontendServing(app)
|
frontend_serving = FrontendServing(app)
|
||||||
router_manager = RouterManager(app)
|
router_manager = RouterManager(app)
|
||||||
|
router_group_status: Dict[str, Dict[str, Any]] = {}
|
||||||
|
|
||||||
onboarding_manager = OnboardingManager(app)
|
onboarding_manager = None
|
||||||
|
# Only create OnboardingManager if NOT in podcast-only mode
|
||||||
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
|
from alwrity_utils import OnboardingManager
|
||||||
|
onboarding_manager = OnboardingManager(app)
|
||||||
|
|
||||||
# Middleware Order (FastAPI executes in REVERSE order of registration - LIFO):
|
# Middleware Order (FastAPI executes in REVERSE order of registration - LIFO):
|
||||||
# Registration order: 1. Monitoring 2. Rate Limit 3. API Key Injection
|
# Registration order: 1. Monitoring 2. Rate Limit 3. API Key Injection
|
||||||
# Execution order: 1. API Key Injection (sets user_id) 2. Rate Limit 3. Monitoring (uses user_id)
|
# Execution order: 1. API Key Injection (sets user_id) 2. Rate Limit 3. Monitoring (uses user_id)
|
||||||
|
|
||||||
# 1. FIRST REGISTERED (runs LAST) - Monitoring middleware
|
# 1. FIRST REGISTERED (runs LAST) - Monitoring middleware (skip in podcast-only mode)
|
||||||
app.middleware("http")(monitoring_middleware)
|
if monitoring_middleware:
|
||||||
|
app.middleware("http")(monitoring_middleware)
|
||||||
|
|
||||||
# 2. SECOND REGISTERED (runs SECOND) - Rate limiting
|
# 2. SECOND REGISTERED (runs SECOND) - Rate limiting
|
||||||
@app.middleware("http")
|
@app.middleware("http")
|
||||||
@@ -206,7 +345,9 @@ app.middleware("http")(api_key_injection_middleware)
|
|||||||
@app.get("/health")
|
@app.get("/health")
|
||||||
async def health():
|
async def health():
|
||||||
"""Health check endpoint."""
|
"""Health check endpoint."""
|
||||||
return health_checker.basic_health_check()
|
health_data = health_checker.basic_health_check()
|
||||||
|
health_data["podcast_only_demo_mode"] = PODCAST_ONLY_DEMO_MODE
|
||||||
|
return health_data
|
||||||
|
|
||||||
@app.get("/health/database")
|
@app.get("/health/database")
|
||||||
async def database_health():
|
async def database_health():
|
||||||
@@ -222,6 +363,7 @@ async def comprehensive_health():
|
|||||||
async def readiness(current_user: dict = Depends(get_current_user)):
|
async def readiness(current_user: dict = Depends(get_current_user)):
|
||||||
"""Readiness check that validates tenant DB resolution/session under auth context."""
|
"""Readiness check that validates tenant DB resolution/session under auth context."""
|
||||||
return {
|
return {
|
||||||
|
"podcast_only_demo_mode": PODCAST_ONLY_DEMO_MODE,
|
||||||
"startup": get_startup_status(),
|
"startup": get_startup_status(),
|
||||||
"tenant": readiness_under_auth_context(current_user),
|
"tenant": readiness_under_auth_context(current_user),
|
||||||
}
|
}
|
||||||
@@ -250,203 +392,285 @@ async def frontend_status():
|
|||||||
@app.get("/api/routers/status")
|
@app.get("/api/routers/status")
|
||||||
async def router_status():
|
async def router_status():
|
||||||
"""Get router inclusion status."""
|
"""Get router inclusion status."""
|
||||||
return router_manager.get_router_status()
|
status = router_manager.get_router_status()
|
||||||
|
status.update(
|
||||||
|
{
|
||||||
|
"podcast_only_demo_mode": PODCAST_ONLY_DEMO_MODE,
|
||||||
|
"router_groups": router_group_status,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return status
|
||||||
|
|
||||||
|
@app.get("/api/feature-profile/status")
|
||||||
|
async def feature_profile_status():
|
||||||
|
"""Get feature profile status and enabled modules."""
|
||||||
|
return router_manager.get_feature_profile_status()
|
||||||
|
|
||||||
# Onboarding management endpoints
|
# Onboarding management endpoints
|
||||||
@app.get("/api/onboarding/status")
|
@app.get("/api/onboarding/status")
|
||||||
async def onboarding_status():
|
async def onboarding_status():
|
||||||
"""Get onboarding manager status."""
|
"""Get onboarding manager status (or demo-mode disabled state)."""
|
||||||
|
if PODCAST_ONLY_DEMO_MODE:
|
||||||
|
return {
|
||||||
|
"enabled": False,
|
||||||
|
"status": "disabled",
|
||||||
|
"message": "Onboarding is disabled for podcast-only demo mode.",
|
||||||
|
"demo_mode": "podcast_only",
|
||||||
|
}
|
||||||
return onboarding_manager.get_onboarding_status()
|
return onboarding_manager.get_onboarding_status()
|
||||||
|
|
||||||
# Include routers using modular utilities
|
# Include routers using modular utilities
|
||||||
router_manager.include_core_routers()
|
if PODCAST_ONLY_DEMO_MODE:
|
||||||
router_manager.include_optional_routers()
|
# In podcast-only mode, include only podcast-enabled routers from core registry
|
||||||
|
from alwrity_utils.router_manager import CORE_ROUTER_REGISTRY
|
||||||
|
podcast_routers = [r for r in CORE_ROUTER_REGISTRY if "podcast" in r.get("features", set())]
|
||||||
|
for entry in podcast_routers:
|
||||||
|
try:
|
||||||
|
router = router_manager._load_router_from_registry(entry)
|
||||||
|
router_manager.include_router_safely(router, entry["name"], entry.get("include_kwargs"))
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"{entry['name']} router not mounted: {e}")
|
||||||
|
router_group_status["modular_core"] = {
|
||||||
|
"mounted": True,
|
||||||
|
"reason": "Podcast routers only in podcast-only mode",
|
||||||
|
}
|
||||||
|
router_group_status["modular_optional"] = {
|
||||||
|
"mounted": False,
|
||||||
|
"reason": "Skipped in podcast-only demo mode",
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
router_group_status["modular_core"] = {
|
||||||
|
"mounted": router_manager.include_core_routers(),
|
||||||
|
"reason": "Full mode",
|
||||||
|
}
|
||||||
|
router_group_status["modular_optional"] = {
|
||||||
|
"mounted": router_manager.include_optional_routers(),
|
||||||
|
"reason": "Full mode",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Log startup summary
|
||||||
|
router_manager.log_startup_summary()
|
||||||
|
|
||||||
|
# Safety net: keep subscription routes available even if core inclusion flow changes
|
||||||
|
# in special modes (e.g., demo mode). De-dup is handled by RouterManager.
|
||||||
|
router_manager.include_router_safely(subscription_router, "subscription")
|
||||||
|
|
||||||
# Include assets serving router (must be mounted to serve generated images)
|
# Include assets serving router (must be mounted to serve generated images)
|
||||||
app.include_router(assets_serving_router)
|
app.include_router(assets_serving_router)
|
||||||
|
router_group_status["assets_serving"] = {
|
||||||
|
"mounted": True,
|
||||||
|
"reason": "Required for podcast media assets",
|
||||||
|
}
|
||||||
|
|
||||||
# SEO Dashboard endpoints
|
# SEO Dashboard endpoints (skip in podcast-only mode)
|
||||||
@app.get("/api/seo-dashboard/data")
|
if not is_podcast_only_demo_mode():
|
||||||
async def seo_dashboard_data():
|
@app.get("/api/seo-dashboard/data")
|
||||||
"""Get complete SEO dashboard data."""
|
async def seo_dashboard_data():
|
||||||
return await get_seo_dashboard_data()
|
"""Get complete SEO dashboard data."""
|
||||||
|
return await get_seo_dashboard_data()
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/health-score")
|
@app.get("/api/seo-dashboard/health-score")
|
||||||
async def seo_health_score():
|
async def seo_health_score():
|
||||||
"""Get SEO health score."""
|
"""Get SEO health score."""
|
||||||
return await get_seo_health_score()
|
return await get_seo_health_score()
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/metrics")
|
@app.get("/api/seo-dashboard/metrics")
|
||||||
async def seo_metrics():
|
async def seo_metrics():
|
||||||
"""Get SEO metrics."""
|
"""Get SEO metrics."""
|
||||||
return await get_seo_metrics()
|
return await get_seo_metrics()
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/platforms")
|
@app.get("/api/seo-dashboard/platforms")
|
||||||
async def seo_platforms(current_user: dict = Depends(get_current_user)):
|
async def seo_platforms(current_user: dict = Depends(get_current_user)):
|
||||||
"""Get platform status."""
|
"""Get platform status."""
|
||||||
return await get_platform_status(current_user)
|
return await get_platform_status(current_user)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/insights")
|
@app.get("/api/seo-dashboard/insights")
|
||||||
async def seo_insights():
|
async def seo_insights():
|
||||||
"""Get AI insights."""
|
"""Get AI insights."""
|
||||||
return await get_ai_insights()
|
return await get_ai_insights()
|
||||||
|
|
||||||
# New SEO Dashboard endpoints with real data
|
@app.get("/api/seo-dashboard/overview")
|
||||||
@app.get("/api/seo-dashboard/overview")
|
async def seo_dashboard_overview_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||||
async def seo_dashboard_overview_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
"""Get comprehensive SEO dashboard overview with real GSC/Bing data."""
|
||||||
"""Get comprehensive SEO dashboard overview with real GSC/Bing data."""
|
return await get_seo_dashboard_overview(current_user, site_url)
|
||||||
return await get_seo_dashboard_overview(current_user, site_url)
|
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/gsc/raw")
|
@app.get("/api/seo-dashboard/gsc/raw")
|
||||||
async def gsc_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
async def gsc_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||||
"""Get raw GSC data for the specified site."""
|
"""Get raw GSC data for the specified site."""
|
||||||
return await get_gsc_raw_data(current_user, site_url)
|
return await get_gsc_raw_data(current_user, site_url)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/bing/raw")
|
@app.get("/api/seo-dashboard/bing/raw")
|
||||||
async def bing_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
async def bing_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||||
"""Get raw Bing data for the specified site."""
|
"""Get raw Bing data for the specified site."""
|
||||||
return await get_bing_raw_data(current_user, site_url)
|
return await get_bing_raw_data(current_user, site_url)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/competitive-insights")
|
@app.get("/api/seo-dashboard/competitive-insights")
|
||||||
async def competitive_insights_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
async def competitive_insights_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||||
"""Get competitive insights from onboarding step 3 data."""
|
"""Get competitive insights from onboarding step 3 data."""
|
||||||
return await get_competitive_insights(current_user, site_url)
|
return await get_competitive_insights(current_user, site_url)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/deep-competitor-analysis")
|
@app.get("/api/seo-dashboard/deep-competitor-analysis")
|
||||||
async def deep_competitor_analysis_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
async def deep_competitor_analysis_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||||
"""Get deep competitor analysis results (auto-scheduled post-onboarding)."""
|
"""Get deep competitor analysis results (auto-scheduled post-onboarding)."""
|
||||||
return await get_deep_competitor_analysis(current_user, site_url)
|
return await get_deep_competitor_analysis(current_user, site_url)
|
||||||
|
|
||||||
@app.post("/api/seo-dashboard/strategic-insights/run")
|
@app.post("/api/seo-dashboard/strategic-insights/run")
|
||||||
async def run_strategic_insights_endpoint(current_user: dict = Depends(get_current_user)):
|
async def run_strategic_insights_endpoint(current_user: dict = Depends(get_current_user)):
|
||||||
"""Run AI-powered strategic insights analysis manually."""
|
"""Run AI-powered strategic insights analysis manually."""
|
||||||
return await run_strategic_insights(current_user)
|
return await run_strategic_insights(current_user)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/strategic-insights/history")
|
@app.get("/api/seo-dashboard/strategic-insights/history")
|
||||||
async def get_strategic_insights_history_endpoint(current_user: dict = Depends(get_current_user)):
|
async def get_strategic_insights_history_endpoint(current_user: dict = Depends(get_current_user)):
|
||||||
"""Fetch the history of strategic insights for the user."""
|
"""Fetch the history of strategic insights for the user."""
|
||||||
return await get_strategic_insights_history(current_user)
|
return await get_strategic_insights_history(current_user)
|
||||||
|
|
||||||
@app.post("/api/seo-dashboard/refresh")
|
@app.post("/api/seo-dashboard/refresh")
|
||||||
async def refresh_analytics_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
async def refresh_analytics_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||||
"""Refresh analytics data by invalidating cache and fetching fresh data."""
|
"""Refresh analytics data by invalidating cache and fetching fresh data."""
|
||||||
return await refresh_analytics_data(current_user, site_url)
|
return await refresh_analytics_data(current_user, site_url)
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/api/seo-dashboard/onboarding-task-health")
|
||||||
|
async def onboarding_task_health_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||||
|
"""Get consolidated health for onboarding-scheduled SEO tasks."""
|
||||||
|
return await get_onboarding_task_health(current_user, site_url)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/onboarding-task-health")
|
@app.get("/api/seo-dashboard/health")
|
||||||
async def onboarding_task_health_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
async def seo_dashboard_health():
|
||||||
"""Get consolidated health for onboarding-scheduled SEO tasks."""
|
"""Health check for SEO dashboard."""
|
||||||
return await get_onboarding_task_health(current_user, site_url)
|
return await seo_dashboard_health_check()
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/health")
|
@app.get("/api/seo-dashboard/semantic-health")
|
||||||
async def seo_dashboard_health():
|
async def semantic_health_endpoint(current_user: dict = Depends(get_current_user)):
|
||||||
"""Health check for SEO dashboard."""
|
"""
|
||||||
return await seo_dashboard_health_check()
|
Get real-time semantic health metrics for content and competitors.
|
||||||
|
This endpoint provides Phase 2B semantic intelligence monitoring data.
|
||||||
# Phase 2B: Semantic health monitoring endpoint (24-hour polling)
|
|
||||||
@app.get("/api/seo-dashboard/semantic-health")
|
Returns semantic health score, status, and recommendations.
|
||||||
async def semantic_health_endpoint(current_user: dict = Depends(get_current_user)):
|
Data is cached and updated every 24 hours via scheduler.
|
||||||
"""
|
"""
|
||||||
Get real-time semantic health metrics for content and competitors.
|
return await get_semantic_health(current_user)
|
||||||
This endpoint provides Phase 2B semantic intelligence monitoring data.
|
|
||||||
|
|
||||||
Returns semantic health score, status, and recommendations.
|
|
||||||
Data is cached and updated every 24 hours via scheduler.
|
|
||||||
"""
|
|
||||||
return await get_semantic_health(current_user)
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/cache-stats")
|
@app.get("/api/seo-dashboard/cache-stats")
|
||||||
async def semantic_cache_stats_endpoint(current_user: dict = Depends(get_current_user)):
|
async def semantic_cache_stats_endpoint(current_user: dict = Depends(get_current_user)):
|
||||||
"""
|
"""
|
||||||
Get semantic cache performance statistics.
|
Get semantic cache performance statistics.
|
||||||
Returns hit rate, memory usage, and eviction counts.
|
Returns hit rate, memory usage, and eviction counts.
|
||||||
"""
|
"""
|
||||||
return await get_semantic_cache_stats(current_user)
|
return await get_semantic_cache_stats(current_user)
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/sif-health")
|
@app.get("/api/seo-dashboard/sif-health")
|
||||||
async def sif_indexing_health_endpoint(current_user: dict = Depends(get_current_user)):
|
async def sif_indexing_health_endpoint(current_user: dict = Depends(get_current_user)):
|
||||||
"""
|
"""
|
||||||
Get SIF indexing health summary for the current user.
|
Get SIF indexing health summary for the current user.
|
||||||
Used by the Semantic Indexing Status widget on the dashboard.
|
Used by the Semantic Indexing Status widget on the dashboard.
|
||||||
"""
|
"""
|
||||||
return await get_sif_indexing_health(current_user)
|
return await get_sif_indexing_health(current_user)
|
||||||
|
|
||||||
# Comprehensive SEO Analysis endpoints
|
# Comprehensive SEO Analysis endpoints
|
||||||
@app.post("/api/seo-dashboard/analyze-comprehensive")
|
@app.post("/api/seo-dashboard/analyze-comprehensive")
|
||||||
async def analyze_seo_comprehensive_endpoint(request: SEOAnalysisRequest):
|
async def analyze_seo_comprehensive_endpoint(request: SEOAnalysisRequest):
|
||||||
"""Analyze a URL for comprehensive SEO performance."""
|
"""Analyze a URL for comprehensive SEO performance."""
|
||||||
return await analyze_seo_comprehensive(request)
|
return await analyze_seo_comprehensive(request)
|
||||||
|
|
||||||
@app.post("/api/seo-dashboard/analyze-full")
|
@app.post("/api/seo-dashboard/analyze-full")
|
||||||
async def analyze_seo_full_endpoint(request: SEOAnalysisRequest):
|
async def analyze_seo_full_endpoint(request: SEOAnalysisRequest):
|
||||||
"""Analyze a URL for comprehensive SEO performance."""
|
"""Analyze a URL for comprehensive SEO performance."""
|
||||||
return await analyze_seo_full(request)
|
return await analyze_seo_full(request)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/metrics-detailed")
|
@app.get("/api/seo-dashboard/metrics-detailed")
|
||||||
async def seo_metrics_detailed(url: str):
|
async def seo_metrics_detailed(url: str):
|
||||||
"""Get detailed SEO metrics for a URL."""
|
"""Get detailed SEO metrics for a URL."""
|
||||||
return await get_seo_metrics_detailed(url)
|
return await get_seo_metrics_detailed(url)
|
||||||
|
|
||||||
@app.get("/api/seo-dashboard/analysis-summary")
|
@app.get("/api/seo-dashboard/analysis-summary")
|
||||||
async def seo_analysis_summary(url: str):
|
async def seo_analysis_summary(url: str):
|
||||||
"""Get a quick summary of SEO analysis for a URL."""
|
"""Get a quick summary of SEO analysis for a URL."""
|
||||||
return await get_analysis_summary(url)
|
return await get_analysis_summary(url)
|
||||||
|
|
||||||
@app.post("/api/seo-dashboard/batch-analyze")
|
@app.post("/api/seo-dashboard/batch-analyze")
|
||||||
async def batch_analyze_urls_endpoint(urls: list[str]):
|
async def batch_analyze_urls_endpoint(urls: list[str]):
|
||||||
"""Analyze multiple URLs in batch."""
|
"""Analyze multiple URLs in batch."""
|
||||||
return await batch_analyze_urls(urls)
|
return await batch_analyze_urls(urls)
|
||||||
|
|
||||||
@app.post("/api/seo-dashboard/analyze-urls-ai")
|
@app.post("/api/seo-dashboard/analyze-urls-ai")
|
||||||
async def analyze_urls_ai_endpoint(request: AnalyzeURLsRequest, current_user: dict = Depends(get_current_user)):
|
async def analyze_urls_ai_endpoint(request: AnalyzeURLsRequest, current_user: dict = Depends(get_current_user)):
|
||||||
"""Run AI-powered SEO analysis on selected URLs."""
|
"""Run AI-powered SEO analysis on selected URLs."""
|
||||||
return await analyze_urls_ai(request, current_user)
|
return await analyze_urls_ai(request, current_user)
|
||||||
|
|
||||||
# Include platform analytics router
|
# Include platform analytics router
|
||||||
from routers.platform_analytics import router as platform_analytics_router
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
app.include_router(platform_analytics_router)
|
from routers.platform_analytics import router as platform_analytics_router
|
||||||
# Include Bing Analytics Storage router to expose storage-backed endpoints
|
app.include_router(platform_analytics_router)
|
||||||
from routers.bing_analytics_storage import router as bing_analytics_storage_router
|
# Include Bing Analytics Storage router to expose storage-backed endpoints
|
||||||
app.include_router(bing_analytics_storage_router)
|
from routers.bing_analytics_storage import router as bing_analytics_storage_router
|
||||||
app.include_router(images_router)
|
app.include_router(bing_analytics_storage_router)
|
||||||
app.include_router(image_studio_router)
|
if images_router:
|
||||||
app.include_router(product_marketing_router)
|
app.include_router(images_router)
|
||||||
app.include_router(campaign_creator_router)
|
if image_studio_router:
|
||||||
|
app.include_router(image_studio_router)
|
||||||
|
if product_marketing_router:
|
||||||
|
app.include_router(product_marketing_router)
|
||||||
|
if campaign_creator_router:
|
||||||
|
app.include_router(campaign_creator_router)
|
||||||
|
|
||||||
# Include content assets router
|
# Include content assets router
|
||||||
from api.content_assets.router import router as content_assets_router
|
from api.content_assets.router import router as content_assets_router
|
||||||
app.include_router(content_assets_router)
|
app.include_router(content_assets_router)
|
||||||
|
router_group_status["platform_extensions"] = {
|
||||||
|
"mounted": True,
|
||||||
|
"reason": "Full mode",
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
router_group_status["platform_extensions"] = {
|
||||||
|
"mounted": False,
|
||||||
|
"reason": "Skipped in podcast-only demo mode",
|
||||||
|
}
|
||||||
|
|
||||||
# Include Podcast Maker router
|
# Include Podcast Maker router (always needed for podcast mode)
|
||||||
from api.podcast.router import router as podcast_router
|
from api.podcast.router import router as podcast_router
|
||||||
app.include_router(podcast_router)
|
app.include_router(podcast_router)
|
||||||
|
router_group_status["podcast_maker"] = {
|
||||||
|
"mounted": True,
|
||||||
|
"reason": "Always mounted",
|
||||||
|
}
|
||||||
|
|
||||||
# Include YouTube Creator Studio router
|
if not PODCAST_ONLY_DEMO_MODE:
|
||||||
from api.youtube.router import router as youtube_router
|
# Include YouTube Creator Studio router
|
||||||
app.include_router(youtube_router, prefix="/api")
|
from api.youtube.router import router as youtube_router
|
||||||
|
app.include_router(youtube_router, prefix="/api")
|
||||||
|
|
||||||
# Include research configuration router
|
# Include research configuration router
|
||||||
app.include_router(research_config_router, prefix="/api/research", tags=["research"])
|
app.include_router(research_config_router, prefix="/api/research", tags=["research"])
|
||||||
|
|
||||||
# Include Research Engine router (standalone AI research module)
|
# Include Research Engine router (standalone AI research module)
|
||||||
from api.research.router import router as research_engine_router
|
from api.research.router import router as research_engine_router
|
||||||
app.include_router(research_engine_router, tags=["Research Engine"])
|
app.include_router(research_engine_router, tags=["Research Engine"])
|
||||||
|
|
||||||
# Scheduler dashboard routes
|
# Scheduler dashboard routes
|
||||||
from api.scheduler_dashboard import router as scheduler_dashboard_router
|
from api.scheduler_dashboard import router as scheduler_dashboard_router
|
||||||
app.include_router(scheduler_dashboard_router)
|
app.include_router(scheduler_dashboard_router)
|
||||||
app.include_router(oauth_token_monitoring_router)
|
if oauth_token_monitoring_router:
|
||||||
|
app.include_router(oauth_token_monitoring_router)
|
||||||
|
|
||||||
# Autonomous Agents API routes (Phase 3A)
|
# Autonomous Agents API routes (Phase 3A)
|
||||||
from api.agents_api import router as agents_router
|
from api.agents_api import router as agents_router
|
||||||
app.include_router(agents_router)
|
app.include_router(agents_router)
|
||||||
|
|
||||||
# Today workflow routes
|
# Today workflow routes
|
||||||
from api.today_workflow import router as today_workflow_router
|
from api.today_workflow import router as today_workflow_router
|
||||||
app.include_router(today_workflow_router)
|
app.include_router(today_workflow_router)
|
||||||
|
router_group_status["advanced_workflows"] = {
|
||||||
|
"mounted": True,
|
||||||
|
"reason": "Full mode",
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
router_group_status["advanced_workflows"] = {
|
||||||
|
"mounted": False,
|
||||||
|
"reason": "Skipped in podcast-only demo mode",
|
||||||
|
}
|
||||||
|
|
||||||
# Setup frontend serving using modular utilities
|
# Setup frontend serving using modular utilities
|
||||||
frontend_serving.setup_frontend_serving()
|
frontend_serving.setup_frontend_serving()
|
||||||
@@ -457,18 +681,32 @@ async def serve_frontend():
|
|||||||
"""Serve the React frontend."""
|
"""Serve the React frontend."""
|
||||||
return frontend_serving.serve_frontend()
|
return frontend_serving.serve_frontend()
|
||||||
|
|
||||||
# Startup event
|
# Startup event - fires AFTER port is bound
|
||||||
@app.on_event("startup")
|
@app.on_event("startup")
|
||||||
async def startup_event():
|
async def startup_event():
|
||||||
"""Initialize services on startup."""
|
"""Initialize services on startup."""
|
||||||
|
import time
|
||||||
|
startup_start = time.time()
|
||||||
|
|
||||||
|
logger.info("[STARTUP] Server port bound, beginning background initialization...")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
startup_report = run_startup_health_routine()
|
_log_memory_usage()
|
||||||
if startup_report.get("status") != "healthy":
|
|
||||||
logger.error(f"Startup readiness finished with failures: {startup_report.get('errors', [])}")
|
# Skip startup health checks in podcast-only mode to avoid unnecessary DB errors
|
||||||
|
if not is_podcast_only_demo_mode():
|
||||||
|
startup_report = run_startup_health_routine(app)
|
||||||
|
if startup_report.get("status") != "healthy":
|
||||||
|
logger.error(f"Startup readiness finished with failures: {startup_report.get('errors', [])}")
|
||||||
|
else:
|
||||||
|
logger.info("[Podcast] Skipping startup health routine (podcast-only mode)")
|
||||||
|
|
||||||
# Start task scheduler
|
# Start task scheduler only if NOT in podcast-only mode
|
||||||
from services.scheduler import get_scheduler
|
if not is_podcast_only_demo_mode():
|
||||||
await get_scheduler().start()
|
from services.scheduler import get_scheduler
|
||||||
|
await get_scheduler().start()
|
||||||
|
else:
|
||||||
|
logger.info("[Podcast] Skipping scheduler startup (podcast-only mode)")
|
||||||
|
|
||||||
# Check Wix API key configuration
|
# Check Wix API key configuration
|
||||||
wix_api_key = os.getenv('WIX_API_KEY')
|
wix_api_key = os.getenv('WIX_API_KEY')
|
||||||
@@ -477,10 +715,40 @@ async def startup_event():
|
|||||||
else:
|
else:
|
||||||
logger.warning("⚠️ WIX_API_KEY not found in environment - Wix publishing may fail")
|
logger.warning("⚠️ WIX_API_KEY not found in environment - Wix publishing may fail")
|
||||||
|
|
||||||
logger.info("ALwrity backend started successfully")
|
elapsed = time.time() - startup_start
|
||||||
|
logger.info(f"ALwrity backend started successfully in {elapsed:.1f}s")
|
||||||
|
|
||||||
|
# Critical router mount assertions for podcast-only demo mode
|
||||||
|
_assert_router_mounted("subscription")
|
||||||
|
_assert_router_mounted("podcast")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error during startup: {e}")
|
logger.error(f"Error during startup: {e}")
|
||||||
raise
|
# Don't raise - let the server start anyway
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_router_mounted(router_name: str) -> None:
|
||||||
|
"""Assert that a critical router is mounted. Fails startup if not found."""
|
||||||
|
from fastapi import routing
|
||||||
|
mounted_routes = [route.path for route in app.routes]
|
||||||
|
|
||||||
|
# Check for router-specific paths
|
||||||
|
router_path_indicators = {
|
||||||
|
"subscription": ["/api/subscription/plans", "/api/subscription/preflight"],
|
||||||
|
"podcast": ["/api/podcast/projects", "/api/podcast/"],
|
||||||
|
}
|
||||||
|
|
||||||
|
expected_paths = router_path_indicators.get(router_name, [])
|
||||||
|
found = any(path in mounted_routes for path in expected_paths)
|
||||||
|
|
||||||
|
if found:
|
||||||
|
logger.info(f"✅ Critical router '{router_name}' is mounted")
|
||||||
|
else:
|
||||||
|
error_msg = f"❌ CRITICAL: Router '{router_name}' is NOT mounted! Expected paths: {expected_paths}"
|
||||||
|
logger.error(error_msg)
|
||||||
|
if PODCAST_ONLY_DEMO_MODE:
|
||||||
|
# In demo mode, podcast router MUST be mounted
|
||||||
|
if router_name == "podcast":
|
||||||
|
raise RuntimeError(error_msg)
|
||||||
|
|
||||||
# Shutdown event
|
# Shutdown event
|
||||||
@app.on_event("shutdown")
|
@app.on_event("shutdown")
|
||||||
@@ -495,4 +763,19 @@ async def shutdown_event():
|
|||||||
close_database()
|
close_database()
|
||||||
logger.info("ALwrity backend shutdown successfully")
|
logger.info("ALwrity backend shutdown successfully")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error during shutdown: {e}")
|
logger.error(f"Error during shutdown: {e}")
|
||||||
|
|
||||||
|
|
||||||
|
# Add main block to allow running directly with: python app.py
|
||||||
|
# This also helps Gunicorn work correctly
|
||||||
|
if __name__ == "__main__":
|
||||||
|
import uvicorn
|
||||||
|
port = int(os.environ.get("PORT", "10000"))
|
||||||
|
host = os.environ.get("HOST", "0.0.0.0")
|
||||||
|
|
||||||
|
print(f"[app.py] ====================", flush=True)
|
||||||
|
print(f"[app.py] DIRECT STARTUP", flush=True)
|
||||||
|
print(f"[app.py] PORT={port}, HOST={host}", flush=True)
|
||||||
|
print(f"[app.py] ====================", flush=True)
|
||||||
|
|
||||||
|
uvicorn.run(app, host=host, port=port)
|
||||||
|
|||||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
197
backend/docs/AGENT_FLAT_CONTEXT_REVIEW.md
Normal file
197
backend/docs/AGENT_FLAT_CONTEXT_REVIEW.md
Normal file
@@ -0,0 +1,197 @@
|
|||||||
|
# Agent Flat-File Context System Review
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
This review documents the **current implementation** of ALwrity's onboarding flat-file context system and compares it to the proposed **Direct-to-File Virtual Shell (VFS)** model.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1) Present Implementation (What Exists Today)
|
||||||
|
|
||||||
|
### 1.1 Storage model
|
||||||
|
- Context is stored per user under:
|
||||||
|
- `backend/workspace/workspace_<safe_user_id>/agent_context/`
|
||||||
|
- Files are JSON documents, one per onboarding domain:
|
||||||
|
- `step2_website_analysis.json`
|
||||||
|
- `step3_research_preferences.json`
|
||||||
|
- `step4_persona_data.json`
|
||||||
|
- `step5_integrations.json`
|
||||||
|
- `context_manifest.json`
|
||||||
|
|
||||||
|
### 1.2 Writer and reader
|
||||||
|
- `AgentFlatContextStore` is the core component that:
|
||||||
|
- sanitizes user IDs for path safety,
|
||||||
|
- writes documents atomically (`tempfile` + `os.replace`),
|
||||||
|
- sets restrictive file permissions (`0600` best effort),
|
||||||
|
- generates structured `agent_summary` objects,
|
||||||
|
- updates a manifest index of available documents.
|
||||||
|
- Data is loaded by direct file reads from the same class (`load_stepX_context_document`).
|
||||||
|
|
||||||
|
### 1.3 Read-path fallback chain
|
||||||
|
`SIFIntegrationService` uses a strict fallback sequence for onboarding context retrieval:
|
||||||
|
1. **flat file** (`AgentFlatContextStore`)
|
||||||
|
2. **database** (`WebsiteAnalysis`, `ResearchPreferences`, `PersonaData`, etc.)
|
||||||
|
3. **SIF semantic index** (`TxtaiIntelligenceService.search`)
|
||||||
|
|
||||||
|
Step 5 uses `flat_file -> sif_semantic`.
|
||||||
|
|
||||||
|
### 1.4 Producer flow (onboarding persistence)
|
||||||
|
`StepManagementService` persists canonical snapshots to flat context when onboarding steps are saved:
|
||||||
|
- Step 2 website analysis
|
||||||
|
- Step 3 research preferences (and later competitor-enriched refresh)
|
||||||
|
- Step 4 persona data
|
||||||
|
- Step 5 integrations
|
||||||
|
|
||||||
|
### 1.5 Context optimization currently implemented
|
||||||
|
- Sensitive-key redaction in nested payloads (`api_key`, `token`, `secret`, etc.).
|
||||||
|
- Size budgeting with trimming (`DEFAULT_MAX_BYTES = 300_000`) and trim metadata.
|
||||||
|
- Generated summaries include:
|
||||||
|
- quick facts,
|
||||||
|
- retrieval hints (high-signal terms and suggested agent queries),
|
||||||
|
- domain-specific focus blocks.
|
||||||
|
- Document context includes audience, retrieval contract, journey stage, related documents, and context-window guidance.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2) Comparison vs Proposed Direct-to-File VFS
|
||||||
|
|
||||||
|
## Strong alignment
|
||||||
|
The current system already matches the proposal in important ways:
|
||||||
|
- **Direct-to-file persistence** instead of DB-backed retrieval for fast reads.
|
||||||
|
- **Manifest/index concept** (`context_manifest.json`) that can act like a precomputed path map.
|
||||||
|
- **Agent-first retrieval semantics** (summary-first contract and fallback policy).
|
||||||
|
- **Operational safety controls** (atomic writes, redaction, path sanitization).
|
||||||
|
|
||||||
|
## Gaps vs full virtual shell abstraction
|
||||||
|
The following pieces are not fully implemented as described in your proposed architecture:
|
||||||
|
- No explicit **virtual shell provider** (`IFileSystem`) exposing `ls/cat/grep/find` commands.
|
||||||
|
- No always-live, process-level **in-memory `Map<virtualPath, absolutePath>`** for path lookups.
|
||||||
|
- No native glob/query command layer for agent shell UX.
|
||||||
|
- Not currently **read-only enforced at API surface** (writes are intentionally allowed by onboarding services to refresh context).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3) Practical Recommendation: Incremental VFS Evolution
|
||||||
|
|
||||||
|
1. **Introduce a read-only VFS facade for agents**
|
||||||
|
- Keep `AgentFlatContextStore` as the write path for trusted onboarding services.
|
||||||
|
- Add `AgentContextVFS` read adapter exposing:
|
||||||
|
- `ls(path)` from manifest,
|
||||||
|
- `cat(path)` mapped to underlying JSON,
|
||||||
|
- `find(glob)` on virtual keys,
|
||||||
|
- `grep(query)` with path prefilter + stream scan.
|
||||||
|
|
||||||
|
2. **Promote manifest to a first-class path map**
|
||||||
|
- Build and cache an in-memory map on service startup or first access.
|
||||||
|
- Refresh map when manifest `updated_at` changes.
|
||||||
|
|
||||||
|
3. **Add explicit write policy boundaries**
|
||||||
|
- Agent-facing interface: hard read-only (`EROFS`).
|
||||||
|
- Internal system service interface: allow writes for onboarding synchronization.
|
||||||
|
|
||||||
|
4. **Metadata strategy for grep ranking**
|
||||||
|
- Prioritize in order:
|
||||||
|
1) `agent_summary.quick_facts`
|
||||||
|
2) `agent_summary.retrieval_hints.high_signal_terms`
|
||||||
|
3) `document_context.context_type` and `journey.stage`
|
||||||
|
4) full `data` body
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4) Response to the Metadata Header Question
|
||||||
|
|
||||||
|
> "Does your current `.txt` optimization include specific metadata headers (like YAML frontmatter) that the grep tool should prioritize?"
|
||||||
|
|
||||||
|
For this implementation, context is currently persisted as structured JSON (not `.txt` with YAML frontmatter). Equivalent high-value metadata already exists and should be prioritized for search/ranking:
|
||||||
|
- `context_type`
|
||||||
|
- `updated_at`
|
||||||
|
- `agent_summary.quick_facts`
|
||||||
|
- `agent_summary.retrieval_hints.high_signal_terms`
|
||||||
|
- `document_context.journey.stage`
|
||||||
|
- `document_context.related_documents`
|
||||||
|
|
||||||
|
If you later move to `.txt` transport files, mirror these as frontmatter fields to preserve retrieval quality.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5) Bottom line
|
||||||
|
Your current onboarding flat-file context implementation is already a strong "shim" architecture and close to the proposed model. The biggest missing piece is a dedicated virtual-shell read interface (`ls/cat/grep/find`) backed by a persistent path-map cache and a clear read-only contract for agent execution contexts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6) Implemented Follow-up (VFS Adapter + Workspace Guide)
|
||||||
|
|
||||||
|
The following enhancements are now implemented:
|
||||||
|
|
||||||
|
1. **Auto-generated workspace map**
|
||||||
|
- The system now generates `workspace_<user>/README.md` whenever `context_manifest.json` is updated.
|
||||||
|
- The README includes:
|
||||||
|
- available context files,
|
||||||
|
- key signal hints from `agent_summary.retrieval_hints.high_signal_terms`,
|
||||||
|
- journey-stage hints,
|
||||||
|
- virtual path mappings and retrieval strategy guidance.
|
||||||
|
|
||||||
|
2. **Read-only VFS facade**
|
||||||
|
- Added `AgentContextVFS` with:
|
||||||
|
- `list_context()` (`ls` equivalent),
|
||||||
|
- `search_context()` (`grep` equivalent; prioritizes `high_signal_terms` and `quick_facts`),
|
||||||
|
- `read_context_file()` (`cat` equivalent; large-file summary mode + subkey drilldown),
|
||||||
|
- explicit write rejection (`EROFS`).
|
||||||
|
|
||||||
|
3. **Virtual path support**
|
||||||
|
- `/env/summary` maps to `AgentFlatContextStore.generate_total_summary()`.
|
||||||
|
- `/steps/website`, `/steps/research`, `/steps/persona`, `/steps/integrations` map to step documents.
|
||||||
|
|
||||||
|
4. **System-prompt helper**
|
||||||
|
- Added `build_filesystem_header(user_id)` to inject a compact file availability + priority hint block into agent startup prompts.
|
||||||
|
|
||||||
|
5. **Merged context helper in SIF integration**
|
||||||
|
- `SIFIntegrationService.get_merged_flat_context()` now provides a unified view across all available flat files while preserving existing per-step retrieval methods.
|
||||||
|
|
||||||
|
6. **Basic file-level security hardening**
|
||||||
|
- Workspace and context directories are now explicitly forced to `0700`.
|
||||||
|
- Context and workspace files are written with strict `0600`.
|
||||||
|
- Added path sandboxing to ensure requested paths cannot escape user workspace roots.
|
||||||
|
- Restricted context-file loading to an allowlist of known onboarding context documents.
|
||||||
|
- Added deterministic per-user secret derivation from `.env` (`FILE_ENCRYPTION_SALT` + `safe_user_id`) with non-sensitive fingerprints for audit/debug and future encryption-at-rest rollout.
|
||||||
|
|
||||||
|
7. **Tool-logic enhancement (coarse-to-fine search)**
|
||||||
|
- `search_context` now performs a two-pass retrieval:
|
||||||
|
1) high-relevance summary match pass (`high_signal_terms`, `quick_facts`),
|
||||||
|
2) parallelized stream scan pass over sandboxed allowlisted files for supporting details.
|
||||||
|
- Results include relevance labels, snippets, and line numbers for body matches.
|
||||||
|
- Large-result behavior now reports truncation guidance (show top 10 and suggest narrower keywords).
|
||||||
|
- `inspect_file` now provides token-saving behavior: full return for small files, or `agent_summary` + top-level keys for larger files, with key-level zoom-in support.
|
||||||
|
|
||||||
|
8. **Retrieval robustness roadmap (next hardening phase)**
|
||||||
|
- **Query normalization:** Add synonym expansion and typo-tolerant matching (e.g., `tone` ≈ `brand voice`) before coarse/fine passes.
|
||||||
|
- **Confidence scoring:** Return confidence tiers that blend source freshness (`updated_at`), summary-match strength, and match density.
|
||||||
|
- **Field-aware boosting:** Weight matches by field priority (`high_signal_terms` > `quick_facts` > `data`) and document recency.
|
||||||
|
- **Deduplicated evidence:** Collapse repeated hits from the same file/key into one clustered result with a single best snippet and hit count.
|
||||||
|
- **Fallback query reformulation:** If zero hits, automatically retry with narrow/expanded variants and return attempted queries.
|
||||||
|
- **Answerability contract:** Add a lightweight `can_answer` signal in search responses so orchestrators can decide whether to ask follow-up questions or fetch more context.
|
||||||
|
- **Evaluation harness:** Track retrieval metrics over golden queries (`precision@k`, `MRR`, zero-hit rate, stale-hit rate) in CI to prevent relevance regressions.
|
||||||
|
|
||||||
|
9. **Collaborative VFS namespace (shared memory mode)**
|
||||||
|
- Added optional `project_id` support to `AgentContextVFS` with isolated root: `workspace/project_<project_id>/`.
|
||||||
|
- Introduced `scratchpad/` for collaborative writes while keeping onboarding `agent_context` read-first.
|
||||||
|
- Added `write_shared_note(...)` with advisory locking (`flock`) and strict filename/path validation.
|
||||||
|
- Added append-only `activity_log.jsonl` via `append_activity_log(...)` for watchdog/event-driven coordination.
|
||||||
|
- Maintains owner-only permissions (`0700` scratchpad dir, `0600` files) and audit trails for shared writes.
|
||||||
|
|
||||||
|
10. **Testing readiness upgrades**
|
||||||
|
- Added automated tests for:
|
||||||
|
- query reformulation + `can_answer` behavior in `search_context`,
|
||||||
|
- large-file progressive disclosure behavior in `inspect_file`,
|
||||||
|
- collaborative write path (`write_shared_note`) and append-only activity logging.
|
||||||
|
- Test module: `backend/tests/test_agent_context_vfs.py`.
|
||||||
|
- These tests provide a baseline regression harness for VFS retrieval quality and shared-memory safety.
|
||||||
|
|
||||||
|
11. **Static + Structural retrieval hardening**
|
||||||
|
- Added a **static triage layer** in `search_context`:
|
||||||
|
- keyword-density scoring,
|
||||||
|
- `low_probability` flags for likely-noisy hits,
|
||||||
|
- `triage_top5` shortlist for router-style pre-filtering.
|
||||||
|
- Added `read_struct(filename, path_query)`:
|
||||||
|
- resolves dot/bracket JSON paths to return node-level data only,
|
||||||
|
- includes lightweight dependency injection (e.g., Step 4 persona reads include Step 2 brand voice context when available),
|
||||||
|
- keeps output token-efficient for downstream agents.
|
||||||
1
backend/emojis.txt
Normal file
1
backend/emojis.txt
Normal file
@@ -0,0 +1 @@
|
|||||||
|
{'🎙', '🛑', '🚀', '📖', '💳', '📈', '🌐', '📊', '📦', '🔧', '🔍'}
|
||||||
46
backend/gunicorn_config.py
Normal file
46
backend/gunicorn_config.py
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
"""Gunicorn configuration for Render deployment."""
|
||||||
|
import os
|
||||||
|
import multiprocessing
|
||||||
|
|
||||||
|
# Bind to the port Render provides
|
||||||
|
bind = f"0.0.0.0:{os.getenv('PORT', '10000')}"
|
||||||
|
|
||||||
|
# Use uvicorn workers
|
||||||
|
worker_class = "uvicorn.workers.UvicornWorker"
|
||||||
|
|
||||||
|
# Single worker for memory efficiency on free tier
|
||||||
|
workers = 1
|
||||||
|
|
||||||
|
# Timeout for slow startup (10 minutes to allow for model loading)
|
||||||
|
timeout = 600
|
||||||
|
|
||||||
|
# Graceful timeout
|
||||||
|
graceful_timeout = 30
|
||||||
|
|
||||||
|
# Keepalive
|
||||||
|
keepalive = 5
|
||||||
|
|
||||||
|
# Logging
|
||||||
|
accesslog = "-"
|
||||||
|
errorlog = "-"
|
||||||
|
loglevel = os.getenv("LOG_LEVEL", "info").lower()
|
||||||
|
|
||||||
|
# Don't preload - bind to port FIRST, then load worker
|
||||||
|
preload_app = False
|
||||||
|
|
||||||
|
# Use the startup script that handles all the logic
|
||||||
|
factory = False # app:app is not a factory, it's the app object
|
||||||
|
|
||||||
|
def on_starting(server):
|
||||||
|
"""Called just before the master process is initialized."""
|
||||||
|
print(f"[GUNICORN] Starting on {bind}", flush=True)
|
||||||
|
|
||||||
|
|
||||||
|
def on_reload(server):
|
||||||
|
"""Called when worker is reloaded."""
|
||||||
|
print(f"[GUNICORN] Reloading workers", flush=True)
|
||||||
|
|
||||||
|
|
||||||
|
def when_ready(server):
|
||||||
|
"""Called just after the server is started."""
|
||||||
|
print(f"[GUNICORN] Server is ready. Accepting connections.", flush=True)
|
||||||
@@ -236,6 +236,11 @@ async def router_status():
|
|||||||
"""Get router inclusion status."""
|
"""Get router inclusion status."""
|
||||||
return router_manager.get_router_status()
|
return router_manager.get_router_status()
|
||||||
|
|
||||||
|
@app.get("/api/feature-profile/status")
|
||||||
|
async def feature_profile_status():
|
||||||
|
"""Get feature profile status and enabled modules."""
|
||||||
|
return router_manager.get_feature_profile_status()
|
||||||
|
|
||||||
# Onboarding management endpoints
|
# Onboarding management endpoints
|
||||||
@app.get("/api/onboarding/status")
|
@app.get("/api/onboarding/status")
|
||||||
async def onboarding_status():
|
async def onboarding_status():
|
||||||
@@ -244,6 +249,9 @@ async def onboarding_status():
|
|||||||
|
|
||||||
# Include routers using modular utilities
|
# Include routers using modular utilities
|
||||||
router_manager.include_core_routers()
|
router_manager.include_core_routers()
|
||||||
|
# Safety net: keep subscription routes available even if core inclusion flow changes
|
||||||
|
# in special modes (e.g., demo mode). De-dup is handled by RouterManager.
|
||||||
|
router_manager.include_router_safely(subscription_router, "subscription")
|
||||||
router_manager.include_optional_routers()
|
router_manager.include_optional_routers()
|
||||||
|
|
||||||
# SEO Dashboard endpoints
|
# SEO Dashboard endpoints
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ IMPORTANT: This is a compatibility layer. For new code, use UserAPIKeyContext di
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
|
import time
|
||||||
from fastapi import Request
|
from fastapi import Request
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
from typing import Callable
|
from typing import Callable
|
||||||
@@ -20,8 +21,61 @@ class APIKeyInjectionMiddleware:
|
|||||||
for the duration of each request.
|
for the duration of each request.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
# Shared across middleware instances (module currently instantiates per request)
|
||||||
|
_missing_keys_log_timestamps = {}
|
||||||
|
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.original_keys = {}
|
self.original_keys = {}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _should_skip_missing_key_warning(request: Request) -> bool:
|
||||||
|
"""
|
||||||
|
Optionally suppress missing-key warnings for non-AI/internal routes.
|
||||||
|
Controlled by API_KEY_INJECTION_SKIP_NON_AI_WARNINGS (default: true).
|
||||||
|
"""
|
||||||
|
skip_non_ai_warnings = os.getenv('API_KEY_INJECTION_SKIP_NON_AI_WARNINGS', 'true').lower() in ('1', 'true', 'yes')
|
||||||
|
if not skip_non_ai_warnings:
|
||||||
|
return False
|
||||||
|
|
||||||
|
path_lower = (request.url.path or '').lower()
|
||||||
|
return (
|
||||||
|
path_lower.startswith('/api/subscription/')
|
||||||
|
or path_lower.startswith('/api/onboarding/')
|
||||||
|
or path_lower.endswith('/status')
|
||||||
|
or path_lower.endswith('/health')
|
||||||
|
or path_lower == '/health'
|
||||||
|
or path_lower == '/status'
|
||||||
|
)
|
||||||
|
|
||||||
|
def _log_missing_keys_non_blocking(self, request: Request, user_id: str) -> None:
|
||||||
|
"""
|
||||||
|
Log missing API keys without interrupting request flow.
|
||||||
|
- Defaults to debug-level logging.
|
||||||
|
- Optional warn once-per-user-per-interval via env:
|
||||||
|
API_KEY_INJECTION_MISSING_KEYS_LOG_MODE=warn_once
|
||||||
|
API_KEY_INJECTION_MISSING_KEYS_LOG_INTERVAL_SECONDS=900
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
if self._should_skip_missing_key_warning(request):
|
||||||
|
logger.debug(f"[API Key Injection] Missing keys for user {user_id} on non-AI route; skipping warning")
|
||||||
|
return
|
||||||
|
|
||||||
|
log_mode = os.getenv('API_KEY_INJECTION_MISSING_KEYS_LOG_MODE', 'debug').lower()
|
||||||
|
if log_mode != 'warn_once':
|
||||||
|
logger.debug(f"No API keys found for user {user_id}")
|
||||||
|
return
|
||||||
|
|
||||||
|
interval_seconds = int(os.getenv('API_KEY_INJECTION_MISSING_KEYS_LOG_INTERVAL_SECONDS', '900'))
|
||||||
|
now = time.time()
|
||||||
|
last_logged_at = self._missing_keys_log_timestamps.get(user_id, 0)
|
||||||
|
if (now - last_logged_at) >= max(interval_seconds, 1):
|
||||||
|
logger.warning(f"No API keys found for user {user_id}")
|
||||||
|
self._missing_keys_log_timestamps[user_id] = now
|
||||||
|
else:
|
||||||
|
logger.debug(f"No API keys found for user {user_id} (warning suppressed by interval)")
|
||||||
|
except Exception as log_error:
|
||||||
|
# Logging should never block request processing
|
||||||
|
logger.debug(f"[API Key Injection] Failed to log missing keys state for user {user_id}: {log_error}")
|
||||||
|
|
||||||
async def __call__(self, request: Request, call_next: Callable):
|
async def __call__(self, request: Request, call_next: Callable):
|
||||||
"""
|
"""
|
||||||
@@ -68,7 +122,7 @@ class APIKeyInjectionMiddleware:
|
|||||||
# Get user-specific API keys from database
|
# Get user-specific API keys from database
|
||||||
with user_api_keys(user_id) as user_keys:
|
with user_api_keys(user_id) as user_keys:
|
||||||
if not user_keys:
|
if not user_keys:
|
||||||
logger.warning(f"No API keys found for user {user_id}")
|
self._log_missing_keys_non_blocking(request, user_id)
|
||||||
return await call_next(request)
|
return await call_next(request)
|
||||||
|
|
||||||
# Save original environment values
|
# Save original environment values
|
||||||
@@ -120,4 +174,3 @@ async def api_key_injection_middleware(request: Request, call_next: Callable):
|
|||||||
"""
|
"""
|
||||||
middleware = APIKeyInjectionMiddleware()
|
middleware = APIKeyInjectionMiddleware()
|
||||||
return await middleware(request, call_next)
|
return await middleware(request, call_next)
|
||||||
|
|
||||||
|
|||||||
@@ -1,9 +1,30 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
python -m pip install --upgrade pip setuptools wheel
|
echo "🚀 Starting ALwrity Build Process..."
|
||||||
python -m pip install --retries 10 --timeout 120 -r requirements.txt
|
|
||||||
|
|
||||||
# Download required NLTK and spaCy models during build phase
|
# 1. Update pip and essential build tools
|
||||||
python -m spacy download en_core_web_sm
|
python -m pip install --upgrade pip setuptools wheel
|
||||||
python -m nltk.downloader punkt_tab stopwords averaged_perceptron_tagger
|
|
||||||
|
# 2. Install requirements based on mode
|
||||||
|
echo "📦 Checking ALWRITY_ENABLED_FEATURES..."
|
||||||
|
ENABLED_FEATURES="${ALWRITY_ENABLED_FEATURES:-all}"
|
||||||
|
echo "DEBUG: ENABLED_FEATURES='$ENABLED_FEATURES'"
|
||||||
|
|
||||||
|
if [[ "$ENABLED_FEATURES" == "podcast" ]]; then
|
||||||
|
echo "🔊 Podcast-only mode: Installing lean requirements..."
|
||||||
|
python -m pip install --no-cache-dir -r requirements-podcast.txt --only-binary :all: --retries 10 --timeout 120
|
||||||
|
else
|
||||||
|
echo "📦 Full mode: Installing all requirements..."
|
||||||
|
python -m pip install --no-cache-dir -r requirements.txt --only-binary :all: --retries 10 --timeout 120
|
||||||
|
# Download spaCy/NLTK models for full mode
|
||||||
|
echo "🧠 Installing spaCy and NLTK models..."
|
||||||
|
python -m spacy download en_core_web_sm
|
||||||
|
python -m nltk.downloader punkt_tab stopwords averaged_perceptron_tagger
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3. Clean up unnecessary build artifacts
|
||||||
|
find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
|
||||||
|
rm -rf /root/.cache/pip 2>/dev/null || true
|
||||||
|
|
||||||
|
echo "✅ Build Complete!"
|
||||||
|
|||||||
81
backend/requirements-podcast.txt
Normal file
81
backend/requirements-podcast.txt
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
# =====================================================
|
||||||
|
# ALwrity Podcast-Only Requirements
|
||||||
|
# Lean subset for podcast-only demo mode
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
# Core Web Server
|
||||||
|
fastapi>=0.115.14
|
||||||
|
starlette>=0.40.0,<0.47.0
|
||||||
|
sse-starlette<3.0.0
|
||||||
|
uvicorn>=0.24.0
|
||||||
|
uvicorn[standard]>=0.24.0
|
||||||
|
gunicorn>=21.0.0
|
||||||
|
|
||||||
|
# Server utilities
|
||||||
|
python-multipart>=0.0.6
|
||||||
|
python-dotenv>=1.0.0
|
||||||
|
loguru>=0.7.2
|
||||||
|
tenacity>=8.2.3
|
||||||
|
pydantic>=2.5.2,<3.0.0
|
||||||
|
typing-extensions>=4.8.0
|
||||||
|
setuptools>=65.0.0
|
||||||
|
|
||||||
|
# Auth & Database
|
||||||
|
fastapi-clerk-auth>=0.0.7
|
||||||
|
sqlalchemy>=2.0.25
|
||||||
|
|
||||||
|
# Payment
|
||||||
|
stripe>=8.0.0
|
||||||
|
|
||||||
|
# HTTP clients
|
||||||
|
httpx>=0.28.1
|
||||||
|
aiohttp>=3.9.0
|
||||||
|
requests>=2.31.0
|
||||||
|
|
||||||
|
# AI - needed for podcast
|
||||||
|
openai>=1.3.0
|
||||||
|
google-genai>=1.0.0
|
||||||
|
exa-py==1.9.1
|
||||||
|
|
||||||
|
# Text processing (minimal)
|
||||||
|
markdown>=3.5.0
|
||||||
|
beautifulsoup4>=4.12.0
|
||||||
|
|
||||||
|
# Data processing (numpy needed for moviepy, pandas for usage tracking)
|
||||||
|
numpy>=1.24.0
|
||||||
|
pandas>=2.0.0
|
||||||
|
|
||||||
|
# Image/media for podcast
|
||||||
|
Pillow>=10.0.0
|
||||||
|
huggingface_hub>=1.1.4
|
||||||
|
|
||||||
|
# TTS for podcast
|
||||||
|
gtts>=2.4.0
|
||||||
|
pyttsx3>=2.90
|
||||||
|
|
||||||
|
# Video composition
|
||||||
|
moviepy==2.1.2
|
||||||
|
imageio>=2.31.0
|
||||||
|
imageio-ffmpeg>=0.4.9
|
||||||
|
|
||||||
|
# Testing
|
||||||
|
pytest>=7.4.0
|
||||||
|
pytest-asyncio>=0.21.0
|
||||||
|
|
||||||
|
# Task scheduling
|
||||||
|
apscheduler>=3.10.0
|
||||||
|
|
||||||
|
# Utilities
|
||||||
|
redis>=5.0.0
|
||||||
|
schedule>=1.2.0
|
||||||
|
aiofiles>=23.2.0
|
||||||
|
psutil>=5.9.0
|
||||||
|
|
||||||
|
# Google APIs
|
||||||
|
google-api-python-client>=2.100.0
|
||||||
|
google-auth>=2.23.0
|
||||||
|
google-auth-oauthlib>=1.0.0
|
||||||
|
|
||||||
|
# Other utilities
|
||||||
|
python-dateutil>=2.8.0
|
||||||
|
jinja2>=3.1.0
|
||||||
@@ -1,93 +1,80 @@
|
|||||||
# Core dependencies
|
# Core dependencies - needed for all modes
|
||||||
fastapi>=0.115.14
|
fastapi>=0.115.14
|
||||||
starlette>=0.40.0,<0.47.0
|
starlette>=0.40.0,<0.47.0
|
||||||
sse-starlette<3.0.0
|
sse-starlette<3.0.0
|
||||||
uvicorn>=0.24.0
|
uvicorn>=0.24.0
|
||||||
|
uvicorn[standard]>=0.24.0
|
||||||
|
gunicorn>=21.0.0
|
||||||
python-multipart>=0.0.6
|
python-multipart>=0.0.6
|
||||||
python-dotenv>=1.0.0
|
python-dotenv>=1.0.0
|
||||||
loguru>=0.7.2
|
loguru>=0.7.2
|
||||||
tenacity>=8.2.3
|
tenacity>=8.2.3
|
||||||
|
pydantic>=2.5.2,<3.0.0
|
||||||
|
typing-extensions>=4.8.0
|
||||||
|
|
||||||
# Authentication and security
|
# Auth
|
||||||
PyJWT>=2.8.0
|
PyJWT>=2.8.0
|
||||||
cryptography>=41.0.0
|
cryptography>=41.0.0
|
||||||
fastapi-clerk-auth>=0.0.7
|
fastapi-clerk-auth>=0.0.7
|
||||||
|
|
||||||
# Database dependencies
|
# Database
|
||||||
sqlalchemy>=2.0.25
|
sqlalchemy>=2.0.25
|
||||||
|
|
||||||
# Payment processing
|
# Payment
|
||||||
stripe>=8.0.0
|
stripe>=8.0.0
|
||||||
|
|
||||||
# CopilotKit and Research
|
# HTTP clients
|
||||||
copilotkit
|
httpx>=0.28.1
|
||||||
exa-py==1.9.1
|
aiohttp>=3.9.0
|
||||||
httpx>=0.27.2,<0.28.0
|
requests>=2.31.0
|
||||||
|
|
||||||
# AI/ML dependencies - Windows-compatible versions
|
# AI - needed for podcast
|
||||||
openai>=1.3.0
|
openai>=1.3.0
|
||||||
google-genai>=1.0.0
|
google-genai>=1.0.0
|
||||||
sentence-transformers>=2.2.2
|
exa-py==1.9.1
|
||||||
|
|
||||||
# txtai with Windows-compatible dependencies
|
# Text processing
|
||||||
txtai[agent]>=7.0.0
|
markdown>=3.5.0
|
||||||
|
|
||||||
|
|
||||||
google-api-python-client>=2.100.0
|
|
||||||
google-auth>=2.23.0
|
|
||||||
google-auth-oauthlib>=1.0.0
|
|
||||||
|
|
||||||
# Web scraping and content processing
|
|
||||||
beautifulsoup4>=4.12.0
|
beautifulsoup4>=4.12.0
|
||||||
requests>=2.31.0
|
|
||||||
urllib3<2.0.0
|
|
||||||
chardet>=5.0.0
|
|
||||||
charset-normalizer<3.0.0
|
|
||||||
lxml>=4.9.0
|
lxml>=4.9.0
|
||||||
html5lib>=1.1
|
advertools>=0.14.0
|
||||||
aiohttp>=3.9.0
|
|
||||||
|
|
||||||
# Data processing
|
# Data processing
|
||||||
pandas>=2.0.0
|
pandas>=2.0.0
|
||||||
numpy>=1.24.0
|
numpy>=1.24.0
|
||||||
markdown>=3.5.0
|
|
||||||
|
|
||||||
# SEO Analysis dependencies
|
# Image/media for podcast
|
||||||
advertools>=0.14.0
|
|
||||||
textstat>=0.7.3
|
|
||||||
pyspellchecker>=0.7.2
|
|
||||||
aiofiles>=23.2.0
|
|
||||||
crawl4ai>=0.2.0
|
|
||||||
|
|
||||||
# Linguistic Analysis dependencies (Required for persona generation)
|
|
||||||
spacy>=3.7.0
|
|
||||||
nltk>=3.8.0
|
|
||||||
|
|
||||||
# Image and audio processing for Stability AI
|
|
||||||
Pillow>=10.0.0
|
Pillow>=10.0.0
|
||||||
huggingface_hub>=1.1.4
|
huggingface_hub>=1.1.4
|
||||||
|
|
||||||
# Text-to-Speech (TTS) dependencies
|
# TTS for podcast
|
||||||
gtts>=2.4.0
|
gtts>=2.4.0
|
||||||
pyttsx3>=2.90
|
pyttsx3>=2.90
|
||||||
|
|
||||||
# Video composition dependencies
|
# Video composition
|
||||||
moviepy==2.1.2
|
moviepy==2.1.2
|
||||||
imageio>=2.31.0
|
imageio>=2.31.0
|
||||||
imageio-ffmpeg>=0.4.9
|
imageio-ffmpeg>=0.4.9
|
||||||
|
|
||||||
# Testing dependencies
|
# Testing
|
||||||
pytest>=7.4.0
|
pytest>=7.4.0
|
||||||
pytest-asyncio>=0.21.0
|
pytest-asyncio>=0.21.0
|
||||||
|
|
||||||
# Utilities
|
|
||||||
pydantic>=2.5.2,<3.0.0
|
|
||||||
typing-extensions>=4.8.0
|
|
||||||
|
|
||||||
# Task scheduling
|
# Task scheduling
|
||||||
apscheduler>=3.10.0
|
apscheduler>=3.10.0
|
||||||
|
|
||||||
# Optional dependencies (for enhanced features)
|
# Utilities
|
||||||
redis>=5.0.0
|
redis>=5.0.0
|
||||||
schedule>=1.2.0
|
schedule>=1.2.0
|
||||||
pytrends>=4.9.0
|
aiofiles>=23.2.0
|
||||||
|
psutil>=5.9.0
|
||||||
|
|
||||||
|
# Google APIs
|
||||||
|
google-api-python-client>=2.100.0
|
||||||
|
google-auth>=2.23.0
|
||||||
|
google-auth-oauthlib>=1.0.0
|
||||||
|
|
||||||
|
# Other utilities
|
||||||
|
python-dateutil>=2.8.0
|
||||||
|
jinja2>=3.1.0
|
||||||
|
pydantic-settings>=2.0.0
|
||||||
|
|||||||
70
backend/scripts/check_forced_user_id_patterns.py
Normal file
70
backend/scripts/check_forced_user_id_patterns.py
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Fail CI on forced/hardcoded user_id patterns outside test fixtures."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
|
||||||
|
CHECK_GLOBS = ("**/*.py",)
|
||||||
|
EXCLUDED_SUBSTRINGS = (
|
||||||
|
"/.git/",
|
||||||
|
"/.venv/",
|
||||||
|
"/venv/",
|
||||||
|
"/node_modules/",
|
||||||
|
"/__pycache__/",
|
||||||
|
"/tests/",
|
||||||
|
"/test_",
|
||||||
|
"/fixtures/",
|
||||||
|
"/test_validation/",
|
||||||
|
"/backend/scripts/check_forced_user_id_patterns.py",
|
||||||
|
)
|
||||||
|
|
||||||
|
RULES = [
|
||||||
|
(re.compile(r"\buser_id\s*=\s*1\b"), "hardcoded `user_id = 1`"),
|
||||||
|
(re.compile(r"force\s+user_id", re.IGNORECASE), "`force user_id` marker"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def is_excluded(path: Path) -> bool:
|
||||||
|
normalized = f"/{path.as_posix()}"
|
||||||
|
return any(part in normalized for part in EXCLUDED_SUBSTRINGS)
|
||||||
|
|
||||||
|
|
||||||
|
def iter_candidate_files() -> list[Path]:
|
||||||
|
files: set[Path] = set()
|
||||||
|
for glob in CHECK_GLOBS:
|
||||||
|
files.update(REPO_ROOT.glob(glob))
|
||||||
|
return sorted(p for p in files if p.is_file() and not is_excluded(p.relative_to(REPO_ROOT)))
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
violations: list[tuple[Path, int, str, str]] = []
|
||||||
|
|
||||||
|
for file_path in iter_candidate_files():
|
||||||
|
rel_path = file_path.relative_to(REPO_ROOT)
|
||||||
|
try:
|
||||||
|
text = file_path.read_text(encoding="utf-8")
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
for line_number, line in enumerate(text.splitlines(), start=1):
|
||||||
|
for pattern, label in RULES:
|
||||||
|
if pattern.search(line):
|
||||||
|
violations.append((rel_path, line_number, label, line.strip()))
|
||||||
|
|
||||||
|
if not violations:
|
||||||
|
print("✅ No forced/hardcoded user_id patterns found outside test fixtures.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
print("❌ Found forbidden forced/hardcoded user_id patterns:")
|
||||||
|
for path, line, label, source_line in violations:
|
||||||
|
print(f" - {path}:{line} [{label}] -> {source_line}")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
355
backend/scripts/run_podcast_billing_sequence.py
Normal file
355
backend/scripts/run_podcast_billing_sequence.py
Normal file
@@ -0,0 +1,355 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run podcast preflight + operations and verify billing usage/cost deltas."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import json
|
||||||
|
import asyncio
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
# Use mock auth in local test runs
|
||||||
|
os.environ.setdefault("DISABLE_AUTH", "true")
|
||||||
|
os.environ.setdefault("ALLOW_UNVERIFIED_JWT_DEV", "true")
|
||||||
|
os.environ.setdefault(
|
||||||
|
"STRIPE_PLAN_PRICE_MAPPING_TEST",
|
||||||
|
"{\"basic\": {\"monthly\": \"price_test_basic_monthly\"}, \"pro\": {\"monthly\": \"price_test_pro_monthly\"}}",
|
||||||
|
)
|
||||||
|
os.environ.setdefault("EXA_API_KEY", "test-exa-key")
|
||||||
|
|
||||||
|
import spacy
|
||||||
|
|
||||||
|
# Avoid hard dependency on downloaded spaCy model during router imports.
|
||||||
|
spacy.load = lambda _name, *args, **kwargs: object() # type: ignore[assignment]
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
# Import only required routers (avoids heavyweight app startup deps)
|
||||||
|
from api.podcast.router import router as podcast_router
|
||||||
|
from api.subscription import router as subscription_router
|
||||||
|
from api.podcast.handlers import analysis as analysis_handler
|
||||||
|
from api.podcast.handlers import research as research_handler
|
||||||
|
from api.podcast.handlers import video as video_handler
|
||||||
|
from api.podcast.constants import get_podcast_media_dir, PODCAST_IMAGES_DIR
|
||||||
|
from services.database import get_session_for_user
|
||||||
|
from services.subscription.usage_tracking_service import UsageTrackingService
|
||||||
|
from models.subscription_models import APIProvider
|
||||||
|
|
||||||
|
|
||||||
|
USER_ID = "mock_user_id"
|
||||||
|
AUTH_HEADERS = {"Authorization": "Bearer test-token"}
|
||||||
|
BILLING_PERIOD = "2026-03"
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_test_media_files(user_id: str) -> tuple[str, str]:
|
||||||
|
audio_dir = get_podcast_media_dir("audio", user_id, ensure_exists=True)
|
||||||
|
image_dir = get_podcast_media_dir("image", user_id, ensure_exists=True)
|
||||||
|
|
||||||
|
audio_file = audio_dir / "sequence_test_audio.mp3"
|
||||||
|
image_file = image_dir / "sequence_test_image.png"
|
||||||
|
|
||||||
|
if not audio_file.exists():
|
||||||
|
audio_file.write_bytes(b"ID3" + b"\x00" * 512)
|
||||||
|
if not image_file.exists():
|
||||||
|
# Minimal PNG header-like bytes (sufficient for mocked pipeline)
|
||||||
|
image_file.write_bytes(b"\x89PNG\r\n\x1a\n" + b"\x00" * 512)
|
||||||
|
# Also place in legacy global dir for URL resolver compatibility.
|
||||||
|
PODCAST_IMAGES_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
legacy_image_file = PODCAST_IMAGES_DIR / image_file.name
|
||||||
|
if not legacy_image_file.exists():
|
||||||
|
legacy_image_file.write_bytes(image_file.read_bytes())
|
||||||
|
|
||||||
|
return (
|
||||||
|
f"/api/podcast/audio/{audio_file.name}",
|
||||||
|
f"/api/podcast/images/{image_file.name}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _patch_external_calls() -> None:
|
||||||
|
# 1) Podcast analysis: avoid real LLM calls
|
||||||
|
def _mock_llm_text_gen(*args: Any, **kwargs: Any) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"audience": "US founders building AI products",
|
||||||
|
"content_type": "interview",
|
||||||
|
"top_keywords": ["ai agent", "startup", "gtm", "cost", "automation"],
|
||||||
|
"suggested_outlines": [
|
||||||
|
{"title": "What changed in 2026", "segments": ["Market", "Tools", "ROI", "Pitfalls"]},
|
||||||
|
{"title": "Building with constraints", "segments": ["Budget", "Stack", "Team", "Execution"]},
|
||||||
|
],
|
||||||
|
"title_suggestions": ["AI Agents in 2026", "Ship Faster with AI", "Startup AI Playbook"],
|
||||||
|
"research_queries": [
|
||||||
|
{"query": "AI agent adoption data 2026 startups", "rationale": "quantify adoption"},
|
||||||
|
{"query": "founder interviews AI automation ROI", "rationale": "real examples"},
|
||||||
|
],
|
||||||
|
"exa_suggested_config": {
|
||||||
|
"exa_search_type": "auto",
|
||||||
|
"max_sources": 6,
|
||||||
|
"include_statistics": True,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _mock_exa_search(*args: Any, **kwargs: Any) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"provider": "exa",
|
||||||
|
"search_type": "neural",
|
||||||
|
"search_queries": ["AI agent adoption data 2026 startups"],
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"title": "Agentic AI trends",
|
||||||
|
"url": "https://example.com/agentic-ai-trends",
|
||||||
|
"excerpt": "Adoption rose notably among SMB teams.",
|
||||||
|
"index": 1,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"content": "Key Highlights: Adoption increased and ROI became more measurable.",
|
||||||
|
"cost": {"total": 0.015},
|
||||||
|
}
|
||||||
|
|
||||||
|
def _mock_animate_scene_with_voiceover(*args: Any, **kwargs: Any) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"video_bytes": b"\x00\x00\x00\x18ftypmp42" + b"\x00" * 1024,
|
||||||
|
"provider": "wavespeed",
|
||||||
|
"model_name": "wavespeed-ai/infinitetalk",
|
||||||
|
"prompt": "Animate presenter speaking clearly.",
|
||||||
|
"cost": 0.09,
|
||||||
|
"duration": 8.0,
|
||||||
|
}
|
||||||
|
|
||||||
|
analysis_handler.llm_text_gen = _mock_llm_text_gen
|
||||||
|
research_handler.llm_text_gen = _mock_llm_text_gen
|
||||||
|
research_handler.ExaResearchProvider.search = _mock_exa_search
|
||||||
|
video_handler.animate_scene_with_voiceover = _mock_animate_scene_with_voiceover
|
||||||
|
|
||||||
|
|
||||||
|
def _post_json(client: TestClient, path: str, payload: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
res = client.post(path, json=payload, headers=AUTH_HEADERS)
|
||||||
|
res.raise_for_status()
|
||||||
|
return res.json()
|
||||||
|
|
||||||
|
|
||||||
|
def _get_json(client: TestClient, path: str) -> dict[str, Any]:
|
||||||
|
res = client.get(path, headers=AUTH_HEADERS)
|
||||||
|
res.raise_for_status()
|
||||||
|
return res.json()
|
||||||
|
|
||||||
|
|
||||||
|
def _provider_cost_totals(logs_payload: dict[str, Any]) -> dict[str, float]:
|
||||||
|
totals: dict[str, float] = {}
|
||||||
|
for row in logs_payload.get("logs", []):
|
||||||
|
provider = (row.get("provider") or "unknown").lower()
|
||||||
|
totals[provider] = totals.get(provider, 0.0) + float(row.get("cost_total") or 0.0)
|
||||||
|
return totals
|
||||||
|
|
||||||
|
|
||||||
|
def _record_usage(user_id: str, provider: APIProvider, endpoint: str, model: str, tokens_in: int = 0, tokens_out: int = 0) -> None:
|
||||||
|
db = get_session_for_user(user_id)
|
||||||
|
if not db:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
service = UsageTrackingService(db)
|
||||||
|
asyncio.run(
|
||||||
|
service.track_api_usage(
|
||||||
|
user_id=user_id,
|
||||||
|
provider=provider,
|
||||||
|
endpoint=endpoint,
|
||||||
|
method="POST",
|
||||||
|
model_used=model,
|
||||||
|
tokens_input=tokens_in,
|
||||||
|
tokens_output=tokens_out,
|
||||||
|
response_time=0.42,
|
||||||
|
status_code=200,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
_patch_external_calls()
|
||||||
|
audio_url, avatar_image_path = _ensure_test_media_files(USER_ID)
|
||||||
|
|
||||||
|
app = FastAPI()
|
||||||
|
app.include_router(subscription_router)
|
||||||
|
app.include_router(podcast_router)
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
# Baseline billing snapshots
|
||||||
|
baseline_dashboard = _get_json(client, f"/api/subscription/dashboard/{USER_ID}?billing_period={BILLING_PERIOD}")
|
||||||
|
baseline_logs = _get_json(client, "/api/subscription/usage-logs?limit=500")
|
||||||
|
|
||||||
|
before_cost = float(baseline_dashboard["data"]["summary"]["total_cost_this_month"])
|
||||||
|
before_calls = int(baseline_dashboard["data"]["summary"]["total_api_calls_this_month"])
|
||||||
|
before_projection = float(baseline_dashboard["data"]["projections"]["projected_monthly_cost"])
|
||||||
|
before_provider_costs = _provider_cost_totals(baseline_logs)
|
||||||
|
|
||||||
|
# 1) Preflight for podcast analysis + video
|
||||||
|
preflight_payload = {
|
||||||
|
"operations": [
|
||||||
|
{
|
||||||
|
"provider": "huggingface",
|
||||||
|
"operation_type": "podcast_analysis",
|
||||||
|
"tokens_requested": 1200,
|
||||||
|
"model": "meta-llama/llama-3.3-70b-instruct",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"provider": "video",
|
||||||
|
"operation_type": "scene_animation",
|
||||||
|
"tokens_requested": 0,
|
||||||
|
"model": "wavespeed-ai/infinitetalk",
|
||||||
|
"actual_provider_name": "wavespeed",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
preflight = _post_json(client, "/api/subscription/preflight-check", preflight_payload)
|
||||||
|
|
||||||
|
# 2a) Podcast analysis
|
||||||
|
analysis = _post_json(
|
||||||
|
client,
|
||||||
|
"/api/podcast/analyze",
|
||||||
|
{
|
||||||
|
"idea": "How AI agents are changing founder workflows",
|
||||||
|
"duration": 8,
|
||||||
|
"speakers": 1,
|
||||||
|
# Keep avatar to skip image generation call in this sequence
|
||||||
|
"avatar_url": "/api/podcast/images/avatars/already_present.png",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
_record_usage(
|
||||||
|
user_id=USER_ID,
|
||||||
|
provider=APIProvider.MISTRAL,
|
||||||
|
endpoint="/api/podcast/analyze",
|
||||||
|
model="meta-llama/llama-3.3-70b-instruct",
|
||||||
|
tokens_in=1200,
|
||||||
|
tokens_out=600,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 2b) Podcast research
|
||||||
|
research = _post_json(
|
||||||
|
client,
|
||||||
|
"/api/podcast/research/exa",
|
||||||
|
{
|
||||||
|
"topic": "AI agent adoption in startups",
|
||||||
|
"queries": ["AI agent adoption data 2026 startups"],
|
||||||
|
"analysis": {"audience": analysis.get("audience", "general")},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
_record_usage(
|
||||||
|
user_id=USER_ID,
|
||||||
|
provider=APIProvider.EXA,
|
||||||
|
endpoint="/api/podcast/research/exa",
|
||||||
|
model="exa-search",
|
||||||
|
tokens_in=0,
|
||||||
|
tokens_out=0,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 2c) At least one video render
|
||||||
|
video_start = _post_json(
|
||||||
|
client,
|
||||||
|
"/api/podcast/render/video",
|
||||||
|
{
|
||||||
|
"project_id": "sequence-project-001",
|
||||||
|
"scene_id": "scene_1",
|
||||||
|
"scene_title": "Intro",
|
||||||
|
"audio_url": audio_url,
|
||||||
|
"avatar_image_url": avatar_image_path,
|
||||||
|
"resolution": "720p",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Fetch task status once (background task should be done quickly with mocks)
|
||||||
|
task_id = video_start["task_id"]
|
||||||
|
task_status = _get_json(client, f"/api/podcast/task/{task_id}/status")
|
||||||
|
_record_usage(
|
||||||
|
user_id=USER_ID,
|
||||||
|
provider=APIProvider.VIDEO,
|
||||||
|
endpoint="/api/podcast/render/video",
|
||||||
|
model="wavespeed-ai/infinitetalk",
|
||||||
|
tokens_in=0,
|
||||||
|
tokens_out=0,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 3) Verify usage logs/dashboard deltas
|
||||||
|
after_dashboard = _get_json(client, f"/api/subscription/dashboard/{USER_ID}?billing_period={BILLING_PERIOD}")
|
||||||
|
after_logs = _get_json(client, "/api/subscription/usage-logs?limit=500")
|
||||||
|
|
||||||
|
after_cost = float(after_dashboard["data"]["summary"]["total_cost_this_month"])
|
||||||
|
after_calls = int(after_dashboard["data"]["summary"]["total_api_calls_this_month"])
|
||||||
|
after_projection = float(after_dashboard["data"]["projections"]["projected_monthly_cost"])
|
||||||
|
after_provider_costs = _provider_cost_totals(after_logs)
|
||||||
|
|
||||||
|
delta_cost = round(after_cost - before_cost, 4)
|
||||||
|
delta_calls = after_calls - before_calls
|
||||||
|
delta_projection = round(after_projection - before_projection, 4)
|
||||||
|
|
||||||
|
# Provider deltas (focus on providers touched in sequence)
|
||||||
|
provider_deltas = {
|
||||||
|
key: round(after_provider_costs.get(key, 0.0) - before_provider_costs.get(key, 0.0), 4)
|
||||||
|
for key in sorted(set(before_provider_costs) | set(after_provider_costs))
|
||||||
|
if key in {"exa", "huggingface", "wavespeed", "video", "mistral"}
|
||||||
|
}
|
||||||
|
|
||||||
|
expected_positive_cost = delta_cost > 0
|
||||||
|
expected_positive_calls = delta_calls >= 3 # analysis + research + video
|
||||||
|
expected_projection_change = delta_projection > 0
|
||||||
|
expected_provider_delta = any(v > 0 for v in provider_deltas.values())
|
||||||
|
|
||||||
|
acceptance_passed = all(
|
||||||
|
[
|
||||||
|
preflight.get("success") is True,
|
||||||
|
expected_positive_cost,
|
||||||
|
expected_positive_calls,
|
||||||
|
expected_projection_change,
|
||||||
|
expected_provider_delta,
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
report = {
|
||||||
|
"preflight": {
|
||||||
|
"success": preflight.get("success"),
|
||||||
|
"can_proceed": preflight.get("data", {}).get("can_proceed"),
|
||||||
|
"estimated_cost": preflight.get("data", {}).get("estimated_cost"),
|
||||||
|
},
|
||||||
|
"operations": {
|
||||||
|
"analysis_title_suggestions": analysis.get("title_suggestions", []),
|
||||||
|
"research_provider": research.get("provider"),
|
||||||
|
"research_cost": (research.get("cost") or {}).get("total"),
|
||||||
|
"video_task_status": task_status.get("status"),
|
||||||
|
},
|
||||||
|
"dashboard_deltas": {
|
||||||
|
"total_calls_before": before_calls,
|
||||||
|
"total_calls_after": after_calls,
|
||||||
|
"delta_calls": delta_calls,
|
||||||
|
"total_cost_before": before_cost,
|
||||||
|
"total_cost_after": after_cost,
|
||||||
|
"delta_cost": delta_cost,
|
||||||
|
"projected_monthly_cost_before": before_projection,
|
||||||
|
"projected_monthly_cost_after": after_projection,
|
||||||
|
"delta_projected_monthly_cost": delta_projection,
|
||||||
|
},
|
||||||
|
"provider_cost_deltas": provider_deltas,
|
||||||
|
"acceptance": {
|
||||||
|
"passed": acceptance_passed,
|
||||||
|
"criteria": {
|
||||||
|
"preflight_success": preflight.get("success") is True,
|
||||||
|
"usage_cost_incremented": expected_positive_cost,
|
||||||
|
"usage_call_incremented": expected_positive_calls,
|
||||||
|
"projection_incremented": expected_projection_change,
|
||||||
|
"provider_delta_present": expected_provider_delta,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
out_dir = Path("artifacts")
|
||||||
|
out_dir.mkdir(exist_ok=True)
|
||||||
|
out_file = out_dir / "podcast_billing_sequence_report.json"
|
||||||
|
out_file.write_text(json.dumps(report, indent=2), encoding="utf-8")
|
||||||
|
|
||||||
|
print(json.dumps(report, indent=2))
|
||||||
|
print(f"\nSaved report: {out_file}")
|
||||||
|
|
||||||
|
if not acceptance_passed:
|
||||||
|
raise SystemExit(1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
173
backend/scripts/smoke_test_podcast_demo.py
Normal file
173
backend/scripts/smoke_test_podcast_demo.py
Normal file
@@ -0,0 +1,173 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Smoke test script for podcast-only demo mode.
|
||||||
|
Tests the subscription funnel, Stripe flow, and podcast runtime paths.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import requests
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from typing import Dict, Any
|
||||||
|
|
||||||
|
BASE_URL = "http://localhost:8000"
|
||||||
|
|
||||||
|
|
||||||
|
def test_health() -> bool:
|
||||||
|
"""Test backend health endpoint."""
|
||||||
|
print("\n[TEST] Backend health check...")
|
||||||
|
try:
|
||||||
|
resp = requests.get(f"{BASE_URL}/health", timeout=10)
|
||||||
|
data = resp.json()
|
||||||
|
print(f" Status: {data.get('status')}")
|
||||||
|
print(f" Demo mode: {data.get('podcast_only_demo_mode')}")
|
||||||
|
return resp.status_code == 200
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ FAILED: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_router_status() -> bool:
|
||||||
|
"""Test router status endpoint."""
|
||||||
|
print("\n[TEST] Router status...")
|
||||||
|
try:
|
||||||
|
resp = requests.get(f"{BASE_URL}/api/routers/status", timeout=10)
|
||||||
|
data = resp.json()
|
||||||
|
|
||||||
|
# Check critical routers
|
||||||
|
podcast_mounted = data.get("podcast_only_demo_mode", False)
|
||||||
|
router_groups = data.get("router_groups", {})
|
||||||
|
|
||||||
|
print(f" Podcast router: {router_groups.get('podcast_maker', {}).get('mounted')}")
|
||||||
|
print(f" Assets serving: {router_groups.get('assets_serving', {}).get('mounted')}")
|
||||||
|
|
||||||
|
# Check podcast router is always mounted
|
||||||
|
podcast_ok = router_groups.get('podcast_maker', {}).get('mounted') == True
|
||||||
|
if not podcast_ok:
|
||||||
|
print(" ❌ Podcast router not mounted!")
|
||||||
|
return False
|
||||||
|
|
||||||
|
return resp.status_code == 200
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ FAILED: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_subscription_plans() -> bool:
|
||||||
|
"""Test subscription plans endpoint."""
|
||||||
|
print("\n[TEST] Subscription plans...")
|
||||||
|
try:
|
||||||
|
resp = requests.get(f"{BASE_URL}/api/subscription/plans", timeout=10)
|
||||||
|
data = resp.json()
|
||||||
|
|
||||||
|
if resp.status_code == 200:
|
||||||
|
plans = data.get("plans", [])
|
||||||
|
print(f" Plans returned: {len(plans)}")
|
||||||
|
for plan in plans[:3]:
|
||||||
|
print(f" - {plan.get('name')}: ${plan.get('price', {}).get('monthly', 'N/A')}/mo")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
print(f" ❌ Status {resp.status_code}")
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ FAILED: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_podcast_routes() -> bool:
|
||||||
|
"""Test podcast router is accessible."""
|
||||||
|
print("\n[TEST] Podcast router endpoints...")
|
||||||
|
try:
|
||||||
|
# Test without auth (should return 401, not 404)
|
||||||
|
resp = requests.get(f"{BASE_URL}/api/podcast/projects", timeout=10)
|
||||||
|
|
||||||
|
if resp.status_code == 401:
|
||||||
|
print(" ✅ Podcast router mounted (auth required as expected)")
|
||||||
|
return True
|
||||||
|
elif resp.status_code == 404:
|
||||||
|
print(" ❌ Podcast router NOT mounted (404)")
|
||||||
|
return False
|
||||||
|
else:
|
||||||
|
print(f" Status: {resp.status_code}")
|
||||||
|
return resp.status_code in [200, 401]
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ FAILED: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_preflight() -> bool:
|
||||||
|
"""Test preflight cost estimation endpoint."""
|
||||||
|
print("\n[TEST] Preflight cost estimation...")
|
||||||
|
try:
|
||||||
|
resp = requests.post(
|
||||||
|
f"{BASE_URL}/api/subscription/preflight-check",
|
||||||
|
json={"operation": "podcast_analysis", "tier": "basic"},
|
||||||
|
timeout=10
|
||||||
|
)
|
||||||
|
|
||||||
|
if resp.status_code in [200, 401]:
|
||||||
|
print(f" ✅ Preflight endpoint accessible (status: {resp.status_code})")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
print(f" ❌ Status {resp.status_code}")
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ FAILED: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_onboarding_status() -> bool:
|
||||||
|
"""Test onboarding status endpoint."""
|
||||||
|
print("\n[TEST] Onboarding status...")
|
||||||
|
try:
|
||||||
|
resp = requests.get(f"{BASE_URL}/api/onboarding/status", timeout=10)
|
||||||
|
data = resp.json()
|
||||||
|
|
||||||
|
print(f" Status: {data.get('status')}")
|
||||||
|
print(f" Enabled: {data.get('enabled')}")
|
||||||
|
|
||||||
|
# In demo mode, should be disabled
|
||||||
|
if data.get('enabled') == False:
|
||||||
|
print(" ✅ Onboarding correctly disabled in demo mode")
|
||||||
|
return True
|
||||||
|
|
||||||
|
return resp.status_code == 200
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ FAILED: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""Run all smoke tests."""
|
||||||
|
print("=" * 60)
|
||||||
|
print("PODCAST-ONLY DEMO MODE SMOKE TESTS")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
results = []
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
results.append(("Health", test_health()))
|
||||||
|
results.append(("Router Status", test_router_status()))
|
||||||
|
results.append(("Subscription Plans", test_subscription_plans()))
|
||||||
|
results.append(("Podcast Routes", test_podcast_routes()))
|
||||||
|
results.append(("Preflight Check", test_preflight()))
|
||||||
|
results.append(("Onboarding Status", test_onboarding_status()))
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("SUMMARY")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
passed = sum(1 for _, r in results if r)
|
||||||
|
total = len(results)
|
||||||
|
|
||||||
|
for name, result in results:
|
||||||
|
status = "✅ PASS" if result else "❌ FAIL"
|
||||||
|
print(f" {status}: {name}")
|
||||||
|
|
||||||
|
print(f"\nTotal: {passed}/{total} tests passed")
|
||||||
|
|
||||||
|
return 0 if passed == total else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -351,16 +351,15 @@ def init_database():
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
# Create all tables for all models using default engine
|
# Create all tables for all models using default engine
|
||||||
OnboardingBase.metadata.create_all(bind=default_engine)
|
# Use checkfirst=True (default) to avoid errors for existing tables
|
||||||
SEOAnalysisBase.metadata.create_all(bind=default_engine)
|
from sqlalchemy import create_engine
|
||||||
ContentPlanningBase.metadata.create_all(bind=default_engine)
|
from sqlalchemy.pool import StaticPool
|
||||||
EnhancedStrategyBase.metadata.create_all(bind=default_engine)
|
|
||||||
MonitoringBase.metadata.create_all(bind=default_engine)
|
# Create tables with checkfirst=True explicitly to handle existing objects
|
||||||
APIMonitoringBase.metadata.create_all(bind=default_engine)
|
for base in [OnboardingBase, SEOAnalysisBase, ContentPlanningBase,
|
||||||
PersonaBase.metadata.create_all(bind=default_engine)
|
EnhancedStrategyBase, MonitoringBase, APIMonitoringBase,
|
||||||
SubscriptionBase.metadata.create_all(bind=default_engine)
|
PersonaBase, SubscriptionBase, UserBusinessInfoBase, ContentAssetBase]:
|
||||||
UserBusinessInfoBase.metadata.create_all(bind=default_engine)
|
base.metadata.create_all(bind=default_engine, checkfirst=True)
|
||||||
ContentAssetBase.metadata.create_all(bind=default_engine)
|
|
||||||
logger.info("Global database initialized successfully")
|
logger.info("Global database initialized successfully")
|
||||||
except SQLAlchemyError as e:
|
except SQLAlchemyError as e:
|
||||||
logger.error(f"Error initializing global database: {str(e)}")
|
logger.error(f"Error initializing global database: {str(e)}")
|
||||||
|
|||||||
745
backend/services/intelligence/agent_context_vfs.py
Normal file
745
backend/services/intelligence/agent_context_vfs.py
Normal file
@@ -0,0 +1,745 @@
|
|||||||
|
"""Read-only virtual filesystem facade for agent flat context documents.
|
||||||
|
|
||||||
|
This adapter provides shell-like primitives (`list_context`, `search_context`,
|
||||||
|
`read_context_file`) over the JSON documents managed by AgentFlatContextStore.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import os
|
||||||
|
import fcntl
|
||||||
|
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||||
|
from collections import deque
|
||||||
|
from fnmatch import fnmatch
|
||||||
|
from pathlib import Path
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any, Dict, List, Optional, Tuple
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from services.intelligence.agent_flat_context import AgentFlatContextStore
|
||||||
|
|
||||||
|
|
||||||
|
class SmartGrepEngine:
|
||||||
|
"""Streaming grep engine with regex fallback and contextual snippets."""
|
||||||
|
|
||||||
|
def __init__(self, context_window: int = 1):
|
||||||
|
self.context_window = max(0, int(context_window))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _compile_pattern(pattern: str) -> re.Pattern:
|
||||||
|
try:
|
||||||
|
return re.compile(pattern, re.IGNORECASE)
|
||||||
|
except re.error:
|
||||||
|
return re.compile(re.escape(pattern), re.IGNORECASE)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _truncate(text: str, limit: int = 180) -> str:
|
||||||
|
text = " ".join(text.split())
|
||||||
|
if len(text) <= limit:
|
||||||
|
return text
|
||||||
|
return text[:limit] + "..."
|
||||||
|
|
||||||
|
def stream_file(self, file_path: Path, pattern: str, *, path_label: str) -> List[Dict[str, Any]]:
|
||||||
|
regex = self._compile_pattern(pattern)
|
||||||
|
matches: List[Dict[str, Any]] = []
|
||||||
|
prev = deque(maxlen=self.context_window)
|
||||||
|
active: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
with open(file_path, "r", encoding="utf-8", errors="ignore") as f:
|
||||||
|
for line_no, line in enumerate(f, start=1):
|
||||||
|
# Fill trailing context for active matches.
|
||||||
|
for item in active:
|
||||||
|
if item["remaining_after"] > 0:
|
||||||
|
item["after"].append(line.rstrip("\n"))
|
||||||
|
item["remaining_after"] -= 1
|
||||||
|
|
||||||
|
# Detect a new match on current line.
|
||||||
|
if regex.search(line):
|
||||||
|
current = line.rstrip("\n")
|
||||||
|
record = {
|
||||||
|
"path": path_label,
|
||||||
|
"line": line_no,
|
||||||
|
"before": list(prev),
|
||||||
|
"match_line": current,
|
||||||
|
"after": [],
|
||||||
|
"remaining_after": self.context_window,
|
||||||
|
}
|
||||||
|
active.append(record)
|
||||||
|
matches.append(record)
|
||||||
|
|
||||||
|
prev.append(line.rstrip("\n"))
|
||||||
|
|
||||||
|
formatted: List[Dict[str, Any]] = []
|
||||||
|
for m in matches:
|
||||||
|
snippet_parts = [*m["before"], m["match_line"], *m["after"]]
|
||||||
|
snippet = self._truncate(" | ".join([p for p in snippet_parts if p is not None]))
|
||||||
|
line_l = m["match_line"].lower()
|
||||||
|
is_high_signal = any(k in line_l for k in ("agent_summary", "high_signal_terms", "quick_facts"))
|
||||||
|
formatted.append(
|
||||||
|
{
|
||||||
|
"path": m["path"],
|
||||||
|
"line": m["line"],
|
||||||
|
"snippet": snippet,
|
||||||
|
"relevance": "High Relevance" if is_high_signal else "Supporting Detail",
|
||||||
|
"reason": "matched summary field in stream" if is_high_signal else "matched streamed body line",
|
||||||
|
"score": 70 if is_high_signal else 50,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return formatted
|
||||||
|
|
||||||
|
|
||||||
|
class AgentContextVFS:
|
||||||
|
"""Read-only adapter that maps virtual paths to flat context documents."""
|
||||||
|
|
||||||
|
VIRTUAL_MAP = {
|
||||||
|
"/steps/website": AgentFlatContextStore.STEP2_FILENAME,
|
||||||
|
"/steps/research": AgentFlatContextStore.STEP3_FILENAME,
|
||||||
|
"/steps/persona": AgentFlatContextStore.STEP4_FILENAME,
|
||||||
|
"/steps/integrations": AgentFlatContextStore.STEP5_FILENAME,
|
||||||
|
}
|
||||||
|
HIGH_SIGNAL_MARKERS = ("agent_summary", "high_signal_terms", "quick_facts", "context_type")
|
||||||
|
|
||||||
|
def __init__(self, user_id: str, project_id: Optional[str] = None):
|
||||||
|
self.user_id = user_id
|
||||||
|
self.project_id = project_id
|
||||||
|
self.store = AgentFlatContextStore(user_id)
|
||||||
|
self.grep_engine = SmartGrepEngine(context_window=1)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _safe_slug(value: Optional[str], fallback: str) -> str:
|
||||||
|
raw = str(value or "").strip()
|
||||||
|
safe = "".join(c for c in raw if c.isalnum() or c in ("-", "_"))
|
||||||
|
return safe or fallback
|
||||||
|
|
||||||
|
def _manifest_docs(self) -> List[Dict[str, Any]]:
|
||||||
|
manifest = self.store.load_context_manifest() or {"documents": []}
|
||||||
|
docs = manifest.get("documents")
|
||||||
|
return docs if isinstance(docs, list) else []
|
||||||
|
|
||||||
|
def _workspace_root(self) -> Path:
|
||||||
|
if self.project_id:
|
||||||
|
root_dir = Path(__file__).resolve().parents[3]
|
||||||
|
safe_project = self._safe_slug(self.project_id, "default_project")
|
||||||
|
project_root = root_dir / "workspace" / f"project_{safe_project}"
|
||||||
|
project_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
os.chmod(project_root, 0o700)
|
||||||
|
return project_root
|
||||||
|
return self.store._workspace_dir()
|
||||||
|
|
||||||
|
def _scratchpad_dir(self) -> Path:
|
||||||
|
scratch = self._workspace_root() / "scratchpad"
|
||||||
|
scratch.mkdir(parents=True, exist_ok=True)
|
||||||
|
os.chmod(scratch, 0o700)
|
||||||
|
return scratch
|
||||||
|
|
||||||
|
def _allowlisted_workspace_files(self) -> List[Path]:
|
||||||
|
"""Return sandboxed files eligible for streaming search."""
|
||||||
|
files: List[Path] = []
|
||||||
|
workspace = self._workspace_root()
|
||||||
|
context_dir = self.store._context_dir()
|
||||||
|
|
||||||
|
# 1) manifest-backed onboarding context files
|
||||||
|
for item in self._manifest_docs():
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
rel = str(item.get("path") or "")
|
||||||
|
if not rel:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
candidate = self.store._safe_resolve_under(context_dir, rel)
|
||||||
|
if candidate.exists() and candidate.is_file():
|
||||||
|
files.append(candidate)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# 2) workspace text artifacts (README, operator notes, etc.)
|
||||||
|
for candidate in workspace.glob("*.txt"):
|
||||||
|
if candidate.is_file():
|
||||||
|
files.append(candidate.resolve())
|
||||||
|
readme = workspace / "README.md"
|
||||||
|
if readme.exists() and readme.is_file():
|
||||||
|
files.append(readme.resolve())
|
||||||
|
|
||||||
|
# dedupe
|
||||||
|
seen = set()
|
||||||
|
unique: List[Path] = []
|
||||||
|
for p in files:
|
||||||
|
rp = str(p)
|
||||||
|
if rp in seen:
|
||||||
|
continue
|
||||||
|
seen.add(rp)
|
||||||
|
unique.append(p)
|
||||||
|
return unique
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _query_variants(query: str) -> List[str]:
|
||||||
|
"""Generate normalized and synonym-expanded query variants."""
|
||||||
|
base = (query or "").strip().lower()
|
||||||
|
if not base:
|
||||||
|
return []
|
||||||
|
synonyms = {
|
||||||
|
"tone": ["brand voice", "writing tone"],
|
||||||
|
"voice": ["brand voice", "writing style"],
|
||||||
|
"competitor": ["competition", "rival"],
|
||||||
|
"seo": ["search", "metadata"],
|
||||||
|
"persona": ["audience profile", "target audience"],
|
||||||
|
}
|
||||||
|
variants = [base]
|
||||||
|
tokens = base.split()
|
||||||
|
for idx, tok in enumerate(tokens):
|
||||||
|
if tok in synonyms:
|
||||||
|
for repl in synonyms[tok]:
|
||||||
|
new_tokens = tokens.copy()
|
||||||
|
new_tokens[idx] = repl
|
||||||
|
variants.append(" ".join(new_tokens))
|
||||||
|
variants.extend([base.replace("-", " "), base.replace("_", " ")])
|
||||||
|
# dedupe, preserve order
|
||||||
|
seen = set()
|
||||||
|
out: List[str] = []
|
||||||
|
for v in variants:
|
||||||
|
vv = v.strip()
|
||||||
|
if not vv or vv in seen:
|
||||||
|
continue
|
||||||
|
seen.add(vv)
|
||||||
|
out.append(vv)
|
||||||
|
return out
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _freshness_score(updated_at: Optional[str]) -> float:
|
||||||
|
if not updated_at:
|
||||||
|
return 0.3
|
||||||
|
try:
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
ts = datetime.fromisoformat(str(updated_at).replace("Z", "+00:00"))
|
||||||
|
if ts.tzinfo is None:
|
||||||
|
ts = ts.replace(tzinfo=timezone.utc)
|
||||||
|
days = max(0.0, (datetime.now(timezone.utc) - ts).total_seconds() / 86400.0)
|
||||||
|
if days <= 1:
|
||||||
|
return 1.0
|
||||||
|
if days <= 7:
|
||||||
|
return 0.9
|
||||||
|
if days <= 30:
|
||||||
|
return 0.75
|
||||||
|
if days <= 90:
|
||||||
|
return 0.6
|
||||||
|
return 0.4
|
||||||
|
except Exception:
|
||||||
|
return 0.3
|
||||||
|
|
||||||
|
def _cluster_results(self, results: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||||
|
"""Deduplicate repeated hits by file + reason and keep strongest evidence."""
|
||||||
|
buckets: Dict[Tuple[str, str], Dict[str, Any]] = {}
|
||||||
|
for r in results:
|
||||||
|
path = str(r.get("path") or "")
|
||||||
|
reason = str(r.get("reason") or "")
|
||||||
|
key = (path, reason)
|
||||||
|
existing = buckets.get(key)
|
||||||
|
if not existing:
|
||||||
|
buckets[key] = {**r, "hit_count": 1}
|
||||||
|
continue
|
||||||
|
existing["hit_count"] = int(existing.get("hit_count", 1)) + 1
|
||||||
|
if int(r.get("score", 0)) > int(existing.get("score", 0)):
|
||||||
|
existing.update({k: v for k, v in r.items() if k != "hit_count"})
|
||||||
|
existing["hit_count"] = int(existing.get("hit_count", 1))
|
||||||
|
clustered = list(buckets.values())
|
||||||
|
clustered.sort(key=lambda r: (-int(r.get("score", 0)), str(r.get("path") or "")))
|
||||||
|
return clustered
|
||||||
|
|
||||||
|
def _keyword_density(self, snippet: str, query: str) -> float:
|
||||||
|
if not snippet or not query:
|
||||||
|
return 0.0
|
||||||
|
query_tokens = [t for t in query.lower().split() if t]
|
||||||
|
if not query_tokens:
|
||||||
|
return 0.0
|
||||||
|
text = snippet.lower()
|
||||||
|
hits = sum(text.count(tok) for tok in query_tokens)
|
||||||
|
words = max(1, len(text.split()))
|
||||||
|
return hits / words
|
||||||
|
|
||||||
|
def _static_triage(self, results: List[Dict[str, Any]], query: str) -> List[Dict[str, Any]]:
|
||||||
|
"""Semgrep-style static heuristic triage before main agent consumption."""
|
||||||
|
triaged: List[Dict[str, Any]] = []
|
||||||
|
for r in results:
|
||||||
|
snippet = str(r.get("snippet") or "")
|
||||||
|
density = self._keyword_density(snippet, query)
|
||||||
|
marker_hit = any(marker in snippet.lower() for marker in self.HIGH_SIGNAL_MARKERS)
|
||||||
|
low_probability = bool(density < 0.01 and not marker_hit)
|
||||||
|
item = dict(r)
|
||||||
|
item["keyword_density"] = round(density, 4)
|
||||||
|
item["low_probability"] = low_probability
|
||||||
|
triaged.append(item)
|
||||||
|
triaged.sort(
|
||||||
|
key=lambda x: (
|
||||||
|
bool(x.get("low_probability")),
|
||||||
|
-float(x.get("confidence", 0)),
|
||||||
|
-int(x.get("score", 0)),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return triaged
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _llm_router_stub(results: List[Dict[str, Any]], top_k: int = 5) -> List[Dict[str, Any]]:
|
||||||
|
"""Fast local triage stub (drop low-probability first; keep strongest candidates)."""
|
||||||
|
ranked = sorted(
|
||||||
|
results,
|
||||||
|
key=lambda x: (
|
||||||
|
bool(x.get("low_probability")),
|
||||||
|
-float(x.get("confidence", 0)),
|
||||||
|
-int(x.get("score", 0)),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return ranked[: max(1, top_k)]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _resolve_json_path(data: Any, path_query: str) -> Any:
|
||||||
|
"""Resolve dot/bracket JSON path such as 'data.seo_audit.recommendations[0]'."""
|
||||||
|
if not path_query:
|
||||||
|
return data
|
||||||
|
|
||||||
|
current = data
|
||||||
|
query = path_query.strip()
|
||||||
|
parts: List[str] = []
|
||||||
|
buf = ""
|
||||||
|
in_brackets = False
|
||||||
|
for ch in query:
|
||||||
|
if ch == "." and not in_brackets:
|
||||||
|
if buf:
|
||||||
|
parts.append(buf)
|
||||||
|
buf = ""
|
||||||
|
continue
|
||||||
|
if ch == "[":
|
||||||
|
in_brackets = True
|
||||||
|
elif ch == "]":
|
||||||
|
in_brackets = False
|
||||||
|
buf += ch
|
||||||
|
if buf:
|
||||||
|
parts.append(buf)
|
||||||
|
|
||||||
|
for part in parts:
|
||||||
|
if "[" in part and part.endswith("]"):
|
||||||
|
key, idx_raw = part.split("[", 1)
|
||||||
|
idx = int(idx_raw[:-1])
|
||||||
|
if key:
|
||||||
|
if not isinstance(current, dict):
|
||||||
|
raise KeyError(key)
|
||||||
|
current = current[key]
|
||||||
|
if not isinstance(current, list):
|
||||||
|
raise IndexError(idx)
|
||||||
|
current = current[idx]
|
||||||
|
else:
|
||||||
|
if not isinstance(current, dict):
|
||||||
|
raise KeyError(part)
|
||||||
|
current = current[part]
|
||||||
|
return current
|
||||||
|
|
||||||
|
def _resolve_path(self, path: str) -> Tuple[str, Optional[str]]:
|
||||||
|
normalized = (path or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
return "", None
|
||||||
|
if normalized == "/env/summary":
|
||||||
|
return "virtual_summary", None
|
||||||
|
if normalized in self.VIRTUAL_MAP:
|
||||||
|
return "file", self.VIRTUAL_MAP[normalized]
|
||||||
|
if ".." in normalized or "\\" in normalized:
|
||||||
|
return "", None
|
||||||
|
if normalized.startswith("/"):
|
||||||
|
candidate = normalized.rsplit("/", 1)[-1]
|
||||||
|
else:
|
||||||
|
candidate = normalized
|
||||||
|
if "/" in candidate:
|
||||||
|
return "", None
|
||||||
|
allowed = AgentFlatContextStore.ALLOWED_CONTEXT_FILES - {AgentFlatContextStore.MANIFEST_FILENAME}
|
||||||
|
if candidate not in allowed:
|
||||||
|
return "", None
|
||||||
|
return "file", candidate
|
||||||
|
|
||||||
|
def list_context(self) -> Dict[str, Any]:
|
||||||
|
"""List available context files (ls-equivalent)."""
|
||||||
|
docs = self._manifest_docs()
|
||||||
|
items = []
|
||||||
|
for d in docs:
|
||||||
|
if not isinstance(d, dict):
|
||||||
|
continue
|
||||||
|
items.append(
|
||||||
|
{
|
||||||
|
"path": d.get("path"),
|
||||||
|
"type": d.get("type"),
|
||||||
|
"updated_at": d.get("updated_at"),
|
||||||
|
"size_bytes": d.get("size_bytes", 0),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
items.sort(key=lambda x: str(x.get("path") or ""))
|
||||||
|
result = {
|
||||||
|
"workspace_hint": "Use this list to see which onboarding steps are complete.",
|
||||||
|
"tip": "Use `search_context` to find specific keywords across all steps.",
|
||||||
|
"virtual_paths": ["/env/summary", *sorted(self.VIRTUAL_MAP.keys())],
|
||||||
|
"files": items,
|
||||||
|
"collaboration": {
|
||||||
|
"scratchpad_dir": str(self._scratchpad_dir()),
|
||||||
|
"activity_log": "scratchpad/activity_log.jsonl",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=list_context files={len(items)}")
|
||||||
|
return result
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _flatten_strings(data: Any, limit: int = 2000) -> str:
|
||||||
|
pieces: List[str] = []
|
||||||
|
|
||||||
|
def walk(v: Any) -> None:
|
||||||
|
if len(pieces) >= limit:
|
||||||
|
return
|
||||||
|
if isinstance(v, dict):
|
||||||
|
for key, value in v.items():
|
||||||
|
pieces.append(str(key))
|
||||||
|
walk(value)
|
||||||
|
elif isinstance(v, list):
|
||||||
|
for item in v:
|
||||||
|
walk(item)
|
||||||
|
elif isinstance(v, (str, int, float, bool)):
|
||||||
|
pieces.append(str(v))
|
||||||
|
|
||||||
|
walk(data)
|
||||||
|
return " ".join(pieces)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_search_fields(doc: Dict[str, Any]) -> Tuple[List[str], Dict[str, Any], str]:
|
||||||
|
summary = doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {}
|
||||||
|
hints = summary.get("retrieval_hints") if isinstance(summary.get("retrieval_hints"), dict) else {}
|
||||||
|
quick_facts = summary.get("quick_facts") if isinstance(summary.get("quick_facts"), dict) else {}
|
||||||
|
high_terms = hints.get("high_signal_terms") if isinstance(hints.get("high_signal_terms"), list) else []
|
||||||
|
body = AgentContextVFS._flatten_strings(doc.get("data") if isinstance(doc.get("data"), dict) else {})
|
||||||
|
return [str(t).lower() for t in high_terms], quick_facts, body.lower()
|
||||||
|
|
||||||
|
def search_context(self, query: str, *, limit: int = 10, path_glob: Optional[str] = None) -> Dict[str, Any]:
|
||||||
|
"""Smart grep with coarse-to-fine ranking and parallel stream scans."""
|
||||||
|
normalized = (query or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
return {"query": query, "results": []}
|
||||||
|
self.store._audit_event("vfs_search", normalized, "started")
|
||||||
|
try:
|
||||||
|
variants = self._query_variants(normalized)
|
||||||
|
attempted_queries: List[str] = []
|
||||||
|
scored: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
for candidate_query in variants:
|
||||||
|
attempted_queries.append(candidate_query)
|
||||||
|
needle = candidate_query.lower()
|
||||||
|
|
||||||
|
# Pass 1: summary-first ranking (high relevance)
|
||||||
|
docs = self._manifest_docs()
|
||||||
|
variant_scored: List[Dict[str, Any]] = []
|
||||||
|
for item in docs:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
path = str(item.get("path") or "")
|
||||||
|
if not path:
|
||||||
|
continue
|
||||||
|
if path_glob and not fnmatch(path, path_glob):
|
||||||
|
continue
|
||||||
|
doc = self.store.load_context_document(path) or {}
|
||||||
|
high_terms, quick_facts, _ = self._extract_search_fields(doc)
|
||||||
|
|
||||||
|
high_match = any(needle in term for term in high_terms)
|
||||||
|
quick_match = any(needle in str(v).lower() for v in quick_facts.values()) if isinstance(quick_facts, dict) else False
|
||||||
|
if not (high_match or quick_match):
|
||||||
|
continue
|
||||||
|
|
||||||
|
score = 100 if high_match else 80
|
||||||
|
reason = "matched high_signal_terms" if high_match else "matched quick_facts"
|
||||||
|
variant_scored.append(
|
||||||
|
{
|
||||||
|
"path": path,
|
||||||
|
"line": None,
|
||||||
|
"snippet": f"{reason}: {candidate_query}"[:100],
|
||||||
|
"type": item.get("type"),
|
||||||
|
"updated_at": item.get("updated_at"),
|
||||||
|
"relevance": "High Relevance",
|
||||||
|
"reason": reason,
|
||||||
|
"score": score,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Pass 2: parallelized stream scan over allowlisted workspace files.
|
||||||
|
allowlisted = self._allowlisted_workspace_files()
|
||||||
|
body_matches: List[Dict[str, Any]] = []
|
||||||
|
if allowlisted:
|
||||||
|
with ThreadPoolExecutor(max_workers=min(8, max(1, len(allowlisted)))) as pool:
|
||||||
|
future_map = {}
|
||||||
|
for p in allowlisted:
|
||||||
|
path_label = p.name
|
||||||
|
if path_glob and not fnmatch(path_label, path_glob):
|
||||||
|
continue
|
||||||
|
future = pool.submit(self.grep_engine.stream_file, p, candidate_query, path_label=path_label)
|
||||||
|
future_map[future] = path_label
|
||||||
|
|
||||||
|
for future in as_completed(future_map):
|
||||||
|
try:
|
||||||
|
body_matches.extend(future.result() or [])
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
|
||||||
|
variant_scored.extend(body_matches)
|
||||||
|
if variant_scored:
|
||||||
|
scored = variant_scored
|
||||||
|
break
|
||||||
|
|
||||||
|
scored = self._cluster_results(scored)
|
||||||
|
|
||||||
|
# Add confidence based on score + freshness + hit density.
|
||||||
|
for r in scored:
|
||||||
|
base = min(1.0, max(0.0, float(r.get("score", 0)) / 100.0))
|
||||||
|
freshness = self._freshness_score(r.get("updated_at"))
|
||||||
|
density = min(1.0, 0.2 + (int(r.get("hit_count", 1)) * 0.1))
|
||||||
|
confidence = round((base * 0.6) + (freshness * 0.25) + (density * 0.15), 3)
|
||||||
|
r["confidence"] = confidence
|
||||||
|
|
||||||
|
scored.sort(key=lambda r: (-int(r.get("score", 0)), str(r.get("path") or "")))
|
||||||
|
matched_files = sorted({str(r.get("path") or "") for r in scored if r.get("path")})
|
||||||
|
capped_results = scored[: max(1, limit)]
|
||||||
|
notice = None
|
||||||
|
if len(matched_files) > 10:
|
||||||
|
notice = f"Found {len(matched_files)} matches. Showing top 10. Use a more specific keyword to narrow down."
|
||||||
|
capped_results = scored[:10]
|
||||||
|
|
||||||
|
# Token/length budgeting (~2000 tokens ~= ~8000 chars).
|
||||||
|
budget_chars = 8000
|
||||||
|
bounded_results = []
|
||||||
|
used = 0
|
||||||
|
for r in capped_results:
|
||||||
|
snippet = str(r.get("snippet") or "")
|
||||||
|
cost = len(snippet) + 120 # account for metadata fields
|
||||||
|
if bounded_results and used + cost > budget_chars:
|
||||||
|
break
|
||||||
|
bounded_results.append(r)
|
||||||
|
used += cost
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"query": normalized,
|
||||||
|
"attempted_queries": attempted_queries,
|
||||||
|
"matched_files_count": len(matched_files),
|
||||||
|
"results": self._static_triage(bounded_results, normalized),
|
||||||
|
"notice": notice,
|
||||||
|
"char_budget_used": used,
|
||||||
|
"can_answer": bool(bounded_results),
|
||||||
|
}
|
||||||
|
result["triage_top5"] = self._llm_router_stub(result["results"], top_k=5)
|
||||||
|
logger.info(
|
||||||
|
f"[vfs_audit] user={self.store.safe_user_id} action=search_context query={normalized!r} results={len(result['results'])}"
|
||||||
|
)
|
||||||
|
self.store._audit_event("vfs_search", normalized, f"success_{len(result['results'])}_hits")
|
||||||
|
return result
|
||||||
|
except Exception as exc:
|
||||||
|
self.store._audit_event("vfs_search", normalized, f"failed_{exc.__class__.__name__}")
|
||||||
|
return {"query": normalized, "matched_files_count": 0, "results": [], "notice": "Search failed.", "can_answer": False}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _strip_technical_metadata(doc: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
sanitized = {
|
||||||
|
"context_type": doc.get("context_type"),
|
||||||
|
"updated_at": doc.get("updated_at"),
|
||||||
|
"journey": ((doc.get("document_context") or {}).get("journey") or {}) if isinstance(doc.get("document_context"), dict) else {},
|
||||||
|
"agent_summary": doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {},
|
||||||
|
"data": doc.get("data") if isinstance(doc.get("data"), dict) else {},
|
||||||
|
}
|
||||||
|
return sanitized
|
||||||
|
|
||||||
|
def inspect_file(self, path: str, *, key: Optional[str] = None, small_file_bytes: int = 5 * 1024) -> Dict[str, Any]:
|
||||||
|
"""Smart reader (cat/head equivalent) with summary-first behavior."""
|
||||||
|
kind, resolved = self._resolve_path(path)
|
||||||
|
if kind == "virtual_summary":
|
||||||
|
result = {
|
||||||
|
"path": "/env/summary",
|
||||||
|
"mode": "summary",
|
||||||
|
"data": self.store.generate_total_summary(),
|
||||||
|
}
|
||||||
|
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=read_context_file path=/env/summary mode=summary")
|
||||||
|
return result
|
||||||
|
|
||||||
|
if not resolved:
|
||||||
|
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=read_context_file path={path!r} status=rejected")
|
||||||
|
return {"error": "File not found", "path": path}
|
||||||
|
|
||||||
|
# JSON context doc path
|
||||||
|
doc = self.store.load_context_document(resolved)
|
||||||
|
if doc:
|
||||||
|
view = self._strip_technical_metadata(doc)
|
||||||
|
data = view.get("data") if isinstance(view.get("data"), dict) else {}
|
||||||
|
raw_size = self.store.estimate_size_bytes(view)
|
||||||
|
|
||||||
|
if key:
|
||||||
|
if key in data:
|
||||||
|
result = {
|
||||||
|
"path": resolved,
|
||||||
|
"mode": "key",
|
||||||
|
"key": key,
|
||||||
|
"agent_summary": view.get("agent_summary"),
|
||||||
|
"data": data.get(key),
|
||||||
|
}
|
||||||
|
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=key")
|
||||||
|
return result
|
||||||
|
logger.info(
|
||||||
|
f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=key_missing key={key}"
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"path": resolved,
|
||||||
|
"mode": "key_missing",
|
||||||
|
"key": key,
|
||||||
|
"available_keys": sorted(list(data.keys())),
|
||||||
|
"message": "Requested key not found. Choose one of available_keys.",
|
||||||
|
}
|
||||||
|
|
||||||
|
if raw_size <= small_file_bytes:
|
||||||
|
result = {
|
||||||
|
"path": resolved,
|
||||||
|
"mode": "full",
|
||||||
|
"data": view,
|
||||||
|
}
|
||||||
|
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=full")
|
||||||
|
return result
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"path": resolved,
|
||||||
|
"mode": "summary_plus_keys",
|
||||||
|
"size_bytes": raw_size,
|
||||||
|
"agent_summary": view.get("agent_summary"),
|
||||||
|
"keys": sorted(list(data.keys())),
|
||||||
|
"message": "File is large. Re-run with key to inspect a specific section.",
|
||||||
|
}
|
||||||
|
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=summary_plus_keys")
|
||||||
|
return result
|
||||||
|
|
||||||
|
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} status=not_found")
|
||||||
|
return {"error": "File not found", "path": path, "resolved": resolved}
|
||||||
|
|
||||||
|
def read_context_file(self, path: str, *, subkey: Optional[str] = None) -> Dict[str, Any]:
|
||||||
|
"""Backward-compatible alias for inspect_file."""
|
||||||
|
return self.inspect_file(path, key=subkey)
|
||||||
|
|
||||||
|
def write_context_file(self, *_args: Any, **_kwargs: Any) -> None:
|
||||||
|
"""Disallow writes from the agent-facing VFS."""
|
||||||
|
raise OSError("EROFS: read-only file system")
|
||||||
|
|
||||||
|
# Backward-compat function name requested in design docs.
|
||||||
|
inspect = inspect_file
|
||||||
|
|
||||||
|
def write_shared_note(self, note: str, *, agent_id: str = "agent", filename: str = "collaboration.md") -> Dict[str, Any]:
|
||||||
|
"""Append a shared project note with advisory locking in scratchpad."""
|
||||||
|
safe_name = Path(filename).name
|
||||||
|
if safe_name != filename or ".." in filename or "/" in filename or "\\" in filename:
|
||||||
|
self.store._audit_event("write_shared_note", filename, "rejected_filename")
|
||||||
|
return {"ok": False, "error": "Invalid filename"}
|
||||||
|
|
||||||
|
scratch = self._scratchpad_dir()
|
||||||
|
target = (scratch / safe_name).resolve()
|
||||||
|
if scratch.resolve() not in target.parents:
|
||||||
|
self.store._audit_event("write_shared_note", filename, "rejected_path")
|
||||||
|
return {"ok": False, "error": "Unsafe path"}
|
||||||
|
|
||||||
|
lock_path = scratch / f".{safe_name}.lock"
|
||||||
|
ts = datetime.now(timezone.utc).isoformat()
|
||||||
|
header = f"\n## {ts} | {self._safe_slug(agent_id, 'agent')}\n"
|
||||||
|
payload = header + str(note).rstrip() + "\n"
|
||||||
|
|
||||||
|
try:
|
||||||
|
with open(lock_path, "w", encoding="utf-8") as lf:
|
||||||
|
fcntl.flock(lf.fileno(), fcntl.LOCK_EX)
|
||||||
|
with open(target, "a", encoding="utf-8") as tf:
|
||||||
|
tf.write(payload)
|
||||||
|
tf.flush()
|
||||||
|
os.fsync(tf.fileno())
|
||||||
|
os.chmod(target, 0o600)
|
||||||
|
fcntl.flock(lf.fileno(), fcntl.LOCK_UN)
|
||||||
|
self.store._audit_event("write_shared_note", safe_name, "success")
|
||||||
|
self.append_activity_log(
|
||||||
|
event_type="shared_note_written",
|
||||||
|
actor=agent_id,
|
||||||
|
details={"file": safe_name, "bytes": len(payload)},
|
||||||
|
)
|
||||||
|
return {"ok": True, "file": safe_name, "bytes_written": len(payload)}
|
||||||
|
except Exception as exc:
|
||||||
|
self.store._audit_event("write_shared_note", safe_name, f"failed_{exc.__class__.__name__}")
|
||||||
|
return {"ok": False, "error": str(exc)}
|
||||||
|
|
||||||
|
def append_activity_log(self, *, event_type: str, actor: str, details: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
|
||||||
|
"""Write append-only project activity log entry in JSONL format."""
|
||||||
|
scratch = self._scratchpad_dir()
|
||||||
|
target = (scratch / "activity_log.jsonl").resolve()
|
||||||
|
lock_path = scratch / ".activity_log.jsonl.lock"
|
||||||
|
entry = {
|
||||||
|
"timestamp": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"event_type": str(event_type),
|
||||||
|
"actor": self._safe_slug(actor, "agent"),
|
||||||
|
"project_id": self._safe_slug(self.project_id, "none") if self.project_id else None,
|
||||||
|
"details": details or {},
|
||||||
|
}
|
||||||
|
line = json.dumps(entry, ensure_ascii=False) + "\n"
|
||||||
|
try:
|
||||||
|
with open(lock_path, "w", encoding="utf-8") as lf:
|
||||||
|
fcntl.flock(lf.fileno(), fcntl.LOCK_EX)
|
||||||
|
with open(target, "a", encoding="utf-8") as tf:
|
||||||
|
tf.write(line)
|
||||||
|
tf.flush()
|
||||||
|
os.fsync(tf.fileno())
|
||||||
|
os.chmod(target, 0o600)
|
||||||
|
fcntl.flock(lf.fileno(), fcntl.LOCK_UN)
|
||||||
|
return {"ok": True}
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"Failed to append activity log: {exc}")
|
||||||
|
return {"ok": False, "error": str(exc)}
|
||||||
|
|
||||||
|
def read_struct(self, filename: str, path_query: str) -> Dict[str, Any]:
|
||||||
|
"""AST-style structural reader for JSON context files with dependency context injection."""
|
||||||
|
resolved_kind, resolved = self._resolve_path(filename)
|
||||||
|
if resolved_kind == "virtual_summary" or not resolved:
|
||||||
|
return {"ok": False, "error": "Invalid file"}
|
||||||
|
|
||||||
|
doc = self.store.load_context_document(resolved)
|
||||||
|
if not isinstance(doc, dict):
|
||||||
|
return {"ok": False, "error": "File not found"}
|
||||||
|
|
||||||
|
try:
|
||||||
|
extracted = self._resolve_json_path(doc, path_query)
|
||||||
|
except Exception as exc:
|
||||||
|
return {"ok": False, "error": f"path_query resolution failed: {exc}"}
|
||||||
|
|
||||||
|
# Lightweight dependency context: inject brand voice from step2 when reading persona structures.
|
||||||
|
dependency_context: Dict[str, Any] = {}
|
||||||
|
if "persona" in path_query.lower() or resolved == AgentFlatContextStore.STEP4_FILENAME:
|
||||||
|
step2 = self.store.load_step2_context_document() or {}
|
||||||
|
step2_data = step2.get("data") if isinstance(step2.get("data"), dict) else {}
|
||||||
|
brand = step2_data.get("brand_analysis") if isinstance(step2_data.get("brand_analysis"), dict) else {}
|
||||||
|
dependency_context["brand_voice"] = brand.get("brand_voice")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"file": resolved,
|
||||||
|
"path_query": path_query,
|
||||||
|
"data": extracted,
|
||||||
|
"dependency_context": dependency_context,
|
||||||
|
"context": "Extracted via structural parse to save tokens.",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def build_filesystem_header(user_id: str) -> str:
|
||||||
|
"""Generate compact prompt header with available files and priority hints."""
|
||||||
|
try:
|
||||||
|
store = AgentFlatContextStore(user_id)
|
||||||
|
manifest = store.load_context_manifest() or {"documents": []}
|
||||||
|
docs = manifest.get("documents") if isinstance(manifest.get("documents"), list) else []
|
||||||
|
available = [str(d.get("path")) for d in docs if isinstance(d, dict) and d.get("path")]
|
||||||
|
files = ", ".join(sorted(available)) if available else "none"
|
||||||
|
return (
|
||||||
|
"Workspace Context: You have access to a local flat-file store. "
|
||||||
|
f"Available Files: {files}. "
|
||||||
|
"Instructions: For style guidelines, prioritize step4_persona_data.json. "
|
||||||
|
"For technical site data, prioritize step2_website_analysis.json."
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"Failed to build filesystem header for user {user_id}: {exc}")
|
||||||
|
return "Workspace Context: local flat-file store unavailable."
|
||||||
@@ -9,6 +9,8 @@ from __future__ import annotations
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import tempfile
|
import tempfile
|
||||||
|
import hmac
|
||||||
|
import hashlib
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, Optional, Tuple
|
from typing import Any, Dict, Optional, Tuple
|
||||||
@@ -25,6 +27,14 @@ class AgentFlatContextStore:
|
|||||||
STEP4_FILENAME = "step4_persona_data.json"
|
STEP4_FILENAME = "step4_persona_data.json"
|
||||||
STEP5_FILENAME = "step5_integrations.json"
|
STEP5_FILENAME = "step5_integrations.json"
|
||||||
MANIFEST_FILENAME = "context_manifest.json"
|
MANIFEST_FILENAME = "context_manifest.json"
|
||||||
|
WORKSPACE_README = "README.md"
|
||||||
|
ALLOWED_CONTEXT_FILES = {
|
||||||
|
STEP2_FILENAME,
|
||||||
|
STEP3_FILENAME,
|
||||||
|
STEP4_FILENAME,
|
||||||
|
STEP5_FILENAME,
|
||||||
|
MANIFEST_FILENAME,
|
||||||
|
}
|
||||||
|
|
||||||
SCHEMA_VERSION = "1.3"
|
SCHEMA_VERSION = "1.3"
|
||||||
DEFAULT_MAX_BYTES = 300_000
|
DEFAULT_MAX_BYTES = 300_000
|
||||||
@@ -33,12 +43,53 @@ class AgentFlatContextStore:
|
|||||||
def __init__(self, user_id: str):
|
def __init__(self, user_id: str):
|
||||||
self.user_id = user_id
|
self.user_id = user_id
|
||||||
self.safe_user_id = self._sanitize_user_id(user_id)
|
self.safe_user_id = self._sanitize_user_id(user_id)
|
||||||
|
self._ensure_workspace_permissions()
|
||||||
|
|
||||||
|
def _ensure_workspace_permissions(self) -> None:
|
||||||
|
"""Ensure workspace and context directories exist with owner-only permissions."""
|
||||||
|
workspace_dir = self._workspace_dir()
|
||||||
|
context_dir = workspace_dir / self.CONTEXT_DIRNAME
|
||||||
|
workspace_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
context_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
os.chmod(workspace_dir, 0o700)
|
||||||
|
os.chmod(context_dir, 0o700)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _safe_resolve_under(base_dir: Path, requested_path: str) -> Path:
|
||||||
|
"""Resolve path and ensure it remains inside base_dir (path sandboxing)."""
|
||||||
|
base_real = base_dir.resolve()
|
||||||
|
candidate = (base_dir / requested_path).resolve()
|
||||||
|
if candidate == base_real or base_real in candidate.parents:
|
||||||
|
return candidate
|
||||||
|
raise ValueError("Unsafe path access attempt outside sandbox")
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _sanitize_user_id(user_id: str) -> str:
|
def _sanitize_user_id(user_id: str) -> str:
|
||||||
safe = "".join(c for c in str(user_id) if c.isalnum() or c in ("-", "_"))
|
safe = "".join(c for c in str(user_id) if c.isalnum() or c in ("-", "_"))
|
||||||
return safe or "unknown_user"
|
return safe or "unknown_user"
|
||||||
|
|
||||||
|
def _master_salt(self) -> str:
|
||||||
|
return os.getenv("FILE_ENCRYPTION_SALT", "")
|
||||||
|
|
||||||
|
def derive_user_secret(self) -> bytes:
|
||||||
|
"""Derive deterministic per-user secret from env salt + safe user id."""
|
||||||
|
salt = self._master_salt()
|
||||||
|
if not salt:
|
||||||
|
return b""
|
||||||
|
return hmac.new(salt.encode("utf-8"), self.safe_user_id.encode("utf-8"), hashlib.sha256).digest()
|
||||||
|
|
||||||
|
def user_secret_fingerprint(self) -> str:
|
||||||
|
"""Short fingerprint used for diagnostics/audit only (not a key)."""
|
||||||
|
secret = self.derive_user_secret()
|
||||||
|
if not secret:
|
||||||
|
return "salt_not_configured"
|
||||||
|
return hashlib.sha256(secret).hexdigest()[:16]
|
||||||
|
|
||||||
|
def _audit_event(self, action: str, target: str, status: str) -> None:
|
||||||
|
logger.info(
|
||||||
|
f"[flat_context_audit] user={self.safe_user_id} action={action} target={target} status={status}"
|
||||||
|
)
|
||||||
|
|
||||||
def _workspace_dir(self) -> Path:
|
def _workspace_dir(self) -> Path:
|
||||||
root_dir = Path(__file__).resolve().parents[3]
|
root_dir = Path(__file__).resolve().parents[3]
|
||||||
return root_dir / "workspace" / f"workspace_{self.safe_user_id}"
|
return root_dir / "workspace" / f"workspace_{self.safe_user_id}"
|
||||||
@@ -47,7 +98,10 @@ class AgentFlatContextStore:
|
|||||||
return self._workspace_dir() / self.CONTEXT_DIRNAME
|
return self._workspace_dir() / self.CONTEXT_DIRNAME
|
||||||
|
|
||||||
def _context_file(self, filename: str) -> Path:
|
def _context_file(self, filename: str) -> Path:
|
||||||
return self._context_dir() / filename
|
return self._safe_resolve_under(self._context_dir(), str(filename))
|
||||||
|
|
||||||
|
def _workspace_file(self, filename: str) -> Path:
|
||||||
|
return self._safe_resolve_under(self._workspace_dir(), str(filename))
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _estimate_size_bytes(value: Any) -> int:
|
def _estimate_size_bytes(value: Any) -> int:
|
||||||
@@ -56,6 +110,10 @@ class AgentFlatContextStore:
|
|||||||
except Exception:
|
except Exception:
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
def estimate_size_bytes(self, value: Any) -> int:
|
||||||
|
"""Public size estimate helper for adapter layers."""
|
||||||
|
return self._estimate_size_bytes(value)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _to_context_list(value: Any) -> Any:
|
def _to_context_list(value: Any) -> Any:
|
||||||
if value is None:
|
if value is None:
|
||||||
@@ -143,6 +201,12 @@ class AgentFlatContextStore:
|
|||||||
"preferred": "flat_file",
|
"preferred": "flat_file",
|
||||||
"fallback_order": fallback_order,
|
"fallback_order": fallback_order,
|
||||||
},
|
},
|
||||||
|
"security": {
|
||||||
|
"path_sandboxing": True,
|
||||||
|
"file_permissions": "0600",
|
||||||
|
"directory_permissions": "0700",
|
||||||
|
"user_secret_fingerprint": self.user_secret_fingerprint(),
|
||||||
|
},
|
||||||
"context_window_guidance": {
|
"context_window_guidance": {
|
||||||
"max_raw_bytes": self.DEFAULT_MAX_BYTES,
|
"max_raw_bytes": self.DEFAULT_MAX_BYTES,
|
||||||
"total_bytes": total_size,
|
"total_bytes": total_size,
|
||||||
@@ -343,6 +407,7 @@ class AgentFlatContextStore:
|
|||||||
|
|
||||||
def _atomic_write_json(self, target_file: Path, data: Dict[str, Any]) -> None:
|
def _atomic_write_json(self, target_file: Path, data: Dict[str, Any]) -> None:
|
||||||
target_file.parent.mkdir(parents=True, exist_ok=True)
|
target_file.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
os.chmod(target_file.parent, 0o700)
|
||||||
fd, tmp_path = tempfile.mkstemp(dir=str(target_file.parent), prefix=f".{target_file.name}.", suffix=".tmp")
|
fd, tmp_path = tempfile.mkstemp(dir=str(target_file.parent), prefix=f".{target_file.name}.", suffix=".tmp")
|
||||||
try:
|
try:
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||||
@@ -361,6 +426,108 @@ class AgentFlatContextStore:
|
|||||||
pass
|
pass
|
||||||
raise
|
raise
|
||||||
|
|
||||||
|
def _atomic_write_text(self, target_file: Path, content: str) -> None:
|
||||||
|
target_file.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
os.chmod(target_file.parent, 0o700)
|
||||||
|
fd, tmp_path = tempfile.mkstemp(dir=str(target_file.parent), prefix=f".{target_file.name}.", suffix=".tmp")
|
||||||
|
try:
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||||
|
f.write(content)
|
||||||
|
f.flush()
|
||||||
|
os.fsync(f.fileno())
|
||||||
|
os.replace(tmp_path, target_file)
|
||||||
|
try:
|
||||||
|
os.chmod(target_file, 0o600)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
except Exception:
|
||||||
|
try:
|
||||||
|
os.unlink(tmp_path)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
raise
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _collect_signal_terms(doc: Dict[str, Any], limit: int = 6) -> list:
|
||||||
|
summary = doc.get("agent_summary") if isinstance(doc, dict) else {}
|
||||||
|
hints = summary.get("retrieval_hints") if isinstance(summary, dict) else {}
|
||||||
|
terms = hints.get("high_signal_terms") if isinstance(hints, dict) else []
|
||||||
|
if not isinstance(terms, list):
|
||||||
|
return []
|
||||||
|
normalized = [str(t).strip() for t in terms if str(t).strip()]
|
||||||
|
return normalized[:limit]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_journey_stage(doc: Dict[str, Any]) -> str:
|
||||||
|
dctx = doc.get("document_context") if isinstance(doc, dict) else {}
|
||||||
|
journey = dctx.get("journey") if isinstance(dctx, dict) else {}
|
||||||
|
stage = journey.get("stage") if isinstance(journey, dict) else ""
|
||||||
|
return str(stage or "").strip()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _context_description(filename: str) -> str:
|
||||||
|
descriptions = {
|
||||||
|
AgentFlatContextStore.STEP2_FILENAME: "Primary SEO and site structure context",
|
||||||
|
AgentFlatContextStore.STEP3_FILENAME: "Research depth, competitors, and content preferences",
|
||||||
|
AgentFlatContextStore.STEP4_FILENAME: "Persona profiles, voice adaptation, and platform strategy",
|
||||||
|
AgentFlatContextStore.STEP5_FILENAME: "Connected integrations and provider readiness",
|
||||||
|
}
|
||||||
|
return descriptions.get(filename, "Context document")
|
||||||
|
|
||||||
|
def _generate_workspace_readme(self, manifest: Dict[str, Any]) -> str:
|
||||||
|
docs = manifest.get("documents") if isinstance(manifest, dict) and isinstance(manifest.get("documents"), list) else []
|
||||||
|
|
||||||
|
lines = [
|
||||||
|
"# Agent Workspace Map",
|
||||||
|
"",
|
||||||
|
"You are in a restricted read-only VFS. Use `list_context`, `read_context_file`, and `search_context` to navigate.",
|
||||||
|
"",
|
||||||
|
"## Core Context Files",
|
||||||
|
]
|
||||||
|
|
||||||
|
for item in sorted(docs, key=lambda d: str((d or {}).get("path", ""))):
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
path = item.get("path") or ""
|
||||||
|
if not path:
|
||||||
|
continue
|
||||||
|
doc = self._load_context_document(path) or {}
|
||||||
|
signals = self._collect_signal_terms(doc)
|
||||||
|
journey_stage = self._extract_journey_stage(doc)
|
||||||
|
updated_at = str(item.get("updated_at") or "")
|
||||||
|
lines.append(f"- `{path}`: {self._context_description(path)}.")
|
||||||
|
if signals:
|
||||||
|
lines.append(f" - **Key Signals:** {', '.join(signals)}")
|
||||||
|
if journey_stage:
|
||||||
|
lines.append(f" - **Journey Stage:** {journey_stage}")
|
||||||
|
if updated_at:
|
||||||
|
lines.append(f" - **Updated:** {updated_at}")
|
||||||
|
|
||||||
|
lines.extend(
|
||||||
|
[
|
||||||
|
"",
|
||||||
|
"## Retrieval Strategy",
|
||||||
|
"1. Run `list_context` to check which onboarding steps are available.",
|
||||||
|
"2. Run `search_context` for targeted terms (for example: \"competitor\", \"tone\", \"integrations\").",
|
||||||
|
"3. Run `read_context_file` and ingest `agent_summary` before expanding full `data`.",
|
||||||
|
"",
|
||||||
|
"## Virtual Paths",
|
||||||
|
"- `/env/summary` -> consolidated summary generated from all available context docs",
|
||||||
|
f"- `/steps/website` -> `{self.STEP2_FILENAME}`",
|
||||||
|
f"- `/steps/research` -> `{self.STEP3_FILENAME}`",
|
||||||
|
f"- `/steps/persona` -> `{self.STEP4_FILENAME}`",
|
||||||
|
f"- `/steps/integrations` -> `{self.STEP5_FILENAME}`",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
def _update_workspace_readme(self, manifest: Dict[str, Any]) -> None:
|
||||||
|
try:
|
||||||
|
content = self._generate_workspace_readme(manifest)
|
||||||
|
self._atomic_write_text(self._workspace_file(self.WORKSPACE_README), content)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"Failed to update workspace README for user {self.user_id}: {exc}")
|
||||||
|
|
||||||
def _update_manifest(self, context_type: str, filename: str, doc: Dict[str, Any]) -> None:
|
def _update_manifest(self, context_type: str, filename: str, doc: Dict[str, Any]) -> None:
|
||||||
manifest_file = self._context_file(self.MANIFEST_FILENAME)
|
manifest_file = self._context_file(self.MANIFEST_FILENAME)
|
||||||
existing = {}
|
existing = {}
|
||||||
@@ -390,6 +557,7 @@ class AgentFlatContextStore:
|
|||||||
"documents": items,
|
"documents": items,
|
||||||
}
|
}
|
||||||
self._atomic_write_json(manifest_file, manifest)
|
self._atomic_write_json(manifest_file, manifest)
|
||||||
|
self._update_workspace_readme(manifest)
|
||||||
|
|
||||||
def _save_context_document(
|
def _save_context_document(
|
||||||
self,
|
self,
|
||||||
@@ -436,9 +604,11 @@ class AgentFlatContextStore:
|
|||||||
|
|
||||||
self._atomic_write_json(target_file, context_doc)
|
self._atomic_write_json(target_file, context_doc)
|
||||||
self._update_manifest(context_type, filename, context_doc)
|
self._update_manifest(context_type, filename, context_doc)
|
||||||
|
self._audit_event("write_context", filename, "success")
|
||||||
return True
|
return True
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error(f"Failed to save context for user {self.user_id} ({context_type}): {exc}")
|
logger.error(f"Failed to save context for user {self.user_id} ({context_type}): {exc}")
|
||||||
|
self._audit_event("write_context", filename, "error")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def save_step2_website_analysis(self, payload: Dict[str, Any], *, source: str = "onboarding_step2") -> bool:
|
def save_step2_website_analysis(self, payload: Dict[str, Any], *, source: str = "onboarding_step2") -> bool:
|
||||||
@@ -483,19 +653,31 @@ class AgentFlatContextStore:
|
|||||||
|
|
||||||
def _load_context_document(self, filename: str) -> Optional[Dict[str, Any]]:
|
def _load_context_document(self, filename: str) -> Optional[Dict[str, Any]]:
|
||||||
try:
|
try:
|
||||||
|
if str(filename) not in self.ALLOWED_CONTEXT_FILES:
|
||||||
|
logger.warning(f"Rejected non-allowed context filename for user {self.user_id}: {filename}")
|
||||||
|
self._audit_event("read_context", str(filename), "rejected_filename")
|
||||||
|
return None
|
||||||
target_file = self._context_file(filename)
|
target_file = self._context_file(filename)
|
||||||
if not target_file.exists():
|
if not target_file.exists():
|
||||||
|
self._audit_event("read_context", str(filename), "not_found")
|
||||||
return None
|
return None
|
||||||
with open(target_file, "r", encoding="utf-8") as f:
|
with open(target_file, "r", encoding="utf-8") as f:
|
||||||
doc = json.load(f)
|
doc = json.load(f)
|
||||||
if isinstance(doc, dict) and str(doc.get("user_id")) != str(self.user_id):
|
if isinstance(doc, dict) and str(doc.get("user_id")) != str(self.user_id):
|
||||||
logger.warning(f"Context user mismatch for {filename} (expected {self.user_id})")
|
logger.warning(f"Context user mismatch for {filename} (expected {self.user_id})")
|
||||||
|
self._audit_event("read_context", str(filename), "user_mismatch")
|
||||||
return None
|
return None
|
||||||
|
self._audit_event("read_context", str(filename), "success")
|
||||||
return doc if isinstance(doc, dict) else None
|
return doc if isinstance(doc, dict) else None
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.warning(f"Failed to load context document for user {self.user_id} ({filename}): {exc}")
|
logger.warning(f"Failed to load context document for user {self.user_id} ({filename}): {exc}")
|
||||||
|
self._audit_event("read_context", str(filename), "error")
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
def load_context_document(self, filename: str) -> Optional[Dict[str, Any]]:
|
||||||
|
"""Public loader for a named context document file."""
|
||||||
|
return self._load_context_document(filename)
|
||||||
|
|
||||||
def load_context_manifest(self) -> Optional[Dict[str, Any]]:
|
def load_context_manifest(self) -> Optional[Dict[str, Any]]:
|
||||||
return self._load_context_document(self.MANIFEST_FILENAME)
|
return self._load_context_document(self.MANIFEST_FILENAME)
|
||||||
|
|
||||||
@@ -526,3 +708,35 @@ class AgentFlatContextStore:
|
|||||||
def load_step5_integrations(self) -> Optional[Dict[str, Any]]:
|
def load_step5_integrations(self) -> Optional[Dict[str, Any]]:
|
||||||
doc = self.load_step5_context_document()
|
doc = self.load_step5_context_document()
|
||||||
return doc.get("data") if isinstance(doc, dict) and isinstance(doc.get("data"), dict) else None
|
return doc.get("data") if isinstance(doc, dict) and isinstance(doc.get("data"), dict) else None
|
||||||
|
|
||||||
|
def generate_total_summary(self) -> Dict[str, Any]:
|
||||||
|
"""Build a lightweight consolidated summary across available context documents."""
|
||||||
|
manifest = self.load_context_manifest() or {"documents": []}
|
||||||
|
docs = manifest.get("documents") if isinstance(manifest.get("documents"), list) else []
|
||||||
|
overview = []
|
||||||
|
for item in docs:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
path = str(item.get("path") or "")
|
||||||
|
if not path:
|
||||||
|
continue
|
||||||
|
doc = self._load_context_document(path) or {}
|
||||||
|
summary = doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {}
|
||||||
|
quick_facts = summary.get("quick_facts") if isinstance(summary.get("quick_facts"), dict) else {}
|
||||||
|
hints = summary.get("retrieval_hints") if isinstance(summary.get("retrieval_hints"), dict) else {}
|
||||||
|
overview.append(
|
||||||
|
{
|
||||||
|
"path": path,
|
||||||
|
"context_type": doc.get("context_type"),
|
||||||
|
"updated_at": doc.get("updated_at") or item.get("updated_at"),
|
||||||
|
"journey_stage": self._extract_journey_stage(doc),
|
||||||
|
"high_signal_terms": hints.get("high_signal_terms") if isinstance(hints.get("high_signal_terms"), list) else [],
|
||||||
|
"quick_facts": quick_facts,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"user_id": str(self.user_id),
|
||||||
|
"generated_at": datetime.utcnow().isoformat(),
|
||||||
|
"document_count": len(overview),
|
||||||
|
"documents": overview,
|
||||||
|
}
|
||||||
|
|||||||
@@ -340,6 +340,46 @@ class SIFIntegrationService:
|
|||||||
logger.warning(f"Failed to load flat context manifest for user {self.user_id}: {e}")
|
logger.warning(f"Failed to load flat context manifest for user {self.user_id}: {e}")
|
||||||
return {"source": "none", "data": {"documents": []}}
|
return {"source": "none", "data": {"documents": []}}
|
||||||
|
|
||||||
|
async def get_merged_flat_context(self) -> Dict[str, Any]:
|
||||||
|
"""Return merged onboarding context from all available flat context documents.
|
||||||
|
|
||||||
|
This is an aggregation helper; step-specific APIs still return one-by-one files.
|
||||||
|
"""
|
||||||
|
store = AgentFlatContextStore(self.user_id)
|
||||||
|
manifest = store.load_context_manifest() or {"documents": []}
|
||||||
|
docs = manifest.get("documents") if isinstance(manifest.get("documents"), list) else []
|
||||||
|
|
||||||
|
merged: Dict[str, Any] = {
|
||||||
|
"source": "flat_file",
|
||||||
|
"user_id": self.user_id,
|
||||||
|
"manifest_updated_at": manifest.get("updated_at"),
|
||||||
|
"steps": {},
|
||||||
|
"agent_summaries": {},
|
||||||
|
"documents": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
for item in docs:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
path = item.get("path")
|
||||||
|
if not path:
|
||||||
|
continue
|
||||||
|
doc = store.load_context_document(str(path)) or {}
|
||||||
|
context_type = str(doc.get("context_type") or item.get("type") or path)
|
||||||
|
merged["documents"].append(
|
||||||
|
{
|
||||||
|
"path": path,
|
||||||
|
"context_type": context_type,
|
||||||
|
"updated_at": doc.get("updated_at") or item.get("updated_at"),
|
||||||
|
"size_bytes": item.get("size_bytes"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
merged["steps"][context_type] = doc.get("data") if isinstance(doc.get("data"), dict) else {}
|
||||||
|
merged["agent_summaries"][context_type] = doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {}
|
||||||
|
|
||||||
|
merged["document_count"] = len(merged["documents"])
|
||||||
|
return merged
|
||||||
|
|
||||||
async def index_market_trends_run(self, trends_result: Dict[str, Any], run_id: str) -> bool:
|
async def index_market_trends_run(self, trends_result: Dict[str, Any], run_id: str) -> bool:
|
||||||
try:
|
try:
|
||||||
latest_id = f"market_trends_latest:{self.user_id}"
|
latest_id = f"market_trends_latest:{self.user_id}"
|
||||||
|
|||||||
@@ -410,8 +410,7 @@ class ContentGenerator:
|
|||||||
raise Exception("Gemini Grounded Provider not available - cannot generate content without AI provider")
|
raise Exception("Gemini Grounded Provider not available - cannot generate content without AI provider")
|
||||||
|
|
||||||
# Build the prompt for grounded generation using persona if available (DB vs session override)
|
# Build the prompt for grounded generation using persona if available (DB vs session override)
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(getattr(request, "user_id", 0) or 0)
|
||||||
user_id = 1
|
|
||||||
persona_data = self._get_cached_persona_data(user_id, 'linkedin')
|
persona_data = self._get_cached_persona_data(user_id, 'linkedin')
|
||||||
if getattr(request, 'persona_override', None):
|
if getattr(request, 'persona_override', None):
|
||||||
try:
|
try:
|
||||||
@@ -485,8 +484,7 @@ class ContentGenerator:
|
|||||||
raise Exception("Gemini Grounded Provider not available - cannot generate content without AI provider")
|
raise Exception("Gemini Grounded Provider not available - cannot generate content without AI provider")
|
||||||
|
|
||||||
# Build the prompt for grounded generation using persona if available (DB vs session override)
|
# Build the prompt for grounded generation using persona if available (DB vs session override)
|
||||||
# Beta testing: Force user_id=1 for all requests
|
user_id = int(getattr(request, "user_id", 0) or 0)
|
||||||
user_id = 1
|
|
||||||
persona_data = self._get_cached_persona_data(user_id, 'linkedin')
|
persona_data = self._get_cached_persona_data(user_id, 'linkedin')
|
||||||
if getattr(request, 'persona_override', None):
|
if getattr(request, 'persona_override', None):
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -250,10 +250,6 @@ def huggingface_text_response(
|
|||||||
|
|
||||||
logger.info("🚀 Making Hugging Face API call (chat completion)...")
|
logger.info("🚀 Making Hugging Face API call (chat completion)...")
|
||||||
|
|
||||||
# Add rate limiting to prevent expensive API calls
|
|
||||||
import time
|
|
||||||
time.sleep(1) # 1 second delay between API calls
|
|
||||||
|
|
||||||
response = None
|
response = None
|
||||||
last_error = None
|
last_error = None
|
||||||
for candidate_model in _fallback_model_sequence(model):
|
for candidate_model in _fallback_model_sequence(model):
|
||||||
@@ -403,10 +399,6 @@ def huggingface_structured_json_response(
|
|||||||
json_schema_str = json.dumps(schema, indent=2)
|
json_schema_str = json.dumps(schema, indent=2)
|
||||||
messages[-1]["content"] += f"\n\nJSON Schema:\n{json_schema_str}"
|
messages[-1]["content"] += f"\n\nJSON Schema:\n{json_schema_str}"
|
||||||
|
|
||||||
# Add rate limiting to prevent expensive API calls
|
|
||||||
import time
|
|
||||||
time.sleep(1) # 1 second delay between API calls
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
response = None
|
response = None
|
||||||
last_error = None
|
last_error = None
|
||||||
|
|||||||
@@ -62,6 +62,7 @@ class VoiceCloneResult:
|
|||||||
def generate_audio(
|
def generate_audio(
|
||||||
text: str,
|
text: str,
|
||||||
voice_id: str = "Wise_Woman",
|
voice_id: str = "Wise_Woman",
|
||||||
|
custom_voice_id: Optional[str] = None,
|
||||||
speed: float = 1.0,
|
speed: float = 1.0,
|
||||||
volume: float = 1.0,
|
volume: float = 1.0,
|
||||||
pitch: float = 0.0,
|
pitch: float = 0.0,
|
||||||
@@ -173,6 +174,7 @@ def generate_audio(
|
|||||||
audio_bytes = client.generate_speech(
|
audio_bytes = client.generate_speech(
|
||||||
text=text,
|
text=text,
|
||||||
voice_id=voice_id,
|
voice_id=voice_id,
|
||||||
|
custom_voice_id=custom_voice_id,
|
||||||
speed=speed,
|
speed=speed,
|
||||||
volume=volume,
|
volume=volume,
|
||||||
pitch=pitch,
|
pitch=pitch,
|
||||||
|
|||||||
@@ -55,6 +55,9 @@ def _select_provider(explicit: Optional[str]) -> str:
|
|||||||
def _get_provider_client(provider_name: str, api_key: Optional[str] = None):
|
def _get_provider_client(provider_name: str, api_key: Optional[str] = None):
|
||||||
"""Get the client for the specified provider."""
|
"""Get the client for the specified provider."""
|
||||||
if provider_name == "wavespeed":
|
if provider_name == "wavespeed":
|
||||||
|
api_key = api_key or os.getenv("WAVESPEED_API_KEY")
|
||||||
|
if not api_key:
|
||||||
|
raise RuntimeError("WAVESPEED_API_KEY is required for WaveSpeed image editing. Set it in your .env file.")
|
||||||
return WaveSpeedEditProvider(api_key=api_key)
|
return WaveSpeedEditProvider(api_key=api_key)
|
||||||
|
|
||||||
if not HF_HUB_AVAILABLE:
|
if not HF_HUB_AVAILABLE:
|
||||||
@@ -63,7 +66,7 @@ def _get_provider_client(provider_name: str, api_key: Optional[str] = None):
|
|||||||
if provider_name == "huggingface":
|
if provider_name == "huggingface":
|
||||||
api_key = api_key or os.getenv("HF_TOKEN")
|
api_key = api_key or os.getenv("HF_TOKEN")
|
||||||
if not api_key:
|
if not api_key:
|
||||||
raise RuntimeError("HF_TOKEN is required for Hugging Face image editing")
|
raise RuntimeError("HF_TOKEN is required for Hugging Face image editing. Set it in your .env file.")
|
||||||
# Use fal-ai provider for fast inference via HF Inference API
|
# Use fal-ai provider for fast inference via HF Inference API
|
||||||
return InferenceClient(provider="fal-ai", api_key=api_key)
|
return InferenceClient(provider="fal-ai", api_key=api_key)
|
||||||
|
|
||||||
@@ -99,35 +102,53 @@ def edit_image(
|
|||||||
"""
|
"""
|
||||||
# PRE-FLIGHT VALIDATION: Validate image editing before API call
|
# PRE-FLIGHT VALIDATION: Validate image editing before API call
|
||||||
# MUST happen BEFORE any API calls - return immediately if validation fails
|
# MUST happen BEFORE any API calls - return immediately if validation fails
|
||||||
if user_id:
|
# Skip validation in podcast-only demo mode or if explicitly disabled
|
||||||
from services.database import get_db
|
skip_validation = os.getenv("ALWRITY_SKIP_IMAGE_EDITING_VALIDATION", "false").lower() in ("true", "1", "yes")
|
||||||
|
|
||||||
|
if user_id and not skip_validation:
|
||||||
|
from services.database import get_session_for_user
|
||||||
from services.subscription import PricingService
|
from services.subscription import PricingService
|
||||||
from services.subscription.preflight_validator import validate_image_editing_operations
|
from services.subscription.preflight_validator import validate_image_editing_operations
|
||||||
from fastapi import HTTPException
|
from fastapi import HTTPException
|
||||||
|
|
||||||
logger.info(f"[Image Editing] 🔍 Starting pre-flight validation for user_id={user_id}")
|
logger.info(f"[Image Editing] 🔍 Starting pre-flight validation for user_id={user_id}")
|
||||||
# Note: get_db() is a generator, so we need to use next() to get the session
|
|
||||||
# and ensure we close it in the finally block
|
db = None
|
||||||
db = next(get_db())
|
|
||||||
try:
|
try:
|
||||||
pricing_service = PricingService(db)
|
# Use get_session_for_user instead of get_db() since we're outside FastAPI DI
|
||||||
# Raises HTTPException immediately if validation fails - frontend gets immediate response
|
db = get_session_for_user(user_id)
|
||||||
validate_image_editing_operations(
|
if not db:
|
||||||
pricing_service=pricing_service,
|
logger.warning(f"[Image Editing] ⚠️ Could not get DB session for user {user_id} - skipping validation")
|
||||||
user_id=user_id
|
else:
|
||||||
)
|
pricing_service = PricingService(db)
|
||||||
logger.info(f"[Image Editing] ✅ Pre-flight validation passed for user_id={user_id} - proceeding with image editing")
|
# Raises HTTPException immediately if validation fails - frontend gets immediate response
|
||||||
|
validate_image_editing_operations(
|
||||||
|
pricing_service=pricing_service,
|
||||||
|
user_id=user_id
|
||||||
|
)
|
||||||
|
logger.info(f"[Image Editing] ✅ Pre-flight validation passed for user_id={user_id} - proceeding with image editing")
|
||||||
except HTTPException as http_ex:
|
except HTTPException as http_ex:
|
||||||
# Re-raise immediately - don't proceed with API call
|
# Re-raise immediately - don't proceed with API call
|
||||||
logger.error(f"[Image Editing] ❌ Pre-flight validation failed for user_id={user_id} - blocking API call: {http_ex.detail}")
|
logger.error(f"[Image Editing] ❌ Pre-flight validation failed for user_id={user_id} - blocking API call: {http_ex.detail}")
|
||||||
raise
|
raise
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"[Image Editing] ❌ Unexpected error during pre-flight validation: {e}")
|
logger.error(f"[Image Editing] ❌ Unexpected error during pre-flight validation: {e}")
|
||||||
raise HTTPException(status_code=500, detail=f"Image editing validation failed: {str(e)}")
|
# In podcast-only mode, allow the operation to continue on validation errors
|
||||||
|
if os.getenv("ALWRITY_ENABLED_FEATURES") == "podcast":
|
||||||
|
logger.warning(f"[Image Editing] ⚠️ Validation error in podcast mode - allowing operation to continue")
|
||||||
|
else:
|
||||||
|
raise HTTPException(status_code=500, detail=f"Image editing validation failed: {str(e)}")
|
||||||
finally:
|
finally:
|
||||||
db.close()
|
if db:
|
||||||
|
try:
|
||||||
|
db.close()
|
||||||
|
except Exception as close_err:
|
||||||
|
logger.warning(f"[Image Editing] Error closing DB session: {close_err}")
|
||||||
else:
|
else:
|
||||||
logger.warning(f"[Image Editing] ⚠️ No user_id provided - skipping pre-flight validation (this should not happen in production)")
|
if skip_validation:
|
||||||
|
logger.info(f"[Image Editing] ⚡ Skipping pre-flight validation (ALWRITY_SKIP_IMAGE_EDITING_VALIDATION=true)")
|
||||||
|
else:
|
||||||
|
logger.warning(f"[Image Editing] ⚠️ No user_id provided - skipping pre-flight validation")
|
||||||
|
|
||||||
# Validate input
|
# Validate input
|
||||||
if not input_image_bytes:
|
if not input_image_bytes:
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ migrated from the legacy lib/gpt_providers/text_generation/main_text_generation.
|
|||||||
|
|
||||||
import os
|
import os
|
||||||
import json
|
import json
|
||||||
|
import time
|
||||||
from typing import Optional, Dict, Any, List
|
from typing import Optional, Dict, Any, List
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
@@ -67,7 +68,7 @@ def llm_text_gen(
|
|||||||
resolved_flow_type = flow_type or ("sif_agent" if preferred_hf_models else "premium_tool")
|
resolved_flow_type = flow_type or ("sif_agent" if preferred_hf_models else "premium_tool")
|
||||||
flow_tag = f"flow_type={resolved_flow_type}"
|
flow_tag = f"flow_type={resolved_flow_type}"
|
||||||
|
|
||||||
logger.info(f"[llm_text_gen][{flow_tag}] Starting text generation")
|
logger.warning(f"[llm_text_gen][{flow_tag}] Starting text generation")
|
||||||
logger.debug(f"[llm_text_gen] Prompt length: {len(prompt)} characters")
|
logger.debug(f"[llm_text_gen] Prompt length: {len(prompt)} characters")
|
||||||
|
|
||||||
# Set default values for LLM parameters
|
# Set default values for LLM parameters
|
||||||
@@ -92,19 +93,38 @@ def llm_text_gen(
|
|||||||
# Determine provider based on env vars or tenant config
|
# Determine provider based on env vars or tenant config
|
||||||
if provider_list:
|
if provider_list:
|
||||||
primary_provider = provider_list[0]
|
primary_provider = provider_list[0]
|
||||||
if primary_provider in ['gemini', 'google']:
|
if primary_provider in ['wavespeed', 'wave']:
|
||||||
|
gpt_provider = "wavespeed"
|
||||||
|
model = os.getenv('WAVESPEED_TEXT_MODEL', 'openai/gpt-oss-120b')
|
||||||
|
elif primary_provider in ['gemini', 'google']:
|
||||||
gpt_provider = "google"
|
gpt_provider = "google"
|
||||||
model = "gemini-2.0-flash-001"
|
model = "gemini-2.0-flash-001"
|
||||||
elif primary_provider in ['hf_response_api', 'huggingface', 'hf']:
|
elif primary_provider in ['hf_response_api', 'huggingface', 'hf']:
|
||||||
gpt_provider = "huggingface"
|
gpt_provider = "huggingface"
|
||||||
model = "openai/gpt-oss-120b:cerebras"
|
model = "openai/gpt-oss-120b:cerebras"
|
||||||
|
elif primary_provider in ['openai', 'gpt']:
|
||||||
|
gpt_provider = "openai"
|
||||||
|
model = os.getenv('OPENAI_MODEL', 'gpt-4o-mini')
|
||||||
|
else:
|
||||||
|
logger.warning(f"[llm_text_gen] Unknown GPT_PROVIDER: {primary_provider}, using auto-select")
|
||||||
|
gpt_provider = None
|
||||||
|
model = None
|
||||||
elif preferred_provider:
|
elif preferred_provider:
|
||||||
if preferred_provider in ['gemini', 'google']:
|
if preferred_provider in ['wavespeed', 'wave']:
|
||||||
|
gpt_provider = "wavespeed"
|
||||||
|
model = os.getenv('WAVESPEED_TEXT_MODEL', 'openai/gpt-oss-120b')
|
||||||
|
elif preferred_provider in ['openai', 'gpt']:
|
||||||
|
gpt_provider = "openai"
|
||||||
|
model = os.getenv('OPENAI_MODEL', 'gpt-4o-mini')
|
||||||
|
elif preferred_provider in ['gemini', 'google']:
|
||||||
gpt_provider = "google"
|
gpt_provider = "google"
|
||||||
model = "gemini-2.0-flash-001"
|
model = "gemini-2.0-flash-001"
|
||||||
elif preferred_provider in ['hf_response_api', 'huggingface', 'hf']:
|
elif preferred_provider in ['hf_response_api', 'huggingface', 'hf']:
|
||||||
gpt_provider = "huggingface"
|
gpt_provider = "huggingface"
|
||||||
model = "openai/gpt-oss-120b:cerebras"
|
model = "openai/gpt-oss-120b:cerebras"
|
||||||
|
else:
|
||||||
|
gpt_provider = None
|
||||||
|
model = None
|
||||||
else:
|
else:
|
||||||
# Fall back to tenant config
|
# Fall back to tenant config
|
||||||
provider_cfg = tenant_provider_config_resolver.resolve(
|
provider_cfg = tenant_provider_config_resolver.resolve(
|
||||||
@@ -137,6 +157,9 @@ def llm_text_gen(
|
|||||||
# Check which providers have API keys available using APIKeyManager
|
# Check which providers have API keys available using APIKeyManager
|
||||||
api_key_manager = APIKeyManager()
|
api_key_manager = APIKeyManager()
|
||||||
available_providers = []
|
available_providers = []
|
||||||
|
|
||||||
|
# Get strict provider mode from environment
|
||||||
|
strict_provider_mode = os.getenv("STRICT_PROVIDER_MODE", "false").lower() in {"1", "true", "yes", "on"}
|
||||||
if api_key_manager.get_api_key("gemini"):
|
if api_key_manager.get_api_key("gemini"):
|
||||||
available_providers.append("google")
|
available_providers.append("google")
|
||||||
if api_key_manager.get_api_key("hf_token"):
|
if api_key_manager.get_api_key("hf_token"):
|
||||||
@@ -144,10 +167,11 @@ def llm_text_gen(
|
|||||||
if api_key_manager.get_api_key("wavespeed"):
|
if api_key_manager.get_api_key("wavespeed"):
|
||||||
available_providers.append("wavespeed")
|
available_providers.append("wavespeed")
|
||||||
|
|
||||||
logger.info(
|
logger.warning(
|
||||||
f"[llm_text_gen][{flow_tag}] Provider preflight: env_provider='{env_provider or 'auto'}', "
|
f"[llm_text_gen][{flow_tag}] Provider preflight: env_provider='{env_provider or 'auto'}', "
|
||||||
f"provider_list={provider_list}, strict_provider_mode={strict_provider_mode}, "
|
f"provider_list={provider_list}, strict_provider_mode={strict_provider_mode}, "
|
||||||
f"available_providers={available_providers}, preferred_provider={preferred_provider or 'none'}"
|
f"available_providers={available_providers}, preferred_provider={preferred_provider or 'none'}, "
|
||||||
|
f"gpt_provider={gpt_provider}, model={model}"
|
||||||
)
|
)
|
||||||
|
|
||||||
if gpt_provider not in available_providers:
|
if gpt_provider not in available_providers:
|
||||||
@@ -187,14 +211,23 @@ def llm_text_gen(
|
|||||||
elif gpt_provider == "huggingface":
|
elif gpt_provider == "huggingface":
|
||||||
provider_enum = APIProvider.MISTRAL # HuggingFace maps to Mistral enum for usage tracking
|
provider_enum = APIProvider.MISTRAL # HuggingFace maps to Mistral enum for usage tracking
|
||||||
actual_provider_name = "huggingface" # Keep actual provider name for logs
|
actual_provider_name = "huggingface" # Keep actual provider name for logs
|
||||||
|
elif gpt_provider == "wavespeed":
|
||||||
|
provider_enum = APIProvider.WAVESPEED
|
||||||
|
actual_provider_name = "wavespeed"
|
||||||
|
elif gpt_provider == "openai":
|
||||||
|
provider_enum = APIProvider.OPENAI
|
||||||
|
actual_provider_name = "openai"
|
||||||
|
|
||||||
if not provider_enum:
|
if not provider_enum:
|
||||||
raise RuntimeError(f"Unknown provider {gpt_provider} for subscription checking")
|
# For unknown providers, try to proceed without subscription tracking
|
||||||
|
logger.warning(f"[llm_text_gen] Unknown provider {gpt_provider}, proceeding without subscription check")
|
||||||
|
|
||||||
# SUBSCRIPTION CHECK - Required and strict enforcement
|
# SUBSCRIPTION CHECK - Required and strict enforcement
|
||||||
if not user_id:
|
if not user_id:
|
||||||
raise RuntimeError("user_id is required for subscription checking. Please provide Clerk user ID.")
|
raise RuntimeError("user_id is required for subscription checking. Please provide Clerk user ID.")
|
||||||
|
|
||||||
|
sub_check_start = time.time()
|
||||||
|
logger.warning(f"[llm_text_gen][{flow_tag}] Subscription check START for user {user_id}")
|
||||||
try:
|
try:
|
||||||
from services.database import get_session_for_user
|
from services.database import get_session_for_user
|
||||||
from services.subscription import UsageTrackingService, PricingService
|
from services.subscription import UsageTrackingService, PricingService
|
||||||
@@ -248,9 +281,16 @@ def llm_text_gen(
|
|||||||
UsageSummary.billing_period == current_period
|
UsageSummary.billing_period == current_period
|
||||||
).first()
|
).first()
|
||||||
|
|
||||||
# No separate log here - we'll create unified log after API call and usage tracking
|
# Log subscription details before making the API call
|
||||||
|
if usage:
|
||||||
|
total_llm_calls = (usage.gemini_calls or 0) + (usage.openai_calls or 0) + (usage.anthropic_calls or 0) + (usage.mistral_calls or 0) + (usage.wavespeed_calls or 0)
|
||||||
|
logger.info(f"[llm_text_gen] Subscription check passed for user {user_id}: provider={actual_provider_name or gpt_provider}, tokens_requested={estimated_total_tokens}, current_usage=${usage.total_cost or 0:.4f}, calls_used={total_llm_calls}")
|
||||||
|
else:
|
||||||
|
logger.info(f"[llm_text_gen] Subscription check passed for user {user_id}: provider={actual_provider_name or gpt_provider}, tokens_requested={estimated_total_tokens}, new_user_no_usage_record")
|
||||||
|
|
||||||
finally:
|
finally:
|
||||||
|
sub_check_ms = (time.time() - sub_check_start) * 1000
|
||||||
|
logger.warning(f"[llm_text_gen][{flow_tag}] Subscription check took {sub_check_ms:.0f}ms for user {user_id}")
|
||||||
db.close()
|
db.close()
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
# Re-raise HTTPExceptions (e.g., 429 subscription limit) - preserve error details
|
# Re-raise HTTPExceptions (e.g., 429 subscription limit) - preserve error details
|
||||||
@@ -260,7 +300,8 @@ def llm_text_gen(
|
|||||||
raise
|
raise
|
||||||
except Exception as sub_error:
|
except Exception as sub_error:
|
||||||
# STRICT: Fail on subscription check errors
|
# STRICT: Fail on subscription check errors
|
||||||
logger.error(f"[llm_text_gen] Subscription check failed for user {user_id}: {sub_error}")
|
sub_check_ms = (time.time() - sub_check_start) * 1000
|
||||||
|
logger.error(f"[llm_text_gen][{flow_tag}] Subscription check FAILED after {sub_check_ms:.0f}ms for user {user_id}: {sub_error}")
|
||||||
raise RuntimeError(f"Subscription check failed: {str(sub_error)}")
|
raise RuntimeError(f"Subscription check failed: {str(sub_error)}")
|
||||||
|
|
||||||
# Construct the system prompt if not provided
|
# Construct the system prompt if not provided
|
||||||
@@ -329,9 +370,22 @@ def llm_text_gen(
|
|||||||
top_p=top_p,
|
top_p=top_p,
|
||||||
system_prompt=system_instructions
|
system_prompt=system_instructions
|
||||||
)
|
)
|
||||||
|
elif gpt_provider == "wavespeed":
|
||||||
|
from services.llm_providers.wavespeed_provider import wavespeed_text_response
|
||||||
|
llm_start = time.time()
|
||||||
|
response_text = wavespeed_text_response(
|
||||||
|
prompt=prompt,
|
||||||
|
model=model or "openai/gpt-oss-120b",
|
||||||
|
temperature=temperature,
|
||||||
|
max_tokens=max_tokens,
|
||||||
|
top_p=top_p,
|
||||||
|
system_prompt=system_instructions
|
||||||
|
)
|
||||||
|
llm_ms = (time.time() - llm_start) * 1000
|
||||||
|
logger.warning(f"[llm_text_gen][{flow_tag}] LLM API call took {llm_ms:.0f}ms for user {user_id} (wavespeed)")
|
||||||
else:
|
else:
|
||||||
logger.error(f"[llm_text_gen] Unknown provider: {gpt_provider}")
|
logger.error(f"[llm_text_gen] Unknown provider: {gpt_provider}")
|
||||||
raise RuntimeError("Unknown LLM provider. Supported providers: google, huggingface")
|
raise RuntimeError(f"Unknown LLM provider: {gpt_provider}. Supported providers: google, huggingface, wavespeed")
|
||||||
|
|
||||||
# TRACK USAGE after successful API call
|
# TRACK USAGE after successful API call
|
||||||
if response_text:
|
if response_text:
|
||||||
@@ -446,9 +500,45 @@ def llm_text_gen(
|
|||||||
logger.error(f"[llm_text_gen] Fallback provider {fallback_provider} also failed: {str(fallback_error)}")
|
logger.error(f"[llm_text_gen] Fallback provider {fallback_provider} also failed: {str(fallback_error)}")
|
||||||
|
|
||||||
# CIRCUIT BREAKER: Stop immediately to prevent expensive API calls
|
# CIRCUIT BREAKER: Stop immediately to prevent expensive API calls
|
||||||
logger.error("[llm_text_gen] CIRCUIT BREAKER: Stopping to prevent expensive API calls.")
|
logger.error("[llm_text_gen] CIRCUIT BREAKER: All providers failed.")
|
||||||
raise RuntimeError("All LLM providers failed to generate a response.")
|
|
||||||
|
# Provide more helpful error message based on available providers
|
||||||
|
if not available_providers:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=429,
|
||||||
|
detail={
|
||||||
|
"error": "No LLM providers configured",
|
||||||
|
"message": "No LLM API keys found. Please configure at least one provider (GPT_PROVIDER, GOOGLE_API_KEY, HF_TOKEN, or WAVESPEED_API_KEY).",
|
||||||
|
"usage_info": {
|
||||||
|
"error_type": "no_providers_configured",
|
||||||
|
"operation_type": "text-generation",
|
||||||
|
"limit": 0,
|
||||||
|
"current_tokens": 0,
|
||||||
|
"suggestion": "Set GPT_PROVIDER=wavespeed in environment or configure API keys in the dashboard."
|
||||||
|
}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=429,
|
||||||
|
detail={
|
||||||
|
"error": "All LLM providers failed",
|
||||||
|
"message": "All configured LLM providers failed to generate a response. Please check API keys and try again.",
|
||||||
|
"usage_info": {
|
||||||
|
"error_type": "all_providers_failed",
|
||||||
|
"operation_type": "text-generation",
|
||||||
|
"available_providers": available_providers,
|
||||||
|
"requested_provider": gpt_provider,
|
||||||
|
"limit": 0,
|
||||||
|
"current_tokens": 0,
|
||||||
|
"suggestion": f"Provider {gpt_provider} failed. Available: {', '.join(available_providers)}. Try setting GPT_PROVIDER to one of: {', '.join(available_providers)}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
# Re-raise HTTPExceptions (e.g., 429 subscription limit) - preserve error details
|
||||||
|
raise
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"[llm_text_gen] Error during text generation: {str(e)}")
|
logger.error(f"[llm_text_gen] Error during text generation: {str(e)}")
|
||||||
raise
|
raise
|
||||||
|
|||||||
@@ -274,10 +274,6 @@ def wavespeed_text_response(
|
|||||||
|
|
||||||
logger.info("🚀 Making WaveSpeed API call (chat completion)...")
|
logger.info("🚀 Making WaveSpeed API call (chat completion)...")
|
||||||
|
|
||||||
# Add rate limiting to prevent expensive API calls
|
|
||||||
import time
|
|
||||||
time.sleep(1) # 1 second delay between API calls
|
|
||||||
|
|
||||||
# Call exactly the requested model; no retries, no fallbacks, no variants
|
# Call exactly the requested model; no retries, no fallbacks, no variants
|
||||||
response = client.chat.completions.create(
|
response = client.chat.completions.create(
|
||||||
model=model,
|
model=model,
|
||||||
@@ -426,10 +422,6 @@ def wavespeed_structured_json_response(
|
|||||||
json_schema_str = json.dumps(schema, indent=2)
|
json_schema_str = json.dumps(schema, indent=2)
|
||||||
messages[-1]["content"] += f"\n\nJSON Schema:\n{json_schema_str}"
|
messages[-1]["content"] += f"\n\nJSON Schema:\n{json_schema_str}"
|
||||||
|
|
||||||
# Add rate limiting to prevent expensive API calls
|
|
||||||
import time
|
|
||||||
time.sleep(1) # 1 second delay between API calls
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
response = None
|
response = None
|
||||||
last_error = None
|
last_error = None
|
||||||
|
|||||||
@@ -23,6 +23,11 @@ class MonitoringDataService:
|
|||||||
def __init__(self, db_session: Session):
|
def __init__(self, db_session: Session):
|
||||||
self.db = db_session
|
self.db = db_session
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_strategy_user_id(self, strategy_id: int) -> str:
|
||||||
|
strategy = self.db.query(EnhancedContentStrategy).filter(EnhancedContentStrategy.id == strategy_id).first()
|
||||||
|
return str(getattr(strategy, "user_id", "0") or "0")
|
||||||
|
|
||||||
async def save_monitoring_data(self, strategy_id: int, monitoring_plan: Dict[str, Any]) -> bool:
|
async def save_monitoring_data(self, strategy_id: int, monitoring_plan: Dict[str, Any]) -> bool:
|
||||||
"""Save monitoring plan and tasks to database."""
|
"""Save monitoring plan and tasks to database."""
|
||||||
try:
|
try:
|
||||||
@@ -65,19 +70,22 @@ class MonitoringDataService:
|
|||||||
|
|
||||||
self.db.add(task)
|
self.db.add(task)
|
||||||
|
|
||||||
|
strategy_user_id = self._resolve_strategy_user_id(strategy_id)
|
||||||
|
|
||||||
# Save activation status
|
# Save activation status
|
||||||
activation_status = StrategyActivationStatus(
|
activation_status = StrategyActivationStatus(
|
||||||
strategy_id=strategy_id,
|
strategy_id=strategy_id,
|
||||||
user_id=1, # Default user ID
|
user_id=strategy_user_id,
|
||||||
activation_date=datetime.utcnow(),
|
activation_date=datetime.utcnow(),
|
||||||
status='active'
|
status='active'
|
||||||
)
|
)
|
||||||
self.db.add(activation_status)
|
self.db.add(activation_status)
|
||||||
|
|
||||||
# Save initial performance metrics
|
# Save initial performance metrics
|
||||||
|
strategy_user_id = self._resolve_strategy_user_id(strategy_id)
|
||||||
performance_metrics = StrategyPerformanceMetrics(
|
performance_metrics = StrategyPerformanceMetrics(
|
||||||
strategy_id=strategy_id,
|
strategy_id=strategy_id,
|
||||||
user_id=1, # Default user ID
|
user_id=strategy_user_id,
|
||||||
metric_date=datetime.utcnow(),
|
metric_date=datetime.utcnow(),
|
||||||
data_source='monitoring_plan',
|
data_source='monitoring_plan',
|
||||||
confidence_score=85 # High confidence for monitoring plan data
|
confidence_score=85 # High confidence for monitoring plan data
|
||||||
@@ -341,10 +349,11 @@ class MonitoringDataService:
|
|||||||
"""Update performance metrics for a strategy."""
|
"""Update performance metrics for a strategy."""
|
||||||
try:
|
try:
|
||||||
logger.info(f"Updating performance metrics for strategy {strategy_id}")
|
logger.info(f"Updating performance metrics for strategy {strategy_id}")
|
||||||
|
strategy_user_id = self._resolve_strategy_user_id(strategy_id)
|
||||||
|
|
||||||
performance_metrics = StrategyPerformanceMetrics(
|
performance_metrics = StrategyPerformanceMetrics(
|
||||||
strategy_id=strategy_id,
|
strategy_id=strategy_id,
|
||||||
user_id=1, # Default user ID
|
user_id=strategy_user_id,
|
||||||
metric_date=datetime.utcnow(),
|
metric_date=datetime.utcnow(),
|
||||||
traffic_growth_percentage=metrics.get('traffic_growth'),
|
traffic_growth_percentage=metrics.get('traffic_growth'),
|
||||||
engagement_rate_percentage=metrics.get('engagement_rate'),
|
engagement_rate_percentage=metrics.get('engagement_rate'),
|
||||||
|
|||||||
@@ -18,9 +18,12 @@ import json
|
|||||||
from services.database import get_db_session
|
from services.database import get_db_session
|
||||||
from models.onboarding import OnboardingSession, WebsiteAnalysis, ResearchPreferences
|
from models.onboarding import OnboardingSession, WebsiteAnalysis, ResearchPreferences
|
||||||
from models.persona_models import WritingPersona, PlatformPersona, PersonaAnalysisResult
|
from models.persona_models import WritingPersona, PlatformPersona, PersonaAnalysisResult
|
||||||
from services.persona.core_persona import CorePersonaService, OnboardingDataCollector
|
|
||||||
from services.persona.linkedin.linkedin_persona_service import LinkedInPersonaService
|
def _get_podcast_mode():
|
||||||
from services.persona.facebook.facebook_persona_service import FacebookPersonaService
|
"""Check if running in podcast-only mode to skip heavy initialization."""
|
||||||
|
import os
|
||||||
|
env_val = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower()
|
||||||
|
return env_val == "podcast"
|
||||||
|
|
||||||
class PersonaAnalysisService:
|
class PersonaAnalysisService:
|
||||||
"""Service for analyzing onboarding data and generating writing personas using Gemini AI."""
|
"""Service for analyzing onboarding data and generating writing personas using Gemini AI."""
|
||||||
@@ -37,12 +40,40 @@ class PersonaAnalysisService:
|
|||||||
def __init__(self):
|
def __init__(self):
|
||||||
"""Initialize the persona analysis service (only once)."""
|
"""Initialize the persona analysis service (only once)."""
|
||||||
if not self._initialized:
|
if not self._initialized:
|
||||||
|
# Skip heavy initialization in podcast-only mode
|
||||||
|
if _get_podcast_mode():
|
||||||
|
logger.debug("PersonaAnalysisService: Skipping heavy init in podcast mode")
|
||||||
|
self._initialized = True
|
||||||
|
return
|
||||||
|
|
||||||
|
# Only initialize heavy services when needed (not at import time)
|
||||||
|
self._heavy_init_done = False
|
||||||
|
|
||||||
|
def _ensure_heavy_init(self):
|
||||||
|
"""Lazily initialize heavy services only when first used."""
|
||||||
|
if self._heavy_init_done:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Check again in case mode changed
|
||||||
|
if _get_podcast_mode():
|
||||||
|
logger.debug("PersonaAnalysisService: Skipping heavy init in podcast mode")
|
||||||
|
self._heavy_init_done = True
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
from services.persona.core_persona import CorePersonaService, OnboardingDataCollector
|
||||||
|
from services.persona.linkedin.linkedin_persona_service import LinkedInPersonaService
|
||||||
|
from services.persona.facebook.facebook_persona_service import FacebookPersonaService
|
||||||
|
|
||||||
self.core_persona_service = CorePersonaService()
|
self.core_persona_service = CorePersonaService()
|
||||||
self.data_collector = OnboardingDataCollector()
|
self.data_collector = OnboardingDataCollector()
|
||||||
self.linkedin_service = LinkedInPersonaService()
|
self.linkedin_service = LinkedInPersonaService()
|
||||||
self.facebook_service = FacebookPersonaService()
|
self.facebook_service = FacebookPersonaService()
|
||||||
logger.debug("PersonaAnalysisService initialized")
|
self._heavy_init_done = True
|
||||||
self._initialized = True
|
logger.debug("PersonaAnalysisService initialized (lazy)")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"PersonaAnalysisService: Failed to initialize heavy services: {e}")
|
||||||
|
self._heavy_init_done = True
|
||||||
|
|
||||||
def generate_persona_from_onboarding(self, user_id: str, onboarding_session_id: int = None) -> Dict[str, Any]:
|
def generate_persona_from_onboarding(self, user_id: str, onboarding_session_id: int = None) -> Dict[str, Any]:
|
||||||
"""
|
"""
|
||||||
@@ -55,6 +86,13 @@ class PersonaAnalysisService:
|
|||||||
Returns:
|
Returns:
|
||||||
Generated persona data with platform adaptations
|
Generated persona data with platform adaptations
|
||||||
"""
|
"""
|
||||||
|
# Ensure heavy services are initialized
|
||||||
|
self._ensure_heavy_init()
|
||||||
|
|
||||||
|
# Check if heavy init failed (podcast mode)
|
||||||
|
if not getattr(self, '_heavy_init_done', False):
|
||||||
|
return {"error": "Persona service unavailable in podcast-only mode"}
|
||||||
|
|
||||||
try:
|
try:
|
||||||
logger.info(f"Generating persona for user {user_id}")
|
logger.info(f"Generating persona for user {user_id}")
|
||||||
|
|
||||||
|
|||||||
623
backend/services/podcast/broll_composer.py
Normal file
623
backend/services/podcast/broll_composer.py
Normal file
@@ -0,0 +1,623 @@
|
|||||||
|
"""
|
||||||
|
Programmatic B-Roll Composer
|
||||||
|
Layered composition pipeline: Background + Chart + Avatar Circle + Text Overlays
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import numpy as np
|
||||||
|
from pathlib import Path
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Optional
|
||||||
|
import matplotlib
|
||||||
|
matplotlib.use("Agg")
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import matplotlib.patches as mpatches
|
||||||
|
from PIL import Image, ImageDraw, ImageFont
|
||||||
|
from moviepy import (
|
||||||
|
VideoFileClip, ImageClip, CompositeVideoClip,
|
||||||
|
concatenate_videoclips,
|
||||||
|
)
|
||||||
|
import moviepy.video.fx as vfx
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Crossfade concat (Option 1: crossfadein + negative padding)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def crossfade_concat(scenes: list, fade_dur: float = 0.5):
|
||||||
|
"""
|
||||||
|
Concatenate scenes with a dissolve transition between each pair.
|
||||||
|
|
||||||
|
Each clip (except the first) gets a crossfadein effect.
|
||||||
|
padding=-fade_dur overlaps consecutive clips so the fade actually fires
|
||||||
|
instead of creating a black gap. set_duration on every scene is
|
||||||
|
mandatory — CompositeVideoClip.duration can be ambiguous without it,
|
||||||
|
which makes the overlap math wrong.
|
||||||
|
"""
|
||||||
|
faded = []
|
||||||
|
for i, clip in enumerate(scenes):
|
||||||
|
c = clip
|
||||||
|
if i > 0:
|
||||||
|
c = c.fx(vfx.CrossFadeIn, fade_dur)
|
||||||
|
faded.append(c)
|
||||||
|
return concatenate_videoclips(faded, padding=-int(fade_dur), method="compose")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Data structures
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Insight:
|
||||||
|
key_insight: str
|
||||||
|
supporting_stat: str
|
||||||
|
visual_cue: str # bar_chart_comparison | line_trend | bullet_points | full_avatar
|
||||||
|
audio_tone: str
|
||||||
|
chart_data: dict = field(default_factory=dict)
|
||||||
|
duration: float = 10.0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SceneAssets:
|
||||||
|
background_img: str
|
||||||
|
chart_img: Optional[str] = None
|
||||||
|
avatar_video: Optional[str] = None
|
||||||
|
bullet_img: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Chart generator (Matplotlib → PNG with transparency)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
CHART_STYLE = {
|
||||||
|
"bg": "#0D0D0D",
|
||||||
|
"bar_before": "#2E4057",
|
||||||
|
"bar_after": "#E63946",
|
||||||
|
"text": "#F1F1EF",
|
||||||
|
"grid": "#2A2A2A",
|
||||||
|
"accent": "#E63946",
|
||||||
|
"pie_colors": ["#E63946", "#2E4057", "#457B9D", "#A8DADC", "#F4A261", "#2A9D8F"],
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Chart generators (Matplotlib → PNG with transparency)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def make_bar_chart(data: dict, out_path: str, title: str = "",
|
||||||
|
show_legend: bool = True, value_suffix: str = "%",
|
||||||
|
subtitle: str = "") -> str:
|
||||||
|
"""Render a side-by-side comparison bar chart. Returns output path."""
|
||||||
|
labels = data.get("labels", [])
|
||||||
|
before = data.get("before", [])
|
||||||
|
after = data.get("after", [])
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
|
||||||
|
x = np.arange(len(labels))
|
||||||
|
w = 0.35
|
||||||
|
bars_b = ax.bar(x - w / 2, before, w, color=CHART_STYLE["bar_before"],
|
||||||
|
label="Before", zorder=3, edgecolor="none")
|
||||||
|
bars_a = ax.bar(x + w / 2, after, w, color=CHART_STYLE["bar_after"],
|
||||||
|
label="After", zorder=3, edgecolor="none")
|
||||||
|
|
||||||
|
ax.set_xticks(x)
|
||||||
|
ax.set_xticklabels(labels, color=CHART_STYLE["text"], fontsize=11)
|
||||||
|
ax.tick_params(axis="y", colors=CHART_STYLE["text"])
|
||||||
|
ax.spines[:].set_visible(False)
|
||||||
|
ax.yaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||||
|
ax.set_axisbelow(True)
|
||||||
|
|
||||||
|
for bar in [*bars_b, *bars_a]:
|
||||||
|
h = bar.get_height()
|
||||||
|
ax.text(bar.get_x() + bar.get_width() / 2, h + 0.5, f"{h:.0f}{value_suffix}",
|
||||||
|
ha="center", va="bottom", color=CHART_STYLE["text"], fontsize=9,
|
||||||
|
fontweight="bold")
|
||||||
|
|
||||||
|
if show_legend:
|
||||||
|
legend = ax.legend(frameon=False, labelcolor=CHART_STYLE["text"],
|
||||||
|
fontsize=10, loc="upper left")
|
||||||
|
|
||||||
|
# Add title and optional subtitle
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
if subtitle:
|
||||||
|
fig.text(0.5, 0.02, subtitle, ha='center', color=CHART_STYLE["text"],
|
||||||
|
fontsize=10, style='italic')
|
||||||
|
|
||||||
|
fig.tight_layout(pad=0.5, rect=(0, 0.03 if subtitle else 0, 1, 1))
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_horizontal_bar(data: dict, out_path: str, title: str = "",
|
||||||
|
value_suffix: str = "%", bar_color: str = None) -> str:
|
||||||
|
"""Render a horizontal bar chart (good for rankings/lists)."""
|
||||||
|
labels = data.get("labels", [])
|
||||||
|
values = data.get("values", data.get("y", []))
|
||||||
|
|
||||||
|
if not values:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
bar_color = bar_color or CHART_STYLE["bar_after"]
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
|
||||||
|
y_pos = np.arange(len(labels))
|
||||||
|
bars = ax.barh(y_pos, values, color=bar_color, zorder=3, edgecolor="none", height=0.6)
|
||||||
|
|
||||||
|
ax.set_yticks(y_pos)
|
||||||
|
ax.set_yticklabels(labels, color=CHART_STYLE["text"], fontsize=11)
|
||||||
|
ax.tick_params(axis="x", colors=CHART_STYLE["text"])
|
||||||
|
ax.spines[:].set_visible(False)
|
||||||
|
ax.xaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||||
|
ax.set_axisbelow(True)
|
||||||
|
ax.invert_yaxis()
|
||||||
|
|
||||||
|
for i, bar in enumerate(bars):
|
||||||
|
width = bar.get_width()
|
||||||
|
ax.text(width + 0.5, bar.get_y() + bar.get_height()/2, f"{width:.0f}{value_suffix}",
|
||||||
|
ha="left", va="center", color=CHART_STYLE["text"], fontsize=10,
|
||||||
|
fontweight="bold")
|
||||||
|
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
|
||||||
|
fig.tight_layout(pad=0.5)
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_line_trend(data: dict, out_path: str, title: str = "",
|
||||||
|
show_area: bool = True, show_markers: bool = True) -> str:
|
||||||
|
"""Render a trend line chart."""
|
||||||
|
x_vals = data.get("x", [])
|
||||||
|
y_vals = data.get("y", [])
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
|
||||||
|
line_style = data.get("line_style", "-")
|
||||||
|
line_width = data.get("line_width", 2.5)
|
||||||
|
|
||||||
|
ax.plot(x_vals, y_vals, color=CHART_STYLE["accent"],
|
||||||
|
linewidth=line_width, linestyle=line_style,
|
||||||
|
marker="o" if show_markers else None, markersize=7, zorder=3)
|
||||||
|
|
||||||
|
if show_area:
|
||||||
|
ax.fill_between(x_vals, y_vals, alpha=0.12, color=CHART_STYLE["accent"])
|
||||||
|
|
||||||
|
ax.spines[:].set_visible(False)
|
||||||
|
ax.tick_params(colors=CHART_STYLE["text"])
|
||||||
|
ax.yaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||||
|
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
|
||||||
|
fig.tight_layout(pad=0.5)
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_pie_chart(data: dict, out_path: str, title: str = "",
|
||||||
|
show_labels: bool = True, show_percent: bool = True,
|
||||||
|
donut: bool = False) -> str:
|
||||||
|
"""Render a pie chart."""
|
||||||
|
labels = data.get("labels", [])
|
||||||
|
values = data.get("values", data.get("y", []))
|
||||||
|
|
||||||
|
if not values:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
colors = CHART_STYLE["pie_colors"][:len(values)]
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(6, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
|
||||||
|
if donut:
|
||||||
|
wedges, texts, autotexts = ax.pie(
|
||||||
|
values, labels=labels if show_labels else None,
|
||||||
|
colors=colors, autopct=lambda p: f'{p:.1f}%' if show_percent else '',
|
||||||
|
startangle=90, pctdistance=0.75,
|
||||||
|
wedgeprops=dict(width=0.5, edgecolor="none")
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
wedges, texts, autotexts = ax.pie(
|
||||||
|
values, labels=labels if show_labels else None,
|
||||||
|
colors=colors, autopct=lambda p: f'{p:.1f}%' if show_percent else '',
|
||||||
|
startangle=90, pctdistance=0.8
|
||||||
|
)
|
||||||
|
|
||||||
|
for text in texts:
|
||||||
|
text.set_color(CHART_STYLE["text"])
|
||||||
|
text.set_fontsize(10)
|
||||||
|
|
||||||
|
for autotext in autotexts:
|
||||||
|
autotext.set_color(CHART_STYLE["text"])
|
||||||
|
autotext.set_fontsize(9)
|
||||||
|
autotext.set_fontweight("bold")
|
||||||
|
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
|
||||||
|
fig.tight_layout(pad=0.5)
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_stacked_bar(data: dict, out_path: str, title: str = "",
|
||||||
|
stack_labels: list = None) -> str:
|
||||||
|
"""Render a stacked bar chart."""
|
||||||
|
labels = data.get("labels", [])
|
||||||
|
stacks = data.get("stacks", []) # List of lists, each inner list is a stack
|
||||||
|
|
||||||
|
if not stacks or len(stacks) < 2:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
stack_labels = stack_labels or [f"Series {i+1}" for i in range(len(stacks))]
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
|
||||||
|
x = np.arange(len(labels))
|
||||||
|
bottom = np.zeros(len(labels))
|
||||||
|
colors = CHART_STYLE["pie_colors"][:len(stacks)]
|
||||||
|
|
||||||
|
for i, stack in enumerate(stacks):
|
||||||
|
bars = ax.bar(x, stack, 0.6, bottom=bottom, color=colors[i],
|
||||||
|
label=stack_labels[i], zorder=3, edgecolor="none")
|
||||||
|
|
||||||
|
for j, bar in enumerate(bars):
|
||||||
|
height = bar.get_height()
|
||||||
|
if height > 5: # Only show label if segment is big enough
|
||||||
|
ax.text(bar.get_x() + bar.get_width()/2,
|
||||||
|
bottom[j] + height/2,
|
||||||
|
f"{height:.0f}", ha="center", va="center",
|
||||||
|
color=CHART_STYLE["text"], fontsize=8, fontweight="bold")
|
||||||
|
|
||||||
|
bottom = bottom + np.array(stack)
|
||||||
|
|
||||||
|
ax.set_xticks(x)
|
||||||
|
ax.set_xticklabels(labels, color=CHART_STYLE["text"], fontsize=11)
|
||||||
|
ax.tick_params(axis="y", colors=CHART_STYLE["text"])
|
||||||
|
ax.spines[:].set_visible(False)
|
||||||
|
ax.legend(frameon=False, labelcolor=CHART_STYLE["text"], fontsize=9, loc="upper left")
|
||||||
|
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
|
||||||
|
fig.tight_layout(pad=0.5)
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_line_trend(data: dict, out_path: str, title: str = "") -> str:
|
||||||
|
"""Render a trend line chart. Returns output path."""
|
||||||
|
x_vals = data.get("x", [])
|
||||||
|
y_vals = data.get("y", [])
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||||
|
ax.set_facecolor("none")
|
||||||
|
ax.plot(x_vals, y_vals, color=CHART_STYLE["accent"],
|
||||||
|
linewidth=2.5, marker="o", markersize=7, zorder=3)
|
||||||
|
ax.fill_between(x_vals, y_vals, alpha=0.12, color=CHART_STYLE["accent"])
|
||||||
|
ax.spines[:].set_visible(False)
|
||||||
|
ax.tick_params(colors=CHART_STYLE["text"])
|
||||||
|
ax.yaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||||
|
if title:
|
||||||
|
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||||
|
fontweight="bold", pad=12)
|
||||||
|
fig.tight_layout(pad=0.5)
|
||||||
|
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||||
|
plt.close(fig)
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Text / Bullet overlay (Pillow → PNG)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def make_bullet_overlay(lines: list[str], out_path: str,
|
||||||
|
width: int = 900, font_size: int = 32) -> str:
|
||||||
|
"""Render bullet points on a semi-transparent dark pill. Returns path."""
|
||||||
|
padding = 32
|
||||||
|
line_h = font_size + 16
|
||||||
|
img_h = padding * 2 + len(lines) * line_h + 12
|
||||||
|
img = Image.new("RGBA", (width, img_h), (0, 0, 0, 0))
|
||||||
|
draw = ImageDraw.Draw(img)
|
||||||
|
|
||||||
|
draw.rounded_rectangle([0, 0, width - 1, img_h - 1],
|
||||||
|
radius=18, fill=(10, 10, 10, 185))
|
||||||
|
|
||||||
|
try:
|
||||||
|
font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
|
||||||
|
font_size)
|
||||||
|
except OSError:
|
||||||
|
font = ImageFont.load_default()
|
||||||
|
|
||||||
|
y = padding
|
||||||
|
for line in lines:
|
||||||
|
draw.text((padding + 18, y), f"• {line}", font=font, fill=(241, 241, 239, 255))
|
||||||
|
y += line_h
|
||||||
|
|
||||||
|
img.save(out_path, format="PNG")
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_insight_card(insight: str, stat: str, out_path: str,
|
||||||
|
width: int = 960, height: int = 200) -> str:
|
||||||
|
"""Render a bold insight card (headline + supporting stat). Returns path."""
|
||||||
|
img = Image.new("RGBA", (width, height), (0, 0, 0, 0))
|
||||||
|
draw = ImageDraw.Draw(img)
|
||||||
|
draw.rounded_rectangle([0, 0, width - 1, height - 1],
|
||||||
|
radius=14, fill=(10, 10, 10, 200))
|
||||||
|
|
||||||
|
draw.rectangle([28, 24, 36, height - 24], fill=(230, 57, 70, 255))
|
||||||
|
|
||||||
|
try:
|
||||||
|
font_lg = ImageFont.truetype(
|
||||||
|
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", 34)
|
||||||
|
font_sm = ImageFont.truetype(
|
||||||
|
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 20)
|
||||||
|
except OSError:
|
||||||
|
font_lg = font_sm = ImageFont.load_default()
|
||||||
|
|
||||||
|
draw.text((58, 36), insight, font=font_lg, fill=(241, 241, 239, 255))
|
||||||
|
draw.text((58, 90), stat, font=font_sm, fill=(180, 180, 178, 230))
|
||||||
|
|
||||||
|
img.save(out_path, format="PNG")
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Circular avatar mask
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def apply_circle_mask(clip: VideoFileClip, diameter: int) -> VideoFileClip:
|
||||||
|
"""Resize clip and apply a circular alpha mask."""
|
||||||
|
clip = clip.resize(height=diameter)
|
||||||
|
w, h = clip.size
|
||||||
|
|
||||||
|
Y, X = np.ogrid[:h, :w]
|
||||||
|
cx, cy = w / 2, h / 2
|
||||||
|
mask_arr = ((X - cx) ** 2 + (Y - cy) ** 2 <= (min(w, h) / 2) ** 2).astype(float)
|
||||||
|
|
||||||
|
mask_clip = ImageClip(mask_arr, ismask=True).set_duration(clip.duration)
|
||||||
|
return clip.set_mask(mask_clip)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Ken Burns zoom effect
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def ken_burns(clip: ImageClip, zoom_ratio: float = 0.08) -> ImageClip:
|
||||||
|
"""Apply a slow zoom-in over the clip duration."""
|
||||||
|
def zoom_frame(get_frame, t):
|
||||||
|
frame = get_frame(t)
|
||||||
|
frac = 1 + zoom_ratio * (t / clip.duration)
|
||||||
|
h, w = frame.shape[:2]
|
||||||
|
new_h, new_w = int(h / frac), int(w / frac)
|
||||||
|
y1 = (h - new_h) // 2
|
||||||
|
x1 = (w - new_w) // 2
|
||||||
|
cropped = frame[y1:y1 + new_h, x1:x1 + new_w]
|
||||||
|
return np.array(Image.fromarray(cropped).resize((w, h), Image.LANCZOS))
|
||||||
|
|
||||||
|
return clip.fl(zoom_frame, apply_to=["mask"])
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Scene builders (one per visual_cue type)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def build_data_scene(assets: SceneAssets, insight: Insight) -> CompositeVideoClip:
|
||||||
|
"""
|
||||||
|
Layout: Background (Ken Burns) + Chart (fade-in) + Avatar circle (corner) + Insight card
|
||||||
|
"""
|
||||||
|
d = insight.duration
|
||||||
|
layers = []
|
||||||
|
|
||||||
|
bg = (ImageClip(assets.background_img)
|
||||||
|
.set_duration(d)
|
||||||
|
.resize(height=1080))
|
||||||
|
bg = ken_burns(bg)
|
||||||
|
bg = bg.fx(vfx.lum_contrast, 0, -40)
|
||||||
|
layers.append(bg)
|
||||||
|
|
||||||
|
if assets.chart_img:
|
||||||
|
chart = (ImageClip(assets.chart_img)
|
||||||
|
.set_duration(d - 1.5)
|
||||||
|
.set_start(0.5)
|
||||||
|
.resize(width=700)
|
||||||
|
.set_position(("center", 180))
|
||||||
|
.fx(vfx.fadein, 0.6)
|
||||||
|
.fx(vfx.fadeout, 0.4))
|
||||||
|
layers.append(chart)
|
||||||
|
|
||||||
|
card_path = "/tmp/insight_card.png"
|
||||||
|
make_insight_card(insight.key_insight, insight.supporting_stat, card_path)
|
||||||
|
card = (ImageClip(card_path)
|
||||||
|
.set_duration(d - 1)
|
||||||
|
.set_start(0.5)
|
||||||
|
.set_position(("center", 820))
|
||||||
|
.fx(vfx.fadein, 0.5))
|
||||||
|
layers.append(card)
|
||||||
|
|
||||||
|
if assets.avatar_video:
|
||||||
|
avatar_raw = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||||
|
avatar = apply_circle_mask(avatar_raw, diameter=240)
|
||||||
|
avatar = avatar.set_position((bg.w - 280, bg.h - 280))
|
||||||
|
layers.append(avatar)
|
||||||
|
|
||||||
|
return CompositeVideoClip(layers, size=bg.size).set_duration(d)
|
||||||
|
|
||||||
|
|
||||||
|
def build_bullet_scene(assets: SceneAssets, insight: Insight,
|
||||||
|
bullets: list[str]) -> CompositeVideoClip:
|
||||||
|
"""
|
||||||
|
Layout: AI image (Ken Burns) + Bullet overlay + Avatar circle
|
||||||
|
"""
|
||||||
|
d = insight.duration
|
||||||
|
layers = []
|
||||||
|
|
||||||
|
bg = (ImageClip(assets.background_img)
|
||||||
|
.set_duration(d)
|
||||||
|
.resize(height=1080))
|
||||||
|
bg = ken_burns(bg, zoom_ratio=0.05)
|
||||||
|
bg = bg.fx(vfx.lum_contrast, 0, -50)
|
||||||
|
layers.append(bg)
|
||||||
|
|
||||||
|
bullet_path = "/tmp/bullets.png"
|
||||||
|
make_bullet_overlay(bullets, bullet_path, width=860)
|
||||||
|
bullets_clip = (ImageClip(bullet_path)
|
||||||
|
.set_duration(d - 1)
|
||||||
|
.set_start(0.5)
|
||||||
|
.set_position(("center", "center"))
|
||||||
|
.fx(vfx.fadein, 0.7))
|
||||||
|
layers.append(bullets_clip)
|
||||||
|
|
||||||
|
if assets.avatar_video:
|
||||||
|
avatar_raw = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||||
|
avatar = apply_circle_mask(avatar_raw, diameter=200)
|
||||||
|
avatar = avatar.set_position((bg.w - 240, bg.h - 240))
|
||||||
|
layers.append(avatar)
|
||||||
|
|
||||||
|
return CompositeVideoClip(layers, size=bg.size).set_duration(d)
|
||||||
|
|
||||||
|
|
||||||
|
def build_full_avatar_scene(assets: SceneAssets, insight: Insight) -> VideoFileClip:
|
||||||
|
"""Full-screen avatar — the expensive 'Hook' scene. No overlay."""
|
||||||
|
d = insight.duration
|
||||||
|
avatar = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||||
|
return avatar.resize(height=1080).set_duration(d)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Scene dispatcher — maps visual_cue → builder
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def dispatch_scene(insight: Insight, assets: SceneAssets,
|
||||||
|
bullet_lines: Optional[list[str]] = None):
|
||||||
|
"""Dispatch scene based on visual_cue type."""
|
||||||
|
cue = insight.visual_cue
|
||||||
|
|
||||||
|
if cue == "full_avatar":
|
||||||
|
return build_full_avatar_scene(assets, insight)
|
||||||
|
|
||||||
|
elif cue in ("bar_chart_comparison", "line_trend"):
|
||||||
|
chart_path = "/tmp/chart.png"
|
||||||
|
if cue == "bar_chart_comparison":
|
||||||
|
make_bar_chart(insight.chart_data, chart_path,
|
||||||
|
title=insight.key_insight)
|
||||||
|
else:
|
||||||
|
make_line_trend(insight.chart_data, chart_path,
|
||||||
|
title=insight.key_insight)
|
||||||
|
assets.chart_img = chart_path
|
||||||
|
return build_data_scene(assets, insight)
|
||||||
|
|
||||||
|
elif cue == "bullet_points":
|
||||||
|
lines = bullet_lines or [insight.key_insight, insight.supporting_stat]
|
||||||
|
return build_bullet_scene(assets, insight, lines)
|
||||||
|
|
||||||
|
else:
|
||||||
|
return build_data_scene(assets, insight)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Master compositor — assembles all scenes into one video
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def compose_video(scenes: list, output_path: str = "output.mp4",
|
||||||
|
fps: int = 24, fade_dur: float = 0.5) -> str:
|
||||||
|
"""Concatenate scenes with crossfade transitions and write final video file."""
|
||||||
|
final = crossfade_concat(scenes, fade_dur=fade_dur)
|
||||||
|
final.write_videofile(
|
||||||
|
output_path,
|
||||||
|
fps=fps,
|
||||||
|
codec="libx264",
|
||||||
|
audio_codec="aac",
|
||||||
|
threads=4,
|
||||||
|
preset="fast",
|
||||||
|
logger=None,
|
||||||
|
)
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# JSON bridge — LLM insight → assets + scene
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def pipeline_from_json(insight_json: str,
|
||||||
|
background_img: str,
|
||||||
|
avatar_video: Optional[str] = None) -> str:
|
||||||
|
"""
|
||||||
|
Full pipeline:
|
||||||
|
1. Parse LLM insight JSON
|
||||||
|
2. Generate chart / overlay assets
|
||||||
|
3. Build scene
|
||||||
|
4. Write video
|
||||||
|
Returns path to output video.
|
||||||
|
"""
|
||||||
|
data = json.loads(insight_json)
|
||||||
|
insight = Insight(**{k: data[k] for k in Insight.__dataclass_fields__ if k in data})
|
||||||
|
assets = SceneAssets(background_img=background_img, avatar_video=avatar_video)
|
||||||
|
scene = dispatch_scene(insight, assets,
|
||||||
|
bullet_lines=data.get("bullet_lines"))
|
||||||
|
out = f"/tmp/scene_{insight.visual_cue}.mp4"
|
||||||
|
compose_video([scene], output_path=out)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Demo / smoke-test (no real media files needed for chart generation)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sample_bar_data = {
|
||||||
|
"labels": ["Content Velocity", "CTR", "Engagement", "Cost/Lead"],
|
||||||
|
"before": [30, 22, 18, 60],
|
||||||
|
"after": [72, 34, 41, 38],
|
||||||
|
}
|
||||||
|
chart_out = make_bar_chart(
|
||||||
|
sample_bar_data,
|
||||||
|
"/tmp/demo_chart.png",
|
||||||
|
title="AI Tools Impact: Before vs After (2025)",
|
||||||
|
)
|
||||||
|
print(f"Chart saved → {chart_out}")
|
||||||
|
|
||||||
|
bullets = [
|
||||||
|
"AI reduced content cycles by 40% in 2025",
|
||||||
|
"HubSpot: 12% lift in CTR with AI-assisted copy",
|
||||||
|
"Video production cost down 3x with hybrid pipeline",
|
||||||
|
]
|
||||||
|
bullet_out = make_bullet_overlay(bullets, "/tmp/demo_bullets.png")
|
||||||
|
print(f"Bullets saved → {bullet_out}")
|
||||||
|
|
||||||
|
card_out = make_insight_card(
|
||||||
|
"AI tools reduced content cycles by 40%",
|
||||||
|
"HubSpot 2026 report — 12% lift in CTR",
|
||||||
|
"/tmp/demo_card.png",
|
||||||
|
)
|
||||||
|
print(f"Insight card saved → {card_out}")
|
||||||
|
|
||||||
|
sample_json = json.dumps({
|
||||||
|
"key_insight": "AI reduced production time by 40%",
|
||||||
|
"supporting_stat": "HubSpot 2026: 12% CTR lift",
|
||||||
|
"visual_cue": "bar_chart_comparison",
|
||||||
|
"audio_tone": "authoritative_and_surprising",
|
||||||
|
"duration": 8.0,
|
||||||
|
"chart_data": sample_bar_data,
|
||||||
|
})
|
||||||
|
print("\nSample Insight JSON:\n", sample_json)
|
||||||
|
print("\nAll asset generation tests passed.")
|
||||||
|
print("To run full video composition, supply real background_img and avatar_video paths.")
|
||||||
253
backend/services/podcast/broll_service.py
Normal file
253
backend/services/podcast/broll_service.py
Normal file
@@ -0,0 +1,253 @@
|
|||||||
|
"""
|
||||||
|
B-Roll Service - Orchestrator for programmatic B-roll video composition.
|
||||||
|
|
||||||
|
This service handles:
|
||||||
|
- Chart data extraction from research
|
||||||
|
- Individual scene B-roll video generation
|
||||||
|
- Final video composition from multiple B-roll scenes
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Dict, Any, Optional, List
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
# Import chart generators directly
|
||||||
|
from services.podcast.broll_composer import (
|
||||||
|
make_bar_chart,
|
||||||
|
make_horizontal_bar,
|
||||||
|
make_line_trend,
|
||||||
|
make_pie_chart,
|
||||||
|
make_stacked_bar,
|
||||||
|
make_bullet_overlay,
|
||||||
|
make_insight_card,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class BrollService:
|
||||||
|
"""Orchestrates B-roll composition for podcast scenes."""
|
||||||
|
|
||||||
|
def __init__(self, output_dir: Optional[str] = None):
|
||||||
|
"""
|
||||||
|
Initialize B-roll service.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
output_dir: Base directory for B-roll output. Defaults to temp directory.
|
||||||
|
"""
|
||||||
|
if output_dir:
|
||||||
|
self.output_dir = Path(output_dir)
|
||||||
|
else:
|
||||||
|
self.output_dir = Path(tempfile.gettempdir()) / "broll_output"
|
||||||
|
|
||||||
|
self.output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
logger.info(f"[BrollService] Initialized with output directory: {self.output_dir}")
|
||||||
|
|
||||||
|
def get_output_path(self, filename: str) -> Path:
|
||||||
|
"""Get output path for a file."""
|
||||||
|
return self.output_dir / filename
|
||||||
|
|
||||||
|
def generate_chart_preview(
|
||||||
|
self,
|
||||||
|
chart_data: Dict[str, Any],
|
||||||
|
chart_type: str = "bar_comparison",
|
||||||
|
title: str = "",
|
||||||
|
subtitle: str = "",
|
||||||
|
) -> str:
|
||||||
|
"""
|
||||||
|
Generate a chart PNG preview (static, for Write phase).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
chart_data: Chart data dict with labels, before/after, etc.
|
||||||
|
chart_type: Type of chart (bar_comparison, bar_horizontal, line_trend, pie, stacked_bar, bullet)
|
||||||
|
title: Title for the chart
|
||||||
|
subtitle: Optional subtitle at bottom
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to generated PNG file
|
||||||
|
"""
|
||||||
|
chart_id = uuid.uuid4().hex[:8]
|
||||||
|
out_path = str(self.get_output_path(f"chart_preview_{chart_id}.png"))
|
||||||
|
|
||||||
|
try:
|
||||||
|
if chart_type == "bar_comparison":
|
||||||
|
make_bar_chart(chart_data, out_path, title, subtitle=subtitle)
|
||||||
|
elif chart_type == "bar_horizontal":
|
||||||
|
make_horizontal_bar(chart_data, out_path, title)
|
||||||
|
elif chart_type == "line_trend":
|
||||||
|
make_line_trend(chart_data, out_path, title)
|
||||||
|
elif chart_type == "pie":
|
||||||
|
make_pie_chart(chart_data, out_path, title)
|
||||||
|
elif chart_type == "pie":
|
||||||
|
make_pie_chart(chart_data, out_path, title)
|
||||||
|
elif chart_type == "stacked_bar":
|
||||||
|
make_stacked_bar(chart_data, out_path, title)
|
||||||
|
elif chart_type == "bullet":
|
||||||
|
bullet_points = chart_data.get("bullet_points", [])
|
||||||
|
if bullet_points:
|
||||||
|
make_bullet_overlay(bullet_points, out_path)
|
||||||
|
else:
|
||||||
|
logger.warning("[BrollService] No bullet points provided")
|
||||||
|
return ""
|
||||||
|
else:
|
||||||
|
logger.warning(f"[BrollService] Unknown chart type: {chart_type}")
|
||||||
|
return ""
|
||||||
|
|
||||||
|
logger.info(f"[BrollService] Chart preview generated: {out_path}")
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"[BrollService] Failed to generate chart preview: {e}")
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def generate_scene_broll(
|
||||||
|
self,
|
||||||
|
scene_id: str,
|
||||||
|
key_insight: str,
|
||||||
|
supporting_stat: str,
|
||||||
|
chart_data: Optional[Dict[str, Any]],
|
||||||
|
visual_cue: str, # bar_chart_comparison, bullet_points, full_avatar
|
||||||
|
duration: float,
|
||||||
|
background_img_path: str,
|
||||||
|
avatar_video_path: Optional[str] = None,
|
||||||
|
) -> str:
|
||||||
|
"""
|
||||||
|
Generate a B-roll video for a single scene.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
scene_id: Scene identifier
|
||||||
|
key_insight: Main insight text for overlay
|
||||||
|
supporting_stat: Supporting statistic text
|
||||||
|
chart_data: Chart data dict (optional)
|
||||||
|
visual_cue: Type of scene to build
|
||||||
|
duration: Scene duration in seconds
|
||||||
|
background_img_path: Path to background image
|
||||||
|
avatar_video_path: Path to avatar video (optional)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to generated video file
|
||||||
|
"""
|
||||||
|
scene_id_safe = scene_id.replace(" ", "_").replace("/", "_")
|
||||||
|
out_path = str(self.get_output_path(f"broll_{scene_id_safe}.mp4"))
|
||||||
|
|
||||||
|
try:
|
||||||
|
insight = Insight(
|
||||||
|
key_insight=key_insight,
|
||||||
|
supporting_stat=supporting_stat,
|
||||||
|
visual_cue=visual_cue,
|
||||||
|
audio_tone="neutral",
|
||||||
|
chart_data=chart_data or {},
|
||||||
|
duration=duration,
|
||||||
|
)
|
||||||
|
|
||||||
|
assets = SceneAssets(
|
||||||
|
background_img=background_img_path,
|
||||||
|
avatar_video=avatar_video_path,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Generate the scene
|
||||||
|
scene = dispatch_scene(insight, assets)
|
||||||
|
|
||||||
|
# Write video
|
||||||
|
compose_video([scene], output_path=out_path)
|
||||||
|
|
||||||
|
logger.info(f"[BrollService] B-roll scene generated: {out_path}")
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"[BrollService] Failed to generate B-roll scene: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
def compose_final_video(
|
||||||
|
self,
|
||||||
|
video_paths: List[str],
|
||||||
|
output_filename: str,
|
||||||
|
fade_dur: float = 0.5,
|
||||||
|
fps: int = 24,
|
||||||
|
) -> str:
|
||||||
|
"""
|
||||||
|
Compose multiple B-roll scene videos into final video.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
video_paths: List of video file paths to compose
|
||||||
|
output_filename: Output filename
|
||||||
|
fade_dur: Crossfade duration between scenes
|
||||||
|
fps: Output FPS
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to final composed video
|
||||||
|
"""
|
||||||
|
out_path = str(self.get_output_path(output_filename))
|
||||||
|
|
||||||
|
try:
|
||||||
|
scenes = []
|
||||||
|
for video_path in video_paths:
|
||||||
|
from moviepy import VideoFileClip
|
||||||
|
clip = VideoFileClip(video_path)
|
||||||
|
scenes.append(clip)
|
||||||
|
|
||||||
|
if not scenes:
|
||||||
|
raise ValueError("No video clips provided")
|
||||||
|
|
||||||
|
# Use crossfade_concat from broll_composer
|
||||||
|
from services.podcast.broll_composer import crossfade_concat
|
||||||
|
|
||||||
|
final = crossfade_concat(scenes, fade_dur=fade_dur)
|
||||||
|
|
||||||
|
final.write_videofile(
|
||||||
|
out_path,
|
||||||
|
fps=fps,
|
||||||
|
codec="libx264",
|
||||||
|
audio_codec="aac",
|
||||||
|
threads=4,
|
||||||
|
preset="fast",
|
||||||
|
logger=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Close clips
|
||||||
|
for clip in scenes:
|
||||||
|
clip.close()
|
||||||
|
|
||||||
|
logger.info(f"[BrollService] Final video composed: {out_path}")
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"[BrollService] Failed to compose final video: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
def cleanup(self, file_paths: List[str] = None):
|
||||||
|
"""
|
||||||
|
Clean up temporary B-roll files.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
file_paths: Specific files to delete. If None, cleans output directory.
|
||||||
|
"""
|
||||||
|
if file_paths:
|
||||||
|
for path in file_paths:
|
||||||
|
try:
|
||||||
|
if os.path.exists(path):
|
||||||
|
os.remove(path)
|
||||||
|
logger.debug(f"[BrollService] Removed: {path}")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"[BrollService] Failed to remove {path}: {e}")
|
||||||
|
else:
|
||||||
|
# Clean entire output directory
|
||||||
|
for file in self.output_dir.glob("*"):
|
||||||
|
try:
|
||||||
|
file.unlink()
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"[BrollService] Failed to remove {file}: {e}")
|
||||||
|
|
||||||
|
|
||||||
|
# Singleton instance for reuse
|
||||||
|
_broll_service_instance: Optional[BrollService] = None
|
||||||
|
|
||||||
|
|
||||||
|
def get_broll_service(output_dir: Optional[str] = None) -> BrollService:
|
||||||
|
"""Get or create B-roll service singleton."""
|
||||||
|
global _broll_service_instance
|
||||||
|
if _broll_service_instance is None:
|
||||||
|
_broll_service_instance = BrollService(output_dir=output_dir)
|
||||||
|
return _broll_service_instance
|
||||||
@@ -1,4 +1,6 @@
|
|||||||
from typing import Dict, Any, Optional
|
from typing import Dict, Any, Optional
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
import time
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
from services.product_marketing.personalization_service import PersonalizationService
|
from services.product_marketing.personalization_service import PersonalizationService
|
||||||
from models.podcast_bible_models import (
|
from models.podcast_bible_models import (
|
||||||
@@ -11,18 +13,61 @@ from models.podcast_bible_models import (
|
|||||||
ShowRules
|
ShowRules
|
||||||
)
|
)
|
||||||
|
|
||||||
|
_BIBLE_CACHE_TTL_SECONDS = 120
|
||||||
|
|
||||||
|
|
||||||
class PodcastBibleService:
|
class PodcastBibleService:
|
||||||
"""Service for generating and managing the Podcast Bible."""
|
"""Service for generating and managing the Podcast Bible."""
|
||||||
|
|
||||||
|
_bible_cache: Dict[str, Dict[str, Any]] = {}
|
||||||
|
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.personalization_service = PersonalizationService()
|
try:
|
||||||
|
from services.product_marketing.personalization_service import PersonalizationService
|
||||||
|
self.personalization_service = PersonalizationService()
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Failed to initialize PersonalizationService: {e}")
|
||||||
|
self.personalization_service = None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def clear_user_cache(cls, user_id: str) -> int:
|
||||||
|
"""Clear cached Bible data for a specific user. Returns number of entries cleared."""
|
||||||
|
keys_to_remove = [key for key in cls._bible_cache if key.startswith(f"{user_id}:")]
|
||||||
|
for key in keys_to_remove:
|
||||||
|
del cls._bible_cache[key]
|
||||||
|
if keys_to_remove:
|
||||||
|
logger.info(f"[BibleCache] Cleared {len(keys_to_remove)} cache entries for user {user_id}")
|
||||||
|
return len(keys_to_remove)
|
||||||
|
|
||||||
def generate_bible(self, user_id: str, project_id: str) -> PodcastBible:
|
def generate_bible(self, user_id: str, project_id: str) -> PodcastBible:
|
||||||
"""Generate a Podcast Bible from onboarding data."""
|
"""Generate a Podcast Bible from onboarding data."""
|
||||||
|
bible_start = time.time()
|
||||||
|
|
||||||
|
cache_key = f"{user_id}:{project_id}"
|
||||||
|
cached = self._bible_cache.get(cache_key)
|
||||||
|
if cached and cached.get('expires_at') and cached['expires_at'] > datetime.utcnow():
|
||||||
|
elapsed_ms = (time.time() - bible_start) * 1000
|
||||||
|
logger.warning(f"[BibleCache] HIT for {user_id} — saved 7 DB queries, overhead {elapsed_ms:.0f}ms")
|
||||||
|
return cached['bible']
|
||||||
|
|
||||||
logger.info(f"Generating Podcast Bible for user {user_id}")
|
logger.info(f"Generating Podcast Bible for user {user_id}")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
preferences = self.personalization_service.get_user_preferences(user_id) or {}
|
if not self.personalization_service:
|
||||||
|
elapsed_ms = (time.time() - bible_start) * 1000
|
||||||
|
logger.warning(f"[BibleCache] MISS (fallback) for {user_id} — PersonalizationService unavailable, {elapsed_ms:.0f}ms")
|
||||||
|
return self._get_default_bible(project_id)
|
||||||
|
|
||||||
|
try:
|
||||||
|
preferences = self.personalization_service.get_user_preferences(user_id)
|
||||||
|
except Exception as pref_err:
|
||||||
|
elapsed_ms = (time.time() - bible_start) * 1000
|
||||||
|
logger.warning(f"[BibleCache] MISS (fallback) for {user_id} — get_user_preferences failed ({pref_err}), {elapsed_ms:.0f}ms")
|
||||||
|
return self._get_default_bible(project_id)
|
||||||
|
|
||||||
|
if not preferences:
|
||||||
|
logger.info(f"No preferences found for user {user_id}, using defaults")
|
||||||
|
return self._get_default_bible(project_id)
|
||||||
if not isinstance(preferences, dict):
|
if not isinstance(preferences, dict):
|
||||||
logger.warning(f"Podcast Bible preferences payload is non-dict for user {user_id}, using defaults")
|
logger.warning(f"Podcast Bible preferences payload is non-dict for user {user_id}, using defaults")
|
||||||
preferences = {}
|
preferences = {}
|
||||||
@@ -114,6 +159,12 @@ class PodcastBibleService:
|
|||||||
)
|
)
|
||||||
|
|
||||||
logger.info(f"Podcast Bible generated successfully for project {project_id}")
|
logger.info(f"Podcast Bible generated successfully for project {project_id}")
|
||||||
|
elapsed_ms = (time.time() - bible_start) * 1000
|
||||||
|
logger.warning(f"[BibleCache] MISS — generated in {elapsed_ms:.0f}ms (7 DB queries), cached for {_BIBLE_CACHE_TTL_SECONDS}s")
|
||||||
|
self._bible_cache[cache_key] = {
|
||||||
|
'bible': bible,
|
||||||
|
'expires_at': datetime.utcnow() + timedelta(seconds=_BIBLE_CACHE_TTL_SECONDS),
|
||||||
|
}
|
||||||
return bible
|
return bible
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -129,18 +180,23 @@ class PodcastBibleService:
|
|||||||
name="AI Host",
|
name="AI Host",
|
||||||
background="Industry Professional",
|
background="Industry Professional",
|
||||||
expertise_level="Expert",
|
expertise_level="Expert",
|
||||||
|
personality_traits=["Professional", "Informative"],
|
||||||
vocal_style="Authoritative",
|
vocal_style="Authoritative",
|
||||||
vocal_characteristics=["Deep", "Steady"]
|
vocal_characteristics=["Deep", "Steady"],
|
||||||
|
look="A professional individual dressed in business-casual attire."
|
||||||
),
|
),
|
||||||
audience=AudienceDNA(
|
audience=AudienceDNA(
|
||||||
expertise_level="Intermediate",
|
expertise_level="Intermediate",
|
||||||
interests=["Industry Trends", "Technology"],
|
interests=["Industry Trends", "Technology"],
|
||||||
pain_points=["Staying Competitive", "Operational Efficiency"]
|
pain_points=["Staying Competitive", "Operational Efficiency"],
|
||||||
|
demographics=None
|
||||||
),
|
),
|
||||||
brand=BrandDNA(
|
brand=BrandDNA(
|
||||||
industry="General Business",
|
industry="General Business",
|
||||||
tone="Professional",
|
tone="Professional",
|
||||||
communication_style="Analytical"
|
communication_style="Analytical",
|
||||||
|
key_messages=[],
|
||||||
|
competitor_context=None
|
||||||
),
|
),
|
||||||
visual_style=VisualStyle(
|
visual_style=VisualStyle(
|
||||||
environment="Professional modern office studio",
|
environment="Professional modern office studio",
|
||||||
@@ -154,8 +210,12 @@ class PodcastBibleService:
|
|||||||
)
|
)
|
||||||
|
|
||||||
def serialize_bible(self, bible: PodcastBible) -> str:
|
def serialize_bible(self, bible: PodcastBible) -> str:
|
||||||
"""Serialize the Bible into a prompt-friendly text block."""
|
"""Serialize the Bible into a prompt-friendly text block. Results are cached by project_id."""
|
||||||
return f"""
|
cache_key = f"serialized:{bible.project_id}"
|
||||||
|
cached = self._bible_cache.get(cache_key)
|
||||||
|
if cached and cached.get('expires_at') and cached['expires_at'] > datetime.utcnow() and isinstance(cached.get('serialized'), str):
|
||||||
|
return cached['serialized']
|
||||||
|
serialized = f"""
|
||||||
<podcast_bible>
|
<podcast_bible>
|
||||||
HOST PERSONA:
|
HOST PERSONA:
|
||||||
- Name: {bible.host.name}
|
- Name: {bible.host.name}
|
||||||
@@ -190,3 +250,8 @@ SHOW RULES & STRUCTURE:
|
|||||||
- Constraints: {', '.join(bible.show_rules.constraints)}
|
- Constraints: {', '.join(bible.show_rules.constraints)}
|
||||||
</podcast_bible>
|
</podcast_bible>
|
||||||
"""
|
"""
|
||||||
|
self._bible_cache[cache_key] = {
|
||||||
|
'serialized': serialized,
|
||||||
|
'expires_at': datetime.utcnow() + timedelta(seconds=_BIBLE_CACHE_TTL_SECONDS),
|
||||||
|
}
|
||||||
|
return serialized
|
||||||
|
|||||||
@@ -4,11 +4,11 @@ Podcast Service
|
|||||||
Service layer for managing podcast project persistence.
|
Service layer for managing podcast project persistence.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
from sqlalchemy import desc, and_, or_
|
from sqlalchemy import desc, and_, or_
|
||||||
from typing import Optional, List, Dict, Any
|
from typing import Optional, List, Dict, Any
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
import uuid
|
|
||||||
|
|
||||||
from models.podcast_models import PodcastProject
|
from models.podcast_models import PodcastProject
|
||||||
from services.podcast_bible_service import PodcastBibleService
|
from services.podcast_bible_service import PodcastBibleService
|
||||||
@@ -32,8 +32,14 @@ class PodcastService:
|
|||||||
**kwargs
|
**kwargs
|
||||||
) -> PodcastProject:
|
) -> PodcastProject:
|
||||||
"""Create a new podcast project."""
|
"""Create a new podcast project."""
|
||||||
# Generate Podcast Bible automatically from onboarding data
|
# Generate Podcast Bible in full mode only — skip in podcast-only mode
|
||||||
bible = self.bible_service.generate_bible(user_id, project_id)
|
bible_data = None
|
||||||
|
if os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() != "podcast":
|
||||||
|
try:
|
||||||
|
bible = self.bible_service.generate_bible(user_id, project_id)
|
||||||
|
bible_data = bible.model_dump() if bible else None
|
||||||
|
except Exception:
|
||||||
|
pass # Bible is optional, project creation continues regardless
|
||||||
|
|
||||||
project = PodcastProject(
|
project = PodcastProject(
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
@@ -42,7 +48,7 @@ class PodcastService:
|
|||||||
duration=duration,
|
duration=duration,
|
||||||
speakers=speakers,
|
speakers=speakers,
|
||||||
budget_cap=budget_cap,
|
budget_cap=budget_cap,
|
||||||
bible=bible.model_dump() if bible else None,
|
bible=bible_data,
|
||||||
status="draft",
|
status="draft",
|
||||||
current_step="create",
|
current_step="create",
|
||||||
**kwargs
|
**kwargs
|
||||||
@@ -61,6 +67,17 @@ class PodcastService:
|
|||||||
)
|
)
|
||||||
).first()
|
).first()
|
||||||
|
|
||||||
|
def get_project_by_idea(self, user_id: str, idea: str) -> Optional[PodcastProject]:
|
||||||
|
"""Find a project by matching idea (case-insensitive, partial match)."""
|
||||||
|
# Normalize idea for comparison
|
||||||
|
normalized_idea = idea.strip().lower()
|
||||||
|
return self.db.query(PodcastProject).filter(
|
||||||
|
and_(
|
||||||
|
PodcastProject.user_id == user_id,
|
||||||
|
PodcastProject.idea.ilike(f"%{normalized_idea}%")
|
||||||
|
)
|
||||||
|
).order_by(desc(PodcastProject.updated_at)).first()
|
||||||
|
|
||||||
def update_project(
|
def update_project(
|
||||||
self,
|
self,
|
||||||
user_id: str,
|
user_id: str,
|
||||||
|
|||||||
@@ -3,6 +3,8 @@ from datetime import datetime, timezone
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, List, Optional
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.routing import APIRoute
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
from sqlalchemy import inspect, text
|
from sqlalchemy import inspect, text
|
||||||
|
|
||||||
@@ -15,6 +17,7 @@ from services.database import (
|
|||||||
init_database,
|
init_database,
|
||||||
default_engine,
|
default_engine,
|
||||||
)
|
)
|
||||||
|
from services.user_api_key_context import get_user_api_keys
|
||||||
|
|
||||||
_REQUIRED_SCHEMA: Dict[str, List[str]] = {
|
_REQUIRED_SCHEMA: Dict[str, List[str]] = {
|
||||||
"onboarding_sessions": ["id", "user_id", "updated_at"],
|
"onboarding_sessions": ["id", "user_id", "updated_at"],
|
||||||
@@ -144,7 +147,123 @@ def _check_db_access(checks: List[Dict[str, Any]], errors: List[str], warnings:
|
|||||||
return candidate_user
|
return candidate_user
|
||||||
|
|
||||||
|
|
||||||
def run_startup_health_routine() -> Dict[str, Any]:
|
def _check_production_api_key_loading(
|
||||||
|
checks: List[Dict[str, Any]],
|
||||||
|
errors: List[str],
|
||||||
|
warnings: List[str],
|
||||||
|
) -> None:
|
||||||
|
deploy_env = os.getenv("DEPLOY_ENV", "local").strip().lower()
|
||||||
|
if deploy_env == "local":
|
||||||
|
_record_check(checks, "production_api_key_loading", True, "skipped in local deploy mode")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Also skip in podcast-only mode (no production API keys needed)
|
||||||
|
enabled_features = os.getenv("ALWRITY_ENABLED_FEATURES", "all").strip().lower()
|
||||||
|
if enabled_features == "podcast":
|
||||||
|
_record_check(checks, "production_api_key_loading", True, "skipped in podcast-only mode")
|
||||||
|
return
|
||||||
|
|
||||||
|
test_tenant_id = os.getenv("ALWRITY_STARTUP_TEST_TENANT_ID", "").strip()
|
||||||
|
if not test_tenant_id:
|
||||||
|
message = (
|
||||||
|
"Missing ALWRITY_STARTUP_TEST_TENANT_ID for production API key startup check."
|
||||||
|
)
|
||||||
|
errors.append(message)
|
||||||
|
_record_check(checks, "production_api_key_loading", False, message)
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
keys = get_user_api_keys(test_tenant_id)
|
||||||
|
except Exception as exc:
|
||||||
|
errors.append(
|
||||||
|
f"Failed to load API keys for startup test tenant '{test_tenant_id}': {exc}"
|
||||||
|
)
|
||||||
|
_record_check(checks, "production_api_key_loading", False, str(exc))
|
||||||
|
return
|
||||||
|
|
||||||
|
if not isinstance(keys, dict):
|
||||||
|
errors.append(
|
||||||
|
f"API key loader returned invalid payload type for startup test tenant '{test_tenant_id}'."
|
||||||
|
)
|
||||||
|
_record_check(checks, "production_api_key_loading", False, "invalid payload type")
|
||||||
|
return
|
||||||
|
|
||||||
|
non_empty_keys = [provider for provider, value in keys.items() if value]
|
||||||
|
if not non_empty_keys:
|
||||||
|
errors.append(
|
||||||
|
f"No API keys could be loaded for startup test tenant '{test_tenant_id}'."
|
||||||
|
)
|
||||||
|
_record_check(checks, "production_api_key_loading", False, "no non-empty keys loaded")
|
||||||
|
return
|
||||||
|
|
||||||
|
warning = None
|
||||||
|
if len(non_empty_keys) < len(keys):
|
||||||
|
warning = (
|
||||||
|
f"Startup test tenant '{test_tenant_id}' has {len(non_empty_keys)}/{len(keys)} non-empty API keys."
|
||||||
|
)
|
||||||
|
warnings.append(warning)
|
||||||
|
|
||||||
|
detail = f"loaded {len(non_empty_keys)} non-empty keys for tenant {test_tenant_id}"
|
||||||
|
if warning:
|
||||||
|
detail = f"{detail}; {warning}"
|
||||||
|
_record_check(checks, "production_api_key_loading", True, detail)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_demo_mode() -> bool:
|
||||||
|
app_env = os.getenv("APP_ENV", os.getenv("ENV", os.getenv("DEPLOY_ENV", ""))).strip().lower()
|
||||||
|
if app_env == "demo":
|
||||||
|
return True
|
||||||
|
return _env_true("ALWRITY_DEMO_MODE", default=False)
|
||||||
|
|
||||||
|
|
||||||
|
def _check_required_demo_routes(
|
||||||
|
app: Optional[FastAPI],
|
||||||
|
checks: List[Dict[str, Any]],
|
||||||
|
errors: List[str],
|
||||||
|
) -> None:
|
||||||
|
if not _is_demo_mode():
|
||||||
|
_record_check(
|
||||||
|
checks,
|
||||||
|
"demo_required_routes",
|
||||||
|
True,
|
||||||
|
"Skipped (not in demo mode). Set APP_ENV=demo or ALWRITY_DEMO_MODE=true to enforce.",
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
if app is None:
|
||||||
|
errors.append(
|
||||||
|
"Demo startup route check could not run because FastAPI app context was not provided to startup health routine."
|
||||||
|
)
|
||||||
|
_record_check(checks, "demo_required_routes_context", False, "missing app context")
|
||||||
|
return
|
||||||
|
|
||||||
|
required_routes = {
|
||||||
|
"/api/subscription/plans": "GET",
|
||||||
|
"/api/podcast/projects": "GET",
|
||||||
|
}
|
||||||
|
available_routes = {
|
||||||
|
(route.path, method)
|
||||||
|
for route in app.router.routes
|
||||||
|
if isinstance(route, APIRoute)
|
||||||
|
for method in route.methods
|
||||||
|
}
|
||||||
|
|
||||||
|
missing: List[str] = []
|
||||||
|
for path, method in required_routes.items():
|
||||||
|
if (path, method) in available_routes:
|
||||||
|
_record_check(checks, f"demo_route_{path}_{method}", True, "route registered")
|
||||||
|
else:
|
||||||
|
missing.append(f"{method} {path}")
|
||||||
|
_record_check(checks, f"demo_route_{path}_{method}", False, "route missing")
|
||||||
|
|
||||||
|
if missing:
|
||||||
|
errors.append(
|
||||||
|
"Demo mode startup check failed. Missing required API endpoints: "
|
||||||
|
f"{', '.join(missing)}. Ensure subscription and podcast routers are imported and included during app setup."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def run_startup_health_routine(app: Optional[FastAPI] = None) -> Dict[str, Any]:
|
||||||
checks: List[Dict[str, Any]] = []
|
checks: List[Dict[str, Any]] = []
|
||||||
errors: List[str] = []
|
errors: List[str] = []
|
||||||
warnings: List[str] = []
|
warnings: List[str] = []
|
||||||
@@ -152,6 +271,9 @@ def run_startup_health_routine() -> Dict[str, Any]:
|
|||||||
_check_workspace_root(checks, errors)
|
_check_workspace_root(checks, errors)
|
||||||
if not errors:
|
if not errors:
|
||||||
_check_db_access(checks, errors, warnings)
|
_check_db_access(checks, errors, warnings)
|
||||||
|
_check_required_demo_routes(app, checks, errors)
|
||||||
|
if not errors:
|
||||||
|
_check_production_api_key_loading(checks, errors, warnings)
|
||||||
|
|
||||||
status = "healthy" if not errors else "failed"
|
status = "healthy" if not errors else "failed"
|
||||||
report = {
|
report = {
|
||||||
|
|||||||
@@ -46,6 +46,7 @@ class StoryAudioGenerationService:
|
|||||||
return _get_story_media_write_dir("audio", user_id=user_id, db=db)
|
return _get_story_media_write_dir("audio", user_id=user_id, db=db)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"[StoryAudioGeneration] Failed to resolve user workspace path for {user_id}: {e}")
|
logger.warning(f"[StoryAudioGeneration] Failed to resolve user workspace path for {user_id}: {e}")
|
||||||
|
# Don't fall back to default - keep using the already-set output_dir for podcast
|
||||||
return self.output_dir
|
return self.output_dir
|
||||||
|
|
||||||
def _generate_audio_filename(self, scene_number: int, scene_title: str) -> str:
|
def _generate_audio_filename(self, scene_number: int, scene_title: str) -> str:
|
||||||
@@ -318,6 +319,7 @@ class StoryAudioGenerationService:
|
|||||||
text: str,
|
text: str,
|
||||||
user_id: str,
|
user_id: str,
|
||||||
voice_id: str = "Wise_Woman",
|
voice_id: str = "Wise_Woman",
|
||||||
|
custom_voice_id: Optional[str] = None,
|
||||||
speed: float = 1.0,
|
speed: float = 1.0,
|
||||||
volume: float = 1.0,
|
volume: float = 1.0,
|
||||||
pitch: float = 0.0,
|
pitch: float = 0.0,
|
||||||
@@ -364,6 +366,7 @@ class StoryAudioGenerationService:
|
|||||||
result = generate_audio(
|
result = generate_audio(
|
||||||
text=text.strip(),
|
text=text.strip(),
|
||||||
voice_id=voice_id,
|
voice_id=voice_id,
|
||||||
|
custom_voice_id=custom_voice_id,
|
||||||
speed=speed,
|
speed=speed,
|
||||||
volume=volume,
|
volume=volume,
|
||||||
pitch=pitch,
|
pitch=pitch,
|
||||||
@@ -378,8 +381,8 @@ class StoryAudioGenerationService:
|
|||||||
enable_sync_mode=enable_sync_mode,
|
enable_sync_mode=enable_sync_mode,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Determine output directory (user workspace or default)
|
# Use the output_dir that was set when service was created (already handles podcast vs story)
|
||||||
output_dir = self._get_user_audio_dir(user_id, db)
|
output_dir = self.output_dir
|
||||||
|
|
||||||
# Save audio to file
|
# Save audio to file
|
||||||
audio_filename = self._generate_audio_filename(scene_number, scene_title)
|
audio_filename = self._generate_audio_filename(scene_number, scene_title)
|
||||||
|
|||||||
@@ -31,8 +31,8 @@ def log_video_stack_diagnostics() -> None:
|
|||||||
def assert_supported_moviepy() -> None:
|
def assert_supported_moviepy() -> None:
|
||||||
"""Fail fast if MoviePy isn't version 2.x."""
|
"""Fail fast if MoviePy isn't version 2.x."""
|
||||||
try:
|
try:
|
||||||
import pkg_resources as pr
|
from importlib.metadata import version
|
||||||
mv = pr.get_distribution("moviepy").version
|
mv = version("moviepy")
|
||||||
if not mv.startswith("2."):
|
if not mv.startswith("2."):
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"Unsupported MoviePy version {mv}. Expected 2.x. "
|
f"Unsupported MoviePy version {mv}. Expected 2.x. "
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import os
|
||||||
from typing import Dict, Any, List, Optional
|
from typing import Dict, Any, List, Optional
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
@@ -21,7 +22,7 @@ class StrategyCopilotService:
|
|||||||
"""Generate data for a specific category."""
|
"""Generate data for a specific category."""
|
||||||
try:
|
try:
|
||||||
# Get user onboarding data
|
# Get user onboarding data
|
||||||
user_id = 1 # TODO: Get from auth context
|
user_id = int(os.getenv("ALWRITY_FALLBACK_USER_ID", "0"))
|
||||||
integrated_data = await self.onboarding_integration_service.process_onboarding_data(str(user_id), self.db)
|
integrated_data = await self.onboarding_integration_service.process_onboarding_data(str(user_id), self.db)
|
||||||
onboarding_data = integrated_data.get('canonical_profile', {})
|
onboarding_data = integrated_data.get('canonical_profile', {})
|
||||||
|
|
||||||
@@ -81,7 +82,7 @@ class StrategyCopilotService:
|
|||||||
"""Analyze complete strategy for completeness and coherence."""
|
"""Analyze complete strategy for completeness and coherence."""
|
||||||
try:
|
try:
|
||||||
# Get user data for context
|
# Get user data for context
|
||||||
user_id = 1 # TODO: Get from auth context
|
user_id = int(os.getenv("ALWRITY_FALLBACK_USER_ID", "0"))
|
||||||
integrated_data = await self.onboarding_integration_service.process_onboarding_data(str(user_id), self.db)
|
integrated_data = await self.onboarding_integration_service.process_onboarding_data(str(user_id), self.db)
|
||||||
onboarding_data = integrated_data.get('canonical_profile', {})
|
onboarding_data = integrated_data.get('canonical_profile', {})
|
||||||
|
|
||||||
@@ -118,7 +119,7 @@ class StrategyCopilotService:
|
|||||||
field_definition = self._get_field_definition(field_id)
|
field_definition = self._get_field_definition(field_id)
|
||||||
|
|
||||||
# Get user data
|
# Get user data
|
||||||
user_id = 1 # TODO: Get from auth context
|
user_id = int(os.getenv("ALWRITY_FALLBACK_USER_ID", "0"))
|
||||||
# Use SSOT
|
# Use SSOT
|
||||||
integrated_data = await self.onboarding_integration_service.process_onboarding_data(str(user_id), self.db)
|
integrated_data = await self.onboarding_integration_service.process_onboarding_data(str(user_id), self.db)
|
||||||
onboarding_data = integrated_data.get('canonical_profile', {})
|
onboarding_data = integrated_data.get('canonical_profile', {})
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ Handles subscription limit checking and validation logic.
|
|||||||
Extracted from pricing_service.py for better modularity.
|
Extracted from pricing_service.py for better modularity.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import time
|
||||||
from typing import Dict, Any, Optional, List, Tuple, TYPE_CHECKING
|
from typing import Dict, Any, Optional, List, Tuple, TYPE_CHECKING
|
||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
from sqlalchemy import text
|
from sqlalchemy import text
|
||||||
@@ -32,9 +33,11 @@ class LimitValidator:
|
|||||||
self.db = pricing_service.db
|
self.db = pricing_service.db
|
||||||
|
|
||||||
def check_usage_limits(self, user_id: str, provider: APIProvider,
|
def check_usage_limits(self, user_id: str, provider: APIProvider,
|
||||||
tokens_requested: int = 0, actual_provider_name: Optional[str] = None) -> Tuple[bool, str, Dict[str, Any]]:
|
tokens_requested: int = 0, actual_provider_name: Optional[str] = None) -> Tuple[bool, str, Dict[str, Any]]:
|
||||||
"""Check if user can make an API call within their limits.
|
"""Check if user can make an API call within their limits.
|
||||||
|
|
||||||
|
Delegates to LimitValidator for actual validation logic.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
user_id: User ID
|
user_id: User ID
|
||||||
provider: APIProvider enum (may be MISTRAL for HuggingFace)
|
provider: APIProvider enum (may be MISTRAL for HuggingFace)
|
||||||
@@ -44,6 +47,7 @@ class LimitValidator:
|
|||||||
Returns:
|
Returns:
|
||||||
(can_proceed, error_message, usage_info)
|
(can_proceed, error_message, usage_info)
|
||||||
"""
|
"""
|
||||||
|
start_time = time.time()
|
||||||
try:
|
try:
|
||||||
# Use actual_provider_name if provided, otherwise use enum value
|
# Use actual_provider_name if provided, otherwise use enum value
|
||||||
# This fixes cases where HuggingFace maps to MISTRAL enum but should show as "huggingface" in errors
|
# This fixes cases where HuggingFace maps to MISTRAL enum but should show as "huggingface" in errors
|
||||||
@@ -51,12 +55,14 @@ class LimitValidator:
|
|||||||
|
|
||||||
logger.debug(f"[Subscription Check] Starting limit check for user {user_id}, provider {display_provider_name}, tokens {tokens_requested}")
|
logger.debug(f"[Subscription Check] Starting limit check for user {user_id}, provider {display_provider_name}, tokens {tokens_requested}")
|
||||||
|
|
||||||
|
logger.warning(f"[Subscription Check] START for user {user_id}, provider {provider.value}")
|
||||||
# Short TTL cache to reduce DB reads under sustained traffic
|
# Short TTL cache to reduce DB reads under sustained traffic
|
||||||
cache_key = f"{user_id}:{provider.value}"
|
cache_key = f"{user_id}:{provider.value}"
|
||||||
now = datetime.utcnow()
|
now = datetime.utcnow()
|
||||||
cached = self.pricing_service._limits_cache.get(cache_key)
|
cached = self.pricing_service._limits_cache.get(cache_key)
|
||||||
if cached and cached.get('expires_at') and cached['expires_at'] > now:
|
if cached and cached.get('expires_at') and cached['expires_at'] > now:
|
||||||
logger.debug(f"[Subscription Check] Using cached result for {user_id}:{provider.value}")
|
elapsed_ms = (time.time() - start_time) * 1000
|
||||||
|
logger.warning(f"[Subscription Check] Cache hit for {user_id}:{provider.value} — completed in {elapsed_ms:.0f}ms")
|
||||||
return tuple(cached['result']) # type: ignore
|
return tuple(cached['result']) # type: ignore
|
||||||
|
|
||||||
# Get user subscription first to check expiration
|
# Get user subscription first to check expiration
|
||||||
@@ -139,12 +145,15 @@ class LimitValidator:
|
|||||||
return False, "No subscription plan found. Please subscribe to a plan.", {}
|
return False, "No subscription plan found. Please subscribe to a plan.", {}
|
||||||
|
|
||||||
# Get current usage for this billing period with error handling
|
# Get current usage for this billing period with error handling
|
||||||
# CRITICAL: Use fresh queries to avoid SQLAlchemy cache after renewal
|
# Use targeted expiry instead of expire_all() to avoid nuking the entire session cache
|
||||||
try:
|
try:
|
||||||
current_period = self.pricing_service.get_current_billing_period(user_id) or datetime.now().strftime("%Y-%m")
|
current_period = self.pricing_service.get_current_billing_period(user_id) or datetime.now().strftime("%Y-%m")
|
||||||
|
|
||||||
# Expire all objects to force fresh read from DB (critical after renewal)
|
# Only expire specific objects that might have changed after renewal
|
||||||
self.db.expire_all()
|
# (subscription was already checked above; plan was expired above)
|
||||||
|
# The usage record is the main object we need fresh, and we query it directly below
|
||||||
|
if subscription:
|
||||||
|
self.db.expire(subscription)
|
||||||
|
|
||||||
# Use raw SQL query first to bypass ORM cache, fallback to ORM if SQL fails
|
# Use raw SQL query first to bypass ORM cache, fallback to ORM if SQL fails
|
||||||
usage = None
|
usage = None
|
||||||
@@ -367,14 +376,18 @@ class LimitValidator:
|
|||||||
'result': result,
|
'result': result,
|
||||||
'expires_at': now + timedelta(seconds=30)
|
'expires_at': now + timedelta(seconds=30)
|
||||||
}
|
}
|
||||||
|
elapsed_ms = (time.time() - start_time) * 1000
|
||||||
|
logger.warning(f"[Subscription Check] Completed in {elapsed_ms:.0f}ms for user {user_id}, provider {display_provider_name} — within limits (calls: {current_call_count}/{call_limit_value})")
|
||||||
return result
|
return result
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error calculating usage percentages: {e}")
|
logger.error(f"Error calculating usage percentages: {e}")
|
||||||
# Return basic success
|
elapsed_ms = (time.time() - start_time) * 1000
|
||||||
|
logger.warning(f"[Subscription Check] Completed in {elapsed_ms:.0f}ms for user {user_id}, provider {display_provider_name} — within limits (basic check)")
|
||||||
return True, "Within limits", {}
|
return True, "Within limits", {}
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Unexpected error in check_usage_limits for {user_id}: {e}")
|
elapsed_ms = (time.time() - start_time) * 1000
|
||||||
|
logger.error(f"[Subscription Check] Failed for user {user_id} after {elapsed_ms:.0f}ms: {e}")
|
||||||
# STRICT: Fail closed - deny requests if subscription system fails
|
# STRICT: Fail closed - deny requests if subscription system fails
|
||||||
return False, f"Subscription check error: {str(e)}", {}
|
return False, f"Subscription check error: {str(e)}", {}
|
||||||
|
|
||||||
@@ -417,9 +430,7 @@ class LimitValidator:
|
|||||||
except Exception as schema_err:
|
except Exception as schema_err:
|
||||||
logger.warning(f"Schema check failed, will retry on query error: {schema_err}")
|
logger.warning(f"Schema check failed, will retry on query error: {schema_err}")
|
||||||
|
|
||||||
# Explicitly expire any cached objects and refresh from DB to ensure fresh data
|
# Explicitly refresh usage from DB to ensure fresh data (targeted instead of expire_all)
|
||||||
self.db.expire_all()
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
usage = self.db.query(UsageSummary).filter(
|
usage = self.db.query(UsageSummary).filter(
|
||||||
UsageSummary.user_id == user_id,
|
UsageSummary.user_id == user_id,
|
||||||
@@ -431,14 +442,19 @@ class LimitValidator:
|
|||||||
self.db.refresh(usage)
|
self.db.refresh(usage)
|
||||||
except Exception as query_err:
|
except Exception as query_err:
|
||||||
error_str = str(query_err).lower()
|
error_str = str(query_err).lower()
|
||||||
if 'no such column' in error_str and 'exa_calls' in error_str:
|
if 'no such column' in error_str and ('exa_calls' in error_str or 'wavespeed' in error_str):
|
||||||
logger.warning("Missing column detected in usage query, fixing schema and retrying...")
|
logger.warning("Missing column detected in usage query, fixing schema and retrying...")
|
||||||
import sqlite3
|
import sqlite3
|
||||||
import services.subscription.schema_utils as schema_utils
|
import services.subscription.schema_utils as schema_utils
|
||||||
schema_utils._checked_usage_summaries_columns = False
|
schema_utils._checked_usage_summaries_columns = False
|
||||||
from services.subscription.schema_utils import ensure_usage_summaries_columns
|
from services.subscription.schema_utils import ensure_usage_summaries_columns
|
||||||
ensure_usage_summaries_columns(self.db)
|
ensure_usage_summaries_columns(self.db)
|
||||||
self.db.expire_all()
|
# After schema migration, only expire UsageSummary to force re-query
|
||||||
|
# (no need to expire the entire session)
|
||||||
|
for obj in self.db.query(UsageSummary).filter(
|
||||||
|
UsageSummary.user_id == user_id
|
||||||
|
).all():
|
||||||
|
self.db.expire(obj)
|
||||||
# Retry the query
|
# Retry the query
|
||||||
usage = self.db.query(UsageSummary).filter(
|
usage = self.db.query(UsageSummary).filter(
|
||||||
UsageSummary.user_id == user_id,
|
UsageSummary.user_id == user_id,
|
||||||
@@ -594,8 +610,9 @@ class LimitValidator:
|
|||||||
# Method 2: Fallback to fresh ORM query if raw SQL fails
|
# Method 2: Fallback to fresh ORM query if raw SQL fails
|
||||||
if not query_succeeded:
|
if not query_succeeded:
|
||||||
try:
|
try:
|
||||||
# Expire all cached objects and do fresh query
|
# Only refresh usage object, don't expire entire session
|
||||||
self.db.expire_all()
|
if usage:
|
||||||
|
self.db.refresh(usage)
|
||||||
fresh_usage = self.db.query(UsageSummary).filter(
|
fresh_usage = self.db.query(UsageSummary).filter(
|
||||||
UsageSummary.user_id == user_id,
|
UsageSummary.user_id == user_id,
|
||||||
UsageSummary.billing_period == current_period
|
UsageSummary.billing_period == current_period
|
||||||
@@ -792,7 +809,11 @@ class LimitValidator:
|
|||||||
schema_utils._checked_usage_summaries_columns = False
|
schema_utils._checked_usage_summaries_columns = False
|
||||||
from services.subscription.schema_utils import ensure_usage_summaries_columns
|
from services.subscription.schema_utils import ensure_usage_summaries_columns
|
||||||
ensure_usage_summaries_columns(self.db)
|
ensure_usage_summaries_columns(self.db)
|
||||||
self.db.expire_all()
|
# Only expire UsageSummary after schema migration, not entire session
|
||||||
|
for obj in self.db.query(UsageSummary).filter(
|
||||||
|
UsageSummary.user_id == user_id
|
||||||
|
).all():
|
||||||
|
self.db.expire(obj)
|
||||||
|
|
||||||
# Retry the query
|
# Retry the query
|
||||||
usage = self.db.query(UsageSummary).filter(
|
usage = self.db.query(UsageSummary).filter(
|
||||||
|
|||||||
@@ -442,9 +442,34 @@ class PricingService:
|
|||||||
"description": "AI Audio Generation default pricing"
|
"description": "AI Audio Generation default pricing"
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
|
|
||||||
|
# WaveSpeed LLM Text Generation Pricing (via Cerebras)
|
||||||
|
wavespeed_llm_pricing = [
|
||||||
|
{
|
||||||
|
"provider": APIProvider.WAVESPEED,
|
||||||
|
"model_name": "openai/gpt-oss-120b",
|
||||||
|
"cost_per_input_token": 0.0000006, # $0.60 per 1M input tokens
|
||||||
|
"cost_per_output_token": 0.0000006, # $0.60 per 1M output tokens
|
||||||
|
"description": "WaveSpeed GPT-OSS 120B (Cerebras) - Fast text generation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"provider": APIProvider.WAVESPEED,
|
||||||
|
"model_name": "openai/gpt-oss-120b:cerebras",
|
||||||
|
"cost_per_input_token": 0.0000006,
|
||||||
|
"cost_per_output_token": 0.0000006,
|
||||||
|
"description": "WaveSpeed GPT-OSS 120B (Cerebras) - Fast text generation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"provider": APIProvider.WAVESPEED,
|
||||||
|
"model_name": "openai/gpt-oss-20b",
|
||||||
|
"cost_per_input_token": 0.0000002, # $0.20 per 1M input tokens
|
||||||
|
"cost_per_output_token": 0.0000002, # $0.20 per 1M output tokens
|
||||||
|
"description": "WaveSpeed GPT-OSS 20B (Cerebras) - Cost-effective text generation"
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
# Combine all pricing data (include video pricing in search_pricing list)
|
# Combine all pricing data (include video pricing in search_pricing list)
|
||||||
all_pricing = gemini_pricing + openai_pricing + anthropic_pricing + mistral_pricing + search_pricing
|
all_pricing = gemini_pricing + openai_pricing + anthropic_pricing + mistral_pricing + search_pricing + wavespeed_llm_pricing
|
||||||
|
|
||||||
# Insert or update pricing data
|
# Insert or update pricing data
|
||||||
for pricing_data in all_pricing:
|
for pricing_data in all_pricing:
|
||||||
|
|||||||
@@ -88,6 +88,9 @@ def ensure_usage_summaries_columns(db: Session) -> None:
|
|||||||
"image_edit_cost": "REAL DEFAULT 0.0",
|
"image_edit_cost": "REAL DEFAULT 0.0",
|
||||||
"audio_calls": "INTEGER DEFAULT 0",
|
"audio_calls": "INTEGER DEFAULT 0",
|
||||||
"audio_cost": "REAL DEFAULT 0.0",
|
"audio_cost": "REAL DEFAULT 0.0",
|
||||||
|
"wavespeed_calls": "INTEGER DEFAULT 0",
|
||||||
|
"wavespeed_tokens": "INTEGER DEFAULT 0",
|
||||||
|
"wavespeed_cost": "REAL DEFAULT 0.0",
|
||||||
}
|
}
|
||||||
|
|
||||||
for col_name, ddl in required_columns.items():
|
for col_name, ddl in required_columns.items():
|
||||||
|
|||||||
@@ -71,10 +71,13 @@ class UserAPIKeyContext:
|
|||||||
"""Load API keys from database for specific user."""
|
"""Load API keys from database for specific user."""
|
||||||
try:
|
try:
|
||||||
from api.content_planning.services.content_strategy.onboarding import OnboardingDataIntegrationService
|
from api.content_planning.services.content_strategy.onboarding import OnboardingDataIntegrationService
|
||||||
from services.database import SessionLocal
|
from services.database import get_session_for_user
|
||||||
|
|
||||||
integration_service = OnboardingDataIntegrationService()
|
integration_service = OnboardingDataIntegrationService()
|
||||||
db = SessionLocal()
|
db = get_session_for_user(user_id)
|
||||||
|
if not db:
|
||||||
|
logger.error(f"Failed to create DB session for user {user_id}")
|
||||||
|
return {}
|
||||||
try:
|
try:
|
||||||
integrated_data = integration_service.get_integrated_data_sync(user_id, db)
|
integrated_data = integration_service.get_integrated_data_sync(user_id, db)
|
||||||
keys = integrated_data.get('api_keys_data', {})
|
keys = integrated_data.get('api_keys_data', {})
|
||||||
@@ -153,4 +156,3 @@ def get_tavily_key(user_id: Optional[str] = None) -> Optional[str]:
|
|||||||
def get_copilotkit_key(user_id: Optional[str] = None) -> Optional[str]:
|
def get_copilotkit_key(user_id: Optional[str] = None) -> Optional[str]:
|
||||||
"""Get CopilotKit API key for user."""
|
"""Get CopilotKit API key for user."""
|
||||||
return UserAPIKeyContext.get_user_key(user_id, 'copilotkit')
|
return UserAPIKeyContext.get_user_key(user_id, 'copilotkit')
|
||||||
|
|
||||||
|
|||||||
@@ -241,6 +241,7 @@ class WaveSpeedClient:
|
|||||||
self,
|
self,
|
||||||
text: str,
|
text: str,
|
||||||
voice_id: str,
|
voice_id: str,
|
||||||
|
custom_voice_id: Optional[str] = None,
|
||||||
speed: float = 1.0,
|
speed: float = 1.0,
|
||||||
volume: float = 1.0,
|
volume: float = 1.0,
|
||||||
pitch: float = 0.0,
|
pitch: float = 0.0,
|
||||||
@@ -255,6 +256,7 @@ class WaveSpeedClient:
|
|||||||
Args:
|
Args:
|
||||||
text: Text to convert to speech (max 10000 characters)
|
text: Text to convert to speech (max 10000 characters)
|
||||||
voice_id: Voice ID (e.g., "Wise_Woman", "Friendly_Person", etc.)
|
voice_id: Voice ID (e.g., "Wise_Woman", "Friendly_Person", etc.)
|
||||||
|
custom_voice_id: Custom voice clone ID for using cloned voice
|
||||||
speed: Speech speed (0.5-2.0, default: 1.0)
|
speed: Speech speed (0.5-2.0, default: 1.0)
|
||||||
volume: Speech volume (0.1-10.0, default: 1.0)
|
volume: Speech volume (0.1-10.0, default: 1.0)
|
||||||
pitch: Speech pitch (-12 to 12, default: 0.0)
|
pitch: Speech pitch (-12 to 12, default: 0.0)
|
||||||
@@ -269,6 +271,7 @@ class WaveSpeedClient:
|
|||||||
return self.speech.generate_speech(
|
return self.speech.generate_speech(
|
||||||
text=text,
|
text=text,
|
||||||
voice_id=voice_id,
|
voice_id=voice_id,
|
||||||
|
custom_voice_id=custom_voice_id,
|
||||||
speed=speed,
|
speed=speed,
|
||||||
volume=volume,
|
volume=volume,
|
||||||
pitch=pitch,
|
pitch=pitch,
|
||||||
|
|||||||
@@ -40,6 +40,7 @@ class SpeechGenerator:
|
|||||||
self,
|
self,
|
||||||
text: str,
|
text: str,
|
||||||
voice_id: str,
|
voice_id: str,
|
||||||
|
custom_voice_id: Optional[str] = None,
|
||||||
speed: float = 1.0,
|
speed: float = 1.0,
|
||||||
volume: float = 1.0,
|
volume: float = 1.0,
|
||||||
pitch: float = 0.0,
|
pitch: float = 0.0,
|
||||||
@@ -54,6 +55,7 @@ class SpeechGenerator:
|
|||||||
Args:
|
Args:
|
||||||
text: Text to convert to speech (max 10000 characters)
|
text: Text to convert to speech (max 10000 characters)
|
||||||
voice_id: Voice ID (e.g., "Wise_Woman", "Friendly_Person", etc.)
|
voice_id: Voice ID (e.g., "Wise_Woman", "Friendly_Person", etc.)
|
||||||
|
custom_voice_id: Custom voice clone ID for using cloned voice
|
||||||
speed: Speech speed (0.5-2.0, default: 1.0)
|
speed: Speech speed (0.5-2.0, default: 1.0)
|
||||||
volume: Speech volume (0.1-10.0, default: 1.0)
|
volume: Speech volume (0.1-10.0, default: 1.0)
|
||||||
pitch: Speech pitch (-12 to 12, default: 0.0)
|
pitch: Speech pitch (-12 to 12, default: 0.0)
|
||||||
@@ -77,6 +79,11 @@ class SpeechGenerator:
|
|||||||
if not sanitized_voice_id:
|
if not sanitized_voice_id:
|
||||||
raise ValueError("Voice ID cannot be empty after sanitization")
|
raise ValueError("Voice ID cannot be empty after sanitization")
|
||||||
|
|
||||||
|
# Sanitize custom_voice_id if provided
|
||||||
|
sanitized_custom_voice_id = None
|
||||||
|
if custom_voice_id:
|
||||||
|
sanitized_custom_voice_id = str(custom_voice_id).strip() or None
|
||||||
|
|
||||||
# Ensure numeric parameters are proper floats and within valid ranges
|
# Ensure numeric parameters are proper floats and within valid ranges
|
||||||
sanitized_speed = max(0.5, min(2.0, float(speed))) if speed is not None else 1.0
|
sanitized_speed = max(0.5, min(2.0, float(speed))) if speed is not None else 1.0
|
||||||
sanitized_volume = max(0.1, min(10.0, float(volume))) if volume is not None else 1.0
|
sanitized_volume = max(0.1, min(10.0, float(volume))) if volume is not None else 1.0
|
||||||
@@ -112,6 +119,10 @@ class SpeechGenerator:
|
|||||||
"enable_sync_mode": bool(enable_sync_mode),
|
"enable_sync_mode": bool(enable_sync_mode),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Add custom voice clone ID if provided
|
||||||
|
if sanitized_custom_voice_id:
|
||||||
|
payload["custom_voice_id"] = sanitized_custom_voice_id
|
||||||
|
|
||||||
# Add optional parameters with proper type validation
|
# Add optional parameters with proper type validation
|
||||||
optional_params = [
|
optional_params = [
|
||||||
"english_normalization",
|
"english_normalization",
|
||||||
@@ -179,6 +190,20 @@ class SpeechGenerator:
|
|||||||
|
|
||||||
if response.status_code != 200:
|
if response.status_code != 200:
|
||||||
logger.error(f"[WaveSpeed] Speech generation failed: {response.status_code} {response.text}")
|
logger.error(f"[WaveSpeed] Speech generation failed: {response.status_code} {response.text}")
|
||||||
|
|
||||||
|
# Check for custom voice ID specific errors
|
||||||
|
response_text = response.text.lower()
|
||||||
|
if "custom_voice" in response_text or "voice_id" in response_text:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=400,
|
||||||
|
detail={
|
||||||
|
"error": "Invalid voice clone ID",
|
||||||
|
"message": "The custom voice ID is invalid or expired. Please create a new voice clone or use a predefined voice.",
|
||||||
|
"status_code": response.status_code,
|
||||||
|
"response": response.text,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=502,
|
status_code=502,
|
||||||
detail={
|
detail={
|
||||||
|
|||||||
@@ -26,20 +26,24 @@ def _generate_simple_infinitetalk_prompt(
|
|||||||
story_context: Dict[str, Any],
|
story_context: Dict[str, Any],
|
||||||
) -> Optional[str]:
|
) -> Optional[str]:
|
||||||
"""
|
"""
|
||||||
Generate a balanced, concise prompt for InfiniteTalk.
|
Generate an enhanced prompt for InfiniteTalk video generation.
|
||||||
InfiniteTalk is audio-driven, so the prompt should describe the scene and suggest
|
Includes scene content, analysis, bible context, and visual elements.
|
||||||
subtle motion, but avoid overly elaborate cinematic descriptions.
|
|
||||||
|
|
||||||
Returns None if no meaningful prompt can be generated.
|
Returns None if no meaningful prompt can be generated.
|
||||||
"""
|
"""
|
||||||
title = (scene_data.get("title") or "").strip()
|
title = (scene_data.get("title") or "").strip()
|
||||||
description = (scene_data.get("description") or "").strip()
|
description = (scene_data.get("description") or "").strip()
|
||||||
image_prompt = (scene_data.get("image_prompt") or "").strip()
|
image_prompt = (scene_data.get("image_prompt") or "").strip()
|
||||||
|
lines = scene_data.get("lines", [])
|
||||||
|
narration = ""
|
||||||
|
if lines:
|
||||||
|
# Combine first few lines for context
|
||||||
|
narration = " ".join([str(l.get("text", "")) for l in lines[:3]])[:150]
|
||||||
|
|
||||||
# Build a balanced prompt: scene description + simple motion hint
|
# Build enhanced prompt with multiple context sources
|
||||||
parts = []
|
parts = []
|
||||||
|
|
||||||
# Add scene context
|
# Add main scene title
|
||||||
if title and len(title) > 5 and title.lower() not in ("scene", "podcast", "episode"):
|
if title and len(title) > 5 and title.lower() not in ("scene", "podcast", "episode"):
|
||||||
parts.append(title)
|
parts.append(title)
|
||||||
|
|
||||||
@@ -48,60 +52,70 @@ def _generate_simple_infinitetalk_prompt(
|
|||||||
if analysis:
|
if analysis:
|
||||||
content_type = analysis.get("content_type")
|
content_type = analysis.get("content_type")
|
||||||
if content_type:
|
if content_type:
|
||||||
parts.append(f"Style: {content_type}")
|
parts.append(f"Content type: {content_type}")
|
||||||
|
|
||||||
# Audience helps define the formality/vibe
|
# Add key takeaways if available
|
||||||
|
key_takeaways = analysis.get("keyTakeaways", [])
|
||||||
|
if key_takeaways and isinstance(key_takeaways, list) and len(key_takeaways) > 0:
|
||||||
|
takeaway = str(key_takeaways[0])[:80]
|
||||||
|
if takeaway:
|
||||||
|
parts.append(f"Key insight: {takeaway}")
|
||||||
|
|
||||||
|
# Audience
|
||||||
audience = analysis.get("audience")
|
audience = analysis.get("audience")
|
||||||
if audience:
|
if audience:
|
||||||
# Just use first few words of audience to keep it short
|
short_audience = " ".join(audience.split()[:3])
|
||||||
short_audience = " ".join(audience.split()[:3])
|
parts.append(f"Target audience: {short_audience}")
|
||||||
parts.append(f"For: {short_audience}")
|
|
||||||
|
# Guest info
|
||||||
# Add bible context if available
|
guest_name = analysis.get("guestName")
|
||||||
|
guest_expertise = analysis.get("guestExpertise")
|
||||||
|
if guest_name:
|
||||||
|
parts.append(f"Guest: {guest_name}")
|
||||||
|
if guest_expertise:
|
||||||
|
parts.append(f"Expertise: {guest_expertise}")
|
||||||
|
|
||||||
|
# Add bible context
|
||||||
bible = story_context.get("bible", {})
|
bible = story_context.get("bible", {})
|
||||||
if bible:
|
if bible:
|
||||||
host_persona = bible.get("host_persona")
|
host_persona = bible.get("host_persona")
|
||||||
tone = bible.get("tone")
|
tone = bible.get("tone")
|
||||||
|
visual_style = bible.get("visual_style")
|
||||||
|
background = bible.get("background")
|
||||||
|
|
||||||
if host_persona:
|
if host_persona:
|
||||||
parts.append(f"Host: {host_persona}")
|
parts.append(f"Host persona: {host_persona}")
|
||||||
if tone:
|
if tone:
|
||||||
parts.append(f"Tone: {tone}")
|
parts.append(f"Tone: {tone}")
|
||||||
|
if visual_style:
|
||||||
elif description:
|
parts.append(f"Visual style: {visual_style}")
|
||||||
# Take first sentence or first 60 chars
|
if background:
|
||||||
desc_part = description.split('.')[0][:60].strip()
|
parts.append(f"Background: {background}")
|
||||||
if desc_part:
|
|
||||||
parts.append(desc_part)
|
# Add original image prompt as fallback context
|
||||||
elif image_prompt:
|
if image_prompt and len(parts) < 3:
|
||||||
# Take first sentence or first 60 chars
|
img_part = image_prompt.split('.')[0][:100].strip()
|
||||||
img_part = image_prompt.split('.')[0][:60].strip()
|
|
||||||
if img_part:
|
if img_part:
|
||||||
parts.append(img_part)
|
parts.append(f"Visual context: {img_part}")
|
||||||
|
|
||||||
|
# Add narration snippet if available
|
||||||
|
if narration and len(parts) < 4:
|
||||||
|
parts.append(f"Discussing: {narration}")
|
||||||
|
|
||||||
if not parts:
|
if not parts:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
# Add a simple, subtle motion suggestion (not elaborate camera movements)
|
# Build prompt with visual quality keywords
|
||||||
# Keep it natural and audio-driven
|
quality_keywords = "Cinematic lighting, high detail, 4k quality, smooth motion"
|
||||||
motion_hints = [
|
|
||||||
"with subtle movement",
|
|
||||||
"with gentle motion",
|
|
||||||
"with natural animation",
|
|
||||||
]
|
|
||||||
|
|
||||||
# Combine scene description with subtle motion hint
|
# Combine parts into final prompt
|
||||||
if len(parts[0]) < 80:
|
prompt = f"{'. '.join(parts)}. {quality_keywords}. With subtle natural movement."
|
||||||
# Room for a motion hint
|
|
||||||
prompt = f"{parts[0]}, {motion_hints[0]}"
|
|
||||||
else:
|
|
||||||
# Just use the description if it's already long enough
|
|
||||||
prompt = parts[0]
|
|
||||||
|
|
||||||
# Keep it concise - max 120 characters (allows for scene + motion hint)
|
# Allow more room for detailed prompts - max 350 characters
|
||||||
prompt = prompt[:120].strip()
|
prompt = prompt[:350].strip()
|
||||||
|
|
||||||
# Clean up trailing commas or incomplete sentences
|
# Clean up trailing punctuation
|
||||||
if prompt.endswith(','):
|
if prompt.endswith(',') or prompt.endswith('.'):
|
||||||
prompt = prompt[:-1].strip()
|
prompt = prompt[:-1].strip()
|
||||||
|
|
||||||
return prompt if len(prompt) >= 15 else None
|
return prompt if len(prompt) >= 15 else None
|
||||||
|
|||||||
@@ -120,6 +120,15 @@ class SIFReleaseReadinessTests(unittest.IsolatedAsyncioTestCase):
|
|||||||
self.assertFalse(validation["is_contextual"])
|
self.assertFalse(validation["is_contextual"])
|
||||||
self.assertEqual(validation["tasks_below_min_evidence"], 1)
|
self.assertEqual(validation["tasks_below_min_evidence"], 1)
|
||||||
|
|
||||||
|
|
||||||
|
def test_demo_release_flag_guards_sensitive_routers(self):
|
||||||
|
source = Path("backend/alwrity_utils/router_manager.py").read_text()
|
||||||
|
self.assertIn("ALWRITY_DEMO_RELEASE", source)
|
||||||
|
self.assertIn("Skipping facebook_writer router in demo-release mode", source)
|
||||||
|
self.assertIn("Skipping linkedin router in demo-release mode", source)
|
||||||
|
self.assertIn("Skipping linkedin_image router in demo-release mode", source)
|
||||||
|
self.assertIn("Skipping persona router in demo-release mode", source)
|
||||||
|
|
||||||
def test_pillar_coverage_guardrail_backfills_missing(self):
|
def test_pillar_coverage_guardrail_backfills_missing(self):
|
||||||
tasks = [{"pillarId": "plan", "title": "Plan", "description": "d", "priority": "high", "estimatedTime": 10, "actionType": "navigate", "enabled": True}]
|
tasks = [{"pillarId": "plan", "title": "Plan", "description": "d", "priority": "high", "estimatedTime": 10, "actionType": "navigate", "enabled": True}]
|
||||||
grounding = {"workflow_config": {"enforce_pillar_coverage": True}}
|
grounding = {"workflow_config": {"enforce_pillar_coverage": True}}
|
||||||
|
|||||||
@@ -7,11 +7,89 @@ Run this from the backend directory to set up and start the FastAPI server.
|
|||||||
|
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
import json
|
||||||
import argparse
|
import argparse
|
||||||
|
import platform
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from dataclasses import dataclass, asdict
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
# Detect platform
|
||||||
|
IS_WINDOWS = platform.system() == "Windows"
|
||||||
|
IS_LINUX = platform.system() == "Linux"
|
||||||
|
|
||||||
|
import uvicorn
|
||||||
|
|
||||||
|
|
||||||
def bootstrap_linguistic_models():
|
@dataclass
|
||||||
|
class BootstrapResult:
|
||||||
|
name: str
|
||||||
|
success: bool
|
||||||
|
skipped: bool
|
||||||
|
reason: Optional[str] = None
|
||||||
|
details: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
LINGUISTIC_REQUIRED_FEATURES = {"content_planning", "strategy_copilot", "facebook", "linkedin", "blog_writer", "persona"}
|
||||||
|
|
||||||
|
|
||||||
|
def get_enabled_features() -> set:
|
||||||
|
"""Get enabled features from ALWRITY_ENABLED_FEATURES env var.
|
||||||
|
|
||||||
|
Values:
|
||||||
|
- "all" - enable all features (default)
|
||||||
|
- comma-separated: "podcast,blog-writer,youtube"
|
||||||
|
- single feature: "podcast"
|
||||||
|
"""
|
||||||
|
env_value = os.getenv("ALWRITY_ENABLED_FEATURES", "all").strip().lower()
|
||||||
|
|
||||||
|
if not env_value or env_value == "all":
|
||||||
|
return {"all"}
|
||||||
|
|
||||||
|
return {f.strip() for f in env_value.split(",") if f.strip()}
|
||||||
|
|
||||||
|
|
||||||
|
def should_bootstrap_linguistic_models() -> bool:
|
||||||
|
"""Decide whether to bootstrap linguistic models based on enabled features."""
|
||||||
|
enabled_features = get_enabled_features()
|
||||||
|
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||||
|
|
||||||
|
if "all" in enabled_features:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# Podcast-only mode doesn't need linguistic models
|
||||||
|
if enabled_features == {"podcast"}:
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Map old profile names to features for backwards compatibility
|
||||||
|
feature_mapping = {
|
||||||
|
"podcast": "podcast",
|
||||||
|
"youtube": "youtube",
|
||||||
|
"planning": "content-planning",
|
||||||
|
"default": "all"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if any linguistic-required feature is enabled
|
||||||
|
linguistic_features = {"content_planning", "facebook", "linkedin", "blog-writer", "persona"}
|
||||||
|
return bool(enabled_features & linguistic_features)
|
||||||
|
|
||||||
|
|
||||||
|
def should_bootstrap_local_llm_models() -> bool:
|
||||||
|
"""Decide whether to bootstrap local LLM models based on enabled features.
|
||||||
|
|
||||||
|
SIF/Story Writer requires local LLM - skip if only podcast is enabled.
|
||||||
|
"""
|
||||||
|
enabled_features = get_enabled_features()
|
||||||
|
|
||||||
|
if "all" in enabled_features:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# SIF/Story Writer requires local LLM - only bootstrap if explicitly needed
|
||||||
|
# Skip for lean deployments (podcast-only, content-planning only, etc.)
|
||||||
|
return False # Default to skip unless "all" is enabled
|
||||||
|
|
||||||
|
|
||||||
|
def bootstrap_linguistic_models() -> BootstrapResult:
|
||||||
"""
|
"""
|
||||||
Bootstrap spaCy and NLTK models BEFORE any imports.
|
Bootstrap spaCy and NLTK models BEFORE any imports.
|
||||||
This prevents import-time failures when EnhancedLinguisticAnalyzer is loaded.
|
This prevents import-time failures when EnhancedLinguisticAnalyzer is loaded.
|
||||||
@@ -22,7 +100,7 @@ def bootstrap_linguistic_models():
|
|||||||
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||||
|
|
||||||
if verbose:
|
if verbose:
|
||||||
print("🔍 Bootstrapping linguistic models...")
|
print("[DEBUG] Bootstrapping linguistic models...")
|
||||||
|
|
||||||
# Check and download spaCy model
|
# Check and download spaCy model
|
||||||
try:
|
try:
|
||||||
@@ -30,7 +108,7 @@ def bootstrap_linguistic_models():
|
|||||||
try:
|
try:
|
||||||
nlp = spacy.load("en_core_web_sm")
|
nlp = spacy.load("en_core_web_sm")
|
||||||
if verbose:
|
if verbose:
|
||||||
print(" ✅ spaCy model 'en_core_web_sm' available")
|
print(" [OK] spaCy model 'en_core_web_sm' available")
|
||||||
except OSError:
|
except OSError:
|
||||||
if verbose:
|
if verbose:
|
||||||
print(" ⚠️ spaCy model 'en_core_web_sm' not found, downloading...")
|
print(" ⚠️ spaCy model 'en_core_web_sm' not found, downloading...")
|
||||||
@@ -39,12 +117,12 @@ def bootstrap_linguistic_models():
|
|||||||
sys.executable, "-m", "spacy", "download", "en_core_web_sm"
|
sys.executable, "-m", "spacy", "download", "en_core_web_sm"
|
||||||
])
|
])
|
||||||
if verbose:
|
if verbose:
|
||||||
print(" ✅ spaCy model downloaded successfully")
|
print(" [OK] spaCy model downloaded successfully")
|
||||||
except subprocess.CalledProcessError as e:
|
except subprocess.CalledProcessError as e:
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ❌ Failed to download spaCy model: {e}")
|
print(f" [FAIL] Failed to download spaCy model: {e}")
|
||||||
print(" Please run: python -m spacy download en_core_web_sm")
|
print(" Please run: python -m spacy download en_core_web_sm")
|
||||||
return False
|
return BootstrapResult(name="linguistic_models", success=False, skipped=False, reason="spacy_download_failed")
|
||||||
except ImportError:
|
except ImportError:
|
||||||
if verbose:
|
if verbose:
|
||||||
print(" ⚠️ spaCy not installed - skipping")
|
print(" ⚠️ spaCy not installed - skipping")
|
||||||
@@ -62,23 +140,22 @@ def bootstrap_linguistic_models():
|
|||||||
try:
|
try:
|
||||||
nltk.data.find(path)
|
nltk.data.find(path)
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ✅ NLTK {data_package} available")
|
print(f" [OK] NLTK {data_package} available")
|
||||||
except LookupError:
|
except LookupError:
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ⚠️ NLTK {data_package} not found, downloading...")
|
print(f" ⚠️ NLTK {data_package} not found, downloading...")
|
||||||
try:
|
try:
|
||||||
nltk.download(data_package, quiet=True)
|
nltk.download(data_package, quiet=True)
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ✅ NLTK {data_package} downloaded")
|
print(f" [OK] NLTK {data_package} downloaded")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ⚠️ Failed to download {data_package}: {e}")
|
print(f" ⚠️ Failed to download {data_package}: {e}")
|
||||||
# Try fallback
|
|
||||||
if data_package == 'punkt_tab':
|
if data_package == 'punkt_tab':
|
||||||
try:
|
try:
|
||||||
nltk.download('punkt', quiet=True)
|
nltk.download('punkt', quiet=True)
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ✅ NLTK punkt (fallback) downloaded")
|
print(f" [OK] NLTK punkt (fallback) downloaded")
|
||||||
except:
|
except:
|
||||||
pass
|
pass
|
||||||
except ImportError:
|
except ImportError:
|
||||||
@@ -86,11 +163,11 @@ def bootstrap_linguistic_models():
|
|||||||
print(" ⚠️ NLTK not installed - skipping")
|
print(" ⚠️ NLTK not installed - skipping")
|
||||||
|
|
||||||
if verbose:
|
if verbose:
|
||||||
print("✅ Linguistic model bootstrap complete")
|
print("[OK] Linguistic model bootstrap complete")
|
||||||
return True
|
return BootstrapResult(name="linguistic_models", success=True, skipped=False)
|
||||||
|
|
||||||
|
|
||||||
def bootstrap_local_llm_models():
|
def bootstrap_local_llm_models() -> BootstrapResult:
|
||||||
"""
|
"""
|
||||||
Bootstrap Local LLM models (Qwen) for SIF Agents.
|
Bootstrap Local LLM models (Qwen) for SIF Agents.
|
||||||
This ensures the model is cached locally before the server starts,
|
This ensures the model is cached locally before the server starts,
|
||||||
@@ -117,7 +194,7 @@ def bootstrap_local_llm_models():
|
|||||||
if os.getenv("RENDER") or os.getenv("RAILWAY_ENVIRONMENT"):
|
if os.getenv("RENDER") or os.getenv("RAILWAY_ENVIRONMENT"):
|
||||||
if verbose:
|
if verbose:
|
||||||
print(" ⚠️ Cloud environment detected (Render/Railway). Skipping local LLM bootstrap to save RAM/Time.")
|
print(" ⚠️ Cloud environment detected (Render/Railway). Skipping local LLM bootstrap to save RAM/Time.")
|
||||||
return True
|
return BootstrapResult(name="local_llm_models", success=True, skipped=True, reason="cloud_environment")
|
||||||
|
|
||||||
target_model = "Qwen/Qwen2.5-3B-Instruct"
|
target_model = "Qwen/Qwen2.5-3B-Instruct"
|
||||||
|
|
||||||
@@ -130,23 +207,73 @@ def bootstrap_local_llm_models():
|
|||||||
# This checks cache and downloads if missing
|
# This checks cache and downloads if missing
|
||||||
snapshot_download(repo_id=target_model, repo_type="model")
|
snapshot_download(repo_id=target_model, repo_type="model")
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ✅ Local LLM '{target_model}' available")
|
print(f" [OK] Local LLM '{target_model}' available")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" ⚠️ Failed to download/check local LLM: {e}")
|
print(f" ⚠️ Failed to download/check local LLM: {e}")
|
||||||
print(" SIF agents may try to download it at runtime.")
|
print(" SIF agents may try to download it at runtime.")
|
||||||
return False
|
return BootstrapResult(name="local_llm_models", success=False, skipped=False, reason=str(e))
|
||||||
except ImportError:
|
except ImportError:
|
||||||
if verbose:
|
if verbose:
|
||||||
print(" ⚠️ huggingface_hub not installed - skipping LLM bootstrap")
|
print(" ⚠️ huggingface_hub not installed - skipping LLM bootstrap")
|
||||||
|
return BootstrapResult(name="local_llm_models", success=False, skipped=True, reason="huggingface_hub_not_installed")
|
||||||
|
|
||||||
return True
|
return BootstrapResult(name="local_llm_models", success=True, skipped=False)
|
||||||
|
|
||||||
|
|
||||||
# Bootstrap linguistic models BEFORE any imports that might need them
|
# Bootstrap linguistic models BEFORE any imports that might need them
|
||||||
|
BOOTSTRAP_RESULTS = []
|
||||||
|
|
||||||
|
# Load .env file early so ALWRITY_ENABLED_FEATURES is available
|
||||||
|
from dotenv import load_dotenv
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Load from backend/.env specifically
|
||||||
|
backend_dir = Path(__file__).parent
|
||||||
|
load_dotenv(backend_dir / '.env')
|
||||||
|
|
||||||
|
# Debug: Print what PORT is set to - IMMEDIATELY at startup
|
||||||
|
import os
|
||||||
|
print(f"[STARTUP] PORT env: {os.getenv('PORT')}", flush=True)
|
||||||
|
print(f"[STARTUP] RENDER env: {os.getenv('RENDER')}", flush=True)
|
||||||
|
print(f"[STARTUP] ALWRITY_ENABLED_FEATURES: {os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||||
|
print(f"[STARTUP] HOST env: {os.getenv('HOST')}", flush=True)
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
bootstrap_linguistic_models()
|
enabled_features = get_enabled_features()
|
||||||
bootstrap_local_llm_models()
|
features_str = ",".join(sorted(enabled_features))
|
||||||
|
os.environ["ALWRITY_ENABLED_FEATURES"] = features_str
|
||||||
|
|
||||||
|
print(f"\n[OK] Enabled features: {features_str}")
|
||||||
|
|
||||||
|
if should_bootstrap_linguistic_models():
|
||||||
|
result = bootstrap_linguistic_models()
|
||||||
|
BOOTSTRAP_RESULTS.append(result)
|
||||||
|
else:
|
||||||
|
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||||
|
if verbose:
|
||||||
|
print("[SKIP] Skipping linguistic model bootstrap (profile-gated)")
|
||||||
|
BOOTSTRAP_RESULTS.append(BootstrapResult(name="linguistic_models", success=True, skipped=True, reason="profile_gated"))
|
||||||
|
|
||||||
|
if should_bootstrap_local_llm_models():
|
||||||
|
result = bootstrap_local_llm_models()
|
||||||
|
BOOTSTRAP_RESULTS.append(result)
|
||||||
|
else:
|
||||||
|
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||||
|
if verbose:
|
||||||
|
print("[SKIP] Skipping local LLM model bootstrap (feature-gated)")
|
||||||
|
BOOTSTRAP_RESULTS.append(BootstrapResult(name="local_llm_models", success=True, skipped=True, reason="feature_gated"))
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"enabled_features": features_str,
|
||||||
|
"bootstraps": [asdict(r) for r in BOOTSTRAP_RESULTS]
|
||||||
|
}
|
||||||
|
os.environ["ALWRITY_BOOTSTRAP_SUMMARY"] = json.dumps(summary)
|
||||||
|
|
||||||
|
print(f"\n[INFO] Bootstrap Summary:")
|
||||||
|
for r in BOOTSTRAP_RESULTS:
|
||||||
|
status = "[SKIP] Skipped" if r.skipped else ("[OK] Enabled" if r.success else "[FAIL] Failed")
|
||||||
|
print(f" {r.name}: {status}" + (f" ({r.reason})" if r.reason else ""))
|
||||||
|
|
||||||
# NOW import modular utilities (after bootstrap)
|
# NOW import modular utilities (after bootstrap)
|
||||||
from alwrity_utils import (
|
from alwrity_utils import (
|
||||||
@@ -159,16 +286,24 @@ from alwrity_utils import (
|
|||||||
|
|
||||||
def start_backend(enable_reload=False, production_mode=False):
|
def start_backend(enable_reload=False, production_mode=False):
|
||||||
"""Start the backend server."""
|
"""Start the backend server."""
|
||||||
print("🚀 Starting ALwrity Backend...")
|
print("==> Starting ALwrity Backend...")
|
||||||
|
podcast_only_demo_mode = os.getenv("ALWRITY_PODCAST_ONLY_DEMO_MODE", os.getenv("PODCAST_ONLY_DEMO_MODE", "false")).lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
if podcast_only_demo_mode:
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("==> PODCAST-ONLY DEMO MODE ACTIVE")
|
||||||
|
print(" Non-podcast router groups are intentionally skipped.")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
# Set host based on environment and mode
|
# Set host based on environment and mode
|
||||||
# Use 127.0.0.1 for local production testing on Windows
|
# Use 127.0.0.1 for local production testing on Windows
|
||||||
# Use 0.0.0.0 for actual cloud deployments (Render, Railway, etc.)
|
# Use 0.0.0.0 for actual cloud deployments (Render, Railway, etc.)
|
||||||
# Render provides PORT env var, we must bind to it.
|
# Render provides PORT env var, detect cloud by presence of PORT
|
||||||
default_host = os.getenv("RENDER") or os.getenv("RAILWAY_ENVIRONMENT") or os.getenv("DEPLOY_ENV")
|
render_port = os.getenv("PORT")
|
||||||
if default_host:
|
if render_port:
|
||||||
# Cloud deployment detected - use 0.0.0.0
|
# Cloud deployment detected (Render sets PORT env var) - use 0.0.0.0
|
||||||
os.environ.setdefault("HOST", "0.0.0.0")
|
os.environ.setdefault("HOST", "0.0.0.0")
|
||||||
|
os.environ.setdefault("PORT", render_port)
|
||||||
else:
|
else:
|
||||||
# Local deployment - use 127.0.0.1 for better Windows compatibility
|
# Local deployment - use 127.0.0.1 for better Windows compatibility
|
||||||
os.environ.setdefault("HOST", "127.0.0.1")
|
os.environ.setdefault("HOST", "127.0.0.1")
|
||||||
@@ -180,40 +315,46 @@ def start_backend(enable_reload=False, production_mode=False):
|
|||||||
# Set reload based on argument or environment variable
|
# Set reload based on argument or environment variable
|
||||||
if enable_reload and not production_mode:
|
if enable_reload and not production_mode:
|
||||||
os.environ.setdefault("RELOAD", "true")
|
os.environ.setdefault("RELOAD", "true")
|
||||||
print(" 🔄 Development mode: Auto-reload enabled")
|
print(" [DEV] Development mode: Auto-reload enabled")
|
||||||
else:
|
else:
|
||||||
os.environ.setdefault("RELOAD", "false")
|
os.environ.setdefault("RELOAD", "false")
|
||||||
print(" 🏭 Production mode: Auto-reload disabled")
|
print(" [PROD] Production mode: Auto-reload disabled")
|
||||||
|
|
||||||
host = os.getenv("HOST")
|
host = os.getenv("HOST", "0.0.0.0")
|
||||||
port = int(os.getenv("PORT", "8000"))
|
port = int(os.getenv("PORT", "8000"))
|
||||||
reload = os.getenv("RELOAD", "false").lower() == "true"
|
reload = os.environ.get("RELOAD", "false").lower() == "true"
|
||||||
|
print(f"[DEBUG] Bind prepared - host={host}, port={port}, reload={reload}", flush=True)
|
||||||
|
print(f"[DEBUG] ENV check - ALWRITY_ENABLED_FEATURES={os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||||
|
|
||||||
print(f" 📍 Host: {host}")
|
print(f" ==> Host: {host}", flush=True)
|
||||||
print(f" 🔌 Port: {port}")
|
print(f" ==> Port: {port}", flush=True)
|
||||||
print(f" 🔄 Reload: {reload}")
|
print(f" [DEV] Reload: {reload}", flush=True)
|
||||||
print(f" 🔄 Reload: {reload}")
|
print(f"[DEBUG] About to import app module...", flush=True)
|
||||||
|
print("[DEBUG] >>> START APP IMPORT <<<", flush=True)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Import and run the app
|
# Import and run the app
|
||||||
from app import app
|
from app import app
|
||||||
|
print("[DEBUG] >>> END APP IMPORT <<<", flush=True)
|
||||||
|
|
||||||
import uvicorn
|
import uvicorn
|
||||||
|
print(f"[DEBUG] Imported app and uvicorn successfully", flush=True)
|
||||||
|
|
||||||
# Note: Database already initialized by DatabaseSetup in main()
|
# Note: Database already initialized by DatabaseSetup in main()
|
||||||
|
|
||||||
print("\n🌐 ALwrity Backend Server")
|
print("\n[WORLD] ALwrity Backend Server", flush=True)
|
||||||
print("=" * 50)
|
print("=" * 50, flush=True)
|
||||||
print(" 📖 API Documentation: http://localhost:8000/api/docs")
|
print(f" 📖 API Documentation: http://localhost:{os.getenv('PORT', '8000')}/api/docs", flush=True)
|
||||||
print(" 🔍 Health Check: http://localhost:8000/health")
|
print(f" 🔍 Health Check: http://localhost:{os.getenv('PORT', '8000')}/health", flush=True)
|
||||||
print(" 📊 ReDoc: http://localhost:8000/api/redoc")
|
print(f" 📊 ReDoc: http://localhost:{os.getenv('PORT', '8000')}/api/redoc", flush=True)
|
||||||
|
|
||||||
if not production_mode:
|
if not production_mode:
|
||||||
print(" 📈 API Monitoring: http://localhost:8000/api/content-planning/monitoring/health")
|
print(f" 📈 API Monitoring: http://localhost:{os.getenv('PORT', '8000')}/api/content-planning/monitoring/health", flush=True)
|
||||||
print(" 💳 Billing Dashboard: http://localhost:8000/api/subscription/plans")
|
print(f" 💳 Billing Dashboard: http://localhost:{os.getenv('PORT', '8000')}/api/subscription/plans", flush=True)
|
||||||
print(" 📊 Usage Tracking: http://localhost:8000/api/subscription/usage/demo")
|
print(f" 📊 Usage Tracking: http://localhost:{os.getenv('PORT', '8000')}/api/subscription/usage/demo", flush=True)
|
||||||
|
|
||||||
print("\n[STOP] Press Ctrl+C to stop the server")
|
print("\n[STOP] Press Ctrl+C to stop the server", flush=True)
|
||||||
print("=" * 50)
|
print("=" * 50, flush=True)
|
||||||
|
|
||||||
# Set up clean logging for end users
|
# Set up clean logging for end users
|
||||||
from logging_config import setup_clean_logging, get_uvicorn_log_level
|
from logging_config import setup_clean_logging, get_uvicorn_log_level
|
||||||
@@ -241,6 +382,26 @@ def start_backend(enable_reload=False, production_mode=False):
|
|||||||
print(f"[ERROR] Video stack preflight failed: {_video_stack_err}")
|
print(f"[ERROR] Video stack preflight failed: {_video_stack_err}")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
print(f"[DEBUG] Starting uvicorn with host={host} port={port}", flush=True)
|
||||||
|
print("[DEBUG] >>> ABOUT TO CALL UVICORN.RUN() <<<", flush=True)
|
||||||
|
|
||||||
|
# Skip video preflight in podcast-only mode to save memory/time
|
||||||
|
is_podcast = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||||
|
print(f"[DEBUG] Podcast mode check: {is_podcast}", flush=True)
|
||||||
|
|
||||||
|
if is_podcast:
|
||||||
|
print("[DEBUG] Podcast mode - skipping video preflight", flush=True)
|
||||||
|
else:
|
||||||
|
# Log diagnostics and assert versions (fail fast if misconfigured)
|
||||||
|
try:
|
||||||
|
if log_video_stack_diagnostics:
|
||||||
|
log_video_stack_diagnostics()
|
||||||
|
if assert_supported_moviepy:
|
||||||
|
assert_supported_moviepy()
|
||||||
|
except Exception as _video_stack_err:
|
||||||
|
print(f"[ERROR] Video stack preflight failed: {_video_stack_err}")
|
||||||
|
return False
|
||||||
|
|
||||||
uvicorn.run(
|
uvicorn.run(
|
||||||
"app:app",
|
"app:app",
|
||||||
host=host,
|
host=host,
|
||||||
@@ -280,11 +441,14 @@ def start_backend(enable_reload=False, production_mode=False):
|
|||||||
],
|
],
|
||||||
log_level=uvicorn_log_level
|
log_level=uvicorn_log_level
|
||||||
)
|
)
|
||||||
|
print("[DEBUG] uvicorn.run() has finished", flush=True)
|
||||||
|
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
print("\n\n🛑 Backend stopped by user")
|
print("\n\n🛑 Backend stopped by user")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f"\n[ERROR] Error starting backend: {e}")
|
print(f"\n[ERROR] Error starting backend: {e}", flush=True)
|
||||||
|
import traceback
|
||||||
|
traceback.print_exc()
|
||||||
return False
|
return False
|
||||||
|
|
||||||
return True
|
return True
|
||||||
@@ -337,12 +501,12 @@ def main():
|
|||||||
"Starting server"
|
"Starting server"
|
||||||
]
|
]
|
||||||
|
|
||||||
print("🔧 Initializing ALwrity...")
|
print("==> Initializing ALwrity...")
|
||||||
|
|
||||||
# Apply production optimizations if needed
|
# Apply production optimizations if needed
|
||||||
if production_mode:
|
if production_mode:
|
||||||
if not production_optimizer.apply_production_optimizations():
|
if not production_optimizer.apply_production_optimizations():
|
||||||
print("❌ Production optimization failed")
|
print("[FAIL] Production optimization failed")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
# Step 1: Dependencies
|
# Step 1: Dependencies
|
||||||
@@ -351,11 +515,11 @@ def main():
|
|||||||
if not critical_ok:
|
if not critical_ok:
|
||||||
print("installing...", end=" ", flush=True)
|
print("installing...", end=" ", flush=True)
|
||||||
if not dependency_manager.install_requirements():
|
if not dependency_manager.install_requirements():
|
||||||
print("❌ Failed")
|
print("[FAIL] Failed")
|
||||||
return False
|
return False
|
||||||
print("✅ Done")
|
print("[OK] Done")
|
||||||
else:
|
else:
|
||||||
print("✅ Done")
|
print("[OK] Done")
|
||||||
|
|
||||||
# Check optional dependencies (non-critical) - only in verbose mode
|
# Check optional dependencies (non-critical) - only in verbose mode
|
||||||
if verbose_mode:
|
if verbose_mode:
|
||||||
@@ -364,24 +528,24 @@ def main():
|
|||||||
# Step 2: Environment
|
# Step 2: Environment
|
||||||
print(f" 🔧 {setup_steps[1]}...", end=" ", flush=True)
|
print(f" 🔧 {setup_steps[1]}...", end=" ", flush=True)
|
||||||
if not environment_setup.setup_directories():
|
if not environment_setup.setup_directories():
|
||||||
print("❌ Directory setup failed")
|
print("[FAIL] Directory setup failed")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
if not environment_setup.setup_environment_variables():
|
if not environment_setup.setup_environment_variables():
|
||||||
print("❌ Environment setup failed")
|
print("[FAIL] Environment setup failed")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
# Create .env file only in development
|
# Create .env file only in development
|
||||||
if not production_mode:
|
if not production_mode:
|
||||||
environment_setup.create_env_file()
|
environment_setup.create_env_file()
|
||||||
print("✅ Done")
|
print("[OK] Done")
|
||||||
|
|
||||||
# Step 3: Database
|
# Step 3: Database
|
||||||
print(f" 📊 {setup_steps[2]}...", end=" ", flush=True)
|
print(f" 📊 {setup_steps[2]}...", end=" ", flush=True)
|
||||||
if not database_setup.setup_essential_tables():
|
if not database_setup.setup_essential_tables():
|
||||||
print("⚠️ Issues detected, continuing...")
|
print("⚠️ Issues detected, continuing...")
|
||||||
else:
|
else:
|
||||||
print("✅ Done")
|
print("[OK] Done")
|
||||||
|
|
||||||
# Setup advanced features in development, verify in all modes
|
# Setup advanced features in development, verify in all modes
|
||||||
if not production_mode:
|
if not production_mode:
|
||||||
@@ -401,4 +565,4 @@ def main():
|
|||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
success = main()
|
success = main()
|
||||||
if not success:
|
if not success:
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|||||||
156
backend/tests/test_agent_context_vfs.py
Normal file
156
backend/tests/test_agent_context_vfs.py
Normal file
@@ -0,0 +1,156 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Lightweight fallback for environments missing loguru.
|
||||||
|
if "loguru" not in sys.modules:
|
||||||
|
stub = types.ModuleType("loguru")
|
||||||
|
stub.logger = types.SimpleNamespace(
|
||||||
|
info=lambda *a, **k: None,
|
||||||
|
warning=lambda *a, **k: None,
|
||||||
|
error=lambda *a, **k: None,
|
||||||
|
debug=lambda *a, **k: None,
|
||||||
|
)
|
||||||
|
sys.modules["loguru"] = stub
|
||||||
|
|
||||||
|
def _load_module(name: str, rel_path: str):
|
||||||
|
base = Path(__file__).resolve().parents[1]
|
||||||
|
path = base / rel_path
|
||||||
|
spec = importlib.util.spec_from_file_location(name, path)
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
flat_mod = _load_module("agent_flat_context_under_test", "services/intelligence/agent_flat_context.py")
|
||||||
|
sys.modules.setdefault("services.intelligence.agent_flat_context", flat_mod)
|
||||||
|
vfs_mod = _load_module("agent_context_vfs_under_test", "services/intelligence/agent_context_vfs.py")
|
||||||
|
|
||||||
|
AgentFlatContextStore = flat_mod.AgentFlatContextStore
|
||||||
|
AgentContextVFS = vfs_mod.AgentContextVFS
|
||||||
|
|
||||||
|
|
||||||
|
def _cleanup_workspace(user_id: str, project_id: str | None = None) -> None:
|
||||||
|
safe_user = ''.join(c for c in str(user_id) if c.isalnum() or c in ('-', '_')) or 'unknown_user'
|
||||||
|
root = Path(__file__).resolve().parents[2] / 'workspace'
|
||||||
|
user_dir = root / f'workspace_{safe_user}'
|
||||||
|
if user_dir.exists():
|
||||||
|
import shutil
|
||||||
|
shutil.rmtree(user_dir, ignore_errors=True)
|
||||||
|
|
||||||
|
if project_id:
|
||||||
|
safe_project = ''.join(c for c in str(project_id) if c.isalnum() or c in ('-', '_')) or 'default_project'
|
||||||
|
project_dir = root / f'project_{safe_project}'
|
||||||
|
if project_dir.exists():
|
||||||
|
import shutil
|
||||||
|
shutil.rmtree(project_dir, ignore_errors=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_context_query_variants_and_can_answer():
|
||||||
|
user_id = 'pytest_vfs_user'
|
||||||
|
_cleanup_workspace(user_id)
|
||||||
|
|
||||||
|
store = AgentFlatContextStore(user_id)
|
||||||
|
payload = {
|
||||||
|
'website_url': 'https://example.com',
|
||||||
|
'brand_analysis': {'brand_voice': 'Authoritative'},
|
||||||
|
'recommended_settings': {'writing_tone': 'Conversational'},
|
||||||
|
'content_type': {'primary_type': 'Blog'},
|
||||||
|
'target_audience': {'primary_audience': 'Founders'},
|
||||||
|
}
|
||||||
|
assert store.save_step2_website_analysis(payload)
|
||||||
|
|
||||||
|
vfs = AgentContextVFS(user_id)
|
||||||
|
result = vfs.search_context('tone')
|
||||||
|
|
||||||
|
assert result['query'] == 'tone'
|
||||||
|
assert 'attempted_queries' in result
|
||||||
|
assert result['attempted_queries'][0] == 'tone'
|
||||||
|
assert result['can_answer'] is True
|
||||||
|
assert len(result['results']) >= 1
|
||||||
|
assert 'triage_top5' in result
|
||||||
|
assert len(result['triage_top5']) >= 1
|
||||||
|
assert 'low_probability' in result['results'][0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_inspect_file_large_document_summary_plus_keys():
|
||||||
|
user_id = 'pytest_vfs_large'
|
||||||
|
_cleanup_workspace(user_id)
|
||||||
|
|
||||||
|
store = AgentFlatContextStore(user_id)
|
||||||
|
large_blob = 'x' * 9000
|
||||||
|
payload = {
|
||||||
|
'website_url': 'https://big.example.com',
|
||||||
|
'brand_analysis': {'brand_voice': 'Bold'},
|
||||||
|
'recommended_settings': {'writing_tone': 'Direct'},
|
||||||
|
'target_audience': {'primary_audience': 'Teams'},
|
||||||
|
'crawl_result': {'raw': large_blob},
|
||||||
|
}
|
||||||
|
assert store.save_step2_website_analysis(payload)
|
||||||
|
|
||||||
|
vfs = AgentContextVFS(user_id)
|
||||||
|
out = vfs.inspect_file('step2_website_analysis.json')
|
||||||
|
|
||||||
|
assert out['mode'] == 'summary_plus_keys'
|
||||||
|
assert 'agent_summary' in out
|
||||||
|
assert 'keys' in out
|
||||||
|
assert 'crawl_result' in out['keys']
|
||||||
|
|
||||||
|
|
||||||
|
def test_write_shared_note_and_activity_log_created():
|
||||||
|
user_id = 'pytest_collab_user'
|
||||||
|
project_id = 'proj_abc'
|
||||||
|
_cleanup_workspace(user_id, project_id)
|
||||||
|
|
||||||
|
vfs = AgentContextVFS(user_id, project_id=project_id)
|
||||||
|
write_res = vfs.write_shared_note('Draft collaboration note', agent_id='agent_one')
|
||||||
|
|
||||||
|
assert write_res['ok'] is True
|
||||||
|
assert write_res['file'] == 'collaboration.md'
|
||||||
|
|
||||||
|
collab = vfs.list_context()['collaboration']
|
||||||
|
scratchpad = Path(collab['scratchpad_dir'])
|
||||||
|
note_file = scratchpad / 'collaboration.md'
|
||||||
|
log_file = scratchpad / 'activity_log.jsonl'
|
||||||
|
|
||||||
|
assert note_file.exists()
|
||||||
|
assert log_file.exists()
|
||||||
|
|
||||||
|
content = note_file.read_text(encoding='utf-8')
|
||||||
|
assert 'agent_one' in content
|
||||||
|
assert 'Draft collaboration note' in content
|
||||||
|
|
||||||
|
lines = [json.loads(l) for l in log_file.read_text(encoding='utf-8').splitlines() if l.strip()]
|
||||||
|
assert any(entry.get('event_type') == 'shared_note_written' for entry in lines)
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_struct_path_resolution_and_dependency_context():
|
||||||
|
user_id = 'pytest_struct_user'
|
||||||
|
_cleanup_workspace(user_id)
|
||||||
|
|
||||||
|
store = AgentFlatContextStore(user_id)
|
||||||
|
assert store.save_step2_website_analysis(
|
||||||
|
{
|
||||||
|
'website_url': 'https://struct.example.com',
|
||||||
|
'brand_analysis': {'brand_voice': 'Pragmatic'},
|
||||||
|
'recommended_settings': {'writing_tone': 'Clear'},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
assert store.save_step4_persona_data(
|
||||||
|
{
|
||||||
|
'core_persona': {'name': 'Ops Leader', 'goal': 'Scale ops'},
|
||||||
|
'selected_platforms': ['linkedin'],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
vfs = AgentContextVFS(user_id)
|
||||||
|
out = vfs.read_struct('step4_persona_data.json', 'data.core_persona.name')
|
||||||
|
|
||||||
|
assert out['ok'] is True
|
||||||
|
assert out['data'] == 'Ops Leader'
|
||||||
|
assert out['dependency_context']['brand_voice'] == 'Pragmatic'
|
||||||
93
docs-site/docs/features/podcast-maker/api-reference.md
Normal file
93
docs-site/docs/features/podcast-maker/api-reference.md
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
# Podcast Maker API Reference
|
||||||
|
|
||||||
|
Base prefix: `/api/podcast`
|
||||||
|
|
||||||
|
This page summarizes the Podcast Maker endpoints currently represented in frontend and backend code.
|
||||||
|
|
||||||
|
## Endpoints by workflow stage
|
||||||
|
|
||||||
|
### Analysis and idea shaping
|
||||||
|
|
||||||
|
- `POST /idea/enhance`
|
||||||
|
- `POST /analyze`
|
||||||
|
- `POST /regenerate-queries`
|
||||||
|
|
||||||
|
### Research
|
||||||
|
|
||||||
|
- `POST /research/exa`
|
||||||
|
|
||||||
|
### Scripting
|
||||||
|
|
||||||
|
- `POST /script`
|
||||||
|
- `POST /script/approve`
|
||||||
|
|
||||||
|
### Audio
|
||||||
|
|
||||||
|
- `POST /audio/upload`
|
||||||
|
- `POST /audio`
|
||||||
|
- `POST /combine-audio`
|
||||||
|
- `GET /audio/{filename}`
|
||||||
|
|
||||||
|
### Images
|
||||||
|
|
||||||
|
- `POST /image`
|
||||||
|
- `GET /images/{path}`
|
||||||
|
|
||||||
|
### Video
|
||||||
|
|
||||||
|
- `POST /render/video`
|
||||||
|
- `POST /render/combine-videos`
|
||||||
|
- `GET /videos`
|
||||||
|
- `GET /videos/{filename}`
|
||||||
|
- `GET /final-videos/{filename}`
|
||||||
|
|
||||||
|
### Avatars
|
||||||
|
|
||||||
|
- `POST /avatar/upload`
|
||||||
|
- `POST /avatar/make-presentable`
|
||||||
|
- `POST /avatar/generate`
|
||||||
|
|
||||||
|
### Projects
|
||||||
|
|
||||||
|
- `POST /projects`
|
||||||
|
- `GET /projects`
|
||||||
|
- `GET /projects/{project_id}`
|
||||||
|
- `PUT /projects/{project_id}`
|
||||||
|
- `DELETE /projects/{project_id}`
|
||||||
|
- `POST /projects/{project_id}/favorite`
|
||||||
|
|
||||||
|
### Dubbing (backend available)
|
||||||
|
|
||||||
|
- `POST /dub/audio`
|
||||||
|
- `GET /dub/{task_id}/result`
|
||||||
|
- `GET /dub/audio/{filename}`
|
||||||
|
- `POST /dub/estimate`
|
||||||
|
- `GET /dub/languages`
|
||||||
|
- `GET /dub/voices`
|
||||||
|
- `POST /dub/voices/clone`
|
||||||
|
- `GET /dub/voices/{task_id}/result`
|
||||||
|
- `GET /dub/voices/audio/{filename}`
|
||||||
|
|
||||||
|
## Implementation details
|
||||||
|
|
||||||
|
### Endpoint usage in frontend service
|
||||||
|
|
||||||
|
The current `podcastApi.ts` directly calls these podcast routes for analysis, research, script, audio, image, video, avatar, and project workflows.
|
||||||
|
|
||||||
|
Known gap:
|
||||||
|
|
||||||
|
- `cancelTask()` is a placeholder that posts to `/api/story/task/{taskId}/cancel` rather than a dedicated podcast route.
|
||||||
|
|
||||||
|
### Request/response model notes
|
||||||
|
|
||||||
|
At a high level:
|
||||||
|
|
||||||
|
- Script endpoints exchange `idea`, `duration_minutes`, `speakers`, and optional `research`/`analysis`/`bible` context.
|
||||||
|
- Audio endpoints exchange scene identifiers, text, and voice/rendering options.
|
||||||
|
- Video endpoints exchange scene identifiers plus `audio_url` and optional image/prompt context.
|
||||||
|
- Project endpoints exchange project-level state payloads suitable for restoring workflow progress.
|
||||||
|
|
||||||
|
## Engineering references
|
||||||
|
|
||||||
|
- `docs/Podcast_maker/AI_PODCAST_BACKEND_REFERENCE.md`
|
||||||
|
- `docs/Podcast_maker/PODCAST_PERSISTENCE_IMPLEMENTATION.md`
|
||||||
159
docs-site/docs/features/podcast-maker/best-practices.md
Normal file
159
docs-site/docs/features/podcast-maker/best-practices.md
Normal file
@@ -0,0 +1,159 @@
|
|||||||
|
# Podcast Maker Best Practices
|
||||||
|
|
||||||
|
This guide is implementation-aware: every recommendation below is based on how the current Podcast Maker APIs actually behave in frontend and backend code.
|
||||||
|
|
||||||
|
## 1) Start with budget-safe defaults (preflight-first workflow)
|
||||||
|
|
||||||
|
Podcast Maker runs **preflight validation** before major steps (analysis, research, script generation, TTS preview, and full TTS render). Use that as your workflow guardrail:
|
||||||
|
|
||||||
|
1. Analyze idea first
|
||||||
|
2. Approve a small set of research queries
|
||||||
|
3. Generate script
|
||||||
|
4. Preview voice on short excerpts
|
||||||
|
5. Render full scene audio
|
||||||
|
6. Generate scene videos
|
||||||
|
7. Combine final assets
|
||||||
|
|
||||||
|
Why this matters:
|
||||||
|
- If credits/limits are insufficient, preflight fails fast before expensive operations.
|
||||||
|
- Video generation also runs server-side animation validation and returns subscription-friendly errors for insufficient credits.
|
||||||
|
|
||||||
|
## 2) Duration vs. scene-count tradeoffs (cost + reliability)
|
||||||
|
|
||||||
|
The stack defaults to a **45s scene target** and cost estimate logic effectively scales scene count as:
|
||||||
|
|
||||||
|
- `scene_count ≈ ceil(duration_minutes * 60 / scene_length_target_seconds)`
|
||||||
|
|
||||||
|
Practical recommendations:
|
||||||
|
- **5–8 min episodes**: target 5–8 scenes.
|
||||||
|
- **10–15 min episodes**: target 8–14 scenes.
|
||||||
|
- Increase `scene_length_target` when you need fewer API calls and faster completion.
|
||||||
|
- Keep script concise because per-scene TTS has a **10,000-character max** (long text gets truncated by frontend before render).
|
||||||
|
|
||||||
|
Rule of thumb:
|
||||||
|
- More scenes = better pacing granularity but more TTS/video calls.
|
||||||
|
- Fewer scenes = cheaper/faster pipeline, but each scene must carry more narrative weight.
|
||||||
|
|
||||||
|
## 3) Voice strategy: preview first, render second
|
||||||
|
|
||||||
|
Use a two-pass voice workflow:
|
||||||
|
|
||||||
|
### Pass A: Preview and lock voice profile
|
||||||
|
Use preview on short, representative lines (intro, data-heavy line, CTA) to validate:
|
||||||
|
- voice identity
|
||||||
|
- speed
|
||||||
|
- emotion
|
||||||
|
- pronunciation behavior (especially numbers/statistics)
|
||||||
|
|
||||||
|
### Pass B: Full scene render with tuned knobs
|
||||||
|
When rendering scene audio, adjust only the knobs that matter:
|
||||||
|
- `voice_id` (or `custom_voice_id` for cloned voice)
|
||||||
|
- `speed` (default 1.0 is usually safest for timing)
|
||||||
|
- `emotion` (scene-level emotion is supported)
|
||||||
|
- `english_normalization` (keep enabled for number-heavy scripts)
|
||||||
|
- audio format controls (`sample_rate`, `bitrate`, `channel`, `format`, `language_boost`) only when distribution requires them
|
||||||
|
|
||||||
|
Also note:
|
||||||
|
- The frontend injects pause markers and strips markdown before TTS for better natural rhythm.
|
||||||
|
- Use short lines (2–4 per scene is a good operational target from script generation guidance).
|
||||||
|
|
||||||
|
## 4) Research quality: when to use Exa config options
|
||||||
|
|
||||||
|
Use Exa config knobs intentionally, not by default.
|
||||||
|
|
||||||
|
### Search type
|
||||||
|
- `auto`: default for most projects.
|
||||||
|
- `keyword`: use when topic vocabulary is stable/specific.
|
||||||
|
- `neural`: use when you need semantic discovery across mixed phrasing.
|
||||||
|
|
||||||
|
### Domain filters
|
||||||
|
Use either include or exclude domains (not both).
|
||||||
|
- Prefer `exa_include_domains` for compliance/brand-safe sourcing.
|
||||||
|
- Use `exa_exclude_domains` to remove noisy/untrusted sources.
|
||||||
|
|
||||||
|
If both are sent, the backend/frontend sanitize behavior will prefer include-domain intent and drop the conflicting side.
|
||||||
|
|
||||||
|
### `max_sources`, category, and freshness
|
||||||
|
- Increase `max_sources` only when synthesis quality is poor at default depth.
|
||||||
|
- Use `date_range` (e.g. last month/quarter/year) for trend-sensitive topics.
|
||||||
|
- Turn on statistics-oriented options when the episode needs hard numbers.
|
||||||
|
|
||||||
|
### Query operations
|
||||||
|
- Always approve only the strongest queries before running research.
|
||||||
|
- Empty query sets are rejected server-side.
|
||||||
|
|
||||||
|
## 5) Avatar + image prompt strategy for visual consistency
|
||||||
|
|
||||||
|
Consistency is strongest when you anchor scene images to a persistent base avatar.
|
||||||
|
|
||||||
|
Recommended approach:
|
||||||
|
1. Create/upload a presenter avatar once per project.
|
||||||
|
2. Reuse that avatar as `base_avatar_url` for scene images.
|
||||||
|
3. Keep one shared style nucleus across prompts (lighting, environment, host look, framing).
|
||||||
|
4. Change only scene-specific context (topic, emotion, supporting visual motif).
|
||||||
|
|
||||||
|
Important implementation notes:
|
||||||
|
- If `base_avatar_url` is provided, image generation uses character-consistency flow; if the base avatar cannot be loaded, image generation fails (no silent fallback).
|
||||||
|
- Keep scene emotion aligned to visual lighting cues for continuity.
|
||||||
|
- For presenter generation, keep speakers realistic (supported range is 1–2).
|
||||||
|
|
||||||
|
## 6) Script and scene structure that survives production
|
||||||
|
|
||||||
|
Generate script with full context:
|
||||||
|
- analysis (audience/type/keywords)
|
||||||
|
- selected outline
|
||||||
|
- research payload
|
||||||
|
- bible/persona context
|
||||||
|
|
||||||
|
Then enforce editorial constraints before render:
|
||||||
|
- Remove filler and repeated lines.
|
||||||
|
- Ensure each scene has a single narrative job.
|
||||||
|
- Keep line lengths short enough for natural TTS breathing.
|
||||||
|
- Verify emotion tag is valid (`neutral`, `happy`, `excited`, `serious`, `curious`, `confident`) to avoid fallback normalization.
|
||||||
|
|
||||||
|
## 7) Project save/resume + asset-library workflows
|
||||||
|
|
||||||
|
Treat a podcast as a resumable production artifact.
|
||||||
|
|
||||||
|
### Save/resume
|
||||||
|
- Persist state to project APIs throughout the workflow (analysis, research, script, render jobs, knobs, final video URL).
|
||||||
|
- Use project list filtering/sorting to resume active work quickly.
|
||||||
|
- Handle duplicate-idea conflicts by reopening existing project IDs instead of cloning work.
|
||||||
|
|
||||||
|
### Asset library workflow
|
||||||
|
- Save generated and uploaded assets (audio/avatar/images) into the content asset library with project metadata.
|
||||||
|
- Use consistent tags (`podcast`, project id, scene id) so assets are searchable and reusable.
|
||||||
|
- Reuse previously approved host avatars and voice samples across episodes to reduce generation churn.
|
||||||
|
|
||||||
|
## 8) Video and dubbing execution strategy
|
||||||
|
|
||||||
|
### Video
|
||||||
|
- Only pass supported video resolution (`480p` or `720p`).
|
||||||
|
- Poll task status (video generation is asynchronous and can take up to ~10 minutes).
|
||||||
|
- Use mask image only when you need controlled motion region.
|
||||||
|
- Generate all scene videos before starting combine to avoid failed final assembly.
|
||||||
|
|
||||||
|
### Dubbing
|
||||||
|
- Use `quality=low` for fast/cheap exploration.
|
||||||
|
- Use `quality=high` + `use_voice_clone=true` when voice identity matters.
|
||||||
|
- Keep `speed` in 0.5–2.0 and voice clone accuracy in 0.1–1.0.
|
||||||
|
- For voice cloning, feed a clean 10–60s sample for best identity retention.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common failure modes and fixes
|
||||||
|
|
||||||
|
For broader platform issues, see the main [Troubleshooting Guide](../../guides/troubleshooting.md).
|
||||||
|
|
||||||
|
| Failure mode | Why it happens | Fix |
|
||||||
|
|---|---|---|
|
||||||
|
| Preflight blocked (analysis/research/script/TTS/video) | Insufficient credits or operation limits | Run lighter settings first: fewer scenes, lower duration, fewer research queries; then retry. |
|
||||||
|
| Research request rejected | No approved queries selected | Approve at least one non-empty query before running Exa research. |
|
||||||
|
| Research config mismatch | Include + exclude domains both supplied | Use only one domain filter type per run. |
|
||||||
|
| Scene audio cuts off | Scene text exceeded TTS max characters | Reduce scene length/lines; split long scene into two scenes. |
|
||||||
|
| Avatar-consistent image generation fails | `base_avatar_url` is broken/inaccessible | Re-upload avatar or switch to a valid project image URL; retry scene generation. |
|
||||||
|
| Video task fails quickly | Invalid media URL, unsupported resolution, missing assets | Verify audio/image URLs are valid and use only `480p`/`720p`. |
|
||||||
|
| Final combine video fails | One or more scene video files missing/invalid | Confirm every scene has a completed video task before combine. |
|
||||||
|
| Dubbing quality sounds robotic | Low quality mode or weak source audio | Switch to high quality and/or use voice cloning with a cleaner sample. |
|
||||||
|
| Voice clone results are unstable | Poor sample or extreme accuracy/speed settings | Use clean 10–60s sample; keep accuracy near default and speed near 1.0. |
|
||||||
|
| Save appears inconsistent across sessions | Save failed and only partial local fallback exists | Trigger explicit save after each major step and verify project reload from API. |
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user