Compare commits
1 Commits
fix/add-ma
...
dependabot
| Author | SHA1 | Date | |
|---|---|---|---|
|
a2163c33aa |
10
.gitignore
vendored
10
.gitignore
vendored
@@ -4,23 +4,15 @@ __pycache__/
|
||||
*.db
|
||||
*.sqlite*
|
||||
|
||||
nul
|
||||
LICENSE
|
||||
CHANGELOG.md
|
||||
|
||||
.trae/
|
||||
.trae
|
||||
|
||||
workspace/
|
||||
workspace/*
|
||||
|
||||
.windsurf
|
||||
artifacts
|
||||
|
||||
.opencode
|
||||
|
||||
data/
|
||||
data/*
|
||||
|
||||
.trae/
|
||||
/backend/database/migrations/*
|
||||
@@ -29,7 +21,7 @@ backend/*.db
|
||||
backend\youtube_audio
|
||||
youtube_avatars
|
||||
backend\youtube_images
|
||||
data/media/podcast_videos/AI_Videos
|
||||
|
||||
backend/.trae_*
|
||||
|
||||
# Onboarding progress files
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
# ALwrity Project
|
||||
|
||||
## What This Is
|
||||
ALwrity is an AI-powered content creation platform that helps users generate various types of content including podcasts, videos, blogs, and social media content. The platform features a React frontend and a FastAPI backend with onboarding workflows, API key management, and content generation capabilities.
|
||||
|
||||
## Core Value
|
||||
To provide an all-in-one AI content creation suite that simplifies the content production process for creators, marketers, and businesses.
|
||||
|
||||
## Current Focus
|
||||
Based on recent git commits, the team has been working on:
|
||||
- Podcast production features (voice cloning, avatar generation, B-roll integration)
|
||||
- Onboarding flow improvements
|
||||
- Backend stability and debugging
|
||||
- Frontend UI/UX enhancements
|
||||
|
||||
## Requirements
|
||||
|
||||
### Validated
|
||||
- User authentication (Clerk)
|
||||
- API key management for AI providers
|
||||
- Basic podcast generation workflow
|
||||
- File storage and media handling
|
||||
|
||||
### Active
|
||||
- Podcast script generation and editing
|
||||
- Voice cloning and avatar creation
|
||||
- B-roll scene rendering and integration
|
||||
- Onboarding flow completion tracking
|
||||
- API endpoint stability and debugging
|
||||
|
||||
### Out of Scope
|
||||
- Mobile applications (currently web-only)
|
||||
- Enterprise team collaboration features
|
||||
- Advanced analytics dashboard
|
||||
|
||||
## Key Decisions
|
||||
- Using FastAPI for backend performance
|
||||
- React with Material-UI for frontend consistency
|
||||
- Modular API design for extensibility
|
||||
- Database-first approach for persistence
|
||||
|
||||
## Constraints
|
||||
- Must maintain backward compatibility with existing API
|
||||
- Deployment targets include both development and production environments
|
||||
- Must support multiple AI providers (OpenAI, HuggingFace, etc.)
|
||||
- Budget-conscious resource usage for AI API calls
|
||||
@@ -1,40 +0,0 @@
|
||||
# Project State
|
||||
|
||||
## Project Reference
|
||||
**Core Value**: ALwrity is an AI-powered content creation platform that helps users generate various types of content including podcasts, videos, blogs, and social media content.
|
||||
|
||||
**Current Focus**: Based on recent development activity, the team is implementing Phase 2 of the WaveSpeed AI integration roadmap - Hyper-Personalization features for the Persona system, including voice training and avatar creation.
|
||||
|
||||
## Current Position
|
||||
**Phase**: 2 of 3 - Hyper-Personalization
|
||||
**Plan**: 3 of 5 - Persona Avatar Creation & Integration
|
||||
**Status**: In Progress - Working on avatar service implementation and frontend UI for avatar creation
|
||||
|
||||
## Progress
|
||||
Progress: [███████░░] 70%
|
||||
|
||||
## Recent Decisions
|
||||
1. **Avatar Service Architecture**: Decided to create a shared avatar service in backend/services/wavespeed/avatar/ for reuse across LinkedIn and Persona modules
|
||||
2. **UI Framework**: Continuing with Material-UI (MUI) for consistent avatar creation interface
|
||||
3. **Storage Strategy**: Using cloud storage for avatar assets with metadata tracking in PostgreSQL
|
||||
4. **Generation Queue**: Implementing asynchronous processing for avatar generation to prevent API timeouts
|
||||
|
||||
## Pending Todos
|
||||
- [ ] Complete avatar generation API endpoints
|
||||
- [ ] Implement avatar library management UI
|
||||
- [ ] Add avatar preview functionality
|
||||
- [ ] Create avatar upload/download capabilities
|
||||
- [ ] Integrate avatar selection into Persona dashboard
|
||||
- [ ] Add usage tracking and cost estimation for avatar generation
|
||||
- [ ] Write comprehensive tests for avatar service
|
||||
- [ ] Update documentation for avatar feature
|
||||
|
||||
## Blockers/Concerns
|
||||
- **WaveSpeed API Rate Limits**: Need to implement proper queuing and retry mechanisms
|
||||
- **Storage Costs**: Avatar storage could become expensive at scale - need to implement cleanup policies
|
||||
- **Generation Time**: Avatar generation can take 30-60 seconds - need to improve user experience during wait
|
||||
- **Quality Consistency**: Ensuring generated avatars maintain consistent quality across different inputs
|
||||
|
||||
Last session: 2026-04-21 07:02:08
|
||||
Stopped at: Session resumed, proceeding to discuss Phase 2 context
|
||||
Resume file: [updated if applicable]
|
||||
13
Procfile
13
Procfile
@@ -1,13 +0,0 @@
|
||||
web: cd backend && ALWRITY_ENABLED_FEATURES=podcast python -c "
|
||||
import os
|
||||
import sys
|
||||
# Ensure podcast mode
|
||||
os.environ.setdefault('ALWRITY_ENABLED_FEATURES', 'podcast')
|
||||
# Set HOST/PORT for Render
|
||||
port = os.getenv('PORT', '10000')
|
||||
host = os.getenv('HOST', '0.0.0.0')
|
||||
print(f'[STARTUP] Starting uvicorn on {host}:{port}', flush=True)
|
||||
sys.stdout.flush()
|
||||
import uvicorn
|
||||
uvicorn.run('app:app', host=host, port=int(port), reload=False)
|
||||
"
|
||||
14
README.md
14
README.md
@@ -1,14 +0,0 @@
|
||||
# Render CLI
|
||||
|
||||
## Installation
|
||||
|
||||
- [Homebrew](https://render.com/docs/cli#homebrew-macos-linux)
|
||||
- [Direct Download](https://render.com/docs/cli#direct-download)
|
||||
|
||||
## Documentation
|
||||
|
||||
Documentation is hosted at https://render.com/docs/cli.
|
||||
|
||||
## Contributing
|
||||
|
||||
To create a new command, use the `cmd/template.go` template file as a starting point. Reference the [CLI Style Guide](docs/STYLE.md) to learn more about command naming, flags, arguments, and help text conventions.
|
||||
@@ -1,672 +0,0 @@
|
||||
import React from 'react';
|
||||
import { BrowserRouter as Router, Routes, Route, Navigate, useLocation } from 'react-router-dom';
|
||||
import { Box, CircularProgress, Typography } from '@mui/material';
|
||||
import { CopilotKit } from "@copilotkit/react-core";
|
||||
import { ClerkProvider, useAuth } from '@clerk/clerk-react';
|
||||
import "@copilotkit/react-ui/styles.css";
|
||||
import Wizard from './components/OnboardingWizard/Wizard';
|
||||
import MainDashboard from './components/MainDashboard/MainDashboard';
|
||||
import SEODashboard from './components/SEODashboard/SEODashboard';
|
||||
import ContentPlanningDashboard from './components/ContentPlanningDashboard/ContentPlanningDashboard';
|
||||
import FacebookWriter from './components/FacebookWriter/FacebookWriter';
|
||||
import LinkedInWriter from './components/LinkedInWriter/LinkedInWriter';
|
||||
import BlogWriter from './components/BlogWriter/BlogWriter';
|
||||
import StoryWriter from './components/StoryWriter/StoryWriter';
|
||||
import { StoryProjectList } from './components/StoryWriter/StoryProjectList';
|
||||
import YouTubeCreator from './components/YouTubeCreator/YouTubeCreator';
|
||||
import { CreateStudio, EditStudio, UpscaleStudio, ControlStudio, SocialOptimizer, AssetLibrary, ImageStudioDashboard, FaceSwapStudio, CompressionStudio, ImageProcessingStudio } from './components/ImageStudio';
|
||||
import {
|
||||
VideoStudioDashboard,
|
||||
CreateVideo,
|
||||
AvatarVideo,
|
||||
EnhanceVideo,
|
||||
ExtendVideo,
|
||||
EditVideo,
|
||||
TransformVideo,
|
||||
SocialVideo,
|
||||
FaceSwap,
|
||||
VideoTranslate,
|
||||
VideoBackgroundRemover,
|
||||
AddAudioToVideo,
|
||||
LibraryVideo,
|
||||
} from './components/VideoStudio';
|
||||
import {
|
||||
ProductMarketingDashboard,
|
||||
ProductPhotoshootStudio,
|
||||
ProductAnimationStudio,
|
||||
ProductVideoStudio,
|
||||
ProductAvatarStudio,
|
||||
} from './components/ProductMarketing';
|
||||
import PodcastDashboard from './components/PodcastMaker/PodcastDashboard';
|
||||
import PricingPage from './components/Pricing/PricingPage';
|
||||
import WixTestPage from './components/WixTestPage/WixTestPage';
|
||||
import WixCallbackPage from './components/WixCallbackPage/WixCallbackPage';
|
||||
import WordPressCallbackPage from './components/WordPressCallbackPage/WordPressCallbackPage';
|
||||
import BingCallbackPage from './components/BingCallbackPage/BingCallbackPage';
|
||||
import BingAnalyticsStorage from './components/BingAnalyticsStorage/BingAnalyticsStorage';
|
||||
import ResearchDashboard from './pages/ResearchDashboard';
|
||||
import IntentResearchTest from './pages/IntentResearchTest';
|
||||
import SchedulerDashboard from './pages/SchedulerDashboard';
|
||||
import BillingPage from './pages/BillingPage';
|
||||
import ApprovalsPage from './pages/ApprovalsPage';
|
||||
import TeamActivityPage from './pages/TeamActivityPage';
|
||||
import StripeDisputesDashboard from './pages/StripeDisputesDashboard';
|
||||
import ProtectedRoute from './components/shared/ProtectedRoute';
|
||||
import GSCAuthCallback from './components/SEODashboard/components/GSCAuthCallback';
|
||||
import Landing from './components/Landing/Landing';
|
||||
import ErrorBoundary from './components/shared/ErrorBoundary';
|
||||
import ErrorBoundaryTest from './components/shared/ErrorBoundaryTest';
|
||||
import CopilotKitDegradedBanner from './components/shared/CopilotKitDegradedBanner';
|
||||
import { OnboardingProvider } from './contexts/OnboardingContext';
|
||||
import { SubscriptionProvider, useSubscription } from './contexts/SubscriptionContext';
|
||||
import { CopilotKitHealthProvider } from './contexts/CopilotKitHealthContext';
|
||||
import { useOAuthTokenAlerts } from './hooks/useOAuthTokenAlerts';
|
||||
|
||||
import { setAuthTokenGetter, setClerkSignOut } from './api/client';
|
||||
import { setMediaAuthTokenGetter } from './utils/fetchMediaBlobUrl';
|
||||
import { setBillingAuthTokenGetter } from './services/billingService';
|
||||
import { useOnboarding } from './contexts/OnboardingContext';
|
||||
import { useState, useEffect } from 'react';
|
||||
import ConnectionErrorPage from './components/shared/ConnectionErrorPage';
|
||||
import { isPodcastOnlyDemoMode } from './utils/demoMode';
|
||||
|
||||
// interface OnboardingStatus {
|
||||
// onboarding_required: boolean;
|
||||
// onboarding_complete: boolean;
|
||||
// current_step?: number;
|
||||
// total_steps?: number;
|
||||
// completion_percentage?: number;
|
||||
// }
|
||||
|
||||
// Conditional CopilotKit wrapper that only shows sidebar on content-planning route
|
||||
const ConditionalCopilotKit: React.FC<{ children: React.ReactNode }> = ({ children }) => {
|
||||
// Do not render CopilotSidebar here. Let specific pages/components control it.
|
||||
return <>{children}</>;
|
||||
};
|
||||
|
||||
// Wrapper to only enable CopilotKit checks/provider when user is authenticated
|
||||
// This prevents CopilotKit from running on the Landing page
|
||||
const AuthenticatedCopilotWrapper: React.FC<{
|
||||
children: React.ReactNode;
|
||||
apiKey: string;
|
||||
}> = ({ children, apiKey }) => {
|
||||
const { isSignedIn } = useAuth();
|
||||
const location = useLocation();
|
||||
|
||||
// Exclude CopilotKit from running on:
|
||||
// 1. Landing page (handled by !isSignedIn)
|
||||
// 2. Onboarding pages (to prevent health check timeouts)
|
||||
// 3. Podcast-only demo mode (CopilotKit not needed)
|
||||
const isPodcastOnly = isPodcastOnlyDemoMode();
|
||||
const shouldExcludeCopilot = !isSignedIn || location.pathname.startsWith('/onboarding') || isPodcastOnly;
|
||||
|
||||
if (shouldExcludeCopilot) {
|
||||
return <>{children}</>;
|
||||
}
|
||||
|
||||
const hasKey = apiKey && apiKey.trim();
|
||||
|
||||
if (hasKey) {
|
||||
// Enhanced error handler that updates health context
|
||||
const handleCopilotKitError = (e: any) => {
|
||||
console.error("CopilotKit Error:", e);
|
||||
|
||||
// Try to get health context if available
|
||||
// We'll use a custom event to notify health context since we can't access it directly here
|
||||
const errorMessage = e?.error?.message || e?.message || 'CopilotKit error occurred';
|
||||
const errorType = errorMessage.toLowerCase();
|
||||
|
||||
// Differentiate between fatal and transient errors
|
||||
const isFatalError =
|
||||
errorType.includes('cors') ||
|
||||
errorType.includes('ssl') ||
|
||||
errorType.includes('certificate') ||
|
||||
errorType.includes('403') ||
|
||||
errorType.includes('forbidden') ||
|
||||
errorType.includes('ERR_CERT_COMMON_NAME_INVALID');
|
||||
|
||||
// Dispatch event for health context to listen to
|
||||
window.dispatchEvent(new CustomEvent('copilotkit-error', {
|
||||
detail: {
|
||||
error: e,
|
||||
errorMessage,
|
||||
isFatal: isFatalError,
|
||||
}
|
||||
}));
|
||||
};
|
||||
|
||||
return (
|
||||
<CopilotKitHealthProvider initialHealthStatus={true}>
|
||||
<CopilotKitDegradedBanner />
|
||||
<ErrorBoundary
|
||||
context="CopilotKit"
|
||||
showDetails={process.env.NODE_ENV === 'development'}
|
||||
fallback={
|
||||
<Box sx={{ p: 3, textAlign: 'center' }}>
|
||||
<Typography variant="h6" color="warning" gutterBottom>
|
||||
Chat Unavailable
|
||||
</Typography>
|
||||
<Typography variant="body2" color="textSecondary">
|
||||
CopilotKit encountered an error. The app continues to work with manual controls.
|
||||
</Typography>
|
||||
</Box>
|
||||
}
|
||||
>
|
||||
<CopilotKit
|
||||
publicApiKey={apiKey}
|
||||
showDevConsole={false}
|
||||
onError={handleCopilotKitError}
|
||||
>
|
||||
{children}
|
||||
</CopilotKit>
|
||||
</ErrorBoundary>
|
||||
</CopilotKitHealthProvider>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<CopilotKitHealthProvider initialHealthStatus={false}>
|
||||
<CopilotKitDegradedBanner />
|
||||
{children}
|
||||
</CopilotKitHealthProvider>
|
||||
);
|
||||
};
|
||||
|
||||
// Component to handle initial routing based on subscription and onboarding status
|
||||
// Flow: Subscription → Onboarding → Dashboard
|
||||
const InitialRouteHandler: React.FC = () => {
|
||||
const { loading, error, isOnboardingComplete, initializeOnboarding, data } = useOnboarding();
|
||||
const { subscription, loading: subscriptionLoading, checkSubscription } = useSubscription();
|
||||
const [connectionError, setConnectionError] = useState<{
|
||||
hasError: boolean;
|
||||
error: Error | null;
|
||||
}>({
|
||||
hasError: false,
|
||||
error: null,
|
||||
});
|
||||
|
||||
// Poll for OAuth token alerts and show toast notifications
|
||||
// Only enabled when user is authenticated (has subscription)
|
||||
useOAuthTokenAlerts({
|
||||
enabled: subscription?.active === true,
|
||||
interval: 60000, // Poll every 1 minute
|
||||
});
|
||||
|
||||
// Check subscription on mount (non-blocking - don't wait for it to route)
|
||||
useEffect(() => {
|
||||
// Delay subscription check slightly to allow auth token getter to be installed first
|
||||
const timeoutId = setTimeout(async () => {
|
||||
// Retry logic for initial subscription check
|
||||
const maxRetries = 3;
|
||||
for (let attempt = 0; attempt < maxRetries; attempt++) {
|
||||
try {
|
||||
await checkSubscription();
|
||||
break; // Success
|
||||
} catch (err) {
|
||||
console.error(`App: Subscription check attempt ${attempt + 1} failed:`, err);
|
||||
|
||||
// If it's a connection error and we have retries left, wait and retry
|
||||
const isConnectionError = err instanceof Error && (err.name === 'NetworkError' || err.name === 'ConnectionError');
|
||||
|
||||
if (isConnectionError && attempt < maxRetries - 1) {
|
||||
const delay = 1000 * Math.pow(2, attempt); // 1s, 2s
|
||||
await new Promise(resolve => setTimeout(resolve, delay));
|
||||
continue;
|
||||
}
|
||||
|
||||
// If final attempt or not a connection error, handle it
|
||||
if (attempt === maxRetries - 1 || !isConnectionError) {
|
||||
if (isConnectionError) {
|
||||
setConnectionError({
|
||||
hasError: true,
|
||||
error: err as Error,
|
||||
});
|
||||
}
|
||||
// Don't block routing on other errors
|
||||
}
|
||||
}
|
||||
}
|
||||
}, 100); // Small delay to ensure TokenInstaller has run
|
||||
|
||||
return () => clearTimeout(timeoutId);
|
||||
}, []); // Remove checkSubscription dependency to prevent loop
|
||||
|
||||
// Initialize onboarding only after subscription is confirmed
|
||||
useEffect(() => {
|
||||
if (subscription && !subscriptionLoading) {
|
||||
// Check if user is new (no subscription record at all)
|
||||
const isNewUser = !subscription || subscription.plan === 'none';
|
||||
|
||||
console.log('InitialRouteHandler: Subscription data received:', {
|
||||
plan: subscription.plan,
|
||||
active: subscription.active,
|
||||
isNewUser,
|
||||
subscriptionLoading
|
||||
});
|
||||
|
||||
if (subscription.active && !isNewUser) {
|
||||
console.log('InitialRouteHandler: Subscription confirmed, initializing onboarding...');
|
||||
initializeOnboarding();
|
||||
}
|
||||
}
|
||||
}, [subscription, subscriptionLoading, initializeOnboarding]);
|
||||
|
||||
// Handle connection error - show connection error page
|
||||
if (connectionError.hasError) {
|
||||
const handleRetry = () => {
|
||||
setConnectionError({
|
||||
hasError: false,
|
||||
error: null,
|
||||
});
|
||||
// Re-trigger the subscription check using context
|
||||
checkSubscription().catch((err) => {
|
||||
if (err instanceof Error && (err.name === 'NetworkError' || err.name === 'ConnectionError')) {
|
||||
setConnectionError({
|
||||
hasError: true,
|
||||
error: err,
|
||||
});
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
const handleGoHome = () => {
|
||||
window.location.href = '/';
|
||||
};
|
||||
|
||||
return (
|
||||
<ConnectionErrorPage
|
||||
onRetry={handleRetry}
|
||||
onGoHome={handleGoHome}
|
||||
message={connectionError.error?.message || "Backend service is not available. Please check if the server is running."}
|
||||
title="Connection Error"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
// Loading state - only wait for onboarding init, not subscription check
|
||||
// Subscription check is non-blocking and happens in background
|
||||
const waitingForOnboardingInit = loading || !data;
|
||||
if (loading || waitingForOnboardingInit) {
|
||||
return (
|
||||
<Box
|
||||
display="flex"
|
||||
flexDirection="column"
|
||||
alignItems="center"
|
||||
justifyContent="center"
|
||||
minHeight="100vh"
|
||||
gap={2}
|
||||
>
|
||||
<CircularProgress size={60} />
|
||||
<Typography variant="h6" color="textSecondary">
|
||||
{subscriptionLoading ? 'Checking subscription...' : 'Preparing your workspace...'}
|
||||
</Typography>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
// Error state
|
||||
if (error) {
|
||||
return (
|
||||
<Box
|
||||
display="flex"
|
||||
flexDirection="column"
|
||||
alignItems="center"
|
||||
justifyContent="center"
|
||||
minHeight="100vh"
|
||||
gap={2}
|
||||
p={3}
|
||||
>
|
||||
<Typography variant="h5" color="error" gutterBottom>
|
||||
Error
|
||||
</Typography>
|
||||
<Typography variant="body1" color="textSecondary" textAlign="center">
|
||||
{error}
|
||||
</Typography>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
// Decision tree for SIGNED-IN users:
|
||||
// Priority: Subscription → Onboarding → Dashboard (as per user flow: Landing → Subscription → Onboarding → Dashboard)
|
||||
|
||||
// 1. If subscription is still loading, show loading state
|
||||
if (subscriptionLoading) {
|
||||
return (
|
||||
<Box
|
||||
display="flex"
|
||||
flexDirection="column"
|
||||
alignItems="center"
|
||||
justifyContent="center"
|
||||
minHeight="100vh"
|
||||
gap={2}
|
||||
>
|
||||
<CircularProgress size={60} />
|
||||
<Typography variant="h6" color="textSecondary">
|
||||
Checking subscription...
|
||||
</Typography>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
// 2. No subscription data yet - handle gracefully
|
||||
// If onboarding is complete, allow access to dashboard (user already went through flow)
|
||||
// If onboarding not complete, check if subscription check is still loading or failed
|
||||
if (!subscription) {
|
||||
if (isOnboardingComplete) {
|
||||
console.log('InitialRouteHandler: Onboarding complete but no subscription data → Dashboard (allow access)');
|
||||
return <Navigate to="/dashboard" replace />;
|
||||
}
|
||||
|
||||
// Onboarding not complete and no subscription data
|
||||
// If subscription check is still loading, show loading state
|
||||
if (subscriptionLoading) {
|
||||
return (
|
||||
<Box
|
||||
display="flex"
|
||||
flexDirection="column"
|
||||
alignItems="center"
|
||||
justifyContent="center"
|
||||
minHeight="100vh"
|
||||
gap={2}
|
||||
>
|
||||
<CircularProgress size={60} />
|
||||
<Typography variant="h6" color="textSecondary">
|
||||
Checking subscription...
|
||||
</Typography>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
// Subscription check completed but returned null/undefined
|
||||
// This likely means no subscription - redirect to pricing
|
||||
console.log('InitialRouteHandler: No subscription data after check → Pricing page');
|
||||
return <Navigate to="/pricing" replace />;
|
||||
}
|
||||
|
||||
// 3. Check subscription status first
|
||||
const isNewUser = !subscription || subscription.plan === 'none';
|
||||
|
||||
// No active subscription → Show modal (SubscriptionContext handles this)
|
||||
// Don't redirect immediately - let the modal show first
|
||||
// User can click "Renew Subscription" button in modal to go to pricing
|
||||
// Or click "Maybe Later" to dismiss (but they still can't use features)
|
||||
if (isNewUser || !subscription.active) {
|
||||
console.log('InitialRouteHandler: No active subscription - modal will be shown by SubscriptionContext');
|
||||
// Note: SubscriptionContext will show the modal automatically when subscription is inactive
|
||||
// We still redirect to pricing for new users, but allow existing users with expired subscriptions
|
||||
// to see the modal first. The modal has a "Renew Subscription" button that navigates to pricing.
|
||||
// For new users (no subscription at all), redirect to pricing immediately
|
||||
if (isNewUser) {
|
||||
console.log('InitialRouteHandler: New user (no subscription) → Pricing page');
|
||||
return <Navigate to="/pricing" replace />;
|
||||
}
|
||||
// For existing users with inactive subscription, show modal but don't redirect immediately
|
||||
// The modal will be shown by SubscriptionContext, and user can click "Renew Subscription"
|
||||
// Allow access to dashboard (modal will be shown and block functionality)
|
||||
console.log('InitialRouteHandler: Inactive subscription - allowing access to show modal');
|
||||
// Continue to onboarding/dashboard flow - modal will be shown by SubscriptionContext
|
||||
}
|
||||
|
||||
// 4. Has active subscription, check onboarding status
|
||||
if (!isOnboardingComplete) {
|
||||
console.log('InitialRouteHandler: Subscription active but onboarding incomplete → Onboarding');
|
||||
return <Navigate to="/onboarding" replace />;
|
||||
}
|
||||
|
||||
// 5. Has subscription AND completed onboarding → Dashboard
|
||||
console.log('InitialRouteHandler: All set (subscription + onboarding) → Dashboard');
|
||||
return <Navigate to="/dashboard" replace />;
|
||||
};
|
||||
|
||||
// Root route that chooses Landing (signed out) or InitialRouteHandler (signed in)
|
||||
const RootRoute: React.FC = () => {
|
||||
const { isSignedIn } = useAuth();
|
||||
if (isSignedIn) {
|
||||
return <InitialRouteHandler />;
|
||||
}
|
||||
return <Landing />;
|
||||
};
|
||||
|
||||
// Installs Clerk auth token getter into axios clients and stores user_id
|
||||
// Must render under ClerkProvider
|
||||
const TokenInstaller: React.FC = () => {
|
||||
const { getToken, userId, isSignedIn, signOut } = useAuth();
|
||||
|
||||
// Store user_id in localStorage when user signs in
|
||||
useEffect(() => {
|
||||
if (isSignedIn && userId) {
|
||||
console.log('TokenInstaller: Storing user_id in localStorage:', userId);
|
||||
localStorage.setItem('user_id', userId);
|
||||
|
||||
// Trigger event to notify SubscriptionContext that user is authenticated
|
||||
window.dispatchEvent(new CustomEvent('user-authenticated', { detail: { userId } }));
|
||||
} else if (!isSignedIn) {
|
||||
// Clear user_id when signed out
|
||||
console.log('TokenInstaller: Clearing user_id from localStorage');
|
||||
localStorage.removeItem('user_id');
|
||||
}
|
||||
}, [isSignedIn, userId]);
|
||||
|
||||
// Install token getter for API calls
|
||||
useEffect(() => {
|
||||
const tokenGetter = async () => {
|
||||
try {
|
||||
const template = process.env.REACT_APP_CLERK_JWT_TEMPLATE;
|
||||
// If a template is provided and it's not a placeholder, request a template-specific JWT
|
||||
if (template && template !== 'your_jwt_template_name_here') {
|
||||
// @ts-ignore Clerk types allow options object
|
||||
return await getToken({ template });
|
||||
}
|
||||
return await getToken();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
// Set token getter for main API client
|
||||
setAuthTokenGetter(tokenGetter);
|
||||
|
||||
// Set token getter for billing API client (same function)
|
||||
setBillingAuthTokenGetter(tokenGetter);
|
||||
|
||||
// Set token getter for media blob URL fetcher (for authenticated image/video requests)
|
||||
setMediaAuthTokenGetter(tokenGetter);
|
||||
}, [getToken]);
|
||||
|
||||
// Install Clerk signOut function for handling expired tokens
|
||||
useEffect(() => {
|
||||
if (signOut) {
|
||||
setClerkSignOut(async () => {
|
||||
await signOut();
|
||||
});
|
||||
}
|
||||
}, [signOut]);
|
||||
|
||||
return null;
|
||||
};
|
||||
|
||||
const App: React.FC = () => {
|
||||
// React Hooks MUST be at the top before any conditionals
|
||||
const [loading, setLoading] = useState(true);
|
||||
|
||||
// Get CopilotKit key from localStorage or .env
|
||||
const [copilotApiKey, setCopilotApiKey] = useState(() => {
|
||||
const savedKey = localStorage.getItem('copilotkit_api_key');
|
||||
const envKey = process.env.REACT_APP_COPILOTKIT_API_KEY || '';
|
||||
const key = (savedKey || envKey).trim();
|
||||
|
||||
// Validate key format if present
|
||||
if (key && !key.startsWith('ck_pub_')) {
|
||||
console.warn('CopilotKit API key format invalid - must start with ck_pub_');
|
||||
}
|
||||
|
||||
return key;
|
||||
});
|
||||
|
||||
// Initialize app - loading state will be managed by InitialRouteHandler
|
||||
useEffect(() => {
|
||||
// Remove manual health check - connection errors are handled by ErrorBoundary
|
||||
setLoading(false);
|
||||
}, []);
|
||||
|
||||
// Listen for CopilotKit key updates
|
||||
useEffect(() => {
|
||||
const handleKeyUpdate = (event: CustomEvent) => {
|
||||
const newKey = event.detail?.apiKey;
|
||||
if (newKey) {
|
||||
console.log('App: CopilotKit key updated, reloading...');
|
||||
setCopilotApiKey(newKey);
|
||||
setTimeout(() => window.location.reload(), 500);
|
||||
}
|
||||
};
|
||||
|
||||
window.addEventListener('copilotkit-key-updated', handleKeyUpdate as EventListener);
|
||||
return () => window.removeEventListener('copilotkit-key-updated', handleKeyUpdate as EventListener);
|
||||
}, []);
|
||||
|
||||
// Token installer must be inside ClerkProvider; see TokenInstaller below
|
||||
|
||||
if (loading) {
|
||||
return (
|
||||
<Box
|
||||
display="flex"
|
||||
flexDirection="column"
|
||||
alignItems="center"
|
||||
justifyContent="center"
|
||||
minHeight="100vh"
|
||||
gap={2}
|
||||
>
|
||||
<CircularProgress size={60} />
|
||||
<Typography variant="h6" color="textSecondary">
|
||||
Connecting to ALwrity...
|
||||
</Typography>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
// Get environment variables with fallbacks
|
||||
const clerkPublishableKey = process.env.REACT_APP_CLERK_PUBLISHABLE_KEY || '';
|
||||
const clerkJSUrl = process.env.REACT_APP_CLERK_JS_URL;
|
||||
|
||||
// Show error if required keys are missing
|
||||
if (!clerkPublishableKey) {
|
||||
return (
|
||||
<Box sx={{ p: 3, textAlign: 'center' }}>
|
||||
<Typography color="error" variant="h6">
|
||||
Missing Clerk Publishable Key
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ mt: 1 }}>
|
||||
Please add REACT_APP_CLERK_PUBLISHABLE_KEY to your .env file
|
||||
</Typography>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
// Render app with or without CopilotKit based on whether we have a key
|
||||
const renderApp = () => {
|
||||
return (
|
||||
<Router>
|
||||
<AuthenticatedCopilotWrapper apiKey={copilotApiKey}>
|
||||
<ConditionalCopilotKit>
|
||||
<TokenInstaller />
|
||||
<Routes>
|
||||
<Route path="/" element={<RootRoute />} />
|
||||
<Route
|
||||
path="/onboarding"
|
||||
element={
|
||||
<ErrorBoundary context="Onboarding Wizard" showDetails>
|
||||
<Wizard />
|
||||
</ErrorBoundary>
|
||||
}
|
||||
/>
|
||||
{/* Error Boundary Testing - Development Only */}
|
||||
{process.env.NODE_ENV === 'development' && (
|
||||
<Route path="/error-test" element={<ErrorBoundaryTest />} />
|
||||
)}
|
||||
<Route path="/dashboard" element={<ProtectedRoute><MainDashboard /></ProtectedRoute>} />
|
||||
<Route path="/seo" element={<ProtectedRoute><SEODashboard /></ProtectedRoute>} />
|
||||
<Route path="/seo-dashboard" element={<ProtectedRoute><SEODashboard /></ProtectedRoute>} />
|
||||
<Route path="/content-planning" element={<ProtectedRoute><ContentPlanningDashboard /></ProtectedRoute>} />
|
||||
<Route path="/facebook-writer" element={<ProtectedRoute><FacebookWriter /></ProtectedRoute>} />
|
||||
<Route path="/linkedin-writer" element={<ProtectedRoute><LinkedInWriter /></ProtectedRoute>} />
|
||||
<Route path="/blog-writer" element={<ProtectedRoute><BlogWriter /></ProtectedRoute>} />
|
||||
<Route path="/story-writer" element={<ProtectedRoute><StoryWriter /></ProtectedRoute>} />
|
||||
<Route path="/story-projects" element={<ProtectedRoute><StoryProjectList /></ProtectedRoute>} />
|
||||
<Route path="/youtube-creator" element={<ProtectedRoute><YouTubeCreator /></ProtectedRoute>} />
|
||||
<Route path="/podcast-maker" element={<ProtectedRoute><PodcastDashboard /></ProtectedRoute>} />
|
||||
<Route path="/image-studio" element={<ProtectedRoute><ImageStudioDashboard /></ProtectedRoute>} />
|
||||
<Route path="/video-studio" element={<ProtectedRoute><VideoStudioDashboard /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/create" element={<ProtectedRoute><CreateVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/avatar" element={<ProtectedRoute><AvatarVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/enhance" element={<ProtectedRoute><EnhanceVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/extend" element={<ProtectedRoute><ExtendVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/edit" element={<ProtectedRoute><EditVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/transform" element={<ProtectedRoute><TransformVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/social" element={<ProtectedRoute><SocialVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/face-swap" element={<ProtectedRoute><FaceSwap /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/video-translate" element={<ProtectedRoute><VideoTranslate /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/video-background-remover" element={<ProtectedRoute><VideoBackgroundRemover /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/add-audio-to-video" element={<ProtectedRoute><AddAudioToVideo /></ProtectedRoute>} />
|
||||
<Route path="/video-studio/library" element={<ProtectedRoute><LibraryVideo /></ProtectedRoute>} />
|
||||
<Route path="/image-generator" element={<ProtectedRoute><CreateStudio /></ProtectedRoute>} />
|
||||
<Route path="/image-editor" element={<ProtectedRoute><EditStudio /></ProtectedRoute>} />
|
||||
<Route path="/image-upscale" element={<ProtectedRoute><UpscaleStudio /></ProtectedRoute>} />
|
||||
<Route path="/image-control" element={<ProtectedRoute><ControlStudio /></ProtectedRoute>} />
|
||||
<Route path="/image-studio/face-swap" element={<ProtectedRoute><FaceSwapStudio /></ProtectedRoute>} />
|
||||
<Route path="/image-studio/compress" element={<ProtectedRoute><CompressionStudio /></ProtectedRoute>} />
|
||||
<Route path="/image-studio/processing" element={<ProtectedRoute><ImageProcessingStudio /></ProtectedRoute>} />
|
||||
<Route path="/image-studio/social-optimizer" element={<ProtectedRoute><SocialOptimizer /></ProtectedRoute>} />
|
||||
<Route path="/asset-library" element={<ProtectedRoute><AssetLibrary /></ProtectedRoute>} />
|
||||
<Route path="/campaign-creator" element={<ProtectedRoute><ProductMarketingDashboard /></ProtectedRoute>} />
|
||||
<Route path="/campaign-creator/photoshoot" element={<ProtectedRoute><ProductPhotoshootStudio /></ProtectedRoute>} />
|
||||
<Route path="/campaign-creator/animation" element={<ProtectedRoute><ProductAnimationStudio /></ProtectedRoute>} />
|
||||
<Route path="/campaign-creator/video" element={<ProtectedRoute><ProductVideoStudio /></ProtectedRoute>} />
|
||||
<Route path="/campaign-creator/avatar" element={<ProtectedRoute><ProductAvatarStudio /></ProtectedRoute>} />
|
||||
<Route path="/product-marketing" element={<Navigate to="/campaign-creator" replace />} />
|
||||
<Route path="/scheduler-dashboard" element={<ProtectedRoute><SchedulerDashboard /></ProtectedRoute>} />
|
||||
<Route path="/billing" element={<ProtectedRoute><BillingPage /></ProtectedRoute>} />
|
||||
<Route path="/approvals" element={<ProtectedRoute><ApprovalsPage /></ProtectedRoute>} />
|
||||
<Route path="/team-activity" element={<ProtectedRoute><TeamActivityPage /></ProtectedRoute>} />
|
||||
<Route path="/stripe-disputes" element={<ProtectedRoute><StripeDisputesDashboard /></ProtectedRoute>} />
|
||||
<Route path="/pricing" element={<PricingPage />} />
|
||||
<Route path="/research-test" element={<ResearchDashboard />} />
|
||||
<Route path="/research-dashboard" element={<ResearchDashboard />} />
|
||||
<Route path="/alwrity-researcher" element={<ResearchDashboard />} />
|
||||
<Route path="/intent-research" element={<IntentResearchTest />} />
|
||||
<Route path="/wix-test" element={<WixTestPage />} />
|
||||
<Route path="/wix-test-direct" element={<WixTestPage />} />
|
||||
<Route path="/wix/callback" element={<WixCallbackPage />} />
|
||||
<Route path="/wp/callback" element={<WordPressCallbackPage />} />
|
||||
<Route path="/gsc/callback" element={<GSCAuthCallback />} />
|
||||
<Route path="/bing/callback" element={<BingCallbackPage />} />
|
||||
<Route path="/bing-analytics-storage" element={<ProtectedRoute><BingAnalyticsStorage /></ProtectedRoute>} />
|
||||
</Routes>
|
||||
</ConditionalCopilotKit>
|
||||
</AuthenticatedCopilotWrapper>
|
||||
</Router>
|
||||
);
|
||||
};
|
||||
|
||||
return (
|
||||
<ErrorBoundary
|
||||
context="Application Root"
|
||||
showDetails={process.env.NODE_ENV === 'development'}
|
||||
onError={(error, errorInfo) => {
|
||||
// Custom error handler - send to analytics/monitoring
|
||||
console.error('Global error caught:', { error, errorInfo });
|
||||
// TODO: Send to error tracking service (Sentry, LogRocket, etc.)
|
||||
}}
|
||||
>
|
||||
<ClerkProvider publishableKey={clerkPublishableKey} clerkJSUrl={clerkJSUrl}>
|
||||
<SubscriptionProvider>
|
||||
<OnboardingProvider>
|
||||
{renderApp()}
|
||||
</OnboardingProvider>
|
||||
</SubscriptionProvider>
|
||||
</ClerkProvider>
|
||||
</ErrorBoundary>
|
||||
);
|
||||
};
|
||||
|
||||
export default App;
|
||||
@@ -1,537 +0,0 @@
|
||||
import React, { useMemo, useCallback } from "react";
|
||||
import { Stack, Typography, Chip, Divider, Box, alpha, Paper, Tooltip } from "@mui/material";
|
||||
import {
|
||||
Insights as InsightsIcon,
|
||||
Search as SearchIcon,
|
||||
AttachMoney as AttachMoneyIcon,
|
||||
EditNote as EditNoteIcon,
|
||||
Article as ArticleIcon,
|
||||
AutoAwesome as AutoAwesomeIcon,
|
||||
FormatQuote as FormatQuoteIcon,
|
||||
Campaign as CampaignIcon,
|
||||
Explore as ExploreIcon,
|
||||
} from "@mui/icons-material";
|
||||
import { Research, ResearchInsight } from "../types";
|
||||
import { GlassyCard, glassyCardSx, PrimaryButton } from "../ui";
|
||||
import { FactCard } from "../FactCard";
|
||||
|
||||
interface ResearchSummaryProps {
|
||||
research: Research;
|
||||
canGenerateScript: boolean;
|
||||
onGenerateScript: () => void;
|
||||
}
|
||||
|
||||
export const ResearchSummary: React.FC<ResearchSummaryProps> = ({
|
||||
research,
|
||||
canGenerateScript,
|
||||
onGenerateScript,
|
||||
}) => {
|
||||
// Simple markdown-to-HTML converter
|
||||
const renderMarkdown = useCallback((text: string) => {
|
||||
if (!text) return null;
|
||||
return text
|
||||
.split('\n')
|
||||
.filter(line => line.trim() !== '') // Remove empty lines
|
||||
.map((line, i) => {
|
||||
// Handle bold
|
||||
let processedLine = line.replace(/\*\*(.*?)\*\*/g, '<strong>$1</strong>');
|
||||
// Handle lists
|
||||
if (processedLine.trim().startsWith('- ') || processedLine.trim().startsWith('* ')) {
|
||||
return <li key={i} dangerouslySetInnerHTML={{ __html: processedLine.trim().substring(2) }} style={{ marginBottom: '4px', fontSize: '0.9rem' }} />;
|
||||
}
|
||||
// Handle headers - make them smaller
|
||||
if (processedLine.startsWith('### ')) {
|
||||
return <Typography key={i} variant="subtitle2" fontWeight={700} sx={{ mt: 1.5, mb: 0.5, color: '#1e293b' }}>{processedLine.substring(4)}</Typography>;
|
||||
}
|
||||
if (processedLine.startsWith('## ')) {
|
||||
return <Typography key={i} variant="subtitle1" fontWeight={700} sx={{ mt: 1.5, mb: 0.5, color: '#0f172a' }}>{processedLine.substring(3)}</Typography>;
|
||||
}
|
||||
// Paragraphs - compact spacing
|
||||
return processedLine.trim() ? <p key={i} dangerouslySetInnerHTML={{ __html: processedLine }} style={{ margin: '4px 0', fontSize: '0.9rem' }} /> : null;
|
||||
});
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<GlassyCard sx={glassyCardSx}>
|
||||
<Stack spacing={3}>
|
||||
<Stack direction="row" justifyContent="space-between" alignItems="center" flexWrap="wrap" gap={2}>
|
||||
<Stack direction="row" alignItems="center" spacing={2} sx={{ flex: 1 }}>
|
||||
<Typography variant="h6" sx={{ display: "flex", alignItems: "center", gap: 1, color: "#0f172a", fontWeight: 700 }}>
|
||||
<InsightsIcon />
|
||||
Research Summary
|
||||
</Typography>
|
||||
|
||||
{/* Research Metadata - Moved alongside title */}
|
||||
<Stack direction="row" spacing={1.5} flexWrap="wrap">
|
||||
{research.searchQueries && research.searchQueries.length > 0 && (
|
||||
<Chip
|
||||
icon={<SearchIcon sx={{ fontSize: "1rem !important" }} />}
|
||||
label={`${research.searchQueries.length} search${research.searchQueries.length > 1 ? "es" : ""}`}
|
||||
size="small"
|
||||
sx={{
|
||||
background: alpha("#667eea", 0.1),
|
||||
color: "#667eea",
|
||||
fontWeight: 600,
|
||||
border: "1px solid rgba(102, 126, 234, 0.2)",
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
{research.searchType && (
|
||||
<Chip
|
||||
label={`${research.searchType.charAt(0).toUpperCase() + research.searchType.slice(1)} search`}
|
||||
size="small"
|
||||
sx={{
|
||||
background: alpha("#10b981", 0.1),
|
||||
color: "#059669",
|
||||
fontWeight: 600,
|
||||
border: "1px solid rgba(16, 185, 129, 0.2)",
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
{research.sourceCount !== undefined && (
|
||||
<Chip
|
||||
label={`${research.sourceCount} source${research.sourceCount !== 1 ? "s" : ""}`}
|
||||
size="small"
|
||||
sx={{
|
||||
background: alpha("#6366f1", 0.1),
|
||||
color: "#4f46e5",
|
||||
fontWeight: 600,
|
||||
border: "1px solid rgba(99, 102, 241, 0.2)",
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
{research.cost !== undefined && (
|
||||
<Chip
|
||||
icon={<AttachMoneyIcon sx={{ fontSize: "0.875rem !important" }} />}
|
||||
label={`$${research.cost.toFixed(3)}`}
|
||||
size="small"
|
||||
sx={{
|
||||
background: alpha("#f59e0b", 0.1),
|
||||
color: "#d97706",
|
||||
fontWeight: 600,
|
||||
border: "1px solid rgba(245, 158, 11, 0.2)",
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</Stack>
|
||||
</Stack>
|
||||
|
||||
<PrimaryButton
|
||||
onClick={onGenerateScript}
|
||||
disabled={!canGenerateScript}
|
||||
startIcon={<EditNoteIcon />}
|
||||
tooltip={!canGenerateScript ? "Complete research to generate script" : "Generate AI-powered script from research"}
|
||||
>
|
||||
Generate Script
|
||||
</PrimaryButton>
|
||||
</Stack>
|
||||
|
||||
<Box sx={{ width: "100%" }}>
|
||||
{/* Main Summary */}
|
||||
{research.summary && (
|
||||
<Paper
|
||||
elevation={0}
|
||||
sx={{
|
||||
p: 2.5,
|
||||
mb: 3,
|
||||
background: "#f8fafc",
|
||||
border: "1px solid rgba(0,0,0,0.06)",
|
||||
borderRadius: 2,
|
||||
}}
|
||||
>
|
||||
<Typography variant="subtitle2" sx={{ mb: 1.5, color: "#64748b", fontWeight: 700, fontSize: "0.75rem", textTransform: "uppercase", letterSpacing: "0.05em", display: "flex", alignItems: "center", gap: 1 }}>
|
||||
<AutoAwesomeIcon fontSize="small" sx={{ color: "#667eea", fontSize: "1rem" }} />
|
||||
Executive Summary
|
||||
</Typography>
|
||||
<Box sx={{
|
||||
lineHeight: 1.6,
|
||||
fontSize: "0.9rem",
|
||||
color: "#334155",
|
||||
"& p": { m: 0, mb: 1 },
|
||||
"& ul": { m: 0, mb: 1, pl: 2.5 },
|
||||
"& li": { mb: 0.5 },
|
||||
"& strong": { color: "#0f172a", fontWeight: 600 }
|
||||
}}>
|
||||
{renderMarkdown(research.summary)}
|
||||
</Box>
|
||||
</Paper>
|
||||
)}
|
||||
|
||||
{/* Deep Insights */}
|
||||
{(research.keyInsights && research.keyInsights.length > 0) ? (
|
||||
<Box sx={{ mb: 4 }}>
|
||||
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||
<ArticleIcon sx={{ color: "#667eea" }} />
|
||||
Deep Insights
|
||||
</Typography>
|
||||
<Stack spacing={2.5}>
|
||||
{research.keyInsights.map((insight: ResearchInsight, idx: number) => (
|
||||
<Paper
|
||||
key={idx}
|
||||
elevation={0}
|
||||
sx={{
|
||||
p: 2.5,
|
||||
background: "#ffffff",
|
||||
border: "1px solid rgba(0,0,0,0.06)",
|
||||
boxShadow: "0 2px 12px rgba(0,0,0,0.03)",
|
||||
borderRadius: 2,
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" justifyContent="space-between" alignItems="flex-start" sx={{ mb: 1.5 }}>
|
||||
<Typography variant="subtitle1" sx={{ color: "#0f172a", fontWeight: 700 }}>
|
||||
{insight.title}
|
||||
</Typography>
|
||||
{insight.source_indices && insight.source_indices.length > 0 && (
|
||||
<Stack direction="row" spacing={0.5}>
|
||||
{insight.source_indices.map(sIdx => {
|
||||
const sourceIdx = sIdx - 1;
|
||||
const fact = research.factCards[sourceIdx];
|
||||
const sourceUrl = fact?.url;
|
||||
const hasUrl = !!sourceUrl;
|
||||
const hue = (sIdx * 47 + 220) % 360;
|
||||
const gradientFrom = `hsl(${hue}, 70%, 55%)`;
|
||||
const gradientTo = `hsl(${(hue + 30) % 360}, 80%, 65%)`;
|
||||
return (
|
||||
<Tooltip
|
||||
key={sIdx}
|
||||
title={hasUrl ? (
|
||||
<Box sx={{ maxWidth: 300, wordBreak: "break-all" }}>
|
||||
<Typography variant="caption" sx={{ color: "#fff", fontWeight: 600 }}>Source {sIdx}</Typography>
|
||||
<br />
|
||||
<Typography variant="caption" sx={{ color: "rgba(255,255,255,0.8)", fontSize: "0.65rem" }}>{sourceUrl}</Typography>
|
||||
</Box>
|
||||
) : `Source ${sIdx}`}
|
||||
arrow
|
||||
placement="top"
|
||||
>
|
||||
<Chip
|
||||
label={hasUrl ? `S${sIdx} ↗` : `S${sIdx}`}
|
||||
size="small"
|
||||
onClick={hasUrl ? () => window.open(sourceUrl, "_blank", "noopener,noreferrer") : undefined}
|
||||
sx={{
|
||||
height: 24,
|
||||
minWidth: 36,
|
||||
fontSize: '0.7rem',
|
||||
fontWeight: 800,
|
||||
fontFamily: "'Inter', 'Roboto', monospace",
|
||||
letterSpacing: "0.02em",
|
||||
border: "none",
|
||||
background: hasUrl
|
||||
? `linear-gradient(135deg, ${gradientFrom}, ${gradientTo})`
|
||||
: `linear-gradient(135deg, ${alpha(gradientFrom, 0.3)}, ${alpha(gradientTo, 0.3)})`,
|
||||
color: hasUrl ? "#fff" : alpha("#fff", 0.7),
|
||||
cursor: hasUrl ? "pointer" : "default",
|
||||
borderRadius: "8px",
|
||||
px: 0.5,
|
||||
boxShadow: hasUrl
|
||||
? `0 2px 8px ${alpha(gradientFrom, 0.35)}, inset 0 1px 0 ${alpha("#fff", 0.2)}`
|
||||
: "none",
|
||||
transition: "all 0.2s ease",
|
||||
"&:hover": hasUrl ? {
|
||||
background: `linear-gradient(135deg, ${gradientTo}, ${gradientFrom})`,
|
||||
boxShadow: `0 4px 14px ${alpha(gradientFrom, 0.5)}, inset 0 1px 0 ${alpha("#fff", 0.3)}`,
|
||||
transform: "translateY(-1px)",
|
||||
} : {},
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
);
|
||||
})}
|
||||
</Stack>
|
||||
)}
|
||||
</Stack>
|
||||
<Box sx={{
|
||||
color: "#475569",
|
||||
lineHeight: 1.7,
|
||||
fontSize: "0.9rem",
|
||||
"& p": { m: 0, mb: 1.5 },
|
||||
"& ul": { m: 0, mb: 1.5, pl: 2 }
|
||||
}}>
|
||||
{renderMarkdown(insight.content)}
|
||||
</Box>
|
||||
</Paper>
|
||||
))}
|
||||
</Stack>
|
||||
</Box>
|
||||
) : (
|
||||
/* Fallback if keyInsights is missing but we have summary paragraphs */
|
||||
research.summary && research.summary.length > 500 && !research.keyInsights && (
|
||||
<Box sx={{ mb: 4 }}>
|
||||
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||
<ArticleIcon sx={{ color: "#667eea" }} />
|
||||
Additional Insights
|
||||
</Typography>
|
||||
<Paper
|
||||
elevation={0}
|
||||
sx={{
|
||||
p: 2.5,
|
||||
background: "#ffffff",
|
||||
border: "1px solid rgba(0,0,0,0.06)",
|
||||
boxShadow: "0 2px 12px rgba(0,0,0,0.03)",
|
||||
borderRadius: 2,
|
||||
}}
|
||||
>
|
||||
<Box sx={{
|
||||
color: "#475569",
|
||||
lineHeight: 1.7,
|
||||
fontSize: "0.9rem",
|
||||
}}>
|
||||
{/* Render parts of summary that might contain insights if structured data is missing */}
|
||||
{renderMarkdown(research.summary.split('\n\n').slice(1).join('\n\n'))}
|
||||
</Box>
|
||||
</Paper>
|
||||
</Box>
|
||||
)
|
||||
)}
|
||||
|
||||
{/* Expert Quotes Section */}
|
||||
{research.expertQuotes && research.expertQuotes.length > 0 && (
|
||||
<Box sx={{ mt: 4, pt: 3, borderTop: "1px solid rgba(0,0,0,0.04)" }}>
|
||||
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||
<FormatQuoteIcon sx={{ color: "#8b5cf6" }} />
|
||||
Expert Quotes ({research.expertQuotes.length})
|
||||
</Typography>
|
||||
<Stack spacing={2}>
|
||||
{research.expertQuotes.map((eq, idx) => (
|
||||
<Paper
|
||||
key={idx}
|
||||
elevation={0}
|
||||
sx={{
|
||||
p: 2.5,
|
||||
background: "linear-gradient(135deg, rgba(139, 92, 246, 0.04) 0%, rgba(99, 102, 241, 0.04) 100%)",
|
||||
border: "1px solid rgba(139, 92, 246, 0.15)",
|
||||
borderLeft: "4px solid #8b5cf6",
|
||||
borderRadius: 2,
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" spacing={1.5} alignItems="flex-start">
|
||||
<FormatQuoteIcon sx={{ color: "#8b5cf6", fontSize: "1.5rem", mt: -0.5, opacity: 0.7 }} />
|
||||
<Box sx={{ flex: 1 }}>
|
||||
<Typography variant="body2" sx={{ color: "#1e293b", fontStyle: "italic", lineHeight: 1.7, fontSize: "0.95rem" }}>
|
||||
“{eq.quote}”
|
||||
</Typography>
|
||||
{eq.source_index !== undefined && (() => {
|
||||
const fact = research.factCards[eq.source_index - 1];
|
||||
const sourceUrl = fact?.url;
|
||||
const hasUrl = !!sourceUrl;
|
||||
const hue = (eq.source_index * 47 + 270) % 360;
|
||||
const gradientFrom = `hsl(${hue}, 70%, 55%)`;
|
||||
const gradientTo = `hsl(${(hue + 30) % 360}, 80%, 65%)`;
|
||||
return (
|
||||
<Box sx={{ mt: 1 }}>
|
||||
<Tooltip title={hasUrl ? (
|
||||
<Box sx={{ maxWidth: 300, wordBreak: "break-all" }}>
|
||||
<Typography variant="caption" sx={{ color: "#fff", fontWeight: 600 }}>Source {eq.source_index}</Typography>
|
||||
<br />
|
||||
<Typography variant="caption" sx={{ color: "rgba(255,255,255,0.8)", fontSize: "0.65rem" }}>{sourceUrl}</Typography>
|
||||
</Box>
|
||||
) : `Source ${eq.source_index}`} arrow placement="top">
|
||||
<Chip
|
||||
label={hasUrl ? `Source ${eq.source_index} ↗` : `Source ${eq.source_index}`}
|
||||
size="small"
|
||||
onClick={hasUrl ? () => window.open(sourceUrl, "_blank", "noopener,noreferrer") : undefined}
|
||||
sx={{
|
||||
height: 24,
|
||||
fontSize: "0.7rem",
|
||||
fontWeight: 800,
|
||||
fontFamily: "'Inter', 'Roboto', monospace",
|
||||
border: "none",
|
||||
background: hasUrl
|
||||
? `linear-gradient(135deg, ${gradientFrom}, ${gradientTo})`
|
||||
: `linear-gradient(135deg, ${alpha(gradientFrom, 0.3)}, ${alpha(gradientTo, 0.3)})`,
|
||||
color: hasUrl ? "#fff" : alpha("#fff", 0.7),
|
||||
cursor: hasUrl ? "pointer" : "default",
|
||||
borderRadius: "8px",
|
||||
px: 1,
|
||||
boxShadow: hasUrl
|
||||
? `0 2px 8px ${alpha(gradientFrom, 0.35)}, inset 0 1px 0 ${alpha("#fff", 0.2)}`
|
||||
: "none",
|
||||
transition: "all 0.2s ease",
|
||||
"&:hover": hasUrl ? {
|
||||
background: `linear-gradient(135deg, ${gradientTo}, ${gradientFrom})`,
|
||||
boxShadow: `0 4px 14px ${alpha(gradientFrom, 0.5)}, inset 0 1px 0 ${alpha("#fff", 0.3)}`,
|
||||
transform: "translateY(-1px)",
|
||||
} : {},
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
</Box>
|
||||
);
|
||||
})()}
|
||||
</Box>
|
||||
</Stack>
|
||||
</Paper>
|
||||
))}
|
||||
</Stack>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{/* Search Queries Used */}
|
||||
{research.searchQueries && research.searchQueries.length > 0 && (
|
||||
<Box sx={{ mt: 4, pt: 3, borderTop: "1px solid rgba(0,0,0,0.04)" }}>
|
||||
<Typography variant="subtitle2" sx={{ mb: 1.5, color: "#64748b", fontWeight: 700, fontSize: "0.7rem", textTransform: "uppercase", letterSpacing: "0.05em" }}>
|
||||
Search Queries Used
|
||||
</Typography>
|
||||
<Stack direction="row" spacing={1} flexWrap="wrap" useFlexGap>
|
||||
{research.searchQueries.map((query, idx) => (
|
||||
<Chip
|
||||
key={idx}
|
||||
label={query}
|
||||
size="small"
|
||||
variant="outlined"
|
||||
sx={{
|
||||
borderColor: "rgba(102, 126, 234, 0.15)",
|
||||
color: "#94a3b8",
|
||||
background: alpha("#f8fafc", 0.3),
|
||||
fontSize: "0.7rem",
|
||||
borderRadius: 1,
|
||||
}}
|
||||
/>
|
||||
))}
|
||||
</Stack>
|
||||
</Box>
|
||||
)}
|
||||
</Box>
|
||||
|
||||
{research.factCards.length > 0 && (
|
||||
<>
|
||||
<Divider sx={{ borderColor: "rgba(0,0,0,0.08)" }} />
|
||||
<Stack direction="row" justifyContent="space-between" alignItems="center" sx={{ mb: 1.5, flexWrap: "wrap", gap: 1 }}>
|
||||
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600 }}>
|
||||
Research Sources & Facts ({research.factCards.length})
|
||||
</Typography>
|
||||
<Typography variant="caption" sx={{ color: "#64748b", fontSize: "0.75rem" }}>
|
||||
Click to expand • Hover to see source
|
||||
</Typography>
|
||||
</Stack>
|
||||
<Box
|
||||
sx={{
|
||||
display: "grid",
|
||||
gridTemplateColumns: { xs: "1fr", sm: "repeat(2, 1fr)", md: "repeat(3, 1fr)", lg: "repeat(4, 1fr)" },
|
||||
gap: 1.5,
|
||||
width: "100%",
|
||||
overflow: "hidden",
|
||||
}}
|
||||
>
|
||||
{research.factCards.map((fact) => (
|
||||
<FactCard key={fact.id} fact={fact} />
|
||||
))}
|
||||
</Box>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Listener CTA Section */}
|
||||
{research.listenerCta && research.listenerCta.length > 0 && (
|
||||
<>
|
||||
<Divider sx={{ borderColor: "rgba(0,0,0,0.08)" }} />
|
||||
<Box>
|
||||
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||
<CampaignIcon sx={{ color: "#f59e0b" }} />
|
||||
Listener Call-to-Action Ideas ({research.listenerCta.length})
|
||||
</Typography>
|
||||
<Stack spacing={1.5}>
|
||||
{research.listenerCta.map((cta, idx) => (
|
||||
<Paper
|
||||
key={idx}
|
||||
elevation={0}
|
||||
sx={{
|
||||
p: 2,
|
||||
background: "linear-gradient(135deg, rgba(245, 158, 11, 0.05) 0%, rgba(251, 191, 36, 0.05) 100%)",
|
||||
border: "1px solid rgba(245, 158, 11, 0.15)",
|
||||
borderRadius: 2,
|
||||
display: "flex",
|
||||
alignItems: "flex-start",
|
||||
gap: 1.5,
|
||||
}}
|
||||
>
|
||||
<Chip
|
||||
label={`#${idx + 1}`}
|
||||
size="small"
|
||||
sx={{
|
||||
bgcolor: alpha("#f59e0b", 0.15),
|
||||
color: "#b45309",
|
||||
fontWeight: 700,
|
||||
fontSize: "0.7rem",
|
||||
height: 24,
|
||||
minWidth: 32,
|
||||
}}
|
||||
/>
|
||||
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.6, flex: 1, pt: 0.2 }}>
|
||||
{cta}
|
||||
</Typography>
|
||||
</Paper>
|
||||
))}
|
||||
</Stack>
|
||||
</Box>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Mapped Angles Section */}
|
||||
{research.mappedAngles && research.mappedAngles.length > 0 && (
|
||||
<>
|
||||
<Divider sx={{ borderColor: "rgba(0,0,0,0.08)" }} />
|
||||
<Box>
|
||||
<Typography variant="h6" sx={{ mb: 2, color: "#0f172a", fontWeight: 700, display: "flex", alignItems: "center", gap: 1 }}>
|
||||
<ExploreIcon sx={{ color: "#06b6d4" }} />
|
||||
Content Angles ({research.mappedAngles.length})
|
||||
</Typography>
|
||||
<Stack spacing={2}>
|
||||
{research.mappedAngles.map((angle, idx) => (
|
||||
<Paper
|
||||
key={idx}
|
||||
elevation={0}
|
||||
sx={{
|
||||
p: 2.5,
|
||||
background: "#ffffff",
|
||||
border: "1px solid rgba(0,0,0,0.06)",
|
||||
borderLeft: "4px solid #06b6d4",
|
||||
boxShadow: "0 2px 12px rgba(0,0,0,0.03)",
|
||||
borderRadius: 2,
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" justifyContent="space-between" alignItems="flex-start" sx={{ mb: 1 }}>
|
||||
<Typography variant="subtitle1" sx={{ color: "#0f172a", fontWeight: 700 }}>
|
||||
{angle.title}
|
||||
</Typography>
|
||||
{angle.mappedFactIds && angle.mappedFactIds.length > 0 && (
|
||||
<Stack direction="row" spacing={0.5}>
|
||||
{angle.mappedFactIds.slice(0, 4).map((fid: string) => (
|
||||
<Chip
|
||||
key={fid}
|
||||
label={fid.replace("fact_", "F")}
|
||||
size="small"
|
||||
variant="outlined"
|
||||
sx={{
|
||||
height: 18,
|
||||
fontSize: "0.6rem",
|
||||
fontWeight: 700,
|
||||
borderColor: alpha("#06b6d4", 0.3),
|
||||
color: "#06b6d4",
|
||||
bgcolor: alpha("#06b6d4", 0.05),
|
||||
}}
|
||||
/>
|
||||
))}
|
||||
{angle.mappedFactIds.length > 4 && (
|
||||
<Chip
|
||||
label={`+${angle.mappedFactIds.length - 4}`}
|
||||
size="small"
|
||||
sx={{ height: 18, fontSize: "0.6rem", color: "#64748b" }}
|
||||
/>
|
||||
)}
|
||||
</Stack>
|
||||
)}
|
||||
</Stack>
|
||||
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7, fontSize: "0.9rem" }}>
|
||||
{angle.why}
|
||||
</Typography>
|
||||
</Paper>
|
||||
))}
|
||||
</Stack>
|
||||
</Box>
|
||||
</>
|
||||
)}
|
||||
</Stack>
|
||||
</GlassyCard>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -1,811 +0,0 @@
|
||||
import React, { useState, useEffect } from "react";
|
||||
import { Stack, Box, Typography, Divider, Chip, alpha, CircularProgress, LinearProgress, IconButton, Tooltip } from "@mui/material";
|
||||
import {
|
||||
EditNote as EditNoteIcon,
|
||||
CheckCircle as CheckCircleIcon,
|
||||
RadioButtonUnchecked as RadioButtonUncheckedIcon,
|
||||
VolumeUp as VolumeUpIcon,
|
||||
PlayArrow as PlayArrowIcon,
|
||||
Image as ImageIcon,
|
||||
Delete as DeleteIcon,
|
||||
} from "@mui/icons-material";
|
||||
import { Scene, Line, Knobs } from "../types";
|
||||
import { GlassyCard, glassyCardSx, PrimaryButton } from "../ui";
|
||||
import { LineEditor } from "./LineEditor";
|
||||
import { ImageRegenerateModal, ImageGenerationSettings } from "./ImageRegenerateModal";
|
||||
import { AudioRegenerateModal, AudioGenerationSettings } from "./AudioRegenerateModal";
|
||||
import { podcastApi } from "../../../services/podcastApi";
|
||||
import { aiApiClient } from "../../../api/client";
|
||||
import { getCachedMedia, setCachedMedia } from "../../../utils/mediaCache";
|
||||
|
||||
interface SceneEditorProps {
|
||||
scene: Scene;
|
||||
onUpdateScene: (s: Scene) => void;
|
||||
onApprove: (id: string) => Promise<void>;
|
||||
onDelete: (sceneId: string) => void;
|
||||
knobs: Knobs;
|
||||
approvingSceneId?: string | null;
|
||||
generatingAudioId?: string | null;
|
||||
onAudioGenerationStart?: (sceneId: string) => void;
|
||||
onAudioGenerated?: (sceneId: string, audioUrl: string) => void;
|
||||
idea?: string; // Podcast idea for image generation context
|
||||
avatarUrl?: string | null; // Base avatar URL for consistent scene image generation
|
||||
totalScenes?: number; // Total number of scenes in the script
|
||||
}
|
||||
|
||||
export const SceneEditor: React.FC<SceneEditorProps> = ({
|
||||
scene,
|
||||
onUpdateScene,
|
||||
onApprove,
|
||||
onDelete,
|
||||
knobs,
|
||||
approvingSceneId,
|
||||
generatingAudioId,
|
||||
onAudioGenerationStart,
|
||||
onAudioGenerated,
|
||||
idea,
|
||||
avatarUrl,
|
||||
totalScenes,
|
||||
}) => {
|
||||
const [localGenerating, setLocalGenerating] = useState(false);
|
||||
const [generatingImage, setGeneratingImage] = useState(false);
|
||||
const [imageGenerationStatus, setImageGenerationStatus] = useState<string>("");
|
||||
const [imageGenerationProgress, setImageGenerationProgress] = useState<number>(0);
|
||||
const [audioBlobUrl, setAudioBlobUrl] = useState<string | null>(null);
|
||||
const [imageBlobUrl, setImageBlobUrl] = useState<string | null>(null);
|
||||
const [imageLoading, setImageLoading] = useState(false);
|
||||
const [showRegenerateModal, setShowRegenerateModal] = useState(false);
|
||||
const [showAudioModal, setShowAudioModal] = useState(false);
|
||||
const [audioSettings, setAudioSettings] = useState<AudioGenerationSettings>({
|
||||
voiceId: "Wise_Woman",
|
||||
speed: 1.0,
|
||||
volume: 1.0,
|
||||
pitch: 0.0,
|
||||
emotion: scene.emotion || "neutral",
|
||||
englishNormalization: true,
|
||||
sampleRate: 24000,
|
||||
bitrate: 64000,
|
||||
channel: "1",
|
||||
format: "mp3",
|
||||
languageBoost: "auto",
|
||||
});
|
||||
|
||||
// Load audio as blob when audioUrl is available
|
||||
useEffect(() => {
|
||||
if (!scene.audioUrl) {
|
||||
// Clean up blob URL if audioUrl is removed
|
||||
setAudioBlobUrl((currentBlobUrl) => {
|
||||
if (currentBlobUrl) {
|
||||
URL.revokeObjectURL(currentBlobUrl);
|
||||
}
|
||||
return null;
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
let isMounted = true;
|
||||
const currentAudioUrl = scene.audioUrl; // Capture current value
|
||||
|
||||
const loadAudioBlob = async () => {
|
||||
try {
|
||||
// Normalize path
|
||||
let audioPath = currentAudioUrl.startsWith('/') ? currentAudioUrl : `/${currentAudioUrl}`;
|
||||
|
||||
// Convert /api/story/audio/ to /api/podcast/audio/ if needed
|
||||
if (audioPath.includes('/api/story/audio/')) {
|
||||
const filename = audioPath.split('/api/story/audio/').pop() || '';
|
||||
audioPath = `/api/podcast/audio/${filename}`;
|
||||
}
|
||||
|
||||
// Ensure it's a podcast audio endpoint
|
||||
if (!audioPath.includes('/api/podcast/audio/')) {
|
||||
const filename = audioPath.split('/').pop() || currentAudioUrl;
|
||||
audioPath = `/api/podcast/audio/${filename}`;
|
||||
}
|
||||
|
||||
// Remove query parameters if present
|
||||
audioPath = audioPath.split('?')[0];
|
||||
|
||||
const response = await aiApiClient.get(audioPath, {
|
||||
responseType: 'blob',
|
||||
});
|
||||
|
||||
if (!isMounted) {
|
||||
// Component unmounted or audioUrl changed, don't set blob URL
|
||||
return;
|
||||
}
|
||||
|
||||
// Double-check that audioUrl hasn't changed
|
||||
if (scene.audioUrl !== currentAudioUrl) {
|
||||
return;
|
||||
}
|
||||
|
||||
const blob = response.data;
|
||||
const blobUrl = URL.createObjectURL(blob);
|
||||
|
||||
setAudioBlobUrl((prevBlobUrl) => {
|
||||
// Clean up previous blob URL if exists
|
||||
if (prevBlobUrl && prevBlobUrl !== blobUrl) {
|
||||
URL.revokeObjectURL(prevBlobUrl);
|
||||
}
|
||||
return blobUrl;
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(`Failed to load audio blob for scene ${scene.id}:`, error);
|
||||
// Don't set blob URL on error - will show error state
|
||||
}
|
||||
};
|
||||
|
||||
loadAudioBlob();
|
||||
|
||||
// Cleanup: only mark as unmounted, don't revoke blob URL here
|
||||
// The blob URL will be cleaned up when audioUrl changes (new effect) or component unmounts
|
||||
return () => {
|
||||
isMounted = false;
|
||||
};
|
||||
}, [scene.audioUrl, scene.id]);
|
||||
|
||||
// Load image as blob when imageUrl is available
|
||||
useEffect(() => {
|
||||
if (!scene.imageUrl) {
|
||||
// Clean up blob URL if imageUrl is removed
|
||||
setImageBlobUrl((currentBlobUrl) => {
|
||||
if (currentBlobUrl && currentBlobUrl.startsWith('blob:')) {
|
||||
URL.revokeObjectURL(currentBlobUrl);
|
||||
}
|
||||
return null;
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// Check cache first with scene context
|
||||
const cachedUrl = getCachedMedia(scene.imageUrl, scene.id);
|
||||
if (cachedUrl) {
|
||||
console.log('[SceneEditor] Using cached image:', scene.imageUrl, `(scene: ${scene.id})`);
|
||||
setImageBlobUrl(cachedUrl);
|
||||
setImageLoading(false);
|
||||
return;
|
||||
}
|
||||
|
||||
let isMounted = true;
|
||||
const currentImageUrl = scene.imageUrl; // Capture current value
|
||||
|
||||
const loadImageBlob = async () => {
|
||||
try {
|
||||
setImageLoading(true);
|
||||
|
||||
// Check cache again in case it was loaded while we were waiting
|
||||
const cachedUrl = getCachedMedia(currentImageUrl, scene.id);
|
||||
if (cachedUrl) {
|
||||
if (isMounted) {
|
||||
setImageBlobUrl(cachedUrl);
|
||||
setImageLoading(false);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[SceneEditor] Loading image blob for:', currentImageUrl);
|
||||
|
||||
// Normalize path
|
||||
let imagePath = currentImageUrl.startsWith('/') ? currentImageUrl : `/${currentImageUrl}`;
|
||||
|
||||
// Convert /api/story/images/ to /api/podcast/images/ if needed
|
||||
if (imagePath.includes('/api/story/images/')) {
|
||||
const filename = imagePath.split('/api/story/images/').pop() || '';
|
||||
imagePath = `/api/podcast/images/${filename}`;
|
||||
}
|
||||
|
||||
// Ensure it's a podcast image endpoint
|
||||
if (!imagePath.includes('/api/podcast/images/')) {
|
||||
const filename = imagePath.split('/').pop() || currentImageUrl;
|
||||
imagePath = `/api/podcast/images/${filename}`;
|
||||
}
|
||||
|
||||
// Remove query parameters if present
|
||||
imagePath = imagePath.split('?')[0];
|
||||
|
||||
const response = await aiApiClient.get(imagePath, {
|
||||
responseType: 'blob',
|
||||
});
|
||||
|
||||
if (!isMounted) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Double-check that imageUrl hasn't changed
|
||||
if (scene.imageUrl !== currentImageUrl) {
|
||||
return;
|
||||
}
|
||||
|
||||
const blob = response.data;
|
||||
const blobUrl = URL.createObjectURL(blob);
|
||||
|
||||
// Cache the blob URL with scene context
|
||||
setCachedMedia(currentImageUrl, blobUrl, 'image', blob.size, scene.id);
|
||||
|
||||
setImageBlobUrl((prevBlobUrl) => {
|
||||
// Clean up previous blob URL if exists
|
||||
if (prevBlobUrl && prevBlobUrl !== blobUrl && prevBlobUrl.startsWith('blob:')) {
|
||||
URL.revokeObjectURL(prevBlobUrl);
|
||||
}
|
||||
return blobUrl;
|
||||
});
|
||||
console.log('[SceneEditor] Image blob loaded and cached successfully:', currentImageUrl);
|
||||
} catch (error) {
|
||||
console.error('[SceneEditor] Failed to load image blob:', error);
|
||||
if (isMounted) {
|
||||
// Try adding query token as fallback
|
||||
try {
|
||||
const token = localStorage.getItem('clerk_dashboard_token') || '';
|
||||
if (token) {
|
||||
const urlWithToken = `${currentImageUrl}?token=${encodeURIComponent(token)}`;
|
||||
setImageBlobUrl(urlWithToken);
|
||||
setCachedMedia(currentImageUrl, urlWithToken, 'image', undefined, scene.id);
|
||||
}
|
||||
} catch (fallbackError) {
|
||||
console.error('[SceneEditor] Fallback image loading failed:', fallbackError);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
if (isMounted) {
|
||||
setImageLoading(false);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
loadImageBlob();
|
||||
|
||||
return () => {
|
||||
isMounted = false;
|
||||
// Don't cleanup blob URL here - let the cache handle it
|
||||
};
|
||||
}, [scene.imageUrl]);
|
||||
|
||||
const updateLine = (updatedLine: Line) => {
|
||||
const updated = { ...scene, lines: scene.lines.map((l) => (l.id === updatedLine.id ? updatedLine : l)) };
|
||||
onUpdateScene(updated);
|
||||
};
|
||||
|
||||
const approving = approvingSceneId === scene.id;
|
||||
const generating = generatingAudioId === scene.id || localGenerating;
|
||||
const hasAudio = Boolean(scene.audioUrl && audioBlobUrl);
|
||||
const hasImage = Boolean(scene.imageUrl);
|
||||
|
||||
const handleApproveAndGenerate = async (settings?: AudioGenerationSettings) => {
|
||||
const wasAlreadyApproved = scene.approved;
|
||||
const sceneId = scene.id;
|
||||
|
||||
try {
|
||||
// Set generating state
|
||||
setLocalGenerating(true);
|
||||
if (onAudioGenerationStart) {
|
||||
onAudioGenerationStart(sceneId);
|
||||
}
|
||||
|
||||
// If scene is not approved yet, approve it first
|
||||
// This will update the parent script state
|
||||
if (!scene.approved) {
|
||||
await onApprove(sceneId);
|
||||
// The parent's approveScene already updated the script state
|
||||
// We need to wait for React to propagate the updated scene prop
|
||||
// For now, we'll update it locally too to ensure UI updates immediately
|
||||
onUpdateScene({ ...scene, approved: true });
|
||||
}
|
||||
|
||||
// Use the current scene (which should now be approved)
|
||||
// If scene prop hasn't updated yet, use the local update we just made
|
||||
const currentScene = { ...scene, approved: true };
|
||||
|
||||
// Generate audio
|
||||
const effectiveSettings = settings || audioSettings;
|
||||
const result = await podcastApi.renderSceneAudio({
|
||||
scene: currentScene,
|
||||
voiceId: effectiveSettings.voiceId || "Wise_Woman",
|
||||
emotion: effectiveSettings.emotion || scene.emotion || knobs.voice_emotion || "neutral",
|
||||
speed: effectiveSettings.speed ?? knobs.voice_speed ?? 1.0,
|
||||
volume: effectiveSettings.volume ?? 1.0,
|
||||
pitch: effectiveSettings.pitch ?? 0.0,
|
||||
englishNormalization: effectiveSettings.englishNormalization ?? true,
|
||||
sampleRate: effectiveSettings.sampleRate,
|
||||
bitrate: effectiveSettings.bitrate,
|
||||
channel: effectiveSettings.channel,
|
||||
format: effectiveSettings.format,
|
||||
languageBoost: effectiveSettings.languageBoost,
|
||||
});
|
||||
|
||||
// Update scene with audio URL and ensure approved state
|
||||
// This will sync with parent script state
|
||||
const updatedScene = { ...currentScene, audioUrl: result.audioUrl, approved: true };
|
||||
onUpdateScene(updatedScene);
|
||||
|
||||
if (onAudioGenerated) {
|
||||
onAudioGenerated(sceneId, result.audioUrl);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to approve and generate audio:", error);
|
||||
// On error, revert approval only if we just approved it in this call
|
||||
if (!wasAlreadyApproved) {
|
||||
onUpdateScene({ ...scene, approved: false, audioUrl: undefined });
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
setLocalGenerating(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleGenerateImage = async (settings?: ImageGenerationSettings) => {
|
||||
const sceneId = scene.id;
|
||||
const startTime = Date.now();
|
||||
let progressInterval: NodeJS.Timeout | null = null;
|
||||
|
||||
try {
|
||||
setGeneratingImage(true);
|
||||
setShowRegenerateModal(false);
|
||||
setImageGenerationStatus("Submitting image generation request...");
|
||||
setImageGenerationProgress(10);
|
||||
|
||||
// Build scene content from lines for context
|
||||
const sceneContent = scene.lines.map((line) => line.text).join(" ");
|
||||
|
||||
// Log avatar URL for debugging
|
||||
console.log("[SceneEditor] Generating image with avatarUrl:", avatarUrl);
|
||||
console.log("[SceneEditor] Custom settings:", settings);
|
||||
|
||||
// Simulate progress updates during API call
|
||||
progressInterval = setInterval(() => {
|
||||
const elapsed = Date.now() - startTime;
|
||||
const seconds = Math.floor(elapsed / 1000);
|
||||
|
||||
// Update status based on elapsed time
|
||||
if (seconds < 5) {
|
||||
setImageGenerationStatus("Submitting request to AI service...");
|
||||
setImageGenerationProgress(15);
|
||||
} else if (seconds < 15) {
|
||||
setImageGenerationStatus("AI is generating your image...");
|
||||
setImageGenerationProgress(30);
|
||||
} else if (seconds < 30) {
|
||||
setImageGenerationStatus("Creating character-consistent scene image...");
|
||||
setImageGenerationProgress(50);
|
||||
} else if (seconds < 60) {
|
||||
setImageGenerationStatus("Rendering image details...");
|
||||
setImageGenerationProgress(70);
|
||||
} else {
|
||||
setImageGenerationStatus(`Processing... (${seconds}s elapsed)`);
|
||||
setImageGenerationProgress(Math.min(90, 50 + (seconds - 30) / 2));
|
||||
}
|
||||
}, 1000);
|
||||
|
||||
const result = await podcastApi.generateSceneImage({
|
||||
sceneId: scene.id,
|
||||
sceneTitle: scene.title,
|
||||
sceneContent: sceneContent,
|
||||
baseAvatarUrl: avatarUrl || undefined, // Pass base avatar URL for character consistency
|
||||
idea: idea,
|
||||
width: 1024,
|
||||
height: 1024,
|
||||
// Pass custom settings if provided
|
||||
customPrompt: settings?.prompt,
|
||||
style: settings?.style,
|
||||
renderingSpeed: settings?.renderingSpeed,
|
||||
aspectRatio: settings?.aspectRatio,
|
||||
});
|
||||
|
||||
if (progressInterval) {
|
||||
clearInterval(progressInterval);
|
||||
progressInterval = null;
|
||||
}
|
||||
|
||||
setImageGenerationStatus("Finalizing image...");
|
||||
setImageGenerationProgress(95);
|
||||
|
||||
// Update scene with image URL
|
||||
const updatedScene = { ...scene, imageUrl: result.image_url };
|
||||
onUpdateScene(updatedScene);
|
||||
|
||||
const elapsed = Math.floor((Date.now() - startTime) / 1000);
|
||||
setImageGenerationStatus(`Image generated successfully in ${elapsed}s`);
|
||||
setImageGenerationProgress(100);
|
||||
|
||||
// Clear status after a moment
|
||||
setTimeout(() => {
|
||||
setImageGenerationStatus("");
|
||||
setImageGenerationProgress(0);
|
||||
}, 2000);
|
||||
} catch (error: any) {
|
||||
// Clear interval on error
|
||||
if (progressInterval) {
|
||||
clearInterval(progressInterval);
|
||||
progressInterval = null;
|
||||
}
|
||||
|
||||
console.error("Failed to generate image:", error);
|
||||
// Extract error message from response if available
|
||||
const errorMessage = error?.response?.data?.detail?.message
|
||||
|| error?.response?.data?.detail?.error
|
||||
|| error?.response?.data?.detail
|
||||
|| error?.message
|
||||
|| "Failed to generate image. Please try again.";
|
||||
console.error("Error details:", {
|
||||
status: error?.response?.status,
|
||||
statusText: error?.response?.statusText,
|
||||
data: error?.response?.data,
|
||||
message: errorMessage,
|
||||
});
|
||||
|
||||
setImageGenerationStatus(`Error: ${errorMessage}`);
|
||||
setImageGenerationProgress(0);
|
||||
|
||||
// Show user-friendly error message
|
||||
alert(`Image generation failed: ${errorMessage}`);
|
||||
throw error;
|
||||
} finally {
|
||||
// Ensure interval is cleared
|
||||
if (progressInterval) {
|
||||
clearInterval(progressInterval);
|
||||
}
|
||||
setGeneratingImage(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleRegenerateClick = () => {
|
||||
setShowRegenerateModal(true);
|
||||
};
|
||||
|
||||
const handleAudioRegenerateClick = () => {
|
||||
if (hasAudio) {
|
||||
setShowAudioModal(true);
|
||||
} else {
|
||||
handleApproveAndGenerate(audioSettings);
|
||||
}
|
||||
};
|
||||
|
||||
const handleAudioRegenerate = (settings: AudioGenerationSettings) => {
|
||||
setAudioSettings(settings);
|
||||
setShowAudioModal(false);
|
||||
handleApproveAndGenerate(settings);
|
||||
};
|
||||
|
||||
return (
|
||||
<GlassyCard sx={glassyCardSx}>
|
||||
<Stack spacing={2.5}>
|
||||
<Stack direction="row" justifyContent="space-between" alignItems="flex-start">
|
||||
<Box sx={{ flex: 1 }}>
|
||||
<Typography
|
||||
variant="h6"
|
||||
sx={{
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
gap: 1.5,
|
||||
mb: 1,
|
||||
color: "#0f172a",
|
||||
fontWeight: 600,
|
||||
fontSize: "1.25rem",
|
||||
letterSpacing: "-0.01em",
|
||||
}}
|
||||
>
|
||||
<EditNoteIcon fontSize="small" sx={{ color: "#667eea", fontSize: "1.5rem" }} />
|
||||
{scene.title}
|
||||
</Typography>
|
||||
<Stack direction="row" spacing={1.5} alignItems="center" flexWrap="wrap">
|
||||
<Chip
|
||||
icon={scene.approved ? <CheckCircleIcon /> : <RadioButtonUncheckedIcon />}
|
||||
label={scene.approved ? "Approved" : "Pending Approval"}
|
||||
size="small"
|
||||
color={scene.approved ? "success" : "warning"}
|
||||
sx={{
|
||||
background: scene.approved
|
||||
? "linear-gradient(135deg, rgba(16, 185, 129, 0.12) 0%, rgba(5, 150, 105, 0.12) 100%)"
|
||||
: "linear-gradient(135deg, rgba(245, 158, 11, 0.12) 0%, rgba(217, 119, 6, 0.12) 100%)",
|
||||
color: scene.approved ? "#059669" : "#d97706",
|
||||
border: scene.approved
|
||||
? "1px solid rgba(16, 185, 129, 0.25)"
|
||||
: "1px solid rgba(245, 158, 11, 0.25)",
|
||||
fontWeight: 600,
|
||||
fontSize: "0.75rem",
|
||||
height: 26,
|
||||
boxShadow: "0 1px 2px rgba(0, 0, 0, 0.05)",
|
||||
}}
|
||||
/>
|
||||
<Typography variant="caption" sx={{ color: "#64748b", fontWeight: 500, fontSize: "0.8125rem" }}>
|
||||
Duration: {scene.duration}s
|
||||
</Typography>
|
||||
</Stack>
|
||||
</Box>
|
||||
<Stack direction="row" spacing={1.5} flexWrap="wrap" useFlexGap>
|
||||
<PrimaryButton
|
||||
onClick={handleAudioRegenerateClick}
|
||||
disabled={approving || generating}
|
||||
loading={approving || generating}
|
||||
startIcon={
|
||||
hasAudio && !generating ? (
|
||||
<VolumeUpIcon />
|
||||
) : generating ? (
|
||||
<CircularProgress size={16} sx={{ color: "white" }} />
|
||||
) : (
|
||||
<PlayArrowIcon />
|
||||
)
|
||||
}
|
||||
tooltip={
|
||||
hasAudio && !generating
|
||||
? "Regenerate audio for this scene with custom settings"
|
||||
: generating
|
||||
? "Generating audio..."
|
||||
: scene.approved
|
||||
? "Generate audio for this scene"
|
||||
: "Approve scene and generate audio"
|
||||
}
|
||||
sx={{
|
||||
minWidth: 200,
|
||||
}}
|
||||
>
|
||||
{hasAudio && !generating
|
||||
? "Regenerate Audio"
|
||||
: generating
|
||||
? "Generating Audio..."
|
||||
: scene.approved
|
||||
? "Generate Audio"
|
||||
: "Approve & Generate Audio"}
|
||||
</PrimaryButton>
|
||||
<PrimaryButton
|
||||
onClick={hasImage ? handleRegenerateClick : () => handleGenerateImage()}
|
||||
disabled={generatingImage}
|
||||
loading={generatingImage}
|
||||
startIcon={
|
||||
hasImage && !generatingImage ? (
|
||||
<ImageIcon />
|
||||
) : generatingImage ? (
|
||||
<CircularProgress size={16} sx={{ color: "white" }} />
|
||||
) : (
|
||||
<ImageIcon />
|
||||
)
|
||||
}
|
||||
tooltip={
|
||||
hasImage
|
||||
? "Regenerate image for this scene"
|
||||
: generatingImage
|
||||
? "Generating image..."
|
||||
: "Generate image for video (optional)"
|
||||
}
|
||||
sx={{
|
||||
minWidth: 180,
|
||||
background: hasImage
|
||||
? "linear-gradient(135deg, #10b981 0%, #059669 100%)"
|
||||
: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||
"&:hover": {
|
||||
background: hasImage
|
||||
? "linear-gradient(135deg, #059669 0%, #047857 100%)"
|
||||
: "linear-gradient(135deg, #764ba2 0%, #667eea 100%)",
|
||||
},
|
||||
}}
|
||||
>
|
||||
{hasImage && !generatingImage
|
||||
? "Regenerate Image"
|
||||
: generatingImage
|
||||
? "Generating Image..."
|
||||
: "Generate Image"}
|
||||
</PrimaryButton>
|
||||
|
||||
<Tooltip title={totalScenes && totalScenes <= 1 ? "Cannot delete the last scene" : "Delete this scene"}>
|
||||
<IconButton
|
||||
onClick={() => onDelete(scene.id)}
|
||||
disabled={approving || generating || (totalScenes !== undefined && totalScenes <= 1)}
|
||||
sx={{
|
||||
color: "#ef4444",
|
||||
backgroundColor: "rgba(239, 68, 68, 0.1)",
|
||||
border: "1px solid rgba(239, 68, 68, 0.2)",
|
||||
borderRadius: 2,
|
||||
padding: 1.5,
|
||||
"&:hover": {
|
||||
backgroundColor: "rgba(239, 68, 68, 0.15)",
|
||||
borderColor: "rgba(239, 68, 68, 0.3)",
|
||||
},
|
||||
"&:disabled": {
|
||||
backgroundColor: "rgba(156, 163, 175, 0.1)",
|
||||
borderColor: "rgba(156, 163, 175, 0.2)",
|
||||
color: "#9ca3af",
|
||||
},
|
||||
}}
|
||||
>
|
||||
<DeleteIcon sx={{ fontSize: "1.25rem" }} />
|
||||
</IconButton>
|
||||
</Tooltip>
|
||||
</Stack>
|
||||
</Stack>
|
||||
|
||||
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1 }} />
|
||||
|
||||
<Stack spacing={2}>
|
||||
{scene.lines.map((line) => (
|
||||
<LineEditor key={line.id} line={line} onChange={updateLine} />
|
||||
))}
|
||||
</Stack>
|
||||
|
||||
{scene.audioUrl && (
|
||||
<>
|
||||
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1, mt: 1 }} />
|
||||
<Box
|
||||
sx={{
|
||||
p: 2,
|
||||
background: hasAudio
|
||||
? "linear-gradient(135deg, rgba(16, 185, 129, 0.08) 0%, rgba(5, 150, 105, 0.08) 100%)"
|
||||
: "linear-gradient(135deg, rgba(245, 158, 11, 0.08) 0%, rgba(217, 119, 6, 0.08) 100%)",
|
||||
borderRadius: 2,
|
||||
border: hasAudio
|
||||
? "1px solid rgba(16, 185, 129, 0.2)"
|
||||
: "1px solid rgba(245, 158, 11, 0.2)",
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" alignItems="center" spacing={1.5} sx={{ mb: 1.5 }}>
|
||||
<VolumeUpIcon sx={{ color: hasAudio ? "#059669" : "#d97706", fontSize: "1.25rem" }} />
|
||||
<Typography variant="subtitle2" sx={{ color: hasAudio ? "#059669" : "#d97706", fontWeight: 600 }}>
|
||||
{hasAudio ? "Audio Generated" : "Loading Audio..."}
|
||||
</Typography>
|
||||
</Stack>
|
||||
{hasAudio && audioBlobUrl ? (
|
||||
<audio controls style={{ width: "100%", borderRadius: 8 }}>
|
||||
<source src={audioBlobUrl} type="audio/mpeg" />
|
||||
Your browser does not support the audio element.
|
||||
</audio>
|
||||
) : (
|
||||
<Box sx={{ display: "flex", alignItems: "center", justifyContent: "center", py: 2 }}>
|
||||
<CircularProgress size={24} sx={{ color: "#d97706" }} />
|
||||
</Box>
|
||||
)}
|
||||
</Box>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Image Generation Progress - Show when generating */}
|
||||
{generatingImage && (
|
||||
<>
|
||||
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1, mt: 1 }} />
|
||||
<Box
|
||||
sx={{
|
||||
p: 2,
|
||||
background: "linear-gradient(135deg, rgba(102, 126, 234, 0.08) 0%, rgba(118, 75, 162, 0.08) 100%)",
|
||||
borderRadius: 2,
|
||||
border: "1px solid rgba(102, 126, 234, 0.2)",
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" alignItems="center" spacing={1.5} sx={{ mb: 1.5 }}>
|
||||
<ImageIcon sx={{ color: "#667eea", fontSize: "1.25rem" }} />
|
||||
<Typography variant="subtitle2" sx={{ color: "#667eea", fontWeight: 600 }}>
|
||||
Generating Image...
|
||||
</Typography>
|
||||
</Stack>
|
||||
|
||||
{/* Progress Bar */}
|
||||
<Box sx={{ mb: 1.5 }}>
|
||||
<LinearProgress
|
||||
variant="determinate"
|
||||
value={imageGenerationProgress}
|
||||
sx={{
|
||||
height: 8,
|
||||
borderRadius: 4,
|
||||
backgroundColor: alpha("#667eea", 0.1),
|
||||
"& .MuiLinearProgress-bar": {
|
||||
backgroundColor: "#667eea",
|
||||
borderRadius: 4,
|
||||
}
|
||||
}}
|
||||
/>
|
||||
<Typography variant="caption" sx={{ color: "#667eea", mt: 0.5, display: "block", textAlign: "right" }}>
|
||||
{imageGenerationProgress}%
|
||||
</Typography>
|
||||
</Box>
|
||||
|
||||
{/* Status Message */}
|
||||
{imageGenerationStatus && (
|
||||
<Typography variant="body2" sx={{ color: "#667eea", fontSize: "0.875rem", lineHeight: 1.6, mb: 1 }}>
|
||||
{imageGenerationStatus}
|
||||
</Typography>
|
||||
)}
|
||||
|
||||
{/* Spinner */}
|
||||
<Box sx={{ display: "flex", alignItems: "center", justifyContent: "center", mt: 1 }}>
|
||||
<CircularProgress size={32} sx={{ color: "#667eea" }} />
|
||||
</Box>
|
||||
</Box>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Generated Image Display - Show when image exists and not generating */}
|
||||
{scene.imageUrl && !generatingImage && (
|
||||
<>
|
||||
<Divider sx={{ borderColor: "rgba(15, 23, 42, 0.08)", borderWidth: 1, mt: 1 }} />
|
||||
<Box
|
||||
sx={{
|
||||
p: 2,
|
||||
background: imageBlobUrl && !imageLoading
|
||||
? "linear-gradient(135deg, rgba(102, 126, 234, 0.08) 0%, rgba(118, 75, 162, 0.08) 100%)"
|
||||
: "linear-gradient(135deg, rgba(245, 158, 11, 0.08) 0%, rgba(217, 119, 6, 0.08) 100%)",
|
||||
borderRadius: 2,
|
||||
border: imageBlobUrl && !imageLoading
|
||||
? "1px solid rgba(102, 126, 234, 0.2)"
|
||||
: "1px solid rgba(245, 158, 11, 0.2)",
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" alignItems="center" spacing={1.5} sx={{ mb: 1.5 }}>
|
||||
<ImageIcon sx={{ color: imageBlobUrl && !imageLoading ? "#667eea" : "#d97706", fontSize: "1.25rem" }} />
|
||||
<Typography variant="subtitle2" sx={{ color: imageBlobUrl && !imageLoading ? "#667eea" : "#d97706", fontWeight: 600 }}>
|
||||
{imageBlobUrl && !imageLoading ? "Image Generated" : "Loading Image..."}
|
||||
</Typography>
|
||||
</Stack>
|
||||
{imageBlobUrl && !imageLoading ? (
|
||||
<Box
|
||||
sx={{
|
||||
width: "100%",
|
||||
borderRadius: 2,
|
||||
overflow: "hidden",
|
||||
border: "1px solid rgba(102,126,234,0.2)",
|
||||
background: alpha("#667eea", 0.05),
|
||||
}}
|
||||
>
|
||||
<Box
|
||||
component="img"
|
||||
src={imageBlobUrl}
|
||||
alt={scene.title}
|
||||
sx={{
|
||||
width: "100%",
|
||||
height: "auto",
|
||||
display: "block",
|
||||
maxHeight: 400,
|
||||
objectFit: "cover",
|
||||
}}
|
||||
onError={(e) => {
|
||||
console.error('[SceneEditor] Image failed to load:', {
|
||||
src: e.currentTarget.src,
|
||||
imageUrl: scene.imageUrl,
|
||||
imageBlobUrl,
|
||||
});
|
||||
}}
|
||||
onLoad={() => {
|
||||
console.log('[SceneEditor] Image loaded successfully');
|
||||
}}
|
||||
/>
|
||||
</Box>
|
||||
) : (
|
||||
<Box sx={{ display: "flex", alignItems: "center", justifyContent: "center", py: 2 }}>
|
||||
<CircularProgress size={24} sx={{ color: "#d97706" }} />
|
||||
</Box>
|
||||
)}
|
||||
</Box>
|
||||
</>
|
||||
)}
|
||||
</Stack>
|
||||
|
||||
{/* Image Regeneration Modal */}
|
||||
<ImageRegenerateModal
|
||||
open={showRegenerateModal}
|
||||
onClose={() => setShowRegenerateModal(false)}
|
||||
onRegenerate={handleGenerateImage}
|
||||
initialPrompt={(() => {
|
||||
const promptParts = [
|
||||
`Scene: ${scene.title}`,
|
||||
"Professional podcast recording studio",
|
||||
"Modern microphone setup",
|
||||
"Clean background, professional lighting",
|
||||
"16:9 aspect ratio, video-optimized composition"
|
||||
];
|
||||
if (idea) {
|
||||
promptParts.push(`Topic: ${idea.substring(0, 60)}`);
|
||||
}
|
||||
return promptParts.join(", ");
|
||||
})()}
|
||||
initialStyle="Realistic"
|
||||
initialRenderingSpeed="Quality"
|
||||
initialAspectRatio="16:9"
|
||||
isGenerating={generatingImage}
|
||||
/>
|
||||
|
||||
<AudioRegenerateModal
|
||||
open={showAudioModal}
|
||||
onClose={() => setShowAudioModal(false)}
|
||||
onRegenerate={handleAudioRegenerate}
|
||||
initialSettings={audioSettings}
|
||||
isGenerating={generating}
|
||||
/>
|
||||
</GlassyCard>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -1,818 +0,0 @@
|
||||
import React, { useEffect, useState, useCallback } from "react";
|
||||
import { Box, Stack, Typography, Alert, Paper, LinearProgress, CircularProgress, alpha, Collapse, IconButton, Divider } from "@mui/material";
|
||||
import { EditNote as EditNoteIcon, CheckCircle as CheckCircleIcon, PlayArrow as PlayArrowIcon, ArrowBack as ArrowBackIcon, Info as InfoIcon, ExpandMore as ExpandMoreIcon, ExpandLess as ExpandLessIcon, Download as DownloadIcon, Refresh as RefreshIcon } from "@mui/icons-material";
|
||||
import { Script, Knobs, Scene } from "../types";
|
||||
import { BlogResearchResponse } from "../../../services/blogWriterApi";
|
||||
import { podcastApi } from "../../../services/podcastApi";
|
||||
import { GlassyCard, PrimaryButton, SecondaryButton } from "../ui";
|
||||
import { SceneEditor } from "./SceneEditor";
|
||||
import { InlineAudioPlayer } from "../InlineAudioPlayer";
|
||||
import { aiApiClient } from "../../../api/client";
|
||||
|
||||
interface ScriptEditorProps {
|
||||
projectId: string;
|
||||
idea: string;
|
||||
research: any; // Research type
|
||||
rawResearch: BlogResearchResponse | null;
|
||||
knobs: Knobs;
|
||||
speakers: number;
|
||||
durationMinutes: number;
|
||||
script: Script | null;
|
||||
onScriptChange: (script: Script) => void;
|
||||
onBackToResearch: () => void;
|
||||
onProceedToRendering: (script: Script) => void;
|
||||
onError: (message: string) => void;
|
||||
avatarUrl?: string | null; // Base avatar URL for consistent scene image generation
|
||||
analysis?: any;
|
||||
outline?: any;
|
||||
}
|
||||
|
||||
export const ScriptEditor: React.FC<ScriptEditorProps> = ({
|
||||
projectId,
|
||||
idea,
|
||||
research,
|
||||
rawResearch,
|
||||
knobs,
|
||||
speakers,
|
||||
durationMinutes,
|
||||
script: initialScript,
|
||||
onScriptChange,
|
||||
onBackToResearch,
|
||||
onProceedToRendering,
|
||||
onError,
|
||||
avatarUrl,
|
||||
analysis,
|
||||
outline,
|
||||
}) => {
|
||||
const [script, setScript] = useState<Script | null>(initialScript);
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [approvingSceneId, setApprovingSceneId] = useState<string | null>(null);
|
||||
const [generatingAudioId, setGeneratingAudioId] = useState<string | null>(null);
|
||||
const [showScriptFormatInfo, setShowScriptFormatInfo] = useState(true);
|
||||
const [combiningAudio, setCombiningAudio] = useState(false);
|
||||
const [combinedAudioResult, setCombinedAudioResult] = useState<{
|
||||
url: string;
|
||||
filename: string;
|
||||
duration: number;
|
||||
sceneCount: number;
|
||||
} | null>(null);
|
||||
|
||||
// Defer upward script updates to avoid setState during render warnings
|
||||
const emitScriptChange = useCallback(
|
||||
(next: Script) => Promise.resolve().then(() => onScriptChange(next)),
|
||||
[onScriptChange]
|
||||
);
|
||||
|
||||
// Sync with parent state
|
||||
useEffect(() => {
|
||||
if (initialScript) {
|
||||
setScript(initialScript);
|
||||
}
|
||||
}, [initialScript]);
|
||||
|
||||
useEffect(() => {
|
||||
// If script already exists, don't regenerate
|
||||
if (script) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Only generate if we have research data
|
||||
if (!rawResearch) {
|
||||
return;
|
||||
}
|
||||
|
||||
let mounted = true;
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
podcastApi
|
||||
.generateScript({
|
||||
projectId,
|
||||
idea,
|
||||
research: rawResearch,
|
||||
knobs,
|
||||
speakers,
|
||||
durationMinutes,
|
||||
analysis,
|
||||
outline,
|
||||
})
|
||||
.then((res) => {
|
||||
if (mounted) {
|
||||
setScript(res);
|
||||
emitScriptChange(res);
|
||||
setError(null);
|
||||
}
|
||||
})
|
||||
.catch((err) => {
|
||||
const message = err instanceof Error ? err.message : "Failed to generate script";
|
||||
setError(message);
|
||||
onError(message);
|
||||
})
|
||||
.finally(() => mounted && setLoading(false));
|
||||
return () => {
|
||||
mounted = false;
|
||||
};
|
||||
}, [projectId, rawResearch, idea, knobs, speakers, durationMinutes, analysis, outline, emitScriptChange, onError, script]);
|
||||
|
||||
const updateScene = (updated: Scene) => {
|
||||
// Use functional update to ensure we're working with latest state
|
||||
setScript((currentScript) => {
|
||||
if (!currentScript) return currentScript;
|
||||
const updatedScript = {
|
||||
...currentScript,
|
||||
scenes: currentScript.scenes.map((s) => (s.id === updated.id ? { ...s, ...updated } : s))
|
||||
};
|
||||
emitScriptChange(updatedScript);
|
||||
return updatedScript;
|
||||
});
|
||||
};
|
||||
|
||||
const approveScene = async (sceneId: string) => {
|
||||
try {
|
||||
setApprovingSceneId(sceneId);
|
||||
await podcastApi.approveScene({ projectId, sceneId });
|
||||
// Use functional update to ensure we're working with latest state
|
||||
setScript((currentScript) => {
|
||||
if (!currentScript) return currentScript;
|
||||
const updatedScript = {
|
||||
...currentScript,
|
||||
scenes: currentScript.scenes.map((s) => (s.id === sceneId ? { ...s, approved: true } : s)),
|
||||
};
|
||||
emitScriptChange(updatedScript);
|
||||
return updatedScript;
|
||||
});
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : "Failed to approve scene";
|
||||
setError(message);
|
||||
onError(message);
|
||||
throw err;
|
||||
} finally {
|
||||
setApprovingSceneId((current) => (current === sceneId ? null : current));
|
||||
}
|
||||
};
|
||||
|
||||
const deleteScene = useCallback((sceneId: string) => {
|
||||
if (!script) return;
|
||||
|
||||
// Prevent deleting if it's the last scene
|
||||
if (script.scenes.length <= 1) {
|
||||
onError("Cannot delete the last scene. At least one scene is required.");
|
||||
return;
|
||||
}
|
||||
|
||||
// Add confirmation dialog
|
||||
const sceneToDelete = script.scenes.find(s => s.id === sceneId);
|
||||
if (!sceneToDelete) return;
|
||||
|
||||
const confirmDelete = window.confirm(
|
||||
`Are you sure you want to delete "${sceneToDelete.title}"? This action cannot be undone.`
|
||||
);
|
||||
|
||||
if (!confirmDelete) return;
|
||||
|
||||
// Remove the scene from the script
|
||||
const updatedScenes = script.scenes.filter(s => s.id !== sceneId);
|
||||
const updatedScript = { ...script, scenes: updatedScenes };
|
||||
|
||||
emitScriptChange(updatedScript);
|
||||
setScript(updatedScript);
|
||||
|
||||
// Show success message
|
||||
console.log(`[ScriptEditor] Scene "${sceneToDelete.title}" deleted successfully`);
|
||||
}, [script, emitScriptChange, onError]);
|
||||
|
||||
const allApproved = script && script.scenes.every((s) => s.approved);
|
||||
const approvedCount = script ? script.scenes.filter((s) => s.approved).length : 0;
|
||||
const totalScenes = script ? script.scenes.length : 0;
|
||||
|
||||
// Check if all scenes have both audio and images (required for video rendering)
|
||||
const allScenesHaveAudioAndImages = script && script.scenes.every((s) => s.audioUrl && s.imageUrl);
|
||||
const scenesWithAudio = script ? script.scenes.filter((s) => s.audioUrl).length : 0;
|
||||
const allScenesHaveAudio = script && script.scenes.every((s) => s.audioUrl);
|
||||
|
||||
const combineAudio = useCallback(async () => {
|
||||
if (!script || !projectId) return;
|
||||
|
||||
try {
|
||||
setCombiningAudio(true);
|
||||
|
||||
const sceneIds: string[] = [];
|
||||
const sceneAudioUrls: string[] = [];
|
||||
|
||||
script.scenes.forEach((scene) => {
|
||||
if (scene.audioUrl) {
|
||||
// Ensure we're using the correct URL format (not blob URLs)
|
||||
const audioUrl = scene.audioUrl.startsWith('blob:') ? '' : scene.audioUrl;
|
||||
if (audioUrl) {
|
||||
sceneIds.push(scene.id);
|
||||
sceneAudioUrls.push(audioUrl);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (sceneIds.length === 0) {
|
||||
onError("No audio files found to combine.");
|
||||
return;
|
||||
}
|
||||
|
||||
const result = await podcastApi.combineAudio({
|
||||
projectId,
|
||||
sceneIds,
|
||||
sceneAudioUrls,
|
||||
});
|
||||
|
||||
// Store combined audio result for preview
|
||||
setCombinedAudioResult({
|
||||
url: result.combined_audio_url,
|
||||
filename: result.combined_audio_filename,
|
||||
duration: result.total_duration,
|
||||
sceneCount: result.scene_count,
|
||||
});
|
||||
|
||||
// Download the combined audio as blob (for authenticated endpoints)
|
||||
try {
|
||||
// Normalize path
|
||||
let audioPath = result.combined_audio_url.startsWith('/')
|
||||
? result.combined_audio_url
|
||||
: `/${result.combined_audio_url}`;
|
||||
|
||||
// Ensure it's a podcast audio endpoint
|
||||
if (!audioPath.includes('/api/podcast/audio/')) {
|
||||
const filename = audioPath.split('/').pop() || result.combined_audio_filename;
|
||||
audioPath = `/api/podcast/audio/${filename}`;
|
||||
}
|
||||
|
||||
// Remove query parameters if present
|
||||
audioPath = audioPath.split('?')[0];
|
||||
|
||||
// Fetch as blob using authenticated client
|
||||
const response = await aiApiClient.get(audioPath, {
|
||||
responseType: 'blob',
|
||||
});
|
||||
|
||||
// Create blob URL and download
|
||||
const blob = response.data;
|
||||
const blobUrl = URL.createObjectURL(blob);
|
||||
|
||||
const link = document.createElement("a");
|
||||
link.href = blobUrl;
|
||||
link.download = result.combined_audio_filename || `podcast-episode-${projectId.slice(-8)}.mp3`;
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
document.body.removeChild(link);
|
||||
|
||||
// Clean up blob URL after a delay
|
||||
setTimeout(() => {
|
||||
URL.revokeObjectURL(blobUrl);
|
||||
}, 100);
|
||||
} catch (downloadError) {
|
||||
console.error('Failed to download combined audio:', downloadError);
|
||||
onError('Failed to download audio file. You can try downloading again from the preview.');
|
||||
}
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Failed to combine audio";
|
||||
onError(`Failed to combine audio: ${message}`);
|
||||
} finally {
|
||||
setCombiningAudio(false);
|
||||
}
|
||||
}, [script, projectId, onError]);
|
||||
|
||||
return (
|
||||
<Box sx={{ mt: 4 }}>
|
||||
<Stack direction="row" spacing={2} alignItems="center" sx={{ mb: 4 }}>
|
||||
<SecondaryButton onClick={onBackToResearch} startIcon={<ArrowBackIcon />}>
|
||||
Back to Research
|
||||
</SecondaryButton>
|
||||
<Box sx={{ flex: 1 }}>
|
||||
<Typography
|
||||
variant="h4"
|
||||
sx={{
|
||||
background: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||
WebkitBackgroundClip: "text",
|
||||
WebkitTextFillColor: "transparent",
|
||||
fontWeight: 700,
|
||||
letterSpacing: "-0.02em",
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
gap: 1.5,
|
||||
fontSize: { xs: "1.75rem", md: "2rem" },
|
||||
}}
|
||||
>
|
||||
<EditNoteIcon sx={{ fontSize: "2rem" }} />
|
||||
Script Editor
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#64748b", mt: 0.5, ml: 5.5 }}>
|
||||
Review and refine your podcast script before rendering
|
||||
</Typography>
|
||||
</Box>
|
||||
</Stack>
|
||||
|
||||
{loading && (
|
||||
<Alert
|
||||
severity="info"
|
||||
icon={<CircularProgress size={20} />}
|
||||
sx={{
|
||||
mb: 3,
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.08) 0%, rgba(139, 92, 246, 0.08) 100%)",
|
||||
border: "1px solid rgba(99, 102, 241, 0.2)",
|
||||
borderRadius: 2,
|
||||
boxShadow: "0 1px 2px rgba(99, 102, 241, 0.05)",
|
||||
"& .MuiAlert-icon": {
|
||||
color: "#6366f1",
|
||||
},
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#0f172a", fontWeight: 500 }}>
|
||||
Generating script with AI... This may take a moment.
|
||||
</Typography>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{error && (
|
||||
<Alert
|
||||
severity="error"
|
||||
sx={{
|
||||
mb: 3,
|
||||
background: "linear-gradient(135deg, rgba(239, 68, 68, 0.08) 0%, rgba(220, 38, 38, 0.08) 100%)",
|
||||
border: "1px solid rgba(239, 68, 68, 0.2)",
|
||||
borderRadius: 2,
|
||||
boxShadow: "0 1px 2px rgba(239, 68, 68, 0.05)",
|
||||
"& .MuiAlert-icon": {
|
||||
color: "#ef4444",
|
||||
},
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#0f172a", fontWeight: 500 }}>
|
||||
{error}
|
||||
</Typography>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{script && (
|
||||
<Stack spacing={3}>
|
||||
{/* Script Format Explanation Panel */}
|
||||
<Paper
|
||||
sx={{
|
||||
p: 3,
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.05) 0%, rgba(139, 92, 246, 0.05) 100%)",
|
||||
border: "1px solid rgba(99, 102, 241, 0.15)",
|
||||
borderRadius: 2,
|
||||
boxShadow: "0 2px 8px rgba(99, 102, 241, 0.08)",
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" alignItems="center" justifyContent="space-between" sx={{ mb: showScriptFormatInfo ? 2 : 0 }}>
|
||||
<Stack direction="row" alignItems="center" spacing={1.5}>
|
||||
<Box
|
||||
sx={{
|
||||
width: 40,
|
||||
height: 40,
|
||||
borderRadius: "50%",
|
||||
background: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
boxShadow: "0 2px 8px rgba(102, 126, 234, 0.3)",
|
||||
}}
|
||||
>
|
||||
<InfoIcon sx={{ color: "#ffffff", fontSize: "1.5rem" }} />
|
||||
</Box>
|
||||
<Box>
|
||||
<Typography variant="h6" sx={{ color: "#0f172a", fontWeight: 600, fontSize: "1.1rem" }}>
|
||||
Why This Script Format?
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#64748b", mt: 0.25 }}>
|
||||
Understanding how your script creates natural, human-like audio
|
||||
</Typography>
|
||||
</Box>
|
||||
</Stack>
|
||||
<IconButton
|
||||
onClick={() => setShowScriptFormatInfo(!showScriptFormatInfo)}
|
||||
sx={{
|
||||
color: "#6366f1",
|
||||
"&:hover": {
|
||||
background: "rgba(99, 102, 241, 0.1)",
|
||||
},
|
||||
}}
|
||||
>
|
||||
{showScriptFormatInfo ? <ExpandLessIcon /> : <ExpandMoreIcon />}
|
||||
</IconButton>
|
||||
</Stack>
|
||||
|
||||
<Collapse in={showScriptFormatInfo}>
|
||||
<Stack spacing={2.5}>
|
||||
<Box>
|
||||
<Typography variant="body2" sx={{ color: "#0f172a", lineHeight: 1.8, mb: 2 }}>
|
||||
Our AI script generator creates scripts specifically optimized for <strong style={{ fontWeight: 600 }}>high-quality text-to-speech</strong>.
|
||||
The format you see here is designed to produce audio that sounds natural and human-like, not robotic.
|
||||
</Typography>
|
||||
</Box>
|
||||
|
||||
<Stack spacing={2}>
|
||||
<Box sx={{ display: "flex", gap: 2 }}>
|
||||
<Box
|
||||
sx={{
|
||||
minWidth: 32,
|
||||
height: 32,
|
||||
borderRadius: "8px",
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
flexShrink: 0,
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||
1
|
||||
</Typography>
|
||||
</Box>
|
||||
<Box>
|
||||
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||
Natural Pauses & Rhythm
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||
The script includes strategic pauses between lines and when speakers change. This creates natural breathing patterns
|
||||
and conversation flow, just like real human speech. Without these pauses, the audio would sound rushed and robotic.
|
||||
</Typography>
|
||||
</Box>
|
||||
</Box>
|
||||
|
||||
<Box sx={{ display: "flex", gap: 2 }}>
|
||||
<Box
|
||||
sx={{
|
||||
minWidth: 32,
|
||||
height: 32,
|
||||
borderRadius: "8px",
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
flexShrink: 0,
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||
2
|
||||
</Typography>
|
||||
</Box>
|
||||
<Box>
|
||||
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||
Emphasis Markers
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||
Lines marked with emphasis help highlight important points, statistics, or key insights. The AI voice will naturally
|
||||
stress these parts, making your podcast more engaging and easier to follow—just like a real host would emphasize important information.
|
||||
</Typography>
|
||||
</Box>
|
||||
</Box>
|
||||
|
||||
<Box sx={{ display: "flex", gap: 2 }}>
|
||||
<Box
|
||||
sx={{
|
||||
minWidth: 32,
|
||||
height: 32,
|
||||
borderRadius: "8px",
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
flexShrink: 0,
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||
3
|
||||
</Typography>
|
||||
</Box>
|
||||
<Box>
|
||||
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||
Short, Conversational Sentences
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||
The script uses shorter sentences (15-20 words) written in a conversational style. This matches how people actually
|
||||
speak, making the audio sound more natural. Long, complex sentences would sound awkward when spoken aloud.
|
||||
</Typography>
|
||||
</Box>
|
||||
</Box>
|
||||
|
||||
<Box sx={{ display: "flex", gap: 2 }}>
|
||||
<Box
|
||||
sx={{
|
||||
minWidth: 32,
|
||||
height: 32,
|
||||
borderRadius: "8px",
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
flexShrink: 0,
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||
4
|
||||
</Typography>
|
||||
</Box>
|
||||
<Box>
|
||||
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||
Scene-Specific Emotions
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||
Each scene has an emotional tone (excited, serious, curious, etc.) that guides the AI voice's delivery. This creates
|
||||
variety and keeps listeners engaged, just like a real podcast host would vary their tone based on the topic.
|
||||
</Typography>
|
||||
</Box>
|
||||
</Box>
|
||||
|
||||
<Box sx={{ display: "flex", gap: 2 }}>
|
||||
<Box
|
||||
sx={{
|
||||
minWidth: 32,
|
||||
height: 32,
|
||||
borderRadius: "8px",
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.1) 0%, rgba(139, 92, 246, 0.1) 100%)",
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
flexShrink: 0,
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#6366f1", fontWeight: 700 }}>
|
||||
5
|
||||
</Typography>
|
||||
</Box>
|
||||
<Box>
|
||||
<Typography variant="subtitle2" sx={{ color: "#0f172a", fontWeight: 600, mb: 0.5 }}>
|
||||
Optimized for Podcast Narration
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#475569", lineHeight: 1.7 }}>
|
||||
The script is optimized with slightly slower pacing and natural pronunciation settings specifically for podcast narration.
|
||||
This ensures clarity and makes the content easy to understand, even when listeners are multitasking.
|
||||
</Typography>
|
||||
</Box>
|
||||
</Box>
|
||||
</Stack>
|
||||
|
||||
<Alert
|
||||
severity="info"
|
||||
sx={{
|
||||
mt: 1,
|
||||
background: "rgba(99, 102, 241, 0.06)",
|
||||
border: "1px solid rgba(99, 102, 241, 0.15)",
|
||||
"& .MuiAlert-icon": {
|
||||
color: "#6366f1",
|
||||
},
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#0f172a", lineHeight: 1.7 }}>
|
||||
<strong style={{ fontWeight: 600 }}>Tip:</strong> You can edit any line or scene to match your preferences.
|
||||
The format will be preserved when rendering, ensuring your audio still sounds natural and professional.
|
||||
</Typography>
|
||||
</Alert>
|
||||
</Stack>
|
||||
</Collapse>
|
||||
</Paper>
|
||||
|
||||
<Alert
|
||||
severity="info"
|
||||
sx={{
|
||||
background: "linear-gradient(135deg, rgba(99, 102, 241, 0.08) 0%, rgba(139, 92, 246, 0.08) 100%)",
|
||||
border: "1px solid rgba(99, 102, 241, 0.2)",
|
||||
borderRadius: 2,
|
||||
boxShadow: "0 1px 2px rgba(99, 102, 241, 0.05)",
|
||||
"& .MuiAlert-icon": {
|
||||
color: "#6366f1",
|
||||
},
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#0f172a", fontWeight: 500, lineHeight: 1.6 }}>
|
||||
<strong style={{ fontWeight: 600 }}>Approval Required:</strong> Each scene must be approved before rendering. Review and edit lines as needed, then approve each scene.
|
||||
</Typography>
|
||||
</Alert>
|
||||
|
||||
<Stack spacing={2}>
|
||||
{script.scenes.map((scene, idx) => (
|
||||
<GlassyCard
|
||||
key={scene.id}
|
||||
initial={{ opacity: 0, y: 8 }}
|
||||
animate={{ opacity: 1, y: 0 }}
|
||||
transition={{ duration: 0.3, delay: idx * 0.1 }}
|
||||
>
|
||||
<SceneEditor
|
||||
scene={scene}
|
||||
onUpdateScene={updateScene}
|
||||
onApprove={approveScene}
|
||||
onDelete={deleteScene}
|
||||
knobs={knobs}
|
||||
approvingSceneId={approvingSceneId}
|
||||
generatingAudioId={generatingAudioId}
|
||||
totalScenes={script.scenes.length}
|
||||
onAudioGenerationStart={(sceneId) => {
|
||||
setGeneratingAudioId(sceneId);
|
||||
}}
|
||||
onAudioGenerated={async (sceneId, audioUrl) => {
|
||||
setGeneratingAudioId(null);
|
||||
// Use functional update to ensure we're working with latest state
|
||||
// Ensure scene is marked as approved and has audioUrl
|
||||
setScript((currentScript) => {
|
||||
if (!currentScript) return currentScript;
|
||||
const updatedScenes = currentScript.scenes.map((s) =>
|
||||
s.id === sceneId ? { ...s, audioUrl, approved: true } : s
|
||||
);
|
||||
const updatedScript = { ...currentScript, scenes: updatedScenes };
|
||||
emitScriptChange(updatedScript);
|
||||
return updatedScript;
|
||||
});
|
||||
}}
|
||||
idea={idea}
|
||||
avatarUrl={avatarUrl}
|
||||
/>
|
||||
</GlassyCard>
|
||||
))}
|
||||
</Stack>
|
||||
|
||||
<Paper
|
||||
sx={{
|
||||
p: 3.5,
|
||||
background: allApproved
|
||||
? "linear-gradient(135deg, rgba(16, 185, 129, 0.05) 0%, rgba(5, 150, 105, 0.05) 100%)"
|
||||
: "#ffffff",
|
||||
border: allApproved
|
||||
? "2px solid rgba(16, 185, 129, 0.25)"
|
||||
: "1px solid rgba(15, 23, 42, 0.08)",
|
||||
borderRadius: 3,
|
||||
boxShadow: allApproved
|
||||
? "0 4px 6px rgba(16, 185, 129, 0.08), 0 8px 24px rgba(16, 185, 129, 0.06)"
|
||||
: "0 1px 3px rgba(15, 23, 42, 0.06), 0 4px 12px rgba(15, 23, 42, 0.04)",
|
||||
transition: "all 0.3s cubic-bezier(0.4, 0, 0.2, 1)",
|
||||
}}
|
||||
>
|
||||
<Stack direction="row" justifyContent="space-between" alignItems="center">
|
||||
<Box>
|
||||
<Typography variant="subtitle1" sx={{ mb: 1, display: "flex", alignItems: "center", gap: 1.5, color: "#0f172a", fontWeight: 600, fontSize: "1.1rem" }}>
|
||||
<CheckCircleIcon fontSize="small" sx={{ color: allApproved ? "#10b981" : "#94a3b8", fontSize: "1.25rem" }} />
|
||||
Approval Status
|
||||
</Typography>
|
||||
<Typography variant="body2" sx={{ color: "#64748b", fontWeight: 400, lineHeight: 1.6 }}>
|
||||
{approvedCount} of {totalScenes} scenes approved
|
||||
{allScenesHaveAudioAndImages && " • All scenes ready for video rendering"}
|
||||
{!allScenesHaveAudioAndImages && allApproved && " • Generate images for all scenes to enable video rendering"}
|
||||
{!allApproved && " — Approve all scenes first"}
|
||||
</Typography>
|
||||
{!allScenesHaveAudioAndImages && (
|
||||
<LinearProgress
|
||||
variant="determinate"
|
||||
value={
|
||||
allScenesHaveAudioAndImages
|
||||
? 100
|
||||
: script
|
||||
? (script.scenes.filter((s) => s.audioUrl && s.imageUrl).length / totalScenes) * 100
|
||||
: 0
|
||||
}
|
||||
sx={{ mt: 1, height: 6, borderRadius: 3 }}
|
||||
/>
|
||||
)}
|
||||
</Box>
|
||||
<PrimaryButton
|
||||
onClick={() => script && onProceedToRendering(script)}
|
||||
disabled={!allScenesHaveAudioAndImages}
|
||||
startIcon={<PlayArrowIcon />}
|
||||
tooltip={
|
||||
!allScenesHaveAudioAndImages
|
||||
? "Generate audio and images for all scenes to proceed to video rendering"
|
||||
: "Proceed to video rendering (all scenes have audio and images)"
|
||||
}
|
||||
>
|
||||
Proceed to Rendering
|
||||
</PrimaryButton>
|
||||
</Stack>
|
||||
</Paper>
|
||||
|
||||
{/* Download Audio-Only Podcast Section */}
|
||||
{allScenesHaveAudio && (
|
||||
<Paper
|
||||
sx={{
|
||||
p: 3,
|
||||
background: "linear-gradient(135deg, rgba(102, 126, 234, 0.05) 0%, rgba(118, 75, 162, 0.05) 100%)",
|
||||
border: "1px solid rgba(102, 126, 234, 0.15)",
|
||||
borderRadius: 2,
|
||||
}}
|
||||
>
|
||||
<Stack spacing={3}>
|
||||
<Typography variant="h6" sx={{ color: "#0f172a", fontWeight: 600 }}>
|
||||
Download Audio-Only Podcast
|
||||
</Typography>
|
||||
|
||||
{!combinedAudioResult ? (
|
||||
<>
|
||||
<PrimaryButton
|
||||
onClick={combineAudio}
|
||||
disabled={combiningAudio}
|
||||
loading={combiningAudio}
|
||||
startIcon={<DownloadIcon />}
|
||||
tooltip="Combine all scene audio files into a single podcast episode"
|
||||
sx={{
|
||||
minWidth: 280,
|
||||
fontSize: "1rem",
|
||||
py: 1.5,
|
||||
background: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
|
||||
"&:hover": {
|
||||
background: "linear-gradient(135deg, #764ba2 0%, #667eea 100%)",
|
||||
},
|
||||
}}
|
||||
>
|
||||
{combiningAudio ? "Combining Audio..." : "Download Audio-Only Podcast"}
|
||||
</PrimaryButton>
|
||||
<Typography variant="caption" sx={{ color: "#64748b", fontStyle: "italic" }}>
|
||||
This will combine all {scenesWithAudio} scene audio files into one complete podcast episode.
|
||||
</Typography>
|
||||
</>
|
||||
) : (
|
||||
<Stack spacing={2}>
|
||||
{/* Success Alert */}
|
||||
<Alert
|
||||
severity="success"
|
||||
sx={{
|
||||
background: alpha("#10b981", 0.1),
|
||||
border: "1px solid rgba(16,185,129,0.3)",
|
||||
"& .MuiAlert-icon": { color: "#10b981" },
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ color: "#059669", fontWeight: 500 }}>
|
||||
✅ Combined audio generated successfully! ({combinedAudioResult.sceneCount} scenes,{" "}
|
||||
{Math.round(combinedAudioResult.duration)}s)
|
||||
</Typography>
|
||||
</Alert>
|
||||
|
||||
{/* Combined Audio Preview */}
|
||||
<InlineAudioPlayer audioUrl={combinedAudioResult.url} title="Complete Podcast Episode" />
|
||||
|
||||
{/* Action Buttons */}
|
||||
<Stack direction="row" spacing={2}>
|
||||
<SecondaryButton
|
||||
onClick={async () => {
|
||||
try {
|
||||
// Normalize path
|
||||
let audioPath = combinedAudioResult.url.startsWith('/')
|
||||
? combinedAudioResult.url
|
||||
: `/${combinedAudioResult.url}`;
|
||||
|
||||
// Ensure it's a podcast audio endpoint
|
||||
if (!audioPath.includes('/api/podcast/audio/')) {
|
||||
const filename = audioPath.split('/').pop() || combinedAudioResult.filename;
|
||||
audioPath = `/api/podcast/audio/${filename}`;
|
||||
}
|
||||
|
||||
// Remove query parameters if present
|
||||
audioPath = audioPath.split('?')[0];
|
||||
|
||||
// Fetch as blob using authenticated client
|
||||
const response = await aiApiClient.get(audioPath, {
|
||||
responseType: 'blob',
|
||||
});
|
||||
|
||||
// Create blob URL and download
|
||||
const blob = response.data;
|
||||
const blobUrl = URL.createObjectURL(blob);
|
||||
|
||||
const link = document.createElement("a");
|
||||
link.href = blobUrl;
|
||||
link.download = combinedAudioResult.filename || `podcast-episode-${projectId.slice(-8)}.mp3`;
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
document.body.removeChild(link);
|
||||
|
||||
// Clean up blob URL after a delay
|
||||
setTimeout(() => {
|
||||
URL.revokeObjectURL(blobUrl);
|
||||
}, 100);
|
||||
} catch (error) {
|
||||
console.error('Failed to download audio:', error);
|
||||
onError('Failed to download audio file. Please try again.');
|
||||
}
|
||||
}}
|
||||
startIcon={<DownloadIcon />}
|
||||
tooltip="Download the combined audio file again"
|
||||
>
|
||||
Download Again
|
||||
</SecondaryButton>
|
||||
<SecondaryButton
|
||||
onClick={() => {
|
||||
setCombinedAudioResult(null);
|
||||
combineAudio();
|
||||
}}
|
||||
disabled={combiningAudio}
|
||||
loading={combiningAudio}
|
||||
startIcon={<RefreshIcon />}
|
||||
tooltip="Regenerate combined audio (useful if scenes were updated)"
|
||||
>
|
||||
Regenerate
|
||||
</SecondaryButton>
|
||||
</Stack>
|
||||
</Stack>
|
||||
)}
|
||||
</Stack>
|
||||
</Paper>
|
||||
)}
|
||||
</Stack>
|
||||
)}
|
||||
</Box>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -1,334 +0,0 @@
|
||||
"""
|
||||
Podcast Analysis Handlers
|
||||
|
||||
Analysis endpoint for podcast ideas.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import Dict, Any
|
||||
import json
|
||||
import uuid
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from services.database import get_db
|
||||
from middleware.auth_middleware import get_current_user
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
from services.llm_providers.main_text_generation import llm_text_gen
|
||||
from services.llm_providers.main_image_generation import generate_image
|
||||
from services.podcast_bible_service import PodcastBibleService
|
||||
from utils.asset_tracker import save_asset_to_library
|
||||
from loguru import logger
|
||||
from ..constants import PODCAST_IMAGES_DIR
|
||||
from ..models import (
|
||||
PodcastAnalyzeRequest,
|
||||
PodcastAnalyzeResponse,
|
||||
PodcastEnhanceIdeaRequest,
|
||||
PodcastEnhanceIdeaResponse
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/idea/enhance", response_model=PodcastEnhanceIdeaResponse)
|
||||
async def enhance_podcast_idea(
|
||||
request: PodcastEnhanceIdeaRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Take raw keywords/topic and use AI to craft a presentable, detailed podcast idea.
|
||||
Uses the user's Podcast Bible for hyper-personalization if available.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Serialize Bible context if provided or generate from onboarding
|
||||
bible_context = ""
|
||||
try:
|
||||
bible_service = PodcastBibleService()
|
||||
if request.bible:
|
||||
from models.podcast_bible_models import PodcastBible
|
||||
bible_data = PodcastBible(**request.bible)
|
||||
bible_context = bible_service.serialize_bible(bible_data)
|
||||
else:
|
||||
# Generate from onboarding data directly
|
||||
bible_obj = bible_service.generate_bible(user_id, "temp_enhance")
|
||||
bible_context = bible_service.serialize_bible(bible_obj)
|
||||
except Exception as exc:
|
||||
logger.warning(f"[Podcast Enhance] Failed to parse or generate bible context: {exc}")
|
||||
|
||||
prompt = f"""
|
||||
You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea.
|
||||
|
||||
{f"USER PERSONALIZATION CONTEXT (Podcast Bible):\n{bible_context}\n" if bible_context else ""}
|
||||
|
||||
RAW IDEA/KEYWORDS: "{request.idea}"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions, each with a unique angle:
|
||||
1. Professional & Expert-led angle (focus on authority, insights, and expertise)
|
||||
2. Storytelling & Human interest angle (focus on narratives, emotions, and personal connections)
|
||||
3. Trendy & Contemporary angle (focus on current trends, modern perspectives, and relevance)
|
||||
|
||||
Each version should be 2-3 sentences, audience-focused, and align with host persona if provided.
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 enhanced episode pitches (in order: Professional, Storytelling, Trendy)
|
||||
- rationales: array of 3 rationales explaining the approach for each version
|
||||
"""
|
||||
|
||||
try:
|
||||
raw = llm_text_gen(
|
||||
prompt=prompt,
|
||||
user_id=user_id,
|
||||
json_struct=None,
|
||||
preferred_provider="huggingface",
|
||||
flow_type="premium_tool",
|
||||
)
|
||||
|
||||
# Normalize response
|
||||
if isinstance(raw, str):
|
||||
data = json.loads(raw)
|
||||
else:
|
||||
data = raw
|
||||
|
||||
# Extract enhanced ideas and rationales with fallbacks
|
||||
enhanced_ideas = data.get("enhanced_ideas", [])
|
||||
rationales = data.get("rationales", [])
|
||||
|
||||
# Ensure we have exactly 3 ideas, fallback to original if needed
|
||||
if not isinstance(enhanced_ideas, list) or len(enhanced_ideas) != 3:
|
||||
# Fallback: create 3 variations of the original idea
|
||||
base_idea = request.idea
|
||||
enhanced_ideas = [
|
||||
f"Expert insights on {base_idea}: A deep dive into industry trends and best practices.",
|
||||
f"The human side of {base_idea}: Personal stories and real-world experiences that resonate.",
|
||||
f"Modern perspectives on {base_idea}: Current trends and forward-thinking approaches."
|
||||
]
|
||||
rationales = [
|
||||
"Professional approach focusing on expertise and authority",
|
||||
"Storytelling approach emphasizing human connection",
|
||||
"Contemporary approach highlighting current relevance"
|
||||
]
|
||||
|
||||
# Ensure rationales match the number of ideas
|
||||
if not isinstance(rationales, list) or len(rationales) != 3:
|
||||
rationales = [
|
||||
"Professional angle with expert insights",
|
||||
"Storytelling angle with human interest",
|
||||
"Trendy angle with contemporary relevance"
|
||||
]
|
||||
|
||||
return PodcastEnhanceIdeaResponse(
|
||||
enhanced_ideas=enhanced_ideas[:3], # Ensure exactly 3
|
||||
rationales=rationales[:3] # Ensure exactly 3
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(f"[Podcast Enhance] Failed for user {user_id}: {exc}")
|
||||
# Fallback to basic variations of original idea
|
||||
base_idea = request.idea
|
||||
return PodcastEnhanceIdeaResponse(
|
||||
enhanced_ideas=[
|
||||
f"Expert insights on {base_idea}: A deep dive into industry trends and best practices.",
|
||||
f"The human side of {base_idea}: Personal stories and real-world experiences that resonate.",
|
||||
f"Modern perspectives on {base_idea}: Current trends and forward-thinking approaches."
|
||||
],
|
||||
rationales=[
|
||||
"Professional approach focusing on expertise and authority",
|
||||
"Storytelling approach emphasizing human connection",
|
||||
"Contemporary approach highlighting current relevance"
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.post("/analyze", response_model=PodcastAnalyzeResponse)
|
||||
async def analyze_podcast_idea(
|
||||
request: PodcastAnalyzeRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Analyze a podcast idea and return podcast-oriented outlines, keywords, and titles.
|
||||
If no avatar_url is provided, it generates one automatically based on the host's look.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Serialize Bible context if provided or generate from onboarding
|
||||
bible_context = ""
|
||||
bible_obj = None
|
||||
try:
|
||||
bible_service = PodcastBibleService()
|
||||
if request.bible:
|
||||
from models.podcast_bible_models import PodcastBible
|
||||
bible_data = PodcastBible(**request.bible)
|
||||
bible_context = bible_service.serialize_bible(bible_data)
|
||||
bible_obj = bible_data
|
||||
else:
|
||||
# Generate from onboarding data directly
|
||||
bible_obj = bible_service.generate_bible(user_id, "temp_analyze")
|
||||
bible_context = bible_service.serialize_bible(bible_obj)
|
||||
bible_obj = bible_obj
|
||||
except Exception as exc:
|
||||
logger.warning(f"[Podcast Analyze] Failed to parse or generate bible context: {exc}")
|
||||
|
||||
# --- NEW: Generate Presenter Avatar if missing ---
|
||||
final_avatar_url = request.avatar_url
|
||||
final_avatar_prompt = None
|
||||
|
||||
if not final_avatar_url:
|
||||
logger.info(f"[Podcast Analyze] No avatar_url provided, generating one for user {user_id}")
|
||||
try:
|
||||
# 1. PRE-FLIGHT VALIDATION: Check subscription limits for image generation
|
||||
from services.subscription import PricingService
|
||||
from services.subscription.preflight_validator import validate_image_generation_operations
|
||||
pricing_service = PricingService(db)
|
||||
validate_image_generation_operations(
|
||||
pricing_service=pricing_service,
|
||||
user_id=user_id,
|
||||
num_images=1
|
||||
)
|
||||
|
||||
# 2. Build avatar prompt from Bible host look or fallback
|
||||
host_look = bible_obj.host.look if bible_obj and bible_obj.host.look else "A professional podcast host"
|
||||
visual_style = bible_obj.visual_style.style_preset if bible_obj else "Realistic Photography"
|
||||
|
||||
final_avatar_prompt = f"Professional headshot of a podcast host, {host_look}, {visual_style} style, clean background, soft studio lighting, center-focused, high resolution, sharp focus, professional photography quality, 16:9 aspect ratio."
|
||||
|
||||
# 3. Generate the image
|
||||
logger.info(f"[Podcast Analyze] Generating avatar with prompt: {final_avatar_prompt}")
|
||||
image_result = generate_image(
|
||||
prompt=final_avatar_prompt,
|
||||
user_id=user_id,
|
||||
width=1024,
|
||||
height=1024
|
||||
)
|
||||
|
||||
# 4. Save to disk and library
|
||||
if image_result and image_result.image_bytes:
|
||||
img_id = str(uuid.uuid4())[:8]
|
||||
filename = f"presenter_podcast_{user_id}_{img_id}.png"
|
||||
output_path = PODCAST_IMAGES_DIR / filename
|
||||
PODCAST_IMAGES_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with open(output_path, "wb") as f:
|
||||
f.write(image_result.image_bytes)
|
||||
|
||||
final_avatar_url = f"/api/podcast/images/avatars/{filename}"
|
||||
|
||||
# Save to asset library for reuse
|
||||
save_asset_to_library(
|
||||
db=db,
|
||||
user_id=user_id,
|
||||
asset_type="image",
|
||||
file_url=final_avatar_url,
|
||||
filename=filename,
|
||||
title=f"Presenter Avatar - {request.idea[:40]}",
|
||||
description=f"AI-generated podcast presenter for: {request.idea}",
|
||||
provider=image_result.provider,
|
||||
model=image_result.model,
|
||||
cost=image_result.cost
|
||||
)
|
||||
logger.info(f"[Podcast Analyze] ✅ Generated and saved avatar to {final_avatar_url}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Podcast Analyze] ❌ Failed to generate avatar: {e}")
|
||||
# Non-fatal: continue analysis even if avatar generation fails
|
||||
|
||||
# --- END: Avatar Generation ---
|
||||
|
||||
# Incorporate user feedback if provided
|
||||
feedback_context = ""
|
||||
if request.feedback:
|
||||
feedback_context = f"""
|
||||
USER REGENERATION FEEDBACK:
|
||||
The user was not satisfied with the previous analysis. They provided the following instructions for improvement:
|
||||
"{request.feedback}"
|
||||
Please prioritize this feedback and adjust the analysis accordingly.
|
||||
"""
|
||||
|
||||
prompt = f"""
|
||||
You are an expert podcast producer and research strategist. Given a podcast idea, craft concise podcast-ready assets
|
||||
that sound like episode plans (not fiction stories).
|
||||
|
||||
{f"USER PERSONALIZATION CONTEXT (Podcast Bible):\n{bible_context}\n" if bible_context else ""}
|
||||
{feedback_context}
|
||||
|
||||
Podcast Idea: "{request.idea}"
|
||||
Duration: ~{request.duration} minutes
|
||||
Speakers: {request.speakers} (host + optional guest)
|
||||
|
||||
TASK:
|
||||
1. Define the target audience and content type aligned with the Bible's "Audience DNA" and "Brand DNA".
|
||||
2. Identify 5 high-impact keywords.
|
||||
3. Propose 2 episode outlines with factual segments.
|
||||
4. Suggest 3 titles.
|
||||
5. IMPORTANT: Generate 4-6 specific research queries for Exa. These queries MUST be highly targeted to the episode's topic, the host's expertise level, and the audience's interests as defined in the Bible.
|
||||
* Do NOT use generic queries like "latest trends in X".
|
||||
* DO use queries that look for case studies, specific data points, expert opinions, or contrasting viewpoints that would make for a deep, insightful podcast conversation.
|
||||
|
||||
Return JSON with:
|
||||
- audience: short target audience description
|
||||
- content_type: podcast style/format
|
||||
- top_keywords: 5 podcast-relevant keywords/phrases
|
||||
- suggested_outlines: 2 items, each with title (<=60 chars) and 4-6 short segments (bullet-friendly, factual)
|
||||
- title_suggestions: 3 concise episode titles
|
||||
- research_queries: array of {{"query": "string", "rationale": "string"}}
|
||||
- exa_suggested_config: suggested Exa search options with:
|
||||
- exa_search_type: "auto" | "neural" | "keyword"
|
||||
- exa_category: one of ["research paper","news","company","github","tweet","personal site","pdf","financial report","linkedin profile"]
|
||||
- exa_include_domains: up to 3 reputable domains
|
||||
- exa_exclude_domains: up to 3 domains
|
||||
- max_sources: 6-10
|
||||
- include_statistics: boolean
|
||||
- date_range: one of ["last_month","last_3_months","last_year","all_time"]
|
||||
|
||||
Requirements:
|
||||
- Keep language factual, actionable, and suited for spoken audio.
|
||||
- Avoid narrative fiction tone.
|
||||
- Prefer 2024-2025 context.
|
||||
"""
|
||||
|
||||
try:
|
||||
raw = llm_text_gen(
|
||||
prompt=prompt,
|
||||
user_id=user_id,
|
||||
json_struct=None,
|
||||
preferred_provider="huggingface",
|
||||
flow_type="premium_tool",
|
||||
)
|
||||
except HTTPException:
|
||||
# Re-raise HTTPExceptions (e.g., 429 subscription limit) - preserve error details
|
||||
raise
|
||||
except Exception as exc:
|
||||
logger.error(f"[Podcast Analyze] Analysis failed for user {user_id}: {exc}")
|
||||
raise HTTPException(status_code=500, detail=f"Analysis failed: {exc}")
|
||||
|
||||
# Normalize response (accept dict or JSON string)
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
except json.JSONDecodeError:
|
||||
raise HTTPException(status_code=500, detail="LLM returned non-JSON output")
|
||||
elif isinstance(raw, dict):
|
||||
data = raw
|
||||
else:
|
||||
raise HTTPException(status_code=500, detail="Unexpected LLM response format")
|
||||
|
||||
audience = data.get("audience") or "Growth-focused professionals"
|
||||
content_type = data.get("content_type") or "Interview + insights"
|
||||
top_keywords = data.get("top_keywords") or []
|
||||
suggested_outlines = data.get("suggested_outlines") or []
|
||||
title_suggestions = data.get("title_suggestions") or []
|
||||
research_queries = data.get("research_queries") or []
|
||||
exa_suggested_config = data.get("exa_suggested_config") or None
|
||||
|
||||
return PodcastAnalyzeResponse(
|
||||
audience=audience,
|
||||
content_type=content_type,
|
||||
top_keywords=top_keywords,
|
||||
suggested_outlines=suggested_outlines,
|
||||
title_suggestions=title_suggestions,
|
||||
research_queries=research_queries,
|
||||
exa_suggested_config=exa_suggested_config,
|
||||
bible=bible_obj.model_dump() if bible_obj else None,
|
||||
avatar_url=final_avatar_url,
|
||||
avatar_prompt=final_avatar_prompt,
|
||||
)
|
||||
|
||||
@@ -1,422 +0,0 @@
|
||||
"""
|
||||
Podcast API Models
|
||||
|
||||
All Pydantic request/response models for podcast endpoints.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
from typing import List, Optional, Dict, Any
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class PodcastProjectResponse(BaseModel):
|
||||
"""Response model for podcast project."""
|
||||
id: int
|
||||
project_id: str
|
||||
user_id: str
|
||||
idea: str
|
||||
duration: int
|
||||
speakers: int
|
||||
budget_cap: float
|
||||
analysis: Optional[Dict[str, Any]] = None
|
||||
queries: Optional[List[Dict[str, Any]]] = None
|
||||
selected_queries: Optional[List[str]] = None
|
||||
research: Optional[Dict[str, Any]] = None
|
||||
raw_research: Optional[Dict[str, Any]] = None
|
||||
estimate: Optional[Dict[str, Any]] = None
|
||||
script_data: Optional[Dict[str, Any]] = None
|
||||
bible: Optional[Dict[str, Any]] = None
|
||||
render_jobs: Optional[List[Dict[str, Any]]] = None
|
||||
knobs: Optional[Dict[str, Any]] = None
|
||||
research_provider: Optional[str] = None
|
||||
show_script_editor: bool = False
|
||||
show_render_queue: bool = False
|
||||
current_step: Optional[str] = None
|
||||
status: str = "draft"
|
||||
is_favorite: bool = False
|
||||
final_video_url: Optional[str] = None
|
||||
avatar_url: Optional[str] = None
|
||||
avatar_prompt: Optional[str] = None
|
||||
avatar_persona_id: Optional[str] = None
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
class PodcastAnalyzeRequest(BaseModel):
|
||||
"""Request model for podcast idea analysis."""
|
||||
idea: str = Field(..., description="Podcast topic or idea")
|
||||
duration: int = Field(default=10, description="Target duration in minutes")
|
||||
speakers: int = Field(default=1, description="Number of speakers")
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
||||
avatar_url: Optional[str] = Field(None, description="Current avatar URL if selected")
|
||||
feedback: Optional[str] = Field(None, description="User feedback for regeneration")
|
||||
|
||||
|
||||
class PodcastAnalyzeResponse(BaseModel):
|
||||
"""Response model for podcast idea analysis."""
|
||||
audience: str
|
||||
content_type: str
|
||||
top_keywords: list[str]
|
||||
suggested_outlines: list[Dict[str, Any]]
|
||||
title_suggestions: list[str]
|
||||
research_queries: Optional[List[Dict[str, str]]] = None
|
||||
exa_suggested_config: Optional[Dict[str, Any]] = None
|
||||
bible: Optional[Dict[str, Any]] = None
|
||||
avatar_url: Optional[str] = None
|
||||
avatar_prompt: Optional[str] = None
|
||||
|
||||
|
||||
class PodcastEnhanceIdeaRequest(BaseModel):
|
||||
"""Request model for enhancing a podcast idea with AI."""
|
||||
idea: str = Field(..., description="The raw podcast idea or keywords")
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
||||
|
||||
|
||||
class PodcastEnhanceIdeaResponse(BaseModel):
|
||||
"""Response model for enhanced podcast idea."""
|
||||
enhanced_ideas: List[str] = Field(..., description="3 AI-enhanced topic choices")
|
||||
rationales: List[str] = Field(..., description="Rationale for each enhanced idea")
|
||||
|
||||
|
||||
class PodcastScriptRequest(BaseModel):
|
||||
"""Request model for podcast script generation."""
|
||||
idea: str = Field(..., description="Podcast idea or topic")
|
||||
duration_minutes: int = Field(default=10, description="Target duration in minutes")
|
||||
speakers: int = Field(default=1, description="Number of speakers")
|
||||
research: Optional[Dict[str, Any]] = Field(None, description="Optional research payload to ground the script")
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||
outline: Optional[Dict[str, Any]] = Field(None, description="The refined episode outline to follow")
|
||||
analysis: Optional[Dict[str, Any]] = Field(None, description="The full analysis context (audience, keywords, etc.)")
|
||||
|
||||
|
||||
class PodcastSceneLine(BaseModel):
|
||||
speaker: str
|
||||
text: str
|
||||
emphasis: Optional[bool] = False
|
||||
|
||||
|
||||
class PodcastScene(BaseModel):
|
||||
id: str
|
||||
title: str
|
||||
duration: int
|
||||
lines: list[PodcastSceneLine]
|
||||
approved: bool = False
|
||||
emotion: Optional[str] = None
|
||||
imageUrl: Optional[str] = None # Generated image URL for video generation
|
||||
|
||||
|
||||
class PodcastExaConfig(BaseModel):
|
||||
"""Exa config for podcast research."""
|
||||
exa_search_type: Optional[str] = Field(default="auto", description="auto | keyword | neural")
|
||||
exa_category: Optional[str] = None
|
||||
exa_include_domains: List[str] = []
|
||||
exa_exclude_domains: List[str] = []
|
||||
max_sources: int = 8
|
||||
include_statistics: Optional[bool] = False
|
||||
date_range: Optional[str] = Field(default=None, description="last_month | last_3_months | last_year | all_time")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_domains(self):
|
||||
if self.exa_include_domains and self.exa_exclude_domains:
|
||||
# Exa API does not allow both include and exclude domains together with contents
|
||||
# Prefer include_domains and drop exclude_domains
|
||||
self.exa_exclude_domains = []
|
||||
return self
|
||||
|
||||
|
||||
class PodcastExaResearchRequest(BaseModel):
|
||||
"""Request for podcast research using Exa directly (no blog writer)."""
|
||||
topic: str
|
||||
queries: List[str]
|
||||
exa_config: Optional[PodcastExaConfig] = None
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||
analysis: Optional[Dict[str, Any]] = Field(None, description="Podcast analysis context (audience, content type, etc.)")
|
||||
|
||||
|
||||
class PodcastExaSource(BaseModel):
|
||||
title: str = ""
|
||||
url: str = ""
|
||||
excerpt: str = ""
|
||||
published_at: Optional[str] = None
|
||||
highlights: Optional[List[str]] = None
|
||||
summary: Optional[str] = None
|
||||
source_type: Optional[str] = None
|
||||
index: Optional[int] = None
|
||||
image: Optional[str] = None
|
||||
author: Optional[str] = None
|
||||
|
||||
|
||||
class PodcastResearchInsight(BaseModel):
|
||||
"""Deep insight extracted from research."""
|
||||
title: str
|
||||
content: str
|
||||
source_indices: List[int] = []
|
||||
|
||||
|
||||
class PodcastExaResearchResponse(BaseModel):
|
||||
sources: List[PodcastExaSource]
|
||||
search_queries: List[str] = []
|
||||
summary: str = ""
|
||||
key_insights: List[PodcastResearchInsight] = []
|
||||
expert_quotes: List[Dict[str, Any]] = []
|
||||
listener_cta: List[str] = []
|
||||
mapped_angles: List[Dict[str, Any]] = []
|
||||
cost: Optional[Dict[str, Any]] = None
|
||||
search_type: Optional[str] = None
|
||||
provider: str = "exa"
|
||||
content: Optional[str] = None # Raw aggregated content (deprecated)
|
||||
|
||||
|
||||
class PodcastScriptResponse(BaseModel):
|
||||
scenes: list[PodcastScene]
|
||||
|
||||
|
||||
class PodcastAudioRequest(BaseModel):
|
||||
"""Generate TTS for a podcast scene."""
|
||||
scene_id: str
|
||||
scene_title: str
|
||||
text: str
|
||||
voice_id: Optional[str] = "Wise_Woman"
|
||||
speed: Optional[float] = 1.0
|
||||
volume: Optional[float] = 1.0
|
||||
pitch: Optional[float] = 0.0
|
||||
emotion: Optional[str] = "neutral"
|
||||
english_normalization: Optional[bool] = False # Better number reading for statistics
|
||||
sample_rate: Optional[int] = None
|
||||
bitrate: Optional[int] = None
|
||||
channel: Optional[str] = None
|
||||
format: Optional[str] = None
|
||||
language_boost: Optional[str] = None
|
||||
enable_sync_mode: Optional[bool] = True
|
||||
|
||||
|
||||
class PodcastAudioResponse(BaseModel):
|
||||
scene_id: str
|
||||
scene_title: str
|
||||
audio_filename: str
|
||||
audio_url: str
|
||||
provider: str
|
||||
model: str
|
||||
voice_id: str
|
||||
text_length: int
|
||||
file_size: int
|
||||
cost: float
|
||||
|
||||
|
||||
class PodcastProjectListResponse(BaseModel):
|
||||
"""Response model for project list."""
|
||||
projects: List[PodcastProjectResponse]
|
||||
total: int
|
||||
limit: int
|
||||
offset: int
|
||||
|
||||
|
||||
class CreateProjectRequest(BaseModel):
|
||||
"""Request model for creating a project."""
|
||||
project_id: str = Field(..., description="Unique project ID")
|
||||
idea: str = Field(..., description="Episode idea or URL")
|
||||
duration: int = Field(..., description="Duration in minutes")
|
||||
speakers: int = Field(default=1, description="Number of speakers")
|
||||
budget_cap: float = Field(default=50.0, description="Budget cap in USD")
|
||||
avatar_url: Optional[str] = Field(None, description="Optional presenter avatar URL")
|
||||
|
||||
|
||||
class UpdateProjectRequest(BaseModel):
|
||||
"""Request model for updating project state."""
|
||||
analysis: Optional[Dict[str, Any]] = None
|
||||
queries: Optional[List[Dict[str, Any]]] = None
|
||||
selected_queries: Optional[List[str]] = None
|
||||
research: Optional[Dict[str, Any]] = None
|
||||
raw_research: Optional[Dict[str, Any]] = None
|
||||
estimate: Optional[Dict[str, Any]] = None
|
||||
script_data: Optional[Dict[str, Any]] = None
|
||||
bible: Optional[Dict[str, Any]] = None
|
||||
render_jobs: Optional[List[Dict[str, Any]]] = None
|
||||
knobs: Optional[Dict[str, Any]] = None
|
||||
research_provider: Optional[str] = None
|
||||
show_script_editor: Optional[bool] = None
|
||||
show_render_queue: Optional[bool] = None
|
||||
current_step: Optional[str] = None
|
||||
status: Optional[str] = None
|
||||
final_video_url: Optional[str] = None
|
||||
|
||||
|
||||
class PodcastCombineAudioRequest(BaseModel):
|
||||
"""Request model for combining podcast audio files."""
|
||||
project_id: str
|
||||
scene_ids: List[str] = Field(..., description="List of scene IDs to combine")
|
||||
scene_audio_urls: List[str] = Field(..., description="List of audio URLs for each scene")
|
||||
|
||||
|
||||
class PodcastCombineAudioResponse(BaseModel):
|
||||
"""Response model for combined podcast audio."""
|
||||
combined_audio_url: str
|
||||
combined_audio_filename: str
|
||||
total_duration: float
|
||||
file_size: int
|
||||
scene_count: int
|
||||
|
||||
|
||||
class PodcastImageRequest(BaseModel):
|
||||
"""Request for generating an image for a podcast scene."""
|
||||
scene_id: str
|
||||
scene_title: str
|
||||
scene_content: Optional[str] = None # Optional: scene lines text for context
|
||||
idea: Optional[str] = None # Optional: podcast idea for context
|
||||
base_avatar_url: Optional[str] = None # Base avatar image URL for scene variations
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||
width: int = 1024
|
||||
height: int = 1024
|
||||
custom_prompt: Optional[str] = None # Custom prompt from user (overrides auto-generated prompt)
|
||||
style: Optional[str] = None # "Auto", "Fiction", or "Realistic"
|
||||
rendering_speed: Optional[str] = None # "Default", "Turbo", or "Quality"
|
||||
aspect_ratio: Optional[str] = None # "1:1", "16:9", "9:16", "4:3", "3:4"
|
||||
|
||||
|
||||
class PodcastImageResponse(BaseModel):
|
||||
"""Response for podcast scene image generation."""
|
||||
scene_id: str
|
||||
scene_title: str
|
||||
image_filename: str
|
||||
image_url: str
|
||||
width: int
|
||||
height: int
|
||||
provider: str
|
||||
model: Optional[str] = None
|
||||
cost: float
|
||||
|
||||
|
||||
class PodcastVideoGenerationRequest(BaseModel):
|
||||
"""Request model for podcast video generation."""
|
||||
project_id: str = Field(..., description="Podcast project ID")
|
||||
scene_id: str = Field(..., description="Scene ID")
|
||||
scene_title: str = Field(..., description="Scene title")
|
||||
audio_url: str = Field(..., description="URL to the generated audio file")
|
||||
avatar_image_url: Optional[str] = Field(None, description="URL to scene image (required for video generation)")
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||
resolution: str = Field("720p", description="Video resolution (480p or 720p)")
|
||||
prompt: Optional[str] = Field(None, description="Optional animation prompt override")
|
||||
seed: Optional[int] = Field(-1, description="Random seed; -1 for random")
|
||||
mask_image_url: Optional[str] = Field(None, description="Optional mask image URL to specify animated region")
|
||||
|
||||
|
||||
class PodcastVideoGenerationResponse(BaseModel):
|
||||
"""Response model for podcast video generation."""
|
||||
task_id: str
|
||||
status: str
|
||||
message: str
|
||||
|
||||
|
||||
class PodcastCombineVideosRequest(BaseModel):
|
||||
"""Request to combine scene videos into final podcast"""
|
||||
project_id: str = Field(..., description="Project ID")
|
||||
scene_video_urls: list[str] = Field(..., description="List of scene video URLs in order")
|
||||
podcast_title: str = Field(default="Podcast", description="Title for the final podcast video")
|
||||
|
||||
|
||||
class PodcastCombineVideosResponse(BaseModel):
|
||||
"""Response from combine videos endpoint"""
|
||||
task_id: str
|
||||
status: str
|
||||
message: str
|
||||
|
||||
|
||||
class AudioDubbingQuality(str, Enum):
|
||||
LOW = "low"
|
||||
HIGH = "high"
|
||||
|
||||
@classmethod
|
||||
def from_string(cls, value: str) -> "AudioDubbingQuality":
|
||||
if value.lower() == "high":
|
||||
return cls.HIGH
|
||||
return cls.LOW
|
||||
|
||||
|
||||
class PodcastAudioDubRequest(BaseModel):
|
||||
"""Request model for audio dubbing."""
|
||||
source_audio_url: str = Field(..., description="URL or path to source audio file")
|
||||
source_language: Optional[str] = Field(None, description="Source language code (auto-detected if None)")
|
||||
target_language: str = Field(..., description="Target language for dubbing")
|
||||
quality: str = Field(default="low", description="Translation quality: low (DeepL) or high (WaveSpeed)")
|
||||
voice_id: Optional[str] = Field(default="Wise_Woman", description="Voice ID for TTS")
|
||||
speed: Optional[float] = Field(default=1.0, ge=0.5, le=2.0, description="Speech speed (0.5-2.0)")
|
||||
emotion: Optional[str] = Field(default="happy", description="Emotion for TTS voice")
|
||||
preserve_emotion: Optional[bool] = Field(default=True, description="Preserve emotional tone in translation")
|
||||
use_voice_clone: Optional[bool] = Field(default=False, description="Use voice cloning to preserve original speaker's voice")
|
||||
custom_voice_id: Optional[str] = Field(None, description="Custom name for the cloned voice")
|
||||
voice_clone_accuracy: Optional[float] = Field(default=0.7, ge=0.1, le=1.0, description="Voice cloning accuracy (0.1-1.0)")
|
||||
|
||||
|
||||
class PodcastAudioDubResponse(BaseModel):
|
||||
"""Response model for audio dubbing task creation."""
|
||||
task_id: str
|
||||
status: str = "pending"
|
||||
message: str = "Audio dubbing task created"
|
||||
|
||||
|
||||
class PodcastAudioDubResult(BaseModel):
|
||||
"""Response model for completed audio dubbing."""
|
||||
dubbed_audio_url: str
|
||||
dubbed_audio_filename: str
|
||||
original_transcript: str
|
||||
translated_transcript: str
|
||||
source_language: str
|
||||
target_language: str
|
||||
voice_id: str
|
||||
quality: str
|
||||
duration_seconds: int
|
||||
file_size: int
|
||||
cost: float
|
||||
task_id: str
|
||||
status: str = "completed"
|
||||
voice_clone_used: Optional[bool] = Field(default=False, description="Whether voice cloning was used")
|
||||
cloned_voice_id: Optional[str] = Field(None, description="ID of the cloned voice if voice_clone_used=True")
|
||||
|
||||
|
||||
class PodcastAudioDubEstimateRequest(BaseModel):
|
||||
"""Request model for dubbing cost estimation."""
|
||||
audio_duration_seconds: float = Field(..., description="Duration of source audio in seconds")
|
||||
target_language: str = Field(..., description="Target language")
|
||||
quality: str = Field(default="low", description="Translation quality")
|
||||
use_voice_clone: Optional[bool] = Field(default=False, description="Include voice cloning cost")
|
||||
|
||||
|
||||
class PodcastAudioDubEstimateResponse(BaseModel):
|
||||
"""Response model for dubbing cost estimation."""
|
||||
estimated_characters: int
|
||||
translation_cost: float
|
||||
tts_cost: float
|
||||
voice_clone_cost: float = 0.0
|
||||
total_cost: float
|
||||
currency: str = "USD"
|
||||
|
||||
|
||||
class VoiceCloneRequest(BaseModel):
|
||||
"""Request model for voice cloning."""
|
||||
source_audio_url: str = Field(..., description="URL or path to source audio file (10-60 seconds recommended)")
|
||||
custom_voice_id: Optional[str] = Field(None, description="Custom name for the cloned voice")
|
||||
accuracy: Optional[float] = Field(default=0.7, ge=0.1, le=1.0, description="Cloning accuracy (0.1-1.0)")
|
||||
language_boost: Optional[str] = Field(None, description="Language to optimize the voice for")
|
||||
|
||||
|
||||
class VoiceCloneResponse(BaseModel):
|
||||
"""Response model for voice cloning."""
|
||||
task_id: str
|
||||
status: str = "pending"
|
||||
message: str = "Voice cloning task created"
|
||||
|
||||
|
||||
class VoiceCloneResult(BaseModel):
|
||||
"""Response model for completed voice cloning."""
|
||||
voice_id: str
|
||||
voice_url: str
|
||||
source_language: str
|
||||
accuracy: float
|
||||
file_size: int
|
||||
task_id: str
|
||||
status: str = "completed"
|
||||
|
||||
@@ -1,837 +0,0 @@
|
||||
import { ResearchProvider, ResearchConfig } from "./blogWriterApi";
|
||||
import {
|
||||
storyWriterApi,
|
||||
StorySetupGenerationResponse,
|
||||
} from "./storyWriterApi";
|
||||
import { getResearchConfig, ResearchPersona } from "../api/researchConfig";
|
||||
import { aiApiClient } from "../api/client";
|
||||
import {
|
||||
CreateProjectPayload,
|
||||
CreateProjectResult,
|
||||
Fact,
|
||||
Knobs,
|
||||
PodcastAnalysis,
|
||||
PodcastEstimate,
|
||||
Query,
|
||||
RenderJobResult,
|
||||
Research,
|
||||
Scene,
|
||||
Script,
|
||||
} from "../components/PodcastMaker/types";
|
||||
import { checkPreflight, PreflightOperation } from "./billingService";
|
||||
import { TaskStatus } from "./storyWriterApi";
|
||||
|
||||
const DEFAULT_KNOBS: Knobs = {
|
||||
voice_emotion: "neutral",
|
||||
voice_speed: 1,
|
||||
resolution: "720p",
|
||||
scene_length_target: 45,
|
||||
sample_rate: 24000,
|
||||
bitrate: "standard",
|
||||
};
|
||||
|
||||
// const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
const createId = (prefix: string) => {
|
||||
if (typeof crypto !== "undefined" && typeof crypto.randomUUID === "function") {
|
||||
return `${prefix}_${crypto.randomUUID()}`;
|
||||
}
|
||||
return `${prefix}_${Date.now()}_${Math.floor(Math.random() * 10000)}`;
|
||||
};
|
||||
|
||||
type OptionLike = StorySetupGenerationResponse["options"][0] | { plot_elements?: string; premise?: string };
|
||||
|
||||
const deriveSegments = (option?: OptionLike): string[] => {
|
||||
const segments: string[] = [];
|
||||
if (option?.plot_elements) {
|
||||
option.plot_elements
|
||||
.split(/[,.;]+/)
|
||||
.map((p) => p.trim())
|
||||
.filter(Boolean)
|
||||
.forEach((p) => segments.push(p));
|
||||
}
|
||||
if (!segments.length && "premise" in (option || {}) && (option as any)?.premise) {
|
||||
segments.push("Intro", "Key Takeaways", "Examples", "CTA");
|
||||
}
|
||||
return segments.slice(0, 5);
|
||||
};
|
||||
|
||||
const estimateCosts = ({
|
||||
minutes,
|
||||
scenes,
|
||||
chars,
|
||||
quality,
|
||||
avatars,
|
||||
queryCount = 3,
|
||||
}: {
|
||||
minutes: number;
|
||||
scenes: number;
|
||||
chars: number;
|
||||
quality: string;
|
||||
avatars: number;
|
||||
queryCount?: number;
|
||||
}): PodcastEstimate => {
|
||||
const secs = Math.max(60, minutes * 60);
|
||||
const ttsCost = (chars / 1000) * 0.05;
|
||||
const avatarCost = avatars * 0.15;
|
||||
const videoRate = quality === "hd" ? 0.06 : 0.03;
|
||||
const videoCost = secs * videoRate;
|
||||
const researchCost = +(Math.max(1, queryCount) * 0.1).toFixed(2);
|
||||
const total = +(ttsCost + avatarCost + videoCost + researchCost).toFixed(2);
|
||||
return {
|
||||
ttsCost: +ttsCost.toFixed(2),
|
||||
avatarCost: +avatarCost.toFixed(2),
|
||||
videoCost: +videoCost.toFixed(2),
|
||||
researchCost,
|
||||
total,
|
||||
};
|
||||
};
|
||||
|
||||
const mapPersonaQueries = (persona: ResearchPersona | undefined, seed: string): Query[] => {
|
||||
const baseIdea = seed || "AI marketing for small businesses";
|
||||
const personaKeywords = persona?.suggested_keywords?.filter(Boolean) || [];
|
||||
const angles = persona?.research_angles ?? [];
|
||||
const generated: Query[] = [];
|
||||
|
||||
const addQuery = (q: string, why: string, needsRecent = false) => {
|
||||
if (!q.trim()) return;
|
||||
generated.push({
|
||||
id: createId("q"),
|
||||
query: q.trim(),
|
||||
rationale: why,
|
||||
needsRecentStats: needsRecent,
|
||||
});
|
||||
};
|
||||
|
||||
if (personaKeywords.length) {
|
||||
personaKeywords.slice(0, 4).forEach((k, idx) =>
|
||||
addQuery(k, angles[idx % Math.max(1, angles.length)] || "Persona-aligned query", /202[45]|latest|trend/i.test(k))
|
||||
);
|
||||
}
|
||||
|
||||
if (!generated.length) {
|
||||
addQuery(`How is ${baseIdea} evolving in 2024?`, "Trend + outcome focus", true);
|
||||
addQuery(`Best practices for ${baseIdea}`, "Actionable guidance", false);
|
||||
addQuery(`${baseIdea} case studies with ROI`, "Proof and outcomes", true);
|
||||
addQuery(`${baseIdea} risks and objections`, "Address listener concerns", false);
|
||||
}
|
||||
|
||||
return generated.slice(0, 6);
|
||||
};
|
||||
|
||||
const mapSourcesToFacts = (sources: ExaSource[]): Fact[] => {
|
||||
if (!sources || !sources.length) return [];
|
||||
return sources.slice(0, 12).map((source: ExaSource, idx: number) => ({
|
||||
id: source.url || createId("fact"),
|
||||
quote: source.excerpt || source.title || "Insight",
|
||||
url: source.url || "",
|
||||
date: source.published_at || "Unknown",
|
||||
confidence: typeof (source as any).credibility_score === "number" ? (source as any).credibility_score : Math.max(0.5, 0.85 - idx * 0.02),
|
||||
image: source.image,
|
||||
author: source.author,
|
||||
highlights: source.highlights,
|
||||
}));
|
||||
};
|
||||
|
||||
type ExaSource = {
|
||||
title?: string;
|
||||
url?: string;
|
||||
excerpt?: string;
|
||||
published_at?: string;
|
||||
highlights?: string[];
|
||||
summary?: string;
|
||||
source_type?: string;
|
||||
index?: number;
|
||||
image?: string;
|
||||
author?: string;
|
||||
};
|
||||
|
||||
type ExaResearchResult = {
|
||||
sources: ExaSource[];
|
||||
search_queries?: string[];
|
||||
cost?: { total?: number };
|
||||
search_type?: string;
|
||||
provider?: string;
|
||||
content?: string;
|
||||
};
|
||||
|
||||
const mapExaResearchResponse = (response: any): Research => {
|
||||
const factCards = mapSourcesToFacts(response.sources);
|
||||
// Use backend summary if available, otherwise use full content (no truncation) or fallback text
|
||||
const summary = response.summary || response.content || "Research completed.";
|
||||
|
||||
const keyInsights = (response.key_insights || []).map((insight: any) => ({
|
||||
title: insight.title || "Insight",
|
||||
content: insight.content || "",
|
||||
source_indices: insight.source_indices || []
|
||||
}));
|
||||
|
||||
const expertQuotes = (response.expert_quotes || []).map((eq: any) => ({
|
||||
quote: eq.quote || eq.text || "",
|
||||
source_index: eq.source_index ?? 0
|
||||
}));
|
||||
|
||||
const listenerCta = response.listener_cta || [];
|
||||
|
||||
const mappedAngles = (response.mapped_angles || []).map((angle: any) => ({
|
||||
title: angle.title || "",
|
||||
why: angle.why || angle.rationale || "",
|
||||
mappedFactIds: angle.mapped_fact_ids || angle.mappedFactIds || []
|
||||
}));
|
||||
|
||||
return {
|
||||
summary,
|
||||
keyInsights,
|
||||
factCards,
|
||||
mappedAngles,
|
||||
expertQuotes,
|
||||
listenerCta,
|
||||
searchQueries: response.search_queries,
|
||||
searchType: response.search_type,
|
||||
provider: response.provider || "exa",
|
||||
cost: response.cost?.total,
|
||||
sourceCount: response.sources?.length || 0,
|
||||
};
|
||||
};
|
||||
|
||||
const ensurePreflight = async (operation: PreflightOperation) => {
|
||||
const result = await checkPreflight(operation);
|
||||
if (!result.can_proceed) {
|
||||
const message = result.operations[0]?.message || "Pre-flight validation failed";
|
||||
throw new Error(message);
|
||||
}
|
||||
return result;
|
||||
};
|
||||
|
||||
export const podcastApi = {
|
||||
async createProject(payload: CreateProjectPayload, bible?: any, feedback?: string): Promise<CreateProjectResult> {
|
||||
const storyIdea = payload.ideaOrUrl || "AI marketing for small businesses";
|
||||
|
||||
await ensurePreflight({
|
||||
provider: "gemini",
|
||||
operation_type: "podcast_analysis",
|
||||
tokens_requested: 1500,
|
||||
actual_provider_name: "gemini",
|
||||
});
|
||||
|
||||
// Podcast-specific analysis (not story setup)
|
||||
const analysisResp = await aiApiClient.post("/api/podcast/analyze", {
|
||||
idea: storyIdea,
|
||||
duration: payload.duration,
|
||||
speakers: payload.speakers,
|
||||
bible: bible,
|
||||
avatar_url: payload.avatarUrl,
|
||||
feedback: feedback, // Pass feedback to backend
|
||||
});
|
||||
|
||||
const outlines = (analysisResp.data?.suggested_outlines || []).map((o: any, idx: number) => ({
|
||||
id: o.id || `outline-${idx + 1}`,
|
||||
title: o.title || `Outline ${idx + 1}`,
|
||||
segments: Array.isArray(o.segments) ? o.segments : deriveSegments({ plot_elements: o.segments }),
|
||||
}));
|
||||
|
||||
const analysis: PodcastAnalysis = {
|
||||
audience: analysisResp.data?.audience || "Growth-minded pros",
|
||||
contentType: analysisResp.data?.content_type || "Podcast interview",
|
||||
topKeywords: analysisResp.data?.top_keywords || outlines[0]?.segments?.slice(0, 3) || [],
|
||||
suggestedOutlines: outlines,
|
||||
suggestedKnobs: { ...DEFAULT_KNOBS, ...payload.knobs },
|
||||
titleSuggestions: (analysisResp.data?.title_suggestions || []).filter(Boolean),
|
||||
research_queries: analysisResp.data?.research_queries || [],
|
||||
exaSuggestedConfig: analysisResp.data?.exa_suggested_config || undefined,
|
||||
};
|
||||
|
||||
const researchConfig = await getResearchConfig().catch(() => null);
|
||||
|
||||
// Use AI-generated queries if available, fallback to legacy mapping
|
||||
let queries: Query[] = [];
|
||||
if (analysis.research_queries && analysis.research_queries.length > 0) {
|
||||
queries = analysis.research_queries.map(rq => ({
|
||||
id: createId("q"),
|
||||
query: rq.query,
|
||||
rationale: rq.rationale,
|
||||
needsRecentStats: /202[45]|latest|trend/i.test(rq.query)
|
||||
}));
|
||||
} else {
|
||||
queries = mapPersonaQueries(researchConfig?.research_persona, storyIdea);
|
||||
}
|
||||
|
||||
const projectId = createId("podcast");
|
||||
const estimate = estimateCosts({
|
||||
minutes: payload.duration,
|
||||
scenes: Math.ceil((payload.duration * 60) / (payload.knobs.scene_length_target || DEFAULT_KNOBS.scene_length_target)),
|
||||
chars: Math.max(1000, payload.duration * 900),
|
||||
quality: payload.knobs.bitrate || "standard",
|
||||
avatars: payload.speakers,
|
||||
queryCount: queries.length || 3,
|
||||
});
|
||||
|
||||
return {
|
||||
projectId,
|
||||
analysis,
|
||||
estimate,
|
||||
queries,
|
||||
bible: analysisResp.data?.bible || undefined,
|
||||
avatar_url: analysisResp.data?.avatar_url || null,
|
||||
avatar_prompt: analysisResp.data?.avatar_prompt || null,
|
||||
};
|
||||
},
|
||||
|
||||
async enhanceIdea(params: { idea: string; bible?: any }): Promise<{ enhanced_ideas: string[]; rationales: string[] }> {
|
||||
const response = await aiApiClient.post("/api/podcast/idea/enhance", params);
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async runResearch(params: {
|
||||
projectId: string;
|
||||
topic: string;
|
||||
approvedQueries: Query[];
|
||||
provider?: ResearchProvider;
|
||||
exaConfig?: ResearchConfig;
|
||||
bible?: any;
|
||||
analysis?: PodcastAnalysis | null;
|
||||
onProgress?: (message: string) => void;
|
||||
}): Promise<{ research: Research; raw: any }> {
|
||||
const keywords = params.approvedQueries.map((q) => q.query).filter(Boolean);
|
||||
if (!keywords.length) {
|
||||
throw new Error("At least one query must be approved for research.");
|
||||
}
|
||||
|
||||
// Ensure Exa payload respects API constraint: when requesting contents, only one of includeDomains or excludeDomains.
|
||||
let sanitizedExaConfig: ResearchConfig | undefined = params.exaConfig;
|
||||
if (sanitizedExaConfig && sanitizedExaConfig.exa_include_domains?.length) {
|
||||
sanitizedExaConfig = {
|
||||
...sanitizedExaConfig,
|
||||
exa_exclude_domains: undefined,
|
||||
};
|
||||
} else if (sanitizedExaConfig && sanitizedExaConfig.exa_exclude_domains?.length) {
|
||||
sanitizedExaConfig = {
|
||||
...sanitizedExaConfig,
|
||||
exa_include_domains: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
await ensurePreflight({
|
||||
provider: "exa",
|
||||
operation_type: "exa_neural_search",
|
||||
tokens_requested: 0,
|
||||
actual_provider_name: "exa",
|
||||
});
|
||||
|
||||
const response = await aiApiClient.post("/api/podcast/research/exa", {
|
||||
topic: params.topic || keywords[0],
|
||||
queries: keywords,
|
||||
exa_config: sanitizedExaConfig,
|
||||
bible: params.bible,
|
||||
analysis: params.analysis,
|
||||
});
|
||||
|
||||
const exaResult = response.data as ExaResearchResult;
|
||||
if (params.onProgress) {
|
||||
params.onProgress("Deep research completed with Exa.");
|
||||
}
|
||||
const mapped = mapExaResearchResponse(exaResult);
|
||||
return { research: mapped, raw: exaResult };
|
||||
},
|
||||
|
||||
async generateScript(params: {
|
||||
projectId: string;
|
||||
idea: string;
|
||||
research?: ExaResearchResult | null;
|
||||
knobs: Knobs;
|
||||
speakers: number;
|
||||
durationMinutes: number;
|
||||
bible?: any;
|
||||
outline?: any;
|
||||
analysis?: PodcastAnalysis | null;
|
||||
}): Promise<Script> {
|
||||
await ensurePreflight({
|
||||
provider: "gemini",
|
||||
operation_type: "script_generation",
|
||||
tokens_requested: 2000,
|
||||
actual_provider_name: "gemini",
|
||||
});
|
||||
|
||||
const response = await aiApiClient.post("/api/podcast/script", {
|
||||
idea: params.idea,
|
||||
duration_minutes: params.durationMinutes,
|
||||
speakers: params.speakers,
|
||||
research: params.research,
|
||||
bible: params.bible,
|
||||
outline: params.outline,
|
||||
analysis: params.analysis,
|
||||
});
|
||||
|
||||
const scenes = response.data?.scenes || [];
|
||||
const scriptScenes: Scene[] = scenes.map((scene: any) => ({
|
||||
id: scene.id || createId("scene"),
|
||||
title: scene.title || "Scene",
|
||||
duration: scene.duration || Math.max(20, params.knobs.scene_length_target || DEFAULT_KNOBS.scene_length_target),
|
||||
lines:
|
||||
Array.isArray(scene.lines) && scene.lines.length
|
||||
? scene.lines.map((l: any) => ({
|
||||
id: createId("line"),
|
||||
speaker: l.speaker || "Host",
|
||||
text: l.text || "",
|
||||
}))
|
||||
: [
|
||||
{
|
||||
id: createId("line"),
|
||||
speaker: "Host",
|
||||
text: "Let's dive into today's topic.",
|
||||
},
|
||||
],
|
||||
approved: false,
|
||||
}));
|
||||
|
||||
return { scenes: scriptScenes };
|
||||
},
|
||||
|
||||
async previewLine(
|
||||
text: string,
|
||||
options: { voiceId?: string; speed?: number; emotion?: string } = {}
|
||||
): Promise<{ ok: boolean; message: string; audioUrl?: string }> {
|
||||
await ensurePreflight({
|
||||
provider: "audio",
|
||||
operation_type: "tts_preview",
|
||||
tokens_requested: text.length,
|
||||
actual_provider_name: "wavespeed",
|
||||
});
|
||||
|
||||
const response = await storyWriterApi.generateAIAudio({
|
||||
scene_number: 0,
|
||||
scene_title: "Preview",
|
||||
text,
|
||||
voice_id: options.voiceId || "Wise_Woman",
|
||||
speed: options.speed || 1.0,
|
||||
emotion: options.emotion || "neutral",
|
||||
});
|
||||
|
||||
if (!response.success) {
|
||||
throw new Error(response.error || "Preview failed");
|
||||
}
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
message: "Preview ready – opening audio in new tab.",
|
||||
audioUrl: response.audio_url,
|
||||
};
|
||||
},
|
||||
|
||||
async renderSceneAudio(params: {
|
||||
scene: Scene;
|
||||
voiceId?: string;
|
||||
emotion?: string; // Fallback if scene doesn't have emotion
|
||||
speed?: number;
|
||||
volume?: number;
|
||||
pitch?: number;
|
||||
englishNormalization?: boolean;
|
||||
sampleRate?: number;
|
||||
bitrate?: number;
|
||||
channel?: "1" | "2";
|
||||
format?: "mp3" | "wav" | "pcm" | "flac";
|
||||
languageBoost?: string;
|
||||
}): Promise<RenderJobResult> {
|
||||
// Use scene-specific emotion if available, otherwise fallback to provided/default
|
||||
const sceneEmotion = params.scene.emotion || params.emotion || "neutral";
|
||||
|
||||
// Optimize text for Minimax Speech-02-HD TTS
|
||||
// - Strip markdown formatting (bold, italic, etc.) - TTS reads it literally
|
||||
// - Use pause markers <#x#> for natural speech rhythm
|
||||
// - Add longer pauses for speaker changes
|
||||
// - Preserve punctuation for natural breathing
|
||||
// - Add emphasis pauses for important points
|
||||
const text = params.scene.lines
|
||||
.map((line, idx) => {
|
||||
let lineText = line.text.trim();
|
||||
|
||||
// Strip markdown formatting - TTS reads asterisks and other markdown literally
|
||||
// Remove bold (**text** or __text__)
|
||||
lineText = lineText.replace(/\*\*([^*]+)\*\*/g, '$1'); // **bold**
|
||||
lineText = lineText.replace(/\*([^*]+)\*/g, '$1'); // *bold* (single asterisk)
|
||||
lineText = lineText.replace(/__([^_]+)__/g, '$1'); // __bold__
|
||||
lineText = lineText.replace(/_([^_]+)_/g, '$1'); // _italic_ (single underscore)
|
||||
// Remove any remaining stray asterisks or underscores
|
||||
lineText = lineText.replace(/\*+/g, ''); // Remove any remaining asterisks
|
||||
lineText = lineText.replace(/_+/g, ''); // Remove any remaining underscores
|
||||
// Clean up extra spaces
|
||||
lineText = lineText.replace(/\s+/g, ' ').trim();
|
||||
|
||||
// Preserve punctuation (Minimax uses it for natural breathing)
|
||||
// Don't strip punctuation - it helps TTS understand natural pauses
|
||||
|
||||
// Add emphasis pause after lines marked with emphasis
|
||||
if (line.emphasis) {
|
||||
// Minimal pause after emphasized content (0.15s for subtle emphasis)
|
||||
lineText = `${lineText}<#0.15#>`;
|
||||
}
|
||||
|
||||
// Check for speaker change (longer pause for natural conversation flow)
|
||||
const prevLine = idx > 0 ? params.scene.lines[idx - 1] : null;
|
||||
const isSpeakerChange = prevLine && prevLine.speaker !== line.speaker;
|
||||
|
||||
if (isSpeakerChange) {
|
||||
// Short pause for speaker changes (0.2s - enough for natural transition)
|
||||
lineText = `<#0.2#>${lineText}`;
|
||||
}
|
||||
|
||||
// Add minimal pause between lines (only between regular lines, very short)
|
||||
if (idx < params.scene.lines.length - 1) {
|
||||
if (!line.emphasis && !isSpeakerChange) {
|
||||
// Very short pause between lines (0.08s - barely noticeable but helps flow)
|
||||
lineText = `${lineText}<#0.08#>`;
|
||||
}
|
||||
// If emphasis or speaker change, the pause is already added above
|
||||
}
|
||||
|
||||
return lineText;
|
||||
})
|
||||
.join(" ");
|
||||
|
||||
// Validate character limit (Minimax max: 10,000 characters)
|
||||
const MAX_CHARS = 10000;
|
||||
let textToUse = text;
|
||||
if (text.length > MAX_CHARS) {
|
||||
console.warn(
|
||||
`[Podcast] Scene "${params.scene.title}" exceeds ${MAX_CHARS} character limit (${text.length} chars). Truncating...`
|
||||
);
|
||||
// Truncate at word boundary to avoid cutting mid-word
|
||||
const truncated = text.substring(0, MAX_CHARS);
|
||||
const lastSpace = truncated.lastIndexOf(" ");
|
||||
textToUse = lastSpace > 0 ? truncated.substring(0, lastSpace) : truncated;
|
||||
}
|
||||
|
||||
await ensurePreflight({
|
||||
provider: "audio",
|
||||
operation_type: "tts_full_render",
|
||||
tokens_requested: textToUse.length,
|
||||
actual_provider_name: "wavespeed",
|
||||
});
|
||||
|
||||
const response = await aiApiClient.post("/api/podcast/audio", {
|
||||
scene_id: params.scene.id,
|
||||
scene_title: params.scene.title,
|
||||
text: textToUse,
|
||||
voice_id: params.voiceId || "Wise_Woman",
|
||||
speed: params.speed ?? 1.0, // Normal speed (was 0.9, but too slow - causing duration issues)
|
||||
volume: params.volume ?? 1.0,
|
||||
pitch: params.pitch ?? 0.0,
|
||||
emotion: sceneEmotion,
|
||||
english_normalization: params.englishNormalization ?? true, // Better number reading for statistics
|
||||
sample_rate: params.sampleRate || null,
|
||||
bitrate: params.bitrate || null,
|
||||
channel: params.channel || null,
|
||||
format: params.format || null,
|
||||
language_boost: params.languageBoost || null,
|
||||
});
|
||||
|
||||
return {
|
||||
audioUrl: response.data.audio_url,
|
||||
audioFilename: response.data.audio_filename,
|
||||
provider: response.data.provider,
|
||||
model: response.data.model,
|
||||
cost: response.data.cost,
|
||||
voiceId: response.data.voice_id,
|
||||
fileSize: response.data.file_size,
|
||||
};
|
||||
},
|
||||
|
||||
async approveScene(params: { projectId: string; sceneId: string; notes?: string }) {
|
||||
await aiApiClient.post("/api/story/script/approve", {
|
||||
project_id: params.projectId,
|
||||
scene_id: params.sceneId,
|
||||
approved: true,
|
||||
notes: params.notes,
|
||||
});
|
||||
},
|
||||
|
||||
// Project persistence endpoints
|
||||
async saveProject(projectId: string, state: any): Promise<void> {
|
||||
try {
|
||||
await aiApiClient.put(`/api/podcast/projects/${projectId}`, state);
|
||||
} catch (error) {
|
||||
console.error("Failed to save project to database:", error);
|
||||
// Don't throw - localStorage fallback is acceptable
|
||||
}
|
||||
},
|
||||
|
||||
async loadProject(projectId: string): Promise<any> {
|
||||
const response = await aiApiClient.get(`/api/podcast/projects/${projectId}`);
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async listProjects(params?: {
|
||||
status?: string;
|
||||
favorites_only?: boolean;
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
order_by?: "updated_at" | "created_at";
|
||||
}): Promise<{ projects: any[]; total: number; limit: number; offset: number }> {
|
||||
const response = await aiApiClient.get("/api/podcast/projects", { params });
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async createProjectInDb(params: {
|
||||
project_id: string;
|
||||
idea: string;
|
||||
duration: number;
|
||||
speakers: number;
|
||||
budget_cap: number;
|
||||
avatar_url?: string | null;
|
||||
}): Promise<any> {
|
||||
const response = await aiApiClient.post("/api/podcast/projects", params);
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async updateProject(projectId: string, updates: any): Promise<any> {
|
||||
const response = await aiApiClient.put(`/api/podcast/projects/${projectId}`, updates);
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async deleteProject(projectId: string): Promise<void> {
|
||||
await aiApiClient.delete(`/api/podcast/projects/${projectId}`);
|
||||
},
|
||||
|
||||
async toggleFavorite(projectId: string): Promise<any> {
|
||||
const response = await aiApiClient.post(`/api/podcast/projects/${projectId}/favorite`);
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async saveAudioToAssetLibrary(params: {
|
||||
audioUrl: string;
|
||||
filename: string;
|
||||
title: string;
|
||||
description?: string;
|
||||
projectId: string;
|
||||
sceneId?: string;
|
||||
cost?: number;
|
||||
provider?: string;
|
||||
model?: string;
|
||||
fileSize?: number;
|
||||
}): Promise<{ assetId: number }> {
|
||||
const response = await aiApiClient.post("/api/content-assets/", {
|
||||
asset_type: "audio",
|
||||
source_module: "podcast_maker",
|
||||
filename: params.filename,
|
||||
file_url: params.audioUrl,
|
||||
title: params.title,
|
||||
description: params.description || `Podcast episode audio: ${params.title}`,
|
||||
tags: ["podcast", "audio", params.projectId],
|
||||
asset_metadata: {
|
||||
project_id: params.projectId,
|
||||
scene_id: params.sceneId,
|
||||
provider: params.provider,
|
||||
model: params.model,
|
||||
},
|
||||
provider: params.provider,
|
||||
model: params.model,
|
||||
cost: params.cost || 0,
|
||||
file_size: params.fileSize,
|
||||
mime_type: "audio/mpeg",
|
||||
});
|
||||
return { assetId: response.data.id };
|
||||
},
|
||||
|
||||
async generateVideo(params: {
|
||||
projectId: string;
|
||||
sceneId: string;
|
||||
sceneTitle: string;
|
||||
audioUrl: string;
|
||||
avatarImageUrl?: string;
|
||||
bible?: any;
|
||||
resolution?: string;
|
||||
prompt?: string;
|
||||
seed?: number;
|
||||
maskImageUrl?: string;
|
||||
}): Promise<{ taskId: string; status: string; message: string }> {
|
||||
const response = await aiApiClient.post("/api/podcast/render/video", {
|
||||
project_id: params.projectId,
|
||||
scene_id: params.sceneId,
|
||||
scene_title: params.sceneTitle,
|
||||
audio_url: params.audioUrl,
|
||||
avatar_image_url: params.avatarImageUrl,
|
||||
bible: params.bible,
|
||||
resolution: params.resolution || "720p",
|
||||
prompt: params.prompt,
|
||||
seed: params.seed ?? -1,
|
||||
mask_image_url: params.maskImageUrl,
|
||||
});
|
||||
|
||||
// Backend returns snake_case (task_id); normalize to camelCase for callers
|
||||
const { task_id, status, message } = response.data || {};
|
||||
return {
|
||||
taskId: task_id,
|
||||
status,
|
||||
message,
|
||||
};
|
||||
},
|
||||
|
||||
async pollTaskStatus(taskId: string): Promise<TaskStatus | null> {
|
||||
const response = await aiApiClient.get(`/api/podcast/task/${taskId}/status`);
|
||||
// Backend returns null if task not found
|
||||
return response.data || null;
|
||||
},
|
||||
|
||||
async listVideos(projectId?: string): Promise<{
|
||||
videos: Array<{
|
||||
scene_number: number;
|
||||
filename: string;
|
||||
video_url: string;
|
||||
file_size: number;
|
||||
}>;
|
||||
}> {
|
||||
const params = projectId ? { project_id: projectId } : {};
|
||||
const response = await aiApiClient.get("/api/podcast/videos", { params });
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async combineVideos(params: {
|
||||
projectId: string;
|
||||
sceneVideoUrls: string[];
|
||||
podcastTitle?: string;
|
||||
}): Promise<{
|
||||
taskId: string;
|
||||
status: string;
|
||||
message: string;
|
||||
}> {
|
||||
const response = await aiApiClient.post("/api/podcast/render/combine-videos", {
|
||||
project_id: params.projectId,
|
||||
scene_video_urls: params.sceneVideoUrls,
|
||||
podcast_title: params.podcastTitle || "Podcast",
|
||||
});
|
||||
|
||||
const { task_id, status, message } = response.data || {};
|
||||
return {
|
||||
taskId: task_id,
|
||||
status,
|
||||
message,
|
||||
};
|
||||
},
|
||||
|
||||
async generateSceneImage(params: {
|
||||
sceneId: string;
|
||||
sceneTitle: string;
|
||||
sceneContent?: string;
|
||||
baseAvatarUrl?: string;
|
||||
bible?: any;
|
||||
idea?: string;
|
||||
width?: number;
|
||||
height?: number;
|
||||
customPrompt?: string;
|
||||
style?: "Auto" | "Fiction" | "Realistic";
|
||||
renderingSpeed?: "Default" | "Turbo" | "Quality";
|
||||
aspectRatio?: "1:1" | "16:9" | "9:16" | "4:3" | "3:4";
|
||||
}): Promise<{
|
||||
scene_id: string;
|
||||
scene_title: string;
|
||||
image_filename: string;
|
||||
image_url: string;
|
||||
width: number;
|
||||
height: number;
|
||||
provider: string;
|
||||
model?: string;
|
||||
cost: number;
|
||||
}> {
|
||||
const response = await aiApiClient.post("/api/podcast/image", {
|
||||
scene_id: params.sceneId,
|
||||
scene_title: params.sceneTitle,
|
||||
scene_content: params.sceneContent,
|
||||
base_avatar_url: params.baseAvatarUrl || null,
|
||||
bible: params.bible,
|
||||
idea: params.idea || null,
|
||||
width: params.width || 1024,
|
||||
height: params.height || 1024,
|
||||
custom_prompt: params.customPrompt || null,
|
||||
style: params.style || null,
|
||||
rendering_speed: params.renderingSpeed || null,
|
||||
aspect_ratio: params.aspectRatio || null,
|
||||
});
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async cancelTask(taskId: string): Promise<void> {
|
||||
// Note: Task cancellation may not be fully supported by backend yet
|
||||
// This is a placeholder for future implementation
|
||||
try {
|
||||
await aiApiClient.post(`/api/story/task/${taskId}/cancel`);
|
||||
} catch (error) {
|
||||
console.warn("Task cancellation not supported:", error);
|
||||
}
|
||||
},
|
||||
|
||||
async combineAudio(params: {
|
||||
projectId: string;
|
||||
sceneIds: string[];
|
||||
sceneAudioUrls: string[];
|
||||
}): Promise<{
|
||||
combined_audio_url: string;
|
||||
combined_audio_filename: string;
|
||||
total_duration: number;
|
||||
file_size: number;
|
||||
scene_count: number;
|
||||
}> {
|
||||
const response = await aiApiClient.post("/api/podcast/combine-audio", {
|
||||
project_id: params.projectId,
|
||||
scene_ids: params.sceneIds,
|
||||
scene_audio_urls: params.sceneAudioUrls,
|
||||
});
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async uploadAvatar(file: File, projectId?: string): Promise<{ avatar_url: string; avatar_filename: string }> {
|
||||
const formData = new FormData();
|
||||
formData.append('file', file);
|
||||
if (projectId) {
|
||||
formData.append('project_id', projectId);
|
||||
}
|
||||
const response = await aiApiClient.post('/api/podcast/avatar/upload', formData, {
|
||||
headers: { 'Content-Type': 'multipart/form-data' },
|
||||
});
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async generatePresenters(
|
||||
speakers: number,
|
||||
projectId?: string,
|
||||
audience?: string,
|
||||
contentType?: string,
|
||||
topKeywords?: string[]
|
||||
): Promise<{
|
||||
avatars: Array<{ avatar_url: string; speaker_number: number; prompt?: string; persona_id?: string; seed?: number }>;
|
||||
persona_id?: string;
|
||||
}> {
|
||||
const formData = new FormData();
|
||||
formData.append('speakers', speakers.toString());
|
||||
if (projectId) {
|
||||
formData.append('project_id', projectId);
|
||||
}
|
||||
if (audience) {
|
||||
formData.append('audience', audience);
|
||||
}
|
||||
if (contentType) {
|
||||
formData.append('content_type', contentType);
|
||||
}
|
||||
if (topKeywords && Array.isArray(topKeywords) && topKeywords.length > 0) {
|
||||
formData.append('top_keywords', JSON.stringify(topKeywords));
|
||||
}
|
||||
const response = await aiApiClient.post('/api/podcast/avatar/generate', formData, {
|
||||
headers: { 'Content-Type': 'multipart/form-data' },
|
||||
});
|
||||
return response.data;
|
||||
},
|
||||
|
||||
async makeAvatarPresentable(avatarUrl: string, projectId?: string): Promise<{ avatar_url: string; avatar_filename: string }> {
|
||||
const formData = new FormData();
|
||||
formData.append('avatar_url', avatarUrl);
|
||||
if (projectId) {
|
||||
formData.append('project_id', projectId);
|
||||
}
|
||||
const response = await aiApiClient.post('/api/podcast/avatar/make-presentable', formData, {
|
||||
headers: { 'Content-Type': 'multipart/form-data' },
|
||||
});
|
||||
return response.data;
|
||||
},
|
||||
};
|
||||
|
||||
export type PodcastApi = typeof podcastApi;
|
||||
|
||||
@@ -1,244 +0,0 @@
|
||||
"""
|
||||
Podcast Research Handlers
|
||||
|
||||
Research endpoints using Exa provider and LLM summarization.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import Dict, Any, List
|
||||
from types import SimpleNamespace
|
||||
import json
|
||||
|
||||
from middleware.auth_middleware import get_current_user
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
from services.blog_writer.research.exa_provider import ExaResearchProvider
|
||||
from services.llm_providers.main_text_generation import llm_text_gen
|
||||
from services.podcast_bible_service import PodcastBibleService
|
||||
from loguru import logger
|
||||
from ..models import (
|
||||
PodcastExaResearchRequest,
|
||||
PodcastExaResearchResponse,
|
||||
PodcastExaSource,
|
||||
PodcastExaConfig,
|
||||
PodcastResearchInsight,
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/research/exa", response_model=PodcastExaResearchResponse)
|
||||
async def podcast_research_exa(
|
||||
request: PodcastExaResearchRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Run podcast research via Exa and then use LLM to extract deep insights.
|
||||
Uses Podcast Bible and Analysis context for hyper-personalization.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
queries = [q.strip() for q in request.queries if q and q.strip()]
|
||||
if not queries:
|
||||
raise HTTPException(status_code=400, detail="At least one query is required for research.")
|
||||
|
||||
exa_cfg = request.exa_config or PodcastExaConfig()
|
||||
cfg = SimpleNamespace(
|
||||
exa_search_type=exa_cfg.exa_search_type or "auto",
|
||||
exa_category=exa_cfg.exa_category,
|
||||
exa_include_domains=exa_cfg.exa_include_domains or [],
|
||||
exa_exclude_domains=exa_cfg.exa_exclude_domains or [],
|
||||
max_sources=exa_cfg.max_sources or 8,
|
||||
source_types=[],
|
||||
)
|
||||
|
||||
provider = ExaResearchProvider()
|
||||
|
||||
# --- Context Building ---
|
||||
bible_service = PodcastBibleService()
|
||||
bible_context = ""
|
||||
if request.bible:
|
||||
try:
|
||||
from models.podcast_bible_models import PodcastBible
|
||||
bible_data = PodcastBible(**request.bible)
|
||||
bible_context = bible_service.serialize_bible(bible_data)
|
||||
except Exception as exc:
|
||||
logger.warning(f"[Podcast Research] Failed to serialize bible: {exc}")
|
||||
|
||||
analysis_context = ""
|
||||
if request.analysis:
|
||||
analysis_context = f"""
|
||||
PODCAST ANALYSIS CONTEXT:
|
||||
Audience: {request.analysis.get('audience', 'General')}
|
||||
Content Type: {request.analysis.get('content_type', 'Informative')}
|
||||
Top Keywords: {', '.join(request.analysis.get('top_keywords', []))}
|
||||
"""
|
||||
|
||||
# Exa search params
|
||||
industry = request.bible.get("brand", {}).get("industry", "") if request.bible else ""
|
||||
target_audience = ""
|
||||
if request.bible:
|
||||
audience_dna = request.bible.get("audience", {})
|
||||
if audience_dna:
|
||||
interests = ", ".join(audience_dna.get("interests", []))
|
||||
target_audience = f"Expertise: {audience_dna.get('expertise_level', '')}. Interests: {interests}."
|
||||
|
||||
try:
|
||||
# 1. RUN EXA SEARCH
|
||||
result = await provider.search(
|
||||
prompt=request.topic,
|
||||
topic=request.topic,
|
||||
industry=industry,
|
||||
target_audience=target_audience,
|
||||
config=cfg,
|
||||
user_id=user_id,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(f"[Podcast Exa Research] Search failed for user {user_id}: {exc}")
|
||||
raise HTTPException(status_code=500, detail=f"Exa research failed: {exc}")
|
||||
|
||||
# 2. EXTRACT INSIGHTS VIA LLM
|
||||
raw_content = result.get("content", "")
|
||||
sources = result.get("sources", [])
|
||||
|
||||
summary = ""
|
||||
key_insights = []
|
||||
expert_quotes = []
|
||||
listener_cta = []
|
||||
mapped_angles = []
|
||||
|
||||
if raw_content and sources:
|
||||
logger.info(f"[Podcast Research] Extracting insights from {len(sources)} sources for user {user_id}")
|
||||
|
||||
prompt = f"""
|
||||
You are an expert research analyst for a high-end podcast production team.
|
||||
Your task is to analyze the following research data and extract deep, actionable insights for a podcast episode.
|
||||
|
||||
PODCAST CONTEXT:
|
||||
Topic: {request.topic}
|
||||
{bible_context}
|
||||
{analysis_context}
|
||||
|
||||
RESEARCH DATA (from {len(sources)} sources):
|
||||
{raw_content}
|
||||
|
||||
TASK:
|
||||
1. Provide a comprehensive summary (2-3 paragraphs) of the most important findings. Use Markdown for formatting (bolding, lists).
|
||||
2. Extract 3-5 "Key Insights". Each insight should have a title and a detailed explanation.
|
||||
3. For each insight, identify which source indices (e.g. 1, 2) it was derived from.
|
||||
4. Extract notable "Expert Quotes" - direct quotes from industry leaders, researchers, or authoritative voices found in the sources.
|
||||
5. Suggest 2-4 "Listener CTA" (call-to-action) ideas that the podcast host can use to engage the audience.
|
||||
6. Identify 3-5 "Mapped Angles" - unique content angles with rationale for why they matter for this topic.
|
||||
|
||||
NOTE: The research data includes "Key Highlights", "Summaries", and "Excerpts" from various sources.
|
||||
Pay special attention to the "Key Highlights" sections as they contain the most relevant information extracted by the neural search engine.
|
||||
|
||||
Return JSON structure:
|
||||
{{
|
||||
"summary": "Detailed markdown summary...",
|
||||
"key_insights": [
|
||||
{{
|
||||
"title": "Insight Title",
|
||||
"content": "Detailed markdown content...",
|
||||
"source_indices": [1, 2]
|
||||
}}
|
||||
],
|
||||
"expert_quotes": [
|
||||
{{
|
||||
"quote": "Exact quote from source...",
|
||||
"source_index": 1
|
||||
}}
|
||||
],
|
||||
"listener_cta": [
|
||||
"Call-to-action suggestion 1",
|
||||
"Call-to-action suggestion 2"
|
||||
],
|
||||
"mapped_angles": [
|
||||
{{
|
||||
"title": "Angle Title",
|
||||
"why": "Why this angle matters for the audience...",
|
||||
"mapped_fact_ids": ["fact_1", "fact_2"]
|
||||
}}
|
||||
]
|
||||
}}
|
||||
|
||||
Requirements:
|
||||
- Ensure insights are deep, not just superficial facts. Look for trends, expert opinions, and specific data points.
|
||||
- Expert quotes should be exact or near-exact quotes from the sources, with attribution.
|
||||
- Listener CTAs should be practical and engaging (e.g., "Share your experience with X on social media").
|
||||
- Mapped angles should be unique perspectives that make the episode stand out.
|
||||
- Tone should be professional, insightful, and ready for a podcast host to discuss.
|
||||
- Avoid generic filler.
|
||||
"""
|
||||
try:
|
||||
llm_response = llm_text_gen(
|
||||
prompt=prompt,
|
||||
user_id=user_id,
|
||||
json_struct=None,
|
||||
preferred_provider="huggingface",
|
||||
flow_type="premium_tool",
|
||||
)
|
||||
|
||||
# Normalize response
|
||||
if isinstance(llm_response, str):
|
||||
data = json.loads(llm_response)
|
||||
else:
|
||||
data = llm_response
|
||||
|
||||
summary = data.get("summary", "")
|
||||
key_insights = [PodcastResearchInsight(**insight) for insight in data.get("key_insights", [])]
|
||||
expert_quotes = data.get("expert_quotes", [])
|
||||
listener_cta = data.get("listener_cta", [])
|
||||
mapped_angles = data.get("mapped_angles", [])
|
||||
except Exception as exc:
|
||||
logger.error(f"[Podcast Research] LLM Insight extraction failed: {exc}")
|
||||
# Fallback to a basic summary if LLM fails
|
||||
summary = f"Research completed for '{request.topic}'. Found {len(sources)} sources."
|
||||
|
||||
# Fallback: if summary is still empty (e.g. LLM returned empty string), use raw content first paragraph or basic text
|
||||
if not summary:
|
||||
if raw_content:
|
||||
summary = raw_content[:2000] # Use first 2000 chars of raw content as summary
|
||||
else:
|
||||
summary = f"Research completed for '{request.topic}'. Found {len(sources)} sources."
|
||||
|
||||
# 3. TRACK USAGE
|
||||
try:
|
||||
cost_total = 0.0
|
||||
if isinstance(result, dict):
|
||||
cost_total = result.get("cost", {}).get("total", 0.005) if result.get("cost") else 0.005
|
||||
provider.track_exa_usage(user_id, cost_total)
|
||||
except Exception as track_err:
|
||||
logger.warning(f"[Podcast Exa Research] Failed to track usage: {track_err}")
|
||||
|
||||
sources_payload = []
|
||||
for src in sources:
|
||||
try:
|
||||
sources_payload.append(PodcastExaSource(**src))
|
||||
except Exception:
|
||||
sources_payload.append(PodcastExaSource(**{
|
||||
"title": src.get("title", ""),
|
||||
"url": src.get("url", ""),
|
||||
"excerpt": src.get("excerpt", ""),
|
||||
"published_at": src.get("published_at"),
|
||||
"highlights": src.get("highlights"),
|
||||
"summary": src.get("summary"),
|
||||
"source_type": src.get("source_type"),
|
||||
"index": src.get("index"),
|
||||
"image": src.get("image"),
|
||||
"author": src.get("author"),
|
||||
}))
|
||||
|
||||
return PodcastExaResearchResponse(
|
||||
sources=sources_payload,
|
||||
search_queries=result.get("search_queries", queries) if isinstance(result, dict) else queries,
|
||||
summary=summary,
|
||||
key_insights=key_insights,
|
||||
expert_quotes=expert_quotes,
|
||||
listener_cta=listener_cta,
|
||||
mapped_angles=mapped_angles,
|
||||
cost=result.get("cost") if isinstance(result, dict) else None,
|
||||
search_type=result.get("search_type") if isinstance(result, dict) else None,
|
||||
provider=result.get("provider", "exa") if isinstance(result, dict) else "exa",
|
||||
content=raw_content,
|
||||
)
|
||||
|
||||
@@ -1,183 +0,0 @@
|
||||
"""
|
||||
Podcast Script Handlers
|
||||
|
||||
Script generation endpoint.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import Dict, Any
|
||||
import json
|
||||
|
||||
from middleware.auth_middleware import get_current_user
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
from services.llm_providers.main_text_generation import llm_text_gen
|
||||
from services.podcast_bible_service import PodcastBibleService
|
||||
from models.podcast_bible_models import PodcastBible
|
||||
from loguru import logger
|
||||
from ..models import (
|
||||
PodcastScriptRequest,
|
||||
PodcastScriptResponse,
|
||||
PodcastScene,
|
||||
PodcastSceneLine,
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/script", response_model=PodcastScriptResponse)
|
||||
async def generate_podcast_script(
|
||||
request: PodcastScriptRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Generate a podcast script outline (scenes + lines) using podcast-oriented prompting.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Build comprehensive research context for higher-quality scripts
|
||||
research_context = ""
|
||||
if request.research:
|
||||
try:
|
||||
key_insights = request.research.get("keyword_analysis", {}).get("key_insights") or []
|
||||
fact_cards = request.research.get("factCards", []) or []
|
||||
mapped_angles = request.research.get("mappedAngles", []) or []
|
||||
sources = request.research.get("sources", []) or []
|
||||
|
||||
top_facts = [f.get("quote", "") for f in fact_cards[:5] if f.get("quote")]
|
||||
angles_summary = [
|
||||
f"{a.get('title', '')}: {a.get('why', '')}" for a in mapped_angles[:3] if a.get("title") or a.get("why")
|
||||
]
|
||||
top_sources = [s.get("url") for s in sources[:3] if s.get("url")]
|
||||
|
||||
research_parts = []
|
||||
if key_insights:
|
||||
research_parts.append(f"Key Insights: {', '.join(key_insights[:5])}")
|
||||
if top_facts:
|
||||
research_parts.append(f"Key Facts: {', '.join(top_facts)}")
|
||||
if angles_summary:
|
||||
research_parts.append(f"Research Angles: {' | '.join(angles_summary)}")
|
||||
if top_sources:
|
||||
research_parts.append(f"Top Sources: {', '.join(top_sources)}")
|
||||
|
||||
research_context = "\n".join(research_parts)
|
||||
except Exception as exc:
|
||||
logger.warning(f"Failed to parse research context: {exc}")
|
||||
research_context = ""
|
||||
|
||||
# Extract Podcast Bible context for hyper-personalization
|
||||
bible_context = ""
|
||||
if request.bible:
|
||||
try:
|
||||
bible_service = PodcastBibleService()
|
||||
bible_obj = PodcastBible(**request.bible)
|
||||
bible_context = bible_service.serialize_bible(bible_obj)
|
||||
except Exception as exc:
|
||||
logger.warning(f"Failed to serialize podcast bible: {exc}")
|
||||
|
||||
# Extract Analysis and Outline context for grounding
|
||||
analysis_context = ""
|
||||
if request.analysis:
|
||||
analysis_context = f"""
|
||||
TARGET AUDIENCE: {request.analysis.get('audience', 'General')}
|
||||
CONTENT TYPE: {request.analysis.get('contentType', 'Conversational')}
|
||||
TOP KEYWORDS: {', '.join(request.analysis.get('topKeywords', []))}
|
||||
"""
|
||||
|
||||
outline_context = ""
|
||||
if request.outline:
|
||||
outline_context = f"""
|
||||
REFINED EPISODE OUTLINE (Follow this structure closely):
|
||||
Title: {request.outline.get('title', 'N/A')}
|
||||
Segments: {' | '.join(request.outline.get('segments', []))}
|
||||
"""
|
||||
|
||||
prompt = f"""You are an expert podcast script planner. Create natural, conversational podcast scenes.
|
||||
|
||||
{f"PODCAST BIBLE (Hyper-Personalization Context):\n{bible_context}\n" if bible_context else ""}
|
||||
{f"ANALYSIS CONTEXT:\n{analysis_context}\n" if analysis_context else ""}
|
||||
{f"REFINED OUTLINE:\n{outline_context}\n" if outline_context else ""}
|
||||
|
||||
Podcast Idea: "{request.idea}"
|
||||
Duration: ~{request.duration_minutes} minutes
|
||||
Speakers: {request.speakers} (Host + optional Guest)
|
||||
|
||||
{f"RESEARCH CONTEXT:\n{research_context}\n" if research_context else ""}
|
||||
|
||||
Return JSON with:
|
||||
- scenes: array of scenes. Each scene has:
|
||||
- id: string
|
||||
- title: short scene title (<= 60 chars)
|
||||
- duration: duration in seconds (evenly split across total duration)
|
||||
- emotion: string (one of: "neutral", "happy", "excited", "serious", "curious", "confident")
|
||||
- lines: array of {{"speaker": "...", "text": "...", "emphasis": boolean}}
|
||||
* Write natural, conversational dialogue
|
||||
* Each line can be a sentence or a few sentences that flow together
|
||||
* Use plain text only - no markdown formatting (no asterisks, underscores, etc.)
|
||||
* Mark "emphasis": true for key statistics or important points
|
||||
|
||||
Guidelines:
|
||||
- Write for spoken delivery: conversational, natural, with contractions.
|
||||
- Follow the interaction tone specified in the Bible.
|
||||
- Ensure the Host persona matches the background and personality traits from the Bible.
|
||||
- Structure the intro and outro scenes according to the Bible's "Intro Format" and "Outro Format".
|
||||
- Adhere to any constraints mentioned in the Bible.
|
||||
- Use insights from the Research Context to ground the conversation in facts.
|
||||
- IMPORTANT: Follow the REFINED OUTLINE segments as the primary structure for the episode.
|
||||
"""
|
||||
|
||||
try:
|
||||
raw = llm_text_gen(
|
||||
prompt=prompt,
|
||||
user_id=user_id,
|
||||
json_struct=None,
|
||||
preferred_provider="huggingface",
|
||||
flow_type="premium_tool",
|
||||
)
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=f"Script generation failed: {exc}")
|
||||
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
except json.JSONDecodeError:
|
||||
raise HTTPException(status_code=500, detail="LLM returned non-JSON output")
|
||||
elif isinstance(raw, dict):
|
||||
data = raw
|
||||
else:
|
||||
raise HTTPException(status_code=500, detail="Unexpected LLM response format")
|
||||
|
||||
scenes_data = data.get("scenes") or []
|
||||
if not isinstance(scenes_data, list):
|
||||
raise HTTPException(status_code=500, detail="LLM response missing scenes array")
|
||||
|
||||
valid_emotions = {"neutral", "happy", "excited", "serious", "curious", "confident"}
|
||||
|
||||
# Normalize scenes
|
||||
scenes: list[PodcastScene] = []
|
||||
for idx, scene in enumerate(scenes_data):
|
||||
title = scene.get("title") or f"Scene {idx + 1}"
|
||||
duration = int(scene.get("duration") or max(30, (request.duration_minutes * 60) // max(1, len(scenes_data))))
|
||||
emotion = scene.get("emotion") or "neutral"
|
||||
if emotion not in valid_emotions:
|
||||
emotion = "neutral"
|
||||
lines_raw = scene.get("lines") or []
|
||||
lines: list[PodcastSceneLine] = []
|
||||
for line in lines_raw:
|
||||
speaker = line.get("speaker") or ("Host" if len(lines) % request.speakers == 0 else "Guest")
|
||||
text = line.get("text") or ""
|
||||
emphasis = line.get("emphasis", False)
|
||||
if text:
|
||||
lines.append(PodcastSceneLine(speaker=speaker, text=text, emphasis=emphasis))
|
||||
scenes.append(
|
||||
PodcastScene(
|
||||
id=scene.get("id") or f"scene-{idx + 1}",
|
||||
title=title,
|
||||
duration=duration,
|
||||
lines=lines,
|
||||
approved=False,
|
||||
emotion=emotion,
|
||||
)
|
||||
)
|
||||
|
||||
return PodcastScriptResponse(scenes=scenes)
|
||||
|
||||
@@ -1,209 +0,0 @@
|
||||
export type Knobs = {
|
||||
voice_emotion: string;
|
||||
voice_speed: number;
|
||||
resolution: string;
|
||||
scene_length_target: number;
|
||||
sample_rate: number;
|
||||
bitrate: string;
|
||||
};
|
||||
|
||||
export type Query = {
|
||||
id: string;
|
||||
query: string;
|
||||
rationale: string;
|
||||
needsRecentStats: boolean;
|
||||
};
|
||||
|
||||
export type Fact = {
|
||||
id: string;
|
||||
quote: string;
|
||||
url: string;
|
||||
date: string;
|
||||
confidence: number;
|
||||
image?: string;
|
||||
author?: string;
|
||||
highlights?: string[];
|
||||
};
|
||||
|
||||
export type ResearchInsight = {
|
||||
title: string;
|
||||
content: string;
|
||||
source_indices: number[];
|
||||
};
|
||||
|
||||
export type Research = {
|
||||
summary: string;
|
||||
keyInsights: ResearchInsight[];
|
||||
factCards: Fact[];
|
||||
mappedAngles: {
|
||||
title: string;
|
||||
why: string;
|
||||
mappedFactIds: string[];
|
||||
}[];
|
||||
searchQueries?: string[];
|
||||
searchType?: string;
|
||||
provider?: string;
|
||||
cost?: number;
|
||||
sourceCount?: number;
|
||||
expertQuotes?: { quote: string; source_index: number }[];
|
||||
listenerCta?: string[];
|
||||
};
|
||||
|
||||
export type Line = {
|
||||
id: string;
|
||||
speaker: string;
|
||||
text: string;
|
||||
usedFactIds?: string[];
|
||||
emphasis?: boolean; // Mark lines that need vocal emphasis
|
||||
};
|
||||
|
||||
export type Scene = {
|
||||
id: string;
|
||||
title: string;
|
||||
duration: number;
|
||||
lines: Line[];
|
||||
approved?: boolean;
|
||||
emotion?: string; // Scene-specific emotion
|
||||
audioUrl?: string; // Generated audio URL for this scene
|
||||
imageUrl?: string; // Generated image URL for this scene (for video generation)
|
||||
};
|
||||
|
||||
export type Script = {
|
||||
scenes: Scene[];
|
||||
};
|
||||
|
||||
export type JobStatus =
|
||||
| "idle"
|
||||
| "previewing"
|
||||
| "queued"
|
||||
| "running"
|
||||
| "completed"
|
||||
| "cancelled"
|
||||
| "failed";
|
||||
|
||||
export type Job = {
|
||||
sceneId: string;
|
||||
title: string;
|
||||
status: JobStatus;
|
||||
progress: number;
|
||||
previewUrl?: string | null;
|
||||
finalUrl?: string | null;
|
||||
videoUrl?: string | null;
|
||||
jobId?: string | null;
|
||||
taskId?: string | null;
|
||||
cost?: number | null;
|
||||
provider?: string | null;
|
||||
voiceId?: string | null;
|
||||
fileSize?: number | null;
|
||||
avatarImageUrl?: string | null;
|
||||
imageUrl?: string | null; // Scene-specific image URL
|
||||
};
|
||||
|
||||
export type PodcastAnalysis = {
|
||||
audience: string;
|
||||
contentType: string;
|
||||
topKeywords: string[];
|
||||
suggestedOutlines: { id: number | string; title: string; segments: string[] }[];
|
||||
suggestedKnobs: Knobs;
|
||||
titleSuggestions: string[];
|
||||
research_queries?: { query: string; rationale: string }[];
|
||||
exaSuggestedConfig?: {
|
||||
exa_search_type?: "auto" | "keyword" | "neural";
|
||||
exa_category?: string;
|
||||
exa_include_domains?: string[];
|
||||
exa_exclude_domains?: string[];
|
||||
max_sources?: number;
|
||||
include_statistics?: boolean;
|
||||
date_range?: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type PodcastEstimate = {
|
||||
ttsCost: number;
|
||||
avatarCost: number;
|
||||
videoCost: number;
|
||||
researchCost: number;
|
||||
total: number;
|
||||
};
|
||||
|
||||
export type HostPersona = {
|
||||
name: string;
|
||||
background: string;
|
||||
expertise_level: string;
|
||||
personality_traits: string[];
|
||||
vocal_style: string;
|
||||
catchphrases: string[];
|
||||
};
|
||||
|
||||
export type AudienceDNA = {
|
||||
expertise_level: string;
|
||||
interests: string[];
|
||||
pain_points: string[];
|
||||
demographics?: string;
|
||||
};
|
||||
|
||||
export type BrandDNA = {
|
||||
industry: string;
|
||||
tone: string;
|
||||
communication_style: string;
|
||||
key_messages: string[];
|
||||
competitor_context?: string;
|
||||
};
|
||||
|
||||
export type PodcastBible = {
|
||||
project_id?: string;
|
||||
host: HostPersona;
|
||||
audience: AudienceDNA;
|
||||
brand: BrandDNA;
|
||||
};
|
||||
|
||||
export type CreateProjectPayload = {
|
||||
ideaOrUrl: string;
|
||||
speakers: number;
|
||||
duration: number;
|
||||
knobs: Knobs;
|
||||
budgetCap: number;
|
||||
files: { voiceFile?: File | null; avatarFile?: File | null };
|
||||
avatarUrl?: string | null;
|
||||
};
|
||||
|
||||
export type CreateProjectResult = {
|
||||
projectId: string;
|
||||
analysis: PodcastAnalysis;
|
||||
estimate: PodcastEstimate;
|
||||
queries: Query[];
|
||||
bible?: PodcastBible;
|
||||
avatar_url?: string | null;
|
||||
avatar_prompt?: string | null;
|
||||
};
|
||||
|
||||
export type RenderJobResult = {
|
||||
audioUrl: string;
|
||||
audioFilename: string;
|
||||
provider: string;
|
||||
model: string;
|
||||
cost: number;
|
||||
voiceId: string;
|
||||
fileSize: number;
|
||||
videoUrl?: string;
|
||||
videoFilename?: string;
|
||||
};
|
||||
|
||||
export interface VideoGenerationSettings {
|
||||
prompt: string;
|
||||
resolution: "480p" | "720p";
|
||||
seed?: number | null;
|
||||
maskImageUrl?: string | null;
|
||||
}
|
||||
|
||||
export type TaskStatus = {
|
||||
task_id: string;
|
||||
status: "pending" | "processing" | "completed" | "failed";
|
||||
progress?: number;
|
||||
message?: string;
|
||||
result?: any;
|
||||
error?: string;
|
||||
created_at?: string;
|
||||
updated_at?: string;
|
||||
};
|
||||
|
||||
@@ -1,425 +0,0 @@
|
||||
import { useState, useEffect, useMemo, useCallback } from "react";
|
||||
import { podcastApi } from "../../../services/podcastApi";
|
||||
import { usePreflightCheck } from "../../../hooks/usePreflightCheck";
|
||||
import { useBudgetTracking } from "../../../hooks/useBudgetTracking";
|
||||
import { CreateProjectPayload, Script } from "../types";
|
||||
import { usePodcastProjectState } from "../../../hooks/usePodcastProjectState";
|
||||
import { sanitizeExaConfig, announceError, getStepLabel } from "./utils";
|
||||
|
||||
type PodcastProjectStateReturn = ReturnType<typeof usePodcastProjectState>;
|
||||
|
||||
interface UsePodcastWorkflowProps {
|
||||
projectState: PodcastProjectStateReturn;
|
||||
onError: (message: string) => void;
|
||||
}
|
||||
|
||||
export const usePodcastWorkflow = ({ projectState, onError }: UsePodcastWorkflowProps) => {
|
||||
const {
|
||||
project,
|
||||
analysis,
|
||||
queries,
|
||||
selectedQueries,
|
||||
research,
|
||||
rawResearch,
|
||||
researchProvider,
|
||||
showScriptEditor,
|
||||
showRenderQueue,
|
||||
currentStep,
|
||||
renderJobs,
|
||||
budgetCap,
|
||||
setProject,
|
||||
setAnalysis,
|
||||
setQueries,
|
||||
setSelectedQueries,
|
||||
setResearch,
|
||||
setRawResearch,
|
||||
setEstimate,
|
||||
setScriptData,
|
||||
setShowScriptEditor,
|
||||
setShowRenderQueue,
|
||||
setKnobs,
|
||||
setResearchProvider,
|
||||
setBudgetCap,
|
||||
updateRenderJob,
|
||||
initializeProject,
|
||||
setBible,
|
||||
} = projectState;
|
||||
|
||||
const [isAnalyzing, setIsAnalyzing] = useState(false);
|
||||
const [isResearching, setIsResearching] = useState(false);
|
||||
const [announcement, setAnnouncement] = useState("");
|
||||
const [showResumeAlert, setShowResumeAlert] = useState(false);
|
||||
const [showPreflightDialog, setShowPreflightDialog] = useState(false);
|
||||
const [preflightResponse, setPreflightResponse] = useState<any>(null);
|
||||
const [preflightOperationName, setPreflightOperationName] = useState<string>("");
|
||||
|
||||
const budgetTracking = useBudgetTracking(budgetCap || 50);
|
||||
const preflightCheck = usePreflightCheck({
|
||||
onBlocked: (response) => {
|
||||
setPreflightResponse(response);
|
||||
setShowPreflightDialog(true);
|
||||
},
|
||||
});
|
||||
|
||||
// Update budget cap when project state changes
|
||||
useEffect(() => {
|
||||
if (budgetCap) {
|
||||
budgetTracking.setBudgetCap(budgetCap);
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [budgetCap]);
|
||||
|
||||
// Check if we have a saved project on mount
|
||||
useEffect(() => {
|
||||
if (project && currentStep && currentStep !== "create") {
|
||||
setShowResumeAlert(true);
|
||||
setTimeout(() => setShowResumeAlert(false), 5000);
|
||||
}
|
||||
}, [project, currentStep]);
|
||||
|
||||
useEffect(() => {
|
||||
if (announcement) {
|
||||
const t = setTimeout(() => setAnnouncement(""), 4000);
|
||||
return () => clearTimeout(t);
|
||||
}
|
||||
return undefined;
|
||||
}, [announcement]);
|
||||
|
||||
const handleCreate = useCallback(async (payload: CreateProjectPayload, feedback?: string) => {
|
||||
if (isAnalyzing) return;
|
||||
setResearch(null);
|
||||
setRawResearch(null);
|
||||
setScriptData(null);
|
||||
setShowScriptEditor(false);
|
||||
setShowRenderQueue(false);
|
||||
try {
|
||||
setIsAnalyzing(true);
|
||||
|
||||
// Use existing avatar URL if provided (e.g. brand avatar), or upload new file
|
||||
let avatarUrl: string | null = payload.avatarUrl || null;
|
||||
if (payload.files.avatarFile) {
|
||||
try {
|
||||
setAnnouncement("Uploading presenter avatar...");
|
||||
const uploadResponse = await podcastApi.uploadAvatar(payload.files.avatarFile);
|
||||
avatarUrl = uploadResponse.avatar_url;
|
||||
} catch (error) {
|
||||
console.error('Avatar upload failed:', error);
|
||||
// Continue without avatar - will generate one later
|
||||
}
|
||||
}
|
||||
|
||||
// NEW FLOW: Create project first to generate/get the Podcast Bible
|
||||
// This allows the analysis to be personalized using the Bible context
|
||||
const projectId = project?.id || `podcast_${Date.now()}_${Math.floor(Math.random() * 1000)}`;
|
||||
setAnnouncement("Initializing project and brand context...");
|
||||
const dbProject = project ? null : await initializeProject(payload, projectId, avatarUrl);
|
||||
const bible = dbProject?.bible || projectState.bible;
|
||||
|
||||
setAnnouncement(feedback ? "Regenerating analysis using your feedback..." : "Analyzing your idea — AI suggestions incoming");
|
||||
const result = await podcastApi.createProject(payload, bible, feedback);
|
||||
|
||||
if (result.bible) {
|
||||
setBible(result.bible);
|
||||
} else if (dbProject?.bible) {
|
||||
setBible(dbProject.bible);
|
||||
}
|
||||
|
||||
// Update the project in database with the analysis results
|
||||
try {
|
||||
await podcastApi.updateProject(projectId, {
|
||||
analysis: result.analysis,
|
||||
estimate: result.estimate,
|
||||
queries: result.queries,
|
||||
selected_queries: result.queries.map(q => q.id),
|
||||
avatar_url: result.avatar_url,
|
||||
avatar_prompt: result.avatar_prompt,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Failed to update project with analysis results:', error);
|
||||
}
|
||||
|
||||
setProject({
|
||||
id: projectId,
|
||||
idea: payload.ideaOrUrl,
|
||||
duration: payload.duration,
|
||||
speakers: payload.speakers,
|
||||
avatarUrl: result.avatar_url || avatarUrl,
|
||||
avatarPrompt: result.avatar_prompt || null,
|
||||
avatarPersonaId: null,
|
||||
});
|
||||
|
||||
setAnalysis(result.analysis);
|
||||
setEstimate(result.estimate);
|
||||
setQueries(result.queries);
|
||||
setSelectedQueries(new Set(result.queries.map((q) => q.id)));
|
||||
setKnobs(payload.knobs);
|
||||
setBudgetCap(payload.budgetCap);
|
||||
|
||||
// Generate presenters AFTER analysis completes (to use analysis insights)
|
||||
// This happens only if no avatar was uploaded
|
||||
if (!avatarUrl && payload.speakers > 0 && result.analysis) {
|
||||
try {
|
||||
setAnnouncement("Generating presenter avatars using AI insights...");
|
||||
const presentersResponse = await podcastApi.generatePresenters(
|
||||
payload.speakers,
|
||||
result.projectId,
|
||||
result.analysis.audience,
|
||||
result.analysis.contentType,
|
||||
result.analysis.topKeywords
|
||||
);
|
||||
if (presentersResponse.avatars && presentersResponse.avatars.length > 0) {
|
||||
// Store the first presenter avatar URL and prompt
|
||||
const firstAvatar = presentersResponse.avatars[0];
|
||||
const prompt = firstAvatar.prompt || null;
|
||||
setProject({
|
||||
id: result.projectId,
|
||||
idea: payload.ideaOrUrl,
|
||||
duration: payload.duration,
|
||||
speakers: payload.speakers,
|
||||
avatarUrl: firstAvatar.avatar_url,
|
||||
avatarPrompt: prompt,
|
||||
avatarPersonaId: firstAvatar.persona_id || presentersResponse.persona_id || null,
|
||||
});
|
||||
setAnnouncement("Analysis complete - Presenter avatars generated");
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Presenter generation failed:', error);
|
||||
setAnnouncement("Analysis complete - Avatar generation will happen later");
|
||||
// Continue without presenters - can generate later
|
||||
}
|
||||
} else {
|
||||
setAnnouncement("Analysis complete");
|
||||
}
|
||||
} catch (error: any) {
|
||||
if (error?.response?.status === 429 || error?.response?.data?.detail) {
|
||||
const errorDetail = error.response.data.detail;
|
||||
if (typeof errorDetail === 'object' && errorDetail.error && errorDetail.error.includes('limit')) {
|
||||
const usageInfo = errorDetail.usage_info || {};
|
||||
const blockedResponse = {
|
||||
can_proceed: false,
|
||||
estimated_cost: 0,
|
||||
operations: [{
|
||||
provider: errorDetail.provider || 'huggingface',
|
||||
operation_type: 'ai_text_generation',
|
||||
cost: 0,
|
||||
allowed: false,
|
||||
limit_info: usageInfo.limit_info || null,
|
||||
message: errorDetail.message || errorDetail.error || 'Subscription limit exceeded',
|
||||
}],
|
||||
total_cost: 0,
|
||||
usage_summary: usageInfo.usage_summary || null,
|
||||
cached: false,
|
||||
};
|
||||
setPreflightResponse(blockedResponse);
|
||||
setPreflightOperationName('Podcast Analysis');
|
||||
setShowPreflightDialog(true);
|
||||
setAnnouncement("Subscription limit reached. Please upgrade to continue.");
|
||||
} else {
|
||||
const message = typeof errorDetail === 'string' ? errorDetail : errorDetail.message || errorDetail.error || 'Request limit exceeded';
|
||||
announceError(setAnnouncement, new Error(message));
|
||||
}
|
||||
} else {
|
||||
announceError(setAnnouncement, error);
|
||||
}
|
||||
} finally {
|
||||
setIsAnalyzing(false);
|
||||
}
|
||||
}, [isAnalyzing, setResearch, setRawResearch, setScriptData, setShowScriptEditor, setShowRenderQueue, initializeProject, setProject, setAnalysis, setEstimate, setQueries, setSelectedQueries, setKnobs, setBudgetCap, setBible]);
|
||||
|
||||
const handleRunResearch = useCallback(async () => {
|
||||
if (isResearching) return;
|
||||
if (!project) {
|
||||
setAnnouncement("Create a project first.");
|
||||
return;
|
||||
}
|
||||
if (selectedQueries.size === 0) {
|
||||
setAnnouncement("Select at least one query to research.");
|
||||
return;
|
||||
}
|
||||
|
||||
setPreflightOperationName("Research");
|
||||
const approvedQueries = queries.filter((q) => selectedQueries.has(q.id));
|
||||
const preflightResult = await preflightCheck.check({
|
||||
provider: researchProvider === "exa" ? "exa" : "gemini",
|
||||
operation_type: researchProvider === "exa" ? "exa_neural_search" : "google_grounding",
|
||||
tokens_requested: researchProvider === "exa" ? 0 : 1200,
|
||||
actual_provider_name: researchProvider || "exa",
|
||||
});
|
||||
|
||||
if (!preflightResult.can_proceed) {
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
setIsResearching(true);
|
||||
setAnnouncement(`Starting ${researchProvider === "exa" ? "deep" : "standard"} research — this may take a moment...`);
|
||||
setResearch(null);
|
||||
setRawResearch(null);
|
||||
setScriptData(null);
|
||||
setShowScriptEditor(false);
|
||||
setShowRenderQueue(false);
|
||||
|
||||
try {
|
||||
const { research: mapped, raw } = await podcastApi.runResearch({
|
||||
projectId: project.id,
|
||||
topic: project.idea,
|
||||
approvedQueries,
|
||||
provider: researchProvider,
|
||||
exaConfig: sanitizeExaConfig(analysis?.exaSuggestedConfig),
|
||||
bible: projectState.bible,
|
||||
analysis: analysis,
|
||||
onProgress: (message) => {
|
||||
setAnnouncement(message);
|
||||
},
|
||||
});
|
||||
setResearch(mapped);
|
||||
setRawResearch(raw);
|
||||
setAnnouncement("Research complete — review fact cards below");
|
||||
} catch (researchError) {
|
||||
const errorMessage = researchError instanceof Error
|
||||
? researchError.message
|
||||
: "Research failed. Please try again or switch to Standard Research.";
|
||||
|
||||
if (errorMessage.includes("Exa") || errorMessage.includes("exa")) {
|
||||
setAnnouncement(`Deep research failed: ${errorMessage}. Try Standard Research instead.`);
|
||||
} else if (errorMessage.includes("timeout")) {
|
||||
setAnnouncement("Research timed out. Please try again with fewer queries.");
|
||||
} else {
|
||||
setAnnouncement(`Research failed: ${errorMessage}`);
|
||||
}
|
||||
|
||||
console.error("Research error:", researchError);
|
||||
throw researchError;
|
||||
}
|
||||
} catch (error) {
|
||||
announceError(setAnnouncement, error);
|
||||
} finally {
|
||||
setIsResearching(false);
|
||||
}
|
||||
}, [isResearching, project, selectedQueries, queries, researchProvider, preflightCheck, analysis, setResearch, setRawResearch, setScriptData, setShowScriptEditor, setShowRenderQueue, projectState.bible]);
|
||||
|
||||
const handleGenerateScript = useCallback(async () => {
|
||||
if (showScriptEditor) return;
|
||||
if (!project || !research) {
|
||||
setAnnouncement("Project or research missing — cannot generate script");
|
||||
return;
|
||||
}
|
||||
|
||||
setPreflightOperationName("Script Generation");
|
||||
const preflightResult = await preflightCheck.check({
|
||||
provider: "gemini",
|
||||
operation_type: "script_generation",
|
||||
tokens_requested: 2000,
|
||||
actual_provider_name: "gemini",
|
||||
});
|
||||
|
||||
if (!preflightResult.can_proceed) {
|
||||
return;
|
||||
}
|
||||
|
||||
setScriptData(null);
|
||||
setShowRenderQueue(false);
|
||||
setShowScriptEditor(true);
|
||||
|
||||
try {
|
||||
const result = await podcastApi.generateScript({
|
||||
projectId: project.id,
|
||||
idea: project.idea,
|
||||
research: rawResearch,
|
||||
knobs: projectState.knobs,
|
||||
speakers: project.speakers,
|
||||
durationMinutes: project.duration,
|
||||
bible: projectState.bible,
|
||||
outline: analysis?.suggestedOutlines?.[0], // Pass the first (possibly refined) outline
|
||||
analysis: analysis, // Pass full analysis context
|
||||
});
|
||||
|
||||
setScriptData(result);
|
||||
} catch (error) {
|
||||
announceError(setAnnouncement, error);
|
||||
}
|
||||
}, [showScriptEditor, project, research, preflightCheck, setScriptData, setShowRenderQueue, setShowScriptEditor, rawResearch, projectState.knobs, projectState.bible])
|
||||
|
||||
const handleProceedToRendering = useCallback((script: Script) => {
|
||||
setScriptData(script);
|
||||
if (renderJobs.length === 0) {
|
||||
script.scenes.forEach((scene) => {
|
||||
const hasExistingAudio = Boolean(scene.audioUrl);
|
||||
updateRenderJob(scene.id, {
|
||||
sceneId: scene.id,
|
||||
title: scene.title,
|
||||
status: hasExistingAudio ? ("completed" as const) : ("idle" as const),
|
||||
progress: hasExistingAudio ? 100 : 0,
|
||||
previewUrl: null,
|
||||
finalUrl: hasExistingAudio ? scene.audioUrl : null,
|
||||
jobId: null,
|
||||
});
|
||||
});
|
||||
}
|
||||
setShowRenderQueue(true);
|
||||
setShowScriptEditor(false);
|
||||
}, [renderJobs.length, setScriptData, updateRenderJob, setShowRenderQueue, setShowScriptEditor]);
|
||||
|
||||
const toggleQuery = useCallback((id: string) => {
|
||||
if (isResearching) return;
|
||||
const current = selectedQueries;
|
||||
const next = new Set<string>(current);
|
||||
if (next.has(id)) next.delete(id);
|
||||
else next.add(id);
|
||||
setSelectedQueries(next);
|
||||
}, [isResearching, selectedQueries, setSelectedQueries]);
|
||||
|
||||
const activeStep = useMemo(() => {
|
||||
if (showRenderQueue) return 3;
|
||||
if (showScriptEditor) return 2;
|
||||
if (currentStep === 'research' || research) return 1;
|
||||
if (currentStep === 'analysis' || analysis) return 0;
|
||||
return -1;
|
||||
}, [showRenderQueue, showScriptEditor, currentStep, research, analysis]);
|
||||
|
||||
const canGenerateScript = Boolean(project && research && rawResearch);
|
||||
|
||||
const handleRegenerate = useCallback(async (feedback?: string) => {
|
||||
if (!project) return;
|
||||
|
||||
// Prepare the payload from existing project state
|
||||
const payload: CreateProjectPayload = {
|
||||
ideaOrUrl: project.idea,
|
||||
duration: project.duration,
|
||||
speakers: project.speakers,
|
||||
knobs: projectState.knobs,
|
||||
budgetCap: projectState.budgetCap,
|
||||
avatarUrl: project.avatarUrl,
|
||||
files: {} // No new files for regeneration
|
||||
};
|
||||
|
||||
await handleCreate(payload, feedback);
|
||||
}, [project, projectState.knobs, projectState.budgetCap, handleCreate]);
|
||||
|
||||
return {
|
||||
// State
|
||||
isAnalyzing,
|
||||
isResearching,
|
||||
announcement,
|
||||
showResumeAlert,
|
||||
showPreflightDialog,
|
||||
preflightResponse,
|
||||
preflightOperationName,
|
||||
activeStep,
|
||||
canGenerateScript,
|
||||
// Handlers
|
||||
handleCreate,
|
||||
handleRegenerate,
|
||||
handleRunResearch,
|
||||
handleGenerateScript,
|
||||
handleProceedToRendering,
|
||||
toggleQuery,
|
||||
setAnnouncement,
|
||||
setShowResumeAlert,
|
||||
setShowPreflightDialog,
|
||||
setPreflightResponse,
|
||||
setResearchProvider,
|
||||
getStepLabel,
|
||||
};
|
||||
};
|
||||
|
||||
@@ -1,184 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Migration script to add missing columns to usage_summaries table.
|
||||
Run this once to fix the database schema.
|
||||
|
||||
Usage:
|
||||
python add_missing_columns.py
|
||||
"""
|
||||
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
def get_db_path():
|
||||
"""Find the database path."""
|
||||
possible_paths = [
|
||||
Path(__file__).parent / "backend" / "alwrity.db",
|
||||
Path(__file__).parent.parent / "backend" / "alwrity.db",
|
||||
Path("C:/Users/diksha rawat/Desktop/ALwrity_github/windsurf/ALwrity/backend/alwrity.db"),
|
||||
]
|
||||
|
||||
for db_path in possible_paths:
|
||||
if db_path.exists():
|
||||
print(f"Using database: {db_path}")
|
||||
return db_path
|
||||
|
||||
backend_dir = Path(__file__).parent / "backend"
|
||||
if backend_dir.exists():
|
||||
db_files = list(backend_dir.glob("*.db"))
|
||||
if db_files:
|
||||
print(f"Found database: {db_files[0]}")
|
||||
return db_files[0]
|
||||
|
||||
raise FileNotFoundError(f"Database not found. Searched: {possible_paths}")
|
||||
|
||||
def create_usage_summaries_table(cursor):
|
||||
"""Create the usage_summaries table if it doesn't exist."""
|
||||
cursor.execute("""
|
||||
CREATE TABLE IF NOT EXISTS usage_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
user_id VARCHAR(100) NOT NULL,
|
||||
billing_period VARCHAR(20) NOT NULL,
|
||||
|
||||
-- API Call Counts
|
||||
gemini_calls INTEGER DEFAULT 0,
|
||||
openai_calls INTEGER DEFAULT 0,
|
||||
anthropic_calls INTEGER DEFAULT 0,
|
||||
mistral_calls INTEGER DEFAULT 0,
|
||||
wavespeed_calls INTEGER DEFAULT 0,
|
||||
tavily_calls INTEGER DEFAULT 0,
|
||||
serper_calls INTEGER DEFAULT 0,
|
||||
metaphor_calls INTEGER DEFAULT 0,
|
||||
firecrawl_calls INTEGER DEFAULT 0,
|
||||
stability_calls INTEGER DEFAULT 0,
|
||||
exa_calls INTEGER DEFAULT 0,
|
||||
video_calls INTEGER DEFAULT 0,
|
||||
image_edit_calls INTEGER DEFAULT 0,
|
||||
audio_calls INTEGER DEFAULT 0,
|
||||
|
||||
-- Token Usage
|
||||
gemini_tokens INTEGER DEFAULT 0,
|
||||
openai_tokens INTEGER DEFAULT 0,
|
||||
anthropic_tokens INTEGER DEFAULT 0,
|
||||
mistral_tokens INTEGER DEFAULT 0,
|
||||
wavespeed_tokens INTEGER DEFAULT 0,
|
||||
|
||||
-- Cost Tracking
|
||||
gemini_cost REAL DEFAULT 0.0,
|
||||
openai_cost REAL DEFAULT 0.0,
|
||||
anthropic_cost REAL DEFAULT 0.0,
|
||||
mistral_cost REAL DEFAULT 0.0,
|
||||
wavespeed_cost REAL DEFAULT 0.0,
|
||||
tavily_cost REAL DEFAULT 0.0,
|
||||
serper_cost REAL DEFAULT 0.0,
|
||||
metaphor_cost REAL DEFAULT 0.0,
|
||||
firecrawl_cost REAL DEFAULT 0.0,
|
||||
stability_cost REAL DEFAULT 0.0,
|
||||
exa_cost REAL DEFAULT 0.0,
|
||||
video_cost REAL DEFAULT 0.0,
|
||||
image_edit_cost REAL DEFAULT 0.0,
|
||||
audio_cost REAL DEFAULT 0.0,
|
||||
|
||||
-- Totals
|
||||
total_calls INTEGER DEFAULT 0,
|
||||
total_tokens INTEGER DEFAULT 0,
|
||||
total_cost REAL DEFAULT 0.0,
|
||||
|
||||
-- Performance Metrics
|
||||
avg_response_time REAL DEFAULT 0.0,
|
||||
error_rate REAL DEFAULT 0.0,
|
||||
usage_status VARCHAR(20) DEFAULT 'active',
|
||||
warnings_sent INTEGER DEFAULT 0,
|
||||
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
UNIQUE(user_id, billing_period)
|
||||
)
|
||||
""")
|
||||
print("Created usage_summaries table")
|
||||
|
||||
def add_missing_columns():
|
||||
db_path = get_db_path()
|
||||
print(f"Using database: {db_path}")
|
||||
|
||||
conn = sqlite3.connect(db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Check what tables exist
|
||||
cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||
tables = [row[0] for row in cursor.fetchall()]
|
||||
print(f"Tables in database: {tables}")
|
||||
|
||||
# Check if usage_summaries exists
|
||||
if "usage_summaries" not in tables:
|
||||
print("usage_summaries table doesn't exist. Creating it...")
|
||||
create_usage_summaries_table(cursor)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
print("Done! Table created successfully.")
|
||||
return
|
||||
|
||||
# Get existing columns
|
||||
cursor.execute("PRAGMA table_info(usage_summaries)")
|
||||
existing_columns = {row[1] for row in cursor.fetchall()}
|
||||
print(f"Existing columns in usage_summaries: {len(existing_columns)}")
|
||||
|
||||
# Columns to add (name, type, default)
|
||||
columns_to_add = [
|
||||
# Call counts
|
||||
("wavespeed_calls", "INTEGER", "0"),
|
||||
("tavily_calls", "INTEGER", "0"),
|
||||
("serper_calls", "INTEGER", "0"),
|
||||
("metaphor_calls", "INTEGER", "0"),
|
||||
("firecrawl_calls", "INTEGER", "0"),
|
||||
("stability_calls", "INTEGER", "0"),
|
||||
("exa_calls", "INTEGER", "0"),
|
||||
("video_calls", "INTEGER", "0"),
|
||||
("image_edit_calls", "INTEGER", "0"),
|
||||
("audio_calls", "INTEGER", "0"),
|
||||
# Token usage
|
||||
("wavespeed_tokens", "INTEGER", "0"),
|
||||
# Cost tracking
|
||||
("wavespeed_cost", "REAL", "0.0"),
|
||||
("tavily_cost", "REAL", "0.0"),
|
||||
("serper_cost", "REAL", "0.0"),
|
||||
("metaphor_cost", "REAL", "0.0"),
|
||||
("firecrawl_cost", "REAL", "0.0"),
|
||||
("stability_cost", "REAL", "0.0"),
|
||||
("exa_cost", "REAL", "0.0"),
|
||||
("video_cost", "REAL", "0.0"),
|
||||
("image_edit_cost", "REAL", "0.0"),
|
||||
("audio_cost", "REAL", "0.0"),
|
||||
]
|
||||
|
||||
added = []
|
||||
skipped = []
|
||||
|
||||
for col_name, col_type, default in columns_to_add:
|
||||
if col_name in existing_columns:
|
||||
skipped.append(col_name)
|
||||
continue
|
||||
|
||||
try:
|
||||
sql = f"ALTER TABLE usage_summaries ADD COLUMN {col_name} {col_type} DEFAULT {default}"
|
||||
cursor.execute(sql)
|
||||
added.append(col_name)
|
||||
print(f" Added: {col_name}")
|
||||
except sqlite3.Error as e:
|
||||
print(f" Error adding {col_name}: {e}")
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
print(f"\nSummary:")
|
||||
print(f" Added: {len(added)} columns")
|
||||
print(f" Skipped (already exist): {len(skipped)} columns")
|
||||
|
||||
if added:
|
||||
print(f"\nColumns added: {', '.join(added)}")
|
||||
if skipped:
|
||||
print(f"Already existed: {', '.join(skipped)}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
add_missing_columns()
|
||||
43
artifacts/podcast_billing_sequence_report.json
Normal file
43
artifacts/podcast_billing_sequence_report.json
Normal file
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"preflight": {
|
||||
"success": true,
|
||||
"can_proceed": true,
|
||||
"estimated_cost": 0.3
|
||||
},
|
||||
"operations": {
|
||||
"analysis_title_suggestions": [
|
||||
"AI Agents in 2026",
|
||||
"Ship Faster with AI",
|
||||
"Startup AI Playbook"
|
||||
],
|
||||
"research_provider": "exa",
|
||||
"research_cost": 0.015,
|
||||
"video_task_status": "completed"
|
||||
},
|
||||
"dashboard_deltas": {
|
||||
"total_calls_before": 1,
|
||||
"total_calls_after": 5,
|
||||
"delta_calls": 4,
|
||||
"total_cost_before": 0.09,
|
||||
"total_cost_after": 0.488,
|
||||
"delta_cost": 0.398,
|
||||
"projected_monthly_cost_before": 0.09,
|
||||
"projected_monthly_cost_after": 0.49,
|
||||
"delta_projected_monthly_cost": 0.4
|
||||
},
|
||||
"provider_cost_deltas": {
|
||||
"exa": 0.005,
|
||||
"huggingface": 0.003,
|
||||
"wavespeed": 0.39
|
||||
},
|
||||
"acceptance": {
|
||||
"passed": true,
|
||||
"criteria": {
|
||||
"preflight_success": true,
|
||||
"usage_cost_incremented": true,
|
||||
"usage_call_incremented": true,
|
||||
"projection_incremented": true,
|
||||
"provider_delta_present": true
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,2 +0,0 @@
|
||||
# Use start_alwrity_backend.py for deployment
|
||||
web: python start_alwrity_backend.py --production
|
||||
@@ -3,11 +3,6 @@ ALwrity Utilities Package
|
||||
Modular utilities for ALwrity backend startup and configuration.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
# Check podcast mode early to skip heavy imports
|
||||
_is_podcast = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||
|
||||
from .dependency_manager import DependencyManager
|
||||
from .environment_setup import EnvironmentSetup
|
||||
from .database_setup import DatabaseSetup
|
||||
@@ -16,6 +11,7 @@ from .health_checker import HealthChecker
|
||||
from .rate_limiter import RateLimiter
|
||||
from .frontend_serving import FrontendServing
|
||||
from .router_manager import RouterManager
|
||||
from .onboarding_manager import OnboardingManager
|
||||
from .feature_runtime import (
|
||||
get_active_profiles,
|
||||
get_enabled_groups,
|
||||
@@ -25,42 +21,20 @@ from .feature_runtime import (
|
||||
is_enabled,
|
||||
)
|
||||
|
||||
# Lazy load OnboardingManager - it triggers heavy imports (aiohttp, etc.)
|
||||
if not _is_podcast:
|
||||
from .onboarding_manager import OnboardingManager
|
||||
__all__ = [
|
||||
'DependencyManager',
|
||||
'EnvironmentSetup',
|
||||
'DatabaseSetup',
|
||||
'ProductionOptimizer',
|
||||
'HealthChecker',
|
||||
'RateLimiter',
|
||||
'FrontendServing',
|
||||
'RouterManager',
|
||||
'OnboardingManager',
|
||||
'get_active_profiles',
|
||||
'get_enabled_groups',
|
||||
'get_enabled_optional_services',
|
||||
'get_enabled_routers',
|
||||
'get_enabled_startup_hooks',
|
||||
'is_enabled'
|
||||
]
|
||||
else:
|
||||
OnboardingManager = None
|
||||
__all__ = [
|
||||
'DependencyManager',
|
||||
'EnvironmentSetup',
|
||||
'DatabaseSetup',
|
||||
'ProductionOptimizer',
|
||||
'HealthChecker',
|
||||
'RateLimiter',
|
||||
'FrontendServing',
|
||||
'RouterManager',
|
||||
'OnboardingManager',
|
||||
'get_active_profiles',
|
||||
'get_enabled_groups',
|
||||
'get_enabled_optional_services',
|
||||
'get_enabled_routers',
|
||||
'get_enabled_startup_hooks',
|
||||
'is_enabled'
|
||||
]
|
||||
__all__ = [
|
||||
'DependencyManager',
|
||||
'EnvironmentSetup',
|
||||
'DatabaseSetup',
|
||||
'ProductionOptimizer',
|
||||
'HealthChecker',
|
||||
'RateLimiter',
|
||||
'FrontendServing',
|
||||
'RouterManager',
|
||||
'OnboardingManager',
|
||||
'get_active_profiles',
|
||||
'get_enabled_groups',
|
||||
'get_enabled_optional_services',
|
||||
'get_enabled_routers',
|
||||
'get_enabled_startup_hooks',
|
||||
'is_enabled'
|
||||
]
|
||||
|
||||
@@ -55,28 +55,22 @@ class EnvironmentSetup:
|
||||
print("🔧 Setting up environment variables...")
|
||||
|
||||
# Production environment variables
|
||||
# IMPORTANT: Don't override PORT if already set by Render cloud
|
||||
render_port = os.getenv("PORT")
|
||||
|
||||
if self.production_mode:
|
||||
env_vars = {
|
||||
"HOST": "0.0.0.0",
|
||||
"PORT": "8000",
|
||||
"RELOAD": "false",
|
||||
"LOG_LEVEL": "INFO",
|
||||
"DEBUG": "false"
|
||||
}
|
||||
# Only set PORT if not already provided by cloud (Render sets PORT)
|
||||
if not render_port:
|
||||
env_vars["PORT"] = "8000"
|
||||
else:
|
||||
env_vars = {
|
||||
"HOST": "0.0.0.0",
|
||||
"PORT": "8000",
|
||||
"RELOAD": "true",
|
||||
"LOG_LEVEL": "DEBUG",
|
||||
"DEBUG": "true"
|
||||
}
|
||||
if not render_port:
|
||||
env_vars["PORT"] = "8000"
|
||||
|
||||
for key, value in env_vars.items():
|
||||
os.environ.setdefault(key, value)
|
||||
|
||||
@@ -39,10 +39,9 @@ class ProductionOptimizer:
|
||||
def _set_production_env_vars(self) -> None:
|
||||
"""Set production-specific environment variables."""
|
||||
production_vars = {
|
||||
# Note: PORT is NOT set here - it's provided by the deployment platform (e.g., Render)
|
||||
# Don't override PORT as it must come from the environment
|
||||
# Note: HOST is not set here - it's auto-detected by start_backend()
|
||||
# Based on deployment environment (cloud vs local)
|
||||
'PORT': '8000',
|
||||
'RELOAD': 'false',
|
||||
'LOG_LEVEL': 'INFO',
|
||||
'DEBUG': 'false',
|
||||
|
||||
@@ -16,7 +16,7 @@ CORE_ROUTER_REGISTRY = [
|
||||
{"name": "component_logic", "module": "api.component_logic", "attr": "router", "features": {"all", "core"}},
|
||||
{"name": "subscription", "module": "api.subscription", "attr": "router", "features": {"all", "core", "podcast", "blog-writer", "youtube"}},
|
||||
{"name": "step3_research", "module": "api.onboarding_utils.step3_routes", "attr": "router", "features": {"all", "core"}},
|
||||
{"name": "step4_assets", "module": "api.onboarding_utils.step4_asset_routes", "attr": "router", "features": {"all", "core", "podcast"}},
|
||||
{"name": "step4_assets", "module": "api.onboarding_utils.step4_asset_routes", "attr": "router", "features": {"all", "core"}},
|
||||
{"name": "step4_persona", "module": "api.onboarding_utils.step4_persona_routes_optimized", "attr": "router", "features": {"all", "core"}},
|
||||
{"name": "gsc_auth", "module": "routers.gsc_auth", "attr": "router", "features": {"all", "core", "seo"}},
|
||||
{"name": "wordpress_oauth", "module": "routers.wordpress_oauth", "attr": "router", "features": {"all", "core"}},
|
||||
@@ -116,6 +116,10 @@ class RouterManager:
|
||||
if "all" in enabled_features:
|
||||
return True
|
||||
|
||||
# Skip core routers in podcast-only mode (they require non-podcast features)
|
||||
if enabled_features == {"podcast"}:
|
||||
return False
|
||||
|
||||
# If no required features specified, include by default
|
||||
if not required_features:
|
||||
return True
|
||||
|
||||
@@ -5,60 +5,50 @@ The onboarding endpoints are re-exported from a stable module
|
||||
`onboarding.py`.
|
||||
"""
|
||||
|
||||
import os
|
||||
from .onboarding_endpoints import (
|
||||
health_check,
|
||||
get_onboarding_status,
|
||||
get_onboarding_progress_full,
|
||||
get_step_data,
|
||||
complete_step,
|
||||
skip_step,
|
||||
validate_step_access,
|
||||
get_api_keys,
|
||||
save_api_key,
|
||||
validate_api_keys,
|
||||
start_onboarding,
|
||||
complete_onboarding,
|
||||
reset_onboarding,
|
||||
get_resume_info,
|
||||
get_onboarding_config,
|
||||
generate_writing_personas,
|
||||
generate_writing_personas_async,
|
||||
get_persona_task_status,
|
||||
assess_persona_quality,
|
||||
regenerate_persona,
|
||||
get_persona_generation_options
|
||||
)
|
||||
|
||||
# Check podcast mode early
|
||||
_is_podcast = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||
|
||||
# In podcast mode, don't import heavy onboarding endpoints
|
||||
# They trigger heavy dependencies (exa_py, etc.)
|
||||
if _is_podcast:
|
||||
__all__ = []
|
||||
else:
|
||||
from .onboarding_endpoints import (
|
||||
health_check,
|
||||
get_onboarding_status,
|
||||
get_onboarding_progress_full,
|
||||
get_step_data,
|
||||
complete_step,
|
||||
skip_step,
|
||||
validate_step_access,
|
||||
get_api_keys,
|
||||
save_api_key,
|
||||
validate_api_keys,
|
||||
start_onboarding,
|
||||
complete_onboarding,
|
||||
reset_onboarding,
|
||||
get_resume_info,
|
||||
get_onboarding_config,
|
||||
generate_writing_personas,
|
||||
generate_writing_personas_async,
|
||||
get_persona_task_status,
|
||||
assess_persona_quality,
|
||||
regenerate_persona,
|
||||
get_persona_generation_options
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
'health_check',
|
||||
'get_onboarding_status',
|
||||
'get_onboarding_progress_full',
|
||||
'get_step_data',
|
||||
'complete_step',
|
||||
'skip_step',
|
||||
'validate_step_access',
|
||||
'get_api_keys',
|
||||
'save_api_key',
|
||||
'validate_api_keys',
|
||||
'start_onboarding',
|
||||
'complete_onboarding',
|
||||
'reset_onboarding',
|
||||
'get_resume_info',
|
||||
'get_onboarding_config',
|
||||
'generate_writing_personas',
|
||||
'generate_writing_personas_async',
|
||||
'get_persona_task_status',
|
||||
'assess_persona_quality',
|
||||
'regenerate_persona',
|
||||
'get_persona_generation_options'
|
||||
]
|
||||
__all__ = [
|
||||
'health_check',
|
||||
'get_onboarding_status',
|
||||
'get_onboarding_progress_full',
|
||||
'get_step_data',
|
||||
'complete_step',
|
||||
'skip_step',
|
||||
'validate_step_access',
|
||||
'get_api_keys',
|
||||
'save_api_key',
|
||||
'validate_api_keys',
|
||||
'start_onboarding',
|
||||
'complete_onboarding',
|
||||
'reset_onboarding',
|
||||
'get_resume_info',
|
||||
'get_onboarding_config',
|
||||
'generate_writing_personas',
|
||||
'generate_writing_personas_async',
|
||||
'get_persona_task_status',
|
||||
'assess_persona_quality',
|
||||
'regenerate_persona',
|
||||
'get_persona_generation_options'
|
||||
]
|
||||
@@ -1,104 +1,52 @@
|
||||
"""
|
||||
Assets Serving Router
|
||||
|
||||
Serves user-uploaded assets (avatars, voice samples) from workspace storage.
|
||||
Uses authenticated or query-token access for security.
|
||||
Audio MIME types are set correctly based on file extension so browsers
|
||||
can play voice clone previews without NotSupportedError.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi.responses import FileResponse
|
||||
import os
|
||||
from pathlib import Path
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi.responses import FileResponse
|
||||
from loguru import logger
|
||||
from typing import Dict, Any
|
||||
|
||||
from middleware.auth_middleware import get_current_user_with_query_token
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
from utils.storage_paths import get_repo_root, sanitize_user_id
|
||||
from services.database import WORKSPACE_DIR, get_user_db_path
|
||||
|
||||
router = APIRouter(prefix="/api/assets", tags=["Assets Serving"])
|
||||
|
||||
MIME_MAP = {
|
||||
".wav": "audio/wav",
|
||||
".mp3": "audio/mpeg",
|
||||
".ogg": "audio/ogg",
|
||||
".opus": "audio/opus",
|
||||
".webm": "audio/webm",
|
||||
".m4a": "audio/mp4",
|
||||
".aac": "audio/aac",
|
||||
".flac": "audio/flac",
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".gif": "image/gif",
|
||||
".webp": "image/webp",
|
||||
".svg": "image/svg+xml",
|
||||
}
|
||||
|
||||
|
||||
def _resolve_asset_path(user_id: str, category: str, filename: str) -> Path:
|
||||
"""Resolve asset path in user workspace with path-traversal protection."""
|
||||
safe_user_id = sanitize_user_id(user_id)
|
||||
repo_root = get_repo_root()
|
||||
|
||||
file_path = (repo_root / "workspace" / f"workspace_{safe_user_id}" / "assets" / category / filename).resolve()
|
||||
|
||||
workspace_dir = (repo_root / "workspace" / f"workspace_{safe_user_id}").resolve()
|
||||
if not str(file_path).startswith(str(workspace_dir)):
|
||||
raise HTTPException(status_code=403, detail="Access denied")
|
||||
|
||||
return file_path
|
||||
|
||||
|
||||
def _get_media_type(filename: str) -> str:
|
||||
"""Determine MIME type from file extension, with fallback."""
|
||||
ext = Path(filename).suffix.lower()
|
||||
return MIME_MAP.get(ext, "application/octet-stream")
|
||||
|
||||
|
||||
@router.get("/{user_id}/avatars/{filename}")
|
||||
async def serve_avatar(
|
||||
user_id: str,
|
||||
filename: str,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user_with_query_token),
|
||||
):
|
||||
"""Serve avatar images. Supports auth via Authorization header or ?token= query param."""
|
||||
require_authenticated_user(current_user)
|
||||
|
||||
async def serve_avatar(user_id: str, filename: str):
|
||||
"""
|
||||
Serve avatar images directly.
|
||||
Public endpoint relying on unguessable filenames.
|
||||
"""
|
||||
# Sanitize user_id (simple check to prevent directory traversal)
|
||||
safe_user_id = "".join(c for c in user_id if c.isalnum() or c in ('-', '_'))
|
||||
if safe_user_id != user_id:
|
||||
raise HTTPException(status_code=400, detail="Invalid user ID")
|
||||
|
||||
# Sanitize filename
|
||||
safe_filename = os.path.basename(filename)
|
||||
file_path = _resolve_asset_path(user_id, "avatars", safe_filename)
|
||||
|
||||
|
||||
# Construct path
|
||||
# workspace/workspace_{user_id}/assets/avatars/{filename}
|
||||
file_path = Path(WORKSPACE_DIR) / f"workspace_{safe_user_id}" / "assets" / "avatars" / safe_filename
|
||||
|
||||
if not file_path.exists():
|
||||
raise HTTPException(status_code=404, detail="Asset not found")
|
||||
|
||||
media_type = _get_media_type(safe_filename)
|
||||
return FileResponse(file_path, media_type=media_type)
|
||||
|
||||
|
||||
return FileResponse(file_path)
|
||||
|
||||
@router.get("/{user_id}/voice_samples/{filename}")
|
||||
async def serve_voice_sample(
|
||||
user_id: str,
|
||||
filename: str,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user_with_query_token),
|
||||
):
|
||||
"""Serve voice sample audio files.
|
||||
|
||||
Supports auth via Authorization header or ?token= query param.
|
||||
The ?token= param is essential for <audio> elements and new Audio()
|
||||
which cannot send Authorization headers.
|
||||
async def serve_voice_sample(user_id: str, filename: str):
|
||||
"""
|
||||
require_authenticated_user(current_user)
|
||||
|
||||
Serve voice sample audio files directly.
|
||||
"""
|
||||
# Sanitize user_id
|
||||
safe_user_id = "".join(c for c in user_id if c.isalnum() or c in ('-', '_'))
|
||||
if safe_user_id != user_id:
|
||||
raise HTTPException(status_code=400, detail="Invalid user ID")
|
||||
|
||||
# Sanitize filename
|
||||
safe_filename = os.path.basename(filename)
|
||||
file_path = _resolve_asset_path(user_id, "voice_samples", safe_filename)
|
||||
|
||||
|
||||
# Construct path
|
||||
# workspace/workspace_{user_id}/assets/voice_samples/{filename}
|
||||
file_path = Path(WORKSPACE_DIR) / f"workspace_{safe_user_id}" / "assets" / "voice_samples" / safe_filename
|
||||
|
||||
if not file_path.exists():
|
||||
logger.info(f"[Assets] Voice sample not found: {file_path}")
|
||||
raise HTTPException(status_code=404, detail="Asset not found")
|
||||
|
||||
media_type = _get_media_type(safe_filename)
|
||||
file_size = file_path.stat().st_size
|
||||
logger.warning(f"[Assets] Serving voice sample: {safe_filename} ({media_type}, {file_size} bytes)")
|
||||
return FileResponse(file_path, media_type=media_type)
|
||||
|
||||
return FileResponse(file_path)
|
||||
|
||||
@@ -9,27 +9,13 @@ from fastapi.responses import FileResponse
|
||||
from sqlalchemy.orm import Session
|
||||
from pydantic import BaseModel
|
||||
from loguru import logger
|
||||
from .step4_persona_routes import _extract_user_id
|
||||
from middleware.auth_middleware import get_current_user
|
||||
|
||||
|
||||
def _extract_user_id(user: Dict[str, Any]) -> str:
|
||||
"""Extract a stable user ID from Clerk-authenticated user payloads.
|
||||
Prefers 'clerk_user_id' or 'id', falls back to 'user_id', else 'unknown'.
|
||||
"""
|
||||
if not isinstance(user, dict):
|
||||
return 'unknown'
|
||||
return (
|
||||
user.get('clerk_user_id')
|
||||
or user.get('id')
|
||||
or user.get('user_id')
|
||||
or 'unknown'
|
||||
)
|
||||
import base64
|
||||
import os
|
||||
from pathlib import Path
|
||||
from utils.file_storage import save_file_safely, generate_unique_filename
|
||||
from services.database import get_db
|
||||
from utils.storage_paths import get_user_workspace, sanitize_user_id
|
||||
from services.database import get_db, WORKSPACE_DIR
|
||||
from utils.asset_tracker import save_asset_to_library
|
||||
from models.content_asset_models import ContentAsset, AssetType, AssetSource
|
||||
from sqlalchemy import desc
|
||||
@@ -87,8 +73,6 @@ async def get_latest_avatar(
|
||||
try:
|
||||
user_id = _extract_user_id(current_user)
|
||||
|
||||
logger.warning(f"[latest-avatar] Looking for avatar for user_id: {user_id}")
|
||||
|
||||
# Search for assets that are either:
|
||||
# 1. Saved with source_module=BRAND_AVATAR_GENERATOR (new)
|
||||
# 2. Saved with source_module=STORY_WRITER but have metadata category='brand_avatar' (legacy)
|
||||
@@ -103,8 +87,6 @@ async def get_latest_avatar(
|
||||
])
|
||||
).order_by(desc(ContentAsset.created_at)).limit(50).all()
|
||||
|
||||
logger.warning(f"[latest-avatar] Found {len(candidates)} candidate(s)")
|
||||
|
||||
asset = None
|
||||
for candidate in candidates:
|
||||
# Check for direct match (new assets)
|
||||
@@ -185,7 +167,7 @@ async def generate_avatar(
|
||||
try:
|
||||
user_id = _extract_user_id(current_user)
|
||||
|
||||
logger.warning(f"Generating avatar for user {user_id} with prompt: {request.prompt}")
|
||||
logger.info(f"Generating avatar for user {user_id} with prompt: {request.prompt}")
|
||||
|
||||
# 1. Generate Image
|
||||
result = await generate_image_with_provider(
|
||||
@@ -235,7 +217,7 @@ async def generate_avatar(
|
||||
content_to_save = base64.b64decode(image_data) if isinstance(image_data, str) else image_data
|
||||
|
||||
# Construct user assets directory
|
||||
user_assets_dir = get_user_workspace(user_id) / "assets" / "avatars"
|
||||
user_assets_dir = Path(WORKSPACE_DIR) / f"workspace_{user_id}" / "assets" / "avatars"
|
||||
|
||||
saved_path, error = save_file_safely(
|
||||
content_to_save,
|
||||
@@ -288,7 +270,7 @@ async def enhance_prompt_route(
|
||||
"""Enhance a simple prompt into a detailed midjourney-style prompt."""
|
||||
try:
|
||||
user_id = _extract_user_id(current_user)
|
||||
logger.warning(f"Enhancing prompt for user {user_id}: {request.prompt}")
|
||||
logger.info(f"Enhancing prompt for user {user_id}: {request.prompt}")
|
||||
|
||||
enhanced_prompt = await enhance_image_prompt(request.prompt, user_id=user_id)
|
||||
|
||||
@@ -312,7 +294,7 @@ async def create_variation_route(
|
||||
"""Generate a variation of an existing avatar."""
|
||||
try:
|
||||
user_id = _extract_user_id(current_user)
|
||||
logger.warning(f"Creating variation for user {user_id} with prompt: {prompt}")
|
||||
logger.info(f"Creating variation for user {user_id} with prompt: {prompt}")
|
||||
|
||||
# Read file
|
||||
file_content = await file.read()
|
||||
@@ -333,7 +315,7 @@ async def create_variation_route(
|
||||
content_to_save = base64.b64decode(image_data)
|
||||
|
||||
# Construct user assets directory
|
||||
user_assets_dir = get_user_workspace(user_id) / "assets" / "avatars"
|
||||
user_assets_dir = Path(WORKSPACE_DIR) / f"workspace_{user_id}" / "assets" / "avatars"
|
||||
|
||||
saved_path, error = save_file_safely(
|
||||
content_to_save,
|
||||
@@ -387,7 +369,7 @@ async def enhance_avatar_route(
|
||||
"""Enhance/Upscale an existing avatar."""
|
||||
try:
|
||||
user_id = _extract_user_id(current_user)
|
||||
logger.warning(f"Enhancing avatar for user {user_id}")
|
||||
logger.info(f"Enhancing avatar for user {user_id}")
|
||||
|
||||
# Read file
|
||||
file_content = await file.read()
|
||||
@@ -407,7 +389,7 @@ async def enhance_avatar_route(
|
||||
content_to_save = base64.b64decode(image_data)
|
||||
|
||||
# Construct user assets directory
|
||||
user_assets_dir = get_user_workspace(user_id) / "assets" / "avatars"
|
||||
user_assets_dir = Path(WORKSPACE_DIR) / f"workspace_{user_id}" / "assets" / "avatars"
|
||||
|
||||
saved_path, error = save_file_safely(
|
||||
content_to_save,
|
||||
@@ -464,13 +446,13 @@ async def create_voice_clone(
|
||||
"""Create a voice clone from an audio file."""
|
||||
try:
|
||||
user_id = _extract_user_id(current_user)
|
||||
logger.warning(f"[VoiceClone] Creating voice clone '{voice_name}' (engine={engine}) for user {user_id}")
|
||||
logger.info(f"Creating voice clone '{voice_name}' (engine={engine}) for user {user_id}")
|
||||
|
||||
# 1. Save uploaded audio file
|
||||
file_content = await file.read()
|
||||
filename = generate_unique_filename("voice_sample", Path(file.filename).suffix.lstrip("."))
|
||||
|
||||
user_voice_dir = get_user_workspace(user_id) / "assets" / "voice_samples"
|
||||
user_voice_dir = Path(WORKSPACE_DIR) / f"workspace_{user_id}" / "assets" / "voice_samples"
|
||||
saved_path, error = save_file_safely(file_content, user_voice_dir, filename)
|
||||
|
||||
if error or not saved_path:
|
||||
@@ -492,7 +474,7 @@ async def create_voice_clone(
|
||||
random_suffix = ''.join(random.choices(string.ascii_letters + string.digits, k=8))
|
||||
custom_voice_id = f"vc_{random_suffix}"
|
||||
|
||||
logger.warning(f"Cloning voice with Minimax, ID: {custom_voice_id}")
|
||||
logger.info(f"Cloning voice with Minimax, ID: {custom_voice_id}")
|
||||
|
||||
# Run blocking call in executor
|
||||
result = await loop.run_in_executor(
|
||||
@@ -507,7 +489,7 @@ async def create_voice_clone(
|
||||
preview_audio_bytes = result.preview_audio_bytes
|
||||
|
||||
elif engine.lower() == "cosyvoice":
|
||||
logger.warning("Cloning voice with CosyVoice")
|
||||
logger.info("Cloning voice with CosyVoice")
|
||||
result = await loop.run_in_executor(
|
||||
None,
|
||||
lambda: cosyvoice_voice_clone(
|
||||
@@ -522,7 +504,7 @@ async def create_voice_clone(
|
||||
custom_voice_id = f"vc_cosy_{asset_uuid}"
|
||||
|
||||
else: # qwen3 (default)
|
||||
logger.warning("Cloning voice with Qwen3")
|
||||
logger.info("Cloning voice with Qwen3")
|
||||
result = await loop.run_in_executor(
|
||||
None,
|
||||
lambda: qwen3_voice_clone(
|
||||
@@ -538,48 +520,27 @@ async def create_voice_clone(
|
||||
|
||||
# 3. Save Preview Audio (if generated)
|
||||
preview_url = None
|
||||
preview_mime_type = "audio/wav"
|
||||
actual_filename = None # Default if preview save fails
|
||||
|
||||
if preview_audio_bytes and len(preview_audio_bytes) > 0:
|
||||
from utils.media_utils import detect_audio_format, ensure_audio_extension
|
||||
if preview_audio_bytes:
|
||||
preview_filename = f"preview_{filename}"
|
||||
# Ensure it ends with .wav
|
||||
if not preview_filename.endswith(".wav"):
|
||||
preview_filename = str(Path(preview_filename).with_suffix('.wav'))
|
||||
|
||||
detected_fmt, preview_mime_type = detect_audio_format(preview_audio_bytes)
|
||||
logger.warning(f"[VoiceClone] Detected preview audio format: {detected_fmt} ({preview_mime_type}), {len(preview_audio_bytes)} bytes")
|
||||
|
||||
# Build filename with correct extension based on actual content format
|
||||
original_stem = Path(filename).stem
|
||||
preview_filename = f"preview_{original_stem}"
|
||||
preview_filename = ensure_audio_extension(preview_filename, preview_audio_bytes)
|
||||
|
||||
user_voice_dir = get_user_workspace(user_id) / "assets" / "voice_samples"
|
||||
user_voice_dir = Path(WORKSPACE_DIR) / f"workspace_{user_id}" / "assets" / "voice_samples"
|
||||
saved_preview_path, error = save_file_safely(preview_audio_bytes, user_voice_dir, preview_filename)
|
||||
|
||||
if not error and saved_preview_path:
|
||||
# Use actual saved filename (may have UUID suffix added by save_file_safely)
|
||||
actual_filename = saved_preview_path.name
|
||||
preview_url = f"/api/assets/{user_id}/voice_samples/{actual_filename}"
|
||||
logger.warning(f"[VoiceClone] Saved preview: {actual_filename} ({saved_preview_path.stat().st_size} bytes, {preview_mime_type})")
|
||||
|
||||
# Verify file exists
|
||||
if not saved_preview_path.exists():
|
||||
logger.warning(f"[VoiceClone] Preview file does not exist after save: {saved_preview_path}")
|
||||
preview_url = None
|
||||
else:
|
||||
logger.warning(f"[VoiceClone] Failed to save preview audio: {error}")
|
||||
preview_url = f"/api/assets/{user_id}/voice_samples/{preview_filename}"
|
||||
|
||||
# 4. Save to Asset Library
|
||||
# Use the preview file (with corrected .wav extension) as the main asset file
|
||||
has_valid_preview = preview_audio_bytes and len(preview_audio_bytes) > 0 and saved_preview_path
|
||||
stored_filename = actual_filename if has_valid_preview else filename
|
||||
asset_id = save_asset_to_library(
|
||||
db=db,
|
||||
user_id=user_id,
|
||||
file_path=file_path,
|
||||
asset_type="audio",
|
||||
source_module="voice_cloner",
|
||||
filename=stored_filename,
|
||||
file_url=f"/api/assets/{user_id}/voice_samples/{stored_filename}",
|
||||
filename=filename,
|
||||
file_url=f"/api/assets/{user_id}/voice_samples/{filename}",
|
||||
asset_metadata={
|
||||
"voice_name": voice_name,
|
||||
"engine": engine,
|
||||
@@ -594,7 +555,7 @@ async def create_voice_clone(
|
||||
return {
|
||||
"success": True,
|
||||
"custom_voice_id": custom_voice_id,
|
||||
"preview_audio_url": preview_url or f"/api/assets/{user_id}/voice_samples/{stored_filename}",
|
||||
"preview_audio_url": preview_url or f"/api/assets/{user_id}/voice_samples/{filename}",
|
||||
"asset_id": asset_id,
|
||||
"message": "Voice clone created successfully"
|
||||
}
|
||||
@@ -613,7 +574,7 @@ async def create_voice_design(
|
||||
"""Create a voice from text description (Voice Design)."""
|
||||
try:
|
||||
user_id = _extract_user_id(current_user)
|
||||
logger.warning(f"Designing voice for user {user_id}")
|
||||
logger.info(f"Designing voice for user {user_id}")
|
||||
|
||||
loop = asyncio.get_event_loop()
|
||||
|
||||
@@ -627,15 +588,9 @@ async def create_voice_design(
|
||||
)
|
||||
)
|
||||
|
||||
# Save the result to a file with correct extension based on content
|
||||
from utils.media_utils import detect_audio_format, ensure_audio_extension
|
||||
detected_fmt, mime_type = detect_audio_format(result.preview_audio_bytes)
|
||||
logger.warning(f"[VoiceDesign] Detected audio format: {detected_fmt} ({mime_type})")
|
||||
|
||||
filename = generate_unique_filename("voice_design_preview", detected_fmt)
|
||||
filename = ensure_audio_extension(filename, result.preview_audio_bytes)
|
||||
|
||||
user_voice_dir = get_user_workspace(user_id) / "assets" / "voice_samples"
|
||||
# Save the result to a temporary file
|
||||
filename = generate_unique_filename("voice_design_preview", "wav")
|
||||
user_voice_dir = Path(WORKSPACE_DIR) / f"workspace_{user_id}" / "assets" / "voice_samples"
|
||||
saved_path, error = save_file_safely(result.preview_audio_bytes, user_voice_dir, filename)
|
||||
|
||||
if error or not saved_path:
|
||||
|
||||
@@ -2,26 +2,34 @@
|
||||
Podcast API Constants
|
||||
|
||||
Centralized constants and directory configuration for podcast module.
|
||||
All workspace paths use utils.storage_paths for root resolution.
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
from loguru import logger
|
||||
from services.story_writer.audio_generation_service import StoryAudioGenerationService
|
||||
from utils.storage_paths import get_repo_root, sanitize_user_id as _sanitize_user_id
|
||||
|
||||
ROOT_DIR = get_repo_root()
|
||||
# Directory paths
|
||||
# router.py is at: backend/api/podcast/router.py
|
||||
# parents[0] = backend/api/podcast/
|
||||
# parents[1] = backend/api/
|
||||
# parents[2] = backend/
|
||||
# parents[3] = root/
|
||||
ROOT_DIR = Path(__file__).resolve().parents[3] # root/
|
||||
DATA_MEDIA_DIR = ROOT_DIR / "data" / "media"
|
||||
|
||||
# Video subdirectory (relative to workspace media dir)
|
||||
PODCAST_AUDIO_DIR = (DATA_MEDIA_DIR / "podcast_audio").resolve()
|
||||
PODCAST_IMAGES_DIR = (DATA_MEDIA_DIR / "podcast_images").resolve()
|
||||
PODCAST_VIDEOS_DIR = (DATA_MEDIA_DIR / "podcast_videos").resolve()
|
||||
|
||||
# Video subdirectory
|
||||
AI_VIDEO_SUBDIR = Path("AI_Videos")
|
||||
|
||||
# Legacy constants - DEPRECATED, use get_podcast_media_dir() instead
|
||||
# Kept for backward compatibility with some handlers
|
||||
PODCAST_AVATARS_SUBDIR = Path("avatars")
|
||||
MediaType = Literal["audio", "image", "video"]
|
||||
|
||||
MediaType = Literal["audio", "image", "video", "chart"]
|
||||
|
||||
def _sanitize_user_id(user_id: str) -> str:
|
||||
return "".join(c for c in user_id if c.isalnum() or c in ("-", "_"))
|
||||
|
||||
|
||||
def get_podcast_media_dir(
|
||||
@@ -30,30 +38,21 @@ def get_podcast_media_dir(
|
||||
*,
|
||||
ensure_exists: bool = False,
|
||||
) -> Path:
|
||||
"""
|
||||
Resolve podcast media directory (workspace-only for multi-tenant isolation).
|
||||
|
||||
Requires user_id for tenant isolation. Falls back to default workspace
|
||||
only if no user_id provided (for backward compat in development).
|
||||
Logs a warning in production when user_id is missing.
|
||||
"""
|
||||
"""Resolve podcast media directory (tenant workspace first, legacy global fallback)."""
|
||||
media_subdir = {
|
||||
"audio": "podcast_audio",
|
||||
"image": "podcast_images",
|
||||
"video": "podcast_videos",
|
||||
"chart": "podcast_charts",
|
||||
}[media_type]
|
||||
|
||||
if user_id:
|
||||
sanitized = _sanitize_user_id(user_id)
|
||||
resolved_dir = (
|
||||
ROOT_DIR / "workspace" / f"workspace_{sanitized}" / "media" / media_subdir
|
||||
).resolve()
|
||||
tenant_media_dir = ROOT_DIR / "workspace" / f"workspace_{sanitized}" / "media" / media_subdir
|
||||
resolved_dir = tenant_media_dir.resolve()
|
||||
else:
|
||||
logger.warning(f"[Podcast] get_podcast_media_dir called without user_id for {media_type} — using default workspace. This should not happen in production.")
|
||||
resolved_dir = (
|
||||
ROOT_DIR / "workspace" / "workspace_alwrity" / "media" / media_subdir
|
||||
).resolve()
|
||||
resolved_dir = (DATA_MEDIA_DIR / media_subdir).resolve()
|
||||
|
||||
logger.debug(f"[Podcast] get_podcast_media_dir: type={media_type}, user_id={user_id}, sanitized={user_id and _sanitize_user_id(user_id)}, resolved={resolved_dir}")
|
||||
|
||||
if ensure_exists:
|
||||
resolved_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -62,11 +61,14 @@ def get_podcast_media_dir(
|
||||
|
||||
|
||||
def get_podcast_media_read_dirs(media_type: MediaType, user_id: str | None = None) -> list[Path]:
|
||||
"""
|
||||
Return directories to search for podcast media.
|
||||
Now workspace-only (no legacy fallback).
|
||||
"""
|
||||
return [get_podcast_media_dir(media_type, user_id)]
|
||||
"""Return ordered directories to search (tenant path first, then legacy global path)."""
|
||||
dirs: list[Path] = []
|
||||
if user_id:
|
||||
dirs.append(get_podcast_media_dir(media_type, user_id))
|
||||
logger.debug(f"[Podcast] get_podcast_media_read_dirs: added user dir for {user_id}")
|
||||
dirs.append(get_podcast_media_dir(media_type, None))
|
||||
logger.debug(f"[Podcast] get_podcast_media_read_dirs: dirs={dirs}")
|
||||
return dirs
|
||||
|
||||
|
||||
def get_podcast_audio_service(user_id: str | None = None) -> StoryAudioGenerationService:
|
||||
|
||||
@@ -1,216 +0,0 @@
|
||||
"""
|
||||
Podcast cost estimation helpers.
|
||||
|
||||
Builds user-facing podcast estimates from the subscription pricing catalog
|
||||
instead of hard-coded frontend heuristics.
|
||||
|
||||
Supports multiple models for each component:
|
||||
- Audio TTS: minimax/speech-02-hd (default), qwen3-tts, cosyvoice-tts
|
||||
- Voice Clone: qwen3, cosyvoice, minimax
|
||||
- Image: qwen-image (default), ideogram-v3-turbo
|
||||
- Video: wan-2.5 (default), kling-v2.5, infinitetalk
|
||||
- LLM: gemini-2.5-flash (default)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, Optional
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from models.subscription_models import APIProvider
|
||||
from services.subscription.pricing_service import PricingService
|
||||
|
||||
|
||||
def _round_money(value: float) -> float:
|
||||
return round(float(value), 4)
|
||||
|
||||
|
||||
def _load_pricing(
|
||||
pricing_service: PricingService,
|
||||
provider: APIProvider,
|
||||
preferred_model: str,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""Load pricing for a provider and model, with fallback to default."""
|
||||
pricing = pricing_service.get_pricing_for_provider_model(provider, preferred_model)
|
||||
if pricing:
|
||||
return pricing
|
||||
# Fallback to provider default model row (if configured).
|
||||
return pricing_service.get_pricing_for_provider_model(provider, "default")
|
||||
|
||||
|
||||
# Default models used in podcast generation
|
||||
DEFAULT_MODELS = {
|
||||
"gemini": "gemini-2.5-flash",
|
||||
"exa": "exa-search",
|
||||
"audio_tts": "minimax/speech-02-hd",
|
||||
"voice_clone": "wavespeed-ai/qwen3-tts/voice-clone",
|
||||
"image": "qwen-image",
|
||||
"video": "wan-2.5",
|
||||
}
|
||||
|
||||
|
||||
def estimate_podcast_cost(
|
||||
*,
|
||||
db: Session,
|
||||
duration_minutes: int,
|
||||
speakers: int,
|
||||
query_count: int,
|
||||
include_avatar_phase: bool = True,
|
||||
# Optional model overrides
|
||||
gemini_model: str = "gemini-2.5-flash",
|
||||
audio_tts_model: str = "minimax/speech-02-hd",
|
||||
voice_clone_engine: str = "qwen3",
|
||||
image_model: str = "qwen-image",
|
||||
video_model: str = "wan-2.5",
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Compute a backend estimate for podcast creation.
|
||||
|
||||
Supports customizable models for each component.
|
||||
Uses pricing_catalog for accurate cost calculation.
|
||||
"""
|
||||
pricing_service = PricingService(db)
|
||||
|
||||
# Load pricing for each component and model
|
||||
gemini_pricing = _load_pricing(pricing_service, APIProvider.GEMINI, gemini_model)
|
||||
exa_pricing = _load_pricing(pricing_service, APIProvider.EXA, "exa-search")
|
||||
|
||||
# Audio TTS pricing (minimax/speech-02-hd)
|
||||
audio_pricing = _load_pricing(pricing_service, APIProvider.AUDIO, audio_tts_model)
|
||||
|
||||
# Voice clone pricing (different engines)
|
||||
voice_clone_model = f"wavespeed-ai/{voice_clone_engine}-tts/voice-clone"
|
||||
voice_clone_pricing = _load_pricing(pricing_service, APIProvider.AUDIO, voice_clone_model)
|
||||
if not voice_clone_pricing:
|
||||
# Try alternate model names
|
||||
voice_clone_pricing = _load_pricing(pricing_service, APIProvider.AUDIO, f"{voice_clone_engine}/voice-clone")
|
||||
|
||||
# Image pricing (qwen-image or ideogram)
|
||||
image_pricing = _load_pricing(pricing_service, APIProvider.STABILITY, image_model)
|
||||
|
||||
# Video pricing (wan-2.5, kling, or infinitetalk)
|
||||
video_pricing = _load_pricing(pricing_service, APIProvider.VIDEO, video_model)
|
||||
|
||||
# Return None if critical pricing unavailable (fail fast)
|
||||
if not gemini_pricing:
|
||||
return None
|
||||
|
||||
# Configuration
|
||||
minutes = max(1, int(duration_minutes or 1))
|
||||
speaker_count = max(1, int(speakers or 1))
|
||||
research_queries = max(1, int(query_count or 1))
|
||||
|
||||
# Token usage assumptions per phase
|
||||
analysis_input_tokens = 1800
|
||||
analysis_output_tokens = 1000
|
||||
research_synthesis_input_tokens = 2200
|
||||
research_synthesis_output_tokens = 900
|
||||
script_input_tokens = max(1800, minutes * 300)
|
||||
script_output_tokens = max(2200, minutes * 700)
|
||||
|
||||
# TTS: ~900 chars per minute per speaker
|
||||
estimated_tts_tokens = max(900, minutes * 900 * speaker_count)
|
||||
|
||||
# Voice clone: 1 clone operation per speaker
|
||||
voice_clone_count = speaker_count
|
||||
|
||||
# ===== COST CALCULATIONS =====
|
||||
|
||||
# 1. Analysis phase (LLM)
|
||||
analysis_cost = (
|
||||
analysis_input_tokens * float(gemini_pricing.get("cost_per_input_token") or 0.0)
|
||||
+ analysis_output_tokens * float(gemini_pricing.get("cost_per_output_token") or 0.0)
|
||||
)
|
||||
|
||||
# 2. Research phase
|
||||
# 2a. LLM for research synthesis
|
||||
research_llm_cost = (
|
||||
research_synthesis_input_tokens * float(gemini_pricing.get("cost_per_input_token") or 0.0)
|
||||
+ research_synthesis_output_tokens * float(gemini_pricing.get("cost_per_output_token") or 0.0)
|
||||
)
|
||||
# 2b. Search API (Exa)
|
||||
research_search_cost = 0.0
|
||||
if exa_pricing:
|
||||
research_search_cost = research_queries * float(exa_pricing.get("cost_per_request") or 0.0)
|
||||
research_cost = research_search_cost + research_llm_cost
|
||||
|
||||
# 3. Script generation (LLM)
|
||||
script_cost = (
|
||||
script_input_tokens * float(gemini_pricing.get("cost_per_input_token") or 0.0)
|
||||
+ script_output_tokens * float(gemini_pricing.get("cost_per_output_token") or 0.0)
|
||||
)
|
||||
|
||||
# 4. Audio TTS
|
||||
tts_cost = 0.0
|
||||
if audio_pricing:
|
||||
tts_cost = estimated_tts_tokens * float(audio_pricing.get("cost_per_input_token") or 0.0)
|
||||
|
||||
# 5. Voice cloning (if needed)
|
||||
voice_clone_cost = 0.0
|
||||
if voice_clone_pricing:
|
||||
voice_clone_cost = voice_clone_count * (
|
||||
float(voice_clone_pricing.get("cost_per_request") or 0.0)
|
||||
+ estimated_tts_tokens * float(voice_clone_pricing.get("cost_per_input_token") or 0.0)
|
||||
)
|
||||
|
||||
# 6. Avatar image generation
|
||||
avatar_cost = 0.0
|
||||
if include_avatar_phase and image_pricing:
|
||||
image_unit = float(image_pricing.get("cost_per_image") or image_pricing.get("cost_per_request") or 0.0)
|
||||
avatar_cost = speaker_count * image_unit
|
||||
|
||||
# 7. Video rendering
|
||||
video_cost = 0.0
|
||||
if video_pricing:
|
||||
# Assume 1 video render per minute (upper bound)
|
||||
video_cost = minutes * float(video_pricing.get("cost_per_request") or 0.0)
|
||||
|
||||
# ===== TOTALS =====
|
||||
llm_total = analysis_cost + research_llm_cost + script_cost
|
||||
audio_total = tts_cost + voice_clone_cost
|
||||
media_total = avatar_cost + video_cost
|
||||
total = llm_total + research_search_cost + audio_total + media_total
|
||||
|
||||
return {
|
||||
# Cost breakdown
|
||||
"analysisCost": _round_money(analysis_cost),
|
||||
"researchCost": _round_money(research_cost),
|
||||
"researchSearchCost": _round_money(research_search_cost),
|
||||
"researchLlmCost": _round_money(research_llm_cost),
|
||||
"scriptCost": _round_money(script_cost),
|
||||
"ttsCost": _round_money(tts_cost),
|
||||
"voiceCloneCost": _round_money(voice_clone_cost),
|
||||
"avatarCost": _round_money(avatar_cost),
|
||||
"videoCost": _round_money(video_cost),
|
||||
"total": _round_money(total),
|
||||
# Totals by category
|
||||
"llmCost": _round_money(llm_total),
|
||||
"audioCost": _round_money(audio_total),
|
||||
"mediaCost": _round_money(media_total),
|
||||
# Currency
|
||||
"currency": "USD",
|
||||
"source": "pricing_catalog",
|
||||
# Models used for this estimate
|
||||
"models": {
|
||||
"llm": gemini_model,
|
||||
"research": "exa-search",
|
||||
"audio_tts": audio_tts_model,
|
||||
"voice_clone": voice_clone_model,
|
||||
"image": image_model,
|
||||
"video": video_model,
|
||||
},
|
||||
# Assumptions used
|
||||
"assumptions": {
|
||||
"analysis_input_tokens": analysis_input_tokens,
|
||||
"analysis_output_tokens": analysis_output_tokens,
|
||||
"research_synthesis_input_tokens": research_synthesis_input_tokens,
|
||||
"research_synthesis_output_tokens": research_synthesis_output_tokens,
|
||||
"script_input_tokens": script_input_tokens,
|
||||
"script_output_tokens": script_output_tokens,
|
||||
"estimated_tts_tokens": estimated_tts_tokens,
|
||||
"research_queries": research_queries,
|
||||
"voice_clone_count": voice_clone_count,
|
||||
"video_requests": minutes,
|
||||
"avatar_requests": speaker_count if include_avatar_phase else 0,
|
||||
},
|
||||
}
|
||||
@@ -4,9 +4,8 @@ Podcast Analysis Handlers
|
||||
Analysis endpoint for podcast ideas.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import Dict, Any, Optional, List
|
||||
from datetime import datetime
|
||||
import json
|
||||
import uuid
|
||||
from sqlalchemy.orm import Session
|
||||
@@ -20,99 +19,17 @@ from services.llm_providers.main_image_generation import generate_image
|
||||
from services.podcast_bible_service import PodcastBibleService
|
||||
from utils.asset_tracker import save_asset_to_library
|
||||
from loguru import logger
|
||||
import os
|
||||
from ..constants import get_podcast_media_dir
|
||||
from ..prompts import get_enhance_topic_prompt, format_website_context
|
||||
from ..constants import PODCAST_IMAGES_DIR
|
||||
from ..models import (
|
||||
PodcastAnalyzeRequest,
|
||||
PodcastAnalyzeResponse,
|
||||
PodcastEnhanceIdeaRequest,
|
||||
PodcastEnhanceIdeaResponse,
|
||||
ExtractUrlRequest,
|
||||
ExtractUrlResponse,
|
||||
WebsiteAnalysisRequest,
|
||||
WebsiteAnalysisResponse,
|
||||
PodcastPreEstimateRequest,
|
||||
PodcastPreEstimateResponse,
|
||||
PodcastEnhanceIdeaResponse
|
||||
)
|
||||
from ..cost_estimator import estimate_podcast_cost
|
||||
|
||||
# Check if running in podcast-only demo mode
|
||||
def _is_podcast_only_mode() -> bool:
|
||||
"""Check if podcast-only demo mode is enabled."""
|
||||
return os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/pre-estimate", response_model=PodcastPreEstimateResponse)
|
||||
async def pre_estimate_cost(
|
||||
request: PodcastPreEstimateRequest,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Lightweight endpoint to estimate podcast creation cost before analysis.
|
||||
|
||||
Takes user configuration (duration, speakers, query_count, podcast_mode) and returns
|
||||
a cost estimate WITHOUT running full analysis.
|
||||
|
||||
Optional model overrides can be specified to estimate with different models.
|
||||
"""
|
||||
try:
|
||||
include_avatar_phase = request.podcast_mode != "audio_only"
|
||||
|
||||
estimate = estimate_podcast_cost(
|
||||
db=db,
|
||||
duration_minutes=request.duration,
|
||||
speakers=request.speakers,
|
||||
query_count=request.query_count,
|
||||
include_avatar_phase=include_avatar_phase,
|
||||
# Model overrides if provided
|
||||
gemini_model=request.gemini_model or "gemini-2.5-flash",
|
||||
audio_tts_model=request.audio_tts_model or "minimax/speech-02-hd",
|
||||
voice_clone_engine=request.voice_clone_engine or "qwen3",
|
||||
image_model=request.image_model or "qwen-image",
|
||||
video_model=request.video_model or "wan-2.5",
|
||||
)
|
||||
|
||||
# Debug: get pricing row count and providers
|
||||
from models.subscription_models import APIProviderPricing
|
||||
pricing_count = db.query(APIProviderPricing).count()
|
||||
providers = db.query(APIProviderPricing.provider).distinct().all()
|
||||
provider_list = sorted([p[0].value for p in providers]) if providers else []
|
||||
|
||||
debug_info = {
|
||||
"pricing_rows": pricing_count,
|
||||
"providers": provider_list,
|
||||
}
|
||||
|
||||
# Log pricing debug info at warning level
|
||||
logger.warning(f"[PRE-ESTIMATE] Pricing debug: rows={pricing_count}, providers={provider_list}")
|
||||
logger.warning(f"[PRE-ESTIMATE] Models: llm={request.gemini_model}, tts={request.audio_tts_model}, video={request.video_model}")
|
||||
|
||||
if estimate is None:
|
||||
return PodcastPreEstimateResponse(
|
||||
estimate=None,
|
||||
error="Pricing data unavailable. Please try again later.",
|
||||
pricing_available=False,
|
||||
debug=debug_info,
|
||||
)
|
||||
|
||||
return PodcastPreEstimateResponse(
|
||||
estimate=estimate,
|
||||
error=None,
|
||||
pricing_available=True,
|
||||
debug=debug_info,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Pre-estimate error: {e}")
|
||||
return PodcastPreEstimateResponse(
|
||||
estimate=None,
|
||||
error=str(e),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/idea/enhance", response_model=PodcastEnhanceIdeaResponse)
|
||||
async def enhance_podcast_idea(
|
||||
request: PodcastEnhanceIdeaRequest,
|
||||
@@ -125,55 +42,39 @@ async def enhance_podcast_idea(
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Serialize Bible context if provided or generate from onboarding
|
||||
# In podcast-only mode, skip bible generation since onboarding is disabled
|
||||
bible_context = ""
|
||||
if not _is_podcast_only_mode():
|
||||
logger.warning(f"[Podcast Enhance] Podcast mode=full — attempting Bible generation for user {user_id}")
|
||||
try:
|
||||
bible_service = PodcastBibleService()
|
||||
if request.bible:
|
||||
from models.podcast_bible_models import PodcastBible
|
||||
bible_data = PodcastBible(**request.bible)
|
||||
bible_context = bible_service.serialize_bible(bible_data)
|
||||
else:
|
||||
# Generate from onboarding data directly
|
||||
bible_obj = bible_service.generate_bible(user_id, "temp_enhance")
|
||||
bible_context = bible_service.serialize_bible(bible_obj)
|
||||
except Exception as exc:
|
||||
logger.warning(f"[Podcast Enhance] Failed to parse or generate bible context: {exc}")
|
||||
else:
|
||||
# In podcast mode, use the provided bible directly if available
|
||||
logger.warning(f"[Podcast Enhance] Podcast mode=podcast_only — skipping Bible generation for user {user_id}")
|
||||
try:
|
||||
bible_service = PodcastBibleService()
|
||||
if request.bible:
|
||||
try:
|
||||
from models.podcast_bible_models import PodcastBible
|
||||
bible_data = PodcastBible(**request.bible)
|
||||
bible_service = PodcastBibleService()
|
||||
bible_context = bible_service.serialize_bible(bible_data)
|
||||
except Exception as exc:
|
||||
logger.debug(f"[Podcast Enhance] Bible parsing skipped in podcast mode: {exc}")
|
||||
from models.podcast_bible_models import PodcastBible
|
||||
bible_data = PodcastBible(**request.bible)
|
||||
bible_context = bible_service.serialize_bible(bible_data)
|
||||
else:
|
||||
# Generate from onboarding data directly
|
||||
bible_obj = bible_service.generate_bible(user_id, "temp_enhance")
|
||||
bible_context = bible_service.serialize_bible(bible_obj)
|
||||
except Exception as exc:
|
||||
logger.warning(f"[Podcast Enhance] Failed to parse or generate bible context: {exc}")
|
||||
|
||||
# Log what's being used for context
|
||||
context_used = []
|
||||
if bible_context:
|
||||
context_used.append("Podcast Bible")
|
||||
if request.website_data:
|
||||
context_used.append("Website Extraction")
|
||||
if request.topic_context:
|
||||
category = request.topic_context.get("category", "unknown")
|
||||
context_used.append(f"Category Research ({category})")
|
||||
|
||||
logger.warning(f"[Podcast Enhance] Generating with context: {', '.join(context_used) if context_used else 'basic idea only'}")
|
||||
prompt = f"""
|
||||
You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea.
|
||||
|
||||
# Use new context builder for prompt generation
|
||||
from services.podcast_context_builder import context_builder
|
||||
context_result = context_builder.build_enhance_context(
|
||||
idea=request.idea,
|
||||
bible_context=bible_context,
|
||||
website_data=request.website_data,
|
||||
topic_context=request.topic_context,
|
||||
)
|
||||
prompt = context_result["prompt"]
|
||||
{f"USER PERSONALIZATION CONTEXT (Podcast Bible):\n{bible_context}\n" if bible_context else ""}
|
||||
|
||||
RAW IDEA/KEYWORDS: "{request.idea}"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions, each with a unique angle:
|
||||
1. Professional & Expert-led angle (focus on authority, insights, and expertise)
|
||||
2. Storytelling & Human interest angle (focus on narratives, emotions, and personal connections)
|
||||
3. Trendy & Contemporary angle (focus on current trends, modern perspectives, and relevance)
|
||||
|
||||
Each version should be 2-3 sentences, audience-focused, and align with host persona if provided.
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 enhanced episode pitches (in order: Professional, Storytelling, Trendy)
|
||||
- rationales: array of 3 rationales explaining the approach for each version
|
||||
"""
|
||||
|
||||
try:
|
||||
raw = llm_text_gen(
|
||||
@@ -194,19 +95,6 @@ async def enhance_podcast_idea(
|
||||
enhanced_ideas = data.get("enhanced_ideas", [])
|
||||
rationales = data.get("rationales", [])
|
||||
|
||||
# Handle case where LLM returns objects instead of strings
|
||||
normalized_ideas = []
|
||||
for idea in enhanced_ideas:
|
||||
if isinstance(idea, dict):
|
||||
# Extract title and description from object
|
||||
title = idea.get("title", "")
|
||||
description = idea.get("description", "") or idea.get("content", "")
|
||||
normalized_ideas.append(f"{title}: {description}" if description else title)
|
||||
elif isinstance(idea, str):
|
||||
normalized_ideas.append(idea)
|
||||
|
||||
enhanced_ideas = normalized_ideas
|
||||
|
||||
# Ensure we have exactly 3 ideas, fallback to original if needed
|
||||
if not isinstance(enhanced_ideas, list) or len(enhanced_ideas) != 3:
|
||||
# Fallback: create 3 variations of the original idea
|
||||
@@ -276,11 +164,7 @@ async def analyze_podcast_idea(
|
||||
final_avatar_url = request.avatar_url
|
||||
final_avatar_prompt = None
|
||||
|
||||
# Skip avatar generation for audio_only mode
|
||||
podcast_mode = getattr(request, 'podcast_mode', None) or 'video_only'
|
||||
should_generate_avatar = not final_avatar_url and podcast_mode != 'audio_only'
|
||||
|
||||
if should_generate_avatar:
|
||||
if not final_avatar_url:
|
||||
logger.info(f"[Podcast Analyze] No avatar_url provided, generating one for user {user_id}")
|
||||
try:
|
||||
# 1. PRE-FLIGHT VALIDATION: Check subscription limits for image generation
|
||||
@@ -311,10 +195,8 @@ async def analyze_podcast_idea(
|
||||
if image_result and image_result.image_bytes:
|
||||
img_id = str(uuid.uuid4())[:8]
|
||||
filename = f"presenter_podcast_{user_id}_{img_id}.png"
|
||||
images_dir = get_podcast_media_dir("image", user_id, ensure_exists=True)
|
||||
avatars_dir = images_dir / "avatars"
|
||||
avatars_dir.mkdir(parents=True, exist_ok=True)
|
||||
output_path = avatars_dir / filename
|
||||
output_path = PODCAST_IMAGES_DIR / filename
|
||||
PODCAST_IMAGES_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with open(output_path, "wb") as f:
|
||||
f.write(image_result.image_bytes)
|
||||
@@ -326,14 +208,13 @@ async def analyze_podcast_idea(
|
||||
db=db,
|
||||
user_id=user_id,
|
||||
asset_type="image",
|
||||
source_module="podcast_analysis",
|
||||
filename=filename,
|
||||
file_url=final_avatar_url,
|
||||
filename=filename,
|
||||
title=f"Presenter Avatar - {request.idea[:40]}",
|
||||
description=f"AI-generated podcast presenter for: {request.idea}",
|
||||
provider=image_result.provider,
|
||||
model=image_result.model,
|
||||
cost=0.0 # Cost tracked in generate_image
|
||||
cost=image_result.cost
|
||||
)
|
||||
logger.info(f"[Podcast Analyze] ✅ Generated and saved avatar to {final_avatar_url}")
|
||||
except Exception as e:
|
||||
@@ -438,13 +319,6 @@ Requirements:
|
||||
listener_cta = data.get("listener_cta") or ""
|
||||
research_queries = data.get("research_queries") or []
|
||||
exa_suggested_config = data.get("exa_suggested_config") or None
|
||||
estimate = estimate_podcast_cost(
|
||||
db=db,
|
||||
duration_minutes=request.duration,
|
||||
speakers=request.speakers,
|
||||
query_count=len(research_queries) if isinstance(research_queries, list) else 0,
|
||||
include_avatar_phase=podcast_mode != "audio_only",
|
||||
)
|
||||
|
||||
return PodcastAnalyzeResponse(
|
||||
audience=audience,
|
||||
@@ -461,7 +335,6 @@ Requirements:
|
||||
bible=bible_obj.model_dump() if bible_obj else None,
|
||||
avatar_url=final_avatar_url,
|
||||
avatar_prompt=final_avatar_prompt,
|
||||
estimate=estimate,
|
||||
)
|
||||
|
||||
|
||||
@@ -567,315 +440,3 @@ Requirements:
|
||||
logger.error(f"[Regenerate Queries] Failed for user {user_id}: {exc}")
|
||||
raise HTTPException(status_code=500, detail=f"Regenerate queries failed: {exc}")
|
||||
|
||||
|
||||
@router.post("/extract-url", response_model=ExtractUrlResponse)
|
||||
async def extract_url_content(
|
||||
request: ExtractUrlRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Extract content from a URL using Exa's get_contents API.
|
||||
|
||||
This allows users to paste a blog post or article URL as their podcast topic,
|
||||
and we'll extract the content to use as the podcast idea.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
from exa_py import Exa
|
||||
import os
|
||||
|
||||
api_key = os.getenv("EXA_API_KEY")
|
||||
if not api_key:
|
||||
raise HTTPException(status_code=500, detail="EXA_API_KEY not configured")
|
||||
|
||||
exa = Exa(api_key)
|
||||
|
||||
logger.warning(f"[ExtractUrl] Extracting content from: {request.url} for user {user_id}")
|
||||
|
||||
try:
|
||||
result = exa.get_contents(
|
||||
urls=[request.url],
|
||||
text=True,
|
||||
highlights=True,
|
||||
summary=True,
|
||||
subpages=2,
|
||||
)
|
||||
except Exception as exa_error:
|
||||
logger.error(f"[ExtractUrl] Exa call error: {exa_error}")
|
||||
return ExtractUrlResponse(
|
||||
success=False,
|
||||
url=request.url,
|
||||
error=f"Exa API error: {str(exa_error)}"
|
||||
)
|
||||
|
||||
# Check for errors using the correct attribute (statuses is array of status objects)
|
||||
if hasattr(result, 'statuses') and result.statuses:
|
||||
for status in result.statuses:
|
||||
if status.status == "error":
|
||||
logger.error(f"[ExtractUrl] Failed to extract {status.id}: {status.error.tag if hasattr(status.error, 'tag') else 'unknown'}")
|
||||
return ExtractUrlResponse(
|
||||
success=False,
|
||||
url=request.url,
|
||||
error=f"Failed to extract content: {status.error.tag if hasattr(status.error, 'tag') else 'unknown error'}"
|
||||
)
|
||||
|
||||
if not result.results:
|
||||
return ExtractUrlResponse(
|
||||
success=False,
|
||||
url=request.url,
|
||||
error="No content found at the provided URL"
|
||||
)
|
||||
|
||||
# Extract content - safe to access result now
|
||||
content = result.results[0]
|
||||
|
||||
# Extract all available fields from Exa response
|
||||
extracted_text = content.text or ""
|
||||
extracted_summary = getattr(content, 'summary', "") or ""
|
||||
extracted_title = content.title or ""
|
||||
|
||||
# Highlights - extract from content.highlights array if available
|
||||
highlights = []
|
||||
if hasattr(content, 'highlights') and content.highlights:
|
||||
highlights = [h for h in content.highlights if h]
|
||||
|
||||
# Additional fields from Exa response
|
||||
image = getattr(content, 'image', None)
|
||||
favicon = getattr(content, 'favicon', None)
|
||||
|
||||
# Subpages - extract with their own content
|
||||
subpages = []
|
||||
if hasattr(content, 'subpages') and content.subpages:
|
||||
for sp in content.subpages:
|
||||
subpages.append({
|
||||
'id': sp.get('id', ''),
|
||||
'title': sp.get('title', ''),
|
||||
'url': sp.get('url', ''),
|
||||
'summary': sp.get('summary', ''),
|
||||
'text': sp.get('text', '')[:500] if sp.get('text') else '', # First 500 chars
|
||||
})
|
||||
|
||||
logger.warning(f"[ExtractUrl] Successfully extracted {len(extracted_text)} chars from {request.url}")
|
||||
logger.warning(f"[ExtractUrl] title={extracted_title[:50]}, summary={extracted_summary[:50]}, highlights={len(highlights)}, subpages={len(subpages)}")
|
||||
|
||||
return ExtractUrlResponse(
|
||||
success=True,
|
||||
title=extracted_title,
|
||||
text=extracted_text,
|
||||
summary=extracted_summary,
|
||||
author=getattr(content, 'author', None),
|
||||
highlights=highlights,
|
||||
url=request.url,
|
||||
image=image,
|
||||
favicon=favicon,
|
||||
subpages=subpages,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/website-analysis", response_model=WebsiteAnalysisResponse)
|
||||
async def save_website_analysis(
|
||||
request: WebsiteAnalysisRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""Save the user's website analysis for reuse in future podcasts."""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
try:
|
||||
from services.user_data_service import user_data_service
|
||||
|
||||
website_data = {
|
||||
"website_url": request.website_url,
|
||||
"extracted_at": datetime.now().isoformat(),
|
||||
"exa_content": request.exa_content,
|
||||
"full_analysis": None,
|
||||
"analysis_status": "pending",
|
||||
}
|
||||
|
||||
success = user_data_service.save_user_data(
|
||||
user_id=user_id,
|
||||
data_key="website_analysis",
|
||||
data_value=website_data,
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.warning(f"[WebsiteAnalysis] Saved analysis for user {user_id}: {request.website_url}")
|
||||
return WebsiteAnalysisResponse(
|
||||
success=True,
|
||||
website_url=request.website_url,
|
||||
message="Website analysis saved successfully",
|
||||
)
|
||||
else:
|
||||
return WebsiteAnalysisResponse(
|
||||
success=False,
|
||||
error="Failed to save website analysis",
|
||||
)
|
||||
|
||||
except Exception as exc:
|
||||
logger.error(f"[WebsiteAnalysis] Failed to save for user {user_id}: {exc}")
|
||||
return WebsiteAnalysisResponse(
|
||||
success=False,
|
||||
error=f"Failed to save: {str(exc)}"
|
||||
)
|
||||
|
||||
|
||||
@router.get("/website-extraction")
|
||||
async def get_saved_website_extraction(request: Request = None):
|
||||
"""Get previously saved website extraction data for this user."""
|
||||
try:
|
||||
# Safely get current_user from Depends
|
||||
if request is None or not hasattr(request, 'state'):
|
||||
logger.warning("[WebsiteExtraction] No request or state - user not authenticated")
|
||||
return {"success": False, "data": None, "error": "Not authenticated"}
|
||||
|
||||
current_user = getattr(request.state, 'user', None)
|
||||
if not current_user:
|
||||
logger.warning("[WebsiteExtraction] No user in request state")
|
||||
return {"success": False, "data": None, "error": "Not authenticated"}
|
||||
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
from services.user_data_service import UserDataService
|
||||
from services.database import get_db
|
||||
db = next(get_db())
|
||||
|
||||
user_service = UserDataService(db)
|
||||
extraction = user_service.get_website_extraction(user_id)
|
||||
|
||||
if extraction:
|
||||
logger.info(f"[WebsiteExtraction] Found saved data for user {user_id}")
|
||||
return {
|
||||
"success": True,
|
||||
"data": extraction
|
||||
}
|
||||
else:
|
||||
logger.info(f"[WebsiteExtraction] No saved data for user {user_id}")
|
||||
return {
|
||||
"success": False,
|
||||
"data": None
|
||||
}
|
||||
|
||||
except Exception as exc:
|
||||
logger.error(f"[WebsiteExtraction] Failed for user: {exc}", exc_info=True)
|
||||
return {
|
||||
"success": False,
|
||||
"error": str(exc)
|
||||
}
|
||||
|
||||
|
||||
@router.post("/website-extraction")
|
||||
async def save_website_extraction(
|
||||
extraction: Dict[str, Any],
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""Save website extraction data for future use."""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
try:
|
||||
from services.user_data_service import UserDataService
|
||||
from services.database import get_db
|
||||
db = next(get_db())
|
||||
|
||||
user_service = UserDataService(db)
|
||||
success = user_service.save_website_extraction(user_id, extraction)
|
||||
|
||||
if success:
|
||||
logger.info(f"[WebsiteExtraction] Saved for user {user_id}")
|
||||
return {
|
||||
"success": True,
|
||||
"message": "Website extraction saved"
|
||||
}
|
||||
else:
|
||||
return {
|
||||
"success": False,
|
||||
"error": "Failed to save"
|
||||
}
|
||||
|
||||
except Exception as exc:
|
||||
logger.error(f"[WebsiteExtraction] Save failed: {exc}")
|
||||
return {
|
||||
"success": False,
|
||||
"error": str(exc)
|
||||
}
|
||||
|
||||
|
||||
@router.post("/project/{project_id}/topic-context")
|
||||
async def save_topic_context(
|
||||
project_id: str,
|
||||
topic_context: Dict[str, Any],
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""Save topic context (category research) to a podcast project."""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
try:
|
||||
from services.database import get_db
|
||||
from models.podcast_models import PodcastProject
|
||||
|
||||
db = next(get_db())
|
||||
|
||||
# Find the project
|
||||
project = db.query(PodcastProject).filter(
|
||||
PodcastProject.project_id == project_id,
|
||||
PodcastProject.user_id == user_id
|
||||
).first()
|
||||
|
||||
if not project:
|
||||
return {
|
||||
"success": False,
|
||||
"error": "Project not found"
|
||||
}
|
||||
|
||||
# Update topic context
|
||||
project.topic_context = topic_context
|
||||
db.commit()
|
||||
|
||||
logger.info(f"[TopicContext] Saved for project {project_id}")
|
||||
return {
|
||||
"success": True,
|
||||
"message": "Topic context saved"
|
||||
}
|
||||
|
||||
except Exception as exc:
|
||||
logger.error(f"[TopicContext] Save failed: {exc}")
|
||||
return {
|
||||
"success": False,
|
||||
"error": str(exc)
|
||||
}
|
||||
|
||||
|
||||
@router.get("/project/{project_id}/topic-context")
|
||||
async def get_topic_context(
|
||||
project_id: str,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""Get topic context from a podcast project."""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
try:
|
||||
from services.database import get_db
|
||||
from models.podcast_models import PodcastProject
|
||||
|
||||
db = next(get_db())
|
||||
|
||||
project = db.query(PodcastProject).filter(
|
||||
PodcastProject.project_id == project_id,
|
||||
PodcastProject.user_id == user_id
|
||||
).first()
|
||||
|
||||
if not project:
|
||||
return {
|
||||
"success": False,
|
||||
"error": "Project not found"
|
||||
}
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"data": project.topic_context
|
||||
}
|
||||
|
||||
except Exception as exc:
|
||||
logger.error(f"[TopicContext] Get failed: {exc}")
|
||||
return {
|
||||
"success": False,
|
||||
"error": str(exc)
|
||||
}
|
||||
|
||||
@@ -12,15 +12,7 @@ from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
import tempfile
|
||||
import uuid
|
||||
import hashlib
|
||||
import time
|
||||
import shutil
|
||||
import requests
|
||||
import asyncio
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
import asyncio
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
from services.database import get_db
|
||||
from middleware.auth_middleware import get_current_user, get_current_user_with_query_token
|
||||
@@ -39,124 +31,6 @@ from ..models import (
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# Thread pool for CPU/IO-intensive voice clone operations
|
||||
_audio_executor = ThreadPoolExecutor(max_workers=2, thread_name_prefix="podcast_audio")
|
||||
|
||||
# In-memory LRU cache for voice samples (per user) to avoid re-downloading
|
||||
_voice_sample_cache: dict[str, tuple[float, bytes]] = {}
|
||||
_VOICE_SAMPLE_CACHE_TTL = 1800 # 30 minutes
|
||||
|
||||
|
||||
def _get_cached_voice_sample(cache_key: str) -> Optional[bytes]:
|
||||
"""Get voice sample bytes from in-memory cache if fresh."""
|
||||
if cache_key in _voice_sample_cache:
|
||||
ts, data = _voice_sample_cache[cache_key]
|
||||
if time.time() - ts < _VOICE_SAMPLE_CACHE_TTL:
|
||||
logger.debug(f"[Podcast] Voice sample cache hit for {cache_key[:16]}...")
|
||||
return data
|
||||
del _voice_sample_cache[cache_key]
|
||||
return None
|
||||
|
||||
|
||||
def _cache_voice_sample(cache_key: str, data: bytes) -> None:
|
||||
"""Store voice sample bytes in in-memory cache."""
|
||||
# Evict oldest entries if cache grows too large
|
||||
if len(_voice_sample_cache) > 50:
|
||||
oldest_key = min(_voice_sample_cache, key=lambda k: _voice_sample_cache[k][0])
|
||||
del _voice_sample_cache[oldest_key]
|
||||
_voice_sample_cache[cache_key] = (time.time(), data)
|
||||
|
||||
|
||||
def _get_latest_voice_sample_url(user_id: str, db) -> Optional[str]:
|
||||
"""Get the latest voice sample URL for a user from their voice clone assets."""
|
||||
try:
|
||||
from models.content_asset_models import ContentAsset, AssetType, AssetSource
|
||||
from sqlalchemy import desc
|
||||
|
||||
asset = db.query(ContentAsset).filter(
|
||||
ContentAsset.user_id == user_id,
|
||||
ContentAsset.asset_type == AssetType.AUDIO,
|
||||
ContentAsset.source_module == AssetSource.VOICE_CLONER,
|
||||
).order_by(desc(ContentAsset.created_at)).first()
|
||||
|
||||
if asset and asset.file_url:
|
||||
logger.info(f"[Podcast] Found voice sample for user {user_id}: {asset.file_url}")
|
||||
return asset.file_url
|
||||
|
||||
logger.warning(f"[Podcast] No voice sample asset found for user {user_id}")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"[Podcast] Error fetching voice sample URL: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _fetch_voice_sample(voice_sample_url: str, user_id: str) -> Optional[bytes]:
|
||||
"""Fetch voice sample audio bytes from URL, with caching."""
|
||||
cache_key = hashlib.md5(f"{user_id}:{voice_sample_url}".encode()).hexdigest()
|
||||
|
||||
# Check in-memory cache first
|
||||
cached = _get_cached_voice_sample(cache_key)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
try:
|
||||
from utils.media_utils import resolve_media_path
|
||||
|
||||
# Try resolving as a local workspace path first (fastest)
|
||||
if "/api/assets/" in voice_sample_url:
|
||||
# Resolve user workspace path directly
|
||||
sanitized_uid = "".join(c for c in user_id if c.isalnum() or c in ("-", "_"))
|
||||
from api.podcast.constants import ROOT_DIR
|
||||
parts = voice_sample_url.split("/")
|
||||
# Expected: /api/assets/{user_id}/voice_samples/{filename}
|
||||
try:
|
||||
idx = parts.index("voice_samples")
|
||||
filename = parts[idx + 1].split("?")[0]
|
||||
local_path = ROOT_DIR / "workspace" / f"workspace_{sanitized_uid}" / "assets" / "voice_samples" / filename
|
||||
if local_path.exists():
|
||||
data = local_path.read_bytes()
|
||||
_cache_voice_sample(cache_key, data)
|
||||
logger.info(f"[Podcast] Voice sample loaded from workspace: {local_path}")
|
||||
return data
|
||||
except (ValueError, IndexError):
|
||||
pass
|
||||
|
||||
# Fall back to media utils resolver
|
||||
local_path = resolve_media_path(voice_sample_url)
|
||||
if local_path and local_path.exists():
|
||||
data = local_path.read_bytes()
|
||||
_cache_voice_sample(cache_key, data)
|
||||
return data
|
||||
|
||||
# Try resolving as a podcast audio file
|
||||
if "/api/podcast/audio/" in voice_sample_url:
|
||||
filename = voice_sample_url.split("/api/podcast/audio/")[-1].split("?")[0]
|
||||
try:
|
||||
audio_dir = get_podcast_media_dir("audio", user_id)
|
||||
local_path = audio_dir / filename
|
||||
if local_path.exists():
|
||||
data = local_path.read_bytes()
|
||||
_cache_voice_sample(cache_key, data)
|
||||
return data
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Try direct HTTP fetch as fallback
|
||||
if voice_sample_url.startswith("http"):
|
||||
logger.info(f"[Podcast] Fetching voice sample via HTTP: {voice_sample_url[:80]}...")
|
||||
resp = requests.get(voice_sample_url, timeout=30)
|
||||
if resp.status_code == 200:
|
||||
data = resp.content
|
||||
_cache_voice_sample(cache_key, data)
|
||||
logger.info(f"[Podcast] Voice sample fetched via HTTP ({len(data)} bytes)")
|
||||
return data
|
||||
|
||||
logger.warning(f"[Podcast] Could not fetch voice sample from: {voice_sample_url}")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"[Podcast] Error fetching voice sample: {e}")
|
||||
return None
|
||||
|
||||
|
||||
@router.post("/audio/upload")
|
||||
async def upload_podcast_audio(
|
||||
@@ -251,190 +125,36 @@ async def generate_podcast_audio(
|
||||
raise HTTPException(status_code=400, detail="Text is required")
|
||||
|
||||
try:
|
||||
# Determine if we should use voice clone path
|
||||
# Voice clone is used when: explicitly requested, OR when voice_id/custom_voice_id indicates a clone
|
||||
# (cloned voice IDs start with "vc_" or match the placeholder "MY_VOICE_CLONE")
|
||||
_vid = request.voice_id or ""
|
||||
_cvid = request.custom_voice_id or ""
|
||||
is_voice_clone = request.use_voice_clone or (
|
||||
_cvid.startswith("vc_") or _cvid == "MY_VOICE_CLONE"
|
||||
) or (
|
||||
_vid.startswith("vc_") or _vid == "MY_VOICE_CLONE"
|
||||
audio_service = get_podcast_audio_service(user_id)
|
||||
logger.warning(f"[Podcast] Generating audio with service dir: {audio_service.output_dir}")
|
||||
result: StoryAudioResult = audio_service.generate_ai_audio(
|
||||
scene_number=0,
|
||||
scene_title=request.scene_title,
|
||||
text=request.text.strip(),
|
||||
user_id=user_id,
|
||||
voice_id=request.voice_id or "Wise_Woman",
|
||||
custom_voice_id=request.custom_voice_id,
|
||||
speed=request.speed or 1.0, # Normal speed (was 0.9, but too slow - causing duration issues)
|
||||
volume=request.volume or 1.0,
|
||||
pitch=request.pitch or 0.0, # Normal pitch (0.0 = neutral)
|
||||
emotion=request.emotion or "neutral",
|
||||
english_normalization=request.english_normalization or False,
|
||||
sample_rate=request.sample_rate,
|
||||
bitrate=request.bitrate,
|
||||
channel=request.channel,
|
||||
format=request.format,
|
||||
language_boost=request.language_boost,
|
||||
enable_sync_mode=request.enable_sync_mode,
|
||||
)
|
||||
|
||||
# If voice_id is a clone ID, normalize it to use Wise_Woman for TTS fallback
|
||||
effective_voice_id = _vid if not (_vid.startswith("vc_") or _vid == "MY_VOICE_CLONE") else "Wise_Woman"
|
||||
|
||||
logger.warning(f"[Podcast] Audio request: use_voice_clone={request.use_voice_clone}, voice_id={request.voice_id}, custom_voice_id={request.custom_voice_id}, is_voice_clone={is_voice_clone}, voice_sample_url={request.voice_sample_url}, voice_clone_engine={request.voice_clone_engine}")
|
||||
|
||||
# Voice clone path: use user's voice sample with scene text as reference
|
||||
if is_voice_clone:
|
||||
# If no voice_sample_url provided, try to fetch it from the user's latest voice clone
|
||||
voice_sample_url = request.voice_sample_url
|
||||
if not voice_sample_url:
|
||||
try:
|
||||
voice_sample_url = _get_latest_voice_sample_url(user_id, db)
|
||||
logger.warning(f"[Podcast] DB fallback voice sample URL for user {user_id}: {voice_sample_url}")
|
||||
except Exception as e:
|
||||
logger.warning(f"[Podcast] Could not fetch voice sample URL: {e}")
|
||||
|
||||
if voice_sample_url:
|
||||
from services.llm_providers.main_audio_generation import qwen3_voice_clone, cosyvoice_voice_clone
|
||||
from utils.media_utils import detect_audio_format
|
||||
|
||||
engine = (request.voice_clone_engine or "qwen3").lower()
|
||||
logger.warning(f"[Podcast] 🔊 Voice clone path: engine={engine}, scene='{request.scene_title}', voice_sample_url={voice_sample_url[:80]}...")
|
||||
|
||||
# Download voice sample from URL (with caching)
|
||||
logger.warning(f"[Podcast] Fetching voice sample from: {voice_sample_url}")
|
||||
try:
|
||||
voice_sample_bytes = _fetch_voice_sample(voice_sample_url, user_id)
|
||||
except Exception as fetch_err:
|
||||
logger.error(f"[Podcast] ❌ Failed to fetch voice sample: {fetch_err}", exc_info=True)
|
||||
raise HTTPException(status_code=400, detail=f"Could not fetch voice sample: {str(fetch_err)}")
|
||||
logger.warning(f"[Podcast] Voice sample fetch result: {len(voice_sample_bytes) if voice_sample_bytes else 0} bytes")
|
||||
if not voice_sample_bytes:
|
||||
raise HTTPException(status_code=400, detail=f"Could not fetch voice sample from {voice_sample_url}")
|
||||
|
||||
# Detect actual audio format from bytes (may differ from file extension)
|
||||
detected_fmt, detected_mime = detect_audio_format(voice_sample_bytes)
|
||||
logger.warning(f"[Podcast] 🔊 Detected voice sample format: {detected_fmt} ({detected_mime}), {len(voice_sample_bytes)} bytes")
|
||||
voice_mime_type = detected_mime or "audio/wav"
|
||||
|
||||
scene_text = request.text.strip()
|
||||
if len(scene_text) > 4000:
|
||||
scene_text = scene_text[:4000]
|
||||
|
||||
# Run voice clone in thread pool to avoid blocking the event loop
|
||||
loop = asyncio.get_event_loop()
|
||||
|
||||
try:
|
||||
if engine == "minimax":
|
||||
from services.llm_providers.main_audio_generation import clone_voice
|
||||
import random
|
||||
import string
|
||||
random_suffix = ''.join(random.choices(string.ascii_letters + string.digits, k=8))
|
||||
custom_vid = request.custom_voice_id or f"vc_{random_suffix}"
|
||||
|
||||
result_obj = await loop.run_in_executor(
|
||||
_audio_executor,
|
||||
lambda cv=custom_vid: clone_voice(
|
||||
audio_bytes=voice_sample_bytes,
|
||||
custom_voice_id=cv,
|
||||
text=scene_text,
|
||||
user_id=user_id,
|
||||
),
|
||||
)
|
||||
audio_bytes = result_obj.preview_audio_bytes
|
||||
provider = "minimax"
|
||||
model = "minimax/voice-clone"
|
||||
elif engine == "cosyvoice":
|
||||
result_obj = await loop.run_in_executor(
|
||||
_audio_executor,
|
||||
lambda: cosyvoice_voice_clone(
|
||||
audio_bytes=voice_sample_bytes,
|
||||
text=scene_text,
|
||||
user_id=user_id,
|
||||
audio_mime_type=voice_mime_type,
|
||||
),
|
||||
)
|
||||
audio_bytes = result_obj.preview_audio_bytes
|
||||
provider = "wavespeed-ai"
|
||||
model = "wavespeed-ai/cosyvoice-tts/voice-clone"
|
||||
else:
|
||||
result_obj = await loop.run_in_executor(
|
||||
_audio_executor,
|
||||
lambda: qwen3_voice_clone(
|
||||
audio_bytes=voice_sample_bytes,
|
||||
text=scene_text,
|
||||
user_id=user_id,
|
||||
audio_mime_type=voice_mime_type,
|
||||
),
|
||||
)
|
||||
audio_bytes = result_obj.preview_audio_bytes
|
||||
provider = "wavespeed-ai"
|
||||
model = "wavespeed-ai/qwen3-tts/voice-clone"
|
||||
|
||||
logger.warning(f"[Podcast] 🔊 Voice clone result: {len(audio_bytes) if audio_bytes else 0} bytes, provider={provider}")
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as clone_err:
|
||||
logger.error(f"[Podcast] ❌ Voice clone failed: {clone_err}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Voice clone generation failed: {str(clone_err)}")
|
||||
|
||||
# Save audio bytes to file
|
||||
audio_service = get_podcast_audio_service(user_id)
|
||||
audio_filename = f"scene_{request.scene_id}_{uuid.uuid4().hex[:8]}.mp3"
|
||||
audio_path = audio_service.output_dir / audio_filename
|
||||
|
||||
with open(audio_path, "wb") as f:
|
||||
f.write(audio_bytes)
|
||||
|
||||
file_size = len(audio_bytes)
|
||||
audio_url = f"/api/podcast/audio/{audio_filename}"
|
||||
cost = max(0.005, 0.005 * (len(scene_text) / 100.0))
|
||||
|
||||
result = {
|
||||
"audio_path": str(audio_path),
|
||||
"audio_filename": audio_filename,
|
||||
"audio_url": audio_url,
|
||||
"file_size": file_size,
|
||||
"provider": provider,
|
||||
"model": model,
|
||||
"cost": cost,
|
||||
"scene_number": 0,
|
||||
"scene_title": request.scene_title,
|
||||
}
|
||||
|
||||
else:
|
||||
# Standard TTS path - but NOT if custom_voice_id is a clone ID
|
||||
# Clone IDs (vc_*, MY_VOICE_CLONE) are not valid for minimax TTS
|
||||
if is_voice_clone:
|
||||
logger.warning(f"[Podcast] ⚠️ Voice clone detected but no voice sample available - falling back to standard TTS with voice_id={effective_voice_id}")
|
||||
effective_custom_voice_id = request.custom_voice_id
|
||||
if effective_custom_voice_id and (
|
||||
effective_custom_voice_id.startswith("vc_") or
|
||||
effective_custom_voice_id == "MY_VOICE_CLONE"
|
||||
):
|
||||
logger.warning(f"[Podcast] Ignoring clone ID '{effective_custom_voice_id}' in standard TTS path - no voice sample URL available")
|
||||
effective_custom_voice_id = None
|
||||
|
||||
audio_service = get_podcast_audio_service(user_id)
|
||||
logger.warning(f"[Podcast] Standard TTS path: voice_id={effective_voice_id}, custom_voice_id={effective_custom_voice_id}")
|
||||
result: StoryAudioResult = audio_service.generate_ai_audio(
|
||||
scene_number=0,
|
||||
scene_title=request.scene_title,
|
||||
text=request.text.strip(),
|
||||
user_id=user_id,
|
||||
voice_id=effective_voice_id,
|
||||
custom_voice_id=effective_custom_voice_id,
|
||||
speed=request.speed or 1.0, # Normal speed (was 0.9, but too slow - causing duration issues)
|
||||
volume=request.volume or 1.0,
|
||||
pitch=request.pitch or 0.0, # Normal pitch (0.0 = neutral)
|
||||
emotion=request.emotion or "neutral",
|
||||
english_normalization=request.english_normalization or False,
|
||||
sample_rate=request.sample_rate,
|
||||
bitrate=request.bitrate,
|
||||
channel=request.channel,
|
||||
format=request.format,
|
||||
language_boost=request.language_boost,
|
||||
enable_sync_mode=request.enable_sync_mode,
|
||||
)
|
||||
|
||||
# Override URL to use podcast endpoint instead of story endpoint
|
||||
if result.get("audio_url") and "/api/story/audio/" in result.get("audio_url", ""):
|
||||
audio_filename = result.get("audio_filename", "")
|
||||
result["audio_url"] = f"/api/podcast/audio/{audio_filename}"
|
||||
|
||||
logger.warning(f"[Podcast] Audio generated - path: {result.get('audio_path')}, url: {result.get('audio_url')}")
|
||||
except HTTPException:
|
||||
raise
|
||||
# Override URL to use podcast endpoint instead of story endpoint
|
||||
if result.get("audio_url") and "/api/story/audio/" in result.get("audio_url", ""):
|
||||
audio_filename = result.get("audio_filename", "")
|
||||
result["audio_url"] = f"/api/podcast/audio/{audio_filename}"
|
||||
|
||||
logger.warning(f"[Podcast] Audio generated - path: {result.get('audio_path')}, url: {result.get('audio_url')}")
|
||||
except Exception as exc:
|
||||
exc_type = type(exc).__name__
|
||||
exc_msg = str(exc)[:500]
|
||||
logger.error(f"[Podcast] Audio generation failed ({exc_type}): {exc_msg}")
|
||||
logger.error(f"[Podcast] Audio generation traceback:", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Audio generation failed ({exc_type}): {exc_msg}")
|
||||
raise HTTPException(status_code=500, detail=f"Audio generation failed: {exc}")
|
||||
|
||||
# Save to asset library (podcast module)
|
||||
try:
|
||||
@@ -671,12 +391,9 @@ async def serve_podcast_audio(
|
||||
raise HTTPException(status_code=400, detail="Invalid filename")
|
||||
|
||||
user_id = require_authenticated_user(current_user)
|
||||
logger.info(f"[Podcast] serve_podcast_audio: filename={filename}, user_id={user_id}")
|
||||
|
||||
logger.warning(f"[Podcast] serve_podcast_audio called: user_id={user_id}, filename={filename}")
|
||||
audio_path = _resolve_podcast_media_file(filename, "audio", user_id)
|
||||
logger.info(f"[Podcast] Audio resolved path: {audio_path}, exists={audio_path.exists()}")
|
||||
audio_path = _resolve_podcast_media_file(filename, "audio", user_id)
|
||||
logger.debug(f"[Podcast] Resolved audio path: {audio_path}")
|
||||
logger.warning(f"[Podcast] Resolved audio path: {audio_path}")
|
||||
|
||||
return FileResponse(audio_path, media_type="audio/mpeg")
|
||||
|
||||
|
||||
@@ -12,39 +12,22 @@ from pathlib import Path
|
||||
import uuid
|
||||
import hashlib
|
||||
|
||||
from services.database import get_db, get_session_for_user
|
||||
from services.database import get_db
|
||||
from middleware.auth_middleware import get_current_user, get_current_user_with_query_token
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
from services.llm_providers.main_image_generation import generate_image
|
||||
from services.llm_providers.main_image_editing import edit_image
|
||||
from utils.asset_tracker import save_asset_to_library
|
||||
from loguru import logger
|
||||
from ..constants import get_podcast_media_dir, PODCAST_AVATARS_SUBDIR
|
||||
from ..constants import PODCAST_IMAGES_DIR
|
||||
from ..presenter_personas import choose_persona_id, get_persona
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# Avatar subdirectory
|
||||
AVATAR_SUBDIR = PODCAST_AVATARS_SUBDIR
|
||||
|
||||
|
||||
async def _get_db_or_none(current_user: Dict[str, Any]):
|
||||
"""Try to get a database session, returning None on failure (non-fatal for uploads)."""
|
||||
try:
|
||||
user_id = current_user.get('id') or current_user.get('clerk_user_id')
|
||||
if not user_id:
|
||||
return None
|
||||
return get_session_for_user(user_id)
|
||||
except Exception as e:
|
||||
logger.warning(f"[Podcast] DB session unavailable (non-fatal): {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _get_podcast_avatars_dir(user_id: str) -> Path:
|
||||
"""Get podcast avatars directory for a user (workspace-aware)."""
|
||||
avatars_dir = get_podcast_media_dir("image", user_id, ensure_exists=True) / AVATAR_SUBDIR
|
||||
avatars_dir.mkdir(parents=True, exist_ok=True)
|
||||
return avatars_dir
|
||||
AVATAR_SUBDIR = "avatars"
|
||||
PODCAST_AVATARS_DIR = PODCAST_IMAGES_DIR / AVATAR_SUBDIR
|
||||
PODCAST_AVATARS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
@router.post("/avatar/upload")
|
||||
@@ -58,16 +41,8 @@ async def upload_podcast_avatar(
|
||||
Upload a presenter avatar image for a podcast project.
|
||||
Returns the avatar URL for use in scene image generation.
|
||||
"""
|
||||
try:
|
||||
user_id = require_authenticated_user(current_user)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"[Podcast] Avatar upload auth failed: {e}", exc_info=True)
|
||||
raise HTTPException(status_code=401, detail="Authentication failed")
|
||||
|
||||
logger.info(f"[Podcast] Avatar upload request - user_id={user_id}, project_id={project_id}, content_type={file.content_type}")
|
||||
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Validate file type
|
||||
if not file.content_type or not file.content_type.startswith('image/'):
|
||||
raise HTTPException(status_code=400, detail="File must be an image")
|
||||
@@ -82,21 +57,19 @@ async def upload_podcast_avatar(
|
||||
file_ext = Path(file.filename).suffix or '.png'
|
||||
unique_id = str(uuid.uuid4())[:8]
|
||||
avatar_filename = f"avatar_{project_id or 'temp'}_{unique_id}{file_ext}"
|
||||
avatars_dir = _get_podcast_avatars_dir(user_id)
|
||||
logger.info(f"[Podcast] Saving avatar to: {avatars_dir / avatar_filename}")
|
||||
avatar_path = avatars_dir / avatar_filename
|
||||
avatar_path = PODCAST_AVATARS_DIR / avatar_filename
|
||||
|
||||
# Save file
|
||||
with open(avatar_path, "wb") as f:
|
||||
f.write(file_content)
|
||||
|
||||
logger.info(f"[Podcast] Avatar uploaded successfully: {avatar_path}")
|
||||
logger.info(f"[Podcast] Avatar uploaded: {avatar_path}")
|
||||
|
||||
# Create avatar URL
|
||||
avatar_url = f"/api/podcast/images/{AVATAR_SUBDIR}/{avatar_filename}"
|
||||
|
||||
# Save to asset library if project_id provided and DB session available
|
||||
if project_id and db:
|
||||
# Save to asset library if project_id provided
|
||||
if project_id:
|
||||
try:
|
||||
save_asset_to_library(
|
||||
db=db,
|
||||
@@ -118,17 +91,13 @@ async def upload_podcast_avatar(
|
||||
},
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"[Podcast] Failed to save avatar asset (non-fatal): {e}")
|
||||
elif project_id and not db:
|
||||
logger.warning(f"[Podcast] DB session unavailable, skipping asset library save for avatar")
|
||||
logger.warning(f"[Podcast] Failed to save avatar asset: {e}")
|
||||
|
||||
return {
|
||||
"avatar_url": avatar_url,
|
||||
"avatar_filename": avatar_filename,
|
||||
"message": "Avatar uploaded successfully"
|
||||
}
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as exc:
|
||||
logger.error(f"[Podcast] Avatar upload failed: {exc}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Avatar upload failed: {str(exc)}")
|
||||
@@ -145,18 +114,12 @@ async def make_avatar_presentable(
|
||||
Transform an uploaded avatar image into a podcast-appropriate presenter.
|
||||
Uses AI image editing to convert the uploaded photo into a professional podcast presenter.
|
||||
"""
|
||||
# CRITICAL: Log at the very start before any logic
|
||||
logger.info(f"[Podcast] ===== MAKE PRESENTABLE ENDPOINT START =====")
|
||||
|
||||
user_id = require_authenticated_user(current_user)
|
||||
logger.info(f"[Podcast] Make presentable request received - user_id={user_id}, avatar_url={avatar_url}, project_id={project_id}")
|
||||
|
||||
try:
|
||||
# Load the uploaded avatar image
|
||||
from ..utils import load_podcast_image_bytes
|
||||
logger.info(f"[Podcast] Loading avatar image from {avatar_url}")
|
||||
avatar_bytes = load_podcast_image_bytes(avatar_url, user_id=user_id)
|
||||
logger.info(f"[Podcast] Avatar loaded successfully - size={len(avatar_bytes)} bytes")
|
||||
avatar_bytes = load_podcast_image_bytes(avatar_url)
|
||||
|
||||
logger.info(f"[Podcast] Transforming avatar to podcast presenter for project {project_id}")
|
||||
|
||||
@@ -178,24 +141,17 @@ async def make_avatar_presentable(
|
||||
"model": None, # Use default model
|
||||
}
|
||||
|
||||
logger.info(f"[Podcast] Calling edit_image with user_id={user_id}")
|
||||
try:
|
||||
result = edit_image(
|
||||
input_image_bytes=avatar_bytes,
|
||||
prompt=transformation_prompt,
|
||||
options=image_options,
|
||||
user_id=user_id
|
||||
)
|
||||
logger.info(f"[Podcast] edit_image completed successfully - provider={result.provider}, model={result.model}")
|
||||
except Exception as edit_err:
|
||||
logger.error(f"[Podcast] edit_image failed: {edit_err}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Image editing failed: {str(edit_err)}")
|
||||
result = edit_image(
|
||||
input_image_bytes=avatar_bytes,
|
||||
prompt=transformation_prompt,
|
||||
options=image_options,
|
||||
user_id=user_id
|
||||
)
|
||||
|
||||
# Save transformed avatar
|
||||
unique_id = str(uuid.uuid4())[:8]
|
||||
transformed_filename = f"presenter_transformed_{project_id or 'temp'}_{unique_id}.png"
|
||||
avatars_dir = _get_podcast_avatars_dir(user_id)
|
||||
transformed_path = avatars_dir / transformed_filename
|
||||
transformed_path = PODCAST_AVATARS_DIR / transformed_filename
|
||||
|
||||
with open(transformed_path, "wb") as f:
|
||||
f.write(result.image_bytes)
|
||||
@@ -238,16 +194,6 @@ async def make_avatar_presentable(
|
||||
"avatar_filename": transformed_filename,
|
||||
"message": "Avatar transformed into podcast presenter successfully"
|
||||
}
|
||||
except HTTPException:
|
||||
# Re-raise HTTP exceptions as-is
|
||||
raise
|
||||
except RuntimeError as rt_err:
|
||||
# Handle missing API keys or configuration errors
|
||||
logger.error(f"[Podcast] Avatar transformation configuration error: {rt_err}")
|
||||
raise HTTPException(
|
||||
status_code=503, # Service Unavailable
|
||||
detail=f"Image editing service not configured: {str(rt_err)}. Please contact support."
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(f"[Podcast] Avatar transformation failed: {exc}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Avatar transformation failed: {str(exc)}")
|
||||
@@ -377,8 +323,7 @@ async def generate_podcast_presenters(
|
||||
# Save avatar
|
||||
unique_id = str(uuid.uuid4())[:8]
|
||||
avatar_filename = f"presenter_{project_id or 'temp'}_{i+1}_{unique_id}.png"
|
||||
avatars_dir = _get_podcast_avatars_dir(user_id)
|
||||
avatar_path = avatars_dir / avatar_filename
|
||||
avatar_path = PODCAST_AVATARS_DIR / avatar_filename
|
||||
|
||||
with open(avatar_path, "wb") as f:
|
||||
f.write(result.image_bytes)
|
||||
|
||||
@@ -1,398 +0,0 @@
|
||||
"""
|
||||
B-Roll Handlers
|
||||
|
||||
API endpoints for B-roll chart preview and video generation.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, BackgroundTasks
|
||||
from fastapi.responses import FileResponse
|
||||
from typing import Dict, Any, Optional, List
|
||||
from pydantic import BaseModel, Field
|
||||
from pathlib import Path
|
||||
import uuid
|
||||
|
||||
from middleware.auth_middleware import get_current_user, get_current_user_with_query_token
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
from api.story_writer.task_manager import task_manager
|
||||
from api.podcast.utils import _resolve_podcast_media_file
|
||||
from services.podcast.broll_service import get_broll_service
|
||||
from utils.media_utils import resolve_media_path
|
||||
from loguru import logger
|
||||
|
||||
|
||||
router = APIRouter(prefix="/broll", tags=["B-Roll"])
|
||||
|
||||
|
||||
def _resolve_broll_background_image_path(background_image_url: str) -> str:
|
||||
"""Resolve background image URL/path to a local file path."""
|
||||
resolved = resolve_media_path(background_image_url)
|
||||
if not resolved:
|
||||
raise HTTPException(status_code=404, detail=f"Background image not found: {background_image_url}")
|
||||
return str(resolved)
|
||||
|
||||
|
||||
def _resolve_broll_avatar_video_path(avatar_video_url: Optional[str], user_id: str) -> Optional[str]:
|
||||
"""Resolve optional avatar video URL/path to a local file path."""
|
||||
if not avatar_video_url:
|
||||
return None
|
||||
|
||||
parsed = urlparse(avatar_video_url)
|
||||
path = parsed.path if parsed.scheme else avatar_video_url
|
||||
|
||||
if "/api/podcast/videos/" in path:
|
||||
filename = path.split("/api/podcast/videos/", 1)[1].split("?", 1)[0].strip()
|
||||
if not filename:
|
||||
raise HTTPException(status_code=400, detail="Invalid avatar video URL")
|
||||
return str(_resolve_podcast_media_file(filename, "video", user_id))
|
||||
|
||||
local_path = Path(path).expanduser().resolve()
|
||||
if local_path.exists() and local_path.is_file():
|
||||
return str(local_path)
|
||||
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=(
|
||||
"Unsupported avatar video URL format. "
|
||||
"Use /api/podcast/videos/{filename} or a valid local file path."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _execute_broll_scene_task(
|
||||
task_id: str,
|
||||
*,
|
||||
scene_id: str,
|
||||
key_insight: str,
|
||||
supporting_stat: str,
|
||||
chart_data: Optional[Dict[str, Any]],
|
||||
visual_cue: str,
|
||||
duration: float,
|
||||
background_img_path: str,
|
||||
avatar_video_path: Optional[str],
|
||||
):
|
||||
"""Background task for rendering a B-roll scene."""
|
||||
try:
|
||||
task_manager.update_task_status(
|
||||
task_id,
|
||||
"processing",
|
||||
progress=10.0,
|
||||
message="Starting B-roll scene render...",
|
||||
)
|
||||
|
||||
broll_service = get_broll_service()
|
||||
task_manager.update_task_status(
|
||||
task_id,
|
||||
"processing",
|
||||
progress=35.0,
|
||||
message="Composing scene layers and overlays...",
|
||||
)
|
||||
|
||||
video_path = broll_service.generate_scene_broll(
|
||||
scene_id=scene_id,
|
||||
key_insight=key_insight,
|
||||
supporting_stat=supporting_stat,
|
||||
chart_data=chart_data,
|
||||
visual_cue=visual_cue,
|
||||
duration=duration,
|
||||
background_img_path=background_img_path,
|
||||
avatar_video_path=avatar_video_path,
|
||||
)
|
||||
|
||||
filename = Path(video_path).name
|
||||
video_url = f"/api/podcast/broll/final/{filename}"
|
||||
|
||||
task_manager.update_task_status(
|
||||
task_id,
|
||||
"completed",
|
||||
progress=100.0,
|
||||
message="B-roll scene render completed.",
|
||||
result={
|
||||
"scene_id": scene_id,
|
||||
"broll_video_path": video_path,
|
||||
"broll_video_url": video_url,
|
||||
},
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(f"[Broll] Task {task_id} failed: {exc}")
|
||||
task_manager.update_task_status(
|
||||
task_id,
|
||||
"failed",
|
||||
error=f"B-roll scene render failed: {str(exc)}",
|
||||
error_status=500,
|
||||
)
|
||||
|
||||
|
||||
class ChartPreviewRequest(BaseModel):
|
||||
"""Request model for chart preview generation."""
|
||||
chart_data: Dict[str, Any] = Field(..., description="Chart data (labels, before/after, etc.)")
|
||||
chart_type: str = Field(
|
||||
default="bar_comparison",
|
||||
description="bar_comparison | bar_horizontal | line_trend | pie | stacked_bar | bullet"
|
||||
)
|
||||
title: str = Field(default="", description="Chart title")
|
||||
subtitle: Optional[str] = Field(default="", description="Optional subtitle at bottom")
|
||||
|
||||
|
||||
class ChartPreviewResponse(BaseModel):
|
||||
"""Response for chart preview."""
|
||||
preview_url: str
|
||||
chart_id: str
|
||||
|
||||
|
||||
class BrollSceneRequest(BaseModel):
|
||||
"""Request for generating B-roll video for a scene."""
|
||||
scene_id: str
|
||||
key_insight: str
|
||||
supporting_stat: str
|
||||
chart_data: Optional[Dict[str, Any]] = None
|
||||
visual_cue: str = Field(default="bar_comparison", description="bar_comparison | bar_horizontal | line_trend | pie | stacked_bar | bullet_points | full_avatar")
|
||||
duration: float = Field(default=10.0, ge=3.0, le=60.0)
|
||||
background_image_url: str
|
||||
avatar_video_url: Optional[str] = None
|
||||
|
||||
|
||||
class BrollSceneResponse(BaseModel):
|
||||
"""Response for B-roll scene generation."""
|
||||
scene_id: str
|
||||
broll_video_url: str = ""
|
||||
broll_video_path: str = ""
|
||||
task_id: Optional[str] = None
|
||||
status: str = "completed"
|
||||
message: Optional[str] = None
|
||||
|
||||
|
||||
class BrollComposeRequest(BaseModel):
|
||||
"""Request for composing multiple B-roll videos."""
|
||||
scene_video_paths: List[str]
|
||||
output_filename: str = "final_broll.mp4"
|
||||
fade_dur: float = Field(default=0.5, ge=0.0, le=2.0)
|
||||
fps: int = Field(default=24, ge=12, le=60)
|
||||
|
||||
|
||||
class BrollComposeResponse(BaseModel):
|
||||
"""Response for B-roll composition."""
|
||||
final_video_url: str
|
||||
final_video_path: str
|
||||
|
||||
|
||||
@router.post("/preview/chart", response_model=ChartPreviewResponse)
|
||||
async def generate_chart_preview(
|
||||
request: ChartPreviewRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Generate a chart PNG preview (static image for Write phase).
|
||||
|
||||
This endpoint is called from the Write phase to show users chart previews
|
||||
before they commit to B-roll video generation.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Debug logging
|
||||
logger.warning(f"[Broll] Chart preview request: type={request.chart_type}, title={request.title}, chart_data keys={list(request.chart_data.keys())}, user_id={user_id}")
|
||||
|
||||
try:
|
||||
broll_service = get_broll_service(user_id=user_id)
|
||||
chart_id = uuid.uuid4().hex[:8]
|
||||
|
||||
preview_path = broll_service.generate_chart_preview(
|
||||
chart_data=request.chart_data,
|
||||
chart_type=request.chart_type,
|
||||
title=request.title,
|
||||
subtitle=request.subtitle or "",
|
||||
chart_id=chart_id,
|
||||
)
|
||||
|
||||
# If chart generation failed (empty path), return a placeholder instead of 500
|
||||
if not preview_path:
|
||||
# Return a fallback response so frontend doesn't crash
|
||||
logger.warning(f"[Broll] Chart preview skipped - invalid data for type: {request.chart_type}")
|
||||
return ChartPreviewResponse(
|
||||
preview_url="",
|
||||
chart_id=chart_id,
|
||||
)
|
||||
|
||||
preview_filename = Path(preview_path).name
|
||||
preview_url = f"/api/podcast/broll/preview/{chart_id}/{preview_filename}"
|
||||
|
||||
logger.warning(f"[Broll] Chart preview generated: chart_id={chart_id}, path={preview_path}, url={preview_url}")
|
||||
|
||||
return ChartPreviewResponse(
|
||||
preview_url=preview_url,
|
||||
chart_id=chart_id,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Broll] Chart preview generation failed: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Chart preview failed: {str(e)}")
|
||||
|
||||
|
||||
@router.post("/render/broll-scene", response_model=BrollSceneResponse)
|
||||
async def generate_broll_scene(
|
||||
request: BrollSceneRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Generate a B-roll video for a single scene.
|
||||
|
||||
This creates a programmatic video with:
|
||||
- Background image with Ken Burns effect
|
||||
- Chart overlay (if chart_data provided)
|
||||
- Avatar circle in corner (if avatar_video_url provided)
|
||||
- Insight card at bottom
|
||||
|
||||
Returns a task_id for polling since video generation can take time.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
try:
|
||||
# Validate visual_cue
|
||||
valid_cues = ["bar_comparison", "bar_chart_comparison", "bar_horizontal", "line_trend", "pie", "stacked_bar", "bullet_points", "full_avatar"]
|
||||
if request.visual_cue not in valid_cues:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid visual_cue. Must be one of: {valid_cues}"
|
||||
)
|
||||
|
||||
background_img_path = _resolve_broll_background_image_path(request.background_image_url)
|
||||
avatar_video_path = _resolve_broll_avatar_video_path(request.avatar_video_url, user_id)
|
||||
|
||||
logger.info(f"[Broll] B-roll scene request for scene: {request.scene_id}")
|
||||
|
||||
# Scene rendering can be expensive, so use task manager/background execution.
|
||||
task_id = task_manager.create_task(
|
||||
"podcast_broll_scene_generation",
|
||||
metadata={"owner_user_id": user_id, "scene_id": request.scene_id},
|
||||
)
|
||||
|
||||
background_tasks.add_task(
|
||||
_execute_broll_scene_task,
|
||||
task_id=task_id,
|
||||
scene_id=request.scene_id,
|
||||
key_insight=request.key_insight,
|
||||
supporting_stat=request.supporting_stat,
|
||||
chart_data=request.chart_data,
|
||||
visual_cue=request.visual_cue,
|
||||
duration=request.duration,
|
||||
background_img_path=background_img_path,
|
||||
avatar_video_path=avatar_video_path,
|
||||
)
|
||||
|
||||
return BrollSceneResponse(
|
||||
scene_id=request.scene_id,
|
||||
task_id=task_id,
|
||||
status="pending",
|
||||
message="B-roll scene render started. Poll /api/podcast/task/{task_id}/status for progress.",
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"[Broll] B-roll scene generation failed: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"B-roll generation failed: {str(e)}")
|
||||
|
||||
|
||||
@router.post("/render/broll-compose", response_model=BrollComposeResponse)
|
||||
async def compose_broll_videos(
|
||||
request: BrollComposeRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Compose multiple B-roll scene videos into a final video.
|
||||
|
||||
Applies crossfade transitions between scenes.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
try:
|
||||
broll_service = get_broll_service()
|
||||
|
||||
final_path = broll_service.compose_final_video(
|
||||
video_paths=request.scene_video_paths,
|
||||
output_filename=request.output_filename,
|
||||
fade_dur=request.fade_dur,
|
||||
fps=request.fps,
|
||||
)
|
||||
|
||||
final_filename = final_path.split('/')[-1]
|
||||
final_url = f"/api/podcast/broll/final/{final_filename}"
|
||||
|
||||
return BrollComposeResponse(
|
||||
final_video_url=final_url,
|
||||
final_video_path=final_path,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Broll] Video composition failed: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Video composition failed: {str(e)}")
|
||||
|
||||
|
||||
@router.get("/preview/{chart_id}/{filename}")
|
||||
async def serve_chart_preview(
|
||||
chart_id: str,
|
||||
filename: str,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user_with_query_token),
|
||||
):
|
||||
"""
|
||||
Serve chart preview PNG files.
|
||||
|
||||
Uses authentication via Authorization header or token query parameter,
|
||||
matching the pattern used by /api/podcast/images/ for browser <img> tags.
|
||||
"""
|
||||
from api.podcast.constants import get_podcast_media_dir
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Validate filename to prevent directory traversal
|
||||
if ".." in filename or "/" in filename or "\\" in filename:
|
||||
raise HTTPException(status_code=400, detail="Invalid filename")
|
||||
|
||||
logger.warning(f"[Broll] serve_chart_preview: chart_id={chart_id}, filename={filename}, user_id={user_id}")
|
||||
|
||||
charts_dir = get_podcast_media_dir("chart", user_id)
|
||||
file_path = charts_dir / filename
|
||||
|
||||
logger.warning(f"[Broll] serve_chart_preview: resolved path={file_path}, exists={file_path.exists()}")
|
||||
|
||||
if not file_path.exists():
|
||||
raise HTTPException(status_code=404, detail="Chart preview not found")
|
||||
|
||||
# Security: ensure resolved path is within charts_dir
|
||||
if not str(file_path.resolve()).startswith(str(charts_dir.resolve())):
|
||||
raise HTTPException(status_code=403, detail="Access denied")
|
||||
|
||||
return FileResponse(
|
||||
path=str(file_path),
|
||||
media_type="image/png",
|
||||
filename=filename,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/final/{filename}")
|
||||
async def serve_final_broll(
|
||||
filename: str,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""Serve final composed B-roll video files."""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
broll_service = get_broll_service()
|
||||
file_path = broll_service.output_dir / filename
|
||||
|
||||
if not file_path.exists():
|
||||
raise HTTPException(status_code=404, detail="Video not found")
|
||||
|
||||
return FileResponse(
|
||||
path=str(file_path),
|
||||
media_type="video/mp4",
|
||||
filename=filename,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/health")
|
||||
async def broll_health():
|
||||
"""Health check for B-roll service."""
|
||||
return {"status": "ok", "service": "broll"}
|
||||
@@ -17,7 +17,7 @@ from api.story_writer.utils.auth import require_authenticated_user
|
||||
from services.llm_providers.main_image_generation import generate_image, generate_character_image
|
||||
from utils.asset_tracker import save_asset_to_library
|
||||
from loguru import logger
|
||||
from ..constants import get_podcast_media_dir
|
||||
from ..constants import PODCAST_IMAGES_DIR
|
||||
from ..models import PodcastImageRequest, PodcastImageResponse
|
||||
|
||||
router = APIRouter()
|
||||
@@ -69,7 +69,7 @@ async def generate_podcast_scene_image(
|
||||
from ..utils import load_podcast_image_bytes
|
||||
try:
|
||||
logger.info(f"[Podcast] Attempting to load base avatar from: {request.base_avatar_url}")
|
||||
base_avatar_bytes = load_podcast_image_bytes(request.base_avatar_url, user_id=user_id)
|
||||
base_avatar_bytes = load_podcast_image_bytes(request.base_avatar_url)
|
||||
logger.info(f"[Podcast] ✅ Successfully loaded base avatar ({len(base_avatar_bytes)} bytes) for scene {request.scene_id}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Podcast] ❌ Failed to load base avatar from {request.base_avatar_url}: {e}", exc_info=True)
|
||||
@@ -377,14 +377,14 @@ async def generate_podcast_scene_image(
|
||||
user_id=user_id
|
||||
)
|
||||
|
||||
# Save image to podcast images directory (workspace-aware)
|
||||
images_dir = get_podcast_media_dir("image", user_id, ensure_exists=True)
|
||||
# Save image to podcast images directory
|
||||
PODCAST_IMAGES_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Generate filename
|
||||
clean_title = "".join(c if c.isalnum() or c in ('-', '_') else '_' for c in request.scene_title[:30])
|
||||
unique_id = str(uuid.uuid4())[:8]
|
||||
image_filename = f"scene_{request.scene_id}_{clean_title}_{unique_id}.png"
|
||||
image_path = images_dir / image_filename
|
||||
image_path = PODCAST_IMAGES_DIR / image_filename
|
||||
|
||||
# Save image
|
||||
with open(image_path, "wb") as f:
|
||||
@@ -470,17 +470,16 @@ async def serve_podcast_image(
|
||||
Query parameter is useful for HTML elements like <img> that cannot send custom headers.
|
||||
Supports subdirectories like avatars/
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
require_authenticated_user(current_user)
|
||||
|
||||
# Security check: ensure path doesn't contain path traversal or absolute paths
|
||||
if ".." in path or path.startswith("/"):
|
||||
raise HTTPException(status_code=400, detail="Invalid path")
|
||||
|
||||
images_dir = get_podcast_media_dir("image", user_id)
|
||||
image_path = (images_dir / path).resolve()
|
||||
image_path = (PODCAST_IMAGES_DIR / path).resolve()
|
||||
|
||||
# Security check: ensure resolved path is within images_dir
|
||||
if not str(image_path).startswith(str(images_dir)):
|
||||
# Security check: ensure resolved path is within PODCAST_IMAGES_DIR
|
||||
if not str(image_path).startswith(str(PODCAST_IMAGES_DIR)):
|
||||
raise HTTPException(status_code=403, detail="Access denied")
|
||||
|
||||
if not image_path.exists():
|
||||
|
||||
@@ -11,7 +11,6 @@ from typing import Optional, Dict, Any
|
||||
from services.database import get_db
|
||||
from middleware.auth_middleware import get_current_user
|
||||
from services.podcast_service import PodcastService
|
||||
from loguru import logger
|
||||
from ..models import (
|
||||
PodcastProjectResponse,
|
||||
CreateProjectRequest,
|
||||
@@ -107,57 +106,25 @@ async def update_project(
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""Update a podcast project state."""
|
||||
import time
|
||||
start_time = time.time()
|
||||
|
||||
try:
|
||||
user_id = current_user.get("user_id") or current_user.get("id")
|
||||
if not user_id:
|
||||
logger.error(f"[Podcast] update_project: No user_id found in current_user: {current_user}")
|
||||
raise HTTPException(status_code=401, detail="User ID not found")
|
||||
|
||||
# Get only field names being updated (not full data to avoid console flooding)
|
||||
request_dict = request.model_dump(exclude_none=True)
|
||||
updated_fields = list(request_dict.keys())
|
||||
|
||||
logger.warning(f"[Podcast] ===== UPDATE_PROJECT_START =====")
|
||||
logger.warning(f"[Podcast] project_id={project_id}, user_id={user_id}, fields={updated_fields}")
|
||||
|
||||
service = PodcastService(db)
|
||||
|
||||
# Check if project exists; if not, create it (upsert behavior for resilience)
|
||||
existing = service.get_project(user_id, project_id)
|
||||
if not existing:
|
||||
logger.warning(f"[Podcast] Project {project_id} not found for user {user_id}, creating new project with default values")
|
||||
# Try to create the project - this handles cases where create succeeded but wasn't found later
|
||||
# (can happen with user_id mismatch or after session refresh)
|
||||
try:
|
||||
project = service.create_project(
|
||||
user_id=user_id,
|
||||
project_id=project_id,
|
||||
idea="Untitled Podcast",
|
||||
status="scripting",
|
||||
duration=10,
|
||||
speakers=1,
|
||||
budget_cap=0.0,
|
||||
)
|
||||
except Exception as create_err:
|
||||
logger.error(f"[Podcast] Failed to create project {project_id}: {create_err}")
|
||||
raise HTTPException(status_code=404, detail=f"Project {project_id} not found and could not create: {create_err}")
|
||||
else:
|
||||
# Convert request to dict, excluding None values
|
||||
updates = request.model_dump(exclude_unset=True)
|
||||
project = service.update_project(user_id, project_id, **updates)
|
||||
# Convert request to dict, excluding None values
|
||||
updates = request.model_dump(exclude_unset=True)
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.warning(f"[Podcast] ===== UPDATE_PROJECT_END (took {duration_ms}ms) =====")
|
||||
project = service.update_project(user_id, project_id, **updates)
|
||||
|
||||
if not project:
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
|
||||
return PodcastProjectResponse.model_validate(project)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.error(f"[Podcast] ===== UPDATE_PROJECT_ERROR (took {duration_ms}ms): {str(e)} =====")
|
||||
raise HTTPException(status_code=500, detail=f"Error updating project: {str(e)}")
|
||||
|
||||
|
||||
|
||||
@@ -9,142 +9,37 @@ from typing import Dict, Any, List
|
||||
from types import SimpleNamespace
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from middleware.auth_middleware import get_current_user
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
from services.database import get_db
|
||||
from services.blog_writer.research.exa_provider import ExaResearchProvider
|
||||
from services.llm_providers.main_text_generation import llm_text_gen
|
||||
from services.podcast_bible_service import PodcastBibleService
|
||||
from services.database import get_db
|
||||
from services.subscription import PricingService
|
||||
from models.subscription_models import APIProvider
|
||||
from loguru import logger
|
||||
from ..cost_estimator import estimate_podcast_cost
|
||||
from ..models import (
|
||||
PodcastExaResearchRequest,
|
||||
PodcastExaResearchResponse,
|
||||
PodcastExaSource,
|
||||
PodcastExaConfig,
|
||||
PodcastResearchInsight,
|
||||
PodcastResearchOutput,
|
||||
PodcastCostEst,
|
||||
PodcastCostBreakdownItem,
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _estimate_tokens(text: str) -> int:
|
||||
if not text:
|
||||
return 0
|
||||
return max(1, len(text) // 4)
|
||||
|
||||
|
||||
def _get_price_from_catalog(
|
||||
pricing_service: PricingService,
|
||||
provider: APIProvider,
|
||||
model_name: str,
|
||||
key: str,
|
||||
fallback: float = 0.0,
|
||||
) -> float:
|
||||
try:
|
||||
pricing = pricing_service.get_pricing_for_provider_model(provider, model_name) or {}
|
||||
value = pricing.get(key)
|
||||
return float(value or fallback)
|
||||
except Exception:
|
||||
return fallback
|
||||
|
||||
|
||||
def _build_research_cost_estimate(
|
||||
request: PodcastExaResearchRequest,
|
||||
raw_content: str,
|
||||
sources_count: int,
|
||||
provider_result: Dict[str, Any],
|
||||
user_id: str = "default",
|
||||
) -> PodcastCostEst:
|
||||
# Fallback defaults mirror current catalog defaults.
|
||||
exa_per_request = 0.005
|
||||
gemini_in_token = 0.00000015
|
||||
gemini_out_token = 0.0000006
|
||||
|
||||
try:
|
||||
from services.database import get_session_for_user
|
||||
db = get_session_for_user(user_id)
|
||||
if db:
|
||||
try:
|
||||
pricing_service = PricingService(db)
|
||||
exa_per_request = _get_price_from_catalog(
|
||||
pricing_service, APIProvider.EXA, "exa-search", "cost_per_request", exa_per_request
|
||||
)
|
||||
gemini_pricing = pricing_service.get_pricing_for_provider_model(APIProvider.GEMINI, "gemini-2.5-flash") or {}
|
||||
gemini_in_token = float(gemini_pricing.get("cost_per_input_token") or gemini_in_token)
|
||||
gemini_out_token = float(gemini_pricing.get("cost_per_output_token") or gemini_out_token)
|
||||
finally:
|
||||
db.close()
|
||||
except Exception as pricing_err:
|
||||
logger.warning(f"[Podcast Research] Failed loading pricing catalog; using defaults: {pricing_err}")
|
||||
|
||||
query_count = max(1, len(request.queries or []))
|
||||
source_count = max(1, sources_count)
|
||||
|
||||
analyze_tokens = _estimate_tokens(request.topic) + sum(_estimate_tokens(q) for q in request.queries or [])
|
||||
gather_search_calls = max(1, query_count)
|
||||
gather_cost = gather_search_calls * exa_per_request
|
||||
|
||||
write_input_tokens = _estimate_tokens(raw_content) + _estimate_tokens(request.topic) + (query_count * 40)
|
||||
write_output_tokens = max(500, int(write_input_tokens * 0.22))
|
||||
write_cost = (write_input_tokens * gemini_in_token) + (write_output_tokens * gemini_out_token)
|
||||
|
||||
# "Produce" is shaping the final API payload and mapped artifacts.
|
||||
produce_tokens = max(120, source_count * 30)
|
||||
produce_cost = (produce_tokens * gemini_in_token) + (produce_tokens * 0.5 * gemini_out_token)
|
||||
|
||||
analyze_cost = analyze_tokens * gemini_in_token
|
||||
|
||||
provider_total = 0.0
|
||||
if isinstance(provider_result, dict):
|
||||
provider_total = float((provider_result.get("cost") or {}).get("total") or 0.0)
|
||||
|
||||
# Prefer transparent estimate built from catalog + usage. If provider reports a higher measured value, keep it.
|
||||
estimated_total = analyze_cost + gather_cost + write_cost + produce_cost
|
||||
scale = (provider_total / estimated_total) if estimated_total > 0 and provider_total > estimated_total else 1.0
|
||||
|
||||
breakdown = [
|
||||
PodcastCostBreakdownItem(phase="Analyze", cost=round(analyze_cost * scale, 6)),
|
||||
PodcastCostBreakdownItem(phase="Gather", cost=round(gather_cost * scale, 6)),
|
||||
PodcastCostBreakdownItem(phase="Write", cost=round(write_cost * scale, 6)),
|
||||
PodcastCostBreakdownItem(phase="Produce", cost=round(produce_cost * scale, 6)),
|
||||
]
|
||||
total = round(sum(item.cost for item in breakdown), 6)
|
||||
|
||||
return PodcastCostEst(
|
||||
total=total,
|
||||
breakdown=breakdown,
|
||||
currency="USD",
|
||||
last_updated=datetime.now(timezone.utc),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/research/exa", response_model=PodcastExaResearchResponse)
|
||||
async def podcast_research_exa(
|
||||
request: PodcastExaResearchRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Run podcast research via Exa and then use LLM to extract deep insights.
|
||||
Uses Podcast Bible and Analysis context for hyper-personalization.
|
||||
"""
|
||||
start_time = time.time()
|
||||
user_id = require_authenticated_user(current_user)
|
||||
|
||||
# Log only essential info, not full request data
|
||||
logger.warning(f"[Podcast Research] ===== RESEARCH_START =====")
|
||||
logger.warning(f"[Podcast Research] user={user_id}, topic='{request.topic[:50]}...', queries={len(request.queries) if request.queries else 0}")
|
||||
logger.warning(f"[Podcast Research] ========== REQUEST START ==========")
|
||||
logger.warning(f"[Podcast Research] User: {user_id}, Topic: {request.topic[:80]}...")
|
||||
logger.warning(f"[Podcast Research] Queries count: {len(request.queries) if request.queries else 0}")
|
||||
|
||||
|
||||
queries = [q.strip() for q in request.queries if q and q.strip()]
|
||||
@@ -224,9 +119,6 @@ Listener CTA: {request.analysis.get('listener_cta', 'N/A')}
|
||||
|
||||
summary = ""
|
||||
key_insights = []
|
||||
expert_quotes = []
|
||||
listener_cta_suggestions = []
|
||||
mapped_angles = []
|
||||
|
||||
if raw_content and sources:
|
||||
logger.warning(f"[Podcast Research] Extracting insights from {len(sources)} sources for user {user_id}")
|
||||
@@ -267,50 +159,43 @@ As a podcast research expert, analyze this data and create content that will:
|
||||
4. Include a compelling call-to-action for listeners
|
||||
|
||||
REQUIRED OUTPUT (JSON):
|
||||
======================
|
||||
=======================
|
||||
{{
|
||||
"summary": "2-3 paragraph comprehensive summary in Markdown. Start with a hook that matches the episode intro.",
|
||||
"summary": "2-3 paragraph comprehensive summary in Markdown. Start with a hook that matches the episode intro. Include specific data points, expert quotes, and trends.",
|
||||
"key_insights": [
|
||||
{{
|
||||
"title": "Insight title",
|
||||
"content": "3-4 sentences with specific facts, quotes, or data for podcast host.",
|
||||
"source_indices": [1, 2],
|
||||
"podcast_talking_points": ["Point host can expand on", "Counter-point"]
|
||||
"title": "Catchy, engaging title for this insight",
|
||||
"content": "3-4 sentences with specific facts, quotes, or data. Write in a conversational tone suitable for a podcast host to discuss.",
|
||||
"source_indices": [1, 2, 3],
|
||||
"podcast_talking_points": ["Point 1 host can expand on", "Counter-point or follow-up", "Question to ask guest"]
|
||||
}}
|
||||
],
|
||||
"expert_quotes": [
|
||||
{{
|
||||
"quote": "Direct quote from source text",
|
||||
"quote": "Direct quote from source",
|
||||
"source_index": 1,
|
||||
"context": "Why this quote matters for the podcast"
|
||||
}}
|
||||
],
|
||||
"listener_cta_suggestions": ["Action listener can take", "Resource to share", "Next episode preview"],
|
||||
"mapped_angles": [
|
||||
{{
|
||||
"title": "Content angle title",
|
||||
"why": "Why compelling for audience",
|
||||
"mapped_fact_ids": [1, 2]
|
||||
}}
|
||||
]
|
||||
"listener_cta_suggestions": ["Specific action listener can take", "Resource to share", "Next episode preview"]
|
||||
}}
|
||||
|
||||
IMPORTANT: You must include ALL fields above with valid data. expert_quotes, listener_cta_suggestions, and mapped_angles must have content - do NOT leave them empty!
|
||||
|
||||
QUALITY STANDARDS:
|
||||
=================
|
||||
- Include at least 2 expert_quotes with source_index
|
||||
- Include at least 2 listener_cta_suggestions
|
||||
- Include at least 2 mapped_angles
|
||||
- Include specific data points, percentages, statistics
|
||||
- Write in conversational tone
|
||||
==================
|
||||
- INSIGHTS MUST BE DEEP, not superficial - avoid generic statements
|
||||
- Include SPECIFIC DATA POINTS, percentages, statistics when available
|
||||
- Extract EXPERT QUOTES that hosts can reference
|
||||
- Identify GAPS in the research where more depth is needed
|
||||
- Make content naturally flow into the planned episode hook and CTA
|
||||
- Write in a CONVERSATIONAL tone - how a host would actually speak
|
||||
- Flag any CONTROVERSIAL or debatable claims for host to address
|
||||
"""
|
||||
try:
|
||||
logger.warning(f"[Podcast Research] Calling LLM with json_struct...")
|
||||
logger.warning(f"[Podcast Research] Calling LLM for insight extraction...")
|
||||
llm_response = llm_text_gen(
|
||||
prompt=prompt,
|
||||
user_id=user_id,
|
||||
json_struct=PodcastResearchOutput.model_json_schema(),
|
||||
json_struct=None,
|
||||
preferred_provider=None,
|
||||
flow_type="premium_tool",
|
||||
)
|
||||
@@ -346,22 +231,13 @@ QUALITY STANDARDS:
|
||||
try:
|
||||
summary = data.get("summary", "")
|
||||
key_insights = [PodcastResearchInsight(**insight) for insight in data.get("key_insights", [])]
|
||||
expert_quotes = data.get("expert_quotes", [])
|
||||
listener_cta_suggestions = data.get("listener_cta_suggestions", [])
|
||||
mapped_angles = data.get("mapped_angles", [])
|
||||
except Exception as insight_err:
|
||||
logger.warning(f"[Podcast Research] Failed to parse insights: {insight_err}. Data keys: {list(data.keys()) if isinstance(data, dict) else 'not a dict'}")
|
||||
summary = data.get("summary", "") if isinstance(data, dict) else ""
|
||||
key_insights = []
|
||||
expert_quotes = data.get("expert_quotes", []) if isinstance(data, dict) else []
|
||||
listener_cta_suggestions = data.get("listener_cta_suggestions", []) if isinstance(data, dict) else []
|
||||
mapped_angles = data.get("mapped_angles", []) if isinstance(data, dict) else []
|
||||
else:
|
||||
summary = ""
|
||||
key_insights = []
|
||||
expert_quotes = []
|
||||
listener_cta_suggestions = []
|
||||
mapped_angles = []
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as exc:
|
||||
@@ -413,41 +289,14 @@ QUALITY STANDARDS:
|
||||
"credibility_score": src.get("credibility_score"),
|
||||
}))
|
||||
|
||||
duration_minutes = 10
|
||||
speakers = 1
|
||||
if request.analysis:
|
||||
duration_minutes = int(request.analysis.get("duration", 10) or 10)
|
||||
speakers = int(request.analysis.get("speakers", 1) or 1)
|
||||
|
||||
estimate = estimate_podcast_cost(
|
||||
db=db,
|
||||
duration_minutes=duration_minutes,
|
||||
speakers=speakers,
|
||||
query_count=len(queries),
|
||||
include_avatar_phase=True,
|
||||
)
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.warning(f"[Podcast Research] ===== RESEARCH_END (took {duration_ms}ms) =====")
|
||||
logger.warning(f"[Podcast Research] sources={len(sources_payload)}, insights={len(key_insights)}, summary_len={len(summary)}")
|
||||
|
||||
return PodcastExaResearchResponse(
|
||||
sources=sources_payload,
|
||||
search_queries=result.get("search_queries", queries) if isinstance(result, dict) else queries,
|
||||
summary=summary,
|
||||
key_insights=key_insights,
|
||||
cost_est=_build_research_cost_estimate(
|
||||
request=request,
|
||||
raw_content=raw_content,
|
||||
sources_count=len(sources_payload),
|
||||
provider_result=result if isinstance(result, dict) else {},
|
||||
user_id=user_id,
|
||||
),
|
||||
cost=result.get("cost") if isinstance(result, dict) else None,
|
||||
search_type=result.get("search_type") if isinstance(result, dict) else None,
|
||||
provider=result.get("provider", "exa") if isinstance(result, dict) else "exa",
|
||||
content=raw_content,
|
||||
mapped_angles=mapped_angles,
|
||||
expert_quotes=expert_quotes,
|
||||
listener_cta_suggestions=listener_cta_suggestions,
|
||||
estimate=estimate,
|
||||
)
|
||||
|
||||
|
||||
@@ -8,8 +8,6 @@ from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import Dict, Any, Optional
|
||||
from pydantic import BaseModel, Field
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
|
||||
from middleware.auth_middleware import get_current_user
|
||||
from api.story_writer.utils.auth import require_authenticated_user
|
||||
@@ -25,8 +23,6 @@ from ..models import (
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
MAX_TTS_CHARS_PER_REQUEST = 10_000
|
||||
TARGET_TTS_CHARS_PER_SCENE = 8_500
|
||||
|
||||
|
||||
class SceneApprovalRequest(BaseModel):
|
||||
@@ -61,46 +57,31 @@ async def generate_podcast_script(
|
||||
Generate a podcast script outline (scenes + lines) using podcast-oriented prompting.
|
||||
"""
|
||||
user_id = require_authenticated_user(current_user)
|
||||
start_time = time.time()
|
||||
logger.warning(f"[ScriptGen] ===== SCRIPT_GEN_START =====")
|
||||
logger.warning(f"[ScriptGen] user={user_id}, topic='{request.idea[:50]}...', duration={request.duration_minutes}min, speakers={request.speakers}")
|
||||
podcast_mode = (request.podcast_mode or "video_only").strip().lower()
|
||||
logger.warning(f"[ScriptGen] research={bool(request.research)}, bible={bool(request.bible)}, analysis={bool(request.analysis)}, mode={podcast_mode}")
|
||||
research_fact_cards = request.research.get("factCards", []) if request.research else []
|
||||
logger.warning(f"[ScriptGen] ========== SCRIPT GENERATION START ==========")
|
||||
logger.warning(f"[ScriptGen] Topic: {request.idea[:60]}...")
|
||||
logger.warning(f"[ScriptGen] Duration: {request.duration_minutes} min, Speakers: {request.speakers}")
|
||||
logger.warning(f"[ScriptGen] Has research: {bool(request.research)}, Has bible: {bool(request.bible)}, Has analysis: {bool(request.analysis)}")
|
||||
|
||||
# Build comprehensive research context for higher-quality scripts
|
||||
research_context = ""
|
||||
if request.research:
|
||||
try:
|
||||
key_insights = request.research.get("keyword_analysis", {}).get("key_insights") or []
|
||||
fact_cards = research_fact_cards or []
|
||||
fact_cards = request.research.get("factCards", []) or []
|
||||
mapped_angles = request.research.get("mappedAngles", []) or []
|
||||
sources = request.research.get("sources", []) or []
|
||||
|
||||
top_facts = [
|
||||
f"[{f.get('id') or f'fact_{idx + 1}'}] {f.get('quote', '')}"
|
||||
for idx, f in enumerate(fact_cards[:10])
|
||||
if f.get("quote")
|
||||
]
|
||||
top_facts = [f.get("quote", "") for f in fact_cards[:5] if f.get("quote")]
|
||||
angles_summary = [
|
||||
f"{a.get('title', '')}: {a.get('why', '')}" for a in mapped_angles[:3] if a.get("title") or a.get("why")
|
||||
]
|
||||
top_sources = [s.get("url") for s in sources[:3] if s.get("url")]
|
||||
numeric_signals = []
|
||||
for f in fact_cards[:12]:
|
||||
quote = (f.get("quote") or "").strip()
|
||||
if any(ch.isdigit() for ch in quote):
|
||||
numeric_signals.append(quote[:180])
|
||||
if len(numeric_signals) >= 5:
|
||||
break
|
||||
|
||||
research_parts = []
|
||||
if key_insights:
|
||||
research_parts.append(f"Key Insights: {', '.join(key_insights[:5])}")
|
||||
if top_facts:
|
||||
research_parts.append(f"Key Facts: {', '.join(top_facts)}")
|
||||
if numeric_signals:
|
||||
research_parts.append(f"Numeric Signals (prefer for chart scenes): {' | '.join(numeric_signals)}")
|
||||
if angles_summary:
|
||||
research_parts.append(f"Research Angles: {' | '.join(angles_summary)}")
|
||||
if top_sources:
|
||||
@@ -111,53 +92,6 @@ async def generate_podcast_script(
|
||||
logger.warning(f"Failed to parse research context: {exc}")
|
||||
research_context = ""
|
||||
|
||||
def _normalize_fact_ids(value: Any) -> Optional[list[str]]:
|
||||
if not value:
|
||||
return None
|
||||
if isinstance(value, list):
|
||||
cleaned = [str(v).strip() for v in value if str(v).strip()]
|
||||
return cleaned or None
|
||||
if isinstance(value, str) and value.strip():
|
||||
return [value.strip()]
|
||||
return None
|
||||
|
||||
def _default_chart_data(scene_title: str) -> Dict[str, Any]:
|
||||
numeric_pairs: list[tuple[str, float]] = []
|
||||
for fact in research_fact_cards[:12]:
|
||||
quote = (fact.get("quote") or "").strip()
|
||||
if not quote:
|
||||
continue
|
||||
nums = re.findall(r"\d+(?:\.\d+)?", quote.replace(",", ""))
|
||||
if not nums:
|
||||
continue
|
||||
label = quote[:48] + ("…" if len(quote) > 48 else "")
|
||||
try:
|
||||
numeric_pairs.append((label, float(nums[0])))
|
||||
except ValueError:
|
||||
continue
|
||||
if len(numeric_pairs) >= 5:
|
||||
break
|
||||
|
||||
if numeric_pairs:
|
||||
labels = [p[0] for p in numeric_pairs]
|
||||
values = [p[1] for p in numeric_pairs]
|
||||
sources = [f.get("url", f.get("source", "")) for f in research_fact_cards[:12] if f.get("url") or f.get("source")]
|
||||
return {
|
||||
"type": "bar_comparison",
|
||||
"title": scene_title,
|
||||
"labels": labels,
|
||||
"values": values,
|
||||
"takeaway": "Data points sourced from research facts used in this scene.",
|
||||
"source": sources[0] if sources else "",
|
||||
}
|
||||
|
||||
return {
|
||||
"type": "bullet_points",
|
||||
"title": scene_title,
|
||||
"bullet_points": ["Key point 1", "Key point 2", "Key point 3"],
|
||||
"takeaway": "Narration summary for this scene.",
|
||||
}
|
||||
|
||||
# Extract Podcast Bible context for hyper-personalization
|
||||
bible_context = ""
|
||||
if request.bible:
|
||||
@@ -188,62 +122,25 @@ async def generate_podcast_script(
|
||||
except:
|
||||
pass
|
||||
|
||||
mode_instructions = ""
|
||||
if podcast_mode == "audio_only":
|
||||
mode_instructions = f"""
|
||||
AUDIO-ONLY MODE RULES (CRITICAL):
|
||||
- This is an audio-only episode. Do NOT include avatar/image/camera instructions.
|
||||
- Keep each scene's total dialogue under {TARGET_TTS_CHARS_PER_SCENE} chars to stay below TTS max request size ({MAX_TTS_CHARS_PER_REQUEST}).
|
||||
- For every scene include chart_data so B-roll charts can be generated while narration plays.
|
||||
- Build script STRICTLY from RESEARCH context and cite fact linkage via usedFactIds.
|
||||
- If evidence is weak, say uncertainty explicitly rather than inventing facts.
|
||||
- Add natural TTS pacing in dialogue with markers like [pause:300ms], [pause:700ms], [emote:curious], [emote:serious].
|
||||
"""
|
||||
elif podcast_mode == "audio_video":
|
||||
mode_instructions = """
|
||||
AUDIO+VIDEO MODE:
|
||||
- Include rich narration that works for both listening and visual storytelling.
|
||||
- Use a balanced pace suitable for TTS and scene visuals.
|
||||
"""
|
||||
else:
|
||||
mode_instructions = """
|
||||
VIDEO-ONLY MODE:
|
||||
- Prioritize visual rhythm and concise narration per scene.
|
||||
"""
|
||||
|
||||
prompt = f"""Create a podcast script with scenes and dialogue.
|
||||
|
||||
{f"BIBLE: {bible_context[:1500]}" if bible_context else ""}
|
||||
{f"{analysis_context}" if analysis_context else ""}
|
||||
{f"{outline_context}" if outline_context else ""}
|
||||
{f"RESEARCH: {research_context[:2500]}" if research_context else ""}
|
||||
{mode_instructions}
|
||||
{f"RESEARCH: {research_context[:1200]}" if research_context else ""}
|
||||
|
||||
Topic: "{request.idea}"
|
||||
Duration: {request.duration_minutes} min | Speakers: {request.speakers}
|
||||
Podcast mode: {podcast_mode}
|
||||
|
||||
Return JSON with scenes array. Each scene:
|
||||
- id: string
|
||||
- title: short title (<=50 chars)
|
||||
- duration: seconds (total/5)
|
||||
- emotion: neutral|happy|excited|serious|curious|confident
|
||||
- lines: array of {{speaker, text, emphasis, usedFactIds, ttsHints}}
|
||||
- lines: array of {{speaker, text, emphasis}}
|
||||
- Use 2-4 LINES PER SCENE (shorter script = lower TTS costs)
|
||||
- Each line: 1-3 sentences, conversational
|
||||
- usedFactIds: include related fact ids when research facts are available (example: ["fact_1", "fact_3"])
|
||||
- ttsHints: optional list from [pause_300ms, pause_700ms, smile, serious_tone, emphasize_data]
|
||||
- Plain text only, no markdown
|
||||
- chart_data: object for B-roll mapping (required in audio_only)
|
||||
- type: bar_comparison|bar_horizontal|line_trend|pie|stacked_bar|bullet_points
|
||||
- title: short chart title
|
||||
- labels: list
|
||||
- values: list (same length as labels, required for bar/line/pie)
|
||||
- before/after: parallel lists of numbers (for bar_comparison only)
|
||||
- segments: list of {{name, values}} (for stacked_bar only)
|
||||
- bullet_points: list of strings (for bullet_points only)
|
||||
- takeaway: one sentence tying chart to narration
|
||||
- source: URL or citation for the data (e.g. "Research fact #3" or a URL from the research context)
|
||||
|
||||
COST OPTIMIZATION:
|
||||
- 5-6 scenes max for {request.duration_minutes} min episode
|
||||
@@ -281,112 +178,25 @@ COST OPTIMIZATION:
|
||||
scenes_data = data.get("scenes") or []
|
||||
if not isinstance(scenes_data, list):
|
||||
raise HTTPException(status_code=500, detail="LLM response missing scenes array")
|
||||
|
||||
if len(scenes_data) == 0:
|
||||
logger.warning("[ScriptGen] LLM returned empty scenes array")
|
||||
raise HTTPException(status_code=500, detail="LLM returned no scenes - please try again")
|
||||
|
||||
logger.warning(f"[ScriptGen] Processing {len(scenes_data)} scenes from LLM response")
|
||||
|
||||
valid_emotions = {"neutral", "happy", "excited", "serious", "curious", "confident"}
|
||||
|
||||
# Normalize scenes
|
||||
scenes: list[PodcastScene] = []
|
||||
total_lines_input = 0
|
||||
total_lines_output = 0
|
||||
dropped_empty_lines = 0
|
||||
|
||||
for idx, scene in enumerate(scenes_data):
|
||||
if not isinstance(scene, dict):
|
||||
logger.warning(f"[ScriptGen] Scene {idx} is not a dict, skipping")
|
||||
continue
|
||||
|
||||
title = scene.get("title") or f"Scene {idx + 1}"
|
||||
duration = int(scene.get("duration") or max(30, (request.duration_minutes * 60) // max(1, len(scenes_data))))
|
||||
emotion = scene.get("emotion") or "neutral"
|
||||
if emotion not in valid_emotions:
|
||||
logger.warning(f"[ScriptGen] Invalid emotion '{emotion}' in scene {idx}, defaulting to 'neutral'")
|
||||
emotion = "neutral"
|
||||
lines_raw = scene.get("lines") or []
|
||||
total_lines_input += len(lines_raw)
|
||||
lines: list[PodcastSceneLine] = []
|
||||
|
||||
for line_idx, line in enumerate(lines_raw):
|
||||
if not isinstance(line, dict):
|
||||
logger.warning(f"[ScriptGen] Line {line_idx} in scene {idx} is not a dict, skipping")
|
||||
continue
|
||||
|
||||
for line in lines_raw:
|
||||
speaker = line.get("speaker") or ("Host" if len(lines) % request.speakers == 0 else "Guest")
|
||||
text = line.get("text") or ""
|
||||
|
||||
# Handle emphasis - convert various values to boolean
|
||||
emphasis_raw = line.get("emphasis", False)
|
||||
if isinstance(emphasis_raw, bool):
|
||||
emphasis = emphasis_raw
|
||||
elif isinstance(emphasis_raw, str):
|
||||
emphasis = emphasis_raw.lower() in ("true", "yes", "1")
|
||||
if emphasis_raw.lower() not in ("true", "false", "yes", "no", "1", "0"):
|
||||
logger.debug(f"[ScriptGen] Unusual emphasis value '{emphasis_raw}' converted to {emphasis}")
|
||||
else:
|
||||
emphasis = bool(emphasis_raw)
|
||||
|
||||
# Generate line ID if not provided
|
||||
line_id = line.get("id") or f"line-{idx + 1}-{line_idx + 1}"
|
||||
|
||||
# Get used fact IDs if provided
|
||||
used_fact_ids = _normalize_fact_ids(line.get("usedFactIds") or line.get("used_fact_ids"))
|
||||
tts_hints = line.get("ttsHints") or line.get("tts_hints") or None
|
||||
|
||||
emphasis = line.get("emphasis", False)
|
||||
if text:
|
||||
lines.append(PodcastSceneLine(
|
||||
speaker=speaker,
|
||||
text=text,
|
||||
emphasis=emphasis,
|
||||
id=line_id,
|
||||
usedFactIds=used_fact_ids,
|
||||
ttsHints=tts_hints if isinstance(tts_hints, list) else None,
|
||||
))
|
||||
total_lines_output += 1
|
||||
else:
|
||||
dropped_empty_lines += 1
|
||||
logger.debug(f"[ScriptGen] Dropped empty line {line_idx} in scene {idx}")
|
||||
|
||||
# Log scene status
|
||||
if scenes_data and isinstance(scene, dict):
|
||||
image_url_raw = scene.get("imageUrl") or scene.get("image_url")
|
||||
audio_url_raw = scene.get("audioUrl") or scene.get("audio_url")
|
||||
if image_url_raw:
|
||||
logger.warning(f"[ScriptGen] Scene {idx} has imageUrl - will be reset to None")
|
||||
if audio_url_raw:
|
||||
logger.warning(f"[ScriptGen] Scene {idx} has audioUrl - will be reset to None")
|
||||
|
||||
# Keep each scene under TTS request size to prevent failures
|
||||
scene_char_count = sum(len((l.text or "").strip()) for l in lines)
|
||||
if scene_char_count > TARGET_TTS_CHARS_PER_SCENE and lines:
|
||||
logger.warning(
|
||||
f"[ScriptGen] Scene {idx} text too long ({scene_char_count} chars). "
|
||||
f"Trimming to {TARGET_TTS_CHARS_PER_SCENE} target."
|
||||
)
|
||||
trimmed_lines: list[PodcastSceneLine] = []
|
||||
remaining = TARGET_TTS_CHARS_PER_SCENE
|
||||
for l in lines:
|
||||
if remaining <= 0:
|
||||
break
|
||||
line_text = (l.text or "").strip()
|
||||
if len(line_text) <= remaining:
|
||||
trimmed_lines.append(l)
|
||||
remaining -= len(line_text)
|
||||
continue
|
||||
l.text = f"{line_text[:max(0, remaining - 1)].rstrip()}…"
|
||||
trimmed_lines.append(l)
|
||||
remaining = 0
|
||||
lines = trimmed_lines
|
||||
|
||||
chart_data = scene.get("chart_data") or scene.get("chartData") or None
|
||||
if podcast_mode == "audio_only" and not chart_data:
|
||||
# Ensure audio-only always has a B-roll mapping fallback
|
||||
chart_data = _default_chart_data(title)
|
||||
|
||||
lines.append(PodcastSceneLine(speaker=speaker, text=text, emphasis=emphasis))
|
||||
scenes.append(
|
||||
PodcastScene(
|
||||
id=scene.get("id") or f"scene-{idx + 1}",
|
||||
@@ -395,19 +205,8 @@ COST OPTIMIZATION:
|
||||
lines=lines,
|
||||
approved=False,
|
||||
emotion=emotion,
|
||||
imageUrl=None, # Will be generated later
|
||||
audioUrl=None, # Will be generated later
|
||||
imagePrompt=None, # Will be generated during image generation
|
||||
chart_data=chart_data if isinstance(chart_data, dict) else None,
|
||||
)
|
||||
)
|
||||
|
||||
# Summary logging
|
||||
logger.warning(f"[ScriptGen] Script generated: {len(scenes)} scenes, {total_lines_output}/{total_lines_input} lines")
|
||||
if dropped_empty_lines > 0:
|
||||
logger.warning(f"[ScriptGen] Dropped {dropped_empty_lines} empty lines")
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.warning(f"[ScriptGen] ===== SCRIPT_GEN_END (took {duration_ms}ms) =====")
|
||||
|
||||
return PodcastScriptResponse(scenes=scenes)
|
||||
|
||||
|
||||
@@ -1,251 +0,0 @@
|
||||
"""
|
||||
Category Research Handlers
|
||||
|
||||
Research endpoints using Tavily or Exa for category-based topic discovery.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import Dict, Any, List, Optional
|
||||
from pydantic import BaseModel
|
||||
from loguru import logger
|
||||
from types import SimpleNamespace
|
||||
|
||||
from middleware.auth_middleware import get_current_user
|
||||
from services.research.tavily_service import TavilyService
|
||||
from services.blog_writer.research.exa_provider import ExaResearchProvider
|
||||
|
||||
router = APIRouter(prefix="/research", tags=["Podcast Category Research"])
|
||||
|
||||
CATEGORY_PROVIDER_MAP = {
|
||||
"news": "tavily",
|
||||
"finance": "tavily",
|
||||
"research-paper": "exa",
|
||||
"personal-site": "exa",
|
||||
}
|
||||
|
||||
EXA_CATEGORY_MAP = {
|
||||
"research-paper": "research paper",
|
||||
"personal-site": "personal site",
|
||||
}
|
||||
|
||||
|
||||
class CategoryResearchRequest(BaseModel):
|
||||
category: str
|
||||
keyword: Optional[str] = None
|
||||
max_results: Optional[int] = 8
|
||||
website_url: Optional[str] = None
|
||||
|
||||
|
||||
class CategoryTopic(BaseModel):
|
||||
title: str
|
||||
url: str
|
||||
snippet: str
|
||||
score: float
|
||||
favicon: Optional[str] = None
|
||||
|
||||
|
||||
class CategoryResearchResponse(BaseModel):
|
||||
success: bool
|
||||
category: str
|
||||
provider: str
|
||||
topics: List[CategoryTopic]
|
||||
query: Optional[str] = None
|
||||
error: Optional[str] = None
|
||||
|
||||
|
||||
def _normalize_tavily_results(results: List[Dict]) -> List[CategoryTopic]:
|
||||
topics = []
|
||||
for item in results:
|
||||
topics.append(CategoryTopic(
|
||||
title=item.get("title", ""),
|
||||
url=item.get("url", ""),
|
||||
snippet=item.get("content", ""),
|
||||
score=item.get("score", 0.0),
|
||||
favicon=item.get("favicon"),
|
||||
))
|
||||
return topics
|
||||
|
||||
|
||||
def _normalize_exa_results(results: List[Dict], query: str) -> List[CategoryTopic]:
|
||||
topics = []
|
||||
for idx, item in enumerate(results):
|
||||
score = 1.0 - (idx * 0.1)
|
||||
topics.append(CategoryTopic(
|
||||
title=item.get("title", "") or f"Result {idx + 1}",
|
||||
url=item.get("url", ""),
|
||||
snippet=item.get("summary", "") or item.get("text", "") or "",
|
||||
score=max(0.5, score),
|
||||
favicon=None,
|
||||
))
|
||||
return topics
|
||||
|
||||
|
||||
async def _search_tavily(category: str, keyword: str, max_results: int) -> CategoryResearchResponse:
|
||||
logger.info(f"[CategoryResearch] Using Tavily for category={category}, keyword={keyword}")
|
||||
|
||||
try:
|
||||
tavily = TavilyService()
|
||||
result = await tavily.search(
|
||||
query=keyword,
|
||||
topic=category,
|
||||
search_depth="basic",
|
||||
max_results=max_results,
|
||||
include_favicon=True,
|
||||
)
|
||||
|
||||
if not result.get("success"):
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=result.get("error", "Tavily search failed")
|
||||
)
|
||||
|
||||
topics = _normalize_tavily_results(result.get("results", []))
|
||||
logger.info(f"[CategoryResearch] Tavily found {len(topics)} topics")
|
||||
|
||||
return CategoryResearchResponse(
|
||||
success=True,
|
||||
category=category,
|
||||
provider="tavily",
|
||||
topics=topics,
|
||||
query=keyword,
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"[CategoryResearch] Tavily error: {e}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
|
||||
async def _search_exa(category: str, keyword: str, max_results: int, website_url: Optional[str] = None) -> CategoryResearchResponse:
|
||||
exa_category = EXA_CATEGORY_MAP.get(category, category)
|
||||
|
||||
logger.info(f"[CategoryResearch] Exa: category={category}, exa_category={exa_category}, keyword={keyword}, website_url={website_url}")
|
||||
|
||||
try:
|
||||
# Import exa directly for more control
|
||||
import os
|
||||
from urllib.parse import urlparse
|
||||
exa_api_key = os.getenv("EXA_API_KEY")
|
||||
if not exa_api_key:
|
||||
raise HTTPException(status_code=500, detail="EXA_API_KEY not configured")
|
||||
|
||||
from exa_py import Exa
|
||||
exa = Exa(exa_api_key)
|
||||
logger.info(f"[CategoryResearch] Exa client initialized")
|
||||
|
||||
# Build search parameters
|
||||
search_params = {
|
||||
"num_results": max_results,
|
||||
"category": exa_category,
|
||||
}
|
||||
|
||||
# For personal-site, extract domain from URL if provided
|
||||
include_domains = None
|
||||
if category == "personal-site" and website_url:
|
||||
try:
|
||||
parsed = urlparse(website_url)
|
||||
if parsed.netloc:
|
||||
include_domains = [parsed.netloc]
|
||||
logger.info(f"[CategoryResearch] Personal site - limiting to domain: {parsed.netloc}")
|
||||
elif parsed.path and "." in parsed.path:
|
||||
# Could be domain without protocol
|
||||
include_domains = [parsed.path]
|
||||
logger.info(f"[CategoryResearch] Personal site - using as domain: {parsed.path}")
|
||||
except Exception as url_err:
|
||||
logger.warning(f"[CategoryResearch] Failed to parse website_url: {url_err}")
|
||||
|
||||
logger.info(f"[CategoryResearch] Calling Exa with params: {search_params}, include_domains={include_domains}")
|
||||
|
||||
# Make the search call
|
||||
results = exa.search_and_contents(
|
||||
query=keyword,
|
||||
type="auto" if category != "personal-site" else "neural",
|
||||
num_results=max_results,
|
||||
category=exa_category,
|
||||
text=True,
|
||||
summary=True,
|
||||
include_domains=include_domains,
|
||||
)
|
||||
|
||||
logger.info(f"[CategoryResearch] Exa search completed, got results")
|
||||
|
||||
# Transform results to our format
|
||||
topics = []
|
||||
if results and hasattr(results, 'results'):
|
||||
for item in results.results:
|
||||
title = getattr(item, 'title', 'Untitled')
|
||||
url = getattr(item, 'url', '')
|
||||
snippet = getattr(item, 'summary', '') or getattr(item, 'text', '') or ''
|
||||
score = 0.8 # Default score for Exa results
|
||||
|
||||
topics.append(CategoryTopic(
|
||||
title=title,
|
||||
url=url,
|
||||
snippet=snippet[:300] if snippet else '',
|
||||
score=score,
|
||||
favicon=None,
|
||||
))
|
||||
|
||||
logger.info(f"[CategoryResearch] Exa found {len(topics)} topics")
|
||||
|
||||
return CategoryResearchResponse(
|
||||
success=True,
|
||||
category=category,
|
||||
provider="exa",
|
||||
topics=topics,
|
||||
query=keyword,
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
import traceback
|
||||
logger.error(f"[CategoryResearch] Exa error: {type(e).__name__}: {e}")
|
||||
logger.error(f"[CategoryResearch] Stack: {traceback.format_exc()}")
|
||||
raise HTTPException(status_code=500, detail=f"Exa search failed: {str(e)}")
|
||||
|
||||
|
||||
@router.post("/tavily-category", response_model=CategoryResearchResponse)
|
||||
async def research_by_category(
|
||||
request: CategoryResearchRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""
|
||||
Research topics by category using Tavily or Exa.
|
||||
|
||||
Categories:
|
||||
- news, finance: Uses Tavily
|
||||
- research-paper, personal-site: Uses Exa
|
||||
"""
|
||||
category = request.category.lower()
|
||||
valid_categories = list(CATEGORY_PROVIDER_MAP.keys())
|
||||
|
||||
logger.info(f"[CategoryResearch] Full request payload: category={request.category}, keyword={request.keyword}, website_url={request.website_url}")
|
||||
|
||||
if category not in valid_categories:
|
||||
logger.error(f"[CategoryResearch] Invalid category: {category}, valid: {valid_categories}")
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Category must be one of: {', '.join(valid_categories)}"
|
||||
)
|
||||
|
||||
keyword = request.keyword or category
|
||||
max_results = min(max(request.max_results or 8, 5), 10)
|
||||
website_url = request.website_url
|
||||
|
||||
logger.info(f"[CategoryResearch] Processing: category={category}, keyword={keyword}, max_results={max_results}, website_url={website_url}")
|
||||
|
||||
provider = CATEGORY_PROVIDER_MAP.get(category, "tavily")
|
||||
logger.info(f"[CategoryResearch] Selected provider: {provider} for category: {category}")
|
||||
|
||||
try:
|
||||
if provider == "tavily":
|
||||
return await _search_tavily(category, keyword, max_results)
|
||||
elif provider == "exa":
|
||||
return await _search_exa(category, keyword, max_results, website_url)
|
||||
else:
|
||||
raise HTTPException(status_code=500, detail="Unknown provider")
|
||||
except Exception as e:
|
||||
logger.error(f"[CategoryResearch] Outer error: {type(e).__name__}: {e}", exc_info=True)
|
||||
raise
|
||||
@@ -1,92 +0,0 @@
|
||||
"""
|
||||
Podcast Trends Handler
|
||||
|
||||
Endpoints for fetching Google Trends data relevant to podcast topics.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import Dict, Any, List, Optional
|
||||
from pydantic import BaseModel, Field
|
||||
from loguru import logger
|
||||
|
||||
from middleware.auth_middleware import get_current_user
|
||||
|
||||
router = APIRouter(prefix="/trends", tags=["Podcast Trends"])
|
||||
|
||||
|
||||
class PodcastTrendsRequest(BaseModel):
|
||||
keywords: List[str] = Field(..., min_length=1, max_length=5, description="1-5 keywords to analyze")
|
||||
timeframe: str = Field(default="today 12-m", description="Timeframe: 'today 3-m', 'today 12-m', 'today 5-y', 'all'")
|
||||
geo: str = Field(default="US", description="Country code: 'US', 'GB', 'IN', etc.")
|
||||
source: str = Field(default="web", description="Data source: 'web' (Google), 'podcast' (YouTube)")
|
||||
|
||||
|
||||
class PodcastTrendsResponse(BaseModel):
|
||||
success: bool
|
||||
data: Optional[Dict[str, Any]] = None
|
||||
error: Optional[str] = None
|
||||
|
||||
|
||||
@router.post("", response_model=PodcastTrendsResponse)
|
||||
async def get_podcast_trends(
|
||||
request: PodcastTrendsRequest,
|
||||
current_user: Dict[str, Any] = Depends(get_current_user),
|
||||
):
|
||||
"""Fetch Google Trends data for podcast topic keywords."""
|
||||
user_id = current_user.get("user_id") or current_user.get("id")
|
||||
if not user_id:
|
||||
raise HTTPException(status_code=401, detail="User ID not found")
|
||||
|
||||
try:
|
||||
from services.research.trends import GoogleTrendsService
|
||||
except (ImportError, RuntimeError) as e:
|
||||
logger.error(f"[Podcast Trends] GoogleTrendsService unavailable: {e}")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="Google Trends service is currently unavailable. Please try again later."
|
||||
)
|
||||
|
||||
try:
|
||||
service = GoogleTrendsService()
|
||||
# Map 'source' to 'gprop' - 'podcast' uses YouTube for video/podcast relevance
|
||||
gprop_map = {"": "", "web": "", "podcast": "youtube", "news": "news", "images": "images", "shopping": "froogle"}
|
||||
gprop = gprop_map.get(request.source, "")
|
||||
|
||||
result = await service.analyze_trends(
|
||||
keywords=request.keywords,
|
||||
timeframe=request.timeframe,
|
||||
geo=request.geo,
|
||||
gprop=gprop,
|
||||
user_id=user_id,
|
||||
)
|
||||
|
||||
has_error = result.get("error")
|
||||
has_data = (
|
||||
len(result.get("interest_over_time", [])) > 0
|
||||
or len(result.get("interest_by_region", [])) > 0
|
||||
or len(result.get("related_topics", {}).get("top", [])) > 0
|
||||
or len(result.get("related_topics", {}).get("rising", [])) > 0
|
||||
or len(result.get("related_queries", {}).get("top", [])) > 0
|
||||
or len(result.get("related_queries", {}).get("rising", [])) > 0
|
||||
)
|
||||
|
||||
# Return error if: has error OR no data (meaning blocked/empty)
|
||||
if has_error and not has_data:
|
||||
error_msg = result.get("error", "")
|
||||
logger.warning(f"[Trends] No data or error: {error_msg[:100]}")
|
||||
return PodcastTrendsResponse(success=False, data=result, error=error_msg or "No trends data available. Google may be blocking requests.")
|
||||
|
||||
# Even if no error but empty data - return error
|
||||
if not has_data:
|
||||
logger.warning("[Trends] Empty data returned")
|
||||
return PodcastTrendsResponse(success=False, data=result, error="No trends data available. Please try different keywords.")
|
||||
|
||||
return PodcastTrendsResponse(success=True, data=result)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
except Exception as e:
|
||||
logger.error(f"[Podcast Trends] Error fetching trends for {request.keywords}: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Failed to fetch trends data: {str(e)}"
|
||||
)
|
||||
@@ -321,7 +321,7 @@ async def generate_podcast_video(
|
||||
|
||||
# Load image bytes (scene image is required for video generation)
|
||||
if body.avatar_image_url:
|
||||
image_bytes = load_podcast_image_bytes(body.avatar_image_url, user_id=user_id)
|
||||
image_bytes = load_podcast_image_bytes(body.avatar_image_url)
|
||||
else:
|
||||
# Scene-specific image should be generated before video generation
|
||||
raise HTTPException(
|
||||
@@ -332,7 +332,7 @@ async def generate_podcast_video(
|
||||
mask_image_bytes = None
|
||||
if body.mask_image_url:
|
||||
try:
|
||||
mask_image_bytes = load_podcast_image_bytes(body.mask_image_url, user_id=user_id)
|
||||
mask_image_bytes = load_podcast_image_bytes(body.mask_image_url)
|
||||
except Exception as e:
|
||||
logger.error(f"[Podcast] Failed to load mask image: {e}")
|
||||
raise HTTPException(
|
||||
|
||||
@@ -5,7 +5,7 @@ All Pydantic request/response models for podcast endpoints.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
from typing import List, Optional, Dict, Any, Literal
|
||||
from typing import List, Optional, Dict, Any
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
|
||||
@@ -54,7 +54,6 @@ class PodcastAnalyzeRequest(BaseModel):
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
||||
avatar_url: Optional[str] = Field(None, description="Current avatar URL if selected")
|
||||
feedback: Optional[str] = Field(None, description="User feedback for regeneration")
|
||||
podcast_mode: Optional[str] = Field(None, description="Podcast mode: audio_only, video_only, or audio_video")
|
||||
|
||||
|
||||
class PodcastAnalyzeResponse(BaseModel):
|
||||
@@ -73,21 +72,12 @@ class PodcastAnalyzeResponse(BaseModel):
|
||||
bible: Optional[Dict[str, Any]] = None
|
||||
avatar_url: Optional[str] = None
|
||||
avatar_prompt: Optional[str] = None
|
||||
estimate: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
class PodcastEnhanceIdeaRequest(BaseModel):
|
||||
"""Request model for enhancing a podcast idea with AI."""
|
||||
idea: str = Field(..., description="The raw podcast idea or keywords")
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Optional Podcast Bible for context")
|
||||
website_data: Optional[Dict[str, Any]] = Field(
|
||||
None,
|
||||
description="Optional website extraction data for enriched context (title, summary, highlights, subpages, url)"
|
||||
)
|
||||
topic_context: Optional[Dict[str, Any]] = Field(
|
||||
None,
|
||||
description="Optional category research context (category, topics, selected_topic)"
|
||||
)
|
||||
|
||||
|
||||
class PodcastEnhanceIdeaResponse(BaseModel):
|
||||
@@ -105,16 +95,12 @@ class PodcastScriptRequest(BaseModel):
|
||||
bible: Optional[Dict[str, Any]] = Field(None, description="Podcast Bible for hyper-personalization")
|
||||
outline: Optional[Dict[str, Any]] = Field(None, description="The refined episode outline to follow")
|
||||
analysis: Optional[Dict[str, Any]] = Field(None, description="The full analysis context (audience, keywords, etc.)")
|
||||
podcast_mode: Optional[str] = Field(default="video_only", description="Podcast mode: audio_only, video_only, or audio_video")
|
||||
|
||||
|
||||
class PodcastSceneLine(BaseModel):
|
||||
speaker: str
|
||||
text: str
|
||||
emphasis: Optional[bool] = False
|
||||
id: Optional[str] = None # Optional line ID for frontend tracking
|
||||
usedFactIds: Optional[List[str]] = None # Facts referenced in this line
|
||||
ttsHints: Optional[List[str]] = None # Optional TTS hints, e.g. pause_300ms, smile, emphasize_data
|
||||
|
||||
|
||||
class PodcastScene(BaseModel):
|
||||
@@ -125,9 +111,6 @@ class PodcastScene(BaseModel):
|
||||
approved: bool = False
|
||||
emotion: Optional[str] = None
|
||||
imageUrl: Optional[str] = None # Generated image URL for video generation
|
||||
audioUrl: Optional[str] = None # Generated audio URL for this scene
|
||||
imagePrompt: Optional[str] = None # Original image generation prompt for video context
|
||||
chart_data: Optional[Dict[str, Any]] = None # Optional chart mapping for B-roll scenes
|
||||
|
||||
|
||||
class PodcastExaConfig(BaseModel):
|
||||
@@ -184,40 +167,15 @@ class PodcastResearchInsight(BaseModel):
|
||||
listener_cta_suggestions: Optional[List[str]] = [] # CTA suggestions
|
||||
|
||||
|
||||
class PodcastResearchOutput(BaseModel):
|
||||
"""Structured JSON output for LLM research extraction using json_struct."""
|
||||
summary: str = ""
|
||||
key_insights: List[PodcastResearchInsight] = []
|
||||
expert_quotes: List[Dict[str, Any]] = [] # [{"quote": str, "source_index": int, "context": str}]
|
||||
listener_cta_suggestions: List[str] = [] # List of CTA suggestions
|
||||
mapped_angles: List[Dict[str, Any]] = [] # [{"title": str, "why": str, "mapped_fact_ids": []}]
|
||||
|
||||
|
||||
class PodcastCostBreakdownItem(BaseModel):
|
||||
phase: Literal["Analyze", "Gather", "Write", "Produce"]
|
||||
cost: float
|
||||
|
||||
|
||||
class PodcastCostEst(BaseModel):
|
||||
total: float
|
||||
breakdown: List[PodcastCostBreakdownItem]
|
||||
currency: Literal["USD"] = "USD"
|
||||
last_updated: datetime
|
||||
|
||||
|
||||
class PodcastExaResearchResponse(BaseModel):
|
||||
sources: List[PodcastExaSource]
|
||||
search_queries: List[str] = []
|
||||
summary: str = ""
|
||||
key_insights: List[PodcastResearchInsight] = []
|
||||
cost_est: PodcastCostEst
|
||||
cost: Optional[Dict[str, Any]] = None
|
||||
search_type: Optional[str] = None
|
||||
provider: str = "exa"
|
||||
content: Optional[str] = None # Raw aggregated content (deprecated)
|
||||
mapped_angles: List[Dict[str, Any]] = [] # Content angles for the episode
|
||||
expert_quotes: List[Dict[str, Any]] = [] # Expert quotes from research
|
||||
listener_cta_suggestions: List[str] = [] # CTA suggestions
|
||||
estimate: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
class PodcastScriptResponse(BaseModel):
|
||||
@@ -231,9 +189,6 @@ class PodcastAudioRequest(BaseModel):
|
||||
text: str
|
||||
voice_id: Optional[str] = "Wise_Woman"
|
||||
custom_voice_id: Optional[str] = None # Voice clone ID for custom voice
|
||||
use_voice_clone: Optional[bool] = False # If True, use voice clone with voice_sample_url
|
||||
voice_sample_url: Optional[str] = None # URL to user's voice sample for cloning
|
||||
voice_clone_engine: Optional[str] = None # Engine: "qwen3", "minimax", "cosyvoice"
|
||||
speed: Optional[float] = 1.0
|
||||
volume: Optional[float] = 1.0
|
||||
pitch: Optional[float] = 0.0
|
||||
@@ -479,58 +434,3 @@ class VoiceCloneResult(BaseModel):
|
||||
task_id: str
|
||||
status: str = "completed"
|
||||
|
||||
|
||||
class ExtractUrlRequest(BaseModel):
|
||||
"""Request to extract content from a URL using Exa."""
|
||||
url: str = Field(..., description="URL to extract content from")
|
||||
|
||||
|
||||
class ExtractUrlResponse(BaseModel):
|
||||
"""Response with extracted content from URL."""
|
||||
success: bool
|
||||
title: Optional[str] = None
|
||||
text: Optional[str] = None
|
||||
summary: Optional[str] = None
|
||||
author: Optional[str] = None
|
||||
highlights: Optional[List[str]] = Field(default_factory=list, description="Key highlights from the content")
|
||||
url: str
|
||||
image: Optional[str] = None
|
||||
favicon: Optional[str] = None
|
||||
subpages: Optional[List[Dict[str, Any]]] = Field(default_factory=list, description="Subpages with their own content")
|
||||
error: Optional[str] = None
|
||||
|
||||
|
||||
class WebsiteAnalysisRequest(BaseModel):
|
||||
"""Request to save user's website analysis."""
|
||||
website_url: str = Field(..., description="The website URL")
|
||||
exa_content: Dict[str, Any] = Field(default_factory=dict, description="Exa extracted content")
|
||||
|
||||
|
||||
class WebsiteAnalysisResponse(BaseModel):
|
||||
"""Response for website analysis."""
|
||||
success: bool
|
||||
website_url: Optional[str] = None
|
||||
message: Optional[str] = None
|
||||
error: Optional[str] = None
|
||||
|
||||
|
||||
class PodcastPreEstimateRequest(BaseModel):
|
||||
"""Request model for pre-analysis cost estimate."""
|
||||
duration: int = Field(default=10, description="Target duration in minutes")
|
||||
speakers: int = Field(default=1, description="Number of speakers")
|
||||
query_count: int = Field(default=3, description="Number of research queries")
|
||||
podcast_mode: str = Field(default="audio_video", description="Podcast mode: audio_only, video_only, or audio_video")
|
||||
# Optional model overrides for cost estimation
|
||||
gemini_model: Optional[str] = Field(default=None, description="LLM model: gemini-2.5-flash, gemini-1.5-flash, etc.")
|
||||
audio_tts_model: Optional[str] = Field(default=None, description="Audio TTS model: minimax/speech-02-hd")
|
||||
voice_clone_engine: Optional[str] = Field(default=None, description="Voice clone engine: qwen3, cosyvoice, minimax")
|
||||
image_model: Optional[str] = Field(default=None, description="Image model: qwen-image, ideogram-v3-turbo")
|
||||
video_model: Optional[str] = Field(default=None, description="Video model: wan-2.5, kling-v2.5-turbo-std-5s, wavespeed-ai/infinitetalk")
|
||||
|
||||
|
||||
class PodcastPreEstimateResponse(BaseModel):
|
||||
"""Response model for pre-analysis cost estimate."""
|
||||
estimate: Optional[Dict[str, Any]] = None
|
||||
error: Optional[str] = None
|
||||
pricing_available: bool = Field(default=False, description="Whether pricing data is available in DB")
|
||||
debug: Optional[Dict[str, Any]] = Field(default=None, description="Debug info: pricing rows count, providers")
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
"""
|
||||
Prompts module for podcast topic enhancement.
|
||||
"""
|
||||
|
||||
from .website_enhance_prompts import (
|
||||
get_enhance_topic_prompt,
|
||||
format_website_context,
|
||||
STANDARD_ENHANCE_PROMPT,
|
||||
WEBSITE_AWARE_ENHANCE_PROMPT,
|
||||
)
|
||||
|
||||
from services.podcast_context_builder import (
|
||||
PodcastContextBuilder,
|
||||
context_builder,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"get_enhance_topic_prompt",
|
||||
"format_website_context",
|
||||
"STANDARD_ENHANCE_PROMPT",
|
||||
"WEBSITE_AWARE_ENHANCE_PROMPT",
|
||||
"PodcastContextBuilder",
|
||||
"context_builder",
|
||||
]
|
||||
@@ -1,187 +0,0 @@
|
||||
"""
|
||||
Website-aware prompts for podcast topic enhancement.
|
||||
|
||||
This module provides prompts for enhancing podcast topics with optional
|
||||
website extraction data for richer context.
|
||||
"""
|
||||
|
||||
from typing import Dict, Any, Optional
|
||||
from string import Template
|
||||
|
||||
|
||||
# Standard prompt for when no website data is available
|
||||
STANDARD_ENHANCE_PROMPT = Template("""">You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea.
|
||||
|
||||
${bible_context}
|
||||
|
||||
RAW IDEA/KEYWORDS: "$idea"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions, each with a unique angle:
|
||||
1. Professional & Expert-led angle (focus on authority, insights, and expertise)
|
||||
2. Storytelling & Human interest angle (focus on narratives, emotions, and personal connections)
|
||||
3. Trendy & Contemporary angle (focus on current trends, modern perspectives, and relevance)
|
||||
|
||||
Each version should be 2-3 sentences, audience-focused, and align with host persona if provided.
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 strings, each string being a complete episode pitch (NOT objects, just plain strings)
|
||||
- rationales: array of 3 strings explaining the approach for each version
|
||||
|
||||
IMPORTANT: enhanced_ideas must be an array of plain strings, NOT objects. Example:
|
||||
{
|
||||
"enhanced_ideas": [
|
||||
"Your expert guide to AI advancement: A practical look at how AI is transforming industries...",
|
||||
"The human stories behind AI innovation: From Silicon Valley to your daily life...",
|
||||
"AI in 2026: What's trending and what's next in artificial intelligence..."
|
||||
],
|
||||
"rationales": [
|
||||
"Professional approach focusing on expertise and authority",
|
||||
"Storytelling approach emphasizing human connection",
|
||||
"Contemporary approach highlighting current relevance"
|
||||
]
|
||||
}
|
||||
""")
|
||||
|
||||
|
||||
# Website-aware prompt for when website data is available
|
||||
WEBSITE_AWARE_ENHANCE_PROMPT = Template("""">You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea, enriched with website content analysis.
|
||||
|
||||
${bible_context}
|
||||
|
||||
WEBSITE CONTENT ANALYSIS:
|
||||
${website_context}
|
||||
|
||||
RAW IDEA/KEYWORDS: "$idea"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions, each with a unique angle, that INCORPORATE the website content context:
|
||||
1. Professional & Expert-led angle (focus on authority, insights, and expertise from the website)
|
||||
2. Storytelling & Human interest angle (focus on narratives, emotions, and personal connections tied to the brand)
|
||||
3. Trendy & Contemporary angle (focus on current trends, modern perspectives, and relevance leveraging the site's focus areas)
|
||||
|
||||
Each version should:
|
||||
- Be 2-3 sentences
|
||||
- Reference specific elements from the website content when relevant
|
||||
- Be audience-focused and align with host persona if provided
|
||||
- NOT just repeat the website summary - create fresh podcast angles
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 strings, each string being a complete episode pitch (NOT objects, just plain strings)
|
||||
- rationales: array of 3 strings explaining the approach for each version
|
||||
|
||||
IMPORTANT: enhanced_ideas must be an array of plain strings, NOT objects. Example:
|
||||
{
|
||||
"enhanced_ideas": [
|
||||
"Your expert guide to AI advancement: A practical look at how AI is transforming industries...",
|
||||
"The human stories behind AI innovation: From Silicon Valley to your daily life...",
|
||||
"AI in 2026: What's trending and what's next in artificial intelligence..."
|
||||
],
|
||||
"rationales": [
|
||||
"Professional approach focusing on expertise and authority",
|
||||
"Storytelling approach emphasizing human connection",
|
||||
"Contemporary approach highlighting current relevance"
|
||||
]
|
||||
}
|
||||
""")
|
||||
|
||||
|
||||
def get_enhance_topic_prompt(
|
||||
idea: str,
|
||||
bible_context: str = "",
|
||||
website_data: Optional[Dict[str, Any]] = None
|
||||
) -> str:
|
||||
"""
|
||||
Returns the appropriate prompt based on available context.
|
||||
|
||||
Args:
|
||||
idea: The raw podcast idea or keywords
|
||||
bible_context: Optional Podcast Bible context string
|
||||
website_data: Optional website extraction data
|
||||
|
||||
Returns:
|
||||
Formatted prompt string with appropriate context
|
||||
"""
|
||||
# Build bible context section
|
||||
bible_section = f"USER PERSONALIZATION CONTEXT (Podcast Bible):\n{bible_context}\n" if bible_context else ""
|
||||
|
||||
if website_data:
|
||||
# Build website context section
|
||||
website_context_parts = []
|
||||
if website_data.get('url'):
|
||||
website_context_parts.append(f"Source: {website_data.get('url')}")
|
||||
if website_data.get('title'):
|
||||
website_context_parts.append(f"Company/Organization: {website_data.get('title')}")
|
||||
if website_data.get('summary'):
|
||||
website_context_parts.append(f"About: {website_data.get('summary')}")
|
||||
if website_data.get('highlights'):
|
||||
highlights_str = ', '.join(website_data.get('highlights', [])[:3])
|
||||
website_context_parts.append(f"Key Highlights: {highlights_str}")
|
||||
if website_data.get('subpages'):
|
||||
subpages_str = ', '.join([
|
||||
sp.get('title', sp.get('url', ''))
|
||||
for sp in website_data.get('subpages', [])[:3]
|
||||
])
|
||||
website_context_parts.append(f"Subpages: {subpages_str}")
|
||||
|
||||
website_context_str = "\n".join(website_context_parts)
|
||||
|
||||
return WEBSITE_AWARE_ENHANCE_PROMPT.substitute(
|
||||
idea=idea,
|
||||
bible_context=bible_section,
|
||||
website_context=website_context_str
|
||||
)
|
||||
else:
|
||||
return STANDARD_ENHANCE_PROMPT.substitute(
|
||||
idea=idea,
|
||||
bible_context=bible_section
|
||||
)
|
||||
|
||||
|
||||
def format_website_context(website_data: Dict[str, Any]) -> str:
|
||||
"""
|
||||
Format website data for inclusion in progress messages.
|
||||
|
||||
Args:
|
||||
website_data: Website extraction data
|
||||
|
||||
Returns:
|
||||
Formatted string describing what's being used
|
||||
"""
|
||||
parts = []
|
||||
|
||||
if website_data.get('title'):
|
||||
parts.append(f"• {website_data['title']}")
|
||||
|
||||
if website_data.get('summary'):
|
||||
summary_preview = website_data['summary'][:100]
|
||||
parts.append(f"• Summary: {summary_preview}...")
|
||||
|
||||
if website_data.get('highlights'):
|
||||
parts.append(f"• {len(website_data['highlights'])} key highlights")
|
||||
|
||||
if website_data.get('subpages'):
|
||||
parts.append(f"• {len(website_data['subpages'])} subpages analyzed")
|
||||
|
||||
if website_data.get('url'):
|
||||
parts.append(f"• Source: {website_data['url']}")
|
||||
|
||||
return "\n".join(parts) if parts else "Basic website analysis"
|
||||
|
||||
if website_data.get('title'):
|
||||
parts.append(f"• {website_data['title']}")
|
||||
|
||||
if website_data.get('summary'):
|
||||
summary_preview = website_data['summary'][:100]
|
||||
parts.append(f"• Summary: {summary_preview}...")
|
||||
|
||||
if website_data.get('highlights'):
|
||||
parts.append(f"• {len(website_data['highlights'])} key highlights")
|
||||
|
||||
if website_data.get('subpages'):
|
||||
parts.append(f"• {len(website_data['subpages'])} subpages analyzed")
|
||||
|
||||
if website_data.get('url'):
|
||||
parts.append(f"• Source: {website_data['url']}")
|
||||
|
||||
return "\n".join(parts) if parts else "Basic website analysis"
|
||||
@@ -12,7 +12,7 @@ from api.story_writer.utils.auth import require_authenticated_user
|
||||
from api.story_writer.task_manager import task_manager
|
||||
|
||||
# Import all handler routers
|
||||
from .handlers import projects, analysis, research, script, audio, images, video, avatar, dubbing, broll, trends, tavily_category_research
|
||||
from .handlers import projects, analysis, research, script, audio, images, video, avatar, dubbing
|
||||
|
||||
# Create main router
|
||||
router = APIRouter(prefix="/api/podcast", tags=["Podcast Maker"])
|
||||
@@ -27,9 +27,6 @@ router.include_router(images.router)
|
||||
router.include_router(video.router)
|
||||
router.include_router(avatar.router)
|
||||
router.include_router(dubbing.router)
|
||||
router.include_router(broll.router)
|
||||
router.include_router(trends.router)
|
||||
router.include_router(tavily_category_research.router)
|
||||
|
||||
|
||||
@router.get("/task/{task_id}/status")
|
||||
|
||||
@@ -67,32 +67,15 @@ def load_podcast_audio_bytes(audio_url: str, user_id: str | None = None) -> byte
|
||||
raise HTTPException(status_code=500, detail=f"Failed to load audio: {str(exc)}")
|
||||
|
||||
|
||||
def load_podcast_image_bytes(image_url: str, user_id: str | None = None) -> bytes:
|
||||
"""Load podcast image bytes from URL. Resolves from workspace first."""
|
||||
def load_podcast_image_bytes(image_url: str) -> bytes:
|
||||
"""Load podcast image bytes from URL. Uses centralized media loader."""
|
||||
if not image_url:
|
||||
raise HTTPException(status_code=400, detail="Image URL is required")
|
||||
|
||||
logger.info(f"[Podcast] Loading image from URL: {image_url}")
|
||||
|
||||
try:
|
||||
# Extract filename from URL path
|
||||
prefix = "/api/podcast/images/"
|
||||
if prefix in image_url:
|
||||
filename = image_url.split(prefix, 1)[1].split("?", 1)[0].strip()
|
||||
# Handle subdirectories like avatars/
|
||||
subdir = None
|
||||
if "/" in filename:
|
||||
subdir_part = filename.rsplit("/", 1)[0]
|
||||
subdir = Path(subdir_part)
|
||||
filename = filename.rsplit("/", 1)[1]
|
||||
|
||||
try:
|
||||
image_path = _resolve_podcast_media_file(filename, "image", user_id, subdir=subdir)
|
||||
return image_path.read_bytes()
|
||||
except HTTPException:
|
||||
pass # Fall through to centralized loader
|
||||
|
||||
# Fall back to centralized media loader
|
||||
# REUSE: Use centralized media loader which handles cross-module lookups
|
||||
image_bytes = load_media_bytes(image_url)
|
||||
|
||||
if not image_bytes:
|
||||
|
||||
@@ -8,14 +8,9 @@ def require_authenticated_user(current_user: Dict[str, Any] | None) -> str:
|
||||
Validates the current user dictionary provided by Clerk middleware and
|
||||
returns the normalized user_id. Raises HTTP 401 if authentication fails.
|
||||
"""
|
||||
# Guard against dependency injection issues where Depends object might be passed
|
||||
if current_user is None or not isinstance(current_user, dict):
|
||||
if not current_user or not isinstance(current_user, dict):
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Authentication required")
|
||||
|
||||
# Additional check: ensure it's actually a dict and not a Depends object or other type
|
||||
if not hasattr(current_user, 'get') or not callable(getattr(current_user, 'get')):
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid authentication context")
|
||||
|
||||
|
||||
user_id = str(current_user.get("id", "")).strip()
|
||||
if not user_id:
|
||||
raise HTTPException(
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
Pre-flight check endpoints for operation validation and cost estimation.
|
||||
"""
|
||||
|
||||
import time
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
from typing import Dict, Any
|
||||
@@ -35,7 +34,6 @@ async def preflight_check(
|
||||
|
||||
Uses caching to minimize DB load (< 100ms with cache hit).
|
||||
"""
|
||||
start_time = time.time()
|
||||
try:
|
||||
user_id = get_user_id_from_token(current_user)
|
||||
|
||||
@@ -231,19 +229,13 @@ async def preflight_check(
|
||||
'remaining': max(0, video_limit - video_current) if video_limit > 0 else float('inf')
|
||||
}
|
||||
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
logger.warning(f"[PreflightCheck] Completed in {elapsed_ms:.0f}ms for user {user_id}")
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"data": response_data
|
||||
}
|
||||
|
||||
except HTTPException:
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
logger.warning(f"[PreflightCheck] HTTP error after {elapsed_ms:.0f}ms")
|
||||
raise
|
||||
except Exception as e:
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
logger.error(f"[PreflightCheck] Error after {elapsed_ms:.0f}ms: {e}")
|
||||
logger.error(f"Error in pre-flight check: {e}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Pre-flight check failed: {str(e)}")
|
||||
|
||||
563
backend/app.py
563
backend/app.py
@@ -1,12 +1,6 @@
|
||||
# Ensure typing constructs and models are available globally for FastAPI type annotation evaluation
|
||||
import os
|
||||
|
||||
# Print env vars immediately - BEFORE any imports
|
||||
print(f"[app.py] EARLY - PORT={os.getenv('PORT')}, HOST={os.getenv('HOST')}", flush=True)
|
||||
|
||||
import typing
|
||||
import builtins
|
||||
import builtins
|
||||
|
||||
# Make common typing constructs available globally
|
||||
builtins.Optional = typing.Optional
|
||||
@@ -20,19 +14,14 @@ from pathlib import Path
|
||||
from dotenv import load_dotenv
|
||||
backend_dir = Path(__file__).parent
|
||||
project_root = backend_dir.parent
|
||||
|
||||
# Load .env but DON'T override existing environment variables (especially PORT from Render)
|
||||
# Use override=False to preserve Render-provided PORT
|
||||
load_dotenv(backend_dir / '.env', override=False)
|
||||
load_dotenv(project_root / '.env', override=False)
|
||||
load_dotenv(override=False)
|
||||
load_dotenv(backend_dir / '.env')
|
||||
load_dotenv(project_root / '.env')
|
||||
load_dotenv()
|
||||
|
||||
# Set LOG_LEVEL early to WARNING to suppress DEBUG persona logs in podcast mode
|
||||
import os
|
||||
if os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast":
|
||||
os.environ["LOG_LEVEL"] = "WARNING"
|
||||
|
||||
print(f"[app.py] Starting... ALWRITY_ENABLED_FEATURES={os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||
|
||||
|
||||
def get_enabled_features() -> set:
|
||||
@@ -43,24 +32,13 @@ def get_enabled_features() -> set:
|
||||
return {f.strip() for f in env_value.split(",") if f.strip()}
|
||||
|
||||
|
||||
# Print env var IMMEDIATELY at module start
|
||||
print(f"[app.py] ALWRITY_ENABLED_FEATURES at start: {os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||
|
||||
def is_podcast_only_demo_mode() -> bool:
|
||||
"""Check if podcast-only mode is enabled."""
|
||||
import os
|
||||
env_val = os.getenv("ALWRITY_ENABLED_FEATURES", "all")
|
||||
enabled = get_enabled_features()
|
||||
result = "podcast" in enabled and "all" not in enabled
|
||||
# Removed debug print - too verbose during startup
|
||||
return result
|
||||
return "podcast" in enabled and "all" not in enabled
|
||||
|
||||
|
||||
# Podcast-only check BEFORE heavy imports
|
||||
PODCAST_ONLY_DEMO_MODE = is_podcast_only_demo_mode()
|
||||
|
||||
|
||||
# Import onboarding models (after env is loaded, before heavy imports)
|
||||
# Import onboarding models (after env is loaded)
|
||||
from models.onboarding import APIKey, WebsiteAnalysis, ResearchPreferences, PersonaData, CompetitorAnalysis
|
||||
|
||||
|
||||
@@ -76,30 +54,14 @@ import asyncio
|
||||
from datetime import datetime
|
||||
from loguru import logger
|
||||
|
||||
def _log_memory_usage():
|
||||
try:
|
||||
import psutil
|
||||
mem_mb = psutil.Process().memory_info().rss // (1024 * 1024)
|
||||
logger.info(f"Memory usage (MB): {mem_mb}")
|
||||
except Exception:
|
||||
# psutil not available or failed; skip silently
|
||||
pass
|
||||
|
||||
# Log memory early in app.py startup
|
||||
_log_memory_usage()
|
||||
logger.info("app.py: Early memory checkpoint after env load")
|
||||
|
||||
|
||||
# Import modular utilities (skip OnboardingManager import in podcast-only mode)
|
||||
from alwrity_utils import HealthChecker, RateLimiter, FrontendServing, RouterManager
|
||||
if not is_podcast_only_demo_mode():
|
||||
from alwrity_utils import OnboardingManager
|
||||
|
||||
# Skip monitoring middleware in podcast-only mode to save memory
|
||||
if not is_podcast_only_demo_mode():
|
||||
from services.subscription import monitoring_middleware
|
||||
else:
|
||||
monitoring_middleware = None
|
||||
# Import monitoring middleware
|
||||
from services.subscription import monitoring_middleware
|
||||
|
||||
|
||||
def should_include_non_podcast_features() -> bool:
|
||||
@@ -119,10 +81,8 @@ setup_clean_logging()
|
||||
# Import middleware
|
||||
from middleware.auth_middleware import get_current_user
|
||||
|
||||
# Import component logic endpoints (skip in podcast-only mode - uses seo_analyzer)
|
||||
component_logic_router = None
|
||||
if not PODCAST_ONLY_DEMO_MODE:
|
||||
from api.component_logic import router as component_logic_router
|
||||
# Import component logic endpoints (needs OnboardingSession, so import after models)
|
||||
from api.component_logic import router as component_logic_router
|
||||
|
||||
# Import subscription API endpoints
|
||||
from api.subscription import router as subscription_router
|
||||
@@ -132,60 +92,38 @@ step3_routes = None
|
||||
if not PODCAST_ONLY_DEMO_MODE:
|
||||
from api.onboarding_utils.step3_routes import router as step3_routes
|
||||
|
||||
# Import SEO tools router (skip in podcast-only mode - uses seo_analyzer)
|
||||
seo_tools_router = None
|
||||
if not PODCAST_ONLY_DEMO_MODE:
|
||||
from routers.seo_tools import router as seo_tools_router
|
||||
# Import SEO tools router
|
||||
from routers.seo_tools import router as seo_tools_router
|
||||
# Import Facebook Writer endpoints
|
||||
from api.facebook_writer.routers import facebook_router
|
||||
# Import LinkedIn content generation router
|
||||
from routers.linkedin import router as linkedin_router
|
||||
# Import LinkedIn image generation router
|
||||
from api.linkedin_image_generation import router as linkedin_image_router
|
||||
from api.brainstorm import router as brainstorm_router
|
||||
from api.images import router as images_router
|
||||
from api.assets_serving import router as assets_serving_router
|
||||
from routers.image_studio import router as image_studio_router
|
||||
from routers.product_marketing import router as product_marketing_router
|
||||
from routers.campaign_creator import router as campaign_creator_router
|
||||
|
||||
# Skip Facebook Writer, LinkedIn, and other non-podcast routes in podcast-only mode
|
||||
# Also skip other heavy services that trigger PersonaAnalysisService initialization
|
||||
if not PODCAST_ONLY_DEMO_MODE:
|
||||
from api.facebook_writer.routers import facebook_router
|
||||
from routers.linkedin import router as linkedin_router
|
||||
from api.linkedin_image_generation import router as linkedin_image_router
|
||||
from api.brainstorm import router as brainstorm_router
|
||||
from api.images import router as images_router
|
||||
from api.assets_serving import router as assets_serving_router
|
||||
from routers.image_studio import router as image_studio_router
|
||||
from routers.product_marketing import router as product_marketing_router
|
||||
from routers.campaign_creator import router as campaign_creator_router
|
||||
else:
|
||||
# In podcast-only mode, only load essential podcast assets router
|
||||
from api.assets_serving import router as assets_serving_router
|
||||
brainstorm_router = None
|
||||
images_router = None
|
||||
image_studio_router = None
|
||||
product_marketing_router = None
|
||||
campaign_creator_router = None
|
||||
# Import hallucination detector router
|
||||
from api.hallucination_detector import router as hallucination_detector_router
|
||||
from api.writing_assistant import router as writing_assistant_router
|
||||
|
||||
# Import hallucination detector router (skip in podcast-only mode - triggers heavy ML)
|
||||
if not PODCAST_ONLY_DEMO_MODE:
|
||||
from api.hallucination_detector import router as hallucination_detector_router
|
||||
from api.writing_assistant import router as writing_assistant_router
|
||||
else:
|
||||
hallucination_detector_router = None
|
||||
writing_assistant_router = None
|
||||
|
||||
# Import research configuration router (skip in podcast-only mode)
|
||||
if not is_podcast_only_demo_mode():
|
||||
from api.research_config import router as research_config_router
|
||||
else:
|
||||
research_config_router = None
|
||||
# Import research configuration router
|
||||
from api.research_config import router as research_config_router
|
||||
|
||||
# Import user data endpoints
|
||||
# Import content planning endpoints (skip in podcast-only mode)
|
||||
if not is_podcast_only_demo_mode():
|
||||
from api.content_planning.api.router import router as content_planning_router
|
||||
from api.content_planning.strategy_copilot import router as strategy_copilot_router
|
||||
else:
|
||||
content_planning_router = None
|
||||
strategy_copilot_router = None
|
||||
# Import content planning endpoints
|
||||
from api.content_planning.api.router import router as content_planning_router
|
||||
from api.user_data import router as user_data_router
|
||||
|
||||
# Import user data endpoints (skip in podcast-only mode to save memory)
|
||||
if not is_podcast_only_demo_mode():
|
||||
from api.user_data import router as user_data_router
|
||||
else:
|
||||
user_data_router = None
|
||||
# Import user environment endpoints
|
||||
from api.user_environment import router as user_environment_router
|
||||
|
||||
# Import strategy copilot endpoints
|
||||
from api.content_planning.strategy_copilot import router as strategy_copilot_router
|
||||
|
||||
# Import database service
|
||||
from services.database import close_database
|
||||
@@ -197,71 +135,39 @@ from services.startup_health import (
|
||||
|
||||
# Trigger reload for monitoring fix
|
||||
|
||||
# Import OAuth token monitoring routes (skip in podcast-only mode)
|
||||
if not is_podcast_only_demo_mode():
|
||||
from api.oauth_token_monitoring_routes import router as oauth_token_monitoring_router
|
||||
else:
|
||||
oauth_token_monitoring_router = None
|
||||
# Import OAuth token monitoring routes
|
||||
from api.oauth_token_monitoring_routes import router as oauth_token_monitoring_router
|
||||
|
||||
# Import SEO Dashboard endpoints (skip in podcast-only mode to save memory)
|
||||
if not is_podcast_only_demo_mode():
|
||||
from api.seo_dashboard import (
|
||||
get_seo_dashboard_data,
|
||||
get_seo_health_score,
|
||||
get_seo_metrics,
|
||||
get_platform_status,
|
||||
get_ai_insights,
|
||||
seo_dashboard_health_check,
|
||||
analyze_seo_comprehensive,
|
||||
analyze_seo_full,
|
||||
get_seo_metrics_detailed,
|
||||
get_analysis_summary,
|
||||
batch_analyze_urls,
|
||||
SEOAnalysisRequest,
|
||||
get_seo_dashboard_overview,
|
||||
get_gsc_raw_data,
|
||||
get_bing_raw_data,
|
||||
get_competitive_insights,
|
||||
get_deep_competitor_analysis,
|
||||
run_strategic_insights,
|
||||
get_strategic_insights_history,
|
||||
refresh_analytics_data,
|
||||
analyze_urls_ai,
|
||||
AnalyzeURLsRequest,
|
||||
get_analyzed_pages,
|
||||
get_semantic_health,
|
||||
get_semantic_cache_stats,
|
||||
get_sif_indexing_health,
|
||||
get_onboarding_task_health,
|
||||
)
|
||||
else:
|
||||
get_seo_dashboard_data = None
|
||||
get_seo_health_score = None
|
||||
get_seo_metrics = None
|
||||
get_platform_status = None
|
||||
get_ai_insights = None
|
||||
seo_dashboard_health_check = None
|
||||
analyze_seo_comprehensive = None
|
||||
analyze_seo_full = None
|
||||
get_seo_metrics_detailed = None
|
||||
get_analysis_summary = None
|
||||
batch_analyze_urls = None
|
||||
SEOAnalysisRequest = None
|
||||
get_seo_dashboard_overview = None
|
||||
get_gsc_raw_data = None
|
||||
get_bing_raw_data = None
|
||||
get_competitive_insights = None
|
||||
get_deep_competitor_analysis = None
|
||||
run_strategic_insights = None
|
||||
get_strategic_insights_history = None
|
||||
refresh_analytics_data = None
|
||||
analyze_urls_ai = None
|
||||
AnalyzeURLsRequest = None
|
||||
get_analyzed_pages = None
|
||||
get_semantic_health = None
|
||||
get_semantic_cache_stats = None
|
||||
get_sif_indexing_health = None
|
||||
get_onboarding_task_health = None
|
||||
# Import SEO Dashboard endpoints
|
||||
from api.seo_dashboard import (
|
||||
get_seo_dashboard_data,
|
||||
get_seo_health_score,
|
||||
get_seo_metrics,
|
||||
get_platform_status,
|
||||
get_ai_insights,
|
||||
seo_dashboard_health_check,
|
||||
analyze_seo_comprehensive,
|
||||
analyze_seo_full,
|
||||
get_seo_metrics_detailed,
|
||||
get_analysis_summary,
|
||||
batch_analyze_urls,
|
||||
SEOAnalysisRequest,
|
||||
get_seo_dashboard_overview,
|
||||
get_gsc_raw_data,
|
||||
get_bing_raw_data,
|
||||
get_competitive_insights,
|
||||
get_deep_competitor_analysis,
|
||||
run_strategic_insights,
|
||||
get_strategic_insights_history,
|
||||
refresh_analytics_data,
|
||||
analyze_urls_ai,
|
||||
AnalyzeURLsRequest,
|
||||
get_analyzed_pages,
|
||||
get_semantic_health,
|
||||
get_semantic_cache_stats,
|
||||
get_sif_indexing_health,
|
||||
get_onboarding_task_health,
|
||||
)
|
||||
|
||||
|
||||
# Initialize FastAPI app
|
||||
@@ -278,23 +184,12 @@ default_allowed_origins = [
|
||||
"http://localhost:8000", # Backend dev server
|
||||
"http://localhost:3001", # Alternative React port
|
||||
"https://alwrity-ai.vercel.app", # Vercel frontend
|
||||
"https://alwrity-5vac2n9su-ajsis-projects.vercel.app", # Current Vercel deployment
|
||||
"https://alwrity.vercel.app", # Vercel app
|
||||
]
|
||||
|
||||
# Optional dynamic origins from environment (comma-separated)
|
||||
env_origins = os.getenv("ALWRITY_ALLOWED_ORIGINS", "").split(",") if os.getenv("ALWRITY_ALLOWED_ORIGINS") else []
|
||||
env_origins = [o.strip() for o in env_origins if o.strip()]
|
||||
|
||||
# Convenience: NGROK_URL env var (single origin)
|
||||
ngrok_origin = os.getenv("NGROK_URL")
|
||||
if ngrok_origin:
|
||||
env_origins.append(ngrok_origin.strip())
|
||||
|
||||
# Optional dynamic origins from environment (comma-separated)
|
||||
env_origins = os.getenv("ALWRITY_ALLOWED_ORIGINS", "").split(",") if os.getenv("ALWRITY_ALLOWED_ORIGINS") else []
|
||||
env_origins = [o.strip() for o in env_origins if o.strip()]
|
||||
|
||||
# Convenience: NGROK_URL env var (single origin)
|
||||
ngrok_origin = os.getenv("NGROK_URL")
|
||||
if ngrok_origin:
|
||||
@@ -327,9 +222,8 @@ if not PODCAST_ONLY_DEMO_MODE:
|
||||
# Registration order: 1. Monitoring 2. Rate Limit 3. API Key Injection
|
||||
# Execution order: 1. API Key Injection (sets user_id) 2. Rate Limit 3. Monitoring (uses user_id)
|
||||
|
||||
# 1. FIRST REGISTERED (runs LAST) - Monitoring middleware (skip in podcast-only mode)
|
||||
if monitoring_middleware:
|
||||
app.middleware("http")(monitoring_middleware)
|
||||
# 1. FIRST REGISTERED (runs LAST) - Monitoring middleware
|
||||
app.middleware("http")(monitoring_middleware)
|
||||
|
||||
# 2. SECOND REGISTERED (runs SECOND) - Rate limiting
|
||||
@app.middleware("http")
|
||||
@@ -421,36 +315,9 @@ async def onboarding_status():
|
||||
|
||||
# Include routers using modular utilities
|
||||
if PODCAST_ONLY_DEMO_MODE:
|
||||
# In podcast-only mode, include only podcast-enabled routers from core registry
|
||||
from alwrity_utils.router_manager import CORE_ROUTER_REGISTRY
|
||||
podcast_routers = [r for r in CORE_ROUTER_REGISTRY if "podcast" in r.get("features", set())]
|
||||
logger.info(f"[PODCAST-ONLY] Found {len(podcast_routers)} podcast routers: {[r['name'] for r in podcast_routers]}")
|
||||
|
||||
# Try to include step4_assets for voice cloning (may fail if nltk not installed)
|
||||
step4_entry = next((r for r in CORE_ROUTER_REGISTRY if r.get("name") == "step4_assets"), None)
|
||||
if step4_entry:
|
||||
try:
|
||||
logger.info(f"[PODCAST-ONLY] Attempting to load step4_assets for voice cloning")
|
||||
router = router_manager._load_router_from_registry(step4_entry)
|
||||
router_manager.include_router_safely(router, step4_entry["name"], step4_entry.get("include_kwargs"))
|
||||
except ImportError as e:
|
||||
logger.warning(f"[PODCAST-ONLY] Skipping step4_assets (missing optional dependency): {e}")
|
||||
except Exception as e:
|
||||
logger.error(f"[PODCAST-ONLY] Failed to mount step4_assets: {e}")
|
||||
|
||||
# Load other podcast routers
|
||||
for entry in podcast_routers:
|
||||
if entry.get("name") == "step4_assets":
|
||||
continue # Already loaded above
|
||||
try:
|
||||
logger.info(f"[PODCAST-ONLY] Loading router: {entry['name']}")
|
||||
router = router_manager._load_router_from_registry(entry)
|
||||
router_manager.include_router_safely(router, entry["name"], entry.get("include_kwargs"))
|
||||
except Exception as e:
|
||||
logger.error(f"[PODCAST-ONLY] Failed to mount {entry.get('name', 'unknown')}: {e}")
|
||||
router_group_status["modular_core"] = {
|
||||
"mounted": True,
|
||||
"reason": "Podcast routers only in podcast-only mode",
|
||||
"mounted": False,
|
||||
"reason": "Skipped in podcast-only demo mode",
|
||||
}
|
||||
router_group_status["modular_optional"] = {
|
||||
"mounted": False,
|
||||
@@ -480,143 +347,145 @@ router_group_status["assets_serving"] = {
|
||||
"reason": "Required for podcast media assets",
|
||||
}
|
||||
|
||||
# SEO Dashboard endpoints (skip in podcast-only mode)
|
||||
if not is_podcast_only_demo_mode():
|
||||
@app.get("/api/seo-dashboard/data")
|
||||
async def seo_dashboard_data():
|
||||
"""Get complete SEO dashboard data."""
|
||||
return await get_seo_dashboard_data()
|
||||
# SEO Dashboard endpoints
|
||||
@app.get("/api/seo-dashboard/data")
|
||||
async def seo_dashboard_data():
|
||||
"""Get complete SEO dashboard data."""
|
||||
return await get_seo_dashboard_data()
|
||||
|
||||
@app.get("/api/seo-dashboard/health-score")
|
||||
async def seo_health_score():
|
||||
"""Get SEO health score."""
|
||||
return await get_seo_health_score()
|
||||
@app.get("/api/seo-dashboard/health-score")
|
||||
async def seo_health_score():
|
||||
"""Get SEO health score."""
|
||||
return await get_seo_health_score()
|
||||
|
||||
@app.get("/api/seo-dashboard/metrics")
|
||||
async def seo_metrics():
|
||||
"""Get SEO metrics."""
|
||||
return await get_seo_metrics()
|
||||
@app.get("/api/seo-dashboard/metrics")
|
||||
async def seo_metrics():
|
||||
"""Get SEO metrics."""
|
||||
return await get_seo_metrics()
|
||||
|
||||
@app.get("/api/seo-dashboard/platforms")
|
||||
async def seo_platforms(current_user: dict = Depends(get_current_user)):
|
||||
"""Get platform status."""
|
||||
return await get_platform_status(current_user)
|
||||
@app.get("/api/seo-dashboard/platforms")
|
||||
async def seo_platforms(current_user: dict = Depends(get_current_user)):
|
||||
"""Get platform status."""
|
||||
return await get_platform_status(current_user)
|
||||
|
||||
@app.get("/api/seo-dashboard/insights")
|
||||
async def seo_insights():
|
||||
"""Get AI insights."""
|
||||
return await get_ai_insights()
|
||||
@app.get("/api/seo-dashboard/insights")
|
||||
async def seo_insights():
|
||||
"""Get AI insights."""
|
||||
return await get_ai_insights()
|
||||
|
||||
@app.get("/api/seo-dashboard/overview")
|
||||
async def seo_dashboard_overview_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get comprehensive SEO dashboard overview with real GSC/Bing data."""
|
||||
return await get_seo_dashboard_overview(current_user, site_url)
|
||||
# New SEO Dashboard endpoints with real data
|
||||
@app.get("/api/seo-dashboard/overview")
|
||||
async def seo_dashboard_overview_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get comprehensive SEO dashboard overview with real GSC/Bing data."""
|
||||
return await get_seo_dashboard_overview(current_user, site_url)
|
||||
|
||||
@app.get("/api/seo-dashboard/gsc/raw")
|
||||
async def gsc_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get raw GSC data for the specified site."""
|
||||
return await get_gsc_raw_data(current_user, site_url)
|
||||
@app.get("/api/seo-dashboard/gsc/raw")
|
||||
async def gsc_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get raw GSC data for the specified site."""
|
||||
return await get_gsc_raw_data(current_user, site_url)
|
||||
|
||||
@app.get("/api/seo-dashboard/bing/raw")
|
||||
async def bing_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get raw Bing data for the specified site."""
|
||||
return await get_bing_raw_data(current_user, site_url)
|
||||
@app.get("/api/seo-dashboard/bing/raw")
|
||||
async def bing_raw_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get raw Bing data for the specified site."""
|
||||
return await get_bing_raw_data(current_user, site_url)
|
||||
|
||||
@app.get("/api/seo-dashboard/competitive-insights")
|
||||
async def competitive_insights_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get competitive insights from onboarding step 3 data."""
|
||||
return await get_competitive_insights(current_user, site_url)
|
||||
@app.get("/api/seo-dashboard/competitive-insights")
|
||||
async def competitive_insights_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get competitive insights from onboarding step 3 data."""
|
||||
return await get_competitive_insights(current_user, site_url)
|
||||
|
||||
@app.get("/api/seo-dashboard/deep-competitor-analysis")
|
||||
async def deep_competitor_analysis_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get deep competitor analysis results (auto-scheduled post-onboarding)."""
|
||||
return await get_deep_competitor_analysis(current_user, site_url)
|
||||
@app.get("/api/seo-dashboard/deep-competitor-analysis")
|
||||
async def deep_competitor_analysis_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get deep competitor analysis results (auto-scheduled post-onboarding)."""
|
||||
return await get_deep_competitor_analysis(current_user, site_url)
|
||||
|
||||
@app.post("/api/seo-dashboard/strategic-insights/run")
|
||||
async def run_strategic_insights_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""Run AI-powered strategic insights analysis manually."""
|
||||
return await run_strategic_insights(current_user)
|
||||
@app.post("/api/seo-dashboard/strategic-insights/run")
|
||||
async def run_strategic_insights_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""Run AI-powered strategic insights analysis manually."""
|
||||
return await run_strategic_insights(current_user)
|
||||
|
||||
@app.get("/api/seo-dashboard/strategic-insights/history")
|
||||
async def get_strategic_insights_history_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""Fetch the history of strategic insights for the user."""
|
||||
return await get_strategic_insights_history(current_user)
|
||||
@app.get("/api/seo-dashboard/strategic-insights/history")
|
||||
async def get_strategic_insights_history_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""Fetch the history of strategic insights for the user."""
|
||||
return await get_strategic_insights_history(current_user)
|
||||
|
||||
@app.post("/api/seo-dashboard/refresh")
|
||||
async def refresh_analytics_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Refresh analytics data by invalidating cache and fetching fresh data."""
|
||||
return await refresh_analytics_data(current_user, site_url)
|
||||
@app.post("/api/seo-dashboard/refresh")
|
||||
async def refresh_analytics_data_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Refresh analytics data by invalidating cache and fetching fresh data."""
|
||||
return await refresh_analytics_data(current_user, site_url)
|
||||
|
||||
|
||||
@app.get("/api/seo-dashboard/onboarding-task-health")
|
||||
async def onboarding_task_health_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get consolidated health for onboarding-scheduled SEO tasks."""
|
||||
return await get_onboarding_task_health(current_user, site_url)
|
||||
|
||||
@app.get("/api/seo-dashboard/health")
|
||||
async def seo_dashboard_health():
|
||||
"""Health check for SEO dashboard."""
|
||||
return await seo_dashboard_health_check()
|
||||
@app.get("/api/seo-dashboard/onboarding-task-health")
|
||||
async def onboarding_task_health_endpoint(current_user: dict = Depends(get_current_user), site_url: str = None):
|
||||
"""Get consolidated health for onboarding-scheduled SEO tasks."""
|
||||
return await get_onboarding_task_health(current_user, site_url)
|
||||
|
||||
@app.get("/api/seo-dashboard/semantic-health")
|
||||
async def semantic_health_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""
|
||||
Get real-time semantic health metrics for content and competitors.
|
||||
This endpoint provides Phase 2B semantic intelligence monitoring data.
|
||||
|
||||
Returns semantic health score, status, and recommendations.
|
||||
Data is cached and updated every 24 hours via scheduler.
|
||||
"""
|
||||
return await get_semantic_health(current_user)
|
||||
@app.get("/api/seo-dashboard/health")
|
||||
async def seo_dashboard_health():
|
||||
"""Health check for SEO dashboard."""
|
||||
return await seo_dashboard_health_check()
|
||||
|
||||
# Phase 2B: Semantic health monitoring endpoint (24-hour polling)
|
||||
@app.get("/api/seo-dashboard/semantic-health")
|
||||
async def semantic_health_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""
|
||||
Get real-time semantic health metrics for content and competitors.
|
||||
This endpoint provides Phase 2B semantic intelligence monitoring data.
|
||||
|
||||
Returns semantic health score, status, and recommendations.
|
||||
Data is cached and updated every 24 hours via scheduler.
|
||||
"""
|
||||
return await get_semantic_health(current_user)
|
||||
|
||||
|
||||
@app.get("/api/seo-dashboard/cache-stats")
|
||||
async def semantic_cache_stats_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""
|
||||
Get semantic cache performance statistics.
|
||||
Returns hit rate, memory usage, and eviction counts.
|
||||
"""
|
||||
return await get_semantic_cache_stats(current_user)
|
||||
@app.get("/api/seo-dashboard/cache-stats")
|
||||
async def semantic_cache_stats_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""
|
||||
Get semantic cache performance statistics.
|
||||
Returns hit rate, memory usage, and eviction counts.
|
||||
"""
|
||||
return await get_semantic_cache_stats(current_user)
|
||||
|
||||
|
||||
@app.get("/api/seo-dashboard/sif-health")
|
||||
async def sif_indexing_health_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""
|
||||
Get SIF indexing health summary for the current user.
|
||||
Used by the Semantic Indexing Status widget on the dashboard.
|
||||
"""
|
||||
return await get_sif_indexing_health(current_user)
|
||||
@app.get("/api/seo-dashboard/sif-health")
|
||||
async def sif_indexing_health_endpoint(current_user: dict = Depends(get_current_user)):
|
||||
"""
|
||||
Get SIF indexing health summary for the current user.
|
||||
Used by the Semantic Indexing Status widget on the dashboard.
|
||||
"""
|
||||
return await get_sif_indexing_health(current_user)
|
||||
|
||||
# Comprehensive SEO Analysis endpoints
|
||||
@app.post("/api/seo-dashboard/analyze-comprehensive")
|
||||
async def analyze_seo_comprehensive_endpoint(request: SEOAnalysisRequest):
|
||||
"""Analyze a URL for comprehensive SEO performance."""
|
||||
return await analyze_seo_comprehensive(request)
|
||||
# Comprehensive SEO Analysis endpoints
|
||||
@app.post("/api/seo-dashboard/analyze-comprehensive")
|
||||
async def analyze_seo_comprehensive_endpoint(request: SEOAnalysisRequest):
|
||||
"""Analyze a URL for comprehensive SEO performance."""
|
||||
return await analyze_seo_comprehensive(request)
|
||||
|
||||
@app.post("/api/seo-dashboard/analyze-full")
|
||||
async def analyze_seo_full_endpoint(request: SEOAnalysisRequest):
|
||||
"""Analyze a URL for comprehensive SEO performance."""
|
||||
return await analyze_seo_full(request)
|
||||
@app.post("/api/seo-dashboard/analyze-full")
|
||||
async def analyze_seo_full_endpoint(request: SEOAnalysisRequest):
|
||||
"""Analyze a URL for comprehensive SEO performance."""
|
||||
return await analyze_seo_full(request)
|
||||
|
||||
@app.get("/api/seo-dashboard/metrics-detailed")
|
||||
async def seo_metrics_detailed(url: str):
|
||||
"""Get detailed SEO metrics for a URL."""
|
||||
return await get_seo_metrics_detailed(url)
|
||||
@app.get("/api/seo-dashboard/metrics-detailed")
|
||||
async def seo_metrics_detailed(url: str):
|
||||
"""Get detailed SEO metrics for a URL."""
|
||||
return await get_seo_metrics_detailed(url)
|
||||
|
||||
@app.get("/api/seo-dashboard/analysis-summary")
|
||||
async def seo_analysis_summary(url: str):
|
||||
"""Get a quick summary of SEO analysis for a URL."""
|
||||
return await get_analysis_summary(url)
|
||||
@app.get("/api/seo-dashboard/analysis-summary")
|
||||
async def seo_analysis_summary(url: str):
|
||||
"""Get a quick summary of SEO analysis for a URL."""
|
||||
return await get_analysis_summary(url)
|
||||
|
||||
@app.post("/api/seo-dashboard/batch-analyze")
|
||||
async def batch_analyze_urls_endpoint(urls: list[str]):
|
||||
"""Analyze multiple URLs in batch."""
|
||||
return await batch_analyze_urls(urls)
|
||||
@app.post("/api/seo-dashboard/batch-analyze")
|
||||
async def batch_analyze_urls_endpoint(urls: list[str]):
|
||||
"""Analyze multiple URLs in batch."""
|
||||
return await batch_analyze_urls(urls)
|
||||
|
||||
@app.post("/api/seo-dashboard/analyze-urls-ai")
|
||||
async def analyze_urls_ai_endpoint(request: AnalyzeURLsRequest, current_user: dict = Depends(get_current_user)):
|
||||
"""Run AI-powered SEO analysis on selected URLs."""
|
||||
return await analyze_urls_ai(request, current_user)
|
||||
@app.post("/api/seo-dashboard/analyze-urls-ai")
|
||||
async def analyze_urls_ai_endpoint(request: AnalyzeURLsRequest, current_user: dict = Depends(get_current_user)):
|
||||
"""Run AI-powered SEO analysis on selected URLs."""
|
||||
return await analyze_urls_ai(request, current_user)
|
||||
|
||||
# Include platform analytics router
|
||||
if not PODCAST_ONLY_DEMO_MODE:
|
||||
@@ -625,14 +494,10 @@ if not PODCAST_ONLY_DEMO_MODE:
|
||||
# Include Bing Analytics Storage router to expose storage-backed endpoints
|
||||
from routers.bing_analytics_storage import router as bing_analytics_storage_router
|
||||
app.include_router(bing_analytics_storage_router)
|
||||
if images_router:
|
||||
app.include_router(images_router)
|
||||
if image_studio_router:
|
||||
app.include_router(image_studio_router)
|
||||
if product_marketing_router:
|
||||
app.include_router(product_marketing_router)
|
||||
if campaign_creator_router:
|
||||
app.include_router(campaign_creator_router)
|
||||
app.include_router(images_router)
|
||||
app.include_router(image_studio_router)
|
||||
app.include_router(product_marketing_router)
|
||||
app.include_router(campaign_creator_router)
|
||||
|
||||
# Include content assets router
|
||||
from api.content_assets.router import router as content_assets_router
|
||||
@@ -647,9 +512,8 @@ else:
|
||||
"reason": "Skipped in podcast-only demo mode",
|
||||
}
|
||||
|
||||
# Include Podcast Maker router (always needed for podcast mode)
|
||||
# Include Podcast Maker router
|
||||
from api.podcast.router import router as podcast_router
|
||||
logger.info(f"[PODCAST] Including podcast_router with prefixes: {podcast_router.routes}")
|
||||
app.include_router(podcast_router)
|
||||
router_group_status["podcast_maker"] = {
|
||||
"mounted": True,
|
||||
@@ -671,8 +535,7 @@ if not PODCAST_ONLY_DEMO_MODE:
|
||||
# Scheduler dashboard routes
|
||||
from api.scheduler_dashboard import router as scheduler_dashboard_router
|
||||
app.include_router(scheduler_dashboard_router)
|
||||
if oauth_token_monitoring_router:
|
||||
app.include_router(oauth_token_monitoring_router)
|
||||
app.include_router(oauth_token_monitoring_router)
|
||||
|
||||
# Autonomous Agents API routes (Phase 3A)
|
||||
from api.agents_api import router as agents_router
|
||||
@@ -700,28 +563,14 @@ async def serve_frontend():
|
||||
"""Serve the React frontend."""
|
||||
return frontend_serving.serve_frontend()
|
||||
|
||||
# Startup event - fires AFTER port is bound
|
||||
# Startup event
|
||||
@app.on_event("startup")
|
||||
async def startup_event():
|
||||
"""Initialize services on startup."""
|
||||
import time
|
||||
startup_start = time.time()
|
||||
|
||||
logger.info("[STARTUP] Server port bound, beginning background initialization...")
|
||||
|
||||
try:
|
||||
_log_memory_usage()
|
||||
|
||||
# Note: Pricing is initialized per-user in services/database.py:init_user_database()
|
||||
# which runs on first database access for each user. No global seeding needed at startup.
|
||||
|
||||
# Skip startup health checks in podcast-only mode to avoid unnecessary DB errors
|
||||
if not is_podcast_only_demo_mode():
|
||||
startup_report = run_startup_health_routine(app)
|
||||
if startup_report.get("status") != "healthy":
|
||||
logger.error(f"Startup readiness finished with failures: {startup_report.get('errors', [])}")
|
||||
else:
|
||||
logger.info("[Podcast] Skipping startup health routine (podcast-only mode)")
|
||||
startup_report = run_startup_health_routine(app)
|
||||
if startup_report.get("status") != "healthy":
|
||||
logger.error(f"Startup readiness finished with failures: {startup_report.get('errors', [])}")
|
||||
|
||||
# Start task scheduler only if NOT in podcast-only mode
|
||||
if not is_podcast_only_demo_mode():
|
||||
@@ -737,15 +586,14 @@ async def startup_event():
|
||||
else:
|
||||
logger.warning("⚠️ WIX_API_KEY not found in environment - Wix publishing may fail")
|
||||
|
||||
elapsed = time.time() - startup_start
|
||||
logger.info(f"ALwrity backend started successfully in {elapsed:.1f}s")
|
||||
logger.info("ALwrity backend started successfully")
|
||||
|
||||
# Critical router mount assertions for podcast-only demo mode
|
||||
_assert_router_mounted("subscription")
|
||||
_assert_router_mounted("podcast")
|
||||
except Exception as e:
|
||||
logger.error(f"Error during startup: {e}")
|
||||
# Don't raise - let the server start anyway
|
||||
raise
|
||||
|
||||
|
||||
def _assert_router_mounted(router_name: str) -> None:
|
||||
@@ -785,19 +633,4 @@ async def shutdown_event():
|
||||
close_database()
|
||||
logger.info("ALwrity backend shutdown successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Error during shutdown: {e}")
|
||||
|
||||
|
||||
# Add main block to allow running directly with: python app.py
|
||||
# This also helps Gunicorn work correctly
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
port = int(os.environ.get("PORT", "10000"))
|
||||
host = os.environ.get("HOST", "0.0.0.0")
|
||||
|
||||
print(f"[app.py] ====================", flush=True)
|
||||
print(f"[app.py] DIRECT STARTUP", flush=True)
|
||||
print(f"[app.py] PORT={port}, HOST={host}", flush=True)
|
||||
print(f"[app.py] ====================", flush=True)
|
||||
|
||||
uvicorn.run(app, host=host, port=port)
|
||||
logger.error(f"Error during shutdown: {e}")
|
||||
|
||||
@@ -1,197 +0,0 @@
|
||||
# Agent Flat-File Context System Review
|
||||
|
||||
## Scope
|
||||
This review documents the **current implementation** of ALwrity's onboarding flat-file context system and compares it to the proposed **Direct-to-File Virtual Shell (VFS)** model.
|
||||
|
||||
---
|
||||
|
||||
## 1) Present Implementation (What Exists Today)
|
||||
|
||||
### 1.1 Storage model
|
||||
- Context is stored per user under:
|
||||
- `backend/workspace/workspace_<safe_user_id>/agent_context/`
|
||||
- Files are JSON documents, one per onboarding domain:
|
||||
- `step2_website_analysis.json`
|
||||
- `step3_research_preferences.json`
|
||||
- `step4_persona_data.json`
|
||||
- `step5_integrations.json`
|
||||
- `context_manifest.json`
|
||||
|
||||
### 1.2 Writer and reader
|
||||
- `AgentFlatContextStore` is the core component that:
|
||||
- sanitizes user IDs for path safety,
|
||||
- writes documents atomically (`tempfile` + `os.replace`),
|
||||
- sets restrictive file permissions (`0600` best effort),
|
||||
- generates structured `agent_summary` objects,
|
||||
- updates a manifest index of available documents.
|
||||
- Data is loaded by direct file reads from the same class (`load_stepX_context_document`).
|
||||
|
||||
### 1.3 Read-path fallback chain
|
||||
`SIFIntegrationService` uses a strict fallback sequence for onboarding context retrieval:
|
||||
1. **flat file** (`AgentFlatContextStore`)
|
||||
2. **database** (`WebsiteAnalysis`, `ResearchPreferences`, `PersonaData`, etc.)
|
||||
3. **SIF semantic index** (`TxtaiIntelligenceService.search`)
|
||||
|
||||
Step 5 uses `flat_file -> sif_semantic`.
|
||||
|
||||
### 1.4 Producer flow (onboarding persistence)
|
||||
`StepManagementService` persists canonical snapshots to flat context when onboarding steps are saved:
|
||||
- Step 2 website analysis
|
||||
- Step 3 research preferences (and later competitor-enriched refresh)
|
||||
- Step 4 persona data
|
||||
- Step 5 integrations
|
||||
|
||||
### 1.5 Context optimization currently implemented
|
||||
- Sensitive-key redaction in nested payloads (`api_key`, `token`, `secret`, etc.).
|
||||
- Size budgeting with trimming (`DEFAULT_MAX_BYTES = 300_000`) and trim metadata.
|
||||
- Generated summaries include:
|
||||
- quick facts,
|
||||
- retrieval hints (high-signal terms and suggested agent queries),
|
||||
- domain-specific focus blocks.
|
||||
- Document context includes audience, retrieval contract, journey stage, related documents, and context-window guidance.
|
||||
|
||||
---
|
||||
|
||||
## 2) Comparison vs Proposed Direct-to-File VFS
|
||||
|
||||
## Strong alignment
|
||||
The current system already matches the proposal in important ways:
|
||||
- **Direct-to-file persistence** instead of DB-backed retrieval for fast reads.
|
||||
- **Manifest/index concept** (`context_manifest.json`) that can act like a precomputed path map.
|
||||
- **Agent-first retrieval semantics** (summary-first contract and fallback policy).
|
||||
- **Operational safety controls** (atomic writes, redaction, path sanitization).
|
||||
|
||||
## Gaps vs full virtual shell abstraction
|
||||
The following pieces are not fully implemented as described in your proposed architecture:
|
||||
- No explicit **virtual shell provider** (`IFileSystem`) exposing `ls/cat/grep/find` commands.
|
||||
- No always-live, process-level **in-memory `Map<virtualPath, absolutePath>`** for path lookups.
|
||||
- No native glob/query command layer for agent shell UX.
|
||||
- Not currently **read-only enforced at API surface** (writes are intentionally allowed by onboarding services to refresh context).
|
||||
|
||||
---
|
||||
|
||||
## 3) Practical Recommendation: Incremental VFS Evolution
|
||||
|
||||
1. **Introduce a read-only VFS facade for agents**
|
||||
- Keep `AgentFlatContextStore` as the write path for trusted onboarding services.
|
||||
- Add `AgentContextVFS` read adapter exposing:
|
||||
- `ls(path)` from manifest,
|
||||
- `cat(path)` mapped to underlying JSON,
|
||||
- `find(glob)` on virtual keys,
|
||||
- `grep(query)` with path prefilter + stream scan.
|
||||
|
||||
2. **Promote manifest to a first-class path map**
|
||||
- Build and cache an in-memory map on service startup or first access.
|
||||
- Refresh map when manifest `updated_at` changes.
|
||||
|
||||
3. **Add explicit write policy boundaries**
|
||||
- Agent-facing interface: hard read-only (`EROFS`).
|
||||
- Internal system service interface: allow writes for onboarding synchronization.
|
||||
|
||||
4. **Metadata strategy for grep ranking**
|
||||
- Prioritize in order:
|
||||
1) `agent_summary.quick_facts`
|
||||
2) `agent_summary.retrieval_hints.high_signal_terms`
|
||||
3) `document_context.context_type` and `journey.stage`
|
||||
4) full `data` body
|
||||
|
||||
---
|
||||
|
||||
## 4) Response to the Metadata Header Question
|
||||
|
||||
> "Does your current `.txt` optimization include specific metadata headers (like YAML frontmatter) that the grep tool should prioritize?"
|
||||
|
||||
For this implementation, context is currently persisted as structured JSON (not `.txt` with YAML frontmatter). Equivalent high-value metadata already exists and should be prioritized for search/ranking:
|
||||
- `context_type`
|
||||
- `updated_at`
|
||||
- `agent_summary.quick_facts`
|
||||
- `agent_summary.retrieval_hints.high_signal_terms`
|
||||
- `document_context.journey.stage`
|
||||
- `document_context.related_documents`
|
||||
|
||||
If you later move to `.txt` transport files, mirror these as frontmatter fields to preserve retrieval quality.
|
||||
|
||||
---
|
||||
|
||||
## 5) Bottom line
|
||||
Your current onboarding flat-file context implementation is already a strong "shim" architecture and close to the proposed model. The biggest missing piece is a dedicated virtual-shell read interface (`ls/cat/grep/find`) backed by a persistent path-map cache and a clear read-only contract for agent execution contexts.
|
||||
|
||||
---
|
||||
|
||||
## 6) Implemented Follow-up (VFS Adapter + Workspace Guide)
|
||||
|
||||
The following enhancements are now implemented:
|
||||
|
||||
1. **Auto-generated workspace map**
|
||||
- The system now generates `workspace_<user>/README.md` whenever `context_manifest.json` is updated.
|
||||
- The README includes:
|
||||
- available context files,
|
||||
- key signal hints from `agent_summary.retrieval_hints.high_signal_terms`,
|
||||
- journey-stage hints,
|
||||
- virtual path mappings and retrieval strategy guidance.
|
||||
|
||||
2. **Read-only VFS facade**
|
||||
- Added `AgentContextVFS` with:
|
||||
- `list_context()` (`ls` equivalent),
|
||||
- `search_context()` (`grep` equivalent; prioritizes `high_signal_terms` and `quick_facts`),
|
||||
- `read_context_file()` (`cat` equivalent; large-file summary mode + subkey drilldown),
|
||||
- explicit write rejection (`EROFS`).
|
||||
|
||||
3. **Virtual path support**
|
||||
- `/env/summary` maps to `AgentFlatContextStore.generate_total_summary()`.
|
||||
- `/steps/website`, `/steps/research`, `/steps/persona`, `/steps/integrations` map to step documents.
|
||||
|
||||
4. **System-prompt helper**
|
||||
- Added `build_filesystem_header(user_id)` to inject a compact file availability + priority hint block into agent startup prompts.
|
||||
|
||||
5. **Merged context helper in SIF integration**
|
||||
- `SIFIntegrationService.get_merged_flat_context()` now provides a unified view across all available flat files while preserving existing per-step retrieval methods.
|
||||
|
||||
6. **Basic file-level security hardening**
|
||||
- Workspace and context directories are now explicitly forced to `0700`.
|
||||
- Context and workspace files are written with strict `0600`.
|
||||
- Added path sandboxing to ensure requested paths cannot escape user workspace roots.
|
||||
- Restricted context-file loading to an allowlist of known onboarding context documents.
|
||||
- Added deterministic per-user secret derivation from `.env` (`FILE_ENCRYPTION_SALT` + `safe_user_id`) with non-sensitive fingerprints for audit/debug and future encryption-at-rest rollout.
|
||||
|
||||
7. **Tool-logic enhancement (coarse-to-fine search)**
|
||||
- `search_context` now performs a two-pass retrieval:
|
||||
1) high-relevance summary match pass (`high_signal_terms`, `quick_facts`),
|
||||
2) parallelized stream scan pass over sandboxed allowlisted files for supporting details.
|
||||
- Results include relevance labels, snippets, and line numbers for body matches.
|
||||
- Large-result behavior now reports truncation guidance (show top 10 and suggest narrower keywords).
|
||||
- `inspect_file` now provides token-saving behavior: full return for small files, or `agent_summary` + top-level keys for larger files, with key-level zoom-in support.
|
||||
|
||||
8. **Retrieval robustness roadmap (next hardening phase)**
|
||||
- **Query normalization:** Add synonym expansion and typo-tolerant matching (e.g., `tone` ≈ `brand voice`) before coarse/fine passes.
|
||||
- **Confidence scoring:** Return confidence tiers that blend source freshness (`updated_at`), summary-match strength, and match density.
|
||||
- **Field-aware boosting:** Weight matches by field priority (`high_signal_terms` > `quick_facts` > `data`) and document recency.
|
||||
- **Deduplicated evidence:** Collapse repeated hits from the same file/key into one clustered result with a single best snippet and hit count.
|
||||
- **Fallback query reformulation:** If zero hits, automatically retry with narrow/expanded variants and return attempted queries.
|
||||
- **Answerability contract:** Add a lightweight `can_answer` signal in search responses so orchestrators can decide whether to ask follow-up questions or fetch more context.
|
||||
- **Evaluation harness:** Track retrieval metrics over golden queries (`precision@k`, `MRR`, zero-hit rate, stale-hit rate) in CI to prevent relevance regressions.
|
||||
|
||||
9. **Collaborative VFS namespace (shared memory mode)**
|
||||
- Added optional `project_id` support to `AgentContextVFS` with isolated root: `workspace/project_<project_id>/`.
|
||||
- Introduced `scratchpad/` for collaborative writes while keeping onboarding `agent_context` read-first.
|
||||
- Added `write_shared_note(...)` with advisory locking (`flock`) and strict filename/path validation.
|
||||
- Added append-only `activity_log.jsonl` via `append_activity_log(...)` for watchdog/event-driven coordination.
|
||||
- Maintains owner-only permissions (`0700` scratchpad dir, `0600` files) and audit trails for shared writes.
|
||||
|
||||
10. **Testing readiness upgrades**
|
||||
- Added automated tests for:
|
||||
- query reformulation + `can_answer` behavior in `search_context`,
|
||||
- large-file progressive disclosure behavior in `inspect_file`,
|
||||
- collaborative write path (`write_shared_note`) and append-only activity logging.
|
||||
- Test module: `backend/tests/test_agent_context_vfs.py`.
|
||||
- These tests provide a baseline regression harness for VFS retrieval quality and shared-memory safety.
|
||||
|
||||
11. **Static + Structural retrieval hardening**
|
||||
- Added a **static triage layer** in `search_context`:
|
||||
- keyword-density scoring,
|
||||
- `low_probability` flags for likely-noisy hits,
|
||||
- `triage_top5` shortlist for router-style pre-filtering.
|
||||
- Added `read_struct(filename, path_query)`:
|
||||
- resolves dot/bracket JSON paths to return node-level data only,
|
||||
- includes lightweight dependency injection (e.g., Step 4 persona reads include Step 2 brand voice context when available),
|
||||
- keeps output token-efficient for downstream agents.
|
||||
@@ -1 +0,0 @@
|
||||
{'🎙', '🛑', '🚀', '📖', '💳', '📈', '🌐', '📊', '📦', '🔧', '🔍'}
|
||||
@@ -1,46 +0,0 @@
|
||||
"""Gunicorn configuration for Render deployment."""
|
||||
import os
|
||||
import multiprocessing
|
||||
|
||||
# Bind to the port Render provides
|
||||
bind = f"0.0.0.0:{os.getenv('PORT', '10000')}"
|
||||
|
||||
# Use uvicorn workers
|
||||
worker_class = "uvicorn.workers.UvicornWorker"
|
||||
|
||||
# Single worker for memory efficiency on free tier
|
||||
workers = 1
|
||||
|
||||
# Timeout for slow startup (10 minutes to allow for model loading)
|
||||
timeout = 600
|
||||
|
||||
# Graceful timeout
|
||||
graceful_timeout = 30
|
||||
|
||||
# Keepalive
|
||||
keepalive = 5
|
||||
|
||||
# Logging
|
||||
accesslog = "-"
|
||||
errorlog = "-"
|
||||
loglevel = os.getenv("LOG_LEVEL", "info").lower()
|
||||
|
||||
# Don't preload - bind to port FIRST, then load worker
|
||||
preload_app = False
|
||||
|
||||
# Use the startup script that handles all the logic
|
||||
factory = False # app:app is not a factory, it's the app object
|
||||
|
||||
def on_starting(server):
|
||||
"""Called just before the master process is initialized."""
|
||||
print(f"[GUNICORN] Starting on {bind}", flush=True)
|
||||
|
||||
|
||||
def on_reload(server):
|
||||
"""Called when worker is reloaded."""
|
||||
print(f"[GUNICORN] Reloading workers", flush=True)
|
||||
|
||||
|
||||
def when_ready(server):
|
||||
"""Called just after the server is started."""
|
||||
print(f"[GUNICORN] Server is ready. Accepting connections.", flush=True)
|
||||
@@ -45,9 +45,6 @@ class PodcastProject(Base):
|
||||
knobs = Column(JSON, nullable=True) # Knobs settings
|
||||
research_provider = Column(String(50), nullable=True, default="google") # Research provider
|
||||
|
||||
# Project-specific topic context (category research, selected topics)
|
||||
topic_context = Column(JSON, nullable=True) # { category: "news"|"finance", topics: [...], selected_topic: {...} }
|
||||
|
||||
# UI state
|
||||
show_script_editor = Column(Boolean, default=False)
|
||||
show_render_queue = Column(Boolean, default=False)
|
||||
|
||||
@@ -80,7 +80,6 @@ class SubscriptionPlan(Base):
|
||||
video_calls_limit = Column(Integer, default=0) # AI video generation
|
||||
image_edit_calls_limit = Column(Integer, default=0) # AI image editing
|
||||
audio_calls_limit = Column(Integer, default=0) # AI audio generation (text-to-speech)
|
||||
wavespeed_calls_limit = Column(Integer, default=0) # WaveSpeed API calls (LLM + TTS + video + image)
|
||||
|
||||
# Token Limits (for LLM providers)
|
||||
gemini_tokens_limit = Column(Integer, default=0)
|
||||
|
||||
@@ -1,30 +1,9 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
echo "🚀 Starting ALwrity Build Process..."
|
||||
|
||||
# 1. Update pip and essential build tools
|
||||
python -m pip install --upgrade pip setuptools wheel
|
||||
python -m pip install --retries 10 --timeout 120 -r requirements.txt
|
||||
|
||||
# 2. Install requirements based on mode
|
||||
echo "📦 Checking ALWRITY_ENABLED_FEATURES..."
|
||||
ENABLED_FEATURES="${ALWRITY_ENABLED_FEATURES:-all}"
|
||||
echo "DEBUG: ENABLED_FEATURES='$ENABLED_FEATURES'"
|
||||
|
||||
if [[ "$ENABLED_FEATURES" == "podcast" ]]; then
|
||||
echo "🔊 Podcast-only mode: Installing lean requirements..."
|
||||
python -m pip install --no-cache-dir -r requirements-podcast.txt --only-binary :all: --retries 10 --timeout 120
|
||||
else
|
||||
echo "📦 Full mode: Installing all requirements..."
|
||||
python -m pip install --no-cache-dir -r requirements.txt --only-binary :all: --retries 10 --timeout 120
|
||||
# Download spaCy/NLTK models for full mode
|
||||
echo "🧠 Installing spaCy and NLTK models..."
|
||||
python -m spacy download en_core_web_sm
|
||||
python -m nltk.downloader punkt_tab stopwords averaged_perceptron_tagger
|
||||
fi
|
||||
|
||||
# 3. Clean up unnecessary build artifacts
|
||||
find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
|
||||
rm -rf /root/.cache/pip 2>/dev/null || true
|
||||
|
||||
echo "✅ Build Complete!"
|
||||
# Download required NLTK and spaCy models during build phase
|
||||
python -m spacy download en_core_web_sm
|
||||
python -m nltk.downloader punkt_tab stopwords averaged_perceptron_tagger
|
||||
|
||||
@@ -1,82 +0,0 @@
|
||||
# =====================================================
|
||||
# ALwrity Podcast-Only Requirements
|
||||
# Lean subset for podcast-only demo mode
|
||||
# =====================================================
|
||||
|
||||
# Core Web Server
|
||||
fastapi>=0.115.14
|
||||
starlette>=0.40.0,<0.47.0
|
||||
sse-starlette<3.0.0
|
||||
uvicorn>=0.24.0
|
||||
uvicorn[standard]>=0.24.0
|
||||
gunicorn>=21.0.0
|
||||
|
||||
# Server utilities
|
||||
python-multipart>=0.0.6
|
||||
python-dotenv>=1.0.0
|
||||
loguru>=0.7.2
|
||||
tenacity>=8.2.3
|
||||
pydantic>=2.5.2,<3.0.0
|
||||
typing-extensions>=4.8.0
|
||||
setuptools>=65.0.0
|
||||
|
||||
# Auth & Database
|
||||
fastapi-clerk-auth>=0.0.7
|
||||
sqlalchemy>=2.0.25
|
||||
|
||||
# Payment
|
||||
stripe>=8.0.0
|
||||
|
||||
# HTTP clients
|
||||
httpx>=0.28.1
|
||||
aiohttp>=3.9.0
|
||||
requests>=2.31.0
|
||||
|
||||
# AI - needed for podcast
|
||||
openai>=1.3.0
|
||||
google-genai>=1.0.0
|
||||
exa-py==1.9.1
|
||||
|
||||
# Text processing (minimal)
|
||||
markdown>=3.5.0
|
||||
beautifulsoup4>=4.12.0
|
||||
|
||||
# Data processing (numpy needed for moviepy, pandas for usage tracking)
|
||||
numpy>=1.24.0
|
||||
pandas>=2.0.0
|
||||
|
||||
# Image/media for podcast
|
||||
Pillow>=10.0.0
|
||||
matplotlib>=3.7.0
|
||||
huggingface_hub>=1.1.4
|
||||
|
||||
# TTS for podcast
|
||||
gtts>=2.4.0
|
||||
pyttsx3>=2.90
|
||||
|
||||
# Video composition
|
||||
moviepy==2.1.2
|
||||
imageio>=2.31.0
|
||||
imageio-ffmpeg>=0.4.9
|
||||
|
||||
# Testing
|
||||
pytest>=7.4.0
|
||||
pytest-asyncio>=0.21.0
|
||||
|
||||
# Task scheduling
|
||||
apscheduler>=3.10.0
|
||||
|
||||
# Utilities
|
||||
redis>=5.0.0
|
||||
schedule>=1.2.0
|
||||
aiofiles>=23.2.0
|
||||
psutil>=5.9.0
|
||||
|
||||
# Google APIs
|
||||
google-api-python-client>=2.100.0
|
||||
google-auth>=2.23.0
|
||||
google-auth-oauthlib>=1.0.0
|
||||
|
||||
# Other utilities
|
||||
python-dateutil>=2.8.0
|
||||
jinja2>=3.1.0
|
||||
@@ -1,81 +1,93 @@
|
||||
# Core dependencies - needed for all modes
|
||||
# Core dependencies
|
||||
fastapi>=0.115.14
|
||||
starlette>=0.40.0,<0.47.0
|
||||
sse-starlette<3.0.0
|
||||
uvicorn>=0.24.0
|
||||
uvicorn[standard]>=0.24.0
|
||||
gunicorn>=21.0.0
|
||||
python-multipart>=0.0.6
|
||||
python-dotenv>=1.0.0
|
||||
loguru>=0.7.2
|
||||
tenacity>=8.2.3
|
||||
pydantic>=2.5.2,<3.0.0
|
||||
typing-extensions>=4.8.0
|
||||
|
||||
# Auth
|
||||
# Authentication and security
|
||||
PyJWT>=2.8.0
|
||||
cryptography>=41.0.0
|
||||
fastapi-clerk-auth>=0.0.7
|
||||
|
||||
# Database
|
||||
# Database dependencies
|
||||
sqlalchemy>=2.0.25
|
||||
|
||||
# Payment
|
||||
# Payment processing
|
||||
stripe>=8.0.0
|
||||
|
||||
# HTTP clients
|
||||
httpx>=0.28.1
|
||||
aiohttp>=3.9.0
|
||||
requests>=2.31.0
|
||||
# CopilotKit and Research
|
||||
copilotkit
|
||||
exa-py==1.9.1
|
||||
httpx>=0.27.2,<0.28.0
|
||||
|
||||
# AI - needed for podcast
|
||||
# AI/ML dependencies - Windows-compatible versions
|
||||
openai>=1.3.0
|
||||
google-genai>=1.0.0
|
||||
exa-py==1.9.1
|
||||
sentence-transformers>=2.2.2
|
||||
|
||||
# Text processing
|
||||
markdown>=3.5.0
|
||||
beautifulsoup4>=4.12.0
|
||||
lxml>=4.9.0
|
||||
advertools>=0.14.0
|
||||
# txtai with Windows-compatible dependencies
|
||||
txtai[agent]>=7.0.0
|
||||
|
||||
# Data processing
|
||||
pandas>=2.0.0
|
||||
numpy>=1.24.0
|
||||
|
||||
# Image/media for podcast
|
||||
Pillow>=10.0.0
|
||||
matplotlib>=3.8.0
|
||||
huggingface_hub>=1.1.4
|
||||
|
||||
# TTS for podcast
|
||||
gtts>=2.4.0
|
||||
pyttsx3>=2.90
|
||||
|
||||
# Video composition
|
||||
moviepy==2.1.2
|
||||
imageio>=2.31.0
|
||||
imageio-ffmpeg>=0.4.9
|
||||
|
||||
# Testing
|
||||
pytest>=7.4.0
|
||||
pytest-asyncio>=0.21.0
|
||||
|
||||
# Task scheduling
|
||||
apscheduler>=3.10.0
|
||||
|
||||
# Utilities
|
||||
redis>=5.0.0
|
||||
schedule>=1.2.0
|
||||
aiofiles>=23.2.0
|
||||
psutil>=5.9.0
|
||||
|
||||
# Google APIs
|
||||
google-api-python-client>=2.100.0
|
||||
google-auth>=2.23.0
|
||||
google-auth-oauthlib>=1.0.0
|
||||
|
||||
# Other utilities
|
||||
python-dateutil>=2.8.0
|
||||
jinja2>=3.1.0
|
||||
pydantic-settings>=2.0.0
|
||||
# Web scraping and content processing
|
||||
beautifulsoup4>=4.12.0
|
||||
requests>=2.31.0
|
||||
urllib3<2.0.0
|
||||
chardet>=5.0.0
|
||||
charset-normalizer<3.0.0
|
||||
lxml>=4.9.0
|
||||
html5lib>=1.1
|
||||
aiohttp>=3.9.0
|
||||
|
||||
# Data processing
|
||||
pandas>=2.0.0
|
||||
numpy>=1.24.0
|
||||
markdown>=3.5.0
|
||||
|
||||
# SEO Analysis dependencies
|
||||
advertools>=0.14.0
|
||||
textstat>=0.7.3
|
||||
pyspellchecker>=0.7.2
|
||||
aiofiles>=23.2.0
|
||||
crawl4ai>=0.2.0
|
||||
|
||||
# Linguistic Analysis dependencies (Required for persona generation)
|
||||
spacy>=3.7.0
|
||||
nltk>=3.8.0
|
||||
|
||||
# Image and audio processing for Stability AI
|
||||
Pillow>=10.0.0
|
||||
huggingface_hub>=1.1.4
|
||||
|
||||
# Text-to-Speech (TTS) dependencies
|
||||
gtts>=2.4.0
|
||||
pyttsx3>=2.90
|
||||
|
||||
# Video composition dependencies
|
||||
moviepy==2.1.2
|
||||
imageio>=2.31.0
|
||||
imageio-ffmpeg>=0.4.9
|
||||
|
||||
# Testing dependencies
|
||||
pytest>=7.4.0
|
||||
pytest-asyncio>=0.21.0
|
||||
|
||||
# Utilities
|
||||
pydantic>=2.5.2,<3.0.0
|
||||
typing-extensions>=4.8.0
|
||||
|
||||
# Task scheduling
|
||||
apscheduler>=3.10.0
|
||||
|
||||
# Optional dependencies (for enhanced features)
|
||||
redis>=5.0.0
|
||||
schedule>=1.2.0
|
||||
pytrends>=4.9.0
|
||||
|
||||
@@ -2,10 +2,6 @@
|
||||
"""
|
||||
Initialize Alpha Tester Subscription Tiers
|
||||
Creates subscription plans for alpha testing with appropriate limits.
|
||||
|
||||
NOTE: Pricing is seeded via PricingService.initialize_default_pricing()
|
||||
which runs in services/database.py:init_user_database()
|
||||
NOT via this script.
|
||||
"""
|
||||
|
||||
import sys
|
||||
@@ -14,7 +10,7 @@ sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
from models.subscription_models import (
|
||||
SubscriptionPlan, SubscriptionTier
|
||||
SubscriptionPlan, SubscriptionTier, APIProviderPricing, APIProvider
|
||||
)
|
||||
from services.database import get_db_session
|
||||
from datetime import datetime
|
||||
@@ -28,7 +24,7 @@ def create_alpha_subscription_tiers():
|
||||
|
||||
db = get_db_session()
|
||||
if not db:
|
||||
logger.error("Could not get database session")
|
||||
logger.error("❌ Could not get database session")
|
||||
return False
|
||||
|
||||
try:
|
||||
@@ -42,12 +38,12 @@ def create_alpha_subscription_tiers():
|
||||
"description": "Free tier for alpha testing - Limited usage",
|
||||
"features": ["blog_writer", "basic_seo", "content_planning"],
|
||||
"limits": {
|
||||
"gemini_calls_limit": 50,
|
||||
"gemini_tokens_limit": 10000,
|
||||
"tavily_calls_limit": 20,
|
||||
"serper_calls_limit": 10,
|
||||
"stability_calls_limit": 5,
|
||||
"monthly_cost_limit": 5.0
|
||||
"gemini_calls_limit": 50, # 50 calls per day
|
||||
"gemini_tokens_limit": 10000, # 10k tokens per day
|
||||
"tavily_calls_limit": 20, # 20 searches per day
|
||||
"serper_calls_limit": 10, # 10 SEO searches per day
|
||||
"stability_calls_limit": 5, # 5 images per day
|
||||
"monthly_cost_limit": 5.0 # $5 monthly limit
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -58,12 +54,12 @@ def create_alpha_subscription_tiers():
|
||||
"description": "Basic alpha tier - Moderate usage for testing",
|
||||
"features": ["blog_writer", "seo_analysis", "content_planning", "strategy_copilot"],
|
||||
"limits": {
|
||||
"gemini_calls_limit": 200,
|
||||
"gemini_tokens_limit": 50000,
|
||||
"tavily_calls_limit": 100,
|
||||
"serper_calls_limit": 50,
|
||||
"stability_calls_limit": 25,
|
||||
"monthly_cost_limit": 25.0
|
||||
"gemini_calls_limit": 200, # 200 calls per day
|
||||
"gemini_tokens_limit": 50000, # 50k tokens per day
|
||||
"tavily_calls_limit": 100, # 100 searches per day
|
||||
"serper_calls_limit": 50, # 50 SEO searches per day
|
||||
"stability_calls_limit": 25, # 25 images per day
|
||||
"monthly_cost_limit": 25.0 # $25 monthly limit
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -74,12 +70,12 @@ def create_alpha_subscription_tiers():
|
||||
"description": "Pro alpha tier - High usage for power users",
|
||||
"features": ["blog_writer", "seo_analysis", "content_planning", "strategy_copilot", "advanced_analytics"],
|
||||
"limits": {
|
||||
"gemini_calls_limit": 500,
|
||||
"gemini_tokens_limit": 150000,
|
||||
"tavily_calls_limit": 300,
|
||||
"serper_calls_limit": 150,
|
||||
"stability_calls_limit": 100,
|
||||
"monthly_cost_limit": 100.0
|
||||
"gemini_calls_limit": 500, # 500 calls per day
|
||||
"gemini_tokens_limit": 150000, # 150k tokens per day
|
||||
"tavily_calls_limit": 300, # 300 searches per day
|
||||
"serper_calls_limit": 150, # 150 SEO searches per day
|
||||
"stability_calls_limit": 100, # 100 images per day
|
||||
"monthly_cost_limit": 100.0 # $100 monthly limit
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -90,31 +86,34 @@ def create_alpha_subscription_tiers():
|
||||
"description": "Enterprise alpha tier - Unlimited usage for enterprise testing",
|
||||
"features": ["blog_writer", "seo_analysis", "content_planning", "strategy_copilot", "advanced_analytics", "custom_integrations"],
|
||||
"limits": {
|
||||
"gemini_calls_limit": 0,
|
||||
"gemini_tokens_limit": 0,
|
||||
"tavily_calls_limit": 0,
|
||||
"serper_calls_limit": 0,
|
||||
"stability_calls_limit": 0,
|
||||
"monthly_cost_limit": 500.0
|
||||
"gemini_calls_limit": 0, # Unlimited calls
|
||||
"gemini_tokens_limit": 0, # Unlimited tokens
|
||||
"tavily_calls_limit": 0, # Unlimited searches
|
||||
"serper_calls_limit": 0, # Unlimited SEO searches
|
||||
"stability_calls_limit": 0, # Unlimited images
|
||||
"monthly_cost_limit": 500.0 # $500 monthly limit
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
# Create subscription plans
|
||||
for tier_data in alpha_tiers:
|
||||
# Check if plan already exists
|
||||
existing_plan = db.query(SubscriptionPlan).filter(
|
||||
SubscriptionPlan.name == tier_data["name"]
|
||||
).first()
|
||||
|
||||
if existing_plan:
|
||||
logger.info(f"Plan '{tier_data['name']}' already exists, updating...")
|
||||
logger.info(f"✅ Plan '{tier_data['name']}' already exists, updating...")
|
||||
# Update existing plan
|
||||
for key, value in tier_data["limits"].items():
|
||||
setattr(existing_plan, key, value)
|
||||
existing_plan.description = tier_data["description"]
|
||||
existing_plan.features = tier_data["features"]
|
||||
existing_plan.updated_at = datetime.utcnow()
|
||||
else:
|
||||
logger.info(f"Creating new plan: {tier_data['name']}")
|
||||
logger.info(f"🆕 Creating new plan: {tier_data['name']}")
|
||||
# Create new plan
|
||||
plan = SubscriptionPlan(
|
||||
name=tier_data["name"],
|
||||
tier=tier_data["tier"],
|
||||
@@ -127,17 +126,106 @@ def create_alpha_subscription_tiers():
|
||||
db.add(plan)
|
||||
|
||||
db.commit()
|
||||
logger.info("Alpha subscription tiers created/updated successfully!")
|
||||
logger.info("✅ Alpha subscription tiers created/updated successfully!")
|
||||
|
||||
# Create API provider pricing
|
||||
create_api_pricing(db)
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating alpha subscription tiers: {e}")
|
||||
logger.error(f"❌ Error creating alpha subscription tiers: {e}")
|
||||
db.rollback()
|
||||
return False
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
def create_api_pricing(db: Session):
|
||||
"""Create API provider pricing configuration."""
|
||||
|
||||
try:
|
||||
# Gemini pricing (based on current Google AI pricing)
|
||||
gemini_pricing = [
|
||||
{
|
||||
"model_name": "gemini-2.0-flash-exp",
|
||||
"cost_per_input_token": 0.00000075, # $0.75 per 1M tokens
|
||||
"cost_per_output_token": 0.000003, # $3 per 1M tokens
|
||||
"description": "Gemini 2.0 Flash Experimental"
|
||||
},
|
||||
{
|
||||
"model_name": "gemini-1.5-flash",
|
||||
"cost_per_input_token": 0.00000075, # $0.75 per 1M tokens
|
||||
"cost_per_output_token": 0.000003, # $3 per 1M tokens
|
||||
"description": "Gemini 1.5 Flash"
|
||||
},
|
||||
{
|
||||
"model_name": "gemini-1.5-pro",
|
||||
"cost_per_input_token": 0.00000125, # $1.25 per 1M tokens
|
||||
"cost_per_output_token": 0.000005, # $5 per 1M tokens
|
||||
"description": "Gemini 1.5 Pro"
|
||||
}
|
||||
]
|
||||
|
||||
# Tavily pricing
|
||||
tavily_pricing = [
|
||||
{
|
||||
"model_name": "search",
|
||||
"cost_per_search": 0.001, # $0.001 per search
|
||||
"description": "Tavily Search API"
|
||||
}
|
||||
]
|
||||
|
||||
# Serper pricing
|
||||
serper_pricing = [
|
||||
{
|
||||
"model_name": "search",
|
||||
"cost_per_search": 0.001, # $0.001 per search
|
||||
"description": "Serper Google Search API"
|
||||
}
|
||||
]
|
||||
|
||||
# Stability AI pricing
|
||||
stability_pricing = [
|
||||
{
|
||||
"model_name": "stable-diffusion-xl",
|
||||
"cost_per_image": 0.01, # $0.01 per image
|
||||
"description": "Stable Diffusion XL"
|
||||
}
|
||||
]
|
||||
|
||||
# Create pricing records
|
||||
pricing_configs = [
|
||||
(APIProvider.GEMINI, gemini_pricing),
|
||||
(APIProvider.TAVILY, tavily_pricing),
|
||||
(APIProvider.SERPER, serper_pricing),
|
||||
(APIProvider.STABILITY, stability_pricing)
|
||||
]
|
||||
|
||||
for provider, pricing_list in pricing_configs:
|
||||
for pricing_data in pricing_list:
|
||||
# Check if pricing already exists
|
||||
existing_pricing = db.query(APIProviderPricing).filter(
|
||||
APIProviderPricing.provider == provider,
|
||||
APIProviderPricing.model_name == pricing_data["model_name"]
|
||||
).first()
|
||||
|
||||
if existing_pricing:
|
||||
logger.info(f"✅ Pricing for {provider.value}/{pricing_data['model_name']} already exists")
|
||||
else:
|
||||
logger.info(f"🆕 Creating pricing for {provider.value}/{pricing_data['model_name']}")
|
||||
pricing = APIProviderPricing(
|
||||
provider=provider,
|
||||
**pricing_data
|
||||
)
|
||||
db.add(pricing)
|
||||
|
||||
db.commit()
|
||||
logger.info("✅ API provider pricing created successfully!")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"❌ Error creating API pricing: {e}")
|
||||
db.rollback()
|
||||
|
||||
def assign_default_plan_to_users():
|
||||
"""Assign Free Alpha plan to all existing users."""
|
||||
if os.getenv('ENABLE_ALPHA', 'false').lower() not in {'1','true','yes','on'}:
|
||||
@@ -146,28 +234,32 @@ def assign_default_plan_to_users():
|
||||
|
||||
db = get_db_session()
|
||||
if not db:
|
||||
logger.error("Could not get database session")
|
||||
logger.error("❌ Could not get database session")
|
||||
return False
|
||||
|
||||
try:
|
||||
# Get Free Alpha plan
|
||||
free_plan = db.query(SubscriptionPlan).filter(
|
||||
SubscriptionPlan.name == "Free Alpha"
|
||||
).first()
|
||||
|
||||
if not free_plan:
|
||||
logger.error("Free Alpha plan not found")
|
||||
logger.error("❌ Free Alpha plan not found")
|
||||
return False
|
||||
|
||||
from models.subscription_models import UserSubscription, BillingCycle, UsageStatus
|
||||
from datetime import timedelta
|
||||
|
||||
# For now, we'll create a default user subscription
|
||||
# In a real system, you'd query actual users
|
||||
from models.subscription_models import UserSubscription, BillingCycle, UsageStatus
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
# Create default user subscription for testing
|
||||
default_user_id = "default_user"
|
||||
existing_subscription = db.query(UserSubscription).filter(
|
||||
UserSubscription.user_id == default_user_id
|
||||
).first()
|
||||
|
||||
if not existing_subscription:
|
||||
logger.info(f"Creating default subscription for {default_user_id}")
|
||||
logger.info(f"🆕 Creating default subscription for {default_user_id}")
|
||||
subscription = UserSubscription(
|
||||
user_id=default_user_id,
|
||||
plan_id=free_plan.id,
|
||||
@@ -180,32 +272,33 @@ def assign_default_plan_to_users():
|
||||
)
|
||||
db.add(subscription)
|
||||
db.commit()
|
||||
logger.info(f"Default subscription created for {default_user_id}")
|
||||
logger.info(f"✅ Default subscription created for {default_user_id}")
|
||||
else:
|
||||
logger.info(f"Default subscription already exists for {default_user_id}")
|
||||
logger.info(f"✅ Default subscription already exists for {default_user_id}")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error assigning default plan: {e}")
|
||||
logger.error(f"❌ Error assigning default plan: {e}")
|
||||
db.rollback()
|
||||
return False
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
if __name__ == "__main__":
|
||||
logger.info("Initializing Alpha Subscription Tiers...")
|
||||
logger.info("🚀 Initializing Alpha Subscription Tiers...")
|
||||
|
||||
success = create_alpha_subscription_tiers()
|
||||
if success:
|
||||
logger.info("Subscription tiers created successfully!")
|
||||
logger.info("✅ Subscription tiers created successfully!")
|
||||
|
||||
# Assign default plan
|
||||
assign_success = assign_default_plan_to_users()
|
||||
if assign_success:
|
||||
logger.info("Default plan assigned successfully!")
|
||||
logger.info("✅ Default plan assigned successfully!")
|
||||
else:
|
||||
logger.error("Failed to assign default plan")
|
||||
logger.error("❌ Failed to assign default plan")
|
||||
else:
|
||||
logger.error("Failed to create subscription tiers")
|
||||
logger.error("❌ Failed to create subscription tiers")
|
||||
|
||||
logger.info("Alpha subscription system initialization complete!")
|
||||
logger.info("🎉 Alpha subscription system initialization complete!")
|
||||
|
||||
@@ -314,14 +314,11 @@ class ExaResearchProvider(BaseProvider):
|
||||
|
||||
def track_exa_usage(self, user_id: str, cost: float):
|
||||
"""Track Exa API usage after successful call."""
|
||||
from services.database import get_session_for_user
|
||||
from services.database import get_db
|
||||
from services.subscription import PricingService
|
||||
from sqlalchemy import text
|
||||
|
||||
db = get_session_for_user(user_id)
|
||||
if not db:
|
||||
logger.warning(f"[track_exa_usage] Could not get DB session for user {user_id}")
|
||||
return
|
||||
db = next(get_db())
|
||||
try:
|
||||
pricing_service = PricingService(db)
|
||||
current_period = pricing_service.get_current_billing_period(user_id)
|
||||
|
||||
@@ -7,7 +7,6 @@ import os
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker, Session
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
from fastapi import HTTPException
|
||||
from loguru import logger
|
||||
from typing import Optional, List
|
||||
|
||||
@@ -352,15 +351,16 @@ def init_database():
|
||||
|
||||
try:
|
||||
# Create all tables for all models using default engine
|
||||
# Use checkfirst=True (default) to avoid errors for existing tables
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.pool import StaticPool
|
||||
|
||||
# Create tables with checkfirst=True explicitly to handle existing objects
|
||||
for base in [OnboardingBase, SEOAnalysisBase, ContentPlanningBase,
|
||||
EnhancedStrategyBase, MonitoringBase, APIMonitoringBase,
|
||||
PersonaBase, SubscriptionBase, UserBusinessInfoBase, ContentAssetBase]:
|
||||
base.metadata.create_all(bind=default_engine, checkfirst=True)
|
||||
OnboardingBase.metadata.create_all(bind=default_engine)
|
||||
SEOAnalysisBase.metadata.create_all(bind=default_engine)
|
||||
ContentPlanningBase.metadata.create_all(bind=default_engine)
|
||||
EnhancedStrategyBase.metadata.create_all(bind=default_engine)
|
||||
MonitoringBase.metadata.create_all(bind=default_engine)
|
||||
APIMonitoringBase.metadata.create_all(bind=default_engine)
|
||||
PersonaBase.metadata.create_all(bind=default_engine)
|
||||
SubscriptionBase.metadata.create_all(bind=default_engine)
|
||||
UserBusinessInfoBase.metadata.create_all(bind=default_engine)
|
||||
ContentAssetBase.metadata.create_all(bind=default_engine)
|
||||
logger.info("Global database initialized successfully")
|
||||
except SQLAlchemyError as e:
|
||||
logger.error(f"Error initializing global database: {str(e)}")
|
||||
@@ -387,15 +387,12 @@ def get_db(current_user: dict = Depends(get_current_user)):
|
||||
"""
|
||||
user_id = current_user.get('id') or current_user.get('clerk_user_id')
|
||||
if not user_id:
|
||||
# Fallback or error? For now log error
|
||||
logger.error("No user ID found in context for DB connection")
|
||||
raise HTTPException(status_code=401, detail="User ID required for database access")
|
||||
# Could raise exception, but let's try to be safe
|
||||
raise Exception("User ID required for database access")
|
||||
|
||||
try:
|
||||
engine = get_engine_for_user(user_id)
|
||||
except Exception as e:
|
||||
logger.error(f"[DB] Failed to create engine for user {user_id}: {e}", exc_info=True)
|
||||
raise HTTPException(status_code=503, detail="Database temporarily unavailable")
|
||||
|
||||
engine = get_engine_for_user(user_id)
|
||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
|
||||
@@ -1,745 +0,0 @@
|
||||
"""Read-only virtual filesystem facade for agent flat context documents.
|
||||
|
||||
This adapter provides shell-like primitives (`list_context`, `search_context`,
|
||||
`read_context_file`) over the JSON documents managed by AgentFlatContextStore.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import os
|
||||
import fcntl
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from collections import deque
|
||||
from fnmatch import fnmatch
|
||||
from pathlib import Path
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from services.intelligence.agent_flat_context import AgentFlatContextStore
|
||||
|
||||
|
||||
class SmartGrepEngine:
|
||||
"""Streaming grep engine with regex fallback and contextual snippets."""
|
||||
|
||||
def __init__(self, context_window: int = 1):
|
||||
self.context_window = max(0, int(context_window))
|
||||
|
||||
@staticmethod
|
||||
def _compile_pattern(pattern: str) -> re.Pattern:
|
||||
try:
|
||||
return re.compile(pattern, re.IGNORECASE)
|
||||
except re.error:
|
||||
return re.compile(re.escape(pattern), re.IGNORECASE)
|
||||
|
||||
@staticmethod
|
||||
def _truncate(text: str, limit: int = 180) -> str:
|
||||
text = " ".join(text.split())
|
||||
if len(text) <= limit:
|
||||
return text
|
||||
return text[:limit] + "..."
|
||||
|
||||
def stream_file(self, file_path: Path, pattern: str, *, path_label: str) -> List[Dict[str, Any]]:
|
||||
regex = self._compile_pattern(pattern)
|
||||
matches: List[Dict[str, Any]] = []
|
||||
prev = deque(maxlen=self.context_window)
|
||||
active: List[Dict[str, Any]] = []
|
||||
|
||||
with open(file_path, "r", encoding="utf-8", errors="ignore") as f:
|
||||
for line_no, line in enumerate(f, start=1):
|
||||
# Fill trailing context for active matches.
|
||||
for item in active:
|
||||
if item["remaining_after"] > 0:
|
||||
item["after"].append(line.rstrip("\n"))
|
||||
item["remaining_after"] -= 1
|
||||
|
||||
# Detect a new match on current line.
|
||||
if regex.search(line):
|
||||
current = line.rstrip("\n")
|
||||
record = {
|
||||
"path": path_label,
|
||||
"line": line_no,
|
||||
"before": list(prev),
|
||||
"match_line": current,
|
||||
"after": [],
|
||||
"remaining_after": self.context_window,
|
||||
}
|
||||
active.append(record)
|
||||
matches.append(record)
|
||||
|
||||
prev.append(line.rstrip("\n"))
|
||||
|
||||
formatted: List[Dict[str, Any]] = []
|
||||
for m in matches:
|
||||
snippet_parts = [*m["before"], m["match_line"], *m["after"]]
|
||||
snippet = self._truncate(" | ".join([p for p in snippet_parts if p is not None]))
|
||||
line_l = m["match_line"].lower()
|
||||
is_high_signal = any(k in line_l for k in ("agent_summary", "high_signal_terms", "quick_facts"))
|
||||
formatted.append(
|
||||
{
|
||||
"path": m["path"],
|
||||
"line": m["line"],
|
||||
"snippet": snippet,
|
||||
"relevance": "High Relevance" if is_high_signal else "Supporting Detail",
|
||||
"reason": "matched summary field in stream" if is_high_signal else "matched streamed body line",
|
||||
"score": 70 if is_high_signal else 50,
|
||||
}
|
||||
)
|
||||
return formatted
|
||||
|
||||
|
||||
class AgentContextVFS:
|
||||
"""Read-only adapter that maps virtual paths to flat context documents."""
|
||||
|
||||
VIRTUAL_MAP = {
|
||||
"/steps/website": AgentFlatContextStore.STEP2_FILENAME,
|
||||
"/steps/research": AgentFlatContextStore.STEP3_FILENAME,
|
||||
"/steps/persona": AgentFlatContextStore.STEP4_FILENAME,
|
||||
"/steps/integrations": AgentFlatContextStore.STEP5_FILENAME,
|
||||
}
|
||||
HIGH_SIGNAL_MARKERS = ("agent_summary", "high_signal_terms", "quick_facts", "context_type")
|
||||
|
||||
def __init__(self, user_id: str, project_id: Optional[str] = None):
|
||||
self.user_id = user_id
|
||||
self.project_id = project_id
|
||||
self.store = AgentFlatContextStore(user_id)
|
||||
self.grep_engine = SmartGrepEngine(context_window=1)
|
||||
|
||||
@staticmethod
|
||||
def _safe_slug(value: Optional[str], fallback: str) -> str:
|
||||
raw = str(value or "").strip()
|
||||
safe = "".join(c for c in raw if c.isalnum() or c in ("-", "_"))
|
||||
return safe or fallback
|
||||
|
||||
def _manifest_docs(self) -> List[Dict[str, Any]]:
|
||||
manifest = self.store.load_context_manifest() or {"documents": []}
|
||||
docs = manifest.get("documents")
|
||||
return docs if isinstance(docs, list) else []
|
||||
|
||||
def _workspace_root(self) -> Path:
|
||||
if self.project_id:
|
||||
root_dir = Path(__file__).resolve().parents[3]
|
||||
safe_project = self._safe_slug(self.project_id, "default_project")
|
||||
project_root = root_dir / "workspace" / f"project_{safe_project}"
|
||||
project_root.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(project_root, 0o700)
|
||||
return project_root
|
||||
return self.store._workspace_dir()
|
||||
|
||||
def _scratchpad_dir(self) -> Path:
|
||||
scratch = self._workspace_root() / "scratchpad"
|
||||
scratch.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(scratch, 0o700)
|
||||
return scratch
|
||||
|
||||
def _allowlisted_workspace_files(self) -> List[Path]:
|
||||
"""Return sandboxed files eligible for streaming search."""
|
||||
files: List[Path] = []
|
||||
workspace = self._workspace_root()
|
||||
context_dir = self.store._context_dir()
|
||||
|
||||
# 1) manifest-backed onboarding context files
|
||||
for item in self._manifest_docs():
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
rel = str(item.get("path") or "")
|
||||
if not rel:
|
||||
continue
|
||||
try:
|
||||
candidate = self.store._safe_resolve_under(context_dir, rel)
|
||||
if candidate.exists() and candidate.is_file():
|
||||
files.append(candidate)
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
# 2) workspace text artifacts (README, operator notes, etc.)
|
||||
for candidate in workspace.glob("*.txt"):
|
||||
if candidate.is_file():
|
||||
files.append(candidate.resolve())
|
||||
readme = workspace / "README.md"
|
||||
if readme.exists() and readme.is_file():
|
||||
files.append(readme.resolve())
|
||||
|
||||
# dedupe
|
||||
seen = set()
|
||||
unique: List[Path] = []
|
||||
for p in files:
|
||||
rp = str(p)
|
||||
if rp in seen:
|
||||
continue
|
||||
seen.add(rp)
|
||||
unique.append(p)
|
||||
return unique
|
||||
|
||||
@staticmethod
|
||||
def _query_variants(query: str) -> List[str]:
|
||||
"""Generate normalized and synonym-expanded query variants."""
|
||||
base = (query or "").strip().lower()
|
||||
if not base:
|
||||
return []
|
||||
synonyms = {
|
||||
"tone": ["brand voice", "writing tone"],
|
||||
"voice": ["brand voice", "writing style"],
|
||||
"competitor": ["competition", "rival"],
|
||||
"seo": ["search", "metadata"],
|
||||
"persona": ["audience profile", "target audience"],
|
||||
}
|
||||
variants = [base]
|
||||
tokens = base.split()
|
||||
for idx, tok in enumerate(tokens):
|
||||
if tok in synonyms:
|
||||
for repl in synonyms[tok]:
|
||||
new_tokens = tokens.copy()
|
||||
new_tokens[idx] = repl
|
||||
variants.append(" ".join(new_tokens))
|
||||
variants.extend([base.replace("-", " "), base.replace("_", " ")])
|
||||
# dedupe, preserve order
|
||||
seen = set()
|
||||
out: List[str] = []
|
||||
for v in variants:
|
||||
vv = v.strip()
|
||||
if not vv or vv in seen:
|
||||
continue
|
||||
seen.add(vv)
|
||||
out.append(vv)
|
||||
return out
|
||||
|
||||
@staticmethod
|
||||
def _freshness_score(updated_at: Optional[str]) -> float:
|
||||
if not updated_at:
|
||||
return 0.3
|
||||
try:
|
||||
from datetime import datetime, timezone
|
||||
|
||||
ts = datetime.fromisoformat(str(updated_at).replace("Z", "+00:00"))
|
||||
if ts.tzinfo is None:
|
||||
ts = ts.replace(tzinfo=timezone.utc)
|
||||
days = max(0.0, (datetime.now(timezone.utc) - ts).total_seconds() / 86400.0)
|
||||
if days <= 1:
|
||||
return 1.0
|
||||
if days <= 7:
|
||||
return 0.9
|
||||
if days <= 30:
|
||||
return 0.75
|
||||
if days <= 90:
|
||||
return 0.6
|
||||
return 0.4
|
||||
except Exception:
|
||||
return 0.3
|
||||
|
||||
def _cluster_results(self, results: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
"""Deduplicate repeated hits by file + reason and keep strongest evidence."""
|
||||
buckets: Dict[Tuple[str, str], Dict[str, Any]] = {}
|
||||
for r in results:
|
||||
path = str(r.get("path") or "")
|
||||
reason = str(r.get("reason") or "")
|
||||
key = (path, reason)
|
||||
existing = buckets.get(key)
|
||||
if not existing:
|
||||
buckets[key] = {**r, "hit_count": 1}
|
||||
continue
|
||||
existing["hit_count"] = int(existing.get("hit_count", 1)) + 1
|
||||
if int(r.get("score", 0)) > int(existing.get("score", 0)):
|
||||
existing.update({k: v for k, v in r.items() if k != "hit_count"})
|
||||
existing["hit_count"] = int(existing.get("hit_count", 1))
|
||||
clustered = list(buckets.values())
|
||||
clustered.sort(key=lambda r: (-int(r.get("score", 0)), str(r.get("path") or "")))
|
||||
return clustered
|
||||
|
||||
def _keyword_density(self, snippet: str, query: str) -> float:
|
||||
if not snippet or not query:
|
||||
return 0.0
|
||||
query_tokens = [t for t in query.lower().split() if t]
|
||||
if not query_tokens:
|
||||
return 0.0
|
||||
text = snippet.lower()
|
||||
hits = sum(text.count(tok) for tok in query_tokens)
|
||||
words = max(1, len(text.split()))
|
||||
return hits / words
|
||||
|
||||
def _static_triage(self, results: List[Dict[str, Any]], query: str) -> List[Dict[str, Any]]:
|
||||
"""Semgrep-style static heuristic triage before main agent consumption."""
|
||||
triaged: List[Dict[str, Any]] = []
|
||||
for r in results:
|
||||
snippet = str(r.get("snippet") or "")
|
||||
density = self._keyword_density(snippet, query)
|
||||
marker_hit = any(marker in snippet.lower() for marker in self.HIGH_SIGNAL_MARKERS)
|
||||
low_probability = bool(density < 0.01 and not marker_hit)
|
||||
item = dict(r)
|
||||
item["keyword_density"] = round(density, 4)
|
||||
item["low_probability"] = low_probability
|
||||
triaged.append(item)
|
||||
triaged.sort(
|
||||
key=lambda x: (
|
||||
bool(x.get("low_probability")),
|
||||
-float(x.get("confidence", 0)),
|
||||
-int(x.get("score", 0)),
|
||||
)
|
||||
)
|
||||
return triaged
|
||||
|
||||
@staticmethod
|
||||
def _llm_router_stub(results: List[Dict[str, Any]], top_k: int = 5) -> List[Dict[str, Any]]:
|
||||
"""Fast local triage stub (drop low-probability first; keep strongest candidates)."""
|
||||
ranked = sorted(
|
||||
results,
|
||||
key=lambda x: (
|
||||
bool(x.get("low_probability")),
|
||||
-float(x.get("confidence", 0)),
|
||||
-int(x.get("score", 0)),
|
||||
),
|
||||
)
|
||||
return ranked[: max(1, top_k)]
|
||||
|
||||
@staticmethod
|
||||
def _resolve_json_path(data: Any, path_query: str) -> Any:
|
||||
"""Resolve dot/bracket JSON path such as 'data.seo_audit.recommendations[0]'."""
|
||||
if not path_query:
|
||||
return data
|
||||
|
||||
current = data
|
||||
query = path_query.strip()
|
||||
parts: List[str] = []
|
||||
buf = ""
|
||||
in_brackets = False
|
||||
for ch in query:
|
||||
if ch == "." and not in_brackets:
|
||||
if buf:
|
||||
parts.append(buf)
|
||||
buf = ""
|
||||
continue
|
||||
if ch == "[":
|
||||
in_brackets = True
|
||||
elif ch == "]":
|
||||
in_brackets = False
|
||||
buf += ch
|
||||
if buf:
|
||||
parts.append(buf)
|
||||
|
||||
for part in parts:
|
||||
if "[" in part and part.endswith("]"):
|
||||
key, idx_raw = part.split("[", 1)
|
||||
idx = int(idx_raw[:-1])
|
||||
if key:
|
||||
if not isinstance(current, dict):
|
||||
raise KeyError(key)
|
||||
current = current[key]
|
||||
if not isinstance(current, list):
|
||||
raise IndexError(idx)
|
||||
current = current[idx]
|
||||
else:
|
||||
if not isinstance(current, dict):
|
||||
raise KeyError(part)
|
||||
current = current[part]
|
||||
return current
|
||||
|
||||
def _resolve_path(self, path: str) -> Tuple[str, Optional[str]]:
|
||||
normalized = (path or "").strip()
|
||||
if not normalized:
|
||||
return "", None
|
||||
if normalized == "/env/summary":
|
||||
return "virtual_summary", None
|
||||
if normalized in self.VIRTUAL_MAP:
|
||||
return "file", self.VIRTUAL_MAP[normalized]
|
||||
if ".." in normalized or "\\" in normalized:
|
||||
return "", None
|
||||
if normalized.startswith("/"):
|
||||
candidate = normalized.rsplit("/", 1)[-1]
|
||||
else:
|
||||
candidate = normalized
|
||||
if "/" in candidate:
|
||||
return "", None
|
||||
allowed = AgentFlatContextStore.ALLOWED_CONTEXT_FILES - {AgentFlatContextStore.MANIFEST_FILENAME}
|
||||
if candidate not in allowed:
|
||||
return "", None
|
||||
return "file", candidate
|
||||
|
||||
def list_context(self) -> Dict[str, Any]:
|
||||
"""List available context files (ls-equivalent)."""
|
||||
docs = self._manifest_docs()
|
||||
items = []
|
||||
for d in docs:
|
||||
if not isinstance(d, dict):
|
||||
continue
|
||||
items.append(
|
||||
{
|
||||
"path": d.get("path"),
|
||||
"type": d.get("type"),
|
||||
"updated_at": d.get("updated_at"),
|
||||
"size_bytes": d.get("size_bytes", 0),
|
||||
}
|
||||
)
|
||||
items.sort(key=lambda x: str(x.get("path") or ""))
|
||||
result = {
|
||||
"workspace_hint": "Use this list to see which onboarding steps are complete.",
|
||||
"tip": "Use `search_context` to find specific keywords across all steps.",
|
||||
"virtual_paths": ["/env/summary", *sorted(self.VIRTUAL_MAP.keys())],
|
||||
"files": items,
|
||||
"collaboration": {
|
||||
"scratchpad_dir": str(self._scratchpad_dir()),
|
||||
"activity_log": "scratchpad/activity_log.jsonl",
|
||||
},
|
||||
}
|
||||
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=list_context files={len(items)}")
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _flatten_strings(data: Any, limit: int = 2000) -> str:
|
||||
pieces: List[str] = []
|
||||
|
||||
def walk(v: Any) -> None:
|
||||
if len(pieces) >= limit:
|
||||
return
|
||||
if isinstance(v, dict):
|
||||
for key, value in v.items():
|
||||
pieces.append(str(key))
|
||||
walk(value)
|
||||
elif isinstance(v, list):
|
||||
for item in v:
|
||||
walk(item)
|
||||
elif isinstance(v, (str, int, float, bool)):
|
||||
pieces.append(str(v))
|
||||
|
||||
walk(data)
|
||||
return " ".join(pieces)
|
||||
|
||||
@staticmethod
|
||||
def _extract_search_fields(doc: Dict[str, Any]) -> Tuple[List[str], Dict[str, Any], str]:
|
||||
summary = doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {}
|
||||
hints = summary.get("retrieval_hints") if isinstance(summary.get("retrieval_hints"), dict) else {}
|
||||
quick_facts = summary.get("quick_facts") if isinstance(summary.get("quick_facts"), dict) else {}
|
||||
high_terms = hints.get("high_signal_terms") if isinstance(hints.get("high_signal_terms"), list) else []
|
||||
body = AgentContextVFS._flatten_strings(doc.get("data") if isinstance(doc.get("data"), dict) else {})
|
||||
return [str(t).lower() for t in high_terms], quick_facts, body.lower()
|
||||
|
||||
def search_context(self, query: str, *, limit: int = 10, path_glob: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Smart grep with coarse-to-fine ranking and parallel stream scans."""
|
||||
normalized = (query or "").strip()
|
||||
if not normalized:
|
||||
return {"query": query, "results": []}
|
||||
self.store._audit_event("vfs_search", normalized, "started")
|
||||
try:
|
||||
variants = self._query_variants(normalized)
|
||||
attempted_queries: List[str] = []
|
||||
scored: List[Dict[str, Any]] = []
|
||||
|
||||
for candidate_query in variants:
|
||||
attempted_queries.append(candidate_query)
|
||||
needle = candidate_query.lower()
|
||||
|
||||
# Pass 1: summary-first ranking (high relevance)
|
||||
docs = self._manifest_docs()
|
||||
variant_scored: List[Dict[str, Any]] = []
|
||||
for item in docs:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
path = str(item.get("path") or "")
|
||||
if not path:
|
||||
continue
|
||||
if path_glob and not fnmatch(path, path_glob):
|
||||
continue
|
||||
doc = self.store.load_context_document(path) or {}
|
||||
high_terms, quick_facts, _ = self._extract_search_fields(doc)
|
||||
|
||||
high_match = any(needle in term for term in high_terms)
|
||||
quick_match = any(needle in str(v).lower() for v in quick_facts.values()) if isinstance(quick_facts, dict) else False
|
||||
if not (high_match or quick_match):
|
||||
continue
|
||||
|
||||
score = 100 if high_match else 80
|
||||
reason = "matched high_signal_terms" if high_match else "matched quick_facts"
|
||||
variant_scored.append(
|
||||
{
|
||||
"path": path,
|
||||
"line": None,
|
||||
"snippet": f"{reason}: {candidate_query}"[:100],
|
||||
"type": item.get("type"),
|
||||
"updated_at": item.get("updated_at"),
|
||||
"relevance": "High Relevance",
|
||||
"reason": reason,
|
||||
"score": score,
|
||||
}
|
||||
)
|
||||
|
||||
# Pass 2: parallelized stream scan over allowlisted workspace files.
|
||||
allowlisted = self._allowlisted_workspace_files()
|
||||
body_matches: List[Dict[str, Any]] = []
|
||||
if allowlisted:
|
||||
with ThreadPoolExecutor(max_workers=min(8, max(1, len(allowlisted)))) as pool:
|
||||
future_map = {}
|
||||
for p in allowlisted:
|
||||
path_label = p.name
|
||||
if path_glob and not fnmatch(path_label, path_glob):
|
||||
continue
|
||||
future = pool.submit(self.grep_engine.stream_file, p, candidate_query, path_label=path_label)
|
||||
future_map[future] = path_label
|
||||
|
||||
for future in as_completed(future_map):
|
||||
try:
|
||||
body_matches.extend(future.result() or [])
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
variant_scored.extend(body_matches)
|
||||
if variant_scored:
|
||||
scored = variant_scored
|
||||
break
|
||||
|
||||
scored = self._cluster_results(scored)
|
||||
|
||||
# Add confidence based on score + freshness + hit density.
|
||||
for r in scored:
|
||||
base = min(1.0, max(0.0, float(r.get("score", 0)) / 100.0))
|
||||
freshness = self._freshness_score(r.get("updated_at"))
|
||||
density = min(1.0, 0.2 + (int(r.get("hit_count", 1)) * 0.1))
|
||||
confidence = round((base * 0.6) + (freshness * 0.25) + (density * 0.15), 3)
|
||||
r["confidence"] = confidence
|
||||
|
||||
scored.sort(key=lambda r: (-int(r.get("score", 0)), str(r.get("path") or "")))
|
||||
matched_files = sorted({str(r.get("path") or "") for r in scored if r.get("path")})
|
||||
capped_results = scored[: max(1, limit)]
|
||||
notice = None
|
||||
if len(matched_files) > 10:
|
||||
notice = f"Found {len(matched_files)} matches. Showing top 10. Use a more specific keyword to narrow down."
|
||||
capped_results = scored[:10]
|
||||
|
||||
# Token/length budgeting (~2000 tokens ~= ~8000 chars).
|
||||
budget_chars = 8000
|
||||
bounded_results = []
|
||||
used = 0
|
||||
for r in capped_results:
|
||||
snippet = str(r.get("snippet") or "")
|
||||
cost = len(snippet) + 120 # account for metadata fields
|
||||
if bounded_results and used + cost > budget_chars:
|
||||
break
|
||||
bounded_results.append(r)
|
||||
used += cost
|
||||
|
||||
result = {
|
||||
"query": normalized,
|
||||
"attempted_queries": attempted_queries,
|
||||
"matched_files_count": len(matched_files),
|
||||
"results": self._static_triage(bounded_results, normalized),
|
||||
"notice": notice,
|
||||
"char_budget_used": used,
|
||||
"can_answer": bool(bounded_results),
|
||||
}
|
||||
result["triage_top5"] = self._llm_router_stub(result["results"], top_k=5)
|
||||
logger.info(
|
||||
f"[vfs_audit] user={self.store.safe_user_id} action=search_context query={normalized!r} results={len(result['results'])}"
|
||||
)
|
||||
self.store._audit_event("vfs_search", normalized, f"success_{len(result['results'])}_hits")
|
||||
return result
|
||||
except Exception as exc:
|
||||
self.store._audit_event("vfs_search", normalized, f"failed_{exc.__class__.__name__}")
|
||||
return {"query": normalized, "matched_files_count": 0, "results": [], "notice": "Search failed.", "can_answer": False}
|
||||
|
||||
@staticmethod
|
||||
def _strip_technical_metadata(doc: Dict[str, Any]) -> Dict[str, Any]:
|
||||
sanitized = {
|
||||
"context_type": doc.get("context_type"),
|
||||
"updated_at": doc.get("updated_at"),
|
||||
"journey": ((doc.get("document_context") or {}).get("journey") or {}) if isinstance(doc.get("document_context"), dict) else {},
|
||||
"agent_summary": doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {},
|
||||
"data": doc.get("data") if isinstance(doc.get("data"), dict) else {},
|
||||
}
|
||||
return sanitized
|
||||
|
||||
def inspect_file(self, path: str, *, key: Optional[str] = None, small_file_bytes: int = 5 * 1024) -> Dict[str, Any]:
|
||||
"""Smart reader (cat/head equivalent) with summary-first behavior."""
|
||||
kind, resolved = self._resolve_path(path)
|
||||
if kind == "virtual_summary":
|
||||
result = {
|
||||
"path": "/env/summary",
|
||||
"mode": "summary",
|
||||
"data": self.store.generate_total_summary(),
|
||||
}
|
||||
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=read_context_file path=/env/summary mode=summary")
|
||||
return result
|
||||
|
||||
if not resolved:
|
||||
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=read_context_file path={path!r} status=rejected")
|
||||
return {"error": "File not found", "path": path}
|
||||
|
||||
# JSON context doc path
|
||||
doc = self.store.load_context_document(resolved)
|
||||
if doc:
|
||||
view = self._strip_technical_metadata(doc)
|
||||
data = view.get("data") if isinstance(view.get("data"), dict) else {}
|
||||
raw_size = self.store.estimate_size_bytes(view)
|
||||
|
||||
if key:
|
||||
if key in data:
|
||||
result = {
|
||||
"path": resolved,
|
||||
"mode": "key",
|
||||
"key": key,
|
||||
"agent_summary": view.get("agent_summary"),
|
||||
"data": data.get(key),
|
||||
}
|
||||
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=key")
|
||||
return result
|
||||
logger.info(
|
||||
f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=key_missing key={key}"
|
||||
)
|
||||
return {
|
||||
"path": resolved,
|
||||
"mode": "key_missing",
|
||||
"key": key,
|
||||
"available_keys": sorted(list(data.keys())),
|
||||
"message": "Requested key not found. Choose one of available_keys.",
|
||||
}
|
||||
|
||||
if raw_size <= small_file_bytes:
|
||||
result = {
|
||||
"path": resolved,
|
||||
"mode": "full",
|
||||
"data": view,
|
||||
}
|
||||
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=full")
|
||||
return result
|
||||
|
||||
result = {
|
||||
"path": resolved,
|
||||
"mode": "summary_plus_keys",
|
||||
"size_bytes": raw_size,
|
||||
"agent_summary": view.get("agent_summary"),
|
||||
"keys": sorted(list(data.keys())),
|
||||
"message": "File is large. Re-run with key to inspect a specific section.",
|
||||
}
|
||||
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} mode=summary_plus_keys")
|
||||
return result
|
||||
|
||||
logger.info(f"[vfs_audit] user={self.store.safe_user_id} action=inspect_file path={resolved} status=not_found")
|
||||
return {"error": "File not found", "path": path, "resolved": resolved}
|
||||
|
||||
def read_context_file(self, path: str, *, subkey: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Backward-compatible alias for inspect_file."""
|
||||
return self.inspect_file(path, key=subkey)
|
||||
|
||||
def write_context_file(self, *_args: Any, **_kwargs: Any) -> None:
|
||||
"""Disallow writes from the agent-facing VFS."""
|
||||
raise OSError("EROFS: read-only file system")
|
||||
|
||||
# Backward-compat function name requested in design docs.
|
||||
inspect = inspect_file
|
||||
|
||||
def write_shared_note(self, note: str, *, agent_id: str = "agent", filename: str = "collaboration.md") -> Dict[str, Any]:
|
||||
"""Append a shared project note with advisory locking in scratchpad."""
|
||||
safe_name = Path(filename).name
|
||||
if safe_name != filename or ".." in filename or "/" in filename or "\\" in filename:
|
||||
self.store._audit_event("write_shared_note", filename, "rejected_filename")
|
||||
return {"ok": False, "error": "Invalid filename"}
|
||||
|
||||
scratch = self._scratchpad_dir()
|
||||
target = (scratch / safe_name).resolve()
|
||||
if scratch.resolve() not in target.parents:
|
||||
self.store._audit_event("write_shared_note", filename, "rejected_path")
|
||||
return {"ok": False, "error": "Unsafe path"}
|
||||
|
||||
lock_path = scratch / f".{safe_name}.lock"
|
||||
ts = datetime.now(timezone.utc).isoformat()
|
||||
header = f"\n## {ts} | {self._safe_slug(agent_id, 'agent')}\n"
|
||||
payload = header + str(note).rstrip() + "\n"
|
||||
|
||||
try:
|
||||
with open(lock_path, "w", encoding="utf-8") as lf:
|
||||
fcntl.flock(lf.fileno(), fcntl.LOCK_EX)
|
||||
with open(target, "a", encoding="utf-8") as tf:
|
||||
tf.write(payload)
|
||||
tf.flush()
|
||||
os.fsync(tf.fileno())
|
||||
os.chmod(target, 0o600)
|
||||
fcntl.flock(lf.fileno(), fcntl.LOCK_UN)
|
||||
self.store._audit_event("write_shared_note", safe_name, "success")
|
||||
self.append_activity_log(
|
||||
event_type="shared_note_written",
|
||||
actor=agent_id,
|
||||
details={"file": safe_name, "bytes": len(payload)},
|
||||
)
|
||||
return {"ok": True, "file": safe_name, "bytes_written": len(payload)}
|
||||
except Exception as exc:
|
||||
self.store._audit_event("write_shared_note", safe_name, f"failed_{exc.__class__.__name__}")
|
||||
return {"ok": False, "error": str(exc)}
|
||||
|
||||
def append_activity_log(self, *, event_type: str, actor: str, details: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
|
||||
"""Write append-only project activity log entry in JSONL format."""
|
||||
scratch = self._scratchpad_dir()
|
||||
target = (scratch / "activity_log.jsonl").resolve()
|
||||
lock_path = scratch / ".activity_log.jsonl.lock"
|
||||
entry = {
|
||||
"timestamp": datetime.now(timezone.utc).isoformat(),
|
||||
"event_type": str(event_type),
|
||||
"actor": self._safe_slug(actor, "agent"),
|
||||
"project_id": self._safe_slug(self.project_id, "none") if self.project_id else None,
|
||||
"details": details or {},
|
||||
}
|
||||
line = json.dumps(entry, ensure_ascii=False) + "\n"
|
||||
try:
|
||||
with open(lock_path, "w", encoding="utf-8") as lf:
|
||||
fcntl.flock(lf.fileno(), fcntl.LOCK_EX)
|
||||
with open(target, "a", encoding="utf-8") as tf:
|
||||
tf.write(line)
|
||||
tf.flush()
|
||||
os.fsync(tf.fileno())
|
||||
os.chmod(target, 0o600)
|
||||
fcntl.flock(lf.fileno(), fcntl.LOCK_UN)
|
||||
return {"ok": True}
|
||||
except Exception as exc:
|
||||
logger.warning(f"Failed to append activity log: {exc}")
|
||||
return {"ok": False, "error": str(exc)}
|
||||
|
||||
def read_struct(self, filename: str, path_query: str) -> Dict[str, Any]:
|
||||
"""AST-style structural reader for JSON context files with dependency context injection."""
|
||||
resolved_kind, resolved = self._resolve_path(filename)
|
||||
if resolved_kind == "virtual_summary" or not resolved:
|
||||
return {"ok": False, "error": "Invalid file"}
|
||||
|
||||
doc = self.store.load_context_document(resolved)
|
||||
if not isinstance(doc, dict):
|
||||
return {"ok": False, "error": "File not found"}
|
||||
|
||||
try:
|
||||
extracted = self._resolve_json_path(doc, path_query)
|
||||
except Exception as exc:
|
||||
return {"ok": False, "error": f"path_query resolution failed: {exc}"}
|
||||
|
||||
# Lightweight dependency context: inject brand voice from step2 when reading persona structures.
|
||||
dependency_context: Dict[str, Any] = {}
|
||||
if "persona" in path_query.lower() or resolved == AgentFlatContextStore.STEP4_FILENAME:
|
||||
step2 = self.store.load_step2_context_document() or {}
|
||||
step2_data = step2.get("data") if isinstance(step2.get("data"), dict) else {}
|
||||
brand = step2_data.get("brand_analysis") if isinstance(step2_data.get("brand_analysis"), dict) else {}
|
||||
dependency_context["brand_voice"] = brand.get("brand_voice")
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"file": resolved,
|
||||
"path_query": path_query,
|
||||
"data": extracted,
|
||||
"dependency_context": dependency_context,
|
||||
"context": "Extracted via structural parse to save tokens.",
|
||||
}
|
||||
|
||||
|
||||
|
||||
def build_filesystem_header(user_id: str) -> str:
|
||||
"""Generate compact prompt header with available files and priority hints."""
|
||||
try:
|
||||
store = AgentFlatContextStore(user_id)
|
||||
manifest = store.load_context_manifest() or {"documents": []}
|
||||
docs = manifest.get("documents") if isinstance(manifest.get("documents"), list) else []
|
||||
available = [str(d.get("path")) for d in docs if isinstance(d, dict) and d.get("path")]
|
||||
files = ", ".join(sorted(available)) if available else "none"
|
||||
return (
|
||||
"Workspace Context: You have access to a local flat-file store. "
|
||||
f"Available Files: {files}. "
|
||||
"Instructions: For style guidelines, prioritize step4_persona_data.json. "
|
||||
"For technical site data, prioritize step2_website_analysis.json."
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(f"Failed to build filesystem header for user {user_id}: {exc}")
|
||||
return "Workspace Context: local flat-file store unavailable."
|
||||
@@ -9,8 +9,6 @@ from __future__ import annotations
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import hmac
|
||||
import hashlib
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
@@ -27,14 +25,6 @@ class AgentFlatContextStore:
|
||||
STEP4_FILENAME = "step4_persona_data.json"
|
||||
STEP5_FILENAME = "step5_integrations.json"
|
||||
MANIFEST_FILENAME = "context_manifest.json"
|
||||
WORKSPACE_README = "README.md"
|
||||
ALLOWED_CONTEXT_FILES = {
|
||||
STEP2_FILENAME,
|
||||
STEP3_FILENAME,
|
||||
STEP4_FILENAME,
|
||||
STEP5_FILENAME,
|
||||
MANIFEST_FILENAME,
|
||||
}
|
||||
|
||||
SCHEMA_VERSION = "1.3"
|
||||
DEFAULT_MAX_BYTES = 300_000
|
||||
@@ -43,53 +33,12 @@ class AgentFlatContextStore:
|
||||
def __init__(self, user_id: str):
|
||||
self.user_id = user_id
|
||||
self.safe_user_id = self._sanitize_user_id(user_id)
|
||||
self._ensure_workspace_permissions()
|
||||
|
||||
def _ensure_workspace_permissions(self) -> None:
|
||||
"""Ensure workspace and context directories exist with owner-only permissions."""
|
||||
workspace_dir = self._workspace_dir()
|
||||
context_dir = workspace_dir / self.CONTEXT_DIRNAME
|
||||
workspace_dir.mkdir(parents=True, exist_ok=True)
|
||||
context_dir.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(workspace_dir, 0o700)
|
||||
os.chmod(context_dir, 0o700)
|
||||
|
||||
@staticmethod
|
||||
def _safe_resolve_under(base_dir: Path, requested_path: str) -> Path:
|
||||
"""Resolve path and ensure it remains inside base_dir (path sandboxing)."""
|
||||
base_real = base_dir.resolve()
|
||||
candidate = (base_dir / requested_path).resolve()
|
||||
if candidate == base_real or base_real in candidate.parents:
|
||||
return candidate
|
||||
raise ValueError("Unsafe path access attempt outside sandbox")
|
||||
|
||||
@staticmethod
|
||||
def _sanitize_user_id(user_id: str) -> str:
|
||||
safe = "".join(c for c in str(user_id) if c.isalnum() or c in ("-", "_"))
|
||||
return safe or "unknown_user"
|
||||
|
||||
def _master_salt(self) -> str:
|
||||
return os.getenv("FILE_ENCRYPTION_SALT", "")
|
||||
|
||||
def derive_user_secret(self) -> bytes:
|
||||
"""Derive deterministic per-user secret from env salt + safe user id."""
|
||||
salt = self._master_salt()
|
||||
if not salt:
|
||||
return b""
|
||||
return hmac.new(salt.encode("utf-8"), self.safe_user_id.encode("utf-8"), hashlib.sha256).digest()
|
||||
|
||||
def user_secret_fingerprint(self) -> str:
|
||||
"""Short fingerprint used for diagnostics/audit only (not a key)."""
|
||||
secret = self.derive_user_secret()
|
||||
if not secret:
|
||||
return "salt_not_configured"
|
||||
return hashlib.sha256(secret).hexdigest()[:16]
|
||||
|
||||
def _audit_event(self, action: str, target: str, status: str) -> None:
|
||||
logger.info(
|
||||
f"[flat_context_audit] user={self.safe_user_id} action={action} target={target} status={status}"
|
||||
)
|
||||
|
||||
def _workspace_dir(self) -> Path:
|
||||
root_dir = Path(__file__).resolve().parents[3]
|
||||
return root_dir / "workspace" / f"workspace_{self.safe_user_id}"
|
||||
@@ -98,10 +47,7 @@ class AgentFlatContextStore:
|
||||
return self._workspace_dir() / self.CONTEXT_DIRNAME
|
||||
|
||||
def _context_file(self, filename: str) -> Path:
|
||||
return self._safe_resolve_under(self._context_dir(), str(filename))
|
||||
|
||||
def _workspace_file(self, filename: str) -> Path:
|
||||
return self._safe_resolve_under(self._workspace_dir(), str(filename))
|
||||
return self._context_dir() / filename
|
||||
|
||||
@staticmethod
|
||||
def _estimate_size_bytes(value: Any) -> int:
|
||||
@@ -110,10 +56,6 @@ class AgentFlatContextStore:
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
def estimate_size_bytes(self, value: Any) -> int:
|
||||
"""Public size estimate helper for adapter layers."""
|
||||
return self._estimate_size_bytes(value)
|
||||
|
||||
@staticmethod
|
||||
def _to_context_list(value: Any) -> Any:
|
||||
if value is None:
|
||||
@@ -201,12 +143,6 @@ class AgentFlatContextStore:
|
||||
"preferred": "flat_file",
|
||||
"fallback_order": fallback_order,
|
||||
},
|
||||
"security": {
|
||||
"path_sandboxing": True,
|
||||
"file_permissions": "0600",
|
||||
"directory_permissions": "0700",
|
||||
"user_secret_fingerprint": self.user_secret_fingerprint(),
|
||||
},
|
||||
"context_window_guidance": {
|
||||
"max_raw_bytes": self.DEFAULT_MAX_BYTES,
|
||||
"total_bytes": total_size,
|
||||
@@ -407,7 +343,6 @@ class AgentFlatContextStore:
|
||||
|
||||
def _atomic_write_json(self, target_file: Path, data: Dict[str, Any]) -> None:
|
||||
target_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(target_file.parent, 0o700)
|
||||
fd, tmp_path = tempfile.mkstemp(dir=str(target_file.parent), prefix=f".{target_file.name}.", suffix=".tmp")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
@@ -426,108 +361,6 @@ class AgentFlatContextStore:
|
||||
pass
|
||||
raise
|
||||
|
||||
def _atomic_write_text(self, target_file: Path, content: str) -> None:
|
||||
target_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(target_file.parent, 0o700)
|
||||
fd, tmp_path = tempfile.mkstemp(dir=str(target_file.parent), prefix=f".{target_file.name}.", suffix=".tmp")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
f.write(content)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(tmp_path, target_file)
|
||||
try:
|
||||
os.chmod(target_file, 0o600)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception:
|
||||
try:
|
||||
os.unlink(tmp_path)
|
||||
except Exception:
|
||||
pass
|
||||
raise
|
||||
|
||||
@staticmethod
|
||||
def _collect_signal_terms(doc: Dict[str, Any], limit: int = 6) -> list:
|
||||
summary = doc.get("agent_summary") if isinstance(doc, dict) else {}
|
||||
hints = summary.get("retrieval_hints") if isinstance(summary, dict) else {}
|
||||
terms = hints.get("high_signal_terms") if isinstance(hints, dict) else []
|
||||
if not isinstance(terms, list):
|
||||
return []
|
||||
normalized = [str(t).strip() for t in terms if str(t).strip()]
|
||||
return normalized[:limit]
|
||||
|
||||
@staticmethod
|
||||
def _extract_journey_stage(doc: Dict[str, Any]) -> str:
|
||||
dctx = doc.get("document_context") if isinstance(doc, dict) else {}
|
||||
journey = dctx.get("journey") if isinstance(dctx, dict) else {}
|
||||
stage = journey.get("stage") if isinstance(journey, dict) else ""
|
||||
return str(stage or "").strip()
|
||||
|
||||
@staticmethod
|
||||
def _context_description(filename: str) -> str:
|
||||
descriptions = {
|
||||
AgentFlatContextStore.STEP2_FILENAME: "Primary SEO and site structure context",
|
||||
AgentFlatContextStore.STEP3_FILENAME: "Research depth, competitors, and content preferences",
|
||||
AgentFlatContextStore.STEP4_FILENAME: "Persona profiles, voice adaptation, and platform strategy",
|
||||
AgentFlatContextStore.STEP5_FILENAME: "Connected integrations and provider readiness",
|
||||
}
|
||||
return descriptions.get(filename, "Context document")
|
||||
|
||||
def _generate_workspace_readme(self, manifest: Dict[str, Any]) -> str:
|
||||
docs = manifest.get("documents") if isinstance(manifest, dict) and isinstance(manifest.get("documents"), list) else []
|
||||
|
||||
lines = [
|
||||
"# Agent Workspace Map",
|
||||
"",
|
||||
"You are in a restricted read-only VFS. Use `list_context`, `read_context_file`, and `search_context` to navigate.",
|
||||
"",
|
||||
"## Core Context Files",
|
||||
]
|
||||
|
||||
for item in sorted(docs, key=lambda d: str((d or {}).get("path", ""))):
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
path = item.get("path") or ""
|
||||
if not path:
|
||||
continue
|
||||
doc = self._load_context_document(path) or {}
|
||||
signals = self._collect_signal_terms(doc)
|
||||
journey_stage = self._extract_journey_stage(doc)
|
||||
updated_at = str(item.get("updated_at") or "")
|
||||
lines.append(f"- `{path}`: {self._context_description(path)}.")
|
||||
if signals:
|
||||
lines.append(f" - **Key Signals:** {', '.join(signals)}")
|
||||
if journey_stage:
|
||||
lines.append(f" - **Journey Stage:** {journey_stage}")
|
||||
if updated_at:
|
||||
lines.append(f" - **Updated:** {updated_at}")
|
||||
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"## Retrieval Strategy",
|
||||
"1. Run `list_context` to check which onboarding steps are available.",
|
||||
"2. Run `search_context` for targeted terms (for example: \"competitor\", \"tone\", \"integrations\").",
|
||||
"3. Run `read_context_file` and ingest `agent_summary` before expanding full `data`.",
|
||||
"",
|
||||
"## Virtual Paths",
|
||||
"- `/env/summary` -> consolidated summary generated from all available context docs",
|
||||
f"- `/steps/website` -> `{self.STEP2_FILENAME}`",
|
||||
f"- `/steps/research` -> `{self.STEP3_FILENAME}`",
|
||||
f"- `/steps/persona` -> `{self.STEP4_FILENAME}`",
|
||||
f"- `/steps/integrations` -> `{self.STEP5_FILENAME}`",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
def _update_workspace_readme(self, manifest: Dict[str, Any]) -> None:
|
||||
try:
|
||||
content = self._generate_workspace_readme(manifest)
|
||||
self._atomic_write_text(self._workspace_file(self.WORKSPACE_README), content)
|
||||
except Exception as exc:
|
||||
logger.warning(f"Failed to update workspace README for user {self.user_id}: {exc}")
|
||||
|
||||
def _update_manifest(self, context_type: str, filename: str, doc: Dict[str, Any]) -> None:
|
||||
manifest_file = self._context_file(self.MANIFEST_FILENAME)
|
||||
existing = {}
|
||||
@@ -557,7 +390,6 @@ class AgentFlatContextStore:
|
||||
"documents": items,
|
||||
}
|
||||
self._atomic_write_json(manifest_file, manifest)
|
||||
self._update_workspace_readme(manifest)
|
||||
|
||||
def _save_context_document(
|
||||
self,
|
||||
@@ -604,11 +436,9 @@ class AgentFlatContextStore:
|
||||
|
||||
self._atomic_write_json(target_file, context_doc)
|
||||
self._update_manifest(context_type, filename, context_doc)
|
||||
self._audit_event("write_context", filename, "success")
|
||||
return True
|
||||
except Exception as exc:
|
||||
logger.error(f"Failed to save context for user {self.user_id} ({context_type}): {exc}")
|
||||
self._audit_event("write_context", filename, "error")
|
||||
return False
|
||||
|
||||
def save_step2_website_analysis(self, payload: Dict[str, Any], *, source: str = "onboarding_step2") -> bool:
|
||||
@@ -653,31 +483,19 @@ class AgentFlatContextStore:
|
||||
|
||||
def _load_context_document(self, filename: str) -> Optional[Dict[str, Any]]:
|
||||
try:
|
||||
if str(filename) not in self.ALLOWED_CONTEXT_FILES:
|
||||
logger.warning(f"Rejected non-allowed context filename for user {self.user_id}: {filename}")
|
||||
self._audit_event("read_context", str(filename), "rejected_filename")
|
||||
return None
|
||||
target_file = self._context_file(filename)
|
||||
if not target_file.exists():
|
||||
self._audit_event("read_context", str(filename), "not_found")
|
||||
return None
|
||||
with open(target_file, "r", encoding="utf-8") as f:
|
||||
doc = json.load(f)
|
||||
if isinstance(doc, dict) and str(doc.get("user_id")) != str(self.user_id):
|
||||
logger.warning(f"Context user mismatch for {filename} (expected {self.user_id})")
|
||||
self._audit_event("read_context", str(filename), "user_mismatch")
|
||||
return None
|
||||
self._audit_event("read_context", str(filename), "success")
|
||||
return doc if isinstance(doc, dict) else None
|
||||
except Exception as exc:
|
||||
logger.warning(f"Failed to load context document for user {self.user_id} ({filename}): {exc}")
|
||||
self._audit_event("read_context", str(filename), "error")
|
||||
return None
|
||||
|
||||
def load_context_document(self, filename: str) -> Optional[Dict[str, Any]]:
|
||||
"""Public loader for a named context document file."""
|
||||
return self._load_context_document(filename)
|
||||
|
||||
def load_context_manifest(self) -> Optional[Dict[str, Any]]:
|
||||
return self._load_context_document(self.MANIFEST_FILENAME)
|
||||
|
||||
@@ -708,35 +526,3 @@ class AgentFlatContextStore:
|
||||
def load_step5_integrations(self) -> Optional[Dict[str, Any]]:
|
||||
doc = self.load_step5_context_document()
|
||||
return doc.get("data") if isinstance(doc, dict) and isinstance(doc.get("data"), dict) else None
|
||||
|
||||
def generate_total_summary(self) -> Dict[str, Any]:
|
||||
"""Build a lightweight consolidated summary across available context documents."""
|
||||
manifest = self.load_context_manifest() or {"documents": []}
|
||||
docs = manifest.get("documents") if isinstance(manifest.get("documents"), list) else []
|
||||
overview = []
|
||||
for item in docs:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
path = str(item.get("path") or "")
|
||||
if not path:
|
||||
continue
|
||||
doc = self._load_context_document(path) or {}
|
||||
summary = doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {}
|
||||
quick_facts = summary.get("quick_facts") if isinstance(summary.get("quick_facts"), dict) else {}
|
||||
hints = summary.get("retrieval_hints") if isinstance(summary.get("retrieval_hints"), dict) else {}
|
||||
overview.append(
|
||||
{
|
||||
"path": path,
|
||||
"context_type": doc.get("context_type"),
|
||||
"updated_at": doc.get("updated_at") or item.get("updated_at"),
|
||||
"journey_stage": self._extract_journey_stage(doc),
|
||||
"high_signal_terms": hints.get("high_signal_terms") if isinstance(hints.get("high_signal_terms"), list) else [],
|
||||
"quick_facts": quick_facts,
|
||||
}
|
||||
)
|
||||
return {
|
||||
"user_id": str(self.user_id),
|
||||
"generated_at": datetime.utcnow().isoformat(),
|
||||
"document_count": len(overview),
|
||||
"documents": overview,
|
||||
}
|
||||
|
||||
@@ -340,46 +340,6 @@ class SIFIntegrationService:
|
||||
logger.warning(f"Failed to load flat context manifest for user {self.user_id}: {e}")
|
||||
return {"source": "none", "data": {"documents": []}}
|
||||
|
||||
async def get_merged_flat_context(self) -> Dict[str, Any]:
|
||||
"""Return merged onboarding context from all available flat context documents.
|
||||
|
||||
This is an aggregation helper; step-specific APIs still return one-by-one files.
|
||||
"""
|
||||
store = AgentFlatContextStore(self.user_id)
|
||||
manifest = store.load_context_manifest() or {"documents": []}
|
||||
docs = manifest.get("documents") if isinstance(manifest.get("documents"), list) else []
|
||||
|
||||
merged: Dict[str, Any] = {
|
||||
"source": "flat_file",
|
||||
"user_id": self.user_id,
|
||||
"manifest_updated_at": manifest.get("updated_at"),
|
||||
"steps": {},
|
||||
"agent_summaries": {},
|
||||
"documents": [],
|
||||
}
|
||||
|
||||
for item in docs:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
path = item.get("path")
|
||||
if not path:
|
||||
continue
|
||||
doc = store.load_context_document(str(path)) or {}
|
||||
context_type = str(doc.get("context_type") or item.get("type") or path)
|
||||
merged["documents"].append(
|
||||
{
|
||||
"path": path,
|
||||
"context_type": context_type,
|
||||
"updated_at": doc.get("updated_at") or item.get("updated_at"),
|
||||
"size_bytes": item.get("size_bytes"),
|
||||
}
|
||||
)
|
||||
merged["steps"][context_type] = doc.get("data") if isinstance(doc.get("data"), dict) else {}
|
||||
merged["agent_summaries"][context_type] = doc.get("agent_summary") if isinstance(doc.get("agent_summary"), dict) else {}
|
||||
|
||||
merged["document_count"] = len(merged["documents"])
|
||||
return merged
|
||||
|
||||
async def index_market_trends_run(self, trends_result: Dict[str, Any], run_id: str) -> bool:
|
||||
try:
|
||||
latest_id = f"market_trends_latest:{self.user_id}"
|
||||
|
||||
@@ -67,11 +67,10 @@ import sys
|
||||
from pathlib import Path
|
||||
import google.genai as genai
|
||||
from google.genai import types
|
||||
from dotenv import load_dotenv
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from utils.logger_utils import get_service_logger
|
||||
from services.api_key_manager import APIKeyManager
|
||||
|
||||
# Use service-specific logger to avoid conflicts
|
||||
logger = get_service_logger("gemini_audio_text")
|
||||
|
||||
@@ -250,6 +250,10 @@ def huggingface_text_response(
|
||||
|
||||
logger.info("🚀 Making Hugging Face API call (chat completion)...")
|
||||
|
||||
# Add rate limiting to prevent expensive API calls
|
||||
import time
|
||||
time.sleep(1) # 1 second delay between API calls
|
||||
|
||||
response = None
|
||||
last_error = None
|
||||
for candidate_model in _fallback_model_sequence(model):
|
||||
@@ -399,6 +403,10 @@ def huggingface_structured_json_response(
|
||||
json_schema_str = json.dumps(schema, indent=2)
|
||||
messages[-1]["content"] += f"\n\nJSON Schema:\n{json_schema_str}"
|
||||
|
||||
# Add rate limiting to prevent expensive API calls
|
||||
import time
|
||||
time.sleep(1) # 1 second delay between API calls
|
||||
|
||||
try:
|
||||
response = None
|
||||
last_error = None
|
||||
|
||||
@@ -55,9 +55,6 @@ def _select_provider(explicit: Optional[str]) -> str:
|
||||
def _get_provider_client(provider_name: str, api_key: Optional[str] = None):
|
||||
"""Get the client for the specified provider."""
|
||||
if provider_name == "wavespeed":
|
||||
api_key = api_key or os.getenv("WAVESPEED_API_KEY")
|
||||
if not api_key:
|
||||
raise RuntimeError("WAVESPEED_API_KEY is required for WaveSpeed image editing. Set it in your .env file.")
|
||||
return WaveSpeedEditProvider(api_key=api_key)
|
||||
|
||||
if not HF_HUB_AVAILABLE:
|
||||
@@ -66,7 +63,7 @@ def _get_provider_client(provider_name: str, api_key: Optional[str] = None):
|
||||
if provider_name == "huggingface":
|
||||
api_key = api_key or os.getenv("HF_TOKEN")
|
||||
if not api_key:
|
||||
raise RuntimeError("HF_TOKEN is required for Hugging Face image editing. Set it in your .env file.")
|
||||
raise RuntimeError("HF_TOKEN is required for Hugging Face image editing")
|
||||
# Use fal-ai provider for fast inference via HF Inference API
|
||||
return InferenceClient(provider="fal-ai", api_key=api_key)
|
||||
|
||||
@@ -102,53 +99,35 @@ def edit_image(
|
||||
"""
|
||||
# PRE-FLIGHT VALIDATION: Validate image editing before API call
|
||||
# MUST happen BEFORE any API calls - return immediately if validation fails
|
||||
# Skip validation in podcast-only demo mode or if explicitly disabled
|
||||
skip_validation = os.getenv("ALWRITY_SKIP_IMAGE_EDITING_VALIDATION", "false").lower() in ("true", "1", "yes")
|
||||
|
||||
if user_id and not skip_validation:
|
||||
from services.database import get_session_for_user
|
||||
if user_id:
|
||||
from services.database import get_db
|
||||
from services.subscription import PricingService
|
||||
from services.subscription.preflight_validator import validate_image_editing_operations
|
||||
from fastapi import HTTPException
|
||||
|
||||
logger.info(f"[Image Editing] 🔍 Starting pre-flight validation for user_id={user_id}")
|
||||
|
||||
db = None
|
||||
# Note: get_db() is a generator, so we need to use next() to get the session
|
||||
# and ensure we close it in the finally block
|
||||
db = next(get_db())
|
||||
try:
|
||||
# Use get_session_for_user instead of get_db() since we're outside FastAPI DI
|
||||
db = get_session_for_user(user_id)
|
||||
if not db:
|
||||
logger.warning(f"[Image Editing] ⚠️ Could not get DB session for user {user_id} - skipping validation")
|
||||
else:
|
||||
pricing_service = PricingService(db)
|
||||
# Raises HTTPException immediately if validation fails - frontend gets immediate response
|
||||
validate_image_editing_operations(
|
||||
pricing_service=pricing_service,
|
||||
user_id=user_id
|
||||
)
|
||||
logger.info(f"[Image Editing] ✅ Pre-flight validation passed for user_id={user_id} - proceeding with image editing")
|
||||
pricing_service = PricingService(db)
|
||||
# Raises HTTPException immediately if validation fails - frontend gets immediate response
|
||||
validate_image_editing_operations(
|
||||
pricing_service=pricing_service,
|
||||
user_id=user_id
|
||||
)
|
||||
logger.info(f"[Image Editing] ✅ Pre-flight validation passed for user_id={user_id} - proceeding with image editing")
|
||||
except HTTPException as http_ex:
|
||||
# Re-raise immediately - don't proceed with API call
|
||||
logger.error(f"[Image Editing] ❌ Pre-flight validation failed for user_id={user_id} - blocking API call: {http_ex.detail}")
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"[Image Editing] ❌ Unexpected error during pre-flight validation: {e}")
|
||||
# In podcast-only mode, allow the operation to continue on validation errors
|
||||
if os.getenv("ALWRITY_ENABLED_FEATURES") == "podcast":
|
||||
logger.warning(f"[Image Editing] ⚠️ Validation error in podcast mode - allowing operation to continue")
|
||||
else:
|
||||
raise HTTPException(status_code=500, detail=f"Image editing validation failed: {str(e)}")
|
||||
raise HTTPException(status_code=500, detail=f"Image editing validation failed: {str(e)}")
|
||||
finally:
|
||||
if db:
|
||||
try:
|
||||
db.close()
|
||||
except Exception as close_err:
|
||||
logger.warning(f"[Image Editing] Error closing DB session: {close_err}")
|
||||
db.close()
|
||||
else:
|
||||
if skip_validation:
|
||||
logger.info(f"[Image Editing] ⚡ Skipping pre-flight validation (ALWRITY_SKIP_IMAGE_EDITING_VALIDATION=true)")
|
||||
else:
|
||||
logger.warning(f"[Image Editing] ⚠️ No user_id provided - skipping pre-flight validation")
|
||||
logger.warning(f"[Image Editing] ⚠️ No user_id provided - skipping pre-flight validation (this should not happen in production)")
|
||||
|
||||
# Validate input
|
||||
if not input_image_bytes:
|
||||
|
||||
@@ -6,7 +6,6 @@ migrated from the legacy lib/gpt_providers/text_generation/main_text_generation.
|
||||
|
||||
import os
|
||||
import json
|
||||
import time
|
||||
from typing import Optional, Dict, Any, List
|
||||
from datetime import datetime
|
||||
from loguru import logger
|
||||
@@ -212,7 +211,7 @@ def llm_text_gen(
|
||||
provider_enum = APIProvider.MISTRAL # HuggingFace maps to Mistral enum for usage tracking
|
||||
actual_provider_name = "huggingface" # Keep actual provider name for logs
|
||||
elif gpt_provider == "wavespeed":
|
||||
provider_enum = APIProvider.WAVESPEED
|
||||
provider_enum = APIProvider.OPENAI # Map to OpenAI for tracking purposes
|
||||
actual_provider_name = "wavespeed"
|
||||
elif gpt_provider == "openai":
|
||||
provider_enum = APIProvider.OPENAI
|
||||
@@ -226,8 +225,6 @@ def llm_text_gen(
|
||||
if not user_id:
|
||||
raise RuntimeError("user_id is required for subscription checking. Please provide Clerk user ID.")
|
||||
|
||||
sub_check_start = time.time()
|
||||
logger.warning(f"[llm_text_gen][{flow_tag}] Subscription check START for user {user_id}")
|
||||
try:
|
||||
from services.database import get_session_for_user
|
||||
from services.subscription import UsageTrackingService, PricingService
|
||||
@@ -289,8 +286,6 @@ def llm_text_gen(
|
||||
logger.info(f"[llm_text_gen] Subscription check passed for user {user_id}: provider={actual_provider_name or gpt_provider}, tokens_requested={estimated_total_tokens}, new_user_no_usage_record")
|
||||
|
||||
finally:
|
||||
sub_check_ms = (time.time() - sub_check_start) * 1000
|
||||
logger.warning(f"[llm_text_gen][{flow_tag}] Subscription check took {sub_check_ms:.0f}ms for user {user_id}")
|
||||
db.close()
|
||||
except HTTPException:
|
||||
# Re-raise HTTPExceptions (e.g., 429 subscription limit) - preserve error details
|
||||
@@ -300,8 +295,7 @@ def llm_text_gen(
|
||||
raise
|
||||
except Exception as sub_error:
|
||||
# STRICT: Fail on subscription check errors
|
||||
sub_check_ms = (time.time() - sub_check_start) * 1000
|
||||
logger.error(f"[llm_text_gen][{flow_tag}] Subscription check FAILED after {sub_check_ms:.0f}ms for user {user_id}: {sub_error}")
|
||||
logger.error(f"[llm_text_gen] Subscription check failed for user {user_id}: {sub_error}")
|
||||
raise RuntimeError(f"Subscription check failed: {str(sub_error)}")
|
||||
|
||||
# Construct the system prompt if not provided
|
||||
@@ -372,7 +366,6 @@ def llm_text_gen(
|
||||
)
|
||||
elif gpt_provider == "wavespeed":
|
||||
from services.llm_providers.wavespeed_provider import wavespeed_text_response
|
||||
llm_start = time.time()
|
||||
response_text = wavespeed_text_response(
|
||||
prompt=prompt,
|
||||
model=model or "openai/gpt-oss-120b",
|
||||
@@ -381,8 +374,6 @@ def llm_text_gen(
|
||||
top_p=top_p,
|
||||
system_prompt=system_instructions
|
||||
)
|
||||
llm_ms = (time.time() - llm_start) * 1000
|
||||
logger.warning(f"[llm_text_gen][{flow_tag}] LLM API call took {llm_ms:.0f}ms for user {user_id} (wavespeed)")
|
||||
else:
|
||||
logger.error(f"[llm_text_gen] Unknown provider: {gpt_provider}")
|
||||
raise RuntimeError(f"Unknown LLM provider: {gpt_provider}. Supported providers: google, huggingface, wavespeed")
|
||||
|
||||
@@ -274,6 +274,10 @@ def wavespeed_text_response(
|
||||
|
||||
logger.info("🚀 Making WaveSpeed API call (chat completion)...")
|
||||
|
||||
# Add rate limiting to prevent expensive API calls
|
||||
import time
|
||||
time.sleep(1) # 1 second delay between API calls
|
||||
|
||||
# Call exactly the requested model; no retries, no fallbacks, no variants
|
||||
response = client.chat.completions.create(
|
||||
model=model,
|
||||
@@ -422,6 +426,10 @@ def wavespeed_structured_json_response(
|
||||
json_schema_str = json.dumps(schema, indent=2)
|
||||
messages[-1]["content"] += f"\n\nJSON Schema:\n{json_schema_str}"
|
||||
|
||||
# Add rate limiting to prevent expensive API calls
|
||||
import time
|
||||
time.sleep(1) # 1 second delay between API calls
|
||||
|
||||
try:
|
||||
response = None
|
||||
last_error = None
|
||||
|
||||
@@ -18,12 +18,9 @@ import json
|
||||
from services.database import get_db_session
|
||||
from models.onboarding import OnboardingSession, WebsiteAnalysis, ResearchPreferences
|
||||
from models.persona_models import WritingPersona, PlatformPersona, PersonaAnalysisResult
|
||||
|
||||
def _get_podcast_mode():
|
||||
"""Check if running in podcast-only mode to skip heavy initialization."""
|
||||
import os
|
||||
env_val = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower()
|
||||
return env_val == "podcast"
|
||||
from services.persona.core_persona import CorePersonaService, OnboardingDataCollector
|
||||
from services.persona.linkedin.linkedin_persona_service import LinkedInPersonaService
|
||||
from services.persona.facebook.facebook_persona_service import FacebookPersonaService
|
||||
|
||||
class PersonaAnalysisService:
|
||||
"""Service for analyzing onboarding data and generating writing personas using Gemini AI."""
|
||||
@@ -40,40 +37,12 @@ class PersonaAnalysisService:
|
||||
def __init__(self):
|
||||
"""Initialize the persona analysis service (only once)."""
|
||||
if not self._initialized:
|
||||
# Skip heavy initialization in podcast-only mode
|
||||
if _get_podcast_mode():
|
||||
logger.debug("PersonaAnalysisService: Skipping heavy init in podcast mode")
|
||||
self._initialized = True
|
||||
return
|
||||
|
||||
# Only initialize heavy services when needed (not at import time)
|
||||
self._heavy_init_done = False
|
||||
|
||||
def _ensure_heavy_init(self):
|
||||
"""Lazily initialize heavy services only when first used."""
|
||||
if self._heavy_init_done:
|
||||
return
|
||||
|
||||
# Check again in case mode changed
|
||||
if _get_podcast_mode():
|
||||
logger.debug("PersonaAnalysisService: Skipping heavy init in podcast mode")
|
||||
self._heavy_init_done = True
|
||||
return
|
||||
|
||||
try:
|
||||
from services.persona.core_persona import CorePersonaService, OnboardingDataCollector
|
||||
from services.persona.linkedin.linkedin_persona_service import LinkedInPersonaService
|
||||
from services.persona.facebook.facebook_persona_service import FacebookPersonaService
|
||||
|
||||
self.core_persona_service = CorePersonaService()
|
||||
self.data_collector = OnboardingDataCollector()
|
||||
self.linkedin_service = LinkedInPersonaService()
|
||||
self.facebook_service = FacebookPersonaService()
|
||||
self._heavy_init_done = True
|
||||
logger.debug("PersonaAnalysisService initialized (lazy)")
|
||||
except Exception as e:
|
||||
logger.warning(f"PersonaAnalysisService: Failed to initialize heavy services: {e}")
|
||||
self._heavy_init_done = True
|
||||
logger.debug("PersonaAnalysisService initialized")
|
||||
self._initialized = True
|
||||
|
||||
def generate_persona_from_onboarding(self, user_id: str, onboarding_session_id: int = None) -> Dict[str, Any]:
|
||||
"""
|
||||
@@ -86,13 +55,6 @@ class PersonaAnalysisService:
|
||||
Returns:
|
||||
Generated persona data with platform adaptations
|
||||
"""
|
||||
# Ensure heavy services are initialized
|
||||
self._ensure_heavy_init()
|
||||
|
||||
# Check if heavy init failed (podcast mode)
|
||||
if not getattr(self, '_heavy_init_done', False):
|
||||
return {"error": "Persona service unavailable in podcast-only mode"}
|
||||
|
||||
try:
|
||||
logger.info(f"Generating persona for user {user_id}")
|
||||
|
||||
|
||||
@@ -1,623 +0,0 @@
|
||||
"""
|
||||
Programmatic B-Roll Composer
|
||||
Layered composition pipeline: Background + Chart + Avatar Circle + Text Overlays
|
||||
"""
|
||||
|
||||
import json
|
||||
import numpy as np
|
||||
from pathlib import Path
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
import matplotlib.patches as mpatches
|
||||
from moviepy import (
|
||||
VideoFileClip, ImageClip, CompositeVideoClip,
|
||||
concatenate_videoclips,
|
||||
)
|
||||
import moviepy.video.fx as vfx
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Crossfade concat (Option 1: crossfadein + negative padding)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def crossfade_concat(scenes: list, fade_dur: float = 0.5):
|
||||
"""
|
||||
Concatenate scenes with a dissolve transition between each pair.
|
||||
|
||||
Each clip (except the first) gets a crossfadein effect.
|
||||
padding=-fade_dur overlaps consecutive clips so the fade actually fires
|
||||
instead of creating a black gap. set_duration on every scene is
|
||||
mandatory — CompositeVideoClip.duration can be ambiguous without it,
|
||||
which makes the overlap math wrong.
|
||||
"""
|
||||
faded = []
|
||||
for i, clip in enumerate(scenes):
|
||||
c = clip
|
||||
if i > 0:
|
||||
c = c.fx(vfx.CrossFadeIn, fade_dur)
|
||||
faded.append(c)
|
||||
return concatenate_videoclips(faded, padding=-int(fade_dur), method="compose")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data structures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class Insight:
|
||||
key_insight: str
|
||||
supporting_stat: str
|
||||
visual_cue: str # bar_comparison|bar_horizontal|line_trend|pie|stacked_bar|bullet_points|full_avatar
|
||||
audio_tone: str
|
||||
chart_data: dict = field(default_factory=dict)
|
||||
duration: float = 10.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class SceneAssets:
|
||||
background_img: str
|
||||
chart_img: Optional[str] = None
|
||||
avatar_video: Optional[str] = None
|
||||
bullet_img: Optional[str] = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Chart generator (Matplotlib → PNG with transparency)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
CHART_STYLE = {
|
||||
"bg": "#0D0D0D",
|
||||
"bar_before": "#2E4057",
|
||||
"bar_after": "#E63946",
|
||||
"text": "#F1F1EF",
|
||||
"grid": "#2A2A2A",
|
||||
"accent": "#E63946",
|
||||
"pie_colors": ["#E63946", "#2E4057", "#457B9D", "#A8DADC", "#F4A261", "#2A9D8F"],
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Chart generators (Matplotlib → PNG with transparency)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def make_bar_chart(data: dict, out_path: str, title: str = "",
|
||||
show_legend: bool = True, value_suffix: str = "%",
|
||||
subtitle: str = "") -> str:
|
||||
"""Render a side-by-side comparison bar chart. Returns output path."""
|
||||
labels = data.get("labels", [])
|
||||
before = data.get("before", [])
|
||||
after = data.get("after", [])
|
||||
|
||||
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||
ax.set_facecolor("none")
|
||||
|
||||
x = np.arange(len(labels))
|
||||
w = 0.35
|
||||
bars_b = ax.bar(x - w / 2, before, w, color=CHART_STYLE["bar_before"],
|
||||
label="Before", zorder=3, edgecolor="none")
|
||||
bars_a = ax.bar(x + w / 2, after, w, color=CHART_STYLE["bar_after"],
|
||||
label="After", zorder=3, edgecolor="none")
|
||||
|
||||
ax.set_xticks(x)
|
||||
ax.set_xticklabels(labels, color=CHART_STYLE["text"], fontsize=11)
|
||||
ax.tick_params(axis="y", colors=CHART_STYLE["text"])
|
||||
ax.spines[:].set_visible(False)
|
||||
ax.yaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||
ax.set_axisbelow(True)
|
||||
|
||||
for bar in [*bars_b, *bars_a]:
|
||||
h = bar.get_height()
|
||||
ax.text(bar.get_x() + bar.get_width() / 2, h + 0.5, f"{h:.0f}{value_suffix}",
|
||||
ha="center", va="bottom", color=CHART_STYLE["text"], fontsize=9,
|
||||
fontweight="bold")
|
||||
|
||||
if show_legend:
|
||||
legend = ax.legend(frameon=False, labelcolor=CHART_STYLE["text"],
|
||||
fontsize=10, loc="upper left")
|
||||
|
||||
# Add title and optional subtitle
|
||||
if title:
|
||||
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||
fontweight="bold", pad=12)
|
||||
if subtitle:
|
||||
fig.text(0.5, 0.02, subtitle, ha='center', color=CHART_STYLE["text"],
|
||||
fontsize=10, style='italic')
|
||||
|
||||
fig.tight_layout(pad=0.5, rect=(0, 0.03 if subtitle else 0, 1, 1))
|
||||
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||
plt.close(fig)
|
||||
return out_path
|
||||
|
||||
|
||||
def make_horizontal_bar(data: dict, out_path: str, title: str = "",
|
||||
value_suffix: str = "%", bar_color: str = None) -> str:
|
||||
"""Render a horizontal bar chart (good for rankings/lists)."""
|
||||
labels = data.get("labels", [])
|
||||
values = data.get("values", data.get("y", []))
|
||||
|
||||
if not values:
|
||||
return ""
|
||||
|
||||
bar_color = bar_color or CHART_STYLE["bar_after"]
|
||||
|
||||
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||
ax.set_facecolor("none")
|
||||
|
||||
y_pos = np.arange(len(labels))
|
||||
bars = ax.barh(y_pos, values, color=bar_color, zorder=3, edgecolor="none", height=0.6)
|
||||
|
||||
ax.set_yticks(y_pos)
|
||||
ax.set_yticklabels(labels, color=CHART_STYLE["text"], fontsize=11)
|
||||
ax.tick_params(axis="x", colors=CHART_STYLE["text"])
|
||||
ax.spines[:].set_visible(False)
|
||||
ax.xaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||
ax.set_axisbelow(True)
|
||||
ax.invert_yaxis()
|
||||
|
||||
for i, bar in enumerate(bars):
|
||||
width = bar.get_width()
|
||||
ax.text(width + 0.5, bar.get_y() + bar.get_height()/2, f"{width:.0f}{value_suffix}",
|
||||
ha="left", va="center", color=CHART_STYLE["text"], fontsize=10,
|
||||
fontweight="bold")
|
||||
|
||||
if title:
|
||||
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||
fontweight="bold", pad=12)
|
||||
|
||||
fig.tight_layout(pad=0.5)
|
||||
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||
plt.close(fig)
|
||||
return out_path
|
||||
|
||||
|
||||
def make_pie_chart(data: dict, out_path: str, title: str = "",
|
||||
show_labels: bool = True, show_percent: bool = True,
|
||||
donut: bool = False) -> str:
|
||||
"""Render a pie chart."""
|
||||
labels = data.get("labels", [])
|
||||
values = data.get("values", data.get("y", []))
|
||||
|
||||
if not values:
|
||||
return ""
|
||||
|
||||
colors = CHART_STYLE["pie_colors"][:len(values)]
|
||||
|
||||
fig, ax = plt.subplots(figsize=(6, 4.5), facecolor="none")
|
||||
ax.set_facecolor("none")
|
||||
|
||||
if donut:
|
||||
wedges, texts, autotexts = ax.pie(
|
||||
values, labels=labels if show_labels else None,
|
||||
colors=colors, autopct=lambda p: f'{p:.1f}%' if show_percent else '',
|
||||
startangle=90, pctdistance=0.75,
|
||||
wedgeprops=dict(width=0.5, edgecolor="none")
|
||||
)
|
||||
else:
|
||||
wedges, texts, autotexts = ax.pie(
|
||||
values, labels=labels if show_labels else None,
|
||||
colors=colors, autopct=lambda p: f'{p:.1f}%' if show_percent else '',
|
||||
startangle=90, pctdistance=0.8
|
||||
)
|
||||
|
||||
for text in texts:
|
||||
text.set_color(CHART_STYLE["text"])
|
||||
text.set_fontsize(10)
|
||||
|
||||
for autotext in autotexts:
|
||||
autotext.set_color(CHART_STYLE["text"])
|
||||
autotext.set_fontsize(9)
|
||||
autotext.set_fontweight("bold")
|
||||
|
||||
if title:
|
||||
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||
fontweight="bold", pad=12)
|
||||
|
||||
fig.tight_layout(pad=0.5)
|
||||
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||
plt.close(fig)
|
||||
return out_path
|
||||
|
||||
|
||||
def make_stacked_bar(data: dict, out_path: str, title: str = "",
|
||||
stack_labels: list = None) -> str:
|
||||
"""Render a stacked bar chart."""
|
||||
labels = data.get("labels", [])
|
||||
stacks = data.get("stacks", []) # List of lists, each inner list is a stack
|
||||
|
||||
if not stacks or len(stacks) < 2:
|
||||
return ""
|
||||
|
||||
stack_labels = stack_labels or [f"Series {i+1}" for i in range(len(stacks))]
|
||||
|
||||
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||
ax.set_facecolor("none")
|
||||
|
||||
x = np.arange(len(labels))
|
||||
bottom = np.zeros(len(labels))
|
||||
colors = CHART_STYLE["pie_colors"][:len(stacks)]
|
||||
|
||||
for i, stack in enumerate(stacks):
|
||||
bars = ax.bar(x, stack, 0.6, bottom=bottom, color=colors[i],
|
||||
label=stack_labels[i], zorder=3, edgecolor="none")
|
||||
|
||||
for j, bar in enumerate(bars):
|
||||
height = bar.get_height()
|
||||
if height > 5: # Only show label if segment is big enough
|
||||
ax.text(bar.get_x() + bar.get_width()/2,
|
||||
bottom[j] + height/2,
|
||||
f"{height:.0f}", ha="center", va="center",
|
||||
color=CHART_STYLE["text"], fontsize=8, fontweight="bold")
|
||||
|
||||
bottom = bottom + np.array(stack)
|
||||
|
||||
ax.set_xticks(x)
|
||||
ax.set_xticklabels(labels, color=CHART_STYLE["text"], fontsize=11)
|
||||
ax.tick_params(axis="y", colors=CHART_STYLE["text"])
|
||||
ax.spines[:].set_visible(False)
|
||||
ax.legend(frameon=False, labelcolor=CHART_STYLE["text"], fontsize=9, loc="upper left")
|
||||
|
||||
if title:
|
||||
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||
fontweight="bold", pad=12)
|
||||
|
||||
fig.tight_layout(pad=0.5)
|
||||
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||
plt.close(fig)
|
||||
return out_path
|
||||
|
||||
|
||||
def make_line_trend(data: dict, out_path: str, title: str = "") -> str:
|
||||
"""Render a trend line chart. Returns output path."""
|
||||
x_labels = data.get("labels", data.get("x", []))
|
||||
y_vals = data.get("values", data.get("y", []))
|
||||
|
||||
if not x_labels or not y_vals:
|
||||
return ""
|
||||
|
||||
fig, ax = plt.subplots(figsize=(8, 4.5), facecolor="none")
|
||||
ax.set_facecolor("none")
|
||||
|
||||
try:
|
||||
x_vals = [float(v) for v in x_labels]
|
||||
except (ValueError, TypeError):
|
||||
x_vals = list(range(len(x_labels)))
|
||||
|
||||
ax.plot(x_vals, y_vals, color=CHART_STYLE["accent"],
|
||||
linewidth=2.5, marker="o", markersize=7, zorder=3)
|
||||
ax.fill_between(x_vals, y_vals, alpha=0.12, color=CHART_STYLE["accent"])
|
||||
ax.spines[:].set_visible(False)
|
||||
ax.tick_params(colors=CHART_STYLE["text"])
|
||||
ax.yaxis.grid(True, color=CHART_STYLE["grid"], linewidth=0.6, zorder=0)
|
||||
|
||||
try:
|
||||
x_labels_f = [float(v) for v in x_labels]
|
||||
except (ValueError, TypeError):
|
||||
ax.set_xticks(x_vals)
|
||||
ax.set_xticklabels(x_labels, color=CHART_STYLE["text"], fontsize=10)
|
||||
|
||||
if title:
|
||||
ax.set_title(title, color=CHART_STYLE["text"], fontsize=13,
|
||||
fontweight="bold", pad=12)
|
||||
fig.tight_layout(pad=0.5)
|
||||
fig.savefig(out_path, dpi=150, transparent=True, bbox_inches="tight")
|
||||
plt.close(fig)
|
||||
return out_path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Text / Bullet overlay (Pillow → PNG)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def make_bullet_overlay(lines: list[str], out_path: str,
|
||||
width: int = 900, font_size: int = 32) -> str:
|
||||
"""Render bullet points on a semi-transparent dark pill. Returns path."""
|
||||
padding = 32
|
||||
line_h = font_size + 16
|
||||
img_h = padding * 2 + len(lines) * line_h + 12
|
||||
img = Image.new("RGBA", (width, img_h), (0, 0, 0, 0))
|
||||
draw = ImageDraw.Draw(img)
|
||||
|
||||
draw.rounded_rectangle([0, 0, width - 1, img_h - 1],
|
||||
radius=18, fill=(10, 10, 10, 185))
|
||||
|
||||
try:
|
||||
font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
|
||||
font_size)
|
||||
except OSError:
|
||||
font = ImageFont.load_default()
|
||||
|
||||
y = padding
|
||||
for line in lines:
|
||||
draw.text((padding + 18, y), f"• {line}", font=font, fill=(241, 241, 239, 255))
|
||||
y += line_h
|
||||
|
||||
img.save(out_path, format="PNG")
|
||||
return out_path
|
||||
|
||||
|
||||
def make_insight_card(insight: str, stat: str, out_path: str,
|
||||
width: int = 960, height: int = 200) -> str:
|
||||
"""Render a bold insight card (headline + supporting stat). Returns path."""
|
||||
img = Image.new("RGBA", (width, height), (0, 0, 0, 0))
|
||||
draw = ImageDraw.Draw(img)
|
||||
draw.rounded_rectangle([0, 0, width - 1, height - 1],
|
||||
radius=14, fill=(10, 10, 10, 200))
|
||||
|
||||
draw.rectangle([28, 24, 36, height - 24], fill=(230, 57, 70, 255))
|
||||
|
||||
try:
|
||||
font_lg = ImageFont.truetype(
|
||||
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", 34)
|
||||
font_sm = ImageFont.truetype(
|
||||
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 20)
|
||||
except OSError:
|
||||
font_lg = font_sm = ImageFont.load_default()
|
||||
|
||||
draw.text((58, 36), insight, font=font_lg, fill=(241, 241, 239, 255))
|
||||
draw.text((58, 90), stat, font=font_sm, fill=(180, 180, 178, 230))
|
||||
|
||||
img.save(out_path, format="PNG")
|
||||
return out_path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Circular avatar mask
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def apply_circle_mask(clip: VideoFileClip, diameter: int) -> VideoFileClip:
|
||||
"""Resize clip and apply a circular alpha mask."""
|
||||
clip = clip.resize(height=diameter)
|
||||
w, h = clip.size
|
||||
|
||||
Y, X = np.ogrid[:h, :w]
|
||||
cx, cy = w / 2, h / 2
|
||||
mask_arr = ((X - cx) ** 2 + (Y - cy) ** 2 <= (min(w, h) / 2) ** 2).astype(float)
|
||||
|
||||
mask_clip = ImageClip(mask_arr, ismask=True).set_duration(clip.duration)
|
||||
return clip.set_mask(mask_clip)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Ken Burns zoom effect
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def ken_burns(clip: ImageClip, zoom_ratio: float = 0.08) -> ImageClip:
|
||||
"""Apply a slow zoom-in over the clip duration."""
|
||||
def zoom_frame(get_frame, t):
|
||||
frame = get_frame(t)
|
||||
frac = 1 + zoom_ratio * (t / clip.duration)
|
||||
h, w = frame.shape[:2]
|
||||
new_h, new_w = int(h / frac), int(w / frac)
|
||||
y1 = (h - new_h) // 2
|
||||
x1 = (w - new_w) // 2
|
||||
cropped = frame[y1:y1 + new_h, x1:x1 + new_w]
|
||||
return np.array(Image.fromarray(cropped).resize((w, h), Image.LANCZOS))
|
||||
|
||||
return clip.fl(zoom_frame, apply_to=["mask"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scene builders (one per visual_cue type)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def build_data_scene(assets: SceneAssets, insight: Insight) -> CompositeVideoClip:
|
||||
"""
|
||||
Layout: Background (Ken Burns) + Chart (fade-in) + Avatar circle (corner) + Insight card
|
||||
"""
|
||||
d = insight.duration
|
||||
layers = []
|
||||
|
||||
bg = (ImageClip(assets.background_img)
|
||||
.set_duration(d)
|
||||
.resize(height=1080))
|
||||
bg = ken_burns(bg)
|
||||
bg = bg.fx(vfx.lum_contrast, 0, -40)
|
||||
layers.append(bg)
|
||||
|
||||
if assets.chart_img:
|
||||
chart = (ImageClip(assets.chart_img)
|
||||
.set_duration(d - 1.5)
|
||||
.set_start(0.5)
|
||||
.resize(width=700)
|
||||
.set_position(("center", 180))
|
||||
.fx(vfx.fadein, 0.6)
|
||||
.fx(vfx.fadeout, 0.4))
|
||||
layers.append(chart)
|
||||
|
||||
card_path = "/tmp/insight_card.png"
|
||||
make_insight_card(insight.key_insight, insight.supporting_stat, card_path)
|
||||
card = (ImageClip(card_path)
|
||||
.set_duration(d - 1)
|
||||
.set_start(0.5)
|
||||
.set_position(("center", 820))
|
||||
.fx(vfx.fadein, 0.5))
|
||||
layers.append(card)
|
||||
|
||||
if assets.avatar_video:
|
||||
avatar_raw = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||
avatar = apply_circle_mask(avatar_raw, diameter=240)
|
||||
avatar = avatar.set_position((bg.w - 280, bg.h - 280))
|
||||
layers.append(avatar)
|
||||
|
||||
return CompositeVideoClip(layers, size=bg.size).set_duration(d)
|
||||
|
||||
|
||||
def build_bullet_scene(assets: SceneAssets, insight: Insight,
|
||||
bullets: list[str]) -> CompositeVideoClip:
|
||||
"""
|
||||
Layout: AI image (Ken Burns) + Bullet overlay + Avatar circle
|
||||
"""
|
||||
d = insight.duration
|
||||
layers = []
|
||||
|
||||
bg = (ImageClip(assets.background_img)
|
||||
.set_duration(d)
|
||||
.resize(height=1080))
|
||||
bg = ken_burns(bg, zoom_ratio=0.05)
|
||||
bg = bg.fx(vfx.lum_contrast, 0, -50)
|
||||
layers.append(bg)
|
||||
|
||||
bullet_path = "/tmp/bullets.png"
|
||||
make_bullet_overlay(bullets, bullet_path, width=860)
|
||||
bullets_clip = (ImageClip(bullet_path)
|
||||
.set_duration(d - 1)
|
||||
.set_start(0.5)
|
||||
.set_position(("center", "center"))
|
||||
.fx(vfx.fadein, 0.7))
|
||||
layers.append(bullets_clip)
|
||||
|
||||
if assets.avatar_video:
|
||||
avatar_raw = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||
avatar = apply_circle_mask(avatar_raw, diameter=200)
|
||||
avatar = avatar.set_position((bg.w - 240, bg.h - 240))
|
||||
layers.append(avatar)
|
||||
|
||||
return CompositeVideoClip(layers, size=bg.size).set_duration(d)
|
||||
|
||||
|
||||
def build_full_avatar_scene(assets: SceneAssets, insight: Insight) -> VideoFileClip:
|
||||
"""Full-screen avatar — the expensive 'Hook' scene. No overlay."""
|
||||
d = insight.duration
|
||||
avatar = VideoFileClip(assets.avatar_video).subclip(0, d)
|
||||
return avatar.resize(height=1080).set_duration(d)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scene dispatcher — maps visual_cue → builder
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def dispatch_scene(insight: Insight, assets: SceneAssets,
|
||||
bullet_lines: Optional[list[str]] = None):
|
||||
"""Dispatch scene based on visual_cue type."""
|
||||
cue = insight.visual_cue
|
||||
|
||||
if cue == "full_avatar":
|
||||
return build_full_avatar_scene(assets, insight)
|
||||
|
||||
elif cue in ("bar_comparison", "bar_chart_comparison", "bar_horizontal", "line_trend", "pie", "stacked_bar"):
|
||||
chart_path = "/tmp/chart.png"
|
||||
chart_data = insight.chart_data or {}
|
||||
if cue in ("bar_comparison", "bar_chart_comparison"):
|
||||
# Normalize {labels, values} -> {labels, before, after} for make_bar_chart
|
||||
if not chart_data.get("before") and not chart_data.get("after"):
|
||||
values = chart_data.get("values", [])
|
||||
labels = chart_data.get("labels", [])
|
||||
if values and labels:
|
||||
n = min(len(labels), len(values))
|
||||
chart_data = {**chart_data, "labels": labels[:n], "before": [0] * n, "after": values[:n]}
|
||||
make_bar_chart(chart_data, chart_path,
|
||||
title=insight.key_insight)
|
||||
elif cue == "bar_horizontal":
|
||||
make_horizontal_bar(chart_data, chart_path,
|
||||
title=insight.key_insight)
|
||||
elif cue == "line_trend":
|
||||
make_line_trend(chart_data, chart_path,
|
||||
title=insight.key_insight)
|
||||
elif cue == "pie":
|
||||
make_pie_chart(chart_data, chart_path,
|
||||
title=insight.key_insight)
|
||||
elif cue == "stacked_bar":
|
||||
make_stacked_bar(chart_data, chart_path,
|
||||
title=insight.key_insight)
|
||||
assets.chart_img = chart_path
|
||||
return build_data_scene(assets, insight)
|
||||
|
||||
elif cue == "bullet_points":
|
||||
lines = bullet_lines or [insight.key_insight, insight.supporting_stat]
|
||||
return build_bullet_scene(assets, insight, lines)
|
||||
|
||||
else:
|
||||
return build_data_scene(assets, insight)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Master compositor — assembles all scenes into one video
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def compose_video(scenes: list, output_path: str = "output.mp4",
|
||||
fps: int = 24, fade_dur: float = 0.5) -> str:
|
||||
"""Concatenate scenes with crossfade transitions and write final video file."""
|
||||
final = crossfade_concat(scenes, fade_dur=fade_dur)
|
||||
final.write_videofile(
|
||||
output_path,
|
||||
fps=fps,
|
||||
codec="libx264",
|
||||
audio_codec="aac",
|
||||
threads=4,
|
||||
preset="fast",
|
||||
logger=None,
|
||||
)
|
||||
return output_path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# JSON bridge — LLM insight → assets + scene
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def pipeline_from_json(insight_json: str,
|
||||
background_img: str,
|
||||
avatar_video: Optional[str] = None) -> str:
|
||||
"""
|
||||
Full pipeline:
|
||||
1. Parse LLM insight JSON
|
||||
2. Generate chart / overlay assets
|
||||
3. Build scene
|
||||
4. Write video
|
||||
Returns path to output video.
|
||||
"""
|
||||
data = json.loads(insight_json)
|
||||
insight = Insight(**{k: data[k] for k in Insight.__dataclass_fields__ if k in data})
|
||||
assets = SceneAssets(background_img=background_img, avatar_video=avatar_video)
|
||||
scene = dispatch_scene(insight, assets,
|
||||
bullet_lines=data.get("bullet_lines"))
|
||||
out = f"/tmp/scene_{insight.visual_cue}.mp4"
|
||||
compose_video([scene], output_path=out)
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Demo / smoke-test (no real media files needed for chart generation)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
if __name__ == "__main__":
|
||||
sample_bar_data = {
|
||||
"labels": ["Content Velocity", "CTR", "Engagement", "Cost/Lead"],
|
||||
"before": [30, 22, 18, 60],
|
||||
"after": [72, 34, 41, 38],
|
||||
}
|
||||
chart_out = make_bar_chart(
|
||||
sample_bar_data,
|
||||
"/tmp/demo_chart.png",
|
||||
title="AI Tools Impact: Before vs After (2025)",
|
||||
)
|
||||
print(f"Chart saved → {chart_out}")
|
||||
|
||||
bullets = [
|
||||
"AI reduced content cycles by 40% in 2025",
|
||||
"HubSpot: 12% lift in CTR with AI-assisted copy",
|
||||
"Video production cost down 3x with hybrid pipeline",
|
||||
]
|
||||
bullet_out = make_bullet_overlay(bullets, "/tmp/demo_bullets.png")
|
||||
print(f"Bullets saved → {bullet_out}")
|
||||
|
||||
card_out = make_insight_card(
|
||||
"AI tools reduced content cycles by 40%",
|
||||
"HubSpot 2026 report — 12% lift in CTR",
|
||||
"/tmp/demo_card.png",
|
||||
)
|
||||
print(f"Insight card saved → {card_out}")
|
||||
|
||||
sample_json = json.dumps({
|
||||
"key_insight": "AI reduced production time by 40%",
|
||||
"supporting_stat": "HubSpot 2026: 12% CTR lift",
|
||||
"visual_cue": "bar_chart_comparison",
|
||||
"audio_tone": "authoritative_and_surprising",
|
||||
"duration": 8.0,
|
||||
"chart_data": sample_bar_data,
|
||||
})
|
||||
print("\nSample Insight JSON:\n", sample_json)
|
||||
print("\nAll asset generation tests passed.")
|
||||
print("To run full video composition, supply real background_img and avatar_video paths.")
|
||||
@@ -1,377 +0,0 @@
|
||||
"""
|
||||
B-Roll Service - Orchestrator for programmatic B-roll video composition.
|
||||
|
||||
This service handles:
|
||||
- Chart data extraction from research
|
||||
- Individual scene B-roll video generation
|
||||
- Final video composition from multiple B-roll scenes
|
||||
"""
|
||||
|
||||
import json
|
||||
import uuid
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional, List, TYPE_CHECKING
|
||||
from loguru import logger
|
||||
|
||||
# Import chart generators directly
|
||||
from services.podcast.broll_composer import (
|
||||
Insight,
|
||||
SceneAssets,
|
||||
dispatch_scene,
|
||||
compose_video,
|
||||
make_bar_chart,
|
||||
make_horizontal_bar,
|
||||
make_line_trend,
|
||||
make_pie_chart,
|
||||
make_stacked_bar,
|
||||
make_bullet_overlay,
|
||||
make_insight_card,
|
||||
)
|
||||
|
||||
|
||||
class BrollService:
|
||||
"""Orchestrates B-roll composition for podcast scenes."""
|
||||
|
||||
def __init__(self, output_dir: Optional[str] = None, user_id: Optional[str] = None):
|
||||
"""
|
||||
Initialize B-roll service.
|
||||
|
||||
Args:
|
||||
output_dir: Base directory for B-roll output. Defaults to workspace chart directory.
|
||||
user_id: User ID for multi-tenant workspace isolation.
|
||||
"""
|
||||
if output_dir:
|
||||
self.output_dir = Path(output_dir)
|
||||
else:
|
||||
self.output_dir = self._get_chart_dir(user_id)
|
||||
|
||||
self.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
logger.warning(f"[BrollService] Initialized with output directory: {self.output_dir}")
|
||||
|
||||
def _get_chart_dir(self, user_id: Optional[str] = None) -> Path:
|
||||
"""Get chart directory from podcast constants (workspace-aware)."""
|
||||
from api.podcast.constants import get_podcast_media_dir
|
||||
return get_podcast_media_dir("chart", user_id, ensure_exists=True)
|
||||
|
||||
def get_output_path(self, filename: str) -> Path:
|
||||
"""Get output path for a file."""
|
||||
return self.output_dir / filename
|
||||
|
||||
def get_chart_preview_filename(self, chart_id: str) -> str:
|
||||
"""Build deterministic chart preview filename from chart ID."""
|
||||
return f"chart_preview_{chart_id}.png"
|
||||
|
||||
def get_chart_preview_path(self, chart_id: str) -> Path:
|
||||
"""Get deterministic chart preview path from chart ID."""
|
||||
return self.get_output_path(self.get_chart_preview_filename(chart_id))
|
||||
|
||||
def generate_chart_preview(
|
||||
self,
|
||||
chart_data: Dict[str, Any],
|
||||
chart_type: str = "bar_comparison",
|
||||
title: str = "",
|
||||
subtitle: str = "",
|
||||
chart_id: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Generate a chart PNG preview (static, for Write phase).
|
||||
|
||||
Args:
|
||||
chart_data: Chart data dict with labels, before/after, etc.
|
||||
chart_type: Type of chart (bar_comparison, bar_horizontal, line_trend, pie, stacked_bar, bullet)
|
||||
title: Title for the chart
|
||||
subtitle: Optional subtitle at bottom
|
||||
|
||||
Returns:
|
||||
Path to generated PNG file
|
||||
"""
|
||||
resolved_chart_id = chart_id or uuid.uuid4().hex[:8]
|
||||
out_path = str(self.get_chart_preview_path(resolved_chart_id))
|
||||
|
||||
# Debug logging
|
||||
logger.warning(f"[BrollService] Generating: type={chart_type}, data keys={list(chart_data.keys())}")
|
||||
|
||||
try:
|
||||
if chart_type == "bar_comparison":
|
||||
# Accept both formats: {labels, before, after} OR {labels, values}
|
||||
labels = chart_data.get("labels", [])
|
||||
before = chart_data.get("before", [])
|
||||
after = chart_data.get("after", [])
|
||||
# If using new format (labels, values), treat as single bar chart
|
||||
if not before and not after:
|
||||
values = chart_data.get("values", [])
|
||||
if values:
|
||||
# Normalize to same length, truncating or padding as needed
|
||||
n = min(len(labels), len(values))
|
||||
labels = labels[:n]
|
||||
before = [0] * n
|
||||
after = values[:n]
|
||||
# Create modified data dict with proper format for make_bar_chart
|
||||
chart_data_for_render = {
|
||||
"labels": labels,
|
||||
"before": before,
|
||||
"after": after
|
||||
}
|
||||
else:
|
||||
chart_data_for_render = chart_data
|
||||
else:
|
||||
chart_data_for_render = chart_data
|
||||
if not labels or (not before and not after):
|
||||
logger.warning(f"[BrollService] Missing required data for bar_comparison: labels={len(labels)}, before={len(before)}, after={len(after)}")
|
||||
return ""
|
||||
if len(labels) != len(before) or len(labels) != len(after):
|
||||
logger.warning(f"[BrollService] Data shape mismatch: labels={len(labels)}, before={len(before)}, after={len(after)}")
|
||||
return ""
|
||||
make_bar_chart(chart_data_for_render, out_path, title, subtitle=subtitle)
|
||||
logger.warning(f"[BrollService] bar_comparison rendered: {out_path}, exists={os.path.exists(out_path)}")
|
||||
elif chart_type == "bar_horizontal":
|
||||
labels = chart_data.get("labels", [])
|
||||
values = chart_data.get("values", [])
|
||||
if not labels or not values:
|
||||
logger.warning("[BrollService] Missing required data for bar_horizontal")
|
||||
return ""
|
||||
make_horizontal_bar(chart_data, out_path, title)
|
||||
logger.warning(f"[BrollService] bar_horizontal rendered: {out_path}, exists={os.path.exists(out_path)}")
|
||||
elif chart_type == "line_trend":
|
||||
labels = chart_data.get("labels", [])
|
||||
values = chart_data.get("values", [])
|
||||
if not labels or not values:
|
||||
logger.warning("[BrollService] Missing required data for line_trend")
|
||||
return ""
|
||||
make_line_trend(chart_data, out_path, title)
|
||||
logger.warning(f"[BrollService] line_trend rendered: {out_path}, exists={os.path.exists(out_path)}")
|
||||
elif chart_type == "pie":
|
||||
labels = chart_data.get("labels", [])
|
||||
values = chart_data.get("values", [])
|
||||
if not labels or not values:
|
||||
logger.warning("[BrollService] Missing required data for pie")
|
||||
return ""
|
||||
make_pie_chart(chart_data, out_path, title)
|
||||
logger.warning(f"[BrollService] pie rendered: {out_path}, exists={os.path.exists(out_path)}")
|
||||
elif chart_type == "stacked_bar":
|
||||
labels = chart_data.get("labels", [])
|
||||
segments = chart_data.get("segments", [])
|
||||
if not labels or not segments:
|
||||
logger.warning("[BrollService] Missing required data for stacked_bar")
|
||||
return ""
|
||||
make_stacked_bar(chart_data, out_path, title)
|
||||
logger.warning(f"[BrollService] stacked_bar rendered: {out_path}, exists={os.path.exists(out_path)}")
|
||||
elif chart_type == "bullet" or chart_type == "bullet_points":
|
||||
# Accept both: bullet_points OR labels
|
||||
bullet_points = chart_data.get("bullet_points", [])
|
||||
# If using new format, use labels as bullet points
|
||||
if not bullet_points:
|
||||
bullet_points = chart_data.get("labels", [])
|
||||
if not bullet_points:
|
||||
labels_fallback = chart_data.get("labels", [])
|
||||
if labels_fallback:
|
||||
bullet_points = labels_fallback
|
||||
if bullet_points:
|
||||
make_bullet_overlay(bullet_points, out_path)
|
||||
logger.warning(f"[BrollService] bullet_points rendered: {out_path}, exists={os.path.exists(out_path)}")
|
||||
else:
|
||||
logger.warning("[BrollService] No bullet points provided")
|
||||
return ""
|
||||
else:
|
||||
logger.warning(f"[BrollService] Unknown chart type: {chart_type}, falling back to bar_comparison")
|
||||
# Try bar_comparison as fallback
|
||||
try:
|
||||
make_bar_chart(chart_data, out_path, title, subtitle=subtitle)
|
||||
return out_path
|
||||
except Exception as fallback_err:
|
||||
logger.warning(f"[BrollService] Fallback also failed: {fallback_err}")
|
||||
return ""
|
||||
|
||||
logger.warning(f"[BrollService] Chart preview generated: {out_path}, exists={os.path.exists(out_path) if out_path else 'N/A'}")
|
||||
|
||||
# Add source attribution overlay if present
|
||||
source = chart_data.get("source", "").strip()
|
||||
if source and out_path and os.path.exists(out_path):
|
||||
try:
|
||||
from PIL import Image as PILImage, ImageDraw, ImageFont
|
||||
img = PILImage.open(out_path).convert("RGBA")
|
||||
draw = ImageDraw.Draw(img)
|
||||
source_text = f"Source: {source[:80]}"
|
||||
try:
|
||||
font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 11)
|
||||
except (OSError, IOError):
|
||||
try:
|
||||
font = ImageFont.truetype("arial.ttf", 11)
|
||||
except (OSError, IOError):
|
||||
font = ImageFont.load_default()
|
||||
text_bbox = draw.textbbox((0, 0), source_text, font=font)
|
||||
text_w = text_bbox[2] - text_bbox[0]
|
||||
text_h = text_bbox[3] - text_bbox[1]
|
||||
x = img.width - text_w - 12
|
||||
y = img.height - text_h - 8
|
||||
draw.rectangle([x - 4, y - 2, x + text_w + 4, y + text_h + 2], fill=(0, 0, 0, 140))
|
||||
draw.text((x, y), source_text, fill=(200, 200, 200, 220), font=font)
|
||||
img.save(out_path)
|
||||
except Exception as src_err:
|
||||
logger.warning(f"[BrollService] Source overlay failed (non-fatal): {src_err}")
|
||||
|
||||
return out_path
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[BrollService] Failed to generate chart preview: {e}")
|
||||
return ""
|
||||
|
||||
def generate_scene_broll(
|
||||
self,
|
||||
scene_id: str,
|
||||
key_insight: str,
|
||||
supporting_stat: str,
|
||||
chart_data: Optional[Dict[str, Any]],
|
||||
visual_cue: str, # bar_comparison, bar_horizontal, line_trend, pie, stacked_bar, bullet_points, full_avatar
|
||||
duration: float,
|
||||
background_img_path: str,
|
||||
avatar_video_path: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Generate a B-roll video for a single scene.
|
||||
|
||||
Args:
|
||||
scene_id: Scene identifier
|
||||
key_insight: Main insight text for overlay
|
||||
supporting_stat: Supporting statistic text
|
||||
chart_data: Chart data dict (optional)
|
||||
visual_cue: Type of scene to build
|
||||
duration: Scene duration in seconds
|
||||
background_img_path: Path to background image
|
||||
avatar_video_path: Path to avatar video (optional)
|
||||
|
||||
Returns:
|
||||
Path to generated video file
|
||||
"""
|
||||
scene_id_safe = scene_id.replace(" ", "_").replace("/", "_")
|
||||
out_path = str(self.get_output_path(f"broll_{scene_id_safe}.mp4"))
|
||||
|
||||
try:
|
||||
insight = Insight(
|
||||
key_insight=key_insight,
|
||||
supporting_stat=supporting_stat,
|
||||
visual_cue=visual_cue,
|
||||
audio_tone="neutral",
|
||||
chart_data=chart_data or {},
|
||||
duration=duration,
|
||||
)
|
||||
|
||||
assets = SceneAssets(
|
||||
background_img=background_img_path,
|
||||
avatar_video=avatar_video_path,
|
||||
)
|
||||
|
||||
# Generate the scene
|
||||
scene = dispatch_scene(insight, assets)
|
||||
|
||||
# Write video
|
||||
compose_video([scene], output_path=out_path)
|
||||
|
||||
logger.info(f"[BrollService] B-roll scene generated: {out_path}")
|
||||
return out_path
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[BrollService] Failed to generate B-roll scene: {e}")
|
||||
raise
|
||||
|
||||
def compose_final_video(
|
||||
self,
|
||||
video_paths: List[str],
|
||||
output_filename: str,
|
||||
fade_dur: float = 0.5,
|
||||
fps: int = 24,
|
||||
) -> str:
|
||||
"""
|
||||
Compose multiple B-roll scene videos into final video.
|
||||
|
||||
Args:
|
||||
video_paths: List of video file paths to compose
|
||||
output_filename: Output filename
|
||||
fade_dur: Crossfade duration between scenes
|
||||
fps: Output FPS
|
||||
|
||||
Returns:
|
||||
Path to final composed video
|
||||
"""
|
||||
out_path = str(self.get_output_path(output_filename))
|
||||
|
||||
try:
|
||||
scenes = []
|
||||
for video_path in video_paths:
|
||||
from moviepy import VideoFileClip
|
||||
clip = VideoFileClip(video_path)
|
||||
scenes.append(clip)
|
||||
|
||||
if not scenes:
|
||||
raise ValueError("No video clips provided")
|
||||
|
||||
# Use crossfade_concat from broll_composer
|
||||
from services.podcast.broll_composer import crossfade_concat
|
||||
|
||||
final = crossfade_concat(scenes, fade_dur=fade_dur)
|
||||
|
||||
final.write_videofile(
|
||||
out_path,
|
||||
fps=fps,
|
||||
codec="libx264",
|
||||
audio_codec="aac",
|
||||
threads=4,
|
||||
preset="fast",
|
||||
logger=None,
|
||||
)
|
||||
|
||||
# Close clips
|
||||
for clip in scenes:
|
||||
clip.close()
|
||||
|
||||
logger.info(f"[BrollService] Final video composed: {out_path}")
|
||||
return out_path
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[BrollService] Failed to compose final video: {e}")
|
||||
raise
|
||||
|
||||
def cleanup(self, file_paths: Optional[List[str]] = None):
|
||||
"""
|
||||
Clean up temporary B-roll files.
|
||||
|
||||
Args:
|
||||
file_paths: Specific files to delete. If None, cleans output directory.
|
||||
"""
|
||||
if file_paths:
|
||||
for path in file_paths:
|
||||
try:
|
||||
if os.path.exists(path):
|
||||
os.remove(path)
|
||||
logger.debug(f"[BrollService] Removed: {path}")
|
||||
except Exception as e:
|
||||
logger.warning(f"[BrollService] Failed to remove {path}: {e}")
|
||||
else:
|
||||
# Clean entire output directory
|
||||
for file in self.output_dir.glob("*"):
|
||||
try:
|
||||
file.unlink()
|
||||
except Exception as e:
|
||||
logger.warning(f"[BrollService] Failed to remove {file}: {e}")
|
||||
|
||||
|
||||
# Per-user service instances for multi-tenant isolation
|
||||
_broll_service_instances: Dict[str, BrollService] = {}
|
||||
|
||||
|
||||
def get_broll_service(output_dir: Optional[str] = None, user_id: Optional[str] = None) -> BrollService:
|
||||
"""
|
||||
Get or create B-roll service for the given user.
|
||||
|
||||
For multi-tenant isolation, pass user_id to get user-specific directory.
|
||||
"""
|
||||
if output_dir:
|
||||
return BrollService(output_dir=output_dir)
|
||||
|
||||
# Create per-user instance based on user_id
|
||||
cache_key = user_id or "default"
|
||||
if cache_key not in _broll_service_instances:
|
||||
_broll_service_instances[cache_key] = BrollService(user_id=user_id)
|
||||
return _broll_service_instances[cache_key]
|
||||
@@ -17,26 +17,20 @@ from loguru import logger
|
||||
class PodcastVideoCombinationService:
|
||||
"""Service for combining podcast scene videos into final episodes."""
|
||||
|
||||
def __init__(self, output_dir: Optional[str] = None, user_id: Optional[str] = None):
|
||||
def __init__(self, output_dir: Optional[str] = None):
|
||||
"""
|
||||
Initialize the podcast video combination service.
|
||||
|
||||
Parameters:
|
||||
output_dir (str, optional): Directory to save combined videos.
|
||||
user_id (str, optional): User ID for workspace-scoped output.
|
||||
|
||||
Either output_dir or user_id must be provided for workspace isolation.
|
||||
Defaults to 'backend/podcast_videos/Final_Videos' if not provided.
|
||||
"""
|
||||
if output_dir:
|
||||
self.output_dir = Path(output_dir)
|
||||
elif user_id:
|
||||
from api.podcast.constants import get_podcast_media_dir
|
||||
self.output_dir = get_podcast_media_dir("video", user_id, ensure_exists=True) / "Final_Videos"
|
||||
else:
|
||||
from utils.storage_paths import get_user_workspace, sanitize_user_id
|
||||
logger.warning("[PodcastVideoCombination] No output_dir or user_id provided — using default workspace. This should not happen in production.")
|
||||
default_user = sanitize_user_id("alwrity")
|
||||
self.output_dir = get_user_workspace(default_user) / "media" / "podcast_videos" / "Final_Videos"
|
||||
# Default to root/data/media/podcast_videos/Final_Videos directory
|
||||
base_dir = Path(__file__).resolve().parents[3]
|
||||
self.output_dir = base_dir / "data" / "media" / "podcast_videos" / "Final_Videos"
|
||||
|
||||
self.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
logger.info(f"[PodcastVideoCombination] Initialized with output directory: {self.output_dir}")
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
from typing import Dict, Any, Optional
|
||||
from datetime import datetime, timedelta
|
||||
import time
|
||||
from loguru import logger
|
||||
from services.product_marketing.personalization_service import PersonalizationService
|
||||
from models.podcast_bible_models import (
|
||||
@@ -13,14 +11,9 @@ from models.podcast_bible_models import (
|
||||
ShowRules
|
||||
)
|
||||
|
||||
_BIBLE_CACHE_TTL_SECONDS = 120
|
||||
|
||||
|
||||
class PodcastBibleService:
|
||||
"""Service for generating and managing the Podcast Bible."""
|
||||
|
||||
_bible_cache: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
def __init__(self):
|
||||
try:
|
||||
from services.product_marketing.personalization_service import PersonalizationService
|
||||
@@ -29,40 +22,19 @@ class PodcastBibleService:
|
||||
logger.warning(f"Failed to initialize PersonalizationService: {e}")
|
||||
self.personalization_service = None
|
||||
|
||||
@classmethod
|
||||
def clear_user_cache(cls, user_id: str) -> int:
|
||||
"""Clear cached Bible data for a specific user. Returns number of entries cleared."""
|
||||
keys_to_remove = [key for key in cls._bible_cache if key.startswith(f"{user_id}:")]
|
||||
for key in keys_to_remove:
|
||||
del cls._bible_cache[key]
|
||||
if keys_to_remove:
|
||||
logger.info(f"[BibleCache] Cleared {len(keys_to_remove)} cache entries for user {user_id}")
|
||||
return len(keys_to_remove)
|
||||
|
||||
def generate_bible(self, user_id: str, project_id: str) -> PodcastBible:
|
||||
"""Generate a Podcast Bible from onboarding data."""
|
||||
bible_start = time.time()
|
||||
|
||||
cache_key = f"{user_id}:{project_id}"
|
||||
cached = self._bible_cache.get(cache_key)
|
||||
if cached and cached.get('expires_at') and cached['expires_at'] > datetime.utcnow():
|
||||
elapsed_ms = (time.time() - bible_start) * 1000
|
||||
logger.warning(f"[BibleCache] HIT for {user_id} — saved 7 DB queries, overhead {elapsed_ms:.0f}ms")
|
||||
return cached['bible']
|
||||
|
||||
logger.info(f"Generating Podcast Bible for user {user_id}")
|
||||
|
||||
try:
|
||||
if not self.personalization_service:
|
||||
elapsed_ms = (time.time() - bible_start) * 1000
|
||||
logger.warning(f"[BibleCache] MISS (fallback) for {user_id} — PersonalizationService unavailable, {elapsed_ms:.0f}ms")
|
||||
logger.warning("PersonalizationService not available, using default bible")
|
||||
return self._get_default_bible(project_id)
|
||||
|
||||
try:
|
||||
preferences = self.personalization_service.get_user_preferences(user_id)
|
||||
except Exception as pref_err:
|
||||
elapsed_ms = (time.time() - bible_start) * 1000
|
||||
logger.warning(f"[BibleCache] MISS (fallback) for {user_id} — get_user_preferences failed ({pref_err}), {elapsed_ms:.0f}ms")
|
||||
logger.warning(f"Failed to get user preferences: {pref_err}, using defaults")
|
||||
return self._get_default_bible(project_id)
|
||||
|
||||
if not preferences:
|
||||
@@ -159,12 +131,6 @@ class PodcastBibleService:
|
||||
)
|
||||
|
||||
logger.info(f"Podcast Bible generated successfully for project {project_id}")
|
||||
elapsed_ms = (time.time() - bible_start) * 1000
|
||||
logger.warning(f"[BibleCache] MISS — generated in {elapsed_ms:.0f}ms (7 DB queries), cached for {_BIBLE_CACHE_TTL_SECONDS}s")
|
||||
self._bible_cache[cache_key] = {
|
||||
'bible': bible,
|
||||
'expires_at': datetime.utcnow() + timedelta(seconds=_BIBLE_CACHE_TTL_SECONDS),
|
||||
}
|
||||
return bible
|
||||
|
||||
except Exception as e:
|
||||
@@ -210,12 +176,8 @@ class PodcastBibleService:
|
||||
)
|
||||
|
||||
def serialize_bible(self, bible: PodcastBible) -> str:
|
||||
"""Serialize the Bible into a prompt-friendly text block. Results are cached by project_id."""
|
||||
cache_key = f"serialized:{bible.project_id}"
|
||||
cached = self._bible_cache.get(cache_key)
|
||||
if cached and cached.get('expires_at') and cached['expires_at'] > datetime.utcnow() and isinstance(cached.get('serialized'), str):
|
||||
return cached['serialized']
|
||||
serialized = f"""
|
||||
"""Serialize the Bible into a prompt-friendly text block."""
|
||||
return f"""
|
||||
<podcast_bible>
|
||||
HOST PERSONA:
|
||||
- Name: {bible.host.name}
|
||||
@@ -250,8 +212,3 @@ SHOW RULES & STRUCTURE:
|
||||
- Constraints: {', '.join(bible.show_rules.constraints)}
|
||||
</podcast_bible>
|
||||
"""
|
||||
self._bible_cache[cache_key] = {
|
||||
'serialized': serialized,
|
||||
'expires_at': datetime.utcnow() + timedelta(seconds=_BIBLE_CACHE_TTL_SECONDS),
|
||||
}
|
||||
return serialized
|
||||
|
||||
@@ -1,281 +0,0 @@
|
||||
"""
|
||||
Podcast Context Builder Service
|
||||
|
||||
Builds unified context for AI prompts from multiple sources:
|
||||
- Podcast Bible (user personalization)
|
||||
- Website Extraction (from Exa)
|
||||
- Topic Context (category research: News/Finance)
|
||||
"""
|
||||
|
||||
from typing import Dict, Any, Optional, List
|
||||
from loguru import logger
|
||||
|
||||
|
||||
class PodcastContextBuilder:
|
||||
"""Builds unified context for AI prompt enhancements."""
|
||||
|
||||
def build_enhance_context(
|
||||
self,
|
||||
idea: str,
|
||||
bible_context: str = "",
|
||||
website_data: Optional[Dict[str, Any]] = None,
|
||||
topic_context: Optional[Dict[str, Any]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Build context for topic enhancement prompt.
|
||||
|
||||
Args:
|
||||
idea: Raw podcast idea/keywords
|
||||
bible_context: Serialized Podcast Bible string
|
||||
website_data: Website extraction data (title, summary, highlights, url, subpages)
|
||||
topic_context: Category research data (category, topics, selected_topic)
|
||||
|
||||
Returns:
|
||||
Dict with:
|
||||
- prompt: The formatted prompt
|
||||
- contexts_used: List of context types being used
|
||||
- context_description: Human-readable description for logging
|
||||
"""
|
||||
contexts_used = []
|
||||
context_parts = []
|
||||
|
||||
# Track what contexts are available
|
||||
if bible_context:
|
||||
contexts_used.append("Podcast Bible")
|
||||
|
||||
if website_data:
|
||||
contexts_used.append("Website Analysis")
|
||||
|
||||
if topic_context:
|
||||
category = topic_context.get("category", "unknown")
|
||||
contexts_used.append(f"Category Research ({category})")
|
||||
|
||||
# Build Bible section
|
||||
if bible_context:
|
||||
context_parts.append(f"USER PERSONALIZATION CONTEXT (Podcast Bible):\n{bible_context}")
|
||||
|
||||
# Build Website section
|
||||
if website_data:
|
||||
website_section = self._format_website_section(website_data)
|
||||
context_parts.append(website_section)
|
||||
|
||||
# Build Topic/Category section
|
||||
if topic_context:
|
||||
topic_section = self._format_topic_section(topic_context)
|
||||
context_parts.append(topic_section)
|
||||
|
||||
# Select appropriate prompt template based on available context
|
||||
prompt = self._select_prompt(idea, context_parts, website_data, topic_context)
|
||||
|
||||
return {
|
||||
"prompt": prompt,
|
||||
"contexts_used": contexts_used,
|
||||
"context_description": ", ".join(contexts_used) if contexts_used else "basic idea only",
|
||||
}
|
||||
|
||||
def _format_website_section(self, website_data: Dict[str, Any]) -> str:
|
||||
"""Format website data for prompt inclusion."""
|
||||
parts = []
|
||||
|
||||
if website_data.get("url"):
|
||||
parts.append(f"Source URL: {website_data['url']}")
|
||||
|
||||
if website_data.get("title"):
|
||||
parts.append(f"Company/Organization: {website_data['title']}")
|
||||
|
||||
if website_data.get("summary"):
|
||||
parts.append(f"About: {website_data['summary']}")
|
||||
|
||||
if website_data.get("highlights"):
|
||||
highlights = website_data.get("highlights", [])
|
||||
if highlights:
|
||||
parts.append(f"Key Highlights: {', '.join(highlights[:3])}")
|
||||
|
||||
if website_data.get("subpages"):
|
||||
subpages = website_data.get("subpages", [])
|
||||
if subpages:
|
||||
subpage_titles = [sp.get("title", sp.get("url", "")) for sp in subpages[:3]]
|
||||
parts.append(f"Subpages: {', '.join(subpage_titles)}")
|
||||
|
||||
return "WEBSITE CONTENT ANALYSIS:\n" + "\n".join(parts)
|
||||
|
||||
def _format_topic_section(self, topic_context: Dict[str, Any]) -> str:
|
||||
"""Format category research data for prompt inclusion."""
|
||||
parts = []
|
||||
|
||||
category = topic_context.get("category", "")
|
||||
if category:
|
||||
parts.append(f"Research Category: {category.upper()}")
|
||||
|
||||
# Include selected topic details
|
||||
selected = topic_context.get("selected_topic", {})
|
||||
if selected:
|
||||
if selected.get("title"):
|
||||
parts.append(f"Selected Topic: {selected['title']}")
|
||||
if selected.get("snippet"):
|
||||
parts.append(f"Context: {selected['snippet']}")
|
||||
if selected.get("url"):
|
||||
parts.append(f"Source: {selected['url']}")
|
||||
|
||||
# Include some alternative topics for reference
|
||||
topics = topic_context.get("topics", [])
|
||||
if topics:
|
||||
alt_titles = [t.get("title", "") for t in topics[:3] if t.get("title")]
|
||||
if alt_titles:
|
||||
parts.append(f"Related Topics: {', '.join(alt_titles)}")
|
||||
|
||||
return "CATEGORY RESEARCH CONTEXT:\n" + "\n".join(parts)
|
||||
|
||||
def _select_prompt(
|
||||
self,
|
||||
idea: str,
|
||||
context_parts: List[str],
|
||||
website_data: Optional[Dict[str, Any]],
|
||||
topic_context: Optional[Dict[str, Any]],
|
||||
) -> str:
|
||||
"""Select and format the appropriate prompt based on available context."""
|
||||
|
||||
context_str = "\n\n".join(context_parts)
|
||||
|
||||
# Full context prompt (all sources available)
|
||||
if website_data and topic_context:
|
||||
return f"""You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea, enriched with website content analysis AND category research.
|
||||
|
||||
{context_str}
|
||||
|
||||
RAW IDEA/KEYWORDS: "{idea}"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions that INCORPORATE both the website content AND category research context:
|
||||
1. Professional & Expert-led angle (leverage website authority + research insights)
|
||||
2. Storytelling & Human interest angle (brand narratives + research findings)
|
||||
3. Trendy & Contemporary angle (current trends + research relevance)
|
||||
|
||||
Each version should:
|
||||
- Be 2-3 sentences
|
||||
- Reference specific elements from both website AND research when relevant
|
||||
- Be audience-focused and align with host persona if provided
|
||||
- NOT just repeat summaries - create fresh podcast angles
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 strings (each a complete episode pitch)
|
||||
- rationales: array of 3 strings explaining each approach
|
||||
|
||||
Example format:
|
||||
{{
|
||||
"enhanced_ideas": ["Pitch 1...", "Pitch 2...", "Pitch 3..."],
|
||||
"rationales": ["Reason 1", "Reason 2", "Reason 3"]
|
||||
}}
|
||||
"""
|
||||
|
||||
# Website-only context
|
||||
elif website_data:
|
||||
return f"""You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea, enriched with website content analysis.
|
||||
|
||||
{context_str}
|
||||
|
||||
RAW IDEA/KEYWORDS: "{idea}"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions that INCORPORATE the website content:
|
||||
1. Professional & Expert-led angle (focus on authority, insights from website)
|
||||
2. Storytelling & Human interest angle (brand narratives, personal connections)
|
||||
3. Trendy & Contemporary angle (modern perspectives, current relevance)
|
||||
|
||||
Each version should:
|
||||
- Be 2-3 sentences
|
||||
- Reference specific elements from the website when relevant
|
||||
- Be audience-focused and align with host persona if provided
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 strings
|
||||
- rationales: array of 3 strings
|
||||
|
||||
Example format:
|
||||
{{
|
||||
"enhanced_ideas": ["Pitch 1...", "Pitch 2...", "Pitch 3..."],
|
||||
"rationales": ["Reason 1", "Reason 2", "Reason 3"]
|
||||
}}
|
||||
"""
|
||||
|
||||
# Category research only context
|
||||
elif topic_context:
|
||||
category = topic_context.get("category", "research").upper()
|
||||
return f"""You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea, enriched with {category} category research.
|
||||
|
||||
{context_str}
|
||||
|
||||
RAW IDEA/KEYWORDS: "{idea}"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions that INCORPORATE the {category} research:
|
||||
1. Professional & Expert-led angle (leverage research insights and data)
|
||||
2. Storytelling & Human interest angle (real-world applications, human impact)
|
||||
3. Trendy & Contemporary angle (cutting-edge trends, future outlook)
|
||||
|
||||
Each version should:
|
||||
- Be 2-3 sentences
|
||||
- Reference specific elements from the research when relevant
|
||||
- Connect the research to the raw idea meaningfully
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 strings
|
||||
- rationales: array of 3 strings
|
||||
|
||||
Example format:
|
||||
{{
|
||||
"enhanced_ideas": ["Pitch 1...", "Pitch 2...", "Pitch 3..."],
|
||||
"rationales": ["Reason 1", "Reason 2", "Reason 3"]
|
||||
}}
|
||||
"""
|
||||
|
||||
# Standard context (no additional context)
|
||||
else:
|
||||
return f"""You are a creative podcast producer. Generate 3 distinct, compelling podcast episode concepts from the raw idea.
|
||||
|
||||
{context_str}
|
||||
|
||||
RAW IDEA/KEYWORDS: "{idea}"
|
||||
|
||||
TASK:
|
||||
Generate 3 different enhanced versions with unique angles:
|
||||
1. Professional & Expert-led angle (focus on authority, insights)
|
||||
2. Storytelling & Human interest angle (focus on narratives, emotions)
|
||||
3. Trendy & Contemporary angle (focus on trends, modern relevance)
|
||||
|
||||
Each version should be 2-3 sentences, audience-focused.
|
||||
|
||||
Return JSON with:
|
||||
- enhanced_ideas: array of 3 strings
|
||||
- rationales: array of 3 strings
|
||||
|
||||
Example format:
|
||||
{{
|
||||
"enhanced_ideas": ["Pitch 1...", "Pitch 2...", "Pitch 3..."],
|
||||
"rationales": ["Reason 1", "Reason 2", "Reason 3"]
|
||||
}}
|
||||
"""
|
||||
|
||||
def format_context_for_logging(
|
||||
self,
|
||||
website_data: Optional[Dict] = None,
|
||||
topic_context: Optional[Dict] = None,
|
||||
) -> str:
|
||||
"""Format context description for logging."""
|
||||
contexts = []
|
||||
|
||||
if website_data:
|
||||
title = website_data.get("title", "Unknown")
|
||||
contexts.append(f"Website: {title[:30]}...")
|
||||
|
||||
if topic_context:
|
||||
category = topic_context.get("category", "unknown")
|
||||
selected = topic_context.get("selected_topic", {})
|
||||
topic_title = selected.get("title", "Not selected")
|
||||
contexts.append(f"Category: {category} ({topic_title[:20]}...)")
|
||||
|
||||
return " | ".join(contexts) if contexts else "No extended context"
|
||||
|
||||
|
||||
# Singleton instance for reuse
|
||||
context_builder = PodcastContextBuilder()
|
||||
@@ -4,11 +4,11 @@ Podcast Service
|
||||
Service layer for managing podcast project persistence.
|
||||
"""
|
||||
|
||||
import os
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy import desc, and_, or_
|
||||
from typing import Optional, List, Dict, Any
|
||||
from datetime import datetime
|
||||
import uuid
|
||||
|
||||
from models.podcast_models import PodcastProject
|
||||
from services.podcast_bible_service import PodcastBibleService
|
||||
@@ -32,14 +32,8 @@ class PodcastService:
|
||||
**kwargs
|
||||
) -> PodcastProject:
|
||||
"""Create a new podcast project."""
|
||||
# Generate Podcast Bible in full mode only — skip in podcast-only mode
|
||||
bible_data = None
|
||||
if os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() != "podcast":
|
||||
try:
|
||||
bible = self.bible_service.generate_bible(user_id, project_id)
|
||||
bible_data = bible.model_dump() if bible else None
|
||||
except Exception:
|
||||
pass # Bible is optional, project creation continues regardless
|
||||
# Generate Podcast Bible automatically from onboarding data
|
||||
bible = self.bible_service.generate_bible(user_id, project_id)
|
||||
|
||||
project = PodcastProject(
|
||||
project_id=project_id,
|
||||
@@ -48,7 +42,7 @@ class PodcastService:
|
||||
duration=duration,
|
||||
speakers=speakers,
|
||||
budget_cap=budget_cap,
|
||||
bible=bible_data,
|
||||
bible=bible.model_dump() if bible else None,
|
||||
status="draft",
|
||||
current_step="create",
|
||||
**kwargs
|
||||
@@ -85,26 +79,18 @@ class PodcastService:
|
||||
**updates
|
||||
) -> Optional[PodcastProject]:
|
||||
"""Update project fields."""
|
||||
from loguru import logger
|
||||
updated_fields = list(updates.keys()) if isinstance(updates, dict) else []
|
||||
logger.warning(f"[PodcastService] update_project: user_id={user_id}, project_id={project_id}, fields={updated_fields}")
|
||||
|
||||
project = self.get_project(user_id, project_id)
|
||||
if not project:
|
||||
logger.warning(f"[PodcastService] update_project: project not found")
|
||||
return None
|
||||
|
||||
# Update fields
|
||||
for key, value in updates.items():
|
||||
if hasattr(project, key):
|
||||
setattr(project, key, value)
|
||||
else:
|
||||
logger.warning(f"[PodcastService] update_project: field '{key}' not in model")
|
||||
|
||||
project.updated_at = datetime.utcnow()
|
||||
self.db.commit()
|
||||
self.db.refresh(project)
|
||||
logger.warning(f"[PodcastService] update_project: success")
|
||||
return project
|
||||
|
||||
def list_projects(
|
||||
|
||||
@@ -4,273 +4,147 @@ Google Trends Service
|
||||
Provides Google Trends data integration for the Research Engine.
|
||||
Handles rate limiting, caching, error handling, and data serialization.
|
||||
|
||||
Key design decisions:
|
||||
- Monkey-patches urllib3 Retry to fix method_whitelist→allowed_methods (urllib3 2.x)
|
||||
- Monkey-patches pytrends related_topics/related_queries to catch IndexError bug
|
||||
- Uses TrendReq built-in retries (3 retries, 1s backoff) for automatic 429 handling
|
||||
- Random user-agent rotation per instance to reduce fingerprinting
|
||||
- 1-second delays between sequential requests to respect rate limits
|
||||
- 24-hour in-memory cache to avoid redundant API calls
|
||||
|
||||
Author: ALwrity Team
|
||||
Version: 2.0
|
||||
Version: 1.0
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import random
|
||||
import time
|
||||
from typing import List, Dict, Any, Optional
|
||||
from datetime import datetime, timedelta
|
||||
from loguru import logger
|
||||
import pandas as pd
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Monkey-patches: fix compatibility issues before importing/using pytrends
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Patch 1: urllib3 2.x renamed Retry's `method_whitelist` to `allowed_methods`.
|
||||
# pytrends 4.9.2 still uses `method_whitelist`, which crashes with urllib3 2.x.
|
||||
# We patch Retry.__init__ to accept `method_whitelist` and remap it.
|
||||
try:
|
||||
from urllib3.util.retry import Retry as _OrigRetry
|
||||
|
||||
_orig_retry_init = _OrigRetry.__init__
|
||||
|
||||
def _patched_retry_init(self, *args, **kwargs):
|
||||
if 'method_whitelist' in kwargs and 'allowed_methods' not in kwargs:
|
||||
kwargs['allowed_methods'] = kwargs.pop('method_whitelist')
|
||||
_orig_retry_init(self, *args, **kwargs)
|
||||
|
||||
_OrigRetry.__init__ = _patched_retry_init
|
||||
logger.debug("[Trends] Patched urllib3 Retry.__init__ for method_whitelist→allowed_methods")
|
||||
except Exception as _patch_err:
|
||||
logger.warning(f"[Trends] Could not patch urllib3 Retry: {_patch_err}")
|
||||
|
||||
# Now safe to import pytrends
|
||||
try:
|
||||
from pytrends.request import TrendReq as _TrendReq
|
||||
from pytrends.request import TrendReq
|
||||
PYTrends_AVAILABLE = True
|
||||
except ImportError:
|
||||
PYTrends_AVAILABLE = False
|
||||
logger.warning("pytrends not installed. Google Trends features will be unavailable.")
|
||||
|
||||
# Patch 2: pytrends related_topics() and related_queries() use keyword[0]
|
||||
# which raises IndexError on empty lists, but only catch KeyError.
|
||||
# We fix this by catching (KeyError, IndexError) for the keyword extraction.
|
||||
if PYTrends_AVAILABLE:
|
||||
import json as _json
|
||||
import pandas as _pd
|
||||
|
||||
def _fixed_related_topics(self):
|
||||
result_dict = {}
|
||||
related_payload = {}
|
||||
for request_json in self.related_topics_widget_list:
|
||||
try:
|
||||
kw = request_json['request']['restriction'][
|
||||
'complexKeywordsRestriction']['keyword'][0]['value']
|
||||
except (KeyError, IndexError):
|
||||
kw = ''
|
||||
related_payload['req'] = _json.dumps(request_json['request'])
|
||||
related_payload['token'] = request_json['token']
|
||||
related_payload['tz'] = self.tz
|
||||
req_json = self._get_data(
|
||||
url=_TrendReq.RELATED_QUERIES_URL,
|
||||
method=_TrendReq.GET_METHOD,
|
||||
trim_chars=5,
|
||||
params=related_payload,
|
||||
)
|
||||
try:
|
||||
top_list = req_json['default']['rankedList'][0]['rankedKeyword']
|
||||
df_top = _pd.json_normalize(top_list, sep='_')
|
||||
except (KeyError, IndexError):
|
||||
df_top = None
|
||||
try:
|
||||
rising_list = req_json['default']['rankedList'][1]['rankedKeyword']
|
||||
df_rising = _pd.json_normalize(rising_list, sep='_')
|
||||
except (KeyError, IndexError):
|
||||
df_rising = None
|
||||
result_dict[kw] = {'rising': df_rising, 'top': df_top}
|
||||
return result_dict
|
||||
|
||||
def _fixed_related_queries(self):
|
||||
result_dict = {}
|
||||
related_payload = {}
|
||||
for request_json in self.related_queries_widget_list:
|
||||
try:
|
||||
kw = request_json['request']['restriction'][
|
||||
'complexKeywordsRestriction']['keyword'][0]['value']
|
||||
except (KeyError, IndexError):
|
||||
kw = ''
|
||||
related_payload['req'] = _json.dumps(request_json['request'])
|
||||
related_payload['token'] = request_json['token']
|
||||
related_payload['tz'] = self.tz
|
||||
req_json = self._get_data(
|
||||
url=_TrendReq.RELATED_QUERIES_URL,
|
||||
method=_TrendReq.GET_METHOD,
|
||||
trim_chars=5,
|
||||
params=related_payload,
|
||||
)
|
||||
try:
|
||||
top_df = _pd.DataFrame(
|
||||
req_json['default']['rankedList'][0]['rankedKeyword'])
|
||||
top_df = top_df[['query', 'value']]
|
||||
except (KeyError, IndexError):
|
||||
top_df = None
|
||||
try:
|
||||
rising_df = _pd.DataFrame(
|
||||
req_json['default']['rankedList'][1]['rankedKeyword'])
|
||||
rising_df = rising_df[['query', 'value']]
|
||||
except (KeyError, IndexError):
|
||||
rising_df = None
|
||||
result_dict[kw] = {'top': top_df, 'rising': rising_df}
|
||||
return result_dict
|
||||
|
||||
_TrendReq.related_topics = _fixed_related_topics
|
||||
_TrendReq.related_queries = _fixed_related_queries
|
||||
logger.debug("[Trends] Patched TrendReq.related_topics/related_queries for IndexError")
|
||||
|
||||
from .rate_limiter import RateLimiter
|
||||
|
||||
|
||||
class GoogleTrendsService:
|
||||
"""
|
||||
Service for fetching and analyzing Google Trends data.
|
||||
|
||||
Uses TrendReq with no retries (fail-fast) to avoid hitting CAPTCHA on blocks.
|
||||
429 retry handling (1s, 2s, 4s backoff). Random user-agent is set
|
||||
per instance to reduce fingerprinting.
|
||||
|
||||
Features:
|
||||
- Interest over time
|
||||
- Interest by region
|
||||
- Related topics
|
||||
- Related queries
|
||||
- Rate limiting (1 req/sec)
|
||||
- Caching (24-hour TTL)
|
||||
- Async support
|
||||
- Error handling with retry logic
|
||||
"""
|
||||
|
||||
USER_AGENTS = [
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0",
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 14_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3 Safari/605.1.15",
|
||||
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0",
|
||||
]
|
||||
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the Google Trends service."""
|
||||
if not PYTrends_AVAILABLE:
|
||||
raise RuntimeError("pytrends library is required. Install with: pip install pytrends")
|
||||
|
||||
self.rate_limiter = RateLimiter(max_calls=1, period=1.0)
|
||||
self.cache: Dict[str, Any] = {}
|
||||
self.cache_ttl = timedelta(hours=24)
|
||||
|
||||
logger.info("GoogleTrendsService initialized (pytrends 4.9.2, fail-fast, 2s delays)")
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Public API
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
|
||||
self.rate_limiter = RateLimiter(max_calls=1, period=1.0) # 1 request per second
|
||||
self.cache: Dict[str, Dict[str, Any]] = {} # Simple in-memory cache
|
||||
self.cache_ttl = timedelta(hours=24) # 24-hour cache
|
||||
|
||||
logger.info("GoogleTrendsService initialized")
|
||||
|
||||
async def analyze_trends(
|
||||
self,
|
||||
keywords: List[str],
|
||||
timeframe: str = "today 12-m",
|
||||
geo: str = "US",
|
||||
gprop: str = "",
|
||||
user_id: Optional[str] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Comprehensive trends analysis.
|
||||
|
||||
|
||||
Fetches all trends data in a single optimized call:
|
||||
- Interest over time
|
||||
- Interest by region
|
||||
- Related topics (top & rising)
|
||||
- Related queries (top & rising)
|
||||
|
||||
Args:
|
||||
keywords: List of keywords to analyze (1-5)
|
||||
timeframe: Timeframe (e.g., "today 12-m", "today 3-m", "today 5-y")
|
||||
keywords: List of keywords to analyze (1-5 keywords recommended)
|
||||
timeframe: Timeframe string (e.g., "today 12-m", "today 1-y", "all")
|
||||
geo: Country code (e.g., "US", "GB", "IN")
|
||||
gprop: Google property filter - '' for web, 'youtube' for YouTube, 'news', 'images', 'froogle'
|
||||
user_id: Optional user ID for tracking
|
||||
|
||||
Fetches: interest over time, interest by region, related topics,
|
||||
and related queries using a single TrendReq session.
|
||||
user_id: User ID for subscription checks (optional for now)
|
||||
|
||||
Returns:
|
||||
Dict containing all trends data in serializable format
|
||||
|
||||
Raises:
|
||||
ValueError: If keywords list is empty or too long
|
||||
RuntimeError: If pytrends is not available or API fails
|
||||
"""
|
||||
if not keywords:
|
||||
raise ValueError("Keywords list cannot be empty")
|
||||
|
||||
|
||||
if len(keywords) > 5:
|
||||
logger.warning(f"Too many keywords ({len(keywords)}), using first 5")
|
||||
keywords = keywords[:5]
|
||||
|
||||
|
||||
# Check cache first
|
||||
cache_key = self._build_cache_key(keywords, timeframe, geo)
|
||||
cached_data = self._get_from_cache(cache_key)
|
||||
if cached_data:
|
||||
logger.info(f"Returning cached trends data for: {keywords}")
|
||||
return {**cached_data, "cached": True}
|
||||
|
||||
|
||||
# Rate limit
|
||||
await self.rate_limiter.acquire()
|
||||
|
||||
total_start = time.monotonic()
|
||||
|
||||
interest_over_time: List[Dict[str, Any]] = []
|
||||
interest_by_region: List[Dict[str, Any]] = []
|
||||
related_topics: Dict[str, List[Dict[str, Any]]] = {"top": [], "rising": []}
|
||||
related_queries: Dict[str, List[Dict[str, Any]]] = {"top": [], "rising": []}
|
||||
|
||||
|
||||
try:
|
||||
logger.info(f"[Trends] ===== START analyze_trends ===== keywords={keywords} timeframe={timeframe} geo={geo}")
|
||||
|
||||
# Initialize TrendReq with gprop (youtube for video/podcast relevance)
|
||||
init_start = time.monotonic()
|
||||
logger.info(f"Fetching Google Trends data for: {keywords} (timeframe: {timeframe}, geo: {geo})")
|
||||
|
||||
# Initialize pytrends (sync operation, run in thread)
|
||||
pytrends = await asyncio.to_thread(
|
||||
self._create_pytrends,
|
||||
self._initialize_pytrends,
|
||||
keywords,
|
||||
timeframe,
|
||||
geo,
|
||||
gprop,
|
||||
geo
|
||||
)
|
||||
init_ms = int((time.monotonic() - init_start) * 1000)
|
||||
logger.info(f"[Trends] TrendReq init + build_payload took {init_ms}ms")
|
||||
|
||||
# --- Interest Over Time ---
|
||||
iot_start = time.monotonic()
|
||||
interest_over_time = await asyncio.to_thread(
|
||||
lambda: self._fetch_interest_over_time(pytrends)
|
||||
|
||||
# Fetch all data in parallel (pytrends methods are sync, so use to_thread)
|
||||
interest_over_time_task = asyncio.to_thread(
|
||||
lambda: self._safe_interest_over_time(pytrends)
|
||||
)
|
||||
iot_ms = int((time.monotonic() - iot_start) * 1000)
|
||||
logger.info(f"[Trends] interest_over_time took {iot_ms}ms, returned {len(interest_over_time)} points")
|
||||
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# --- Interest By Region ---
|
||||
ibr_start = time.monotonic()
|
||||
interest_by_region = await asyncio.to_thread(
|
||||
lambda: self._fetch_interest_by_region(pytrends)
|
||||
interest_by_region_task = asyncio.to_thread(
|
||||
lambda: self._safe_interest_by_region(pytrends)
|
||||
)
|
||||
ibr_ms = int((time.monotonic() - ibr_start) * 1000)
|
||||
logger.info(f"[Trends] interest_by_region took {ibr_ms}ms, returned {len(interest_by_region)} regions")
|
||||
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# --- Related Topics ---
|
||||
rt_start = time.monotonic()
|
||||
related_topics = await asyncio.to_thread(
|
||||
lambda: self._fetch_related_topics(pytrends)
|
||||
related_topics_task = asyncio.to_thread(
|
||||
lambda: self._safe_related_topics(pytrends, keywords)
|
||||
)
|
||||
rt_ms = int((time.monotonic() - rt_start) * 1000)
|
||||
rt_top = len(related_topics.get("top", []))
|
||||
rt_rising = len(related_topics.get("rising", []))
|
||||
logger.info(f"[Trends] related_topics took {rt_ms}ms, top={rt_top} rising={rt_rising}")
|
||||
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# --- Related Queries ---
|
||||
rq_start = time.monotonic()
|
||||
related_queries = await asyncio.to_thread(
|
||||
lambda: self._fetch_related_queries(pytrends)
|
||||
related_queries_task = asyncio.to_thread(
|
||||
lambda: self._safe_related_queries(pytrends, keywords)
|
||||
)
|
||||
rq_ms = int((time.monotonic() - rq_start) * 1000)
|
||||
rq_top = len(related_queries.get("top", []))
|
||||
rq_rising = len(related_queries.get("rising", []))
|
||||
logger.info(f"[Trends] related_queries took {rq_ms}ms, top={rq_top} rising={rq_rising}")
|
||||
|
||||
total_ms = int((time.monotonic() - total_start) * 1000)
|
||||
logger.info(
|
||||
f"[Trends] ===== DONE analyze_trends ===== total={total_ms}ms "
|
||||
f"iot={len(interest_over_time)} ibr={len(interest_by_region)} "
|
||||
f"rt_top={rt_top} rq_top={rq_top}"
|
||||
|
||||
# Wait for all tasks
|
||||
interest_over_time, interest_by_region, related_topics, related_queries = await asyncio.gather(
|
||||
interest_over_time_task,
|
||||
interest_by_region_task,
|
||||
related_topics_task,
|
||||
related_queries_task,
|
||||
return_exceptions=True
|
||||
)
|
||||
|
||||
|
||||
# Handle exceptions
|
||||
if isinstance(interest_over_time, Exception):
|
||||
logger.error(f"Interest over time failed: {interest_over_time}")
|
||||
interest_over_time = []
|
||||
if isinstance(interest_by_region, Exception):
|
||||
logger.error(f"Interest by region failed: {interest_by_region}")
|
||||
interest_by_region = []
|
||||
if isinstance(related_topics, Exception):
|
||||
logger.error(f"Related topics failed: {related_topics}")
|
||||
related_topics = {"top": [], "rising": []}
|
||||
if isinstance(related_queries, Exception):
|
||||
logger.error(f"Related queries failed: {related_queries}")
|
||||
related_queries = {"top": [], "rising": []}
|
||||
|
||||
# Build result
|
||||
result = {
|
||||
"interest_over_time": interest_over_time,
|
||||
"interest_by_region": interest_by_region,
|
||||
@@ -279,268 +153,186 @@ class GoogleTrendsService:
|
||||
"timeframe": timeframe,
|
||||
"geo": geo,
|
||||
"keywords": keywords,
|
||||
"source": "web" if gprop == "" else "podcast" if gprop == "youtube" else gprop,
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"cached": False,
|
||||
"cached": False
|
||||
}
|
||||
|
||||
|
||||
# Cache result
|
||||
self._save_to_cache(cache_key, result)
|
||||
|
||||
logger.info(
|
||||
f"Google Trends data fetched successfully: "
|
||||
f"{len(interest_over_time)} time points, {len(interest_by_region)} regions"
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Google Trends data fetched successfully: {len(interest_over_time)} time points, {len(interest_by_region)} regions")
|
||||
|
||||
return result
|
||||
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Google Trends analysis failed: {e}")
|
||||
return self._create_fallback_response(keywords, timeframe, geo, gprop, str(e))
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# TrendReq factory
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
def _create_pytrends(
|
||||
# Return fallback response
|
||||
return self._create_fallback_response(keywords, timeframe, geo, str(e))
|
||||
|
||||
def _initialize_pytrends(
|
||||
self,
|
||||
keywords: List[str],
|
||||
timeframe: str,
|
||||
geo: str,
|
||||
gprop: str = "",
|
||||
) -> _TrendReq:
|
||||
"""Create TrendReq with optional gprop (e.g., 'youtube' for video trends)."""
|
||||
start = time.monotonic()
|
||||
ua = random.choice(self.USER_AGENTS)
|
||||
logger.info(f"[Trends] Creating TrendReq (fail-fast, gprop='{gprop}', UA={ua[:40]}...)")
|
||||
pytrends = _TrendReq(
|
||||
hl='en-US',
|
||||
tz=360,
|
||||
timeout=(10, 30),
|
||||
retries=0,
|
||||
backoff_factor=0,
|
||||
requests_args={'headers': {'User-Agent': ua}},
|
||||
)
|
||||
# gprop: '' = web, 'youtube' = YouTube, 'news', 'images', 'froogle'
|
||||
pytrends.build_payload(kw_list=keywords, timeframe=timeframe, geo=geo, gprop=gprop)
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
logger.info(f"[Trends] TrendReq init + build_payload completed in {elapsed}ms (gprop={gprop})")
|
||||
geo: str
|
||||
) -> TrendReq:
|
||||
"""Initialize pytrends and build payload (sync operation)."""
|
||||
pytrends = TrendReq(hl='en-US', tz=360)
|
||||
pytrends.build_payload(kw_list=keywords, timeframe=timeframe, geo=geo)
|
||||
return pytrends
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Data fetchers — each catches all exceptions and returns defaults
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
def _fetch_interest_over_time(self, pytrends: _TrendReq, keywords: List[str] = None) -> List[Dict[str, Any]]:
|
||||
"""Fetch interest over time data."""
|
||||
start = time.monotonic()
|
||||
|
||||
def _safe_interest_over_time(self, pytrends: TrendReq) -> List[Dict[str, Any]]:
|
||||
"""Safely fetch interest over time data."""
|
||||
try:
|
||||
df = pytrends.interest_over_time()
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
if df is None or (hasattr(df, 'empty') and df.empty):
|
||||
logger.info(f"[Trends] interest_over_time returned empty in {elapsed}ms")
|
||||
if df.empty:
|
||||
return []
|
||||
# Use pytrends.kw_list if keywords not provided
|
||||
kw = keywords or pytrends.kw_list
|
||||
result = self._format_dataframe(df.reset_index(), kw)
|
||||
logger.info(f"[Trends] interest_over_time returned {len(result)} points in {elapsed}ms")
|
||||
return result
|
||||
return self._format_dataframe(df.reset_index())
|
||||
except Exception as e:
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
logger.error(f"[Trends] interest_over_time failed in {elapsed}ms: {e}")
|
||||
logger.error(f"Error fetching interest over time: {e}")
|
||||
return []
|
||||
|
||||
def _fetch_interest_by_region(self, pytrends: _TrendReq, keywords: List[str] = None) -> List[Dict[str, Any]]:
|
||||
"""Fetch interest by region data."""
|
||||
start = time.monotonic()
|
||||
|
||||
def _safe_interest_by_region(self, pytrends: TrendReq) -> List[Dict[str, Any]]:
|
||||
"""Safely fetch interest by region data."""
|
||||
try:
|
||||
df = pytrends.interest_by_region(resolution='COUNTRY', inc_low_vol=True, inc_geo_code=False)
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
if df is None or (hasattr(df, 'empty') and df.empty):
|
||||
logger.info(f"[Trends] interest_by_region returned empty in {elapsed}ms")
|
||||
if df.empty:
|
||||
return []
|
||||
result = self._format_dataframe(df.reset_index(), keywords or pytrends.kw_list)
|
||||
logger.info(f"[Trends] interest_by_region returned {len(result)} regions in {elapsed}ms")
|
||||
return result
|
||||
return self._format_dataframe(df.reset_index())
|
||||
except Exception as e:
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
logger.error(f"[Trends] interest_by_region failed in {elapsed}ms: {e}")
|
||||
logger.error(f"Error fetching interest by region: {e}")
|
||||
return []
|
||||
|
||||
def _fetch_related_topics(self, pytrends: _TrendReq) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""Fetch related topics. Patches catch IndexError from pytrends bug."""
|
||||
start = time.monotonic()
|
||||
result = {"top": [], "rising": []}
|
||||
|
||||
def _safe_related_topics(
|
||||
self,
|
||||
pytrends: TrendReq,
|
||||
keywords: List[str]
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""Safely fetch related topics."""
|
||||
try:
|
||||
topics_data = pytrends.related_topics()
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
|
||||
if topics_data is None:
|
||||
logger.info(f"[Trends] related_topics returned None in {elapsed}ms")
|
||||
return result
|
||||
|
||||
if not isinstance(topics_data, dict):
|
||||
logger.info(f"[Trends] related_topics returned {type(topics_data).__name__}, expected dict")
|
||||
return result
|
||||
|
||||
for key, keyword_data in topics_data.items():
|
||||
if keyword_data is None or not isinstance(keyword_data, dict):
|
||||
continue
|
||||
|
||||
for section in ["top", "rising"]:
|
||||
section_df = keyword_data.get(section)
|
||||
if section_df is None:
|
||||
continue
|
||||
if hasattr(section_df, 'empty') and section_df.empty:
|
||||
continue
|
||||
if not hasattr(section_df, 'to_dict'):
|
||||
continue
|
||||
|
||||
try:
|
||||
if "topic_title" in section_df.columns and "value" in section_df.columns:
|
||||
data = section_df[["topic_title", "value"]].to_dict('records')
|
||||
else:
|
||||
data = section_df.to_dict('records')
|
||||
result[section].extend(data)
|
||||
except Exception as e:
|
||||
logger.debug(f"Error parsing {section} topics for key '{key}': {e}")
|
||||
continue
|
||||
|
||||
logger.info(f"[Trends] related_topics completed in {elapsed}ms, top={len(result['top'])} rising={len(result['rising'])}")
|
||||
result = {"top": [], "rising": []}
|
||||
|
||||
for keyword in keywords:
|
||||
if keyword in topics_data and isinstance(topics_data[keyword], dict):
|
||||
keyword_topics = topics_data[keyword]
|
||||
|
||||
if "top" in keyword_topics and not keyword_topics["top"].empty:
|
||||
top_df = keyword_topics["top"]
|
||||
# Select relevant columns
|
||||
if "topic_title" in top_df.columns and "value" in top_df.columns:
|
||||
top_data = top_df[["topic_title", "value"]].to_dict('records')
|
||||
result["top"].extend(top_data)
|
||||
|
||||
if "rising" in keyword_topics and not keyword_topics["rising"].empty:
|
||||
rising_df = keyword_topics["rising"]
|
||||
if "topic_title" in rising_df.columns and "value" in rising_df.columns:
|
||||
rising_data = rising_df[["topic_title", "value"]].to_dict('records')
|
||||
result["rising"].extend(rising_data)
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
logger.error(f"[Trends] related_topics failed in {elapsed}ms: {e}")
|
||||
return result
|
||||
|
||||
def _fetch_related_queries(self, pytrends: _TrendReq) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""Fetch related queries. Patches catch IndexError from pytrends bug."""
|
||||
start = time.monotonic()
|
||||
result = {"top": [], "rising": []}
|
||||
logger.error(f"Error fetching related topics: {e}")
|
||||
return {"top": [], "rising": []}
|
||||
|
||||
def _safe_related_queries(
|
||||
self,
|
||||
pytrends: TrendReq,
|
||||
keywords: List[str]
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""Safely fetch related queries."""
|
||||
try:
|
||||
queries_data = pytrends.related_queries()
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
|
||||
if queries_data is None:
|
||||
logger.info(f"[Trends] related_queries returned None in {elapsed}ms")
|
||||
return result
|
||||
|
||||
if not isinstance(queries_data, dict):
|
||||
logger.info(f"[Trends] related_queries returned {type(queries_data).__name__}, expected dict")
|
||||
return result
|
||||
|
||||
for key, keyword_data in queries_data.items():
|
||||
if keyword_data is None or not isinstance(keyword_data, dict):
|
||||
continue
|
||||
|
||||
for section in ["top", "rising"]:
|
||||
section_df = keyword_data.get(section)
|
||||
if section_df is None:
|
||||
continue
|
||||
if hasattr(section_df, 'empty') and section_df.empty:
|
||||
continue
|
||||
if not hasattr(section_df, 'to_dict'):
|
||||
continue
|
||||
|
||||
try:
|
||||
data = section_df.to_dict('records')
|
||||
result[section].extend(data)
|
||||
except Exception as e:
|
||||
logger.debug(f"Error parsing {section} queries for key '{key}': {e}")
|
||||
continue
|
||||
|
||||
logger.info(f"[Trends] related_queries completed in {elapsed}ms, top={len(result['top'])} rising={len(result['rising'])}")
|
||||
result = {"top": [], "rising": []}
|
||||
|
||||
for keyword in keywords:
|
||||
if keyword in queries_data and isinstance(queries_data[keyword], dict):
|
||||
keyword_queries = queries_data[keyword]
|
||||
|
||||
if "top" in keyword_queries and not keyword_queries["top"].empty:
|
||||
top_df = keyword_queries["top"]
|
||||
result["top"].extend(top_df.to_dict('records'))
|
||||
|
||||
if "rising" in keyword_queries and not keyword_queries["rising"].empty:
|
||||
rising_df = keyword_queries["rising"]
|
||||
result["rising"].extend(rising_df.to_dict('records'))
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
elapsed = int((time.monotonic() - start) * 1000)
|
||||
logger.error(f"[Trends] related_queries failed in {elapsed}ms: {e}")
|
||||
return result
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Helpers
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
def _format_dataframe(self, df: pd.DataFrame, keywords: List[str] = None) -> List[Dict[str, Any]]:
|
||||
"""Convert DataFrame to list of dicts. Handles both pytrends and SerpAPI formats."""
|
||||
logger.error(f"Error fetching related queries: {e}")
|
||||
return {"top": [], "rising": []}
|
||||
|
||||
def _format_dataframe(self, df: pd.DataFrame) -> List[Dict[str, Any]]:
|
||||
"""Convert DataFrame to list of dicts (serializable format)."""
|
||||
if df.empty:
|
||||
return []
|
||||
|
||||
# Try to detect and handle SerpAPI-style nested data
|
||||
# Check if the dataframe has 'date' column and 'values' array column
|
||||
records = df.to_dict('records')
|
||||
|
||||
# Check first record for nested values pattern (SerpAPI format)
|
||||
if records and 'values' in records[0] and isinstance(records[0]['values'], list):
|
||||
# SerpAPI-style: need to flatten
|
||||
flat_records = []
|
||||
for record in records:
|
||||
date_str = record.get('date', '')
|
||||
timestamp = record.get('timestamp', '')
|
||||
is_partial = record.get('partial_data', False)
|
||||
|
||||
# Extract values from nested array
|
||||
for val_entry in record['values']:
|
||||
keyword_name = val_entry.get('query', '')
|
||||
value = val_entry.get('value', val_entry.get('extracted_value', 0))
|
||||
flat_record = {
|
||||
'date': date_str,
|
||||
'timestamp': timestamp,
|
||||
keyword_name: int(value) if value else 0,
|
||||
}
|
||||
if is_partial:
|
||||
flat_record['isPartial'] = True
|
||||
flat_records.append(flat_record)
|
||||
records = flat_records
|
||||
|
||||
# Convert datetime columns to strings
|
||||
for record in records:
|
||||
for key, value in record.items():
|
||||
if hasattr(value, 'year'): # datetime-like
|
||||
record[key] = str(value)
|
||||
for col in df.columns:
|
||||
if pd.api.types.is_datetime64_any_dtype(df[col]):
|
||||
df[col] = df[col].astype(str)
|
||||
|
||||
return records
|
||||
|
||||
# Convert to dict records
|
||||
return df.to_dict('records')
|
||||
|
||||
def _build_cache_key(self, keywords: List[str], timeframe: str, geo: str) -> str:
|
||||
"""Build cache key from parameters."""
|
||||
keywords_str = ":".join(sorted(keywords))
|
||||
return f"google_trends:{keywords_str}:{timeframe}:{geo}"
|
||||
|
||||
|
||||
def _get_from_cache(self, cache_key: str) -> Optional[Dict[str, Any]]:
|
||||
"""Get data from cache if not expired."""
|
||||
if cache_key not in self.cache:
|
||||
return None
|
||||
|
||||
cached_entry = self.cache[cache_key]
|
||||
cached_time = datetime.fromisoformat(cached_entry.get("timestamp", ""))
|
||||
|
||||
if datetime.utcnow() - cached_time > self.cache_ttl:
|
||||
# Expired, remove from cache
|
||||
del self.cache[cache_key]
|
||||
return None
|
||||
|
||||
# Return cached data (without cached flag)
|
||||
result = {**cached_entry}
|
||||
result.pop("cached", None)
|
||||
return result
|
||||
|
||||
|
||||
def _save_to_cache(self, cache_key: str, data: Dict[str, Any]):
|
||||
cache_entry = {**data, "cached_at": datetime.utcnow().isoformat()}
|
||||
"""Save data to cache."""
|
||||
# Store with timestamp
|
||||
cache_entry = {
|
||||
**data,
|
||||
"cached_at": datetime.utcnow().isoformat()
|
||||
}
|
||||
self.cache[cache_key] = cache_entry
|
||||
if len(self.cache) > 100:
|
||||
|
||||
# Clean up old cache entries periodically
|
||||
if len(self.cache) > 100: # Limit cache size
|
||||
self._cleanup_cache()
|
||||
|
||||
|
||||
def _cleanup_cache(self):
|
||||
"""Remove expired cache entries."""
|
||||
now = datetime.utcnow()
|
||||
expired_keys = []
|
||||
|
||||
for key, entry in self.cache.items():
|
||||
cached_time = datetime.fromisoformat(entry.get("cached_at", entry.get("timestamp", "")))
|
||||
if now - cached_time > self.cache_ttl:
|
||||
expired_keys.append(key)
|
||||
|
||||
for key in expired_keys:
|
||||
del self.cache[key]
|
||||
|
||||
logger.debug(f"Cleaned up {len(expired_keys)} expired cache entries")
|
||||
|
||||
|
||||
def _create_fallback_response(
|
||||
self,
|
||||
keywords: List[str],
|
||||
timeframe: str,
|
||||
geo: str,
|
||||
gprop: str = "",
|
||||
error_message: str = "",
|
||||
error_message: str
|
||||
) -> Dict[str, Any]:
|
||||
source = "web" if gprop == "" else "podcast" if gprop == "youtube" else gprop
|
||||
"""Create fallback response when trends analysis fails."""
|
||||
return {
|
||||
"interest_over_time": [],
|
||||
"interest_by_region": [],
|
||||
@@ -549,38 +341,40 @@ class GoogleTrendsService:
|
||||
"timeframe": timeframe,
|
||||
"geo": geo,
|
||||
"keywords": keywords,
|
||||
"source": source,
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"cached": False,
|
||||
"error": error_message,
|
||||
"error": error_message
|
||||
}
|
||||
|
||||
|
||||
async def get_trending_searches(
|
||||
self,
|
||||
country: str = "united_states",
|
||||
user_id: Optional[str] = None,
|
||||
user_id: Optional[str] = None
|
||||
) -> List[str]:
|
||||
"""
|
||||
Get current trending searches for a country.
|
||||
|
||||
Args:
|
||||
country: Country name (e.g., "united_states", "united_kingdom")
|
||||
user_id: User ID for subscription checks
|
||||
|
||||
Returns:
|
||||
List of trending search terms
|
||||
"""
|
||||
await self.rate_limiter.acquire()
|
||||
|
||||
|
||||
try:
|
||||
ua = random.choice(self.USER_AGENTS)
|
||||
pytrends = _TrendReq(
|
||||
hl='en-US',
|
||||
tz=360,
|
||||
timeout=(10, 30),
|
||||
retries=0,
|
||||
backoff_factor=0,
|
||||
requests_args={'headers': {'User-Agent': ua}},
|
||||
)
|
||||
pytrends = TrendReq(hl='en-US', tz=360)
|
||||
trending_df = await asyncio.to_thread(
|
||||
lambda: pytrends.trending_searches(pn=country)
|
||||
)
|
||||
|
||||
if trending_df is None or (hasattr(trending_df, 'empty') and trending_df.empty):
|
||||
|
||||
if trending_df.empty:
|
||||
return []
|
||||
|
||||
|
||||
# Return as list of strings
|
||||
return trending_df[0].tolist() if len(trending_df.columns) > 0 else []
|
||||
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching trending searches: {e}")
|
||||
return []
|
||||
return []
|
||||
|
||||
@@ -31,8 +31,8 @@ def log_video_stack_diagnostics() -> None:
|
||||
def assert_supported_moviepy() -> None:
|
||||
"""Fail fast if MoviePy isn't version 2.x."""
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
mv = version("moviepy")
|
||||
import pkg_resources as pr
|
||||
mv = pr.get_distribution("moviepy").version
|
||||
if not mv.startswith("2."):
|
||||
raise RuntimeError(
|
||||
f"Unsupported MoviePy version {mv}. Expected 2.x. "
|
||||
|
||||
@@ -4,7 +4,6 @@ Handles subscription limit checking and validation logic.
|
||||
Extracted from pricing_service.py for better modularity.
|
||||
"""
|
||||
|
||||
import time
|
||||
from typing import Dict, Any, Optional, List, Tuple, TYPE_CHECKING
|
||||
from datetime import datetime, timedelta
|
||||
from sqlalchemy import text
|
||||
@@ -19,15 +18,6 @@ if TYPE_CHECKING:
|
||||
from .pricing_service import PricingService
|
||||
|
||||
|
||||
def _should_enforce_limit(limit_value: int, tier: str) -> bool:
|
||||
"""
|
||||
Determine if a limit should be enforced.
|
||||
- Free tier: 0 means DISABLED (not unlimited)
|
||||
- Basic/Pro/Enterprise: 0 means UNLIMITED
|
||||
"""
|
||||
return limit_value > 0
|
||||
|
||||
|
||||
class LimitValidator:
|
||||
"""Validates subscription limits for API usage."""
|
||||
|
||||
@@ -42,11 +32,9 @@ class LimitValidator:
|
||||
self.db = pricing_service.db
|
||||
|
||||
def check_usage_limits(self, user_id: str, provider: APIProvider,
|
||||
tokens_requested: int = 0, actual_provider_name: Optional[str] = None) -> Tuple[bool, str, Dict[str, Any]]:
|
||||
tokens_requested: int = 0, actual_provider_name: Optional[str] = None) -> Tuple[bool, str, Dict[str, Any]]:
|
||||
"""Check if user can make an API call within their limits.
|
||||
|
||||
Delegates to LimitValidator for actual validation logic.
|
||||
|
||||
Args:
|
||||
user_id: User ID
|
||||
provider: APIProvider enum (may be MISTRAL for HuggingFace)
|
||||
@@ -56,7 +44,6 @@ class LimitValidator:
|
||||
Returns:
|
||||
(can_proceed, error_message, usage_info)
|
||||
"""
|
||||
start_time = time.time()
|
||||
try:
|
||||
# Use actual_provider_name if provided, otherwise use enum value
|
||||
# This fixes cases where HuggingFace maps to MISTRAL enum but should show as "huggingface" in errors
|
||||
@@ -64,14 +51,12 @@ class LimitValidator:
|
||||
|
||||
logger.debug(f"[Subscription Check] Starting limit check for user {user_id}, provider {display_provider_name}, tokens {tokens_requested}")
|
||||
|
||||
logger.warning(f"[Subscription Check] START for user {user_id}, provider {provider.value}")
|
||||
# Short TTL cache to reduce DB reads under sustained traffic
|
||||
cache_key = f"{user_id}:{provider.value}"
|
||||
now = datetime.utcnow()
|
||||
cached = self.pricing_service._limits_cache.get(cache_key)
|
||||
if cached and cached.get('expires_at') and cached['expires_at'] > now:
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
logger.warning(f"[Subscription Check] Cache hit for {user_id}:{provider.value} — completed in {elapsed_ms:.0f}ms")
|
||||
logger.debug(f"[Subscription Check] Using cached result for {user_id}:{provider.value}")
|
||||
return tuple(cached['result']) # type: ignore
|
||||
|
||||
# Get user subscription first to check expiration
|
||||
@@ -153,19 +138,13 @@ class LimitValidator:
|
||||
logger.warning(f"[Subscription Check] No subscription or free tier found for user {user_id}, denying access")
|
||||
return False, "No subscription plan found. Please subscribe to a plan.", {}
|
||||
|
||||
# Extract tier for limit enforcement logic
|
||||
user_tier = limits.get('tier', 'free') if limits else 'free'
|
||||
|
||||
# Get current usage for this billing period with error handling
|
||||
# Use targeted expiry instead of expire_all() to avoid nuking the entire session cache
|
||||
# CRITICAL: Use fresh queries to avoid SQLAlchemy cache after renewal
|
||||
try:
|
||||
current_period = self.pricing_service.get_current_billing_period(user_id) or datetime.now().strftime("%Y-%m")
|
||||
|
||||
# Only expire specific objects that might have changed after renewal
|
||||
# (subscription was already checked above; plan was expired above)
|
||||
# The usage record is the main object we need fresh, and we query it directly below
|
||||
if subscription:
|
||||
self.db.expire(subscription)
|
||||
# Expire all objects to force fresh read from DB (critical after renewal)
|
||||
self.db.expire_all()
|
||||
|
||||
# Use raw SQL query first to bypass ORM cache, fallback to ORM if SQL fails
|
||||
usage = None
|
||||
@@ -257,8 +236,8 @@ class LimitValidator:
|
||||
(usage.mistral_calls or 0)
|
||||
)
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(ai_text_gen_limit, user_tier) and current_total_llm_calls >= ai_text_gen_limit:
|
||||
# Only enforce limit if limit > 0 (0 means unlimited for Enterprise)
|
||||
if ai_text_gen_limit > 0 and current_total_llm_calls >= ai_text_gen_limit:
|
||||
logger.error(f"[Subscription Check] AI text generation call limit exceeded for user {user_id}: {current_total_llm_calls}/{ai_text_gen_limit} (provider: {display_provider_name})")
|
||||
result = (False, f"AI text generation call limit reached. Used {current_total_llm_calls} of {ai_text_gen_limit} total AI text generation calls this billing period.", {
|
||||
'current_calls': current_total_llm_calls,
|
||||
@@ -290,8 +269,8 @@ class LimitValidator:
|
||||
current_calls = getattr(usage, f"{provider_name}_calls", 0) or 0
|
||||
call_limit = limits['limits'].get(f"{provider_name}_calls", 0) or 0
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(call_limit, user_tier) and current_calls >= call_limit:
|
||||
# Only enforce limit if limit > 0 (0 means unlimited for Enterprise)
|
||||
if call_limit > 0 and current_calls >= call_limit:
|
||||
logger.error(f"[Subscription Check] Call limit exceeded for user {user_id}, provider {display_provider_name}: {current_calls}/{call_limit}")
|
||||
result = (False, f"API call limit reached for {display_provider_name}. Used {current_calls} of {call_limit} calls this billing period.", {
|
||||
'current_calls': current_calls,
|
||||
@@ -308,13 +287,7 @@ class LimitValidator:
|
||||
logger.debug(f"[Subscription Check] Call limit check passed for user {user_id}, provider {display_provider_name}: {current_calls}/{call_limit if call_limit > 0 else 'unlimited'}")
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking call limits: {e}")
|
||||
# Fail closed - deny if we can't verify the limit
|
||||
result = (False, f"Unable to verify call limit: {str(e)}", {})
|
||||
self.pricing_service._limits_cache[cache_key] = {
|
||||
'result': result,
|
||||
'expires_at': now + timedelta(seconds=30)
|
||||
}
|
||||
return result
|
||||
# Continue to next check
|
||||
|
||||
# Check token limits for LLM providers with error handling
|
||||
# NOTE: token_limit = 0 means UNLIMITED (Enterprise plans)
|
||||
@@ -323,8 +296,8 @@ class LimitValidator:
|
||||
current_tokens = getattr(usage, f"{provider_name}_tokens", 0) or 0
|
||||
token_limit = limits['limits'].get(f"{provider_name}_tokens", 0) or 0
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(token_limit, user_tier) and (current_tokens + tokens_requested) > token_limit:
|
||||
# Only enforce limit if limit > 0 (0 means unlimited for Enterprise)
|
||||
if token_limit > 0 and (current_tokens + tokens_requested) > token_limit:
|
||||
result = (False, f"Token limit would be exceeded for {display_provider_name}. Current: {current_tokens}, Requested: {tokens_requested}, Limit: {token_limit}", {
|
||||
'current_tokens': current_tokens,
|
||||
'requested_tokens': tokens_requested,
|
||||
@@ -346,19 +319,14 @@ class LimitValidator:
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking token limits: {e}")
|
||||
# Fail closed - deny if we can't verify the limit
|
||||
result = (False, f"Unable to verify token limit: {str(e)}", {})
|
||||
self.pricing_service._limits_cache[cache_key] = {
|
||||
'result': result,
|
||||
'expires_at': now + timedelta(seconds=30)
|
||||
}
|
||||
return result
|
||||
# Continue to next check
|
||||
|
||||
# Check cost limits with error handling
|
||||
# NOTE: cost_limit = 0 means UNLIMITED (Enterprise plans)
|
||||
try:
|
||||
cost_limit = limits['limits'].get('monthly_cost', 0) or 0
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(cost_limit, user_tier) and usage.total_cost >= cost_limit:
|
||||
# Only enforce limit if limit > 0 (0 means unlimited for Enterprise)
|
||||
if cost_limit > 0 and usage.total_cost >= cost_limit:
|
||||
result = (False, f"Monthly cost limit reached. Current cost: ${usage.total_cost:.2f}, Limit: ${cost_limit:.2f}", {
|
||||
'current_cost': usage.total_cost,
|
||||
'limit': cost_limit,
|
||||
@@ -371,13 +339,7 @@ class LimitValidator:
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking cost limits: {e}")
|
||||
# Fail closed - deny if we can't verify the limit
|
||||
result = (False, f"Unable to verify cost limit: {str(e)}", {})
|
||||
self.pricing_service._limits_cache[cache_key] = {
|
||||
'result': result,
|
||||
'expires_at': now + timedelta(seconds=30)
|
||||
}
|
||||
return result
|
||||
# Continue to success case
|
||||
|
||||
# Calculate usage percentages for warnings
|
||||
try:
|
||||
@@ -405,18 +367,14 @@ class LimitValidator:
|
||||
'result': result,
|
||||
'expires_at': now + timedelta(seconds=30)
|
||||
}
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
logger.warning(f"[Subscription Check] Completed in {elapsed_ms:.0f}ms for user {user_id}, provider {display_provider_name} — within limits (calls: {current_call_count}/{call_limit_value})")
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.error(f"Error calculating usage percentages: {e}")
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
logger.warning(f"[Subscription Check] Completed in {elapsed_ms:.0f}ms for user {user_id}, provider {display_provider_name} — within limits (basic check)")
|
||||
# Return basic success
|
||||
return True, "Within limits", {}
|
||||
|
||||
except Exception as e:
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
logger.error(f"[Subscription Check] Failed for user {user_id} after {elapsed_ms:.0f}ms: {e}")
|
||||
logger.error(f"Unexpected error in check_usage_limits for {user_id}: {e}")
|
||||
# STRICT: Fail closed - deny requests if subscription system fails
|
||||
return False, f"Subscription check error: {str(e)}", {}
|
||||
|
||||
@@ -459,7 +417,9 @@ class LimitValidator:
|
||||
except Exception as schema_err:
|
||||
logger.warning(f"Schema check failed, will retry on query error: {schema_err}")
|
||||
|
||||
# Explicitly refresh usage from DB to ensure fresh data (targeted instead of expire_all)
|
||||
# Explicitly expire any cached objects and refresh from DB to ensure fresh data
|
||||
self.db.expire_all()
|
||||
|
||||
try:
|
||||
usage = self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id,
|
||||
@@ -478,12 +438,7 @@ class LimitValidator:
|
||||
schema_utils._checked_usage_summaries_columns = False
|
||||
from services.subscription.schema_utils import ensure_usage_summaries_columns
|
||||
ensure_usage_summaries_columns(self.db)
|
||||
# After schema migration, only expire UsageSummary to force re-query
|
||||
# (no need to expire the entire session)
|
||||
for obj in self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id
|
||||
).all():
|
||||
self.db.expire(obj)
|
||||
self.db.expire_all()
|
||||
# Retry the query
|
||||
usage = self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id,
|
||||
@@ -532,7 +487,6 @@ class LimitValidator:
|
||||
return False, "No subscription plan found. Please subscribe to a plan.", {}
|
||||
|
||||
limits = limits_dict.get('limits', {})
|
||||
tier = limits_dict.get('tier', 'free')
|
||||
|
||||
# Track cumulative usage across all operations
|
||||
total_llm_calls = (
|
||||
@@ -577,8 +531,7 @@ class LimitValidator:
|
||||
# Count this operation as an LLM call
|
||||
projected_total_llm_calls = total_llm_calls + 1
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(ai_text_gen_limit, tier) and projected_total_llm_calls > ai_text_gen_limit:
|
||||
if ai_text_gen_limit > 0 and projected_total_llm_calls > ai_text_gen_limit:
|
||||
error_info = {
|
||||
'current_calls': total_llm_calls,
|
||||
'limit': ai_text_gen_limit,
|
||||
@@ -641,9 +594,8 @@ class LimitValidator:
|
||||
# Method 2: Fallback to fresh ORM query if raw SQL fails
|
||||
if not query_succeeded:
|
||||
try:
|
||||
# Only refresh usage object, don't expire entire session
|
||||
if usage:
|
||||
self.db.refresh(usage)
|
||||
# Expire all cached objects and do fresh query
|
||||
self.db.expire_all()
|
||||
fresh_usage = self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id,
|
||||
UsageSummary.billing_period == current_period
|
||||
@@ -685,8 +637,7 @@ class LimitValidator:
|
||||
|
||||
token_limit = limits.get(provider_tokens_key, 0) or 0
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(token_limit, tier) and tokens_requested > 0:
|
||||
if token_limit > 0 and tokens_requested > 0:
|
||||
projected_tokens = current_provider_tokens + tokens_requested
|
||||
logger.info(f" └─ Token Check: {current_provider_tokens} (current) + {tokens_requested} (requested) = {projected_tokens} (total) / {token_limit} (limit)")
|
||||
|
||||
@@ -748,8 +699,7 @@ class LimitValidator:
|
||||
image_limit = limits.get('stability_calls', 0) or 0
|
||||
projected_images = total_images + 1
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(image_limit, tier) and projected_images > image_limit:
|
||||
if image_limit > 0 and projected_images > image_limit:
|
||||
error_info = {
|
||||
'current_images': total_images,
|
||||
'limit': image_limit,
|
||||
@@ -770,8 +720,7 @@ class LimitValidator:
|
||||
total_video_calls = usage.video_calls or 0
|
||||
projected_video_calls = total_video_calls + 1
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(video_limit, tier) and projected_video_calls > video_limit:
|
||||
if video_limit > 0 and projected_video_calls > video_limit:
|
||||
error_info = {
|
||||
'current_calls': total_video_calls,
|
||||
'limit': video_limit,
|
||||
@@ -790,8 +739,7 @@ class LimitValidator:
|
||||
total_image_edit_calls = getattr(usage, 'image_edit_calls', 0) or 0
|
||||
projected_image_edit_calls = total_image_edit_calls + 1
|
||||
|
||||
# Enforce limit based on tier (Free: 0=disabled, others: 0=unlimited)
|
||||
if _should_enforce_limit(image_edit_limit, tier) and projected_image_edit_calls > image_edit_limit:
|
||||
if image_edit_limit > 0 and projected_image_edit_calls > image_edit_limit:
|
||||
error_info = {
|
||||
'current_calls': total_image_edit_calls,
|
||||
'limit': image_edit_limit,
|
||||
@@ -824,25 +772,6 @@ class LimitValidator:
|
||||
'error_type': 'call_limit',
|
||||
'usage_info': error_info
|
||||
}
|
||||
|
||||
# Check WaveSpeed combined limit if actual_provider is WaveSpeed
|
||||
if actual_provider_name == 'wavespeed':
|
||||
wavespeed_limit = limits.get('wavespeed_calls', 0) or 0
|
||||
if _should_enforce_limit(wavespeed_limit, tier):
|
||||
wavespeed_usage = usage.wavespeed_calls or 0
|
||||
projected_wavespeed = wavespeed_usage + 1
|
||||
if projected_wavespeed > wavespeed_limit:
|
||||
error_info = {
|
||||
'current_calls': wavespeed_usage,
|
||||
'limit': wavespeed_limit,
|
||||
'provider': 'wavespeed',
|
||||
'operation_type': operation_type,
|
||||
'operation_index': op_idx
|
||||
}
|
||||
return False, f"WaveSpeed API limit would be exceeded. Would use {projected_wavespeed} of {wavespeed_limit} WaveSpeed calls this billing period.", {
|
||||
'error_type': 'wavespeed_limit',
|
||||
'usage_info': error_info
|
||||
}
|
||||
|
||||
# All checks passed
|
||||
logger.info(f"[Pre-flight Check] ✅ All {len(operations)} operation(s) validated successfully")
|
||||
@@ -863,11 +792,7 @@ class LimitValidator:
|
||||
schema_utils._checked_usage_summaries_columns = False
|
||||
from services.subscription.schema_utils import ensure_usage_summaries_columns
|
||||
ensure_usage_summaries_columns(self.db)
|
||||
# Only expire UsageSummary after schema migration, not entire session
|
||||
for obj in self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id
|
||||
).all():
|
||||
self.db.expire(obj)
|
||||
self.db.expire_all()
|
||||
|
||||
# Retry the query
|
||||
usage = self.db.query(UsageSummary).filter(
|
||||
|
||||
@@ -494,16 +494,7 @@ class PricingService:
|
||||
logger.debug(f"Added new pricing for {pricing_data['provider'].value}:{pricing_data['model_name']}")
|
||||
|
||||
self.db.commit()
|
||||
|
||||
# Debug: count pricing rows seeded
|
||||
total_rows = self.db.query(APIProviderPricing).count()
|
||||
providers = self.db.query(APIProviderPricing.provider).distinct().all()
|
||||
provider_list = sorted([p[0].value for p in providers]) if providers else []
|
||||
logger.info(f"[PRICING_INIT] Default API pricing initialized: {len(all_pricing)} rows configured, {total_rows} rows in DB, providers: {provider_list}")
|
||||
|
||||
# Warning-level log that will be visible
|
||||
logger.warning(f"[PRICING_INIT] Pricing ready: {total_rows} rows for {len(provider_list)} providers")
|
||||
logger.warning("Default API pricing initialized/updated. HuggingFace pricing loaded from env vars if available.")
|
||||
logger.info("Default API pricing initialized/updated. HuggingFace pricing loaded from env vars if available.")
|
||||
|
||||
def initialize_default_plans(self):
|
||||
"""Initialize default subscription plans."""
|
||||
@@ -514,26 +505,21 @@ class PricingService:
|
||||
"tier": SubscriptionTier.FREE,
|
||||
"price_monthly": 0.0,
|
||||
"price_yearly": 0.0,
|
||||
"ai_text_generation_calls_limit": 50, # Explicit: Free gets 50 AI text calls (via Gemini fallback)
|
||||
"gemini_calls_limit": 50,
|
||||
"openai_calls_limit": 0, # DISABLED: OpenAI access not included in Free tier
|
||||
"anthropic_calls_limit": 0, # DISABLED: Anthropic access not included in Free tier
|
||||
"mistral_calls_limit": 0, # DISABLED: HuggingFace not in Free tier
|
||||
"tavily_calls_limit": 10,
|
||||
"serper_calls_limit": 10,
|
||||
"metaphor_calls_limit": 0, # DISABLED: Metaphor not in Free tier
|
||||
"firecrawl_calls_limit": 0, # DISABLED: Firecrawl not in Free tier
|
||||
"stability_calls_limit": 3, # 3 images - enough to try the product
|
||||
"exa_calls_limit": 10, # 10 research queries - enough to try the product
|
||||
"video_calls_limit": 0, # DISABLED: Video generation not in Free tier
|
||||
"image_edit_calls_limit": 5, # 5 image edits - enough to try the product
|
||||
"audio_calls_limit": 5, # 5 audio clips - enough to try the product
|
||||
"wavespeed_calls_limit": 0, # DISABLED: WaveSpeed not included in Free tier
|
||||
"gemini_tokens_limit": 50000,
|
||||
"openai_tokens_limit": 0, # DISABLED
|
||||
"anthropic_tokens_limit": 0, # DISABLED
|
||||
"mistral_tokens_limit": 0, # DISABLED
|
||||
"monthly_cost_limit": 2.0, # $2 cap - prevents runaway costs on free tier
|
||||
"gemini_calls_limit": 100,
|
||||
"openai_calls_limit": 0,
|
||||
"anthropic_calls_limit": 0,
|
||||
"mistral_calls_limit": 50,
|
||||
"tavily_calls_limit": 20,
|
||||
"serper_calls_limit": 20,
|
||||
"metaphor_calls_limit": 10,
|
||||
"firecrawl_calls_limit": 10,
|
||||
"stability_calls_limit": 5,
|
||||
"exa_calls_limit": 100,
|
||||
"video_calls_limit": 0, # No video generation for free tier
|
||||
"image_edit_calls_limit": 10, # 10 AI image editing calls/month
|
||||
"audio_calls_limit": 20, # 20 AI audio generation calls/month
|
||||
"gemini_tokens_limit": 100000,
|
||||
"monthly_cost_limit": 0.0,
|
||||
"features": ["basic_content_generation", "limited_research"],
|
||||
"description": "Perfect for trying out ALwrity"
|
||||
},
|
||||
@@ -542,7 +528,7 @@ class PricingService:
|
||||
"tier": SubscriptionTier.BASIC,
|
||||
"price_monthly": 29.0,
|
||||
"price_yearly": 290.0,
|
||||
"ai_text_generation_calls_limit": 500, # Unified limit for all LLM providers
|
||||
"ai_text_generation_calls_limit": 50, # INCREASED: Unified limit for all LLM providers (OSS-focused strategy)
|
||||
"gemini_calls_limit": 1000, # Legacy, kept for backwards compatibility (not used for enforcement)
|
||||
"openai_calls_limit": 500,
|
||||
"anthropic_calls_limit": 200,
|
||||
@@ -551,17 +537,16 @@ class PricingService:
|
||||
"serper_calls_limit": 200,
|
||||
"metaphor_calls_limit": 100,
|
||||
"firecrawl_calls_limit": 100,
|
||||
"stability_calls_limit": 25, # 25 images - good for podcast episode covers
|
||||
"exa_calls_limit": 100, # 100 research queries
|
||||
"video_calls_limit": 10, # 10 videos - enough for a few podcast episodes
|
||||
"image_edit_calls_limit": 25, # 25 AI image edits
|
||||
"stability_calls_limit": 50, # INCREASED: Now includes WaveSpeed OSS models (Qwen Image $0.03)
|
||||
"exa_calls_limit": 500,
|
||||
"video_calls_limit": 30, # INCREASED: 30 videos/month (WAN 2.5 OSS $0.25)
|
||||
"image_edit_calls_limit": 50, # INCREASED: 50 AI image editing calls/month (Qwen Edit OSS $0.02)
|
||||
"audio_calls_limit": 100, # INCREASED: 100 AI audio generation calls/month (Minimax Speech OSS)
|
||||
"wavespeed_calls_limit": 200, # WaveSpeed combined limit: TTS + video + image + LLM (Minimax Speech $0.002/min, Qwen $0.03/img, Kling $0.25/5s)
|
||||
"gemini_tokens_limit": 100000, # INCREASED: 100K tokens per provider (OSS-focused strategy)
|
||||
"openai_tokens_limit": 100000, # INCREASED: 100K tokens per provider
|
||||
"anthropic_tokens_limit": 100000, # INCREASED: 100K tokens per provider
|
||||
"mistral_tokens_limit": 100000, # INCREASED: 100K tokens per provider
|
||||
"monthly_cost_limit": 25.0, # $25 cap - podcast-focused pricing
|
||||
"monthly_cost_limit": 45.0, # ADJUSTED: $45 cap (aligns with $40-50 hard limit target)
|
||||
"features": ["full_content_generation", "advanced_research", "basic_analytics", "all_tools_access", "oss_models_priority"],
|
||||
"description": "Perfect for individuals and small teams. Access all ALwrity features with generous limits powered by OSS AI models."
|
||||
},
|
||||
@@ -570,7 +555,6 @@ class PricingService:
|
||||
"tier": SubscriptionTier.PRO,
|
||||
"price_monthly": 79.0,
|
||||
"price_yearly": 790.0,
|
||||
"ai_text_generation_calls_limit": 3000, # Explicit: Pro gets 3000 AI text calls
|
||||
"gemini_calls_limit": 5000,
|
||||
"openai_calls_limit": 2500,
|
||||
"anthropic_calls_limit": 1000,
|
||||
@@ -579,17 +563,16 @@ class PricingService:
|
||||
"serper_calls_limit": 1000,
|
||||
"metaphor_calls_limit": 500,
|
||||
"firecrawl_calls_limit": 500,
|
||||
"stability_calls_limit": 100, # 100 images - good for regular podcasts
|
||||
"exa_calls_limit": 500, # 500 research queries
|
||||
"video_calls_limit": 30, # 30 videos - enough for daily episodes
|
||||
"image_edit_calls_limit": 100, # 100 AI image edits
|
||||
"audio_calls_limit": 100, # 100 audio clips - podcast-focused
|
||||
"wavespeed_calls_limit": 500, # WaveSpeed combined limit: TTS + video + image + LLM
|
||||
"stability_calls_limit": 200,
|
||||
"exa_calls_limit": 2000,
|
||||
"video_calls_limit": 50, # 50 videos/month for pro plan
|
||||
"image_edit_calls_limit": 100, # 100 AI image editing calls/month
|
||||
"audio_calls_limit": 200, # 200 AI audio generation calls/month
|
||||
"gemini_tokens_limit": 5000000,
|
||||
"openai_tokens_limit": 2500000,
|
||||
"anthropic_tokens_limit": 1000000,
|
||||
"mistral_tokens_limit": 2500000,
|
||||
"monthly_cost_limit": 100.0, # $100 cap - podcast-focused
|
||||
"monthly_cost_limit": 150.0,
|
||||
"features": ["unlimited_content_generation", "premium_research", "advanced_analytics", "priority_support"],
|
||||
"description": "Perfect for growing businesses"
|
||||
},
|
||||
@@ -598,7 +581,6 @@ class PricingService:
|
||||
"tier": SubscriptionTier.ENTERPRISE,
|
||||
"price_monthly": 199.0,
|
||||
"price_yearly": 1990.0,
|
||||
"ai_text_generation_calls_limit": 0, # Unlimited
|
||||
"gemini_calls_limit": 0, # Unlimited
|
||||
"openai_calls_limit": 0,
|
||||
"anthropic_calls_limit": 0,
|
||||
@@ -612,7 +594,6 @@ class PricingService:
|
||||
"video_calls_limit": 0, # Unlimited for enterprise
|
||||
"image_edit_calls_limit": 0, # Unlimited image editing for enterprise
|
||||
"audio_calls_limit": 0, # Unlimited audio generation for enterprise
|
||||
"wavespeed_calls_limit": 0, # Unlimited for enterprise
|
||||
"gemini_tokens_limit": 0,
|
||||
"openai_tokens_limit": 0,
|
||||
"anthropic_tokens_limit": 0,
|
||||
@@ -834,7 +815,6 @@ class PricingService:
|
||||
'video_calls': getattr(plan, 'video_calls_limit', 0), # Support missing column
|
||||
'image_edit_calls': getattr(plan, 'image_edit_calls_limit', 0), # Support missing column
|
||||
'audio_calls': getattr(plan, 'audio_calls_limit', 0), # Support missing column
|
||||
'wavespeed_calls': getattr(plan, 'wavespeed_calls_limit', 0), # WaveSpeed API calls
|
||||
# Token limits
|
||||
'gemini_tokens': plan.gemini_tokens_limit,
|
||||
'openai_tokens': plan.openai_tokens_limit,
|
||||
|
||||
@@ -29,12 +29,10 @@ def ensure_subscription_plan_columns(db: Session) -> None:
|
||||
|
||||
# Columns we may reference in models but might be missing in older DBs
|
||||
required_columns = {
|
||||
"ai_text_generation_calls_limit": "INTEGER DEFAULT 0",
|
||||
"exa_calls_limit": "INTEGER DEFAULT 0",
|
||||
"video_calls_limit": "INTEGER DEFAULT 0",
|
||||
"image_edit_calls_limit": "INTEGER DEFAULT 0",
|
||||
"audio_calls_limit": "INTEGER DEFAULT 0",
|
||||
"wavespeed_calls_limit": "INTEGER DEFAULT 0",
|
||||
}
|
||||
|
||||
for col_name, ddl in required_columns.items():
|
||||
|
||||
@@ -13,7 +13,6 @@ from sqlalchemy.orm import Session
|
||||
from sqlalchemy import desc
|
||||
from loguru import logger
|
||||
import json
|
||||
from api.subscription.cache import clear_dashboard_cache
|
||||
|
||||
from models.subscription_models import (
|
||||
APIUsageLog, UsageSummary, APIProvider, UsageAlert,
|
||||
@@ -45,12 +44,12 @@ class UsageTrackingService:
|
||||
self._enforce_cache: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
def _get_authoritative_billing_period_keys(self, user_id: str, billing_period: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Return authoritative billing period lookup keys. Always uses calendar month for consistency."""
|
||||
"""Return authoritative billing period lookup keys anchored to subscription period boundaries."""
|
||||
subscription = self.db.query(UserSubscription).filter(
|
||||
UserSubscription.user_id == user_id
|
||||
).first()
|
||||
|
||||
# If caller explicitly requested a billing period, use it
|
||||
# If caller explicitly requested a billing period, keep it authoritative for that read.
|
||||
if billing_period:
|
||||
return {
|
||||
"billing_period": billing_period,
|
||||
@@ -59,15 +58,23 @@ class UsageTrackingService:
|
||||
"period_end": subscription.current_period_end if subscription else None,
|
||||
}
|
||||
|
||||
# ALWAYS use current calendar month for billing period to ensure consistency
|
||||
# This prevents data loss when subscription spans month boundaries
|
||||
current_period = datetime.now().strftime("%Y-%m")
|
||||
|
||||
if subscription and subscription.current_period_start and subscription.current_period_end:
|
||||
start_key = subscription.current_period_start.strftime("%Y-%m")
|
||||
end_key = subscription.current_period_end.strftime("%Y-%m")
|
||||
lookup_periods = [start_key] if start_key == end_key else [start_key, end_key]
|
||||
return {
|
||||
"billing_period": start_key,
|
||||
"lookup_periods": lookup_periods,
|
||||
"period_start": subscription.current_period_start,
|
||||
"period_end": subscription.current_period_end,
|
||||
}
|
||||
|
||||
resolved_period = self.pricing_service.get_current_billing_period(user_id) or datetime.now().strftime("%Y-%m")
|
||||
return {
|
||||
"billing_period": current_period,
|
||||
"lookup_periods": [current_period],
|
||||
"period_start": subscription.current_period_start if subscription else None,
|
||||
"period_end": subscription.current_period_end if subscription else None,
|
||||
"billing_period": resolved_period,
|
||||
"lookup_periods": [resolved_period],
|
||||
"period_start": None,
|
||||
"period_end": None,
|
||||
}
|
||||
|
||||
async def track_api_usage(self, user_id: str, provider: APIProvider,
|
||||
@@ -163,12 +170,6 @@ class UsageTrackingService:
|
||||
|
||||
self.db.commit()
|
||||
|
||||
# Invalidate dashboard cache so header stats update immediately
|
||||
try:
|
||||
clear_dashboard_cache(user_id)
|
||||
except Exception as cache_err:
|
||||
logger.debug(f"Could not clear dashboard cache: {cache_err}")
|
||||
|
||||
logger.info(f"Tracked API usage: {user_id} -> {provider.value} -> ${cost_data['cost_total']:.6f}")
|
||||
|
||||
return {
|
||||
@@ -199,14 +200,11 @@ class UsageTrackingService:
|
||||
).first()
|
||||
|
||||
if not summary:
|
||||
logger.info(f"[UsageTracking] Creating new UsageSummary for user={user_id}, period={period_keys['billing_period']}")
|
||||
summary = UsageSummary(
|
||||
user_id=user_id,
|
||||
billing_period=period_keys["billing_period"]
|
||||
)
|
||||
self.db.add(summary)
|
||||
else:
|
||||
logger.debug(f"[UsageTracking] Found existing UsageSummary for user={user_id}, period={summary.billing_period}, calls={summary.total_calls}")
|
||||
|
||||
# Update provider-specific counters
|
||||
provider_name = provider.value
|
||||
@@ -379,19 +377,12 @@ class UsageTrackingService:
|
||||
period_keys = self._get_authoritative_billing_period_keys(user_id, requested_billing_period)
|
||||
billing_period = period_keys["billing_period"]
|
||||
|
||||
logger.debug(f"[get_user_usage_stats] user={user_id}, billing_period={billing_period}, lookup_periods={period_keys['lookup_periods']}")
|
||||
|
||||
# Get usage summary
|
||||
summary = self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id,
|
||||
UsageSummary.billing_period.in_(period_keys["lookup_periods"])
|
||||
).first()
|
||||
|
||||
if summary:
|
||||
logger.debug(f"[get_user_usage_stats] Found summary: period={summary.billing_period}, calls={summary.total_calls}, cost={summary.total_cost}")
|
||||
else:
|
||||
logger.debug(f"[get_user_usage_stats] No summary found for user={user_id}, period={billing_period}")
|
||||
|
||||
# Get user limits
|
||||
limits = self.pricing_service.get_user_limits(user_id)
|
||||
|
||||
@@ -530,26 +521,20 @@ class UsageTrackingService:
|
||||
|
||||
async def reset_current_billing_period(self, user_id: str) -> Dict[str, Any]:
|
||||
"""Reset usage status and counters for the current billing period (after plan renewal/change)."""
|
||||
period_keys = self._get_authoritative_billing_period_keys(user_id)
|
||||
billing_period = period_keys["billing_period"]
|
||||
summary = self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id,
|
||||
UsageSummary.billing_period.in_(period_keys["lookup_periods"])
|
||||
).first()
|
||||
|
||||
if not summary:
|
||||
return {"reset": False, "reason": "no_summary"}
|
||||
|
||||
try:
|
||||
period_keys = self._get_authoritative_billing_period_keys(user_id)
|
||||
billing_period = period_keys["billing_period"]
|
||||
summary = self.db.query(UsageSummary).filter(
|
||||
UsageSummary.user_id == user_id,
|
||||
UsageSummary.billing_period.in_(period_keys["lookup_periods"])
|
||||
).first()
|
||||
|
||||
if not summary:
|
||||
return {"reset": False, "reason": "no_summary"}
|
||||
|
||||
reset_usage_summary_counters(summary)
|
||||
self.db.commit()
|
||||
|
||||
# Invalidate dashboard cache so header stats update after reset
|
||||
try:
|
||||
clear_dashboard_cache(user_id)
|
||||
except Exception as cache_err:
|
||||
logger.debug(f"Could not clear dashboard cache: {cache_err}")
|
||||
|
||||
|
||||
logger.info(f"Reset usage counters for user {user_id} in billing period {billing_period} after renewal")
|
||||
return {"reset": True, "counters_reset": True}
|
||||
except Exception as e:
|
||||
|
||||
@@ -4,7 +4,6 @@ Handles fetching user data from the onboarding database.
|
||||
"""
|
||||
|
||||
from typing import Optional, List, Dict, Any
|
||||
from datetime import datetime
|
||||
from sqlalchemy.orm import Session
|
||||
from loguru import logger
|
||||
|
||||
@@ -93,88 +92,5 @@ class UserDataService:
|
||||
return integrated_data.get('website_analysis')
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting user website analysis: {e}")
|
||||
return None
|
||||
|
||||
def save_website_extraction(self, user_id: str, extraction_data: Dict[str, Any]) -> bool:
|
||||
"""
|
||||
Save website extraction data for future use.
|
||||
|
||||
Args:
|
||||
user_id: The user ID
|
||||
extraction_data: Website extraction data (title, summary, highlights, url, subpages)
|
||||
|
||||
Returns:
|
||||
True if saved successfully
|
||||
"""
|
||||
try:
|
||||
# Clean data - remove images/favicon
|
||||
clean_data = {
|
||||
k: v for k, v in extraction_data.items()
|
||||
if k not in ('image', 'favicon')
|
||||
}
|
||||
clean_data['saved_at'] = datetime.now().isoformat()
|
||||
|
||||
# Find or create user session for storing
|
||||
onboarding = self.db.query(OnboardingSession).filter(
|
||||
OnboardingSession.user_id == user_id
|
||||
).first()
|
||||
|
||||
if not onboarding:
|
||||
# Create new session if not exists
|
||||
onboarding = OnboardingSession(user_id=user_id)
|
||||
self.db.add(onboarding)
|
||||
|
||||
# Try to update website_analysis field
|
||||
# The field might be JSON in the model
|
||||
try:
|
||||
existing = onboarding.website_analysis
|
||||
if isinstance(existing, dict):
|
||||
existing.update(clean_data)
|
||||
onboarding.website_analysis = existing
|
||||
else:
|
||||
onboarding.website_analysis = clean_data
|
||||
except Exception as ex:
|
||||
logger.warning(f"Could not update website_analysis: {ex}")
|
||||
onboarding.website_analysis = clean_data
|
||||
|
||||
self.db.commit()
|
||||
logger.info(f"Saved website extraction for user {user_id}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error saving website extraction: {str(e)}")
|
||||
self.db.rollback()
|
||||
return False
|
||||
|
||||
def get_website_extraction(self, user_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Get saved website extraction data.
|
||||
|
||||
Args:
|
||||
user_id: The user ID
|
||||
|
||||
Returns:
|
||||
Website extraction data or None
|
||||
"""
|
||||
try:
|
||||
onboarding = self.db.query(OnboardingSession).filter(
|
||||
OnboardingSession.user_id == user_id
|
||||
).first()
|
||||
|
||||
if not onboarding:
|
||||
return None
|
||||
|
||||
extraction = onboarding.website_analysis
|
||||
if isinstance(extraction, dict):
|
||||
# Return clean data without internal fields
|
||||
return {
|
||||
k: v for k, v in extraction.items()
|
||||
if k not in ('saved_at', 'full_analysis', 'analysis_status')
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting website extraction: {str(e)}")
|
||||
logger.error(f"Error getting user website analysis: {str(e)}")
|
||||
return None
|
||||
|
||||
@@ -481,9 +481,7 @@ class SpeechGenerator:
|
||||
raise HTTPException(status_code=502, detail="WaveSpeed Qwen3 voice clone returned no outputs")
|
||||
|
||||
audio_url = self._extract_audio_url(outputs)
|
||||
downloaded_audio = self._download_audio(audio_url, timeout)
|
||||
logger.warning(f"[WaveSpeed] qwen3_voice_clone downloaded {len(downloaded_audio)} bytes")
|
||||
return downloaded_audio
|
||||
return self._download_audio(audio_url, timeout)
|
||||
|
||||
def cosyvoice_voice_clone(
|
||||
self,
|
||||
|
||||
@@ -9,17 +9,10 @@ import os
|
||||
import sys
|
||||
import json
|
||||
import argparse
|
||||
import platform
|
||||
from pathlib import Path
|
||||
from dataclasses import dataclass, asdict
|
||||
from typing import Optional
|
||||
|
||||
# Detect platform
|
||||
IS_WINDOWS = platform.system() == "Windows"
|
||||
IS_LINUX = platform.system() == "Linux"
|
||||
|
||||
import uvicorn
|
||||
|
||||
|
||||
@dataclass
|
||||
class BootstrapResult:
|
||||
@@ -100,7 +93,7 @@ def bootstrap_linguistic_models() -> BootstrapResult:
|
||||
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||
|
||||
if verbose:
|
||||
print("[DEBUG] Bootstrapping linguistic models...")
|
||||
print("🔍 Bootstrapping linguistic models...")
|
||||
|
||||
# Check and download spaCy model
|
||||
try:
|
||||
@@ -108,7 +101,7 @@ def bootstrap_linguistic_models() -> BootstrapResult:
|
||||
try:
|
||||
nlp = spacy.load("en_core_web_sm")
|
||||
if verbose:
|
||||
print(" [OK] spaCy model 'en_core_web_sm' available")
|
||||
print(" ✅ spaCy model 'en_core_web_sm' available")
|
||||
except OSError:
|
||||
if verbose:
|
||||
print(" ⚠️ spaCy model 'en_core_web_sm' not found, downloading...")
|
||||
@@ -117,10 +110,10 @@ def bootstrap_linguistic_models() -> BootstrapResult:
|
||||
sys.executable, "-m", "spacy", "download", "en_core_web_sm"
|
||||
])
|
||||
if verbose:
|
||||
print(" [OK] spaCy model downloaded successfully")
|
||||
print(" ✅ spaCy model downloaded successfully")
|
||||
except subprocess.CalledProcessError as e:
|
||||
if verbose:
|
||||
print(f" [FAIL] Failed to download spaCy model: {e}")
|
||||
print(f" ❌ Failed to download spaCy model: {e}")
|
||||
print(" Please run: python -m spacy download en_core_web_sm")
|
||||
return BootstrapResult(name="linguistic_models", success=False, skipped=False, reason="spacy_download_failed")
|
||||
except ImportError:
|
||||
@@ -140,14 +133,14 @@ def bootstrap_linguistic_models() -> BootstrapResult:
|
||||
try:
|
||||
nltk.data.find(path)
|
||||
if verbose:
|
||||
print(f" [OK] NLTK {data_package} available")
|
||||
print(f" ✅ NLTK {data_package} available")
|
||||
except LookupError:
|
||||
if verbose:
|
||||
print(f" ⚠️ NLTK {data_package} not found, downloading...")
|
||||
try:
|
||||
nltk.download(data_package, quiet=True)
|
||||
if verbose:
|
||||
print(f" [OK] NLTK {data_package} downloaded")
|
||||
print(f" ✅ NLTK {data_package} downloaded")
|
||||
except Exception as e:
|
||||
if verbose:
|
||||
print(f" ⚠️ Failed to download {data_package}: {e}")
|
||||
@@ -155,7 +148,7 @@ def bootstrap_linguistic_models() -> BootstrapResult:
|
||||
try:
|
||||
nltk.download('punkt', quiet=True)
|
||||
if verbose:
|
||||
print(f" [OK] NLTK punkt (fallback) downloaded")
|
||||
print(f" ✅ NLTK punkt (fallback) downloaded")
|
||||
except:
|
||||
pass
|
||||
except ImportError:
|
||||
@@ -163,7 +156,7 @@ def bootstrap_linguistic_models() -> BootstrapResult:
|
||||
print(" ⚠️ NLTK not installed - skipping")
|
||||
|
||||
if verbose:
|
||||
print("[OK] Linguistic model bootstrap complete")
|
||||
print("✅ Linguistic model bootstrap complete")
|
||||
return BootstrapResult(name="linguistic_models", success=True, skipped=False)
|
||||
|
||||
|
||||
@@ -207,7 +200,7 @@ def bootstrap_local_llm_models() -> BootstrapResult:
|
||||
# This checks cache and downloads if missing
|
||||
snapshot_download(repo_id=target_model, repo_type="model")
|
||||
if verbose:
|
||||
print(f" [OK] Local LLM '{target_model}' available")
|
||||
print(f" ✅ Local LLM '{target_model}' available")
|
||||
except Exception as e:
|
||||
if verbose:
|
||||
print(f" ⚠️ Failed to download/check local LLM: {e}")
|
||||
@@ -226,25 +219,19 @@ BOOTSTRAP_RESULTS = []
|
||||
|
||||
# Load .env file early so ALWRITY_ENABLED_FEATURES is available
|
||||
from dotenv import load_dotenv
|
||||
from pathlib import Path
|
||||
load_dotenv()
|
||||
|
||||
# Load from backend/.env specifically
|
||||
backend_dir = Path(__file__).parent
|
||||
load_dotenv(backend_dir / '.env')
|
||||
|
||||
# Debug: Print what PORT is set to - IMMEDIATELY at startup
|
||||
# Debug: Print what PORT is set to
|
||||
import os
|
||||
print(f"[STARTUP] PORT env: {os.getenv('PORT')}", flush=True)
|
||||
print(f"[STARTUP] RENDER env: {os.getenv('RENDER')}", flush=True)
|
||||
print(f"[STARTUP] ALWRITY_ENABLED_FEATURES: {os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||
print(f"[STARTUP] HOST env: {os.getenv('HOST')}", flush=True)
|
||||
print(f"[DEBUG] PORT env: {os.getenv('PORT')}")
|
||||
print(f"[DEBUG] RENDER env: {os.getenv('RENDER')}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
enabled_features = get_enabled_features()
|
||||
features_str = ",".join(sorted(enabled_features))
|
||||
os.environ["ALWRITY_ENABLED_FEATURES"] = features_str
|
||||
|
||||
print(f"\n[OK] Enabled features: {features_str}")
|
||||
print(f"\n📋 Enabled features: {features_str}")
|
||||
|
||||
if should_bootstrap_linguistic_models():
|
||||
result = bootstrap_linguistic_models()
|
||||
@@ -252,7 +239,7 @@ if __name__ == "__main__":
|
||||
else:
|
||||
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||
if verbose:
|
||||
print("[SKIP] Skipping linguistic model bootstrap (profile-gated)")
|
||||
print("⏭️ Skipping linguistic model bootstrap (profile-gated)")
|
||||
BOOTSTRAP_RESULTS.append(BootstrapResult(name="linguistic_models", success=True, skipped=True, reason="profile_gated"))
|
||||
|
||||
if should_bootstrap_local_llm_models():
|
||||
@@ -261,7 +248,7 @@ if __name__ == "__main__":
|
||||
else:
|
||||
verbose = os.getenv("ALWRITY_VERBOSE", "false").lower() == "true"
|
||||
if verbose:
|
||||
print("[SKIP] Skipping local LLM model bootstrap (feature-gated)")
|
||||
print("⏭️ Skipping local LLM model bootstrap (feature-gated)")
|
||||
BOOTSTRAP_RESULTS.append(BootstrapResult(name="local_llm_models", success=True, skipped=True, reason="feature_gated"))
|
||||
|
||||
summary = {
|
||||
@@ -270,9 +257,9 @@ if __name__ == "__main__":
|
||||
}
|
||||
os.environ["ALWRITY_BOOTSTRAP_SUMMARY"] = json.dumps(summary)
|
||||
|
||||
print(f"\n[INFO] Bootstrap Summary:")
|
||||
print(f"\n📋 Bootstrap Summary:")
|
||||
for r in BOOTSTRAP_RESULTS:
|
||||
status = "[SKIP] Skipped" if r.skipped else ("[OK] Enabled" if r.success else "[FAIL] Failed")
|
||||
status = "⏭️ Skipped" if r.skipped else ("✅ Enabled" if r.success else "❌ Failed")
|
||||
print(f" {r.name}: {status}" + (f" ({r.reason})" if r.reason else ""))
|
||||
|
||||
# NOW import modular utilities (after bootstrap)
|
||||
@@ -286,24 +273,23 @@ from alwrity_utils import (
|
||||
|
||||
def start_backend(enable_reload=False, production_mode=False):
|
||||
"""Start the backend server."""
|
||||
print("==> Starting ALwrity Backend...")
|
||||
print("🚀 Starting ALwrity Backend...")
|
||||
podcast_only_demo_mode = os.getenv("ALWRITY_PODCAST_ONLY_DEMO_MODE", os.getenv("PODCAST_ONLY_DEMO_MODE", "false")).lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
if podcast_only_demo_mode:
|
||||
print("\n" + "=" * 60)
|
||||
print("==> PODCAST-ONLY DEMO MODE ACTIVE")
|
||||
print("🎙️ PODCAST-ONLY DEMO MODE ACTIVE")
|
||||
print(" Non-podcast router groups are intentionally skipped.")
|
||||
print("=" * 60)
|
||||
|
||||
# Set host based on environment and mode
|
||||
# Use 127.0.0.1 for local production testing on Windows
|
||||
# Use 0.0.0.0 for actual cloud deployments (Render, Railway, etc.)
|
||||
# Render provides PORT env var, detect cloud by presence of PORT
|
||||
render_port = os.getenv("PORT")
|
||||
if render_port:
|
||||
# Cloud deployment detected (Render sets PORT env var) - use 0.0.0.0
|
||||
# Render provides PORT env var, we must bind to it.
|
||||
default_host = os.getenv("RENDER") or os.getenv("RAILWAY_ENVIRONMENT") or os.getenv("DEPLOY_ENV")
|
||||
if default_host:
|
||||
# Cloud deployment detected - use 0.0.0.0
|
||||
os.environ.setdefault("HOST", "0.0.0.0")
|
||||
os.environ.setdefault("PORT", render_port)
|
||||
else:
|
||||
# Local deployment - use 127.0.0.1 for better Windows compatibility
|
||||
os.environ.setdefault("HOST", "127.0.0.1")
|
||||
@@ -315,46 +301,40 @@ def start_backend(enable_reload=False, production_mode=False):
|
||||
# Set reload based on argument or environment variable
|
||||
if enable_reload and not production_mode:
|
||||
os.environ.setdefault("RELOAD", "true")
|
||||
print(" [DEV] Development mode: Auto-reload enabled")
|
||||
print(" 🔄 Development mode: Auto-reload enabled")
|
||||
else:
|
||||
os.environ.setdefault("RELOAD", "false")
|
||||
print(" [PROD] Production mode: Auto-reload disabled")
|
||||
print(" 🏭 Production mode: Auto-reload disabled")
|
||||
|
||||
host = os.getenv("HOST", "0.0.0.0")
|
||||
port = int(os.getenv("PORT", "8000"))
|
||||
reload = os.environ.get("RELOAD", "false").lower() == "true"
|
||||
print(f"[DEBUG] Bind prepared - host={host}, port={port}, reload={reload}", flush=True)
|
||||
print(f"[DEBUG] ENV check - ALWRITY_ENABLED_FEATURES={os.getenv('ALWRITY_ENABLED_FEATURES')}", flush=True)
|
||||
reload = os.getenv("RELOAD", "false").lower() == "true"
|
||||
|
||||
print(f" ==> Host: {host}", flush=True)
|
||||
print(f" ==> Port: {port}", flush=True)
|
||||
print(f" [DEV] Reload: {reload}", flush=True)
|
||||
print(f"[DEBUG] About to import app module...", flush=True)
|
||||
print("[DEBUG] >>> START APP IMPORT <<<", flush=True)
|
||||
print(f" 📍 Host: {host}")
|
||||
print(f" 🔌 Port: {port}")
|
||||
print(f" 🔄 Reload: {reload}")
|
||||
print(f"[DEBUG] Starting server with host={host}, port={port}")
|
||||
|
||||
try:
|
||||
# Import and run the app
|
||||
from app import app
|
||||
print("[DEBUG] >>> END APP IMPORT <<<", flush=True)
|
||||
|
||||
import uvicorn
|
||||
print(f"[DEBUG] Imported app and uvicorn successfully", flush=True)
|
||||
|
||||
# Note: Database already initialized by DatabaseSetup in main()
|
||||
|
||||
print("\n[WORLD] ALwrity Backend Server", flush=True)
|
||||
print("=" * 50, flush=True)
|
||||
print(f" 📖 API Documentation: http://localhost:{os.getenv('PORT', '8000')}/api/docs", flush=True)
|
||||
print(f" 🔍 Health Check: http://localhost:{os.getenv('PORT', '8000')}/health", flush=True)
|
||||
print(f" 📊 ReDoc: http://localhost:{os.getenv('PORT', '8000')}/api/redoc", flush=True)
|
||||
print("\n🌐 ALwrity Backend Server")
|
||||
print("=" * 50)
|
||||
print(" 📖 API Documentation: http://localhost:8000/api/docs")
|
||||
print(" 🔍 Health Check: http://localhost:8000/health")
|
||||
print(" 📊 ReDoc: http://localhost:8000/api/redoc")
|
||||
|
||||
if not production_mode:
|
||||
print(f" 📈 API Monitoring: http://localhost:{os.getenv('PORT', '8000')}/api/content-planning/monitoring/health", flush=True)
|
||||
print(f" 💳 Billing Dashboard: http://localhost:{os.getenv('PORT', '8000')}/api/subscription/plans", flush=True)
|
||||
print(f" 📊 Usage Tracking: http://localhost:{os.getenv('PORT', '8000')}/api/subscription/usage/demo", flush=True)
|
||||
print(" 📈 API Monitoring: http://localhost:8000/api/content-planning/monitoring/health")
|
||||
print(" 💳 Billing Dashboard: http://localhost:8000/api/subscription/plans")
|
||||
print(" 📊 Usage Tracking: http://localhost:8000/api/subscription/usage/demo")
|
||||
|
||||
print("\n[STOP] Press Ctrl+C to stop the server", flush=True)
|
||||
print("=" * 50, flush=True)
|
||||
print("\n[STOP] Press Ctrl+C to stop the server")
|
||||
print("=" * 50)
|
||||
|
||||
# Set up clean logging for end users
|
||||
from logging_config import setup_clean_logging, get_uvicorn_log_level
|
||||
@@ -382,26 +362,6 @@ def start_backend(enable_reload=False, production_mode=False):
|
||||
print(f"[ERROR] Video stack preflight failed: {_video_stack_err}")
|
||||
return False
|
||||
|
||||
print(f"[DEBUG] Starting uvicorn with host={host} port={port}", flush=True)
|
||||
print("[DEBUG] >>> ABOUT TO CALL UVICORN.RUN() <<<", flush=True)
|
||||
|
||||
# Skip video preflight in podcast-only mode to save memory/time
|
||||
is_podcast = os.getenv("ALWRITY_ENABLED_FEATURES", "").strip().lower() == "podcast"
|
||||
print(f"[DEBUG] Podcast mode check: {is_podcast}", flush=True)
|
||||
|
||||
if is_podcast:
|
||||
print("[DEBUG] Podcast mode - skipping video preflight", flush=True)
|
||||
else:
|
||||
# Log diagnostics and assert versions (fail fast if misconfigured)
|
||||
try:
|
||||
if log_video_stack_diagnostics:
|
||||
log_video_stack_diagnostics()
|
||||
if assert_supported_moviepy:
|
||||
assert_supported_moviepy()
|
||||
except Exception as _video_stack_err:
|
||||
print(f"[ERROR] Video stack preflight failed: {_video_stack_err}")
|
||||
return False
|
||||
|
||||
uvicorn.run(
|
||||
"app:app",
|
||||
host=host,
|
||||
@@ -441,14 +401,11 @@ def start_backend(enable_reload=False, production_mode=False):
|
||||
],
|
||||
log_level=uvicorn_log_level
|
||||
)
|
||||
print("[DEBUG] uvicorn.run() has finished", flush=True)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
print("\n\n🛑 Backend stopped by user")
|
||||
except Exception as e:
|
||||
print(f"\n[ERROR] Error starting backend: {e}", flush=True)
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
print(f"\n[ERROR] Error starting backend: {e}")
|
||||
return False
|
||||
|
||||
return True
|
||||
@@ -501,12 +458,12 @@ def main():
|
||||
"Starting server"
|
||||
]
|
||||
|
||||
print("==> Initializing ALwrity...")
|
||||
print("🔧 Initializing ALwrity...")
|
||||
|
||||
# Apply production optimizations if needed
|
||||
if production_mode:
|
||||
if not production_optimizer.apply_production_optimizations():
|
||||
print("[FAIL] Production optimization failed")
|
||||
print("❌ Production optimization failed")
|
||||
return False
|
||||
|
||||
# Step 1: Dependencies
|
||||
@@ -515,11 +472,11 @@ def main():
|
||||
if not critical_ok:
|
||||
print("installing...", end=" ", flush=True)
|
||||
if not dependency_manager.install_requirements():
|
||||
print("[FAIL] Failed")
|
||||
print("❌ Failed")
|
||||
return False
|
||||
print("[OK] Done")
|
||||
print("✅ Done")
|
||||
else:
|
||||
print("[OK] Done")
|
||||
print("✅ Done")
|
||||
|
||||
# Check optional dependencies (non-critical) - only in verbose mode
|
||||
if verbose_mode:
|
||||
@@ -528,24 +485,24 @@ def main():
|
||||
# Step 2: Environment
|
||||
print(f" 🔧 {setup_steps[1]}...", end=" ", flush=True)
|
||||
if not environment_setup.setup_directories():
|
||||
print("[FAIL] Directory setup failed")
|
||||
print("❌ Directory setup failed")
|
||||
return False
|
||||
|
||||
if not environment_setup.setup_environment_variables():
|
||||
print("[FAIL] Environment setup failed")
|
||||
print("❌ Environment setup failed")
|
||||
return False
|
||||
|
||||
# Create .env file only in development
|
||||
if not production_mode:
|
||||
environment_setup.create_env_file()
|
||||
print("[OK] Done")
|
||||
print("✅ Done")
|
||||
|
||||
# Step 3: Database
|
||||
print(f" 📊 {setup_steps[2]}...", end=" ", flush=True)
|
||||
if not database_setup.setup_essential_tables():
|
||||
print("⚠️ Issues detected, continuing...")
|
||||
else:
|
||||
print("[OK] Done")
|
||||
print("✅ Done")
|
||||
|
||||
# Setup advanced features in development, verify in all modes
|
||||
if not production_mode:
|
||||
|
||||
@@ -1,156 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import types
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
# Lightweight fallback for environments missing loguru.
|
||||
if "loguru" not in sys.modules:
|
||||
stub = types.ModuleType("loguru")
|
||||
stub.logger = types.SimpleNamespace(
|
||||
info=lambda *a, **k: None,
|
||||
warning=lambda *a, **k: None,
|
||||
error=lambda *a, **k: None,
|
||||
debug=lambda *a, **k: None,
|
||||
)
|
||||
sys.modules["loguru"] = stub
|
||||
|
||||
def _load_module(name: str, rel_path: str):
|
||||
base = Path(__file__).resolve().parents[1]
|
||||
path = base / rel_path
|
||||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
flat_mod = _load_module("agent_flat_context_under_test", "services/intelligence/agent_flat_context.py")
|
||||
sys.modules.setdefault("services.intelligence.agent_flat_context", flat_mod)
|
||||
vfs_mod = _load_module("agent_context_vfs_under_test", "services/intelligence/agent_context_vfs.py")
|
||||
|
||||
AgentFlatContextStore = flat_mod.AgentFlatContextStore
|
||||
AgentContextVFS = vfs_mod.AgentContextVFS
|
||||
|
||||
|
||||
def _cleanup_workspace(user_id: str, project_id: str | None = None) -> None:
|
||||
safe_user = ''.join(c for c in str(user_id) if c.isalnum() or c in ('-', '_')) or 'unknown_user'
|
||||
root = Path(__file__).resolve().parents[2] / 'workspace'
|
||||
user_dir = root / f'workspace_{safe_user}'
|
||||
if user_dir.exists():
|
||||
import shutil
|
||||
shutil.rmtree(user_dir, ignore_errors=True)
|
||||
|
||||
if project_id:
|
||||
safe_project = ''.join(c for c in str(project_id) if c.isalnum() or c in ('-', '_')) or 'default_project'
|
||||
project_dir = root / f'project_{safe_project}'
|
||||
if project_dir.exists():
|
||||
import shutil
|
||||
shutil.rmtree(project_dir, ignore_errors=True)
|
||||
|
||||
|
||||
def test_search_context_query_variants_and_can_answer():
|
||||
user_id = 'pytest_vfs_user'
|
||||
_cleanup_workspace(user_id)
|
||||
|
||||
store = AgentFlatContextStore(user_id)
|
||||
payload = {
|
||||
'website_url': 'https://example.com',
|
||||
'brand_analysis': {'brand_voice': 'Authoritative'},
|
||||
'recommended_settings': {'writing_tone': 'Conversational'},
|
||||
'content_type': {'primary_type': 'Blog'},
|
||||
'target_audience': {'primary_audience': 'Founders'},
|
||||
}
|
||||
assert store.save_step2_website_analysis(payload)
|
||||
|
||||
vfs = AgentContextVFS(user_id)
|
||||
result = vfs.search_context('tone')
|
||||
|
||||
assert result['query'] == 'tone'
|
||||
assert 'attempted_queries' in result
|
||||
assert result['attempted_queries'][0] == 'tone'
|
||||
assert result['can_answer'] is True
|
||||
assert len(result['results']) >= 1
|
||||
assert 'triage_top5' in result
|
||||
assert len(result['triage_top5']) >= 1
|
||||
assert 'low_probability' in result['results'][0]
|
||||
|
||||
|
||||
def test_inspect_file_large_document_summary_plus_keys():
|
||||
user_id = 'pytest_vfs_large'
|
||||
_cleanup_workspace(user_id)
|
||||
|
||||
store = AgentFlatContextStore(user_id)
|
||||
large_blob = 'x' * 9000
|
||||
payload = {
|
||||
'website_url': 'https://big.example.com',
|
||||
'brand_analysis': {'brand_voice': 'Bold'},
|
||||
'recommended_settings': {'writing_tone': 'Direct'},
|
||||
'target_audience': {'primary_audience': 'Teams'},
|
||||
'crawl_result': {'raw': large_blob},
|
||||
}
|
||||
assert store.save_step2_website_analysis(payload)
|
||||
|
||||
vfs = AgentContextVFS(user_id)
|
||||
out = vfs.inspect_file('step2_website_analysis.json')
|
||||
|
||||
assert out['mode'] == 'summary_plus_keys'
|
||||
assert 'agent_summary' in out
|
||||
assert 'keys' in out
|
||||
assert 'crawl_result' in out['keys']
|
||||
|
||||
|
||||
def test_write_shared_note_and_activity_log_created():
|
||||
user_id = 'pytest_collab_user'
|
||||
project_id = 'proj_abc'
|
||||
_cleanup_workspace(user_id, project_id)
|
||||
|
||||
vfs = AgentContextVFS(user_id, project_id=project_id)
|
||||
write_res = vfs.write_shared_note('Draft collaboration note', agent_id='agent_one')
|
||||
|
||||
assert write_res['ok'] is True
|
||||
assert write_res['file'] == 'collaboration.md'
|
||||
|
||||
collab = vfs.list_context()['collaboration']
|
||||
scratchpad = Path(collab['scratchpad_dir'])
|
||||
note_file = scratchpad / 'collaboration.md'
|
||||
log_file = scratchpad / 'activity_log.jsonl'
|
||||
|
||||
assert note_file.exists()
|
||||
assert log_file.exists()
|
||||
|
||||
content = note_file.read_text(encoding='utf-8')
|
||||
assert 'agent_one' in content
|
||||
assert 'Draft collaboration note' in content
|
||||
|
||||
lines = [json.loads(l) for l in log_file.read_text(encoding='utf-8').splitlines() if l.strip()]
|
||||
assert any(entry.get('event_type') == 'shared_note_written' for entry in lines)
|
||||
|
||||
|
||||
def test_read_struct_path_resolution_and_dependency_context():
|
||||
user_id = 'pytest_struct_user'
|
||||
_cleanup_workspace(user_id)
|
||||
|
||||
store = AgentFlatContextStore(user_id)
|
||||
assert store.save_step2_website_analysis(
|
||||
{
|
||||
'website_url': 'https://struct.example.com',
|
||||
'brand_analysis': {'brand_voice': 'Pragmatic'},
|
||||
'recommended_settings': {'writing_tone': 'Clear'},
|
||||
}
|
||||
)
|
||||
assert store.save_step4_persona_data(
|
||||
{
|
||||
'core_persona': {'name': 'Ops Leader', 'goal': 'Scale ops'},
|
||||
'selected_platforms': ['linkedin'],
|
||||
}
|
||||
)
|
||||
|
||||
vfs = AgentContextVFS(user_id)
|
||||
out = vfs.read_struct('step4_persona_data.json', 'data.core_persona.name')
|
||||
|
||||
assert out['ok'] is True
|
||||
assert out['data'] == 'Ops Leader'
|
||||
assert out['dependency_context']['brand_voice'] == 'Pragmatic'
|
||||
@@ -3,10 +3,6 @@ Media Utility Functions
|
||||
|
||||
Centralized helper functions for loading and managing media assets across modules.
|
||||
Promotes reuse between Podcast, YouTube, and other media-heavy modules.
|
||||
|
||||
DEPRECATED: The global DATA_MEDIA_DIR paths below are legacy and will be removed.
|
||||
New code should use workspace-scoped paths via utils.storage_paths or module-specific
|
||||
resolvers (e.g., api.podcast.constants.get_podcast_media_dir).
|
||||
"""
|
||||
|
||||
import logging
|
||||
@@ -16,19 +12,16 @@ from typing import Optional, List
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from services.database import WORKSPACE_DIR
|
||||
from utils.storage_paths import get_repo_root
|
||||
|
||||
# Configure logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Base Directories — use get_repo_root() for consistent resolution
|
||||
ROOT_DIR = get_repo_root()
|
||||
|
||||
# DEPRECATED: Global data/media paths — kept for backward-compat read fallback only.
|
||||
# New writes must go to workspace-scoped paths. Do NOT add new consumers.
|
||||
# Base Directories
|
||||
# backend/utils/media_utils.py -> parents[2] = backend/.. = root
|
||||
ROOT_DIR = Path(__file__).resolve().parents[2]
|
||||
DATA_MEDIA_DIR = ROOT_DIR / "data" / "media"
|
||||
|
||||
# Module-specific directories (DEPRECATED — use workspace-scoped resolvers instead)
|
||||
# Module-specific directories
|
||||
YOUTUBE_AVATARS_DIR = DATA_MEDIA_DIR / "youtube_avatars"
|
||||
YOUTUBE_IMAGES_DIR = DATA_MEDIA_DIR / "youtube_images"
|
||||
PODCAST_IMAGES_DIR = DATA_MEDIA_DIR / "podcast_images"
|
||||
@@ -40,7 +33,7 @@ def ensure_media_dirs() -> None:
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def resolve_media_path(media_url_or_path: str, user_id: Optional[str] = None) -> Optional[Path]:
|
||||
def resolve_media_path(media_url_or_path: str) -> Optional[Path]:
|
||||
"""
|
||||
Resolve a media URL or filename to a concrete file path on disk.
|
||||
|
||||
@@ -48,7 +41,6 @@ def resolve_media_path(media_url_or_path: str, user_id: Optional[str] = None) ->
|
||||
|
||||
Args:
|
||||
media_url_or_path: URL path (e.g. /api/youtube/avatars/foo.png) or filename
|
||||
user_id: Optional user ID for tenant-scoped resolution (recommended)
|
||||
|
||||
Returns:
|
||||
Path object if found, None otherwise
|
||||
@@ -78,9 +70,9 @@ def resolve_media_path(media_url_or_path: str, user_id: Optional[str] = None) ->
|
||||
parsed_path = urlparse(media_url_or_path).path
|
||||
parts = parsed_path.split("/")
|
||||
if len(parts) >= 6:
|
||||
asset_user_id = parts[3]
|
||||
safe_user_id = "".join(c for c in asset_user_id if c.isalnum() or c in ("-", "_"))
|
||||
if safe_user_id == asset_user_id:
|
||||
user_id = parts[3]
|
||||
safe_user_id = "".join(c for c in user_id if c.isalnum() or c in ("-", "_"))
|
||||
if safe_user_id == user_id:
|
||||
safe_filename = os.path.basename(filename)
|
||||
assets_path = Path(WORKSPACE_DIR) / f"workspace_{safe_user_id}" / "assets" / "avatars" / safe_filename
|
||||
if assets_path.exists() and assets_path.is_file():
|
||||
@@ -90,7 +82,7 @@ def resolve_media_path(media_url_or_path: str, user_id: Optional[str] = None) ->
|
||||
logger.error(f"[MediaUtils] Error resolving assets avatar path: {exc}")
|
||||
|
||||
# Define search paths in order of likelihood
|
||||
# We search all avatar/image directories (DEPRECATED: global paths — kept for backward-compat reads)
|
||||
# We search all avatar/image directories
|
||||
search_paths: List[Path] = [
|
||||
YOUTUBE_AVATARS_DIR / filename,
|
||||
PODCAST_AVATARS_DIR / filename,
|
||||
@@ -105,22 +97,13 @@ def resolve_media_path(media_url_or_path: str, user_id: Optional[str] = None) ->
|
||||
# Prioritize YouTube paths
|
||||
pass # Already first in list
|
||||
elif "/api/podcast/" in media_url_or_path:
|
||||
# Prioritize Podcast paths: use centralized podcast media resolution
|
||||
try:
|
||||
# Import the centralized function that checks tenant workspace first
|
||||
from api.podcast.constants import get_podcast_media_read_dirs
|
||||
podcast_dirs = get_podcast_media_read_dirs("image", user_id=user_id)
|
||||
search_paths = []
|
||||
for pod_dir in podcast_dirs:
|
||||
# Add both avatar and image subdirectories
|
||||
search_paths.append(pod_dir / "avatars" / filename)
|
||||
search_paths.append(pod_dir / filename)
|
||||
except ImportError:
|
||||
# Fallback if podcast constants not available
|
||||
search_paths = [
|
||||
PODCAST_AVATARS_DIR / filename,
|
||||
PODCAST_IMAGES_DIR / filename,
|
||||
]
|
||||
# Prioritize Podcast paths
|
||||
search_paths = [
|
||||
PODCAST_AVATARS_DIR / filename,
|
||||
PODCAST_IMAGES_DIR / filename,
|
||||
YOUTUBE_AVATARS_DIR / filename,
|
||||
YOUTUBE_IMAGES_DIR / filename
|
||||
]
|
||||
|
||||
# Iterate and find first existing file
|
||||
for path in search_paths:
|
||||
@@ -154,89 +137,3 @@ def load_media_bytes(media_url_or_path: str) -> Optional[bytes]:
|
||||
logger.error(f"[MediaUtils] Error reading file {path}: {e}")
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
# Audio format magic bytes signatures
|
||||
_AUDIO_SIGNATURES = [
|
||||
(b"\xff\xfb", "mp3"), # MP3 (MPEG-1 Layer 3, common)
|
||||
(b"\xff\xf3", "mp3"), # MP3 (MPEG-2.5 Layer 3)
|
||||
(b"\xff\xf2", "mp3"), # MP3 (MPEG-2 Layer 3)
|
||||
(b"\xff\xfa", "mp3"), # MP3 (MPEG-2 Layer 3 variant)
|
||||
(b"ID3", "mp3"), # MP3 with ID3 tag
|
||||
(b"RIFF", "wav"), # WAV (RIFF header)
|
||||
(b"OggS", "ogg"), # OGG
|
||||
(b"fLaC", "flac"), # FLAC
|
||||
(b"\x1a\x45\xdf\xa3", "webm"), # WebM / Matroska
|
||||
(b"ftyp", "m4a"), # MP4/M4A (ftyp box follows offset 4)
|
||||
]
|
||||
|
||||
|
||||
def detect_audio_format(audio_bytes: bytes) -> tuple[str, str]:
|
||||
"""Detect the actual audio format from content magic bytes.
|
||||
|
||||
Returns:
|
||||
Tuple of (format_name, mime_type).
|
||||
Falls back to ('wav', 'audio/wav') if no signature matches.
|
||||
"""
|
||||
if not audio_bytes or len(audio_bytes) < 4:
|
||||
return "wav", "audio/wav"
|
||||
|
||||
for signature, fmt in _AUDIO_SIGNATURES:
|
||||
if signature == b"ftyp":
|
||||
# M4A/MP4: 'ftyp' appears at offset 4
|
||||
if len(audio_bytes) > 8 and audio_bytes[4:8] == b"ftyp":
|
||||
return "m4a", "audio/mp4"
|
||||
elif audio_bytes[:len(signature)] == signature:
|
||||
mime_map = {
|
||||
"mp3": "audio/mpeg",
|
||||
"wav": "audio/wav",
|
||||
"ogg": "audio/ogg",
|
||||
"flac": "audio/flac",
|
||||
"webm": "audio/webm",
|
||||
"m4a": "audio/mp4",
|
||||
}
|
||||
return fmt, mime_map.get(fmt, "audio/wav")
|
||||
|
||||
# Check for Opus-in-OGG (Opus magic after OGG pages)
|
||||
if b"OpusHead" in audio_bytes[:100]:
|
||||
return "ogg", "audio/ogg"
|
||||
|
||||
# Check for MP4/M4A container (atoms starting with size + type)
|
||||
if len(audio_bytes) > 8:
|
||||
atom_type = audio_bytes[4:8]
|
||||
if atom_type in (b"moov", b"mdat", b"free", b"skip"):
|
||||
return "m4a", "audio/mp4"
|
||||
|
||||
return "wav", "audio/wav"
|
||||
|
||||
|
||||
def ensure_audio_extension(filename: str, audio_bytes: bytes) -> str:
|
||||
"""Adjust filename extension to match the actual audio format in audio_bytes.
|
||||
|
||||
Args:
|
||||
filename: Original filename (may have wrong extension like .wav for mp3 data)
|
||||
audio_bytes: The actual audio data bytes
|
||||
|
||||
Returns:
|
||||
Filename with corrected extension based on content format.
|
||||
"""
|
||||
fmt, _ = detect_audio_format(audio_bytes)
|
||||
ext_map = {
|
||||
"mp3": ".mp3",
|
||||
"wav": ".wav",
|
||||
"ogg": ".ogg",
|
||||
"flac": ".flac",
|
||||
"webm": ".webm",
|
||||
"m4a": ".m4a",
|
||||
"opus": ".ogg",
|
||||
}
|
||||
|
||||
correct_ext = ext_map.get(fmt, ".wav")
|
||||
path = Path(filename)
|
||||
current_ext = path.suffix.lower()
|
||||
|
||||
if current_ext != correct_ext:
|
||||
logger.info(f"[MediaUtils] Correcting audio extension: {filename} -> {path.stem}{correct_ext} (detected format: {fmt})")
|
||||
return f"{path.stem}{correct_ext}"
|
||||
|
||||
return filename
|
||||
|
||||
@@ -1,11 +1,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
from loguru import logger
|
||||
|
||||
_SAFE_CHARS = {"-", "_"}
|
||||
|
||||
|
||||
@@ -19,46 +16,9 @@ def _sanitize_segment(value: str, fallback: str) -> str:
|
||||
return cleaned or fallback
|
||||
|
||||
|
||||
def find_repo_root() -> Path:
|
||||
"""Find the project repository root directory.
|
||||
|
||||
Resolution order:
|
||||
1. ALWRITY_ROOT_DIR environment variable (explicit override for production)
|
||||
2. Deterministic path from this file (storage_paths.py is at utils/)
|
||||
3. Walk-up fallback looking for a 'backend/' directory at project root
|
||||
|
||||
Returns an absolute, resolved Path.
|
||||
"""
|
||||
env_root = os.environ.get("ALWRITY_ROOT_DIR")
|
||||
if env_root:
|
||||
root = Path(env_root).resolve()
|
||||
if root.is_dir():
|
||||
return root
|
||||
|
||||
# storage_paths.py is at backend/utils/storage_paths.py
|
||||
# project root is parents[2] (utils -> backend -> root)
|
||||
this_file = Path(__file__).resolve()
|
||||
candidate = this_file.parents[2]
|
||||
|
||||
if (candidate / "backend").is_dir():
|
||||
return candidate
|
||||
|
||||
# Walk-up fallback for unusual deployments
|
||||
current = this_file.parent
|
||||
for _ in range(10):
|
||||
if (current / "backend").is_dir():
|
||||
return current
|
||||
parent = current.parent
|
||||
if parent == current:
|
||||
break
|
||||
current = parent
|
||||
|
||||
return this_file.parents[2]
|
||||
|
||||
|
||||
def get_repo_root() -> Path:
|
||||
"""Return repository root as an absolute canonical path."""
|
||||
return find_repo_root()
|
||||
return Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def get_workspace_root() -> Path:
|
||||
@@ -107,7 +67,3 @@ def get_legacy_video_studio_upload_dirs() -> list[Path]:
|
||||
(repo_root / "backend" / "data" / "video_studio" / "uploads").resolve(),
|
||||
(repo_root / "backend" / "backend" / "data" / "video_studio" / "uploads").resolve(),
|
||||
]
|
||||
|
||||
|
||||
# Log resolved root at import time for production debugging
|
||||
logger.info(f"[StoragePaths] Repository root resolved to: {get_repo_root()}")
|
||||
|
||||
@@ -1,93 +0,0 @@
|
||||
# Podcast Maker API Reference
|
||||
|
||||
Base prefix: `/api/podcast`
|
||||
|
||||
This page summarizes the Podcast Maker endpoints currently represented in frontend and backend code.
|
||||
|
||||
## Endpoints by workflow stage
|
||||
|
||||
### Analysis and idea shaping
|
||||
|
||||
- `POST /idea/enhance`
|
||||
- `POST /analyze`
|
||||
- `POST /regenerate-queries`
|
||||
|
||||
### Research
|
||||
|
||||
- `POST /research/exa`
|
||||
|
||||
### Scripting
|
||||
|
||||
- `POST /script`
|
||||
- `POST /script/approve`
|
||||
|
||||
### Audio
|
||||
|
||||
- `POST /audio/upload`
|
||||
- `POST /audio`
|
||||
- `POST /combine-audio`
|
||||
- `GET /audio/{filename}`
|
||||
|
||||
### Images
|
||||
|
||||
- `POST /image`
|
||||
- `GET /images/{path}`
|
||||
|
||||
### Video
|
||||
|
||||
- `POST /render/video`
|
||||
- `POST /render/combine-videos`
|
||||
- `GET /videos`
|
||||
- `GET /videos/{filename}`
|
||||
- `GET /final-videos/{filename}`
|
||||
|
||||
### Avatars
|
||||
|
||||
- `POST /avatar/upload`
|
||||
- `POST /avatar/make-presentable`
|
||||
- `POST /avatar/generate`
|
||||
|
||||
### Projects
|
||||
|
||||
- `POST /projects`
|
||||
- `GET /projects`
|
||||
- `GET /projects/{project_id}`
|
||||
- `PUT /projects/{project_id}`
|
||||
- `DELETE /projects/{project_id}`
|
||||
- `POST /projects/{project_id}/favorite`
|
||||
|
||||
### Dubbing (backend available)
|
||||
|
||||
- `POST /dub/audio`
|
||||
- `GET /dub/{task_id}/result`
|
||||
- `GET /dub/audio/{filename}`
|
||||
- `POST /dub/estimate`
|
||||
- `GET /dub/languages`
|
||||
- `GET /dub/voices`
|
||||
- `POST /dub/voices/clone`
|
||||
- `GET /dub/voices/{task_id}/result`
|
||||
- `GET /dub/voices/audio/{filename}`
|
||||
|
||||
## Implementation details
|
||||
|
||||
### Endpoint usage in frontend service
|
||||
|
||||
The current `podcastApi.ts` directly calls these podcast routes for analysis, research, script, audio, image, video, avatar, and project workflows.
|
||||
|
||||
Known gap:
|
||||
|
||||
- `cancelTask()` is a placeholder that posts to `/api/story/task/{taskId}/cancel` rather than a dedicated podcast route.
|
||||
|
||||
### Request/response model notes
|
||||
|
||||
At a high level:
|
||||
|
||||
- Script endpoints exchange `idea`, `duration_minutes`, `speakers`, and optional `research`/`analysis`/`bible` context.
|
||||
- Audio endpoints exchange scene identifiers, text, and voice/rendering options.
|
||||
- Video endpoints exchange scene identifiers plus `audio_url` and optional image/prompt context.
|
||||
- Project endpoints exchange project-level state payloads suitable for restoring workflow progress.
|
||||
|
||||
## Engineering references
|
||||
|
||||
- `docs/Podcast_maker/AI_PODCAST_BACKEND_REFERENCE.md`
|
||||
- `docs/Podcast_maker/PODCAST_PERSISTENCE_IMPLEMENTATION.md`
|
||||
@@ -1,159 +0,0 @@
|
||||
# Podcast Maker Best Practices
|
||||
|
||||
This guide is implementation-aware: every recommendation below is based on how the current Podcast Maker APIs actually behave in frontend and backend code.
|
||||
|
||||
## 1) Start with budget-safe defaults (preflight-first workflow)
|
||||
|
||||
Podcast Maker runs **preflight validation** before major steps (analysis, research, script generation, TTS preview, and full TTS render). Use that as your workflow guardrail:
|
||||
|
||||
1. Analyze idea first
|
||||
2. Approve a small set of research queries
|
||||
3. Generate script
|
||||
4. Preview voice on short excerpts
|
||||
5. Render full scene audio
|
||||
6. Generate scene videos
|
||||
7. Combine final assets
|
||||
|
||||
Why this matters:
|
||||
- If credits/limits are insufficient, preflight fails fast before expensive operations.
|
||||
- Video generation also runs server-side animation validation and returns subscription-friendly errors for insufficient credits.
|
||||
|
||||
## 2) Duration vs. scene-count tradeoffs (cost + reliability)
|
||||
|
||||
The stack defaults to a **45s scene target** and cost estimate logic effectively scales scene count as:
|
||||
|
||||
- `scene_count ≈ ceil(duration_minutes * 60 / scene_length_target_seconds)`
|
||||
|
||||
Practical recommendations:
|
||||
- **5–8 min episodes**: target 5–8 scenes.
|
||||
- **10–15 min episodes**: target 8–14 scenes.
|
||||
- Increase `scene_length_target` when you need fewer API calls and faster completion.
|
||||
- Keep script concise because per-scene TTS has a **10,000-character max** (long text gets truncated by frontend before render).
|
||||
|
||||
Rule of thumb:
|
||||
- More scenes = better pacing granularity but more TTS/video calls.
|
||||
- Fewer scenes = cheaper/faster pipeline, but each scene must carry more narrative weight.
|
||||
|
||||
## 3) Voice strategy: preview first, render second
|
||||
|
||||
Use a two-pass voice workflow:
|
||||
|
||||
### Pass A: Preview and lock voice profile
|
||||
Use preview on short, representative lines (intro, data-heavy line, CTA) to validate:
|
||||
- voice identity
|
||||
- speed
|
||||
- emotion
|
||||
- pronunciation behavior (especially numbers/statistics)
|
||||
|
||||
### Pass B: Full scene render with tuned knobs
|
||||
When rendering scene audio, adjust only the knobs that matter:
|
||||
- `voice_id` (or `custom_voice_id` for cloned voice)
|
||||
- `speed` (default 1.0 is usually safest for timing)
|
||||
- `emotion` (scene-level emotion is supported)
|
||||
- `english_normalization` (keep enabled for number-heavy scripts)
|
||||
- audio format controls (`sample_rate`, `bitrate`, `channel`, `format`, `language_boost`) only when distribution requires them
|
||||
|
||||
Also note:
|
||||
- The frontend injects pause markers and strips markdown before TTS for better natural rhythm.
|
||||
- Use short lines (2–4 per scene is a good operational target from script generation guidance).
|
||||
|
||||
## 4) Research quality: when to use Exa config options
|
||||
|
||||
Use Exa config knobs intentionally, not by default.
|
||||
|
||||
### Search type
|
||||
- `auto`: default for most projects.
|
||||
- `keyword`: use when topic vocabulary is stable/specific.
|
||||
- `neural`: use when you need semantic discovery across mixed phrasing.
|
||||
|
||||
### Domain filters
|
||||
Use either include or exclude domains (not both).
|
||||
- Prefer `exa_include_domains` for compliance/brand-safe sourcing.
|
||||
- Use `exa_exclude_domains` to remove noisy/untrusted sources.
|
||||
|
||||
If both are sent, the backend/frontend sanitize behavior will prefer include-domain intent and drop the conflicting side.
|
||||
|
||||
### `max_sources`, category, and freshness
|
||||
- Increase `max_sources` only when synthesis quality is poor at default depth.
|
||||
- Use `date_range` (e.g. last month/quarter/year) for trend-sensitive topics.
|
||||
- Turn on statistics-oriented options when the episode needs hard numbers.
|
||||
|
||||
### Query operations
|
||||
- Always approve only the strongest queries before running research.
|
||||
- Empty query sets are rejected server-side.
|
||||
|
||||
## 5) Avatar + image prompt strategy for visual consistency
|
||||
|
||||
Consistency is strongest when you anchor scene images to a persistent base avatar.
|
||||
|
||||
Recommended approach:
|
||||
1. Create/upload a presenter avatar once per project.
|
||||
2. Reuse that avatar as `base_avatar_url` for scene images.
|
||||
3. Keep one shared style nucleus across prompts (lighting, environment, host look, framing).
|
||||
4. Change only scene-specific context (topic, emotion, supporting visual motif).
|
||||
|
||||
Important implementation notes:
|
||||
- If `base_avatar_url` is provided, image generation uses character-consistency flow; if the base avatar cannot be loaded, image generation fails (no silent fallback).
|
||||
- Keep scene emotion aligned to visual lighting cues for continuity.
|
||||
- For presenter generation, keep speakers realistic (supported range is 1–2).
|
||||
|
||||
## 6) Script and scene structure that survives production
|
||||
|
||||
Generate script with full context:
|
||||
- analysis (audience/type/keywords)
|
||||
- selected outline
|
||||
- research payload
|
||||
- bible/persona context
|
||||
|
||||
Then enforce editorial constraints before render:
|
||||
- Remove filler and repeated lines.
|
||||
- Ensure each scene has a single narrative job.
|
||||
- Keep line lengths short enough for natural TTS breathing.
|
||||
- Verify emotion tag is valid (`neutral`, `happy`, `excited`, `serious`, `curious`, `confident`) to avoid fallback normalization.
|
||||
|
||||
## 7) Project save/resume + asset-library workflows
|
||||
|
||||
Treat a podcast as a resumable production artifact.
|
||||
|
||||
### Save/resume
|
||||
- Persist state to project APIs throughout the workflow (analysis, research, script, render jobs, knobs, final video URL).
|
||||
- Use project list filtering/sorting to resume active work quickly.
|
||||
- Handle duplicate-idea conflicts by reopening existing project IDs instead of cloning work.
|
||||
|
||||
### Asset library workflow
|
||||
- Save generated and uploaded assets (audio/avatar/images) into the content asset library with project metadata.
|
||||
- Use consistent tags (`podcast`, project id, scene id) so assets are searchable and reusable.
|
||||
- Reuse previously approved host avatars and voice samples across episodes to reduce generation churn.
|
||||
|
||||
## 8) Video and dubbing execution strategy
|
||||
|
||||
### Video
|
||||
- Only pass supported video resolution (`480p` or `720p`).
|
||||
- Poll task status (video generation is asynchronous and can take up to ~10 minutes).
|
||||
- Use mask image only when you need controlled motion region.
|
||||
- Generate all scene videos before starting combine to avoid failed final assembly.
|
||||
|
||||
### Dubbing
|
||||
- Use `quality=low` for fast/cheap exploration.
|
||||
- Use `quality=high` + `use_voice_clone=true` when voice identity matters.
|
||||
- Keep `speed` in 0.5–2.0 and voice clone accuracy in 0.1–1.0.
|
||||
- For voice cloning, feed a clean 10–60s sample for best identity retention.
|
||||
|
||||
---
|
||||
|
||||
## Common failure modes and fixes
|
||||
|
||||
For broader platform issues, see the main [Troubleshooting Guide](../../guides/troubleshooting.md).
|
||||
|
||||
| Failure mode | Why it happens | Fix |
|
||||
|---|---|---|
|
||||
| Preflight blocked (analysis/research/script/TTS/video) | Insufficient credits or operation limits | Run lighter settings first: fewer scenes, lower duration, fewer research queries; then retry. |
|
||||
| Research request rejected | No approved queries selected | Approve at least one non-empty query before running Exa research. |
|
||||
| Research config mismatch | Include + exclude domains both supplied | Use only one domain filter type per run. |
|
||||
| Scene audio cuts off | Scene text exceeded TTS max characters | Reduce scene length/lines; split long scene into two scenes. |
|
||||
| Avatar-consistent image generation fails | `base_avatar_url` is broken/inaccessible | Re-upload avatar or switch to a valid project image URL; retry scene generation. |
|
||||
| Video task fails quickly | Invalid media URL, unsupported resolution, missing assets | Verify audio/image URLs are valid and use only `480p`/`720p`. |
|
||||
| Final combine video fails | One or more scene video files missing/invalid | Confirm every scene has a completed video task before combine. |
|
||||
| Dubbing quality sounds robotic | Low quality mode or weak source audio | Switch to high quality and/or use voice cloning with a cleaner sample. |
|
||||
| Voice clone results are unstable | Poor sample or extreme accuracy/speed settings | Use clean 10–60s sample; keep accuracy near default and speed near 1.0. |
|
||||
| Save appears inconsistent across sessions | Save failed and only partial local fallback exists | Trigger explicit save after each major step and verify project reload from API. |
|
||||
@@ -1,60 +0,0 @@
|
||||
# Podcast Maker Implementation Overview
|
||||
|
||||
This page keeps implementation details in one place for engineering and advanced troubleshooting.
|
||||
|
||||
## Architecture
|
||||
|
||||
Podcast Maker is split into:
|
||||
|
||||
- **Frontend orchestration service**: `frontend/src/services/podcastApi.ts`
|
||||
- Coordinates step flow (analysis → research → script → audio/video)
|
||||
- Runs preflight checks before expensive calls
|
||||
- Maps API payloads into UI-friendly objects
|
||||
- **Backend podcast handlers**: `backend/api/podcast/handlers/*.py`
|
||||
- Route-level APIs for analysis, research, script, media, and projects
|
||||
- Authenticated operations with user-scoped media/project data
|
||||
|
||||
## Frontend orchestration responsibilities
|
||||
|
||||
Primary responsibilities in `podcastApi.ts`:
|
||||
|
||||
- Create project analysis payloads and map response into Podcast Analysis UI data.
|
||||
- Build/validate research query payloads for Exa research route.
|
||||
- Generate script scenes and normalize scene/line structure for editor state.
|
||||
- Render per-scene audio and combine scenes into final audio.
|
||||
- Trigger scene image and video generation workflows.
|
||||
- Persist project state via project CRUD endpoints.
|
||||
|
||||
## Backend handler modules
|
||||
|
||||
- `analysis.py`: idea enhancement, analysis, regenerate-queries.
|
||||
- `research.py`: Exa research endpoint.
|
||||
- `script.py`: script generation and scene approval.
|
||||
- `audio.py`: audio upload, generation, combine, serving audio files.
|
||||
- `images.py`: scene image generation and image serving.
|
||||
- `video.py`: scene video generation, video listing/serving, combine videos.
|
||||
- `avatar.py`: avatar upload, avatar generation, avatar cleanup/presentability.
|
||||
- `projects.py`: create, get, update, list, delete, favorite project records.
|
||||
- `dubbing.py`: dubbing/voice clone lifecycle endpoints (currently backend-available).
|
||||
|
||||
## Data models (functional view)
|
||||
|
||||
At feature level, the flow revolves around:
|
||||
|
||||
- **Project metadata**: `project_id`, idea, duration, speakers, budget and status fields.
|
||||
- **Analysis output**: audience, content type, keywords, outlines, title suggestions.
|
||||
- **Research output**: source list, summarized insights, fact cards for script grounding.
|
||||
- **Script output**: scenes with IDs, durations, emotions, and speaker lines.
|
||||
- **Media output**: audio files, scene images, scene videos, combined episode artifacts.
|
||||
|
||||
## Operational notes
|
||||
|
||||
- Preflight checks are used to fail fast on plan/credit constraints.
|
||||
- Some operations are synchronous (analysis/script/audio/image), while video is async task-based.
|
||||
- Client-side task polling is used for long-running jobs.
|
||||
|
||||
## Engineering references
|
||||
|
||||
- `docs/Podcast_maker/AI_PODCAST_BACKEND_REFERENCE.md`
|
||||
- `docs/Podcast_maker/PODCAST_API_CALL_ANALYSIS.md`
|
||||
- `docs/Podcast_maker/PODCAST_PLAN_COMPLETION_STATUS.md`
|
||||
@@ -1,57 +0,0 @@
|
||||
# Podcast Maker Overview
|
||||
|
||||
Podcast Maker helps you turn a topic idea into a polished episode draft with research, script generation, AI voice narration, and optional video scenes.
|
||||
|
||||
## What you do in the product
|
||||
|
||||
1. **Start with an idea** and episode settings (duration, speakers, style).
|
||||
2. **Review AI analysis** suggestions (audience fit, outline ideas, titles, takeaways).
|
||||
3. **Run research** from selected queries and use source-backed fact cards.
|
||||
4. **Generate and edit a script** scene-by-scene.
|
||||
5. **Generate voice audio** for each scene and combine clips into one episode file.
|
||||
6. **Optionally create scene images and talking-head videos**.
|
||||
7. **Save and revisit projects** from your episode/project list.
|
||||
|
||||
## What you see in the UI
|
||||
|
||||
- Suggested outlines, titles, and hooks after analysis.
|
||||
- A query approval step before research runs.
|
||||
- Fact cards and summarized research insights.
|
||||
- Scene-based script editor with approval actions.
|
||||
- Audio generation controls (voice, emotion, speed, format-related options).
|
||||
- Video task progress and completed video listing.
|
||||
- Project persistence (save/load/list/favorite/delete).
|
||||
|
||||
## Feature status matrix (based on current code)
|
||||
|
||||
| Capability | Status | Notes |
|
||||
|---|---|---|
|
||||
| Idea enhancement + analysis suggestions | **Implemented** | Frontend calls `/api/podcast/idea/enhance` and `/api/podcast/analyze`; backend handlers exist. |
|
||||
| Research with Exa flow | **Implemented** | Frontend uses `/api/podcast/research/exa`; backend Exa research route is present. |
|
||||
| Script generation + scene approval | **Implemented** | Frontend uses `/api/podcast/script` and `/api/podcast/script/approve`; backend handlers exist. |
|
||||
| Scene audio generation + combine audio | **Implemented** | Frontend uses `/api/podcast/audio` and `/api/podcast/combine-audio`; backend handlers exist. |
|
||||
| Scene image generation | **Implemented** | Frontend uses `/api/podcast/image`; backend image handler exists. |
|
||||
| Scene video generation + status polling + combine videos | **Implemented** | Frontend uses `/api/podcast/render/video`, `/api/podcast/task/{id}/status`, `/api/podcast/render/combine-videos`; backend video routes are present. |
|
||||
| Project CRUD + favorites | **Implemented** | Frontend calls `/api/podcast/projects*`; backend create/get/update/list/delete/favorite routes exist. |
|
||||
| Avatar upload/generate/make-presentable | **Implemented** | Frontend calls `/api/podcast/avatar/*`; backend routes exist. |
|
||||
| Audio dubbing + voice clone routes | **Partial** | Backend dubbing routes exist; not wired in `podcastApi.ts` yet. |
|
||||
| Task cancellation from Podcast Maker UI | **Partial** | Frontend has `cancelTask()` placeholder using `/api/story/task/.../cancel`, not a dedicated podcast cancel API path. |
|
||||
| Multi-provider research toggle in podcast service | **Planned/Not active in current frontend** | Podcast frontend currently targets Exa route directly instead of a user-facing provider switch in this API layer. |
|
||||
|
||||
## Advanced / developer notes
|
||||
|
||||
Most users can ignore this section.
|
||||
|
||||
- Podcast Maker uses preflight checks before expensive operations (analysis/script/audio/research) to surface plan/credit issues early.
|
||||
- The frontend normalizes snake_case API responses into camelCase for UI components where needed.
|
||||
- Long-running video operations are task-based and polled from the client.
|
||||
|
||||
## Engineering references
|
||||
|
||||
These are internal planning/reference docs retained as source material:
|
||||
|
||||
- `docs/Podcast_maker/AI_PODCAST_BACKEND_REFERENCE.md`
|
||||
- `docs/Podcast_maker/AI_PODCAST_ENHANCEMENTS.md`
|
||||
- `docs/Podcast_maker/PODCAST_API_CALL_ANALYSIS.md`
|
||||
- `docs/Podcast_maker/PODCAST_PERSISTENCE_IMPLEMENTATION.md`
|
||||
- `docs/Podcast_maker/PODCAST_PLAN_COMPLETION_STATUS.md`
|
||||
@@ -106,13 +106,6 @@ journey
|
||||
- Set up quality control processes
|
||||
- Train team on brand standards
|
||||
|
||||
### Bonus: Team Podcast Production (75-120 minutes)
|
||||
**[Podcast Maker Journey →](podcast-maker-journey.md)**
|
||||
|
||||
- Coordinate analysis, research, script, render, and export across team roles
|
||||
- Maintain brand voice through review checkpoints
|
||||
- Publish episodes on schedule with reusable assets
|
||||
|
||||
## 🎯 Success Stories
|
||||
|
||||
### Sarah - Marketing Team Lead
|
||||
@@ -144,7 +137,6 @@ Once you've established your team workflow, explore these next steps:
|
||||
- **[Performance Analytics](performance-analytics.md)** - Track team and content performance
|
||||
- **[Client Management](client-management.md)** - Manage multiple clients efficiently
|
||||
- **[Team Scaling](team-scaling.md)** - Grow your content team
|
||||
- **[Podcast Maker Journey](podcast-maker-journey.md)** - Add a standardized podcast lane to team workflows
|
||||
|
||||
## 🔧 Technical Requirements
|
||||
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
# Podcast Maker Journey - Content Teams
|
||||
|
||||
Use this workflow to produce consistent podcast episodes across contributors while maintaining editorial quality and brand voice.
|
||||
|
||||
## Overview
|
||||
|
||||
### Entry Conditions
|
||||
- **Inputs:** Editorial brief, role assignments, brand guide, deadline.
|
||||
- **Skill level:** Mixed (editor lead + contributors).
|
||||
- **Expected time:** 75-120 minutes end-to-end for team production.
|
||||
|
||||
### Success Target
|
||||
Release a review-approved episode on schedule with clear ownership at each stage.
|
||||
|
||||
## Setup
|
||||
|
||||
### Recommended Defaults
|
||||
- **Duration:** 18-25 minutes
|
||||
- **Speakers:** Host + 1 guest (or two-host format)
|
||||
- **Voice style:** Brand-consistent, clear pacing
|
||||
- **Research provider:** Tavily (reliable source collection for editorial review)
|
||||
|
||||
### Pre-Production Checklist
|
||||
1. Assign owner for analysis, research, script QA, and publish tasks.
|
||||
2. Confirm audience persona and approved episode angle.
|
||||
3. Set shared template for intro, segment transitions, and outro.
|
||||
4. Define review SLA and escalation path.
|
||||
|
||||
## Production
|
||||
|
||||
### Podcast Maker Workflow
|
||||
1. **Analysis**
|
||||
- Align episode with editorial calendar and campaign priorities.
|
||||
- Freeze episode scope to prevent late-stage rewrites.
|
||||
2. **Research**
|
||||
- Collect and verify sources in a shared reference set.
|
||||
- Flag claims needing legal or product review.
|
||||
3. **Script**
|
||||
- Draft using team template and brand voice standards.
|
||||
- Run editor review for structure, tone, and factual accuracy.
|
||||
4. **Render**
|
||||
- Render staged draft for stakeholder sign-off.
|
||||
- Apply final edits from reviewer checklist.
|
||||
5. **Export**
|
||||
- Export audio + episode summary + channel-specific snippets.
|
||||
- Publish according to calendar and track delivery SLAs.
|
||||
|
||||
## Optimization
|
||||
|
||||
### Success Criteria
|
||||
- All approval gates pass without critical rework.
|
||||
- Episode goes live on schedule with complete metadata.
|
||||
- Style and tone match team brand guidelines.
|
||||
- Reuse assets created for social/email/web repurposing.
|
||||
|
||||
### Checkpoints
|
||||
- **Before render:** Editorial sign-off on script and claims.
|
||||
- **After render:** QA pass for pacing, names, and transitions.
|
||||
- **After publish:** Retrospective on cycle time and revision count.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues and Fixes
|
||||
- **Too many revisions:** Lock brief scope and decision owner early.
|
||||
- **Brand inconsistency:** Enforce reusable script blocks and style checks.
|
||||
- **Missed deadlines:** Add milestone gates for each workflow stage.
|
||||
- **Fact disputes:** Keep source notes attached to each script section.
|
||||
- **Inefficient handoffs:** Use a single shared checklist per episode.
|
||||
|
||||
---
|
||||
|
||||
Next step: combine this with **[Workflow Optimization](workflow-optimization.md)** to reduce cycle time.
|
||||
@@ -116,13 +116,6 @@ journey
|
||||
- Help improve documentation
|
||||
- Participate in the community
|
||||
|
||||
### Bonus: Automated Podcast Pipeline (60-120 minutes)
|
||||
**[Podcast Maker Journey →](podcast-maker-journey.md)**
|
||||
|
||||
- Implement analysis → research → script → render → export as a pipeline
|
||||
- Add schema validation, retries, and stage-level observability
|
||||
- Export artifacts with metadata for downstream integrations
|
||||
|
||||
## 🎯 Success Stories
|
||||
|
||||
### Alex - Full-Stack Developer
|
||||
@@ -154,7 +147,6 @@ Once you've completed your first integration, explore these next steps:
|
||||
- **[Production Deployment](deployment.md)** - Deploy to production
|
||||
- **[Team Collaboration](team-collaboration.md)** - Work with your team
|
||||
- **[Contributing](contributing.md)** - Contribute to ALwrity
|
||||
- **[Podcast Maker Journey](podcast-maker-journey.md)** - Build and automate podcast generation workflows
|
||||
|
||||
## 🔧 Technical Requirements
|
||||
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
# Podcast Maker Journey - Developers
|
||||
|
||||
Use this journey to integrate Podcast Maker into repeatable, testable pipelines for scripted audio generation and distribution.
|
||||
|
||||
## Overview
|
||||
|
||||
### Entry Conditions
|
||||
- **Inputs:** API credentials, topic payload schema, content constraints, output destination.
|
||||
- **Skill level:** Intermediate to advanced (API and workflow automation).
|
||||
- **Expected time:** 60-120 minutes for first implementation.
|
||||
|
||||
### Success Target
|
||||
Automate one full podcast generation path from prompt to exported artifact with predictable quality.
|
||||
|
||||
## Setup
|
||||
|
||||
### Recommended Defaults
|
||||
- **Duration:** 10-20 minutes (configurable per template)
|
||||
- **Speakers:** 1-2 synthetic speakers
|
||||
- **Voice style:** Neutral/professional with stable pacing
|
||||
- **Research provider:** Perplexity (structured fact gathering for scripted outputs)
|
||||
|
||||
### Pre-Production Checklist
|
||||
1. Define request schema for analysis/research/script/render/export stages.
|
||||
2. Store provider credentials via environment variables.
|
||||
3. Configure retry/error policy for external research and render calls.
|
||||
4. Add logging for prompt versions and output hashes.
|
||||
|
||||
## Production
|
||||
|
||||
### Podcast Maker Workflow
|
||||
1. **Analysis**
|
||||
- Validate input payload and enforce required fields.
|
||||
- Derive episode objective and section plan programmatically.
|
||||
2. **Research**
|
||||
- Fetch source context with provider abstraction.
|
||||
- Normalize citations and drop low-confidence results.
|
||||
3. **Script**
|
||||
- Generate structured script JSON (intro/segments/outro/CTA).
|
||||
- Run lint-style checks for length and forbidden terms.
|
||||
4. **Render**
|
||||
- Render audio using configured speaker profile.
|
||||
- Execute post-render QA hooks (duration, loudness, clipping checks).
|
||||
5. **Export**
|
||||
- Persist artifact + metadata to storage.
|
||||
- Trigger downstream publish/webhook integration.
|
||||
|
||||
## Optimization
|
||||
|
||||
### Success Criteria
|
||||
- End-to-end pipeline completes without manual intervention.
|
||||
- Output passes automated quality checks.
|
||||
- Metadata includes provenance for research and prompt version.
|
||||
- Failure paths are observable with actionable logs.
|
||||
|
||||
### Checkpoints
|
||||
- **Before render:** Unit/integration checks pass for script payload.
|
||||
- **After render:** Verify duration bounds and transcript alignment.
|
||||
- **After publish:** Monitor error rate, latency, and output quality metrics.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues and Fixes
|
||||
- **Provider timeouts:** Add retries with exponential backoff and fallback provider.
|
||||
- **Inconsistent scripts:** Pin model settings and enforce schema validation.
|
||||
- **Audio quality failures:** Add deterministic render settings and QA thresholds.
|
||||
- **Broken exports:** Validate storage credentials and file naming conventions.
|
||||
- **Debug difficulty:** Log stage-level inputs/outputs with correlation IDs.
|
||||
|
||||
---
|
||||
|
||||
Next step: integrate this into **[Advanced Usage](advanced-usage.md)** automation patterns.
|
||||
@@ -106,13 +106,6 @@ journey
|
||||
- Implement performance monitoring
|
||||
- Establish business impact measurement
|
||||
|
||||
### Bonus: Governed Podcast Operations (1.5-3 hours)
|
||||
**[Podcast Maker Journey →](podcast-maker-journey.md)**
|
||||
|
||||
- Run compliant episode workflows from analysis to export
|
||||
- Apply legal/compliance checkpoints before publication
|
||||
- Archive governed outputs for audit and KPI reporting
|
||||
|
||||
## 🎯 Success Stories
|
||||
|
||||
### Sarah - CMO at Fortune 500 Company
|
||||
@@ -144,7 +137,6 @@ Once you've completed your enterprise setup, explore these next steps:
|
||||
- **[Performance Optimization](performance-optimization.md)** - Optimize system performance
|
||||
- **[Custom Solutions](custom-solutions.md)** - Develop custom enterprise solutions
|
||||
- **[Strategic Planning](strategic-planning.md)** - Align content strategy with business goals
|
||||
- **[Podcast Maker Journey](podcast-maker-journey.md)** - Launch compliant, scalable enterprise podcast production
|
||||
|
||||
## 🔧 Technical Requirements
|
||||
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
# Podcast Maker Journey - Enterprise
|
||||
|
||||
Use Podcast Maker for compliant, scalable audio production aligned with governance controls and cross-functional approval requirements.
|
||||
|
||||
## Overview
|
||||
|
||||
### Entry Conditions
|
||||
- **Inputs:** Business unit brief, compliance constraints, approved messaging, KPI target.
|
||||
- **Skill level:** Advanced team workflow (marketing + legal + ops).
|
||||
- **Expected time:** 1.5-3 hours including governance review.
|
||||
|
||||
### Success Target
|
||||
Publish a compliant enterprise episode that meets brand, legal, and performance standards.
|
||||
|
||||
## Setup
|
||||
|
||||
### Recommended Defaults
|
||||
- **Duration:** 20-30 minutes
|
||||
- **Speakers:** Executive/SME host + moderator
|
||||
- **Voice style:** Professional, authoritative
|
||||
- **Research provider:** Tavily (traceable source paths for auditability)
|
||||
|
||||
### Pre-Production Checklist
|
||||
1. Map required approval stakeholders (brand, legal, compliance).
|
||||
2. Assign classification level for external statements.
|
||||
3. Define mandatory disclaimers and prohibited claims.
|
||||
4. Prepare measurement framework (pipeline, engagement, retention).
|
||||
|
||||
## Production
|
||||
|
||||
### Podcast Maker Workflow
|
||||
1. **Analysis**
|
||||
- Align episode goals with strategic initiative and audience segment.
|
||||
- Identify risk-sensitive statements before drafting.
|
||||
2. **Research**
|
||||
- Build source-backed evidence pack with references.
|
||||
- Validate data currency and claim boundaries.
|
||||
3. **Script**
|
||||
- Generate script with approved messaging blocks and disclaimers.
|
||||
- Route through legal/compliance checkpoint.
|
||||
4. **Render**
|
||||
- Render controlled draft for executive review.
|
||||
- Confirm pronunciation of product names and regulated terms.
|
||||
5. **Export**
|
||||
- Export approved audio and governed show notes.
|
||||
- Archive final assets and source references for audits.
|
||||
|
||||
## Optimization
|
||||
|
||||
### Success Criteria
|
||||
- No compliance exceptions in final published episode.
|
||||
- Approval timeline meets internal SLA.
|
||||
- Episode metadata and source references are fully archived.
|
||||
- Performance report linked to enterprise KPI dashboard.
|
||||
|
||||
### Checkpoints
|
||||
- **Before render:** Complete legal/compliance sign-off.
|
||||
- **After render:** QA for disclaimers, claims, and brand integrity.
|
||||
- **After publish:** Governance review + KPI impact check.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues and Fixes
|
||||
- **Approval bottlenecks:** Pre-approve claim libraries and disclaimer blocks.
|
||||
- **Compliance rejections:** Tag high-risk sections in analysis stage earlier.
|
||||
- **Version confusion:** Maintain a single source-of-truth script workspace.
|
||||
- **Weak executive adoption:** Provide KPI snapshots with every episode brief.
|
||||
- **Audit gaps:** Attach source and approval logs to exported package.
|
||||
|
||||
---
|
||||
|
||||
Next step: extend this with **[Security & Compliance](security-compliance.md)** controls.
|
||||
@@ -109,13 +109,6 @@ journey
|
||||
- Build your content library
|
||||
- Develop your content strategy
|
||||
|
||||
### Bonus: Launch Your Podcast (45-75 minutes)
|
||||
**[Podcast Maker Journey →](podcast-maker-journey.md)**
|
||||
|
||||
- Turn your topic into a complete podcast episode
|
||||
- Follow analysis → research → script → render → export
|
||||
- Publish polished audio without technical editing complexity
|
||||
|
||||
## 🎯 Success Stories
|
||||
|
||||
### Sarah - Lifestyle Blogger
|
||||
@@ -147,7 +140,6 @@ Once you've completed your first content creation, explore these next steps:
|
||||
- **[SEO Basics](seo-basics.md)** - Learn simple SEO techniques
|
||||
- **[Content Strategy](content-strategy.md)** - Plan your content calendar
|
||||
- **[Performance Tracking](performance-tracking.md)** - Monitor your success
|
||||
- **[Podcast Maker Journey](podcast-maker-journey.md)** - Create and publish episodes with guided defaults
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
# Podcast Maker Journey - Non-Tech Creators
|
||||
|
||||
Use this journey to go from idea to published podcast episode with minimal technical setup.
|
||||
|
||||
## Overview
|
||||
|
||||
### Entry Conditions
|
||||
- **Inputs:** Topic idea, audience goal, 3-5 talking points, optional reference links.
|
||||
- **Skill level:** Beginner (no audio editing experience required).
|
||||
- **Expected time:** 45-75 minutes for a first complete episode.
|
||||
|
||||
### Success Target
|
||||
Publish one clear, on-brand episode and reuse the workflow weekly.
|
||||
|
||||
## Setup
|
||||
|
||||
### Recommended Defaults
|
||||
- **Duration:** 8-12 minutes
|
||||
- **Speakers:** 1 host + optional 1 co-host
|
||||
- **Voice style:** Natural, friendly, medium pace
|
||||
- **Research provider:** Tavily (balanced depth + speed)
|
||||
|
||||
### Pre-Production Checklist
|
||||
1. Pick a single episode objective (teach, announce, or summarize).
|
||||
2. Set audience level (beginner/intermediate).
|
||||
3. Add 2-3 must-cover points to prevent rambling.
|
||||
4. Confirm intro/outro CTA (newsletter, site, product page).
|
||||
|
||||
## Production
|
||||
|
||||
### Podcast Maker Workflow
|
||||
1. **Analysis**
|
||||
- Define episode goal, audience pain point, and key takeaway.
|
||||
- Validate the title so listeners know the value in under 8 words.
|
||||
2. **Research**
|
||||
- Pull supporting facts/examples from trusted sources.
|
||||
- Keep only relevant references to avoid overloading the script.
|
||||
3. **Script**
|
||||
- Generate intro hook, 2-4 core segments, and concise outro CTA.
|
||||
- Add transitions between segments for natural flow.
|
||||
4. **Render**
|
||||
- Choose voice and pacing defaults.
|
||||
- Render a draft and listen for pronunciation/tone issues.
|
||||
5. **Export**
|
||||
- Export final audio (MP3) and episode notes.
|
||||
- Publish to your hosting platform and schedule promotion.
|
||||
|
||||
## Optimization
|
||||
|
||||
### Success Criteria
|
||||
- Episode stays inside target duration window.
|
||||
- Opening 30 seconds clearly states listener benefit.
|
||||
- No unresolved placeholders/fact checks in final script.
|
||||
- Export includes title, description, and CTA.
|
||||
|
||||
### Checkpoints
|
||||
- **Before render:** Read script out loud once for clarity.
|
||||
- **After render:** Spot-check intro, midpoint transition, and outro.
|
||||
- **After publish:** Track listens, retention, and CTA clicks.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues and Fixes
|
||||
- **Output sounds robotic:** Switch to a warmer voice profile and reduce script complexity.
|
||||
- **Episode too long:** Cut to one primary theme and remove secondary tangents.
|
||||
- **Weak structure:** Rebuild around hook → problem → solution → CTA.
|
||||
- **Research overload:** Limit references to top 3 sources relevant to the audience.
|
||||
- **Low engagement:** Strengthen title and first 20 seconds with a sharper promise.
|
||||
|
||||
---
|
||||
|
||||
Next step: pair this with **[Content Optimization](content-optimization.md)** to improve discoverability and repeatability.
|
||||
@@ -106,13 +106,6 @@ journey
|
||||
- Leverage social media for growth
|
||||
- Convert followers into customers
|
||||
|
||||
### Bonus: Authority Podcast Workflow (50-90 minutes)
|
||||
**[Podcast Maker Journey →](podcast-maker-journey.md)**
|
||||
|
||||
- Produce authority-building episodes that support your offer
|
||||
- Run analysis → research → script → render → export in one flow
|
||||
- Add clear CTA and repurposing outputs for social and email
|
||||
|
||||
## 🎯 Success Stories
|
||||
|
||||
### Sarah - Business Coach
|
||||
@@ -144,7 +137,6 @@ Once you've established your foundation, explore these next steps:
|
||||
- **[Content Monetization](content-monetization.md)** - Turn your content into revenue
|
||||
- **[Community Building](community-building.md)** - Build a loyal following
|
||||
- **[Business Growth](business-growth.md)** - Scale your solopreneur business
|
||||
- **[Podcast Maker Journey](podcast-maker-journey.md)** - Build repeatable podcast episodes for lead generation
|
||||
|
||||
## 🔧 Technical Requirements
|
||||
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
# Podcast Maker Journey - Solopreneurs
|
||||
|
||||
Use Podcast Maker to produce authority-building episodes that generate leads without adding a full production team.
|
||||
|
||||
## Overview
|
||||
|
||||
### Entry Conditions
|
||||
- **Inputs:** Offer/theme, ICP (ideal customer profile), episode angle, optional proof points.
|
||||
- **Skill level:** Beginner to intermediate.
|
||||
- **Expected time:** 50-90 minutes per episode (including positioning).
|
||||
|
||||
### Success Target
|
||||
Ship one episode that strengthens personal brand positioning and drives one business CTA.
|
||||
|
||||
## Setup
|
||||
|
||||
### Recommended Defaults
|
||||
- **Duration:** 12-18 minutes
|
||||
- **Speakers:** Solo host (or founder + guest)
|
||||
- **Voice style:** Confident, conversational
|
||||
- **Research provider:** Perplexity (fast market context and trend summaries)
|
||||
|
||||
### Pre-Production Checklist
|
||||
1. Align episode topic with one content pillar.
|
||||
2. Define one conversion CTA (call booking, newsletter, lead magnet).
|
||||
3. Capture one personal story/case insight.
|
||||
4. Set repurposing targets (LinkedIn post, email, short clips).
|
||||
|
||||
## Production
|
||||
|
||||
### Podcast Maker Workflow
|
||||
1. **Analysis**
|
||||
- Clarify business objective (awareness, trust, or conversion).
|
||||
- Frame the episode around one audience pain + practical outcome.
|
||||
2. **Research**
|
||||
- Gather market stats, examples, or competitor framing.
|
||||
- Keep only proof points that support your positioning.
|
||||
3. **Script**
|
||||
- Build authority arc: context → method → example → next step.
|
||||
- Insert 1-2 short personal credibility stories.
|
||||
4. **Render**
|
||||
- Render a preview for tone fit and confidence level.
|
||||
- Adjust emphasis on key offers/CTAs.
|
||||
5. **Export**
|
||||
- Export audio + show notes + CTA links.
|
||||
- Queue repurposing assets for social and email distribution.
|
||||
|
||||
## Optimization
|
||||
|
||||
### Success Criteria
|
||||
- Core message and offer are clear by minute 3.
|
||||
- One clear CTA appears in both script and show notes.
|
||||
- Episode maps to at least two downstream channels.
|
||||
- Audio pacing remains consistent throughout.
|
||||
|
||||
### Checkpoints
|
||||
- **Before render:** Confirm episode supports current business campaign.
|
||||
- **After render:** Verify name, offer, and links are pronounced correctly.
|
||||
- **After publish:** Review lead quality and conversion from podcast traffic.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues and Fixes
|
||||
- **No business impact:** Move CTA earlier and repeat it once near close.
|
||||
- **Episode feels generic:** Add one client case, lesson, or contrarian insight.
|
||||
- **Inconsistent voice:** Save a reusable script template and voice profile.
|
||||
- **Slow production:** Batch analysis/research for 3-4 episodes at once.
|
||||
- **Low retention:** Tighten intro and cut non-essential setup commentary.
|
||||
|
||||
---
|
||||
|
||||
Next step: connect this flow with **[Content Monetization](content-monetization.md)** for stronger revenue outcomes.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user