ALwrity/docs/STORY_WRITER_REVIEW_AND_NEXT_STEPS.md

Story Writer Backend Migration - Review & Next Steps

✅ What Was Accomplished

1. Backend Service Layer (backend/services/story_writer/)

Status: ✅ Complete

• story_service.py - Core story generation service
  • Migrated from ToBeMigrated/ai_writers/ai_story_writer/ai_story_generator.py
  • Updated imports to use services.llm_providers.main_text_generation
  • Added user_id parameter for subscription integration
  • Removed Streamlit dependencies
  • Modular methods:
    • generate_premise() - Generate story premise
    • generate_outline() - Generate story outline
    • generate_story_start() - Generate story beginning
    • continue_story() - Continue story generation
    • generate_full_story() - Complete story generation with iterations

Key Features:

• ✅ Subscription support via main_text_generation
• ✅ Supports both Gemini and HuggingFace providers
• ✅ Proper error handling with HTTPException support
• ✅ Comprehensive logging

2. API Layer (backend/api/story_writer/)

Status: ✅ Complete

• router.py - RESTful API endpoints

  • Synchronous endpoints: premise, outline, start, continue
  • Asynchronous endpoint: full story generation with task management
  • Task status and result endpoints
  • Cache management endpoints
  • Health check endpoint

• task_manager.py - Async task execution

  • Background task execution
  • Progress tracking (0-100%)
  • Status management (pending, processing, completed, failed)
  • Automatic cleanup of old tasks

• cache_manager.py - Result caching

  • MD5-based cache key generation
  • Cache statistics
  • Cache clearing

3. Models (backend/models/story_models.py)

Status: ✅ Complete

• Pydantic models for type-safe API:
  • StoryGenerationRequest - Input parameters
  • StoryPremiseResponse - Premise generation response
  • StoryOutlineResponse - Outline generation response
  • StoryContentResponse - Story content response
  • StoryFullGenerationResponse - Complete story response
  • StoryContinueRequest/Response - Continuation models
  • TaskStatus - Task tracking model

4. Router Registration

Status: ✅ Complete

• Added to alwrity_utils/router_manager.py in optional routers section
• Automatic registration on app startup
• Error handling for graceful failures

📊 API Endpoints Summary

Synchronous Endpoints

POST /api/story/generate-premise
POST /api/story/generate-outline
POST /api/story/generate-start
POST /api/story/continue

Asynchronous Endpoints

POST /api/story/generate-full        → Returns task_id
GET  /api/story/task/{task_id}/status
GET  /api/story/task/{task_id}/result

Utility Endpoints

GET  /api/story/health
GET  /api/story/cache/stats
POST /api/story/cache/clear

🎯 Next Steps - Implementation Roadmap

Phase 1: Backend Testing & Validation (Priority: High)

Estimated Time: 1-2 days

Tasks:

1. API Testing

   • Test all synchronous endpoints with Postman/curl
   • Test async task flow (generate-full → status → result)
   • Verify subscription limits work (429 errors)
   • Test with both Gemini and HuggingFace providers
   • Test error handling (invalid inputs, API failures)

2. Integration Testing

   • Test with real user authentication
   • Verify usage tracking in database
   • Test cache functionality
   • Test task cleanup (old tasks removal)

3. Performance Testing

   • Measure response times for each endpoint
   • Test concurrent requests
   • Monitor memory usage during long story generation

Deliverables:

• API test suite (Postman collection or pytest)
• Test results document
• Performance benchmarks

---

Phase 2: Frontend Foundation (Priority: High)

Estimated Time: 2-3 days

Tasks:

1. Create Frontend Structure

   • Create frontend/src/components/StoryWriter/ directory
   • Create frontend/src/services/storyWriterApi.ts (API client)
   • Create frontend/src/hooks/useStoryWriterState.ts (state management)
   • Create frontend/src/hooks/useStoryWriterPhaseNavigation.ts (phase navigation)

2. API Service Layer

   // frontend/src/services/storyWriterApi.ts
   - generatePremise()
   - generateOutline()
   - generateStoryStart()
   - continueStory()
   - generateFullStory() // async with polling
   - getTaskStatus()
   - getTaskResult()
   

3. State Management Hook

   // frontend/src/hooks/useStoryWriterState.ts
   - Story parameters (persona, setting, characters, etc.)
   - Premise, outline, story content
   - Generation progress
   - Task management
   

4. Phase Navigation Hook

   // Similar to usePhaseNavigation.ts from Blog Writer
   Phases: Setup → Premise → Outline → Writing → Export
   

Deliverables:

• Frontend directory structure
• API service with TypeScript types
• State management hooks
• Phase navigation hook

---

Phase 3: UI Components - Core (Priority: High)

Estimated Time: 3-4 days

Tasks:

1. Main Component

   • StoryWriter.tsx - Main container component
   • Similar structure to BlogWriter.tsx

2. Phase Components

   • StorySetup.tsx - Phase 1: Input story parameters

     • Persona selector (11 options)
     • Story setting input
     • Characters input
     • Plot elements input
     • Writing style, tone, POV selectors
     • Audience age group, content rating, ending preference

   • StoryPremise.tsx - Phase 2: Review premise

     • Display generated premise
     • Regenerate option
     • Continue to outline button

   • StoryOutline.tsx - Phase 3: Review outline

     • Display generated outline
     • Edit/refine option
     • Continue to writing button

   • StoryContent.tsx - Phase 4: Generated story

     • Display story content
     • Markdown editor for editing
     • Continue generation button
     • Progress indicator for async generation

   • StoryExport.tsx - Phase 5: Export options

     • Download as text/markdown
     • Copy to clipboard
     • Share options

3. Utility Components

   • HeaderBar.tsx - Phase navigation header (like Blog Writer)
   • PhaseContent.tsx - Phase content wrapper
   • TaskProgressModal.tsx - Progress modal for async operations

Deliverables:

• All phase components
• Main StoryWriter component
• Utility components

---

Phase 4: CopilotKit Integration (Priority: Medium)

Estimated Time: 2-3 days

Tasks:

1. CopilotKit Actions

   • useStoryWriterCopilotActions.ts hook
   • Actions:
     • generateStoryPremise - Generate premise
     • generateStoryOutline - Generate outline
     • startStoryWriting - Begin story generation
     • continueStoryWriting - Continue story
     • refineStoryOutline - Refine outline
     • exportStory - Export story

2. CopilotKit Sidebar

   • WriterCopilotSidebar.tsx - Suggestions sidebar
   • Context-aware suggestions based on current phase
   • Action buttons for common tasks

3. Integration

   • Register actions in StoryWriter component
   • Connect sidebar to component state
   • Test CopilotKit interactions

Reference: frontend/src/components/BlogWriter/BlogWriterUtils/useBlogWriterCopilotActions.ts

Deliverables:

• CopilotKit actions hook
• CopilotKit sidebar component
• Integrated with main component

---

Phase 5: Polish & Enhancement (Priority: Low)

Estimated Time: 2-3 days

Tasks:

1. Error Handling

   • User-friendly error messages
   • Retry mechanisms
   • Error boundaries

2. Loading States

   • Skeleton loaders
   • Progress indicators
   • Optimistic UI updates

3. UX Improvements

   • Keyboard shortcuts
   • Auto-save draft
   • Undo/redo functionality
   • Story preview

4. Styling

   • Match Blog Writer design system
   • Responsive design
   • Dark mode support (if applicable)

Deliverables:

• Polished UI/UX
• Error handling improvements
• Loading states

---

Phase 6: Illustration Support (Optional - Future)

Estimated Time: 3-4 days

Tasks:

1. Backend Migration

   • Migrate story_illustrator.py to backend service
   • Create illustration API endpoints
   • Integrate with image generation API

2. Frontend Integration

   • Add illustration phase
   • Illustration generation UI
   • Preview and download illustrations

Note: Defer to Phase 2 if core story generation is priority

---

🚀 Quick Start Guide

Testing Backend API

# Health check
curl http://localhost:8000/api/story/health

# Generate premise (requires auth token)
curl -X POST http://localhost:8000/api/story/generate-premise \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "persona": "Award-Winning Science Fiction Author",
    "story_setting": "A futuristic city in 2150",
    "character_input": "John, a brave explorer",
    "plot_elements": "The hero's journey",
    "writing_style": "Formal",
    "story_tone": "Suspenseful",
    "narrative_pov": "Third Person Limited",
    "audience_age_group": "Adults",
    "content_rating": "PG-13",
    "ending_preference": "Happy"
  }'

Frontend Development Order

1. Start with API Service (storyWriterApi.ts)

   • Define all API calls
   • Add TypeScript types
   • Test with mock data

2. Build State Management (useStoryWriterState.ts)

   • Define state structure
   • Add state setters/getters
   • Test state updates

3. Create Phase Navigation (useStoryWriterPhaseNavigation.ts)

   • Define phases
   • Add navigation logic
   • Test phase transitions

4. Build Components (Start with Setup phase)

   • StorySetup component
   • Test form submission
   • Connect to API

5. Add Remaining Phases

   • Premise → Outline → Writing → Export
   • Test each phase independently

6. Integrate CopilotKit

   • Add actions
   • Connect sidebar
   • Test interactions

---

📝 Key Decisions Made

1. Modular Structure: Follows Blog Writer patterns for consistency
2. Async Task Pattern: Long-running operations use task management with polling
3. Subscription Integration: Automatic via main_text_generation
4. Provider Support: Works with both Gemini and HuggingFace automatically
5. Caching: Results cached to avoid duplicate generations
6. Error Handling: Comprehensive with HTTPException support

---

⚠️ Important Notes

1. Authentication Required: All endpoints require valid Clerk authentication token
2. Subscription Limits: Will return 429 if limits exceeded
3. Long Operations: Full story generation can take several minutes - use async pattern
4. Task Cleanup: Tasks older than 1 hour are automatically cleaned up
5. Cache Keys: Based on request parameters - identical requests return cached results

---

🎯 Recommended Immediate Next Steps

1. Test Backend API (Today)

   • Verify all endpoints work
   • Test subscription integration
   • Document any issues

2. Create Frontend API Service (Day 1-2)

   • Set up TypeScript types
   • Create API client functions
   • Test with Postman/curl responses

3. Build StorySetup Component (Day 2-3)

   • Create form with all parameters
   • Connect to API
   • Test premise generation

4. Add Phase Navigation (Day 3-4)

   • Implement phase hook
   • Add HeaderBar component
   • Test phase transitions

5. Complete Remaining Phases (Day 4-7)

   • Build each phase component
   • Connect to API
   • Test full flow

---

📚 Reference Files

• Blog Writer (Reference implementation):

  • frontend/src/components/BlogWriter/BlogWriter.tsx
  • frontend/src/hooks/usePhaseNavigation.ts
  • frontend/src/components/BlogWriter/BlogWriterUtils/useBlogWriterCopilotActions.ts

• Backend Patterns:

  • backend/api/blog_writer/router.py
  • backend/api/blog_writer/task_manager.py
  • backend/services/blog_writer/blog_service.py

---

✅ Success Criteria

• All backend endpoints tested and working
• Frontend API service complete
• All phase components built
• Phase navigation working
• CopilotKit integrated
• Full story generation flow works end-to-end
• Error handling comprehensive
• Loading states implemented
• UI matches Blog Writer design

---

Ready to proceed with Phase 1 (Backend Testing) or Phase 2 (Frontend Foundation)?