feat: Implement Today's Workflow and Agent Huddle enhancements

docs/SIF/SEO_DASHBOARD_INSIGHTS.md

@@ -0,0 +1,81 @@
+# SIF SEO Dashboard Insights
+
+**Last Updated**: 2025-03-01
+**Component**: Semantic Intelligence Dashboard (Frontend/Backend)
+
+---
+
+## 🔍 Overview
+
+The **SEO Dashboard** is the user's window into the Semantic Intelligence Framework (SIF). It visualizes the data stored in the `txtai` vector index, translating complex semantic relationships into actionable marketing insights.
+
+Unlike traditional SEO tools that rely on keyword volume, SIF analyzes **topical authority** and **semantic distance**.
+
+---
+
+## 🏗️ Data Flow
+
+```mermaid
+graph LR
+    A[Raw Data] -->|Indexing| B[(SIF Vector Index)]
+    B -->|Query| C[RealTimeSemanticMonitor]
+    C -->|Analyze| D[SIF Agents]
+    D -->|Tag| E[ContentSemanticInsight]
+    E -->|API| F[Frontend Dashboard]
+```
+
+---
+
+## 📊 Key Insight Modules
+
+### 1. Semantic Health Score
+*   **What it is**: A 0-100 score representing how well the user's content covers their target niche compared to competitors.
+*   **Calculation**:
+    *   `Topic Coverage`: % of core industry topics present in user index.
+    *   `Content Freshness`: Recency of indexed documents.
+    *   `Competitor Overlap`: Semantic similarity score vs. top competitors.
+
+### 2. Content Pillars (The Strategy)
+*   **Visual**: Cards showing core themes (e.g., "AI Marketing", "SEO Tools").
+*   **Agent**: **Strategy Architect**.
+*   **Logic**:
+    1.  `txtai` clusters all user content.
+    2.  Clusters with >5 documents become "Pillars".
+    3.  Relevance score is calculated based on cluster density.
+
+### 3. Semantic Gaps (The Opportunity)
+*   **Visual**: Accordion list of missing topics.
+*   **Agent**: **Content Strategist**.
+*   **Logic**:
+    1.  Compare User Vector Space vs. Competitor Vector Space.
+    2.  Identify dense clusters in Competitor space that are empty in User space.
+    3.  Flag these as "Gaps" (e.g., "Competitors write about 'Voice Search', you don't").
+
+### 4. AI Insights (The Action)
+*   **Visual**: A feed of prioritized recommendations.
+*   **Agents involved**: All.
+*   **Types**:
+    *   **Trend**: "Interest in 'Vector Database' is rising." (Source: Content Strategist)
+    *   **Optimization**: "Low CTR on 'Pricing' page." (Source: SEO Specialist)
+    *   **Threat**: "Competitor X launched a new guide." (Source: Competitor Analyst)
+
+---
+
+## 🕵️ Agent Attribution
+
+To build trust, every insight in the dashboard is attributed to a specific AI agent:
+
+*   **"Identified by Strategy Architect"**: Found a structural issue.
+*   **"Spotted by Content Strategist"**: Found a creative opportunity.
+*   **"Flagged by SEO Specialist"**: Found a technical error.
+
+This connects the dashboard back to the "Team" concept introduced during onboarding.
+
+---
+
+## 🔄 Real-Time Monitoring
+
+The `RealTimeSemanticMonitor` service runs periodically (default: daily or on-demand).
+1.  **Polls SIF**: Checks for new indexed documents.
+2.  **Runs Agents**: Executes agent logic against the fresh index.
+3.  **Generates Alerts**: If a critical threshold is breached (e.g., Health < 50%), it sends a system notification.

docs/SIF/SIF_AGENTS_TEAM_ARCHITECTURE.md

@@ -0,0 +1,121 @@
+# SIF AI Agents Team - Architecture & Capabilities
+
+**Last Updated**: 2025-03-01
+**Component**: Semantic Intelligence Framework (SIF) Agents
+
+---
+
+## 🧠 Executive Summary
+
+The **SIF Agents Team** is a multi-agent system built on top of the Semantic Intelligence Framework (SIF). Unlike generic AI assistants, these agents are "grounded" in a shared semantic index (`txtai`) containing the user's content, competitor data, and search console metrics.
+
+Each agent acts as a specialized "Department Head," continuously monitoring the index to surface insights, propose tasks, and execute workflows autonomously.
+
+---
+
+## 🏗️ Architecture
+
+### The "Committee" Model
+Instead of a single "God Mode" AI, we use a committee of specialized agents orchestrated by a central Manager.
+
+```mermaid
+graph TD
+    A[User / Dashboard] -->|Requests| B(Orchestrator)
+    B -->|Delegates| C[Strategy Architect]
+    B -->|Delegates| D[Content Strategist]
+    B -->|Delegates| E[SEO Specialist]
+    B -->|Delegates| F[Social Manager]
+    B -->|Delegates| G[Competitor Analyst]
+    
+    C & D & E & F & G -->|Reads/Writes| H[(SIF Semantic Index)]
+    H -->|Syncs| I[Google Search Console]
+    H -->|Syncs| J[Competitor Content]
+```
+
+### Shared Brain (SIF Index)
+All agents share the same memory (the SIF Index). 
+- **Example**: If the *Competitor Analyst* indexes a new rival blog post, the *Content Strategist* immediately sees it as a "Content Gap" without needing a manual update.
+
+---
+
+## 🤖 The Agent Roster
+
+### 1. Strategy Architect Agent (Lead)
+*   **Role**: The "VP of Content." Responsible for high-level direction.
+*   **Key Capabilities**:
+    *   **Pillar Discovery**: Clusters content to find de-facto pillars.
+    *   **Strategy Health**: Warns when content deviates from core goals.
+    *   **Planning**: Proposes quarterly themes based on performance.
+*   **SIF Integration**: Queries `txtai` for cluster density and topic coherence.
+
+### 2. Content Strategist Agent (Creative)
+*   **Role**: The "Editor-in-Chief." Focuses on what to write next.
+*   **Key Capabilities**:
+    *   **Gap Analysis**: Identifies topics competitors cover but you don't.
+    *   **Trend Spotting**: Detects rising keywords in the industry.
+    *   **Brief Generation**: Creates detailed outlines for writers.
+*   **SIF Integration**: Compares user vector space vs. competitor vector space.
+
+### 3. SEO Specialist Agent (Technical)
+*   **Role**: The "Technical SEO." Ensures visibility and health.
+*   **Key Capabilities**:
+    *   **Rank Monitoring**: Watches SERP movements for key pages.
+    *   **Health Checks**: Flags 404s, slow pages, or missing meta tags.
+    *   **Opportunity Finding**: "Low hanging fruit" (e.g., high impression, low CTR).
+*   **SIF Integration**: Analyzes GSC performance data mapped to content embeddings.
+
+### 4. Social Manager Agent (Engagement)
+*   **Role**: The "Social Media Manager." Handles distribution and community.
+*   **Key Capabilities**:
+    *   **Repurposing**: Turns blog posts into LinkedIn threads/Tweets.
+    *   **Schedule Optimization**: Predicts best times to post.
+    *   **Engagement**: Drafts replies to high-value comments.
+*   **SIF Integration**: Matches social trends to existing content library.
+
+### 5. Competitor Analyst Agent (Intelligence)
+*   **Role**: The "Spy." Watches the market 24/7.
+*   **Key Capabilities**:
+    *   **Change Detection**: Alerts when a competitor updates their pricing or homepage.
+    *   **Counter-Strategy**: Suggests moves to block competitor launches.
+*   **SIF Integration**: Continuously indexes competitor sitemaps into the shared brain.
+
+---
+
+## 🛠️ Technical Implementation
+
+### Base Agent Interface
+All agents inherit from `BaseALwrityAgent` and implement standard methods:
+```python
+class SpecializedAgent(BaseALwrityAgent):
+    async def propose_daily_tasks(self, context) -> List[TaskProposal]:
+        # Domain specific logic
+        pass
+
+    async def analyze_sif_data(self, query) -> Dict[str, Any]:
+        # Semantic search logic
+        pass
+```
+
+### Task Proposal Protocol
+Agents don't just "chat"; they submit structured `TaskProposal` objects:
+- **Title**: Actionable name.
+- **Priority**: High/Medium/Low.
+- **Reasoning**: "Why?" (e.g., "Because competitor X did Y").
+- **Source**: Agent Name (displayed in UI).
+
+---
+
+## 📊 UI Visibility
+
+The agents are visible to the user in three key areas:
+1.  **Team Huddle Widget**: Real-time status (Active/Thinking) in the Main Dashboard.
+2.  **Today's Tasks**: Each task card shows the agent's badge and reasoning.
+3.  **SEO Dashboard**: Insights are tagged with "Identified by [Agent Name]".
+
+---
+
+## 🚀 Future Roadmap
+
+*   **Inter-Agent Chat**: Allow agents to debate strategy (e.g., SEO Agent vs. Creative Agent).
+*   **Auto-Execution**: Allow agents to *perform* tasks (e.g., fix a broken link) with user approval.
+*   **Voice Interface**: Daily standup meeting via voice.

docs/SIF/TODAYS_TASKS_WORKFLOW.md

@@ -0,0 +1,142 @@
+# Multi-Agent Today's Tasks System - Implementation Plan
+
+**Date**: 2025-03-01
+**Status**: Architecture Plan
+**Target System**: Today's Tasks Workflow (Multi-Agent Committee)
+
+---
+
+## 📋 Executive Summary
+
+This document outlines the implementation plan for transforming the "Today's Tasks" system from a single-prompt generator into a **Multi-Agent "Committee" Architecture**. 
+
+Instead of a generic LLM generating tasks, we will leverage our existing specialized agents (`StrategyArchitect`, `ContentStrategist`, `SEOOptimization`, etc.) to **propose high-value, context-aware tasks** based on their specific domain knowledge. A central "Manager" (Orchestrator) will then consolidate, prioritize, and deduplicate these proposals into a cohesive daily plan.
+
+We will also introduce a **Self-Learning Task Memory** using `txtai` to ensure the system learns from user behavior (acceptances/rejections) and avoids redundant suggestions.
+
+---
+
+## 🏗️ Architecture: The "Committee" Model
+
+### 1. Agent Roles & Responsibilities
+
+Each agent will act as a "Department Head," submitting daily proposals for their specific pillar.
+
+| Workflow Pillar | Owner Agent | Data Sources | Proposal Type Example |
+| :--- | :--- | :--- | :--- |
+| **PLAN** | `StrategyArchitectAgent` | Content Pillars, Strategy Doc | "Review 'AI Trends' pillar strategy - engagement dropped 10%." |
+| **GENERATE** | `ContentStrategyAgent` | Content Gaps, Trends | "Draft a blog post on 'Vector Search' (High Opportunity Gap)." |
+| **PUBLISH** | `SocialAmplificationAgent` | Audience Activity, Calendar | "Schedule your 'Weekly Recap' thread for 10 AM (Peak Audience)." |
+| **ANALYZE** | `SEOOptimizationAgent` | GSC, Site Health, Rankings | "Fix 3 broken links on your pricing page to recover link equity." |
+| **ENGAGE** | `SocialAmplificationAgent` | Social Mentions, Comments | "Reply to 3 unanswered comments on your latest LinkedIn post." |
+| **REMARKET** | `CompetitorResponseAgent` | Competitor Activity | "Competitor X posted about [Topic]. Create a counter-narrative Reel." |
+
+### 2. The Workflow (Daily Cycle)
+
+1.  **Morning Briefing (Parallel)**: `TodayWorkflowManager` polls all agents via `propose_daily_tasks(context)`.
+2.  **Aggregation**: Manager collects raw proposals (~10-15 tasks).
+3.  **Intelligence Filter (Self-Learning)**: 
+    *   Check `TaskMemoryIndex` (txtai).
+    *   Filter out tasks similar to previously **Rejected** tasks.
+    *   Deprioritize tasks similar to recently **Completed** tasks.
+4.  **Consolidation**: Deduplicate overlapping ideas (e.g., SEO & Content agents both suggesting the same topic).
+5.  **Final Selection**: Select top 1-3 tasks per pillar based on user goals (e.g., "Growth" mode = more Publish/Remarket tasks).
+
+---
+
+## 🚀 Implementation Phases
+
+### Phase 1: Agent Interface Standardization (The "Voice")
+**Objective**: Give every agent the ability to speak the "Task Proposal" language.
+**Status**: ✅ Completed
+
+*   **Task 1.1**: Define `TaskProposal` schema (Pydantic model). ✅
+    *   Fields: `title`, `description`, `pillar`, `priority`, `reasoning`, `estimated_time`, `action_type`.
+*   **Task 1.2**: Update `BaseALwrityAgent` with abstract `propose_daily_tasks(context: Dict) -> List[TaskProposal]`. ✅
+*   **Task 1.3**: Implement `propose_daily_tasks` in all specialized agents. ✅
+    *   *StrategyArchitect*: Logic to check pillar health.
+    *   *ContentStrategist*: Logic to check content gaps.
+    *   *SEOAgent*: Logic to check GSC alerts/errors.
+    *   *SocialAmplification*: Logic for publish/engage.
+    *   *CompetitorResponse*: Logic for monitoring.
+
+### Phase 2: The Manager (The "Orchestrator")
+**Objective**: Build the backend service that coordinates the committee.
+**Status**: ✅ Completed
+
+*   **Task 2.1**: Refactor `TodayWorkflowGenerator` in `today_workflow_service.py`. ✅
+    *   Replace single-prompt generation with `gather_agent_proposals()`.
+    *   Implement `asyncio.gather` for parallel agent execution (performance critical).
+*   **Task 2.2**: Implement `consolidate_proposals()` logic. ✅
+    *   Use a lightweight LLM call to merge/rank the raw list if needed, or deterministic logic for speed.
+*   **Task 2.3**: Connect to Frontend. ✅
+    *   Ensure the API response matches the existing `TodayTask` frontend interface.
+
+### Phase 3: Self-Learning Memory (The "Brain")
+**Objective**: Stop the system from nagging users about things they hate or just did.
+**Status**: ✅ Completed
+
+*   **Task 3.1**: Create `TaskHistory` model in DB. ✅
+    *   Store: `task_vector_id`, `original_text`, `status` (completed/rejected/skipped), `user_feedback`.
+*   **Task 3.2**: Implement `TaskMemoryService` using `txtai`. ✅
+    *   Index tasks with metadata.
+    *   Implement `is_redundant_or_rejected(proposal_text)` check.
+*   **Task 3.3**: Wire feedback loop. ✅
+    *   When user clicks "Dismiss" or "Complete" in frontend, update the `txtai` index.
+
+### Phase 4: UI Feedback & Transparency
+**Objective**: Show the user *why* a task was suggested.
+
+*   **Task 4.1**: Update Frontend `TodayTask` card.
+    *   Add "Suggested by [Agent Name]" badge.
+    *   Add "Why?" tooltip (e.g., "Because Competitor X did Y").
+*   **Task 4.2**: Add "Train my Agents" feedback.
+    *   "Don't show this again" vs "Not today".
+
+---
+
+## 📊 Data Models
+
+### 1. TaskProposal (Backend)
+```python
+class TaskProposal(BaseModel):
+    title: str
+    description: str
+    pillar_id: str  # plan, generate, publish, analyze, engage, remarket
+    priority: str   # high, medium, low
+    estimated_time: int # minutes
+    source_agent: str # e.g., "SEOOptimizationAgent"
+    reasoning: str # "Detected 404 error spike"
+    context_data: Optional[Dict] # e.g., {"url": "..."}
+```
+
+### 2. TaskMemoryDocument (txtai)
+```json
+{
+  "id": "uuid",
+  "text": "Write a blog post about AI Trends",
+  "embedding": [vector],
+  "tags": ["generate", "content_strategy"],
+  "user_id": "123",
+  "status": "rejected",
+  "last_updated": "2024-03-01T10:00:00Z"
+}
+```
+
+---
+
+## 🛠️ Technical Considerations
+
+*   **Performance**: Calling 6 agents + LLMs in parallel can be slow.
+    *   *Mitigation*: Set strict timeouts (e.g., 5s) per agent. Use "Lite" logic for proposals (e.g., check DB/Cache instead of live crawling) where possible.
+*   **Fallback**: If agents time out or fail, fall back to the `_fallback_tasks` template currently in place.
+*   **Token Usage**: Summarize context before sending to agents to minimize input tokens.
+
+---
+
+## 📅 Execution Timeline
+
+1.  **Day 1**: Phase 1 (Interfaces & 2 core agents).
+2.  **Day 2**: Phase 2 (Orchestrator wiring).
+3.  **Day 3**: Phase 3 (txtai Memory integration).
+4.  **Day 4**: Phase 1 completion (remaining agents) & Phase 4 (UI polish).

docs/SIF/TODAYS_TASKS_WORKFLOW_DOCS.md

@@ -0,0 +1,109 @@
+# Multi-Agent Today's Tasks Workflow
+
+**Last Updated**: 2025-03-01
+**Component**: Today's Workflow Service
+
+---
+
+## 📅 Overview
+
+The **Today's Tasks Workflow** is an automated, intelligent system that generates a personalized daily to-do list for the user. Unlike static templates or generic AI prompts, this system uses a **Multi-Agent Committee** to analyze real-time data and propose high-value actions.
+
+## 🏗️ Architecture: The "Committee" Model
+
+The workflow follows a **Manager-Worker** pattern where the `TodayWorkflowGenerator` acts as the Orchestrator.
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant Orchestrator
+    participant Agents (Committee)
+    participant Memory (SIF)
+    
+    User->>Orchestrator: Loads Dashboard
+    Orchestrator->>Orchestrator: Checks for existing plan
+    alt No Plan for Today
+        Orchestrator->>Agents: "Propose tasks for [User Context]"
+        par Parallel Execution
+            Agents->>Agents: Analyze GSC, Trends, Gaps
+        end
+        Agents-->>Orchestrator: [Task Proposals]
+        Orchestrator->>Memory: Filter Redundant/Rejected?
+        Memory-->>Orchestrator: Filtered List
+        Orchestrator->>Orchestrator: Consolidate & Prioritize
+        Orchestrator->>User: Daily Plan
+    else Plan Exists
+        Orchestrator->>User: Existing Plan
+    end
+```
+
+---
+
+## 🧠 The Intelligence Layer
+
+### 1. Proposal Phase (The "Workers")
+Each agent submits proposals based on its domain:
+
+| Agent | Data Source | Sample Proposal |
+| :--- | :--- | :--- |
+| **Strategy Architect** | Content Pillars | "Review 'AI Trends' pillar - performance dropped 10%." |
+| **Content Strategist** | Competitor Content | "Draft post on 'Vector Search' (Competitor Gap)." |
+| **SEO Specialist** | Search Console | "Fix 404 error on /pricing page." |
+| **Social Manager** | Engagement Metrics | "Reply to 3 comments on LinkedIn post." |
+| **Competitor Analyst** | Market Signals | "Competitor X launched feature Y. Monitor impact." |
+
+### 2. Orchestration Phase (The "Manager")
+The `TodayWorkflowGenerator`:
+1.  **Gathers**: Collects all proposals via `asyncio.gather`.
+2.  **Deduplicates**: Merges similar tasks (e.g., if SEO and Content agents both suggest the same blog update).
+3.  **Formats**: Converts raw proposals into the frontend-ready `TodayTask` schema.
+
+### 3. Self-Learning Phase (The "Brain")
+The system uses `TaskMemoryService` and `txtai` to improve over time.
+-   **Rejected Tasks**: If a user dismisses a task, it is indexed as "negative feedback." The system semantically checks future proposals against this index to avoid nagging.
+-   **Completed Tasks**: Completed tasks are recorded to prevent suggesting the same non-recurring task too soon.
+
+---
+
+## 🛠️ Data Models
+
+### TaskProposal (Internal)
+```python
+@dataclass
+class TaskProposal:
+    title: str
+    description: str
+    pillar_id: str          # plan, generate, publish, analyze, engage, remarket
+    priority: str           # high, medium, low
+    estimated_time: int     # minutes
+    source_agent: str       # e.g., "SEOOptimizationAgent"
+    reasoning: str          # "Detected 404 error spike"
+    context_data: Dict      # Payload for the action button
+```
+
+### TaskHistory (Database)
+Tracks the lifecycle for learning:
+-   `task_hash`: SHA-256 of title+desc for fast deduplication.
+-   `status`: completed / dismissed.
+-   `feedback`: User provided notes.
+-   `vector_id`: Link to the semantic index entry.
+
+---
+
+## 🎨 UI Experience
+
+1.  **The Card**: Each task appears as a card in the "Today's Workflow" modal.
+2.  **Transparency**: 
+    -   **Badge**: "Suggested by [Agent Name]"
+    -   **Tooltip**: "Why? [Reasoning]" (e.g., "Because traffic dropped 15%").
+3.  **Feedback**:
+    -   **Complete**: Triggers positive reinforcement learning.
+    -   **Dismiss**: Triggers negative reinforcement learning.
+
+---
+
+## 🔄 Lifecycle & Triggers
+
+-   **Daily Reset**: The plan is generated once per day (UTC).
+-   **Persistence**: Tasks remain "in progress" until marked done or the day ends.
+-   **On-Demand**: Users can manually trigger a regeneration if the day's plan is empty or irrelevant.