ALwrity/docs/ALwrity Researcher/INTENT_DRIVEN_RESEARCH_GUIDE.md

Intent-Driven Research Guide

Date: 2025-01-29
Status: Comprehensive Guide to Intent-Driven Research

---

📋 Overview

Intent-driven research is a paradigm shift from manual research configuration to AI-inferred research goals. Instead of users selecting research modes and configuring providers manually, the AI analyzes user input and automatically determines:

1. What the user wants to research (intent inference)
2. How to research it (query generation)
3. Where to search (provider optimization)
4. What to extract (deliverable identification)

---

🎯 Core Concept

Traditional Research (Old)

User Input: "AI marketing tools"
  ↓
User selects: Comprehensive mode
User configures: Exa provider, 20 sources
  ↓
Research executes with user's configuration
  ↓
Generic results

Intent-Driven Research (Current)

User Input: "AI marketing tools"
  ↓
AI Analyzes:
  - Intent: "Find and compare AI marketing automation platforms"
  - Queries: ["AI marketing automation platforms 2025", ...]
  - Provider: Exa (best for company/product info)
  - Deliverables: statistics, expert quotes, case studies
  ↓
Research executes with AI-optimized configuration
  ↓
Intent-aware results (extracted deliverables)

---

🏗️ Architecture

Unified Research Analyzer

Location: backend/services/research/intent/unified_research_analyzer.py

Purpose: Single AI call that performs:

1. Intent inference
2. Query generation
3. Parameter optimization

Why Single Call?

• Reduces LLM calls from 2-3 to 1
• Faster response time
• Lower costs
• More consistent results

Input:

{
  "user_input": "AI marketing tools",
  "industry": "Technology",
  "target_audience": "Marketing professionals"
}

Output:

{
  "intent": {
    "primary_question": "What are the latest AI-powered marketing automation tools?",
    "research_goals": ["identify tools", "compare features", "analyze trends"],
    "deliverables": ["statistics", "expert_quotes", "case_studies"],
    "industry": "Technology",
    "target_audience": "Marketing professionals"
  },
  "queries": [
    {
      "query": "AI marketing automation platforms 2025",
      "provider": "exa",
      "justification": "Exa excels at finding company and product information"
    }
  ],
  "optimized_config": {
    "provider": "exa",
    "exa_category": "company",
    "exa_search_type": "neural",
    "provider_justification": "Exa is best for company/product research"
  },
  "trends_config": {
    "keywords": ["AI marketing", "marketing automation"],
    "enabled": true
  }
}

Intent-Aware Analyzer

Location: backend/services/research/intent/intent_aware_analyzer.py

Purpose: Analyzes raw research results based on user intent to extract specific deliverables

Why Intent-Aware?

• Extracts only relevant information
• Structures results based on user goals
• Provides actionable insights
• Reduces information overload

Input:

{
  "raw_results": {
    "sources": [...],
    "content": "..."
  },
  "intent": {
    "primary_question": "...",
    "deliverables": ["statistics", "expert_quotes", "case_studies"]
  }
}

Output:

{
  "summary": "Comprehensive overview...",
  "deliverables": {
    "statistics": [
      {
        "value": "85%",
        "description": "of marketers use AI tools",
        "citation": {...}
      }
    ],
    "expert_quotes": [
      {
        "quote": "...",
        "author": "...",
        "source": {...}
      }
    ],
    "case_studies": [...],
    "trends": [...]
  },
  "sources": [...],
  "analysis": "Deep insights based on intent..."
}

---

🔄 Research Flow

Step-by-Step Flow

1. User Input
   User enters: "AI marketing tools"
   Industry: "Technology"
   Target Audience: "Marketing professionals"
   ↓

2. Intent Analysis (UnifiedResearchAnalyzer)
   POST /api/research/intent/analyze
   ↓
   AI analyzes:
   - What user wants to research
   - What information they need
   - Best way to research it
   ↓
   Returns:
   - ResearchIntent
   - ResearchQuery[]
   - OptimizedConfig
   - TrendsConfig (if applicable)
   ↓

3. Intent Confirmation (Frontend)
   User reviews:
   - Primary question
   - Generated queries
   - Provider settings
   - Google Trends keywords
   ↓
   User can:
   - Edit primary question
   - Toggle deliverables
   - Select/edit queries
   - Review provider settings
   ↓

4. Research Execution
   POST /api/research/intent/research
   ↓
   Execute queries via:
   - Exa (priority 1)
   - Tavily (priority 2)
   - Google (priority 3)
   ↓
   Parallel execution:
   - Core research queries
   - Google Trends (if enabled)
   ↓

5. Intent-Aware Analysis (IntentAwareAnalyzer)
   Analyze raw results based on intent
   ↓
   Extract:
   - Statistics with citations
   - Expert quotes
   - Case studies
   - Trends
   - Comparisons
   ↓

6. Results Display
   Tabbed view:
   - Summary: AI-generated overview
   - Deliverables: Extracted statistics, quotes, etc.
   - Sources: Citations with credibility scores
   - Analysis: Deep insights

---

🔌 API Endpoints

1. Intent Analysis

Endpoint: POST /api/research/intent/analyze

Request:

{
  "keywords": "AI marketing tools",
  "industry": "Technology",
  "target_audience": "Marketing professionals"
}

Response:

{
  "success": true,
  "intent": {
    "primary_question": "What are the latest AI-powered marketing automation tools?",
    "research_goals": [
      "identify top AI marketing tools",
      "compare features and pricing",
      "analyze market trends"
    ],
    "deliverables": [
      "statistics",
      "expert_quotes",
      "case_studies",
      "trends"
    ],
    "industry": "Technology",
    "target_audience": "Marketing professionals"
  },
  "queries": [
    {
      "query": "AI marketing automation platforms 2025",
      "provider": "exa",
      "justification": "Exa excels at finding company and product information"
    },
    {
      "query": "best AI marketing tools comparison",
      "provider": "tavily",
      "justification": "Tavily is best for recent comparisons and reviews"
    }
  ],
  "optimized_config": {
    "provider": "exa",
    "exa_category": "company",
    "exa_search_type": "neural",
    "max_sources": 20,
    "include_statistics": true,
    "include_expert_quotes": true,
    "include_case_studies": true,
    "include_trends": true,
    "provider_justification": "Exa is best for company/product research"
  },
  "trends_config": {
    "keywords": ["AI marketing", "marketing automation", "AI tools"],
    "enabled": true
  }
}

2. Intent-Driven Research

Endpoint: POST /api/research/intent/research

Request:

{
  "intent": {
    "primary_question": "...",
    "research_goals": [...],
    "deliverables": [...]
  },
  "queries": [
    {
      "query": "...",
      "provider": "exa"
    }
  ],
  "config": {
    "provider": "exa",
    "exa_category": "company",
    "max_sources": 20
  },
  "trends_config": {
    "keywords": [...],
    "enabled": true
  }
}

Response:

{
  "success": true,
  "result": {
    "summary": "Comprehensive overview of AI marketing tools...",
    "deliverables": {
      "statistics": [
        {
          "value": "85%",
          "description": "of marketers use AI tools in their workflow",
          "citation": {
            "source": "Marketing AI Report 2025",
            "url": "https://...",
            "credibility_score": 0.9
          }
        }
      ],
      "expert_quotes": [
        {
          "quote": "AI marketing tools are transforming how we approach customer engagement...",
          "author": "John Doe",
          "title": "CMO at TechCorp",
          "source": {...}
        }
      ],
      "case_studies": [
        {
          "title": "How Company X Increased ROI by 200%",
          "summary": "...",
          "source": {...}
        }
      ],
      "trends": [
        {
          "trend": "AI personalization",
          "description": "...",
          "data": {...}
        }
      ]
    },
    "sources": [
      {
        "title": "...",
        "url": "...",
        "credibility_score": 0.9,
        "relevance_score": 0.95
      }
    ],
    "analysis": "Deep insights based on research intent..."
  }
}

---

🎨 Frontend Integration

useIntentResearch Hook

Location: frontend/src/components/Research/hooks/useIntentResearch.ts

Usage:

import { useIntentResearch } from '../hooks/useIntentResearch';

function ResearchComponent() {
  const intentResearch = useIntentResearch();

  // Analyze intent
  const handleAnalyze = async () => {
    await intentResearch.analyzeIntent(
      "AI marketing tools",
      "Technology",
      "Marketing professionals"
    );
  };

  // Confirm intent
  const handleConfirm = (intent: ResearchIntent) => {
    intentResearch.confirmIntent(intent);
  };

  // Execute research
  const handleExecute = async (queries: ResearchQuery[]) => {
    const result = await intentResearch.executeResearch(queries);
    if (result?.success) {
      // Handle results
    }
  };

  return (
    <div>
      {/* UI */}
    </div>
  );
}

IntentConfirmationPanel Component

Location: frontend/src/components/Research/steps/components/IntentConfirmationPanel/

Purpose: Allows users to review and edit AI-inferred intent

Features:

• Editable primary question
• Toggle deliverables
• Select/edit queries
• Review provider settings
• Google Trends keywords display

Usage:

<IntentConfirmationPanel
  isAnalyzing={execution.isAnalyzingIntent}
  intentAnalysis={execution.intentAnalysis}
  confirmedIntent={execution.confirmedIntent}
  onConfirm={execution.confirmIntent}
  onUpdateField={execution.updateIntentField}
  onExecute={async (selectedQueries) => {
    await execution.executeIntentResearch(state, selectedQueries);
  }}
  onDismiss={execution.clearIntent}
  isExecuting={execution.isExecuting}
  showAdvancedOptions={advanced}
  onAdvancedOptionsChange={setAdvanced}
  providerAvailability={providerAvailability}
  config={state.config}
  onConfigUpdate={handleConfigUpdate}
/>

---

🔧 Backend Implementation

UnifiedResearchAnalyzer

Key Method:

async def analyze(
    user_input: str,
    industry: Optional[str] = None,
    target_audience: Optional[str] = None,
    user_id: Optional[str] = None
) -> UnifiedResearchAnalysis:
    """
    Analyzes user input and returns:
    - Inferred research intent
    - Generated research queries
    - Optimized provider configuration
    - Google Trends keywords (if applicable)
    """
    # Build unified prompt
    prompt = self._build_unified_prompt(
        user_input, industry, target_audience
    )
    
    # Single LLM call
    response = await llm_text_gen(
        prompt=prompt,
        user_id=user_id,
        response_format={"type": "json_object"}
    )
    
    # Parse response
    analysis = UnifiedResearchAnalysis.parse_raw(response)
    
    return analysis

Prompt Structure:

1. User input context
2. Current date/time context (for time-sensitive queries)
3. Intent inference instructions
4. Query generation rules
5. Parameter optimization guidelines
6. Google Trends keyword suggestions

IntentAwareAnalyzer

Key Method:

async def analyze(
    raw_results: Dict[str, Any],
    intent: ResearchIntent,
    user_id: Optional[str] = None
) -> IntentDrivenResearchResult:
    """
    Analyzes raw results based on user intent
    """
    # Build analysis prompt
    prompt = self._build_analysis_prompt(
        raw_results, intent
    )
    
    # LLM call
    response = await llm_text_gen(
        prompt=prompt,
        user_id=user_id,
        response_format={"type": "json_object"}
    )
    
    # Parse and structure results
    result = IntentDrivenResearchResult.parse_raw(response)
    
    return result

---

📊 Benefits

For Users

• Faster: No manual configuration needed
• Smarter: AI optimizes for best results
• Better Results: Intent-aware extraction
• Less Overwhelming: Structured deliverables

For Developers

• Simpler: Single API call instead of multiple
• Consistent: AI ensures consistent quality
• Maintainable: Less configuration logic
• Extensible: Easy to add new providers/features

For Business

• Lower Costs: Fewer LLM calls
• Better UX: Users get results faster
• Higher Quality: AI-optimized research
• Scalable: Handles complex research needs

---

🎯 Best Practices

1. Always Use Intent Analysis First

// Good: Analyze intent before research
const analysis = await analyzeIntent(keywords, industry, audience);
const result = await executeResearch(analysis.queries, analysis.config);

// Avoid: Skip intent analysis
const result = await executeResearch([keywords], defaultConfig);

2. Let Users Review Intent

// Good: Show IntentConfirmationPanel
<IntentConfirmationPanel
  intentAnalysis={analysis}
  onConfirm={handleConfirm}
/>

// Avoid: Auto-execute without confirmation
await executeResearch(analysis.queries); // User can't review

3. Use Intent-Aware Results

// Good: Use structured deliverables
result.deliverables.statistics.forEach(stat => {
  // Use structured data
});

// Avoid: Parse raw results manually
const stats = parseRawResults(result.raw_content); // Manual parsing

---

🔄 Migration Guide

From Traditional Research

Old Code:

// User selects mode
const mode = 'comprehensive';

// User configures provider
const config = {
  provider: 'exa',
  max_sources: 20
};

// Execute research
const result = await executeResearch(keywords, mode, config);

New Code:

// Analyze intent
const analysis = await analyzeIntent(keywords, industry, audience);

// User reviews (optional)
// Execute with AI-optimized config
const result = await executeIntentResearch(
  analysis.intent,
  analysis.queries,
  analysis.optimized_config
);

---

📚 Additional Resources

• Architecture Rules: .cursor/rules/researcher-architecture.mdc
• Implementation Guide: RESEARCH_WIZARD_IMPLEMENTATION.md
• Integration Guide: RESEARCH_COMPONENT_INTEGRATION.md
• Current Architecture: CURRENT_ARCHITECTURE_OVERVIEW.md

---

✅ Implementation Status

• ✅ UnifiedResearchAnalyzer implemented
• ✅ IntentAwareAnalyzer implemented
• ✅ Intent-driven API endpoints working
• ✅ Frontend integration complete
• ✅ Google Trends integrated
• ✅ Research persona integrated

---

Status: Current and Comprehensive