ALwrity/docs/ALwrity Researcher/EXA_API_OPTIONS_AUDIT.md

# Exa API Options Audit

**Date**: 2025-01-29  
**Status**: Comparison of Current Implementation vs Exa API Documentation

---

## 📊 Summary

This document compares our current Exa implementation with the official Exa API documentation to identify missing options and configuration gaps.

---

## ✅ Currently Supported Options

### Main Search Parameters
1. ✅ **`type`** - Search type (auto, neural, fast, deep)
   - **Frontend**: `exa_search_type` dropdown
   - **Backend**: `config.exa_search_type` → `type` parameter
   - **Status**: Fully supported

2. ✅ **`category`** - Content category filter
   - **Frontend**: `exa_category` dropdown
   - **Backend**: `config.exa_category` → `category` parameter
   - **Status**: Fully supported

3. ✅ **`numResults`** - Number of results (5-100)
   - **Frontend**: `exa_num_results` input (5-25 limit shown, but API supports up to 100)
   - **Backend**: Uses `config.max_sources` (capped at 25), should use `config.exa_num_results`
   - **Status**: Partially supported (needs to use `exa_num_results` instead of `max_sources`)

4. ✅ **`includeDomains`** - Domain inclusion filter
   - **Frontend**: `exa_include_domains` text input
   - **Backend**: `config.exa_include_domains` → `include_domains` parameter
   - **Status**: Fully supported

5. ✅ **`excludeDomains`** - Domain exclusion filter
   - **Frontend**: `exa_exclude_domains` text input
   - **Backend**: `config.exa_exclude_domains` → `exclude_domains` parameter
   - **Status**: Fully supported

### Contents Parameters (Currently Hardcoded)
6. ⚠️ **`text`** - Full page text retrieval
   - **Current**: Hardcoded to `{'max_characters': 1000}`
   - **Should be**: Configurable via `exa_text_max_characters` and `exa_text_include_html`
   - **Status**: Needs configuration

7. ⚠️ **`highlights`** - Text snippets extraction
   - **Current**: Hardcoded to `{'num_sentences': 2, 'highlights_per_url': 3}`
   - **Should be**: Configurable via `exa_highlights_num_sentences`, `exa_highlights_per_url`, `exa_highlights_query`
   - **Status**: Needs configuration (we have `exa_highlights` boolean but not the detailed config)

8. ⚠️ **`summary`** - Webpage summary
   - **Current**: Hardcoded to `{'query': f"Key insights about {topic}"}`
   - **Should be**: Configurable via `exa_summary_query` and `exa_summary_schema`
   - **Status**: Needs configuration

9. ⚠️ **`context`** - Context string for RAG
   - **Current**: Not used (we have `exa_context` boolean in config but not applied)
   - **Should be**: Configurable via `exa_context` (boolean) or `exa_context_max_characters` (object)
   - **Status**: Partially supported (config exists but not used)

---

## ❌ Missing Options

### Date Filters
10. ❌ **`startPublishedDate`** - Filter by publish date (start)
    - **Frontend**: We have `exa_date_filter` but it's not being used
    - **Backend**: Not passed to Exa API
    - **Status**: Config exists but not implemented

11. ❌ **`endPublishedDate`** - Filter by publish date (end)
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

12. ❌ **`startCrawlDate`** - Filter by crawl date (start)
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

13. ❌ **`endCrawlDate`** - Filter by crawl date (end)
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

### Text Filters
14. ❌ **`includeText`** - Text that must be present in results
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

15. ❌ **`excludeText`** - Text that must not be present in results
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

### Advanced Options
16. ❌ **`userLocation`** - Two-letter ISO country code
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

17. ❌ **`moderation`** - Content moderation filter
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

18. ❌ **`additionalQueries`** - Additional queries for deep search
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing (only works with `type="deep"`)

### Contents Advanced Options
19. ❌ **`livecrawl`** - Live crawling options (never, fallback, preferred, always)
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

20. ❌ **`livecrawlTimeout`** - Timeout for live crawling (ms)
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

21. ❌ **`subpages`** - Number of subpages to crawl
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

22. ❌ **`subpageTarget`** - Term to find specific subpages
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

23. ❌ **`extras`** - Extra parameters (links, imageLinks)
    - **Frontend**: Not exposed
    - **Backend**: Not implemented
    - **Status**: Missing

---

## 🔧 Implementation Gaps

### 1. Date Filter Not Applied
- **Issue**: `exa_date_filter` exists in config but is not passed to Exa API
- **Fix**: Map `exa_date_filter` → `startPublishedDate` in `exa_provider.py`

### 2. Context Not Applied
- **Issue**: `exa_context` boolean exists but is not used
- **Fix**: Apply `context` parameter based on `exa_context` value

### 3. Num Results Uses Wrong Field
- **Issue**: Uses `config.max_sources` instead of `config.exa_num_results`
- **Fix**: Use `config.exa_num_results` if available, fallback to `max_sources`

### 4. Contents Parameters Hardcoded
- **Issue**: `text`, `highlights`, `summary` are hardcoded
- **Fix**: Make them configurable via ResearchConfig

---

## 📋 Recommended Priority

### Priority 1: Fix Existing Config Not Applied
1. ✅ Apply `exa_date_filter` → `startPublishedDate`
2. ✅ Apply `exa_context` → `context`
3. ✅ Use `exa_num_results` instead of `max_sources`

### Priority 2: Make Contents Configurable
4. ✅ Make `text.max_characters` configurable
5. ✅ Make `highlights` configurable (num_sentences, highlights_per_url, query)
6. ✅ Make `summary.query` configurable

### Priority 3: Add Common Date Filters
7. ✅ Add `endPublishedDate` support
8. ✅ Add `startCrawlDate` / `endCrawlDate` support (if needed)

### Priority 4: Add Text Filters (If Needed)
9. ✅ Add `includeText` / `excludeText` support (if needed)

### Priority 5: Advanced Options (Low Priority)
10. ✅ Add `userLocation`, `moderation`, `livecrawl`, `subpages`, `extras` (if needed)

---

## 🎯 Current Status

**Total Exa API Options**: ~23 options  
**Currently Supported**: 5 fully, 4 partially  
**Missing**: 14 options  
**Hardcoded**: 3 options (text, highlights, summary)

**Recommendation**: Focus on Priority 1 and 2 to make existing config work and make contents configurable.

---

## ✅ Recent Fixes (2025-01-29)

### Fixed Critical Issues
1. ✅ **Updated `type` enum**: Removed `deep`, added `keyword` and `fast` to match latest API
2. ✅ **Updated `category` enum**: Removed `movie` and `song`, kept `linkedin profile` 
3. ✅ **Applied `exa_date_filter`**: Now maps to `start_published_date` parameter
4. ✅ **Applied `exa_context`**: Now properly passed to Exa API when enabled
5. ✅ **Fixed `exa_num_results`**: Now uses `exa_num_results` instead of `max_sources`, supports up to 100 results
6. ✅ **Updated frontend**: Added `fast` option, updated category list, increased num_results limit to 100

### Updated Files
- `backend/services/research/intent/unified_research_analyzer.py` - Updated AI prompt enum values
- `backend/services/blog_writer/research/exa_provider.py` - Applied date filter, context, and num_results
- `frontend/src/components/Research/steps/utils/constants.ts` - Updated search types and categories
- `frontend/src/components/Research/steps/components/ExaOptions.tsx` - Updated num_results limit and type handling