diff --git a/backend/start_alwrity_backend.py b/backend/start_alwrity_backend.py index 2d80de25..58286dd6 100644 --- a/backend/start_alwrity_backend.py +++ b/backend/start_alwrity_backend.py @@ -308,7 +308,7 @@ def main(): # Set global verbose flag for utilities os.environ["ALWRITY_VERBOSE"] = "true" if verbose_mode else "false" - print("🚀 ALwrity Backend Server") + print("[*] ALwrity Backend Server") print("=" * 40) print(f"Mode: {'PRODUCTION' if production_mode else 'DEVELOPMENT'}") print(f"Auto-reload: {'ENABLED' if enable_reload else 'DISABLED'}") diff --git a/docs/TXTAI_NON_BLOCKING_FIX.md b/docs/TXTAI_NON_BLOCKING_FIX.md new file mode 100644 index 00000000..857b4bf7 --- /dev/null +++ b/docs/TXTAI_NON_BLOCKING_FIX.md @@ -0,0 +1,98 @@ +# TxtaiIntelligenceService Non-Blocking Initialization Fix + +## Problem +When users accessed the dashboard, the application would display "Loading weights" messages that: +1. Blocked other API services from responding +2. Occurred on every dashboard refresh (not just at startup) +3. Prevented users from interacting with the application + +The "Loading weights" messages indicated that the embedding model was being loaded synchronously in the main request thread, blocking the event loop. + +## Root Cause +The `TxtaiIntelligenceService` was initialized synchronously when `index_content()` was called: +- Each dashboard refresh triggered `_index_tasks_to_sif()` +- This function called `TxtaiIntelligenceService(user_id)` +- The initialization loaded embedding weights synchronously +- This blocked the FastAPI event loop, preventing other requests from being processed + +## Solution + +### 1. Background Thread Initialization +- Modified `_ensure_initialized()` to spawn a background thread instead of blocking +- The service initialization now happens in a daemon thread while the API returns immediately +- Prevents the event loop from being blocked + +```python +def _ensure_initialized(self): + if self._initialization_in_progress: + return # Skip if already initializing + + self._initialization_in_progress = True + thread = threading.Thread(target=self._initialize_embeddings, daemon=True) + thread.start() +``` + +### 2. Non-Blocking API Responses +- Updated `index_content()` to return immediately without waiting for initialization +- If service isn't initialized, it logs a message and returns gracefully +- Indexing will happen in background once weights are loaded + +```python +async def index_content(self, items): + if not self._initialized and not self._initialization_in_progress: + self._ensure_initialized() # Start in background + return # Return immediately + + if not self._initialized: + return # Initialization still in progress +``` + +### 3. Initialization Tracking +- Added `_initialization_in_progress` flag to track initialization state +- Prevents duplicate initialization attempts +- Uses asyncio locks for thread-safe initialization if needed later + +### 4. Graceful Error Handling +- Background indexing failures are logged but don't crash the API +- API always responds quickly regardless of initialization state + +## Files Modified + +1. **backend/services/intelligence/txtai_service.py** + - Added threading import for background initialization + - Implemented non-blocking `_ensure_initialized()` + - Added `_ensure_initialized_async()` for future truly async operations + - Modified `index_content()` to return immediately without blocking + +2. **backend/api/today_workflow.py** + - Updated `_index_tasks_to_sif()` with better error handling + - Added comments explaining non-blocking behavior + +## Benefits + +1. ✅ **Dashboard loads instantly** - No more "Loading weights" blocking the UI +2. ✅ **Non-blocking operations** - Other API services can respond while weights load +3. ✅ **Single initialization** - Weights are only loaded once per user session, in background +4. ✅ **Graceful degradation** - If indexing fails, API still responds normally +5. ✅ **Better user experience** - UI is responsive even during first-time weight loading + +## Testing + +The fix can be verified by: +1. Restarting the backend +2. Refreshing the dashboard multiple times +3. Observing that there's no "Loading weights" blocking message +4. Verifying other API endpoints respond quickly + +## Technical Details + +- **Thread Safety**: Uses daemon threads which don't prevent process shutdown +- **Event Loop**: Background initialization doesn't block the FastAPI event loop +- **Singleton Pattern**: Maintained - each user gets one TxtaiIntelligenceService instance +- **Caching**: Existing semantic cache functionality is preserved + +## Future Improvements + +1. Could implement pre-initialization in startup event for common users +2. Could add metrics to track initialization completion +3. Could implement true async initialization with asyncio instead of threading