651bd2b5f0
- Changed rocket emoji to ASCII [*] for Windows CP1252 compatibility - This allows the backend to start without UnicodeEncodeError on Windows
99 lines
4.0 KiB
Markdown
99 lines
4.0 KiB
Markdown
# TxtaiIntelligenceService Non-Blocking Initialization Fix
|
|
|
|
## Problem
|
|
When users accessed the dashboard, the application would display "Loading weights" messages that:
|
|
1. Blocked other API services from responding
|
|
2. Occurred on every dashboard refresh (not just at startup)
|
|
3. Prevented users from interacting with the application
|
|
|
|
The "Loading weights" messages indicated that the embedding model was being loaded synchronously in the main request thread, blocking the event loop.
|
|
|
|
## Root Cause
|
|
The `TxtaiIntelligenceService` was initialized synchronously when `index_content()` was called:
|
|
- Each dashboard refresh triggered `_index_tasks_to_sif()`
|
|
- This function called `TxtaiIntelligenceService(user_id)`
|
|
- The initialization loaded embedding weights synchronously
|
|
- This blocked the FastAPI event loop, preventing other requests from being processed
|
|
|
|
## Solution
|
|
|
|
### 1. Background Thread Initialization
|
|
- Modified `_ensure_initialized()` to spawn a background thread instead of blocking
|
|
- The service initialization now happens in a daemon thread while the API returns immediately
|
|
- Prevents the event loop from being blocked
|
|
|
|
```python
|
|
def _ensure_initialized(self):
|
|
if self._initialization_in_progress:
|
|
return # Skip if already initializing
|
|
|
|
self._initialization_in_progress = True
|
|
thread = threading.Thread(target=self._initialize_embeddings, daemon=True)
|
|
thread.start()
|
|
```
|
|
|
|
### 2. Non-Blocking API Responses
|
|
- Updated `index_content()` to return immediately without waiting for initialization
|
|
- If service isn't initialized, it logs a message and returns gracefully
|
|
- Indexing will happen in background once weights are loaded
|
|
|
|
```python
|
|
async def index_content(self, items):
|
|
if not self._initialized and not self._initialization_in_progress:
|
|
self._ensure_initialized() # Start in background
|
|
return # Return immediately
|
|
|
|
if not self._initialized:
|
|
return # Initialization still in progress
|
|
```
|
|
|
|
### 3. Initialization Tracking
|
|
- Added `_initialization_in_progress` flag to track initialization state
|
|
- Prevents duplicate initialization attempts
|
|
- Uses asyncio locks for thread-safe initialization if needed later
|
|
|
|
### 4. Graceful Error Handling
|
|
- Background indexing failures are logged but don't crash the API
|
|
- API always responds quickly regardless of initialization state
|
|
|
|
## Files Modified
|
|
|
|
1. **backend/services/intelligence/txtai_service.py**
|
|
- Added threading import for background initialization
|
|
- Implemented non-blocking `_ensure_initialized()`
|
|
- Added `_ensure_initialized_async()` for future truly async operations
|
|
- Modified `index_content()` to return immediately without blocking
|
|
|
|
2. **backend/api/today_workflow.py**
|
|
- Updated `_index_tasks_to_sif()` with better error handling
|
|
- Added comments explaining non-blocking behavior
|
|
|
|
## Benefits
|
|
|
|
1. ✅ **Dashboard loads instantly** - No more "Loading weights" blocking the UI
|
|
2. ✅ **Non-blocking operations** - Other API services can respond while weights load
|
|
3. ✅ **Single initialization** - Weights are only loaded once per user session, in background
|
|
4. ✅ **Graceful degradation** - If indexing fails, API still responds normally
|
|
5. ✅ **Better user experience** - UI is responsive even during first-time weight loading
|
|
|
|
## Testing
|
|
|
|
The fix can be verified by:
|
|
1. Restarting the backend
|
|
2. Refreshing the dashboard multiple times
|
|
3. Observing that there's no "Loading weights" blocking message
|
|
4. Verifying other API endpoints respond quickly
|
|
|
|
## Technical Details
|
|
|
|
- **Thread Safety**: Uses daemon threads which don't prevent process shutdown
|
|
- **Event Loop**: Background initialization doesn't block the FastAPI event loop
|
|
- **Singleton Pattern**: Maintained - each user gets one TxtaiIntelligenceService instance
|
|
- **Caching**: Existing semantic cache functionality is preserved
|
|
|
|
## Future Improvements
|
|
|
|
1. Could implement pre-initialization in startup event for common users
|
|
2. Could add metrics to track initialization completion
|
|
3. Could implement true async initialization with asyncio instead of threading
|