Extract useful LLM provider improvements from PRs #423-#429

huggingface_provider.py:
- Add retry logic with _should_retry_hf_error and _is_non_retryable_hf_error
- Update default models from :groq to :cerebras (HF_FALLBACK_MODELS)
- Add fallback_models parameter to huggingface_text_response
- Add get_available_models with updated model list

main_text_generation.py:
- Add GPT_PROVIDER and TEXTGEN_AI_MODELS env var support
- Add preferred_provider and flow_type parameters to llm_text_gen
- Add HF_MODEL_MAPPING for short model name resolution
- Add flow_type logging tag for better observability

sif_agents.py:
- Add LOW_COST_SHARED_REMOTE_MODELS for SIF agents
- Update SharedLLMWrapper to use preferred_hf_models and flow_type

These changes preserve the modular textgen_utils structure while incorporating
the useful routing and retry logic improvements from the pending PRs.

backend/services/intelligence/sif_agents.py

@@ -32,9 +32,12 @@ class SharedLLMWrapper:
     def generate(self, prompt: str, **kwargs) -> str:
         """Generate text using the shared LLM provider."""
         try:
-            # We ignore kwargs like 'max_tokens' as llm_text_gen handles defaults,
-            # but we could map them if needed.
-            return llm_text_gen(prompt, user_id=self.user_id)
+            return llm_text_gen(
+                prompt,
+                user_id=self.user_id,
+                preferred_hf_models=LOW_COST_SHARED_REMOTE_MODELS,
+                flow_type="sif_agent",
+            )
         except Exception as e:
             logger.error(f"SharedLLMWrapper failed to generate text: {e}")
             return f"[ERROR: Shared LLM generation failed for user {self.user_id}]"
@@ -44,6 +47,12 @@ class SharedLLMWrapper:
 
 _local_llm_cache = {}
 
+LOW_COST_SHARED_REMOTE_MODELS = [
+    "Qwen/Qwen2.5-1.5B-Instruct",
+    "Qwen/Qwen2.5-0.5B-Instruct",
+    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
+]
+
 LOCAL_LLM_FALLBACKS = [
     "Qwen/Qwen2.5-1.5B-Instruct",
     "Qwen/Qwen2.5-0.5B-Instruct",