Add Llm::Resolver which picks the LLM credential/endpoint for a given account
for the M2 conversation analytics classifier, in priority order:
1. per-account OpenAI integration hook (settings.api_key + optional
settings.base_url, via Account has_many :hooks)
2. instance Captain config (CAPTAIN_OPEN_AI_API_KEY / ENDPOINT)
3. nil => feature disabled for that account (no LLM).
Only https base URLs are accepted (safe_https?); a non-https or blank base_url
is omitted so callers fall back to the provider default (api.openai.com) rather
than an arbitrary host — content never leaks to an unapproved endpoint.
Approved by independent five-key pre-commit review deleg_269e20cb
(passed=true, blocking arrays empty).
The OpenAI Custom Server request targets the per-account OpenAI integration
(/app/accounts/{id}/settings/integrations/openai), not the instance-level
Captain config. The CAPTAIN_OPEN_AI_PROVIDER provider-switch added in 46fe70398
was out of the intended scope and is new behavior that did not previously exist,
so it is removed. This restores the original Captain behavior (CAPTAIN_OPEN_AI_ENDPOINT
remains usable as before, no provider gating). The correct implementation goes in
config/integration/apps.yml for the per-account openai integration.
Add an operator-level provider switch for Captain AI so it can use any
OpenAI-compatible base URL (vLLM, OpenRouter, proxies, etc.) in addition to the
default api.openai.com endpoint.
- installation_config.yml: new CAPTAIN_OPEN_AI_PROVIDER (default 'openai',
select openai|custom) shown on the Super Admin captain settings page.
- lib/llm/config.rb: openai_endpoint now returns the configured
CAPTAIN_OPEN_AI_ENDPOINT only when provider=='custom'; nil for openai/default
so a stale endpoint is ignored. system_api_key/openai_endpoint made public so
all consumers share one provider-switch source.
- Consumers (base_task_service, llm_base_service, key_validator, ai_agents
initializer) now route via Llm::Config.openai_endpoint instead of reading
CAPTAIN_OPEN_AI_ENDPOINT directly.
- installation_config.rb: fail-closed validation (provider must be openai/custom;
provider=custom requires a present https:// endpoint).
Approved by independent five-key reviews deleg_b141098a (found+fixed one logic
gap) and deleg_3088b922 (passed=true, blocking arrays empty).
Conversation completion evaluations now use a dedicated internal LLM
feature with GPT 4.1 as the default. The internal route keeps the
completion model separate from the installation wide Captain model
override and from the assistant route, which can use GPT 5.2 for Captain
V2 accounts. Evaluations continue to use the installation API key and do
not consume Captain response credits.
## What changed
Added an internal `conversation_completion` feature to the LLM model
config and excluded internal features from account preferences, the
Captain settings API, and Super Admin model overrides.
Updated `Captain::ConversationCompletionService` to resolve its model
through `Llm::FeatureRouter`.
Added focused service and request coverage for model routing and
settings visibility.
# Pull Request Template
## Description
defaults new accounts to captain v2
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
locally and specs
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
## Description
Adds the foundation for feature-specific LLM model routing so Captain AI
features can resolve their effective provider/model from code defaults
and account-level overrides. This fixes the provider metadata key in
`config/llm.yml`, adds `Llm::FeatureRouter`, and routes existing
`CaptainFeaturable` model defaults through the shared resolver.
Fixes https://linear.app/chatwoot/issue/CW-7425/test-new-models
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update
## How Has This Been Tested?
- `bundle exec rspec spec/lib/llm/models_spec.rb
spec/lib/llm/feature_router_spec.rb
spec/models/concerns/captain_featurable_spec.rb` - 23 examples, 0
failures
- `bundle exec rubocop lib/llm/models.rb lib/llm/feature_router.rb
app/models/concerns/captain_featurable.rb spec/lib/llm/models_spec.rb
spec/lib/llm/feature_router_spec.rb
spec/models/concerns/captain_featurable_spec.rb` - no offenses
- `bundle exec ruby -e "require 'yaml'; config =
YAML.load_file('config/llm.yml'); abort('missing providers') unless
config['providers']; abort('missing models') unless config['models'];
abort('missing features') unless config['features']; puts 'llm.yml ok'"`
- `git diff --check`
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
RubyLLM bundles a static models.json that doesn't know about models
released after the gem was published. Self-hosted users configuring
newer models hit ModelNotFoundError.
Added a rake task that refreshes the registry from models.dev and saves
to disk. ~~Called during Docker image build so every deploy gets fresh
model data. Falls back silently to the bundled registry if models.dev is
unreachable.~~
Commit the models.json file to code so it is available across
deployments.
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Removes sentry flooding of unnecessary rubyllm logs of wrong API key.
Logs only system api key error since it would be P0.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
