Conversation completion evaluations now use a dedicated internal LLM
feature with GPT 4.1 as the default. The internal route keeps the
completion model separate from the installation wide Captain model
override and from the assistant route, which can use GPT 5.2 for Captain
V2 accounts. Evaluations continue to use the installation API key and do
not consume Captain response credits.
## What changed
Added an internal `conversation_completion` feature to the LLM model
config and excluded internal features from account preferences, the
Captain settings API, and Super Admin model overrides.
Updated `Captain::ConversationCompletionService` to resolve its model
through `Llm::FeatureRouter`.
Added focused service and request coverage for model routing and
settings visibility.
Captain can now use each assistant's saved setting when a customer stops
replying. Captain can review the conversation and resolve or hand it
off, resolve it after the selected time without review, or leave it
pending until the customer replies.
The job checks the conversation again while holding a database lock
before it changes the status. A new customer reply or another worker
cannot cause an outdated resolve or handoff.
## Closes
[AI-163](https://linear.app/chatwoot/issue/AI-163)
## What changed
- Added assistant modes for review, always resolve, and wait for the
customer.
- Kept the account setting as the fallback for assistants that do not
have a saved mode.
- Skipped scheduling when resolution is disabled on the assistant or
through the older account setting.
- Rechecked the conversation status and activity time before each
resolve or handoff.
- Recorded events only after a status change succeeds.
- Kept out of office messages out of campaign conversations.
## How to test
1. Set an assistant to review conversations. Run the inactivity job with
complete and incomplete decisions. Confirm the first conversation is
resolved and the second is handed off.
2. Set the assistant to always resolve. Confirm an eligible pending
conversation is resolved after the selected time.
3. Set the assistant to wait for the customer. Confirm the scheduler
does not enqueue the inactivity job and the conversation remains
pending.
4. Add a customer reply while the review is running. Confirm the job
does not resolve or hand off the updated conversation.
5. Run two workers for the same conversation. Confirm only one status
change and one event are recorded.
---------
Co-authored-by: iamsivin <iamsivin@gmail.com>
Records Captain conversation outcomes at episode grain so reporting can
distinguish initial demand from reopened conversations and measure
replies, handoffs, resolutions, human follow-up, and CSAT.
Eligibility creates the episode at demand time. Message-derived fields
are snapshotted from persisted messages at handoff or resolution,
keeping terminal analytics accurate without writing outcomes for every
message. Outcome tracking remains reporting-only and fail-open.
Builds on the episode-grain schema from #15315.
## Closes
- https://linear.app/chatwoot/issue/CW-7792
## How to test
1. Enable `captain_integration_v2` and connect a Captain assistant to an
inbox.
2. Send an inbound customer message and confirm an initial outcome
episode is created at the message timestamp.
3. Let Captain reply and then resolve or hand off the conversation.
Confirm the episode records Captain reply counts and timestamps, the
outcome timestamp, and the handoff category where applicable.
4. Reply after resolution and confirm a `reopen` episode is created
while preserving the previous episode.
5. Resolve the reopened conversation and submit CSAT. Confirm the
response is attributed to the episode that issued the survey.
## What changed
- Creates the initial episode from demand-level eligibility and appends
a new episode when a resolved conversation reopens.
- Snapshots Captain replies and the first qualifying human reply from
persisted messages at handoff and resolution.
- Attributes asynchronous resolution events using the episode active at
the event timestamp.
- Records later CSAT responses using the survey message timestamp.
- Keeps boundary writes transactional and fail-open without retries,
advisory locks, late-boundary repair, or handoff self-healing.
- Adds schema-constrained handoff reason categories, including lifecycle
coverage for incomplete V2 tool fallback handoffs.
Open, non-terminal episodes may retain empty or stale message-derived
fields until handoff or resolution.
Captain V2 now adds FAQ citations from a structured model response. The
model returns ordered response parts with citation indexes, and Chatwoot
turns only trusted indexes into customer links.
## Before
Captain V2 asked the model to copy text markers such as `[[faq:1]]`.
Chatwoot used one regular expression to replace those markers with links
in the outgoing message and another regular expression to remove the
rendered links before the next model turn. Long conversations depended
on parsing the customer message to recover plain model context.
## After
The FAQ lookup tool now gives each eligible source document a numeric
index and never gives the model a URL. FAQ results from the same
document reuse the same index. The model returns `response_parts`, where
each part contains customer text and the supporting citation indexes.
Chatwoot checks every index against the document IDs registered during
the current run.
Only stored HTTP or HTTPS web-document links without embedded
credentials can appear in the customer reply. Blank links, PDF sources,
attachments, non-HTTP storage links, and unknown indexes do not create
links. Sources receive display numbers in the order they first appear,
and repeated sources keep the same display number.
Chatwoot saves the structured response parts with each newly generated
Captain message. Later Captain V2 turns use the saved plain text for
those messages, so they never need to parse rendered links. Existing
messages remain unchanged and continue to use their stored content. When
citations are disabled, Chatwoot clears citation indexes before it
returns or saves the response.
Captain V1, Copilot, legacy prompts, legacy tools, and the playground
response contract are unchanged. The playground continues to show the
plain `response` field.
## Closes
[AI-138](https://linear.app/chatwoot/issue/AI-138/faq-citation-fix)
## How to test
1. Open a conversation handled by a Captain V2 assistant and turn
citations off. Ask a greeting, an FAQ question, a code question, and a
follow up question. Confirm that the assistant answers normally and
shows no source links.
2. Turn citations on and ask a question that matches one public web
document. Confirm that the reply shows the stored public link after the
supported text.
3. Ask a question that needs two public web documents. Confirm that the
response order stays correct, each link appears after the supported
text, and repeated sources keep the same display number.
4. Ask a question that retrieves several FAQ results from one document.
Confirm that the reply shows the document once at each supported
response part rather than exposing separate FAQ sources.
5. Ask a question supported by a PDF, attachment, blank link, or
non-HTTP storage link. Confirm that Captain can use the information but
does not show a customer link.
6. Ask for a fenced code example with a citation. Confirm that the code
block stays complete and the citation appears after the closing fence.
7. Continue the conversation with a follow up question. Confirm that
Captain uses the earlier plain response text and does not receive or
repeat rendered citation links.
8. Test a scenario handoff in a conversation. Confirm that the handoff
and final response still work.
Captain auto-resolve policy is now owned by each assistant. Existing
assistants first read an assistant-level setting, fall back to the
current account setting during rollout, and are backfilled
asynchronously so behavior is preserved.
## Closes
[AI-163](https://linear.app/chatwoot/issue/AI-163)
## How to test
1. Configure different auto-resolve modes on two assistants in the same
account through the API.
2. Confirm each assistant follows its own mode when the inactivity job
runs.
3. Confirm an assistant without the new setting follows the existing
account mode.
4. Run the migration job and confirm the account mode is copied without
overwriting an existing assistant mode.
5. Confirm evaluated mode falls back to time-based resolution when the
`captain_tasks` capability is unavailable.
## What changed
- Added assistant-level `disabled`, `legacy`, and `evaluated` policy
storage and validation.
- Updated scheduling and resolution runtime reads to use the assistant
policy.
- Added a compatibility fallback and asynchronous backfill from the
account setting.
- Preserved the account-level `captain_tasks` capability gate for
evaluation.
- Kept this foundation independent of the Captain V2 guard; the timer
and advanced settings in stacks 2–5 are Captain V2-only.
Stack 1 of 5. This is the base for #15303.
This introduces a small event layer for the Captain V2 conversation
lifecycle. A new `Captain::ConversationEvents` facade dispatches five
normalized events (`captain.conversation.engaged`,
`captain.conversation.handed_off`, `captain.conversation.resolved`,
`captain.response.completed`, `captain.response.failed`) from the points
where Captain engages a conversation, replies, fails, hands off, or
auto-resolves. Each event carries the conversation, assistant,
timestamp, and a `source`/`reason_category` where relevant.
## Why this, why now
The Captain V2 flow is about to gain several observers at once:
conversation outcome tracking, agent session capture, and analytics all
need to know when Captain engages, replies, fails, hands off, or
resolves. Wiring each of them directly into `ResponseBuilderJob`,
`HookExecutionService`, and the tools would tangle secondary bookkeeping
into the paths that deliver customer-facing behavior, and every future
consumer would deepen that. Landing the event layer first as its own PR
means the flow announces these moments once and stays otherwise
untouched: customer-visible behavior (messages, status changes,
handoffs, usage enforcement) remains synchronous, while secondary
effects subscribe through listeners. The upcoming conversation outcomes
PR then reduces to a listener plus a model instead of another round of
edits to the core flow, which is why this ships now, before that work
merges.
The existing inference reporting behavior is folded into this layer: the
`conversation.captain_inference_*` events and their dispatch helpers on
`Enterprise::Conversation` are removed, and a dedicated
`Captain::ReportingEventListener` (registered on the enterprise async
dispatcher) maps `source: 'inference'` events to the same stored
reporting event names, so recorded analytics and the assistant stats
builder are unaffected.
## What changed
- New `Captain::ConversationEvents` facade and event type constants
- Event emission from `HookExecutionService` (engagement, usage-limit
handoff), `ResponseBuilderJob` (response completed/failed,
generation-failure handoff), `HandoffTool` (tool handoff), and
`InboxPendingConversationsResolutionJob` (inference resolved/handoff)
- A dedicated `Captain::ReportingEventListener` preserves inference
reporting events through the new event names, removing captain logic
from the OSS listener
Captain now handles a burst of customer messages with one reply. If more
messages arrive before Captain replies, the latest job uses the full
conversation history. The change applies only to Captain V2 and works
across every channel that Captain supports.
Blocked on: https://github.com/chatwoot/chatwoot/pull/15212
## Closes
Closes https://github.com/chatwoot/chatwoot/issues/14545
## How to reproduce
1. Start a pending conversation with Captain V2.
2. Send several messages while Captain is preparing a reply.
3. Captain can generate and send a separate reply for each message.
## What changed
* Each Captain V2 job records the incoming message that started it.
* A job stops before generation if a newer message already exists.
* Captain discards a generated reply if a newer message arrived during
generation.
* The latest job replies using the full conversation history.
* Langfuse records whether a generation was discarded and whether a
customer credit was used.
* Captain V1 keeps its existing behavior.
## Tradeoffs
Discarded generations still cost money. Message bursts can also increase
background job work and model provider load. A continuous stream of
incoming messages can delay the reply until one generation finishes
without a newer message.
A small timing window remains if a message arrives after the final check
and before Captain saves the reply. A handoff also cannot be undone if a
newer message arrives after Captain has already changed the conversation
status.
## How to test
1. Enable Captain V2 and start a pending Captain conversation.
2. Send several messages while Captain is preparing a reply.
3. Confirm that Captain sends one reply based on the full message
history.
4. Confirm that Langfuse marks discarded runs with `discarded=true` and
`credit_used=false`.
5. Disable Captain V2 and confirm that Captain V1 behavior is unchanged.
---------
Co-authored-by: Sony Mathew <sony@chatwoot.com>
Records a `Captain::Session` row for every Captain V2 assistant response
delivered in a conversation, so we can show how a response was generated
and report on credit, FAQ, and document usage. Stacked on #14970 (the
`captain_sessions` model).
## What changed
- `FaqLookupTool` now records the retrieved FAQ ids (and their backing
document ids) into the shared run state, accumulated across tool calls.
- `AgentRunnerService` exposes the raw ai-agents run result via
`last_run_result`; the `generate_response` return shape is unchanged, so
the playground path is unaffected.
- New `Captain::Assistant::SessionCaptureService` builds the session:
scenario resolved from the answering agent name, model from
`assistant.agent_model`, token usage plus the trimmed current-turn
conversation history stored in `run_context`.
- `ResponseBuilderJob` captures after delivery: `credits_consumed`
mirrors the actual charge (1.0 for a billed response, 0.0 for handoffs,
where the session points at the customer-facing handoff message).
Capture runs outside the delivery transaction and swallows its own
failures, so a logging bug can never block or roll back a customer
reply.
V1 responses and copilot are out of scope; copilot capture comes next.
## How to test
On an account with `captain_integration_v2` enabled and an inbox
connected to an assistant with approved FAQs, send a customer message on
a pending conversation. After the assistant replies, a
`Captain::Session` row should exist with the conversation as subject,
the reply message as result, the FAQs/documents used, and the run
context for that turn. Asking for a human agent should produce a
zero-credit session pointing at the handoff message.
<img width="2428" height="1058" alt="CleanShot 2026-07-15 at 17 25
40@2x"
src="https://github.com/user-attachments/assets/d8e44923-c17b-494f-8c33-c8fa4219438c"
/>
Captain custom tools configured with API key authentication now send the
configured key in the requested HTTP header. Existing tools begin
working without needing to be recreated or reconfigured, while
credentials remain protected across redirects.
## How to reproduce
1. Create a Captain custom tool using API Key authentication.
2. Configure `X-API-Key` as the header name and save the tool.
3. Invoke the tool and inspect the incoming request.
4. Before this change, the API key header is absent; after this change,
the configured endpoint receives it.
## What changed
The UI persists API key authentication as `name` and `key`, but the
request builder also required an unused `location: header` property. The
request builder now treats API key authentication as header-based,
matching the only mode exposed by the UI.
Custom authentication headers are also registered as sensitive with
`SafeFetch`. They are retained for the configured endpoint and
same-origin redirects, but stripped when a redirect crosses origins to
prevent credential leakage. Factory, request, and redirect specs cover
the real UI payload and both public and private-network fetch paths.
# Pull Request Template
## Description
Please include a summary of the change and issue(s) fixed. Also, mention
relevant motivation, context, and any dependencies that this change
requires.
Fixes
https://linear.app/chatwoot/issue/AI-136/check-conversation-status-while-auto-resolving
- After 60mins of inactivity, we run a job that decides if pending
conversations are resolvable or need handoff
- the prompt was a bit conservative and didn't have conversation state
context
## Type of change
Please delete options that are not relevant.
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
locally ran a sample eval
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Sony Mathew <sony@chatwoot.com>
Fixes https://linear.app/chatwoot/issue/CW-7559/inbox-limit-abuse
## Why
The regular inbox API checked limits in the controller, but WhatsApp
embedded signup creates inboxes through a service using `Inbox.create!`.
That let Enterprise account inbox limits be skipped for embedded signup.
## What this change does
- Adds an Inbox create-time validation hook in OSS and implements the
limit check in the Enterprise Inbox module.
- Removes the duplicate controller/helper limit check so the model is
the single enforcement point.
- Preserves the existing `402 Payment Required` API response for account
inbox limit failures.
- Keeps updates to existing inboxes allowed when an account is already
at its inbox limit.
## Validation
- `bundle exec rspec
spec/controllers/api/v1/accounts/inboxes_controller_spec.rb
spec/enterprise/models/inbox_spec.rb`
# Pull Request Template
## Description
Fixes:
https://linear.app/chatwoot/issue/CW-7167/label-suggestions-bad-ux
## Type of change
Please delete options that are not relevant.
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
# Pull Request Template
## Description
- Validates openai key while configuring hooks
- added backfill logic
Fixes # (issue)
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
locally
<img width="1710" height="1234" alt="CleanShot 2026-04-15 at 16 15
02@2x"
src="https://github.com/user-attachments/assets/3d319fe0-19f9-4fd0-9308-74987daac2e1"
/>
<img width="2884" height="1136" alt="CleanShot 2026-05-11 at 19 22
53@2x"
src="https://github.com/user-attachments/assets/5eae8650-985b-4c4a-af42-35f7175ff52d"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
When a new account finishes onboarding we want to land them on a
dashboard with a working web widget already configured, branded, named,
and assigned to them, instead of an empty inbox list. This PR adds the
services that produce that widget. **No user-visible change yet:** the
services are dormant until the trigger and background job are wired up
in the follow-up PR.
## Context
Milestone 1 added `Account::BrandingEnrichmentJob`, which calls
context.dev during signup and stores brand data on
`account.custom_attributes['brand_info']`, plus the new onboarding form
that captures `domain`, `name`, `industry`, etc. Milestone 2 starts
using that data, and the first thing we want is a web widget
materialized automatically. Splitting the service layer from the
orchestration plumbing (Redis key, `onboarding_step` extension,
controller wiring, ActionCable) keeps this diff focused and lets the
LLM/widget logic merge independently.
## How to test
Run against an existing account that already has `brand_info` populated.
```ruby
account = Account.find(<account_id>)
user = account.administrators.first
inbox = WidgetCreationService.new(account, user).perform
inbox.channel.widget_color # color from brand_info, or '#1f93ff'
inbox.channel.welcome_title # brand_info[:title], or account.name
inbox.channel.welcome_tagline # LLM tagline (Enterprise + system key set),
# else brand_info[:slogan]/[:description]/nil
inbox.inbox_members.pluck(:user_id)
```
Toggle `InstallationConfig['CAPTAIN_OPEN_AI_API_KEY']` to flip between
LLM and brand-text tagline paths. To verify failure isolation, raise
inside `Captain::Llm::WidgetTaglineService#perform` and confirm widget
creation still succeeds with the fallback tagline.
Removes sentry flooding of unnecessary rubyllm logs of wrong API key.
Logs only system api key error since it would be P0.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
# Pull Request Template
## Description
Add account setting and store_accessor for
`captain_force_legacy_auto_resolve`.
Enterprise job now skips LLM evaluation when this flag is true and falls
back to legacy time-based resolution. Add spec to cover the fallback.
## Type of change
We recently rolled out Captain deciding if a conversation is resolved or
not. While it is an improvement for majority of customers, some still
prefer the old way of auto-resolving based on inactivity. This PR adds a
check.
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
legacy_auto_resolve = true
<img width="1282" height="848" alt="CleanShot 2026-03-13 at 19 55 55@2x"
src="https://github.com/user-attachments/assets/dfdcc5d5-6d21-462b-87a6-a5e1b1290a8b"
/>
legacy_auto_resolve = false
<img width="1268" height="864" alt="CleanShot 2026-03-13 at 20 00 50@2x"
src="https://github.com/user-attachments/assets/f4719ec6-922a-4c3b-bc45-7b29eaced565"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
# Pull Request Template
## Description
Add an api_key override so internal conversation completions prefers
using the system API key and do not consume customer OpenAI credits.
## Type of change
Please delete options that are not relevant.
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
specs and locally
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
# Pull Request Template
## Description
captain decides if conversation should be resolved or open
Fixes
https://linear.app/chatwoot/issue/AI-91/make-captain-resolution-time-configurable
Update: Added 2 entries in reporting events:
`conversation_captain_handoff` and `conversation_captain_resolved`
## Type of change
Please delete options that are not relevant.
- [x] New feature (non-breaking change which adds functionality)
- [x] This change requires a documentation update
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
LLM call decides that conversation is resolved, drops a private note
<img width="1228" height="438" alt="image"
src="https://github.com/user-attachments/assets/fb2cf1e9-4b2b-458b-a1e2-45c53d6a0158"
/>
LLM call decides conversation is still open as query was not resolved
<img width="1215" height="573" alt="image"
src="https://github.com/user-attachments/assets/2d1d5322-f567-487e-954e-11ab0798d11c"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Add a temporary `captain_disable_auto_resolve` boolean setting on
accounts to prevent Captain from resolving conversations. Guards both
the scheduled resolution job and the assistant's resolve tool.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
# Pull Request Template
## Description
Adds a new built-in tool that allows Captain scenarios to resolve
conversations programmatically. This enables automated workflows like
the misdirected contact deflector to close conversations after handling
them, while still allowing human review via label filtering.
## Type of change
Please delete options that are not relevant.
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
tested by mentioning it to be used in captain v2 scenario
<img width="1180" height="828" alt="image"
src="https://github.com/user-attachments/assets/e70baf96-0c70-407e-af2c-328500ac5434"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Tanmay Deep Sharma <32020192+tds-1@users.noreply.github.com>
- We previously had this spec inside the enterprise folder which wouldn't be picked up by our build process, so moving to the correct folder instead.
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
