refactor: introduce normalized Captain lifecycle events [CW-7792] (#15213)

This introduces a small event layer for the Captain V2 conversation
lifecycle. A new `Captain::ConversationEvents` facade dispatches five
normalized events (`captain.conversation.engaged`,
`captain.conversation.handed_off`, `captain.conversation.resolved`,
`captain.response.completed`, `captain.response.failed`) from the points
where Captain engages a conversation, replies, fails, hands off, or
auto-resolves. Each event carries the conversation, assistant,
timestamp, and a `source`/`reason_category` where relevant.

## Why this, why now

The Captain V2 flow is about to gain several observers at once:
conversation outcome tracking, agent session capture, and analytics all
need to know when Captain engages, replies, fails, hands off, or
resolves. Wiring each of them directly into `ResponseBuilderJob`,
`HookExecutionService`, and the tools would tangle secondary bookkeeping
into the paths that deliver customer-facing behavior, and every future
consumer would deepen that. Landing the event layer first as its own PR
means the flow announces these moments once and stays otherwise
untouched: customer-visible behavior (messages, status changes,
handoffs, usage enforcement) remains synchronous, while secondary
effects subscribe through listeners. The upcoming conversation outcomes
PR then reduces to a listener plus a model instead of another round of
edits to the core flow, which is why this ships now, before that work
merges.

The existing inference reporting behavior is folded into this layer: the
`conversation.captain_inference_*` events and their dispatch helpers on
`Enterprise::Conversation` are removed, and a dedicated
`Captain::ReportingEventListener` (registered on the enterprise async
dispatcher) maps `source: 'inference'` events to the same stored
reporting event names, so recorded analytics and the assistant stats
builder are unaffected.

## What changed
- New `Captain::ConversationEvents` facade and event type constants
- Event emission from `HookExecutionService` (engagement, usage-limit
handoff), `ResponseBuilderJob` (response completed/failed,
generation-failure handoff), `HandoffTool` (tool handoff), and
`InboxPendingConversationsResolutionJob` (inference resolved/handoff)
- A dedicated `Captain::ReportingEventListener` preserves inference
reporting events through the new event names, removing captain logic
from the OSS listener

enterprise/app/dispatchers/enterprise/async_dispatcher.rb

@@ -1,7 +1,8 @@
 module Enterprise::AsyncDispatcher
   def listeners
     super + [
-      CaptainListener.instance
+      CaptainListener.instance,
+      Captain::ReportingEventListener.instance
     ]
   end
 end

enterprise/app/jobs/captain/conversation/response_builder_job.rb

@@ -1,6 +1,7 @@
 class Captain::Conversation::ResponseBuilderJob < ApplicationJob
   include Captain::Conversation::V1ActionClassifier
   include Captain::Conversation::V1FalsePromiseHandler
+  include Captain::Conversation::V2LifecycleEvents
   include Captain::Conversation::MessageBuilder
 
   MAX_MESSAGE_LENGTH = 10_000
@@ -64,20 +65,30 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
   end
 
   def process_response
+    # The V2 runner rescues its own generation errors and signals them via an error
+    # response instead of raising, so the failure event must be emitted here — the
+    # top-level handle_error path only sees exceptions raised outside the runner.
+    record_v2_response_failure(@response['error_reason']) if v2_generation_errored?
+
     if v2_handoff_tool_fired?
       process_v2_handoff_response
     elsif v1_handoff_requested?
-      # V1 only signals via the response string — no state has been touched yet. If
-      # the conversation isn't pending anymore, a human took over mid-run; bail out
-      # rather than posting a stale handoff message on top of their reply.
-      return unless conversation_pending?
-
-      process_v1_handoff
+      process_v1_handoff_request
     elsif conversation_pending?
       process_standard_response
     end
   end
 
+  def process_v1_handoff_request
+    # V1 only signals via the response string — no state has been touched yet. If
+    # the conversation isn't pending anymore, a human took over mid-run; bail out
+    # rather than posting a stale handoff message on top of their reply.
+    return unless conversation_pending?
+
+    process_v1_handoff
+    record_v2_failure_handoff if v2_generation_errored?
+  end
+
   def process_standard_response
     message = nil
     ActiveRecord::Base.transaction do
@@ -90,6 +101,7 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
     return unless message
 
     capture_assistant_session(result_message: message, credits_consumed: 1.0)
+    record_v2_response_completed(message) if captain_v2_enabled?
   end
 
   def process_v2_handoff_response
@@ -98,6 +110,10 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
     if captain_v2_enabled?
       return unless v2_handoff_tool_completed? || conversation_pending?
 
+      # Known gap, accepted for now: the fallback V1 handoff (tool fired but never
+      # completed) emits no captain.conversation.handed_off event — the tool emits
+      # only after a successful bot_handoff!. If outcome data ever needs it, emit
+      # here with a distinct source such as 'tool_fallback'.
       v2_handoff_tool_completed? ? process_v2_handoff : process_v1_handoff
     else
       conversation_pending? ? process_v1_handoff : process_v2_handoff
@@ -173,10 +189,19 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
     @response ||= {}
     @response['action_source'] ||= 'error'
     @response['action_reason'] ||= error_action_reason(error)
-    process_v1_handoff if conversation_pending? && (!captain_v2_enabled? || !newer_customer_message_arrived?)
+    record_v2_response_failure(error_action_reason(error)) if captain_v2_enabled?
+    process_error_handoff
     true
   end
 
+  def process_error_handoff
+    return unless conversation_pending?
+    return if captain_v2_enabled? && newer_customer_message_arrived?
+
+    process_v1_handoff
+    record_v2_failure_handoff if captain_v2_enabled?
+  end
+
   def log_error(error)
     ChatwootExceptionTracker.new(error, account: account).capture_exception
   end

enterprise/app/jobs/captain/conversation/v2_lifecycle_events.rb

@@ -0,0 +1,30 @@
+module Captain::Conversation::V2LifecycleEvents
+  private
+
+  def v2_generation_errored?
+    captain_v2_enabled? && @response['error'].present?
+  end
+
+  def record_v2_response_completed(message)
+    Captain::ConversationEvents.response_completed(
+      conversation: @conversation,
+      assistant: @assistant,
+      message: message,
+      at: Time.current
+    )
+  end
+
+  def record_v2_response_failure(reason)
+    Captain::ConversationEvents.response_failed(conversation: @conversation, assistant: @assistant, reason: reason, at: Time.current)
+  end
+
+  def record_v2_failure_handoff
+    Captain::ConversationEvents.handed_off(
+      conversation: @conversation,
+      assistant: @assistant,
+      source: Captain::ConversationEvents::Sources::GENERATION_FAILURE,
+      reason_category: :tool_failure,
+      at: Time.current
+    )
+  end
+end

enterprise/app/jobs/captain/inbox_pending_conversations_resolution_job.rb

@@ -28,6 +28,12 @@ class Captain::InboxPendingConversationsResolutionJob < ApplicationJob
     resolvable_pending_conversations(inbox).each do |conversation|
       create_resolution_message(conversation, inbox)
       conversation.resolved!
+      Captain::ConversationEvents.resolved(
+        conversation: conversation,
+        assistant: inbox.captain_assistant,
+        source: Captain::ConversationEvents::Sources::TIME_BASED,
+        at: Time.current
+      )
     end
   end
 
@@ -77,7 +83,12 @@ class Captain::InboxPendingConversationsResolutionJob < ApplicationJob
       reason: CAPTAIN_INFERENCE_RESOLVE_ACTIVITY_REASON,
       reason_type: :inference
     ) { conversation.resolved! }
-    conversation.dispatch_captain_inference_resolved_event
+    Captain::ConversationEvents.resolved(
+      conversation: conversation,
+      assistant: inbox.captain_assistant,
+      source: Captain::ConversationEvents::Sources::INFERENCE,
+      at: Time.current
+    )
   end
 
   def handoff_conversation(conversation, inbox, reason)
@@ -87,7 +98,13 @@ class Captain::InboxPendingConversationsResolutionJob < ApplicationJob
       reason: CAPTAIN_INFERENCE_HANDOFF_ACTIVITY_REASON,
       reason_type: :inference
     ) { conversation.bot_handoff! }
-    conversation.dispatch_captain_inference_handoff_event
+    Captain::ConversationEvents.handed_off(
+      conversation: conversation,
+      assistant: inbox.captain_assistant,
+      source: Captain::ConversationEvents::Sources::INFERENCE,
+      reason_category: :pending_clarification,
+      at: Time.current
+    )
     send_out_of_office_message_if_applicable(conversation.reload)
   end
 

enterprise/app/listeners/captain/reporting_event_listener.rb

@@ -0,0 +1,27 @@
+class Captain::ReportingEventListener < BaseListener
+  def captain_conversation_handed_off(event)
+    create_captain_inference_event(event, 'conversation_captain_inference_handoff') if event.data[:source] == 'inference'
+  end
+
+  def captain_conversation_resolved(event)
+    create_captain_inference_event(event, 'conversation_captain_inference_resolved') if event.data[:source] == 'inference'
+  end
+
+  private
+
+  def create_captain_inference_event(event, event_name)
+    conversation = extract_conversation_and_account(event)[0]
+    time_to_event = event.timestamp.to_i - conversation.created_at.to_i
+
+    ReportingEvent.create!(
+      name: event_name,
+      value: time_to_event,
+      account_id: conversation.account_id,
+      inbox_id: conversation.inbox_id,
+      user_id: conversation.assignee_id,
+      conversation_id: conversation.id,
+      event_start_time: conversation.created_at,
+      event_end_time: event.timestamp
+    )
+  end
+end

enterprise/app/models/enterprise/conversation.rb

@@ -1,14 +1,6 @@
 module Enterprise::Conversation
   attr_accessor :captain_activity_reason, :captain_activity_reason_type
 
-  def dispatch_captain_inference_resolved_event
-    dispatch_captain_inference_event(Events::Types::CONVERSATION_CAPTAIN_INFERENCE_RESOLVED)
-  end
-
-  def dispatch_captain_inference_handoff_event
-    dispatch_captain_inference_event(Events::Types::CONVERSATION_CAPTAIN_INFERENCE_HANDOFF)
-  end
-
   def list_of_keys
     super + %w[sla_policy_id]
   end
@@ -50,10 +42,6 @@ module Enterprise::Conversation
     current_applied_sla.update!(completed_at: resolved? ? Time.current : nil)
   end
 
-  def dispatch_captain_inference_event(event_name)
-    dispatcher_dispatch(event_name)
-  end
-
   def call_attributes_changed?
     return false if previous_changes['additional_attributes'].blank?
 

enterprise/app/services/captain/assistant/agent_runner_service.rb

@@ -34,7 +34,7 @@ class Captain::Assistant::AgentRunnerService
     Rails.logger.error "[Captain V2] AgentRunnerService error: #{e.message}"
     Rails.logger.error e.backtrace.join("\n")
 
-    error_response(e.message)
+    error_response(e)
   end
 
   def response_discarded? = @response_discarded == true
@@ -132,10 +132,12 @@ class Captain::Assistant::AgentRunnerService
     @message_length_limit ||= Captain::MessageLengthLimit.for(@conversation)
   end
 
-  def error_response(error_message)
+  def error_response(error)
     {
       'response' => 'conversation_handoff',
-      'reasoning' => "Error occurred: #{error_message}",
+      'reasoning' => "Error occurred: #{error.message}",
+      'error' => true,
+      'error_reason' => error.class.name.underscore.tr('/', '_'),
       'handoff_tool_called' => @handoff_tool_called
     }
   end

enterprise/app/services/captain/conversation_events.rb

@@ -0,0 +1,72 @@
+class Captain::ConversationEvents
+  module Sources
+    TOOL = 'tool'.freeze
+    GENERATION_FAILURE = 'generation_failure'.freeze
+    TIME_BASED = 'time_based'.freeze
+    INFERENCE = 'inference'.freeze
+    USAGE_LIMIT = 'usage_limit'.freeze
+  end
+
+  class << self
+    def engaged(conversation:, assistant:, at:)
+      dispatch(
+        Events::Types::CAPTAIN_CONVERSATION_ENGAGED,
+        at: at,
+        conversation: conversation,
+        assistant: assistant
+      )
+    end
+
+    def handed_off(conversation:, assistant:, source:, at:, reason_category: nil)
+      dispatch(
+        Events::Types::CAPTAIN_CONVERSATION_HANDED_OFF,
+        at: at,
+        conversation: conversation,
+        assistant: assistant,
+        source: source,
+        reason_category: reason_category
+      )
+    end
+
+    def resolved(conversation:, assistant:, source:, at:)
+      dispatch(
+        Events::Types::CAPTAIN_CONVERSATION_RESOLVED,
+        at: at,
+        conversation: conversation,
+        assistant: assistant,
+        source: source
+      )
+    end
+
+    def response_completed(conversation:, assistant:, message:, at:)
+      dispatch(
+        Events::Types::CAPTAIN_RESPONSE_COMPLETED,
+        at: at,
+        conversation: conversation,
+        assistant: assistant,
+        message: message
+      )
+    end
+
+    def response_failed(conversation:, assistant:, reason:, at:)
+      dispatch(
+        Events::Types::CAPTAIN_RESPONSE_FAILED,
+        at: at,
+        conversation: conversation,
+        assistant: assistant,
+        reason: reason
+      )
+    end
+
+    private
+
+    # Lifecycle events are secondary effects: a dispatch failure must never alter
+    # the customer-facing Captain flow (blocking a response from being scheduled,
+    # or turning an already-delivered reply into a spurious handoff).
+    def dispatch(event_name, at:, **data)
+      Rails.configuration.dispatcher.dispatch(event_name, at, data)
+    rescue StandardError => e
+      ChatwootExceptionTracker.new(e, account: data[:conversation]&.account).capture_exception
+    end
+  end
+end

enterprise/app/services/enterprise/message_templates/hook_execution_service.rb

@@ -6,6 +6,7 @@ module Enterprise::MessageTemplates::HookExecutionService
     return unless should_process_captain_response?
     return perform_handoff unless inbox.captain_active?
 
+    track_captain_engagement
     schedule_captain_response
   end
 
@@ -29,6 +30,20 @@ module Enterprise::MessageTemplates::HookExecutionService
 
   private
 
+  def track_captain_engagement
+    return unless captain_v2_enabled?
+
+    Captain::ConversationEvents.engaged(
+      conversation: conversation,
+      assistant: inbox.captain_assistant,
+      at: message.created_at
+    )
+  end
+
+  def captain_v2_enabled?
+    conversation.account.feature_enabled?('captain_integration_v2')
+  end
+
   def schedule_captain_response
     job_args = [conversation, conversation.inbox.captain_assistant]
     captain_v2_enabled = conversation.account.feature_enabled?('captain_integration_v2')
@@ -81,6 +96,15 @@ module Enterprise::MessageTemplates::HookExecutionService
       content: 'Transferring to another agent for further assistance.'
     )
     conversation.bot_handoff!
+    if captain_v2_enabled?
+      Captain::ConversationEvents.handed_off(
+        conversation: conversation,
+        assistant: inbox.captain_assistant,
+        source: Captain::ConversationEvents::Sources::USAGE_LIMIT,
+        reason_category: :usage_limit,
+        at: Time.current
+      )
+    end
     send_out_of_office_message_after_handoff
   end
 

enterprise/lib/captain/tools/handoff_tool.rb

@@ -54,6 +54,7 @@ class Captain::Tools::HandoffTool < Captain::Tools::BasePublicTool
     tool_context.state[:captain_v2_handoff_tool_completed] = true
     # Queue the event after the state change commits so notification jobs always see the open conversation.
     conversation.dispatch_bot_handoff_event
+    emit_tool_handoff_event(conversation)
 
     # Send out of office message if applicable (since template messages were suppressed while Captain was handling)
     send_out_of_office_message_if_applicable(conversation)
@@ -67,10 +68,16 @@ class Captain::Tools::HandoffTool < Captain::Tools::BasePublicTool
     )
     record_handoff_note(tool_context, note) if reason.present?
     conversation.bot_handoff!
+    emit_tool_handoff_event(conversation)
     send_out_of_office_message_if_applicable(conversation)
     :completed
   end
 
+  def emit_tool_handoff_event(conversation)
+    Captain::ConversationEvents.handed_off(conversation: conversation, assistant: @assistant,
+                                           source: Captain::ConversationEvents::Sources::TOOL, at: Time.current)
+  end
+
   def record_handoff_note(tool_context, note)
     metadata = tool_context.state[:cw_metadata] ||= {}
     metadata[:handoff_note_id] = note.id

enterprise/lib/captain/tools/resolve_conversation_tool.rb

@@ -11,6 +11,8 @@ class Captain::Tools::ResolveConversationTool < Captain::Tools::BasePublicTool
     log_tool_usage('resolve_conversation', { conversation_id: conversation.id, reason: reason })
 
     conversation.with_captain_activity_context(reason: reason, reason_type: :tool) { conversation.resolved! }
+    Captain::ConversationEvents.resolved(conversation: conversation, assistant: @assistant,
+                                         source: Captain::ConversationEvents::Sources::TOOL, at: Time.current)
 
     "Conversation ##{conversation.display_id} resolved#{" (Reason: #{reason})" if reason}"
   end