feat(voice): transcribe Twilio call recordings (#15241)

Twilio voice calls now get an AI transcript alongside the recording.
Once a call ends and its recording is stored, we transcribe it and show
the text under the audio player in the call bubble — the same experience
WhatsApp voice notes already have. Transcription runs on Captain and
consumes Captain response credits, so it only kicks in for accounts with
Captain enabled and audio transcriptions turned on.

## How to test

1. On an account with Captain enabled and Settings → Account → Audio
transcriptions on, make a call on a Twilio voice inbox and hang up.
2. Open the conversation. The voice call bubble shows the recording
player once Twilio delivers the recording.
3. Shortly after, the transcript appears under the player — no refresh
needed.
4. Turn audio transcriptions off (or exhaust Captain credits) and
repeat: the recording still appears, the transcript does not.

## What changed

- `Llm::SpeechToTextService` (new) — blob-in/text-out transcription
engine extracted from `Messages::AudioTranscriptionService`: size limit,
temp-file download, model resolution via `Llm::FeatureRouter`, the
OpenAI call, and Captain credit accounting. `.available_for?` holds the
shared gate.
- `Messages::AudioTranscriptionService` — now a thin wrapper over that
engine; its public contract is unchanged, so
`Captain::OpenAiMessageBuilderService` is unaffected.
- `Voice::CallTranscriptionService` / `Voice::CallTranscriptionJob`
(new) — transcribe `call.recording` into `calls.transcript`, then
rebroadcast the message so clients pick it up over the wire.
- `Voice::Provider::Twilio::RecordingAttachmentService` — enqueues the
job after the recording is attached.

The API and frontend needed no changes: `calls.transcript` already
existed, `_call.json.jbuilder` already serialized it, and
`VoiceCall.vue` already fed it to the audio chip. Nothing had ever
written the column.

Also wires `instrument_audio_transcription`, which existed but was never
called, so both transcription paths now emit LLM spans.

enterprise/app/jobs/voice/call_transcription_job.rb

@@ -0,0 +1,23 @@
+class Voice::CallTranscriptionJob < ApplicationJob
+  queue_as :low
+
+  # A recording OpenAI rejects (corrupt/unsupported audio) or credentials it refuses
+  # will never succeed on retry — drop the job instead of hammering the API.
+  discard_on Faraday::BadRequestError, Faraday::UnauthorizedError do |job, error|
+    log_context = {
+      call_id: job.arguments.first,
+      job_id: job.job_id,
+      status_code: error.response&.dig(:status)
+    }
+
+    Rails.logger.warn("Discarding call transcription job: #{log_context}")
+  end
+  retry_on ActiveStorage::FileNotFoundError, wait: 2.seconds, attempts: 3
+
+  def perform(call_id)
+    call = Call.find_by(id: call_id)
+    return if call.blank?
+
+    Voice::CallTranscriptionService.new(call: call).perform
+  end
+end

enterprise/app/presenters/enterprise/messages/search_data_presenter.rb

@@ -0,0 +1,11 @@
+module Enterprise::Messages::SearchDataPresenter
+  # Voice-call recordings hang off the Call, not off message attachments, so the
+  # transcript is folded into attachments.transcribed_text — the field search
+  # already queries for audio-message transcriptions.
+  def attachment_data
+    transcript = call&.transcript if content_type == 'voice_call'
+    return super if transcript.blank?
+
+    (super || []) + [{ transcribed_text: transcript }]
+  end
+end

enterprise/app/services/llm/speech_to_text_service.rb

@@ -0,0 +1,103 @@
+# Blob-in, text-out audio transcription shared by voice-note attachments
+# (Messages::AudioTranscriptionService) and voice-call recordings
+# (Voice::CallTranscriptionService).
+class Llm::SpeechToTextService < Llm::LegacyBaseOpenAiService
+  include Integrations::LlmInstrumentation
+
+  # OpenAI's transcription endpoint hard limit is 25 MB *decimal* (25_000_000), not
+  # binary (25.megabytes = 26_214_400) — using the binary form leaks the 25.0–26.2 MB
+  # range to the API as 413s. Long audio (~70+ min Opus) keeps the source audio but
+  # skips transcription.
+  BYTE_LIMIT = 25_000_000
+
+  attr_reader :blob, :account, :transcription_model
+
+  # Transcription runs on Captain's OpenAI credentials and consumes its response credits.
+  def self.available_for?(account)
+    return false unless account.feature_enabled?('captain_integration')
+    return false if account.audio_transcriptions.blank?
+
+    account.usage_limits[:captain][:responses][:current_available].positive?
+  end
+
+  def self.too_large?(blob)
+    blob.present? && blob.byte_size > BYTE_LIMIT
+  end
+
+  def initialize(blob:, account:)
+    super()
+    @blob = blob
+    @account = account
+    @transcription_model = Llm::FeatureRouter.resolve(feature: 'audio_transcription', account: account)[:model]
+  end
+
+  def perform
+    temp_file_path = fetch_audio_file
+    transcribed_text = nil
+
+    File.open(temp_file_path, 'rb') do |file|
+      transcribed_text = instrument_audio_transcription(instrumentation_params(temp_file_path)) do
+        # temperature: 0.0 minimises hallucinations on silence / near-silent
+        # audio; non-zero values trigger spiraling repeats — well-documented
+        # behaviour across OpenAI transcription models.
+        response = @client.audio.transcribe(
+          parameters: {
+            model: transcription_model,
+            file: file,
+            temperature: 0.0
+          }
+        )
+        response['text']
+      end
+    end
+
+    account.increment_response_usage if transcribed_text.present?
+    transcribed_text
+  ensure
+    FileUtils.rm_f(temp_file_path) if temp_file_path.present?
+  end
+
+  private
+
+  def fetch_audio_file
+    temp_dir = Rails.root.join('tmp/uploads/audio-transcriptions')
+    FileUtils.mkdir_p(temp_dir)
+    temp_file_name = "#{blob.key}-#{blob.filename}"
+
+    if blob.filename.extension_without_delimiter.blank?
+      extension = extension_from_content_type(blob.content_type)
+      temp_file_name = "#{temp_file_name}.#{extension}" if extension.present?
+    end
+
+    temp_file_path = File.join(temp_dir, temp_file_name)
+
+    File.open(temp_file_path, 'wb') do |file|
+      blob.open do |blob_file|
+        IO.copy_stream(blob_file, file)
+      end
+    end
+
+    temp_file_path
+  end
+
+  def extension_from_content_type(content_type)
+    subtype = content_type.to_s.downcase.split(';').first.to_s.split('/').last.to_s
+    return if subtype.blank?
+
+    {
+      'x-m4a' => 'm4a',
+      'x-wav' => 'wav',
+      'x-mp3' => 'mp3'
+    }.fetch(subtype, subtype)
+  end
+
+  def instrumentation_params(file_path)
+    {
+      span_name: 'llm.messages.audio_transcription',
+      model: transcription_model,
+      account_id: account&.id,
+      feature_name: 'audio_transcription',
+      file_path: file_path
+    }
+  end
+end

enterprise/app/services/messages/audio_transcription_service.rb

@@ -1,26 +1,16 @@
-class Messages::AudioTranscriptionService< Llm::LegacyBaseOpenAiService
-  include Integrations::LlmInstrumentation
-
-  # OpenAI's transcription endpoint hard limit is 25 MB *decimal* (25_000_000), not
-  # binary (25.megabytes = 26_214_400) — using the binary form leaks the 25.0–26.2 MB
-  # range to the API as 413s. Long audio (~70+ min Opus) keeps the attachment but skips
-  # transcription.
-  TRANSCRIPTION_BYTE_LIMIT = 25_000_000
-
-  attr_reader :attachment, :message, :account, :transcription_model
+class Messages::AudioTranscriptionService
+  attr_reader :attachment, :message, :account
 
   def initialize(attachment)
-    super()
     @attachment = attachment
     @message = attachment.message
-    @account = message.account
-    @transcription_model = Llm::FeatureRouter.resolve(feature: 'audio_transcription', account: account)[:model]
+    @account = message&.account
   end
 
   def perform
-    return { error: 'Transcription limit exceeded' } unless can_transcribe?
     return { error: 'Message not found' } if message.blank?
-    return { error: 'Audio too large for Whisper' } if audio_too_large?
+    return { error: 'Transcription limit exceeded' } unless Llm::SpeechToTextService.available_for?(account)
+    return { error: 'Audio too large for transcription' } if Llm::SpeechToTextService.too_large?(attachment.file&.blob)
 
     transcriptions = transcribe_audio
     Rails.logger.info "Audio transcription successful: #{transcriptions}"
@@ -32,77 +22,13 @@ class Messages::AudioTranscriptionService< Llm::LegacyBaseOpenAiService
 
   private
 
-  def can_transcribe?
-    return false unless account.feature_enabled?('captain_integration')
-    return false if account.audio_transcriptions.blank?
-
-    account.usage_limits[:captain][:responses][:current_available].positive?
-  end
-
-  def audio_too_large?
-    blob = attachment.file&.blob
-    return false unless blob
-
-    blob.byte_size > TRANSCRIPTION_BYTE_LIMIT
-  end
-
-  def fetch_audio_file
-    blob = attachment.file.blob
-    temp_dir = Rails.root.join('tmp/uploads/audio-transcriptions')
-    FileUtils.mkdir_p(temp_dir)
-    temp_file_name = "#{blob.key}-#{blob.filename}"
-
-    if blob.filename.extension_without_delimiter.blank?
-      extension = extension_from_content_type(blob.content_type)
-      temp_file_name = "#{temp_file_name}.#{extension}" if extension.present?
-    end
-
-    temp_file_path = File.join(temp_dir, temp_file_name)
-
-    File.open(temp_file_path, 'wb') do |file|
-      blob.open do |blob_file|
-        IO.copy_stream(blob_file, file)
-      end
-    end
-
-    temp_file_path
-  end
-
   def transcribe_audio
     transcribed_text = attachment.meta&.[]('transcribed_text') || ''
     return transcribed_text if transcribed_text.present?
 
-    temp_file_path = fetch_audio_file
-    transcribed_text = nil
-
-    File.open(temp_file_path, 'rb') do |file|
-      # temperature: 0.0 minimises hallucinations on silence / near-silent
-      # audio; non-zero values trigger spiraling repeats — well-documented
-      # behaviour across OpenAI transcription models.
-      response = @client.audio.transcribe(
-        parameters: {
-          model: transcription_model,
-          file: file,
-          temperature: 0.0
-        }
-      )
-      transcribed_text = response['text']
-    end
-
+    transcribed_text = Llm::SpeechToTextService.new(blob: attachment.file.blob, account: account).perform
     update_transcription(transcribed_text)
     transcribed_text
-  ensure
-    FileUtils.rm_f(temp_file_path) if temp_file_path.present?
-  end
-
-  def instrumentation_params(file_path)
-    {
-      span_name: 'llm.messages.audio_transcription',
-      model: transcription_model,
-      account_id: account&.id,
-      feature_name: 'audio_transcription',
-      file_path: file_path
-    }
   end
 
   def update_transcription(transcribed_text)
@@ -110,21 +36,9 @@ class Messages::AudioTranscriptionService< Llm::LegacyBaseOpenAiService
 
     attachment.update!(meta: { transcribed_text: transcribed_text })
     message.reload.send_update_event
-    message.account.increment_response_usage
 
     return unless ChatwootApp.advanced_search_allowed?
 
     message.reindex
   end
-
-  def extension_from_content_type(content_type)
-    subtype = content_type.to_s.downcase.split(';').first.to_s.split('/').last.to_s
-    return if subtype.blank?
-
-    {
-      'x-m4a' => 'm4a',
-      'x-wav' => 'wav',
-      'x-mp3' => 'mp3'
-    }.fetch(subtype, subtype)
-  end
 end

enterprise/app/services/voice/call_transcription_service.rb

@@ -0,0 +1,37 @@
+class Voice::CallTranscriptionService
+  pattr_initialize [:call!]
+
+  def perform
+    # Split so a publish failure can retry without re-running (and re-charging) transcription.
+    transcribe if call.transcript.blank?
+    publish(call.message) if call.transcript.present?
+  end
+
+  private
+
+  def transcribe
+    return unless call.recording.attached?
+    return unless Llm::SpeechToTextService.available_for?(call.account)
+    return if Llm::SpeechToTextService.too_large?(recording_blob)
+
+    transcript = Llm::SpeechToTextService.new(blob: recording_blob, account: call.account).perform
+    call.update!(transcript: transcript) if transcript.present?
+  end
+
+  def publish(message)
+    return if message.blank?
+
+    # Reindex before broadcasting: if reindexing fails and the job retries,
+    # the transcript is already present so only publish reruns. Sending the
+    # update event first would resend it to clients on every such retry.
+    message.reindex if ChatwootApp.advanced_search_allowed?
+
+    # Rebroadcast the message so connected clients pick up the embedded Call
+    # payload (now with transcript) without a refetch.
+    message.reload.send_update_event
+  end
+
+  def recording_blob
+    call.recording.blob
+  end
+end

enterprise/app/services/voice/provider/twilio/recording_attachment_service.rb

@@ -19,6 +19,10 @@ class Voice::Provider::Twilio::RecordingAttachmentService
     # Bump the message updated_at so the message.updated dispatcher rebroadcasts
     # the embedded Call payload (now with recording_url) to connected clients.
     call.message&.touch # rubocop:disable Rails/SkipsModelValidations
+
+    # Duplicate callbacks can both clear the outer already_attached? check, so only
+    # the invocation that actually stored the blob pays for transcription.
+    Voice::CallTranscriptionJob.perform_later(call.id) if @persisted
   end
 
   private
@@ -31,6 +35,7 @@ class Voice::Provider::Twilio::RecordingAttachmentService
       call.recording_sid = recording_sid
       call.duration_seconds ||= normalized_recording_duration
       call.save!
+      @persisted = true
     end
   end