从按住说话到离线转写：H5 录音 Hook 实践

我血条子呢

304人浏览 · 2026-08-17 15:38:50

我血条子呢 · 2026-08-17 15:38:50 发布

从按住说话到离线转写：H5 录音 Hook 实践

这套方案跑在 H5 里，也适用于微信小程序通过 web-view 嵌套的 H5 页面。录音、生成 WAV、上传识别都在 H5 内完成，不依赖小程序原生录音 API，也不需要小程序与 H5 之间实时传音频。小程序只负责承载页面；H5 需部署到 HTTPS，并在小程序后台配置对应的业务域名。真机验证以微信里打开为准，开发者工具的麦克风模拟不能替代。

聊天页需要「按住说话」。用户按住麦克风按钮开始录音，松手后把音频交给后端做离线语音识别，识别文字再作为问题发出去；上滑则取消，不提交。

整条链路拆成两层：

VoiceInput 只处理手势和提示文案
useOfflineVoiceTranscription 负责麦克风、PCM、WAV 和 ASR

本文介绍这个 Hook 的设计，以及聊天输入如何把它接到「按住说话」上。

1. 为什么不把录音写在按钮里

录音不是一次 getUserMedia 就能结束的同步操作。开始之后要持续收 PCM；停止时要拼文件、上传、等 SSE 结果；中途还可能取消、切页、权限被拒。这些状态如果散落在按钮组件里，手势逻辑和音频资源会缠在一起。

所以按钮只发出三个意图：

用户动作	按钮回调	Hook 行为
按下	`onVoiceStart`	`startRecording()`
松手且未上滑	`onVoiceSubmit`	`stopRecordingAndTranscribe()`
上滑松手 / 指针丢失	`onVoiceCancel`	`cancelRecording()`

识别成功后的「把文字当问题发出去」也不在 Hook 里，由 ChatInput 监听 status === 'success' 再调用 onSend。Hook 只产出转写结果，不决定业务怎么用。

2. 状态机

Hook 对外暴露一个明确的状态，而不是一堆互相覆盖的 boolean：

idle → requesting → recording → transcribing → success
                                              ↘ error
任意进行中状态都可以回到 idle（取消 / 重置）

状态	含义
`idle`	空闲
`requesting`	正在要麦克风权限
`recording`	正在采集 PCM
`transcribing`	WAV 已生成，正在走 ASR SSE
`success`	拿到转写文本
`error`	权限失败、没采到音频、识别失败等

组件侧常用派生值：

isStarting: status === 'requesting'
isRecording: status === 'recording'
isTranscribing: status === 'transcribing'

requesting 单独拆出来，是因为微信 WebView 里授权弹窗可能要等几秒。按钮在这段时间要禁用，界面要提示「正在请求录音权限」，不能假装已经开始录音。

3. 开始录音

startRecording 必须由用户手势直接触发。浏览器和微信 WebView 都不会在没有用户动作时弹出麦克风授权。

开始前先做能力检查：

navigator.mediaDevices.getUserMedia 是否存在
是否处于安全上下文（HTTPS；localhost / 127.0.0.1 除外）
AudioContext 或 webkitAudioContext 是否可用
当前是否已经在 requesting / recording / transcribing，避免重复开始

通过后再按这个顺序初始化：

getUserMedia({ audio: true })
  → new AudioContext()
  → 若 state === 'suspended'，先 resume
  → MediaStreamSource
  → ScriptProcessor(4096, 1, 1)
  → Gain(0) 接到 destination

顺序不能反。先创建 AudioContext 再要麦克风，在部分 WebView 里会一直停在 suspended，采集不到数据。getUserMedia 成功后再 resume，才能把「用户刚点了按钮」这个手势带到音频图上。

麦克风流拿到之后立刻创建音频图，不要插入其它异步步骤。授权成功到真正开始采集之间拖得越久，越容易遇到页面切到后台、用户已经松手取消等情况。

4. 采集 PCM，但不回放麦克风

采集用 ScriptProcessorNode，在 onaudioprocess 里把单声道 Float32Array 拷出来存进缓冲区：

processor.onaudioprocess = event => {
  audioSamplesRef.current.push(new Float32Array(event.inputBuffer.getChannelData(0)))
}

必须拷贝一份。getChannelData 返回的是可复用底层缓冲，下一帧会被覆盖。

ScriptProcessor 还要接到输出图上才会持续回调。如果直接接到扬声器，用户会听到自己的声音。所以中间加一个增益为 0 的 GainNode：

MediaStreamSource → ScriptProcessor → Gain(0) → destination

节点活着、回调继续，麦克风声音不会漏到听筒。

这里没有用 AudioWorklet。聊天 H5 要跑在微信小程序 web-view 里，ScriptProcessor 的兼容面更稳。它已经标记为废弃，但对当前运行环境仍然是更务实的选择。

5. 停止：降采样、写 WAV、走 SSE

松手提交时，Hook 做四件事：

把分片 PCM 拼成一段连续的 Float32Array
按设备采样率降到 16 kHz
写成单声道、16-bit PCM WAV
用现有 SSE 封装上传到 /asr/offline/transcribe

设备采样率通常是 44.1 kHz 或 48 kHz，ASR 约定是 16 kHz。降采样用区间平均，而不是简单抽点，毛刺会少一些：

const TARGET_SAMPLE_RATE = 16000

function downsample(samples: Float32Array, sourceSampleRate: number) {
  if (sourceSampleRate === TARGET_SAMPLE_RATE) return samples

  const sampleCount = Math.round((samples.length * TARGET_SAMPLE_RATE) / sourceSampleRate)
  const result = new Float32Array(sampleCount)
  const sampleRatio = sourceSampleRate / TARGET_SAMPLE_RATE

  for (let index = 0; index < sampleCount; index += 1) {
    const start = Math.floor(index * sampleRatio)
    const end = Math.min(Math.floor((index + 1) * sampleRatio), samples.length)
    let total = 0
    for (let sourceIndex = start; sourceIndex < end; sourceIndex += 1) {
      total += samples[sourceIndex]
    }
    result[index] = total / Math.max(end - start, 1)
  }

  return result
}

WAV 头部按 PCM 规范写：RIFF / WAVE / fmt / data，采样率 16000，声道 1，位深 16。Float32 先钳到 [-1, 1]，再乘 0x7fff 写成 Int16。

文件交给 startOfflineTranscription。它把 File 放进 FormData，走项目统一的 createSseConnection，不要自己再开一条 axios / fetchEventSource。SSE 事件大致是：

事件	含义	Hook 处理
`submitted`	任务已受理，带 `aid`	记下 `taskId`
`result`	识别完成，`lattices[].onebest`	拼成 `transcription`，状态改为 `success`
`error`	上游失败	记下 `error`，状态改为 `error`

连接正常结束却还停在 transcribing，说明没收到结果，这时也当成失败，避免界面一直转圈。

如果停止时 AudioContext 不存在，或一帧 PCM 都没有，不会上传空文件，直接报「未采集到音频」。

6. 取消比开始更难：attemptId

getUserMedia 是异步的。用户可能在授权弹窗还没关掉时就上滑取消，或页面已经卸载。如果回来的流不处理，麦克风指示灯会一直亮，下一次录音也可能失败。

Hook 用 recordingAttemptIdRef 给每一次开始编号。startRecording 入口 ++recordingAttemptId；cancelRecording / reset / 卸载时再加一次。麦克风流或 AudioContext.resume 回来后，发现编号已经变了，就立刻停轨、关上下文，并且不再把状态改成 recording。

const attemptId = ++recordingAttemptIdRef.current
const audioStream = await navigator.mediaDevices.getUserMedia({ audio: true })

if (attemptId !== recordingAttemptIdRef.current) {
  audioStream.getTracks().forEach(track => track.stop())
  return false
}

这比「用一个 cancelled boolean」更稳。连续点两次开始、开始后立刻取消再开始，boolean 很容易被后一次覆盖。自增编号只认「还是不是这一次尝试」。

cancelRecording 只丢掉本地音频资源，不碰已经发出去的识别请求。识别阶段要停，走 cancelTranscription 或 reset。

7. 资源必须成对释放

音频图、媒体轨、SSE 连接都是有副作用的对象。停止、取消、失败、组件卸载都走同一套释放：

去掉 AudioContext statechange 监听
disconnect ScriptProcessor / Gain
stop 所有 MediaStreamTrack
close AudioContext
close ASR SSE

漏掉任何一步，常见后果是：系统麦克风灯不灭、第二次 getUserMedia 失败、节点重复连接后出现杂音或空数据。

卸载时的 useEffect 清理函数也会 ++recordingAttemptId，避免卸载后还把异步结果写回已经不存在的组件状态。

8. 按钮：按住说话，上移取消

VoiceInput 用 Pointer Events，而不是分开处理 touch 和 mouse。按下时 setPointerCapture，这样手指滑出按钮区域后仍能收到 pointerup / pointercancel。

取消用一段滞回，避免手指在阈值附近抖动时文案来回跳：

上移超过 48px：进入 cancelling，「松开取消」
再往下移回，距离小于 32px：回到 recording，「松手发送」

松手时只看当前是不是 recording，并且 Hook 是否已经真正进入 isRecording。如果还在 requesting（权限弹窗未结束），松手一律取消，不会拿一段空音频去识别。

pointercancel 和 lostpointercapture 也当取消。系统手势、来电、WebView 抢焦点时，宁可丢掉这一段录音，也不要误提交。

识别过程中按钮禁用。WAV 上传和 SSE 等待可能要一两秒，这段时间不允许再按住说话。

9. 聊天页如何把转写变成问题

ChatInput 接好三个回调，再等成功：

<VoiceInput onVoiceStart={() => void startRecording()} onVoiceSubmit={stopRecordingAndTranscribe} onVoiceCancel={cancelRecording} />

useEffect(() => {
  if (transcriptionStatus !== 'success') {
    hasSubmittedVoiceRef.current = false
    return
  }
  if (hasSubmittedVoiceRef.current) return

  hasSubmittedVoiceRef.current = true
  const question = transcription.trim()
  if (question) onSend(question)
  resetVoiceTranscription()
}, [transcriptionStatus, transcription, onSend, resetVoiceTranscription])

hasSubmittedVoiceRef 防止 success 期间 effect 因其它依赖重跑而重复发消息。发完立刻 reset，状态回到 idle，输入条可以接下一次按住说话。

切到文字输入时也会 reset，避免语音识别还在后台跑，回来后又突然插一条消息。

10. 调试

Hook 接受 { debug: true }。打开后会在内存里保留最近 100 条带时间戳的步骤日志，例如：

startRecording 调用时的安全上下文、协议、getUserMedia 是否存在
getUserMedia 成功后的 track readyState / muted / settings
AudioContext 的 state、sampleRate
是否收到第一帧 PCM
WAV 文件大小和采样率
ASR 事件和请求异常

微信开发者工具里的麦克风经常是假的：页面显示权限 prompt，但弹窗不会落到真实设备上，最终 NotAllowedError。真机 HTTPS 才是有效结论。错误必须渲染在页面上，不能只 console.log——真机 WebView 很难看控制台。

测试页 /test 就是按这个方式接的：开启 debug、展示能力检查、预览 WAV、复制诊断日志。

11. 使用时要注意的几件事

HTTPS。真机微信里用 HTTP 打开 H5，getUserMedia 会直接失败。
合法业务域名。小程序 web-view 只能打开后台配过的 HTTPS 域名。
用户手势。startRecording 必须从 pointerdown / click 里调用，不要包一层 setTimeout。
空录音。按住时间过短可能一帧都没有，Hook 会报错而不是上传空文件。
不要把识别协议写进页面。SSE 事件解析留在 api/asrSse，Hook 只消费 submitted / result / error。
iOS 要单独验。Android 微信真机通过，不代表 iOS WebView 的权限和 AudioContext.resume 行为一致。

12. 完整实现

前面按步骤拆过关键片段。下面按文件给出可直接拷贝的完整实现，文件关系和仓库内路径一致：

src/api/asrSse.ts
src/hooks/useOfflineVoiceTranscription.ts
src/pages/chat/input/VoiceInput.tsx
src/pages/chat/input/ChatInput.tsx

复用时最少需要两块：录音 Hook 和 ASR 上传。按钮可以换成自己的 UI，但要保留 start / submit / cancel 三个时机。VoiceInput 在项目里还依赖键盘图标、停止按钮和声波动画；文中给出去掉这些依赖后的完整按钮，手势判断与源码一致。

startOfflineTranscription 在本项目里走 @/utils/sse 的 createSseConnection。若你的项目没有这层封装，用 12.2 的最小适配替换即可，返回值必须带 done 和 close()。

12.1 ASR 接口

路径：src/api/asrSse.ts

import { createSseConnection, type SseConnection } from '@/utils/sse'

export type AsrSubmittedEvent = {
  requestId: string
  aid: string
  status: 'submitted'
}

export type AsrResultEvent = {
  state: {
    ok: number
    code: number
    success: boolean
  }
  aid: string
  lattices: Array<{
    onebest: string
    begin: number
    end: number
    lid: number
    spk: number
  }>
}

export type AsrErrorEvent = {
  requestId?: string
  code: string
  message: string
}

export type AsrOfflineEvent = AsrSubmittedEvent | AsrResultEvent | AsrErrorEvent

export function parseAsrEvent(rawData: string): AsrOfflineEvent {
  return JSON.parse(rawData) as AsrOfflineEvent
}

/** 上传音频文件并接收离线转写 SSE 事件。 */
export function startOfflineTranscription(
  file: File,
  onEvent: (event: string, data: AsrOfflineEvent) => void,
  onError: (error: unknown) => number | null
): SseConnection {
  const formData = new FormData()
  formData.append('file', file)

  return createSseConnection<AsrOfflineEvent, FormData>('/asr/offline/transcribe', formData, {
    parse: parseAsrEvent,
    onMessage: message => onEvent(message.event, message.data),
    onError
  })
}

没有 createSseConnection 时，可以用 @microsoft/fetch-event-source 做成同样契约：

import { fetchEventSource } from '@microsoft/fetch-event-source'

import { parseAsrEvent, type AsrOfflineEvent } from './asrSse'

export type SseConnection = {
  readonly done: Promise<void>
  close: () => void
}

export function startOfflineTranscription(
  file: File,
  onEvent: (event: string, data: AsrOfflineEvent) => void,
  onError: (error: unknown) => number | null
): SseConnection {
  const controller = new AbortController()
  const formData = new FormData()
  formData.append('file', file)

  const done = fetchEventSource('/asr/offline/transcribe', {
    method: 'POST',
    headers: { Accept: 'text/event-stream' },
    body: formData,
    signal: controller.signal,
    openWhenHidden: true,
    async onopen(response) {
      if (!response.ok) {
        throw new Error(`SSE 请求失败（HTTP ${response.status}）`)
      }
    },
    onmessage(event) {
      onEvent(event.event, parseAsrEvent(event.data))
    },
    onerror(error) {
      if (controller.signal.aborted) return null
      return onError(error)
    }
  })

  return {
    done,
    close: () => controller.abort()
  }
}

鉴权头按自己的项目补，不要手动设置 Content-Type，让浏览器给 FormData 带 boundary。

12.2 录音 Hook

路径：src/hooks/useOfflineVoiceTranscription.ts

import { useEffect, useRef, useState } from 'react'

import { startOfflineTranscription, type AsrOfflineEvent } from '@/api/asrSse'

const TARGET_SAMPLE_RATE = 16000

export type VoiceTranscriptionStatus = 'idle' | 'requesting' | 'recording' | 'transcribing' | 'success' | 'error'

type UseOfflineVoiceTranscriptionOptions = {
  debug?: boolean
}

function stringifyDebugDetails(details: unknown) {
  if (details === undefined) return ''

  try {
    return ` ${JSON.stringify(details)}`
  } catch {
    return ` ${String(details)}`
  }
}

type WebkitWindow = Window &
  typeof globalThis & {
    webkitAudioContext?: typeof AudioContext
  }

type DocumentWithMicrophonePolicy = Document & {
  permissionsPolicy?: {
    allowsFeature?: (feature: string) => boolean
  }
  featurePolicy?: {
    allowsFeature?: (feature: string) => boolean
  }
}

function downsample(samples: Float32Array, sourceSampleRate: number) {
  if (sourceSampleRate === TARGET_SAMPLE_RATE) {
    return samples
  }

  const sampleCount = Math.round((samples.length * TARGET_SAMPLE_RATE) / sourceSampleRate)
  const result = new Float32Array(sampleCount)
  const sampleRatio = sourceSampleRate / TARGET_SAMPLE_RATE

  for (let index = 0; index < sampleCount; index += 1) {
    const start = Math.floor(index * sampleRatio)
    const end = Math.min(Math.floor((index + 1) * sampleRatio), samples.length)
    let total = 0

    for (let sourceIndex = start; sourceIndex < end; sourceIndex += 1) {
      total += samples[sourceIndex]
    }

    result[index] = total / Math.max(end - start, 1)
  }

  return result
}

function createWavFile(samples: Float32Array, sourceSampleRate: number) {
  const pcmSamples = downsample(samples, sourceSampleRate)
  const buffer = new ArrayBuffer(44 + pcmSamples.length * 2)
  const view = new DataView(buffer)
  const writeString = (offset: number, value: string) => {
    for (let index = 0; index < value.length; index += 1) {
      view.setUint8(offset + index, value.charCodeAt(index))
    }
  }

  writeString(0, 'RIFF')
  view.setUint32(4, 36 + pcmSamples.length * 2, true)
  writeString(8, 'WAVE')
  writeString(12, 'fmt ')
  view.setUint32(16, 16, true)
  view.setUint16(20, 1, true)
  view.setUint16(22, 1, true)
  view.setUint32(24, TARGET_SAMPLE_RATE, true)
  view.setUint32(28, TARGET_SAMPLE_RATE * 2, true)
  view.setUint16(32, 2, true)
  view.setUint16(34, 16, true)
  writeString(36, 'data')
  view.setUint32(40, pcmSamples.length * 2, true)

  pcmSamples.forEach((sample, index) => {
    const normalizedSample = Math.max(-1, Math.min(1, sample))
    view.setInt16(44 + index * 2, normalizedSample * 0x7fff, true)
  })

  return new Blob([buffer], { type: 'audio/wav' })
}

/**
 * 录制单声道 16 kHz、16-bit PCM WAV 并调用离线语音转写。
 *
 * 开始录音时调用 startRecording，结束录音时调用 stopRecordingAndTranscribe。
 * 页面切换、上移取消或其他取消场景调用 cancelRecording。
 */
export function useOfflineVoiceTranscription({ debug = false }: UseOfflineVoiceTranscriptionOptions = {}) {
  const [status, setStatus] = useState<VoiceTranscriptionStatus>('idle')
  const [transcription, setTranscription] = useState('')
  const [error, setError] = useState<string | null>(null)
  const [taskId, setTaskId] = useState<string | null>(null)
  const [audioFile, setAudioFile] = useState<File | null>(null)
  const [debugLogs, setDebugLogs] = useState<string[]>([])
  const audioContextRef = useRef<AudioContext | null>(null)
  const audioProcessorRef = useRef<ScriptProcessorNode | null>(null)
  const audioSilentGainRef = useRef<GainNode | null>(null)
  const audioStreamRef = useRef<MediaStream | null>(null)
  const audioSamplesRef = useRef<Float32Array[]>([])
  const audioFrameCountRef = useRef(0)
  const recordingAttemptIdRef = useRef(0)
  const audioContextStateListenerRef = useRef<(() => void) | null>(null)
  const transcriptionConnectionRef = useRef<ReturnType<typeof startOfflineTranscription> | null>(null)

  const appendDebugLog = (message: string, details?: unknown) => {
    if (!debug) return

    const timestamp = new Date().toLocaleTimeString('zh-CN', { hour12: false })
    setDebugLogs(currentLogs => [...currentLogs, `[${timestamp}] ${message}${stringifyDebugDetails(details)}`].slice(-100))
  }

  const releaseAudioResources = (shouldLog = true) => {
    if (shouldLog) {
      appendDebugLog('释放音频资源', {
        contextState: audioContextRef.current?.state,
        streamTracks: audioStreamRef.current?.getTracks().map(track => ({ kind: track.kind, readyState: track.readyState }))
      })
    }

    if (audioContextRef.current && audioContextStateListenerRef.current) {
      audioContextRef.current.removeEventListener('statechange', audioContextStateListenerRef.current)
    }
    audioContextStateListenerRef.current = null
    audioProcessorRef.current?.disconnect()
    audioProcessorRef.current = null
    audioSilentGainRef.current?.disconnect()
    audioSilentGainRef.current = null
    audioStreamRef.current?.getTracks().forEach(track => track.stop())
    audioStreamRef.current = null
    void audioContextRef.current?.close()
    audioContextRef.current = null
  }

  const cancelTranscription = () => {
    appendDebugLog('取消识别连接')
    transcriptionConnectionRef.current?.close()
    transcriptionConnectionRef.current = null
    setStatus('idle')
  }

  useEffect(
    () => () => {
      recordingAttemptIdRef.current += 1
      releaseAudioResources(false)
      transcriptionConnectionRef.current?.close()
    },
    []
  )

  const startRecording = async () => {
    appendDebugLog('startRecording 调用', {
      status,
      secureContext: window.isSecureContext,
      protocol: window.location.protocol,
      hostname: window.location.hostname,
      mediaDevices: Boolean(navigator.mediaDevices),
      getUserMedia: Boolean(navigator.mediaDevices?.getUserMedia)
    })

    if (!navigator.mediaDevices?.getUserMedia) {
      appendDebugLog('能力检查失败：getUserMedia 不可用')
      setError('当前浏览器不支持麦克风录音。')
      setStatus('error')
      return false
    }

    if (status === 'requesting' || status === 'recording' || status === 'transcribing') {
      appendDebugLog('忽略重复开始：当前状态不允许录音', { status })
      return false
    }

    const attemptId = ++recordingAttemptIdRef.current
    setStatus('requesting')
    setError(null)

    let audioContext: AudioContext | null = null
    let audioStream: MediaStream | null = null

    try {
      if (!window.isSecureContext && window.location.hostname !== 'localhost' && window.location.hostname !== '127.0.0.1') {
        throw new Error('录音需要 HTTPS 安全环境。请不要在微信真机中使用 HTTP 地址打开 H5。')
      }

      appendDebugLog('安全环境检查通过')

      const AudioContextConstructor = window.AudioContext ?? (window as WebkitWindow).webkitAudioContext
      if (!AudioContextConstructor) {
        throw new Error('当前微信 WebView 不支持 Web Audio。')
      }

      const documentWithPolicy = document as DocumentWithMicrophonePolicy
      const microphonePolicy = documentWithPolicy.permissionsPolicy ?? documentWithPolicy.featurePolicy
      appendDebugLog('文档权限上下文', {
        visibilityState: document.visibilityState,
        topLevel: window.top === window.self,
        microphoneAllowedByPolicy: microphonePolicy?.allowsFeature?.('microphone')
      })
      appendDebugLog('调用 getUserMedia 请求麦克风')
      audioStream = await navigator.mediaDevices.getUserMedia({ audio: true })
      if (attemptId !== recordingAttemptIdRef.current) {
        appendDebugLog('录音请求已取消：释放刚获取的媒体流')
        audioStream.getTracks().forEach(track => track.stop())
        return false
      }
      appendDebugLog('getUserMedia 成功', {
        active: audioStream.active,
        tracks: audioStream.getTracks().map(track => ({
          kind: track.kind,
          label: track.label,
          enabled: track.enabled,
          muted: track.muted,
          readyState: track.readyState,
          settings: track.getSettings()
        }))
      })

      audioContext = new AudioContextConstructor()
      appendDebugLog('AudioContext 创建完成', {
        state: audioContext.state,
        sampleRate: audioContext.sampleRate,
        outputLatency: 'outputLatency' in audioContext ? audioContext.outputLatency : undefined
      })
      const handleAudioContextStateChange = () => {
        appendDebugLog('AudioContext 状态变化', { state: audioContext?.state })
      }
      audioContextStateListenerRef.current = handleAudioContextStateChange
      audioContext.addEventListener('statechange', handleAudioContextStateChange)

      if (audioContext.state === 'suspended') {
        appendDebugLog('getUserMedia 成功，AudioContext 处于 suspended，尝试 resume')
        await audioContext.resume()
        appendDebugLog('AudioContext resume 完成', { state: audioContext.state })
      }
      if (attemptId !== recordingAttemptIdRef.current) {
        appendDebugLog('录音请求已取消：释放 AudioContext 和媒体流')
        audioStream.getTracks().forEach(track => track.stop())
        if (audioContextStateListenerRef.current) {
          audioContext.removeEventListener('statechange', audioContextStateListenerRef.current)
          audioContextStateListenerRef.current = null
        }
        void audioContext.close()
        return false
      }
      const source = audioContext.createMediaStreamSource(audioStream)
      appendDebugLog('MediaStreamSource 创建完成')
      const processor = audioContext.createScriptProcessor(4096, 1, 1)
      appendDebugLog('ScriptProcessor 创建完成', { bufferSize: processor.bufferSize })
      const silentGain = audioContext.createGain()
      silentGain.gain.value = 0

      audioSamplesRef.current = []
      audioFrameCountRef.current = 0
      processor.onaudioprocess = event => {
        audioFrameCountRef.current += 1
        audioSamplesRef.current.push(new Float32Array(event.inputBuffer.getChannelData(0)))
        if (audioFrameCountRef.current === 1) {
          appendDebugLog('收到首个音频帧', {
            frameLength: event.inputBuffer.length,
            sampleRate: event.inputBuffer.sampleRate,
            channelCount: event.inputBuffer.numberOfChannels
          })
        }
      }
      source.connect(processor)
      processor.connect(silentGain)
      silentGain.connect(audioContext.destination)
      appendDebugLog('音频节点连接完成')

      audioContextRef.current = audioContext
      audioProcessorRef.current = processor
      audioSilentGainRef.current = silentGain
      audioStreamRef.current = audioStream
      setTranscription('')
      setError(null)
      setTaskId(null)
      setAudioFile(null)
      setStatus('recording')
      appendDebugLog('录音状态已切换为 recording')
      return true
    } catch (cause) {
      audioStream?.getTracks().forEach(track => track.stop())
      if (audioContext) {
        if (audioContextStateListenerRef.current) {
          audioContext.removeEventListener('statechange', audioContextStateListenerRef.current)
          audioContextStateListenerRef.current = null
        }
        void audioContext.close()
      }

      let microphonePermission: PermissionState | '查询失败' | '不支持查询' = '不支持查询'
      if (navigator.permissions?.query) {
        try {
          const permission = await navigator.permissions.query({ name: 'microphone' as PermissionName })
          microphonePermission = permission.state
        } catch {
          microphonePermission = '查询失败'
        }
      }

      if (attemptId !== recordingAttemptIdRef.current) {
        appendDebugLog('录音请求已取消：忽略初始化异常')
        return false
      }

      const message = cause instanceof Error && cause.message ? cause.message : '无法使用麦克风，请确认已授予微信或浏览器录音权限。'
      appendDebugLog('startRecording 异常', {
        name: cause instanceof Error ? cause.name : undefined,
        message: cause instanceof Error ? cause.message : String(cause),
        stack: cause instanceof Error ? cause.stack : undefined,
        contextState: audioContext?.state,
        microphonePermission,
        visibilityState: document.visibilityState,
        topLevel: window.top === window.self,
        streamTracks: audioStream?.getTracks().map(track => ({ kind: track.kind, readyState: track.readyState }))
      })
      setError(message)
      setStatus('error')
      return false
    }
  }

  const handleAsrEvent = (event: string, data: AsrOfflineEvent) => {
    appendDebugLog(`收到 ASR 事件：${event}`, data)
    if (event === 'submitted' && 'aid' in data) {
      setTaskId(data.aid)
      return
    }

    if (event === 'result' && 'lattices' in data) {
      if (!data.state.success || data.state.code !== 0) {
        setError('语音识别未成功完成。')
        setStatus('error')
        return
      }

      setTranscription(data.lattices.map(item => item.onebest).join(''))
      setStatus('success')
      return
    }

    if (event === 'error' && 'message' in data) {
      setError(data.message)
      setStatus('error')
    }
  }

  const stopRecordingAndTranscribe = () => {
    const audioContext = audioContextRef.current
    const samples = audioSamplesRef.current
    const totalSampleCount = samples.reduce((length, sample) => length + sample.length, 0)

    appendDebugLog('stopRecordingAndTranscribe 调用', {
      contextState: audioContext?.state,
      sampleBufferCount: samples.length,
      totalSampleCount,
      frameCount: audioFrameCountRef.current,
      streamTracks: audioStreamRef.current?.getTracks().map(track => ({ kind: track.kind, readyState: track.readyState }))
    })

    if (!audioContext || samples.length === 0) {
      appendDebugLog('停止失败：没有可用的 AudioContext 或音频采样')
      releaseAudioResources()
      setError('未采集到音频，请重新录制。')
      setStatus('error')
      return
    }

    const mergedSamples = new Float32Array(totalSampleCount)
    let offset = 0
    samples.forEach(sample => {
      mergedSamples.set(sample, offset)
      offset += sample.length
    })

    const fileName = `asr-${new Date().toISOString().replace(/[:.]/g, '-')}.wav`
    const file = new File([createWavFile(mergedSamples, audioContext.sampleRate)], fileName, { type: 'audio/wav' })
    appendDebugLog('WAV 文件生成完成', {
      fileName: file.name,
      fileSize: file.size,
      sourceSampleRate: audioContext.sampleRate,
      targetSampleRate: TARGET_SAMPLE_RATE
    })

    releaseAudioResources()
    setAudioFile(file)
    setError(null)
    setStatus('transcribing')
    transcriptionConnectionRef.current?.close()
    appendDebugLog('开始上传 WAV 并建立 ASR SSE 连接')

    const connection = startOfflineTranscription(file, handleAsrEvent, requestError => {
      appendDebugLog(
        'ASR 请求异常',
        requestError instanceof Error ? { name: requestError.name, message: requestError.message, stack: requestError.stack } : requestError
      )
      setError(requestError instanceof Error ? requestError.message : '识别请求失败，请稍后重试。')
      setStatus('error')
      return null
    })
    transcriptionConnectionRef.current = connection

    void connection.done.then(() => {
      appendDebugLog('ASR SSE 连接结束')
      setStatus(currentStatus => {
        if (currentStatus === 'transcribing') {
          setError('识别连接已关闭，未收到识别结果。')
          return 'error'
        }
        return currentStatus
      })
    })
  }

  const cancelRecording = () => {
    recordingAttemptIdRef.current += 1
    appendDebugLog('取消录音')
    releaseAudioResources()
    audioSamplesRef.current = []
    setStatus('idle')
  }

  const reset = () => {
    recordingAttemptIdRef.current += 1
    appendDebugLog('重置录音状态')
    releaseAudioResources()
    cancelTranscription()
    setTranscription('')
    setError(null)
    setTaskId(null)
    setAudioFile(null)
  }

  return {
    status,
    isStarting: status === 'requesting',
    isRecording: status === 'recording',
    isTranscribing: status === 'transcribing',
    transcription,
    error,
    taskId,
    audioFile,
    debugLogs,
    clearDebugLogs: () => setDebugLogs([]),
    startRecording,
    stopRecordingAndTranscribe,
    cancelRecording,
    cancelTranscription,
    reset
  }
}

12.3 按住说话按钮

路径：src/pages/chat/input/VoiceInput.tsx

项目里的按钮还引用了 Icon、键盘 SVG、StopButton、VoiceBars。下面是去掉这些依赖后的完整组件，手势、滞回距离和提交判断与源码一致，可单独拷贝。

import { useRef, useState } from 'react'

type VoiceInputMode = 'idle' | 'recording' | 'cancelling'

type VoiceInputProps = {
  isAnswering: boolean
  isStarting: boolean
  isRecording: boolean
  isTranscribing: boolean
  disabled: boolean
  transcriptionError: string | null
  onSwitchToText: () => void
  onStop: () => void
  onVoiceStart: () => void
  onVoiceSubmit: () => void
  onVoiceCancel: () => void
}

const VOICE_CANCEL_DISTANCE = 48
const VOICE_RESUME_DISTANCE = 32

export default function VoiceInput({
  isAnswering,
  isStarting,
  isRecording,
  isTranscribing,
  disabled,
  transcriptionError,
  onSwitchToText,
  onStop,
  onVoiceStart,
  onVoiceSubmit,
  onVoiceCancel
}: VoiceInputProps) {
  const [mode, setMode] = useState<VoiceInputMode>('idle')
  const voicePointerIdRef = useRef<number | null>(null)
  const voiceStartYRef = useRef(0)
  const isVoiceInputActiveRef = useRef(false)
  const isVoiceInputActive = mode !== 'idle' || isStarting || isRecording

  const resetVoiceInput = () => {
    voicePointerIdRef.current = null
    isVoiceInputActiveRef.current = false
    setMode('idle')
  }

  const startVoiceInput = (clientY: number) => {
    if (isAnswering || disabled || isStarting || isRecording || isTranscribing) {
      return false
    }

    voiceStartYRef.current = clientY
    isVoiceInputActiveRef.current = true
    setMode('recording')
    onVoiceStart()
    return true
  }

  const updateVoiceInput = (clientY: number) => {
    const distance = voiceStartYRef.current - clientY
    if (mode === 'recording' && distance >= VOICE_CANCEL_DISTANCE) {
      setMode('cancelling')
    } else if (mode === 'cancelling' && distance < VOICE_RESUME_DISTANCE) {
      setMode('recording')
    }
  }

  const completeVoiceInput = () => {
    if (!isVoiceInputActiveRef.current) {
      return
    }

    const shouldSubmit = mode === 'recording' && isRecording
    resetVoiceInput()
    if (shouldSubmit) {
      onVoiceSubmit()
    } else {
      onVoiceCancel()
    }
  }

  const cancelVoiceInput = () => {
    if (!isVoiceInputActiveRef.current) return

    onVoiceCancel()
    resetVoiceInput()
  }

  const handlePointerDown = (event: React.PointerEvent<HTMLButtonElement>) => {
    if (!event.isPrimary || (event.pointerType === 'mouse' && event.button !== 0)) {
      return
    }

    if (!startVoiceInput(event.clientY)) return

    voicePointerIdRef.current = event.pointerId
    event.currentTarget.setPointerCapture(event.pointerId)
  }

  const handlePointerMove = (event: React.PointerEvent<HTMLButtonElement>) => {
    if (event.pointerId === voicePointerIdRef.current) {
      updateVoiceInput(event.clientY)
    }
  }

  const handlePointerUp = (event: React.PointerEvent<HTMLButtonElement>) => {
    if (event.pointerId === voicePointerIdRef.current) {
      completeVoiceInput()
    }
  }

  const hint = isStarting ? '请在系统提示中允许使用麦克风' : isTranscribing ? '请稍候' : mode === 'cancelling' ? '松开取消' : '松手发送，上移取消'

  return (
    <div className="h-[66px] w-full shrink-0 select-none p-3">
      <div className="relative h-[42px] w-full rounded-full bg-white shadow-[0_3px_16px_1px_rgb(0_0_0/0.1)]">
        <button
          type="button"
          aria-label="切换文字输入"
          className="absolute left-0 top-0 z-10 flex h-[42px] w-[42px] items-center justify-center text-sm text-[#2D3644]"
          disabled={disabled || isStarting || isTranscribing}
          onClick={onSwitchToText}
        >
          键盘
        </button>
        <button
          type="button"
          aria-label={mode === 'recording' ? '松手发送，上移取消' : mode === 'cancelling' ? '松开取消' : '按住说话'}
          className={`absolute left-0 top-0 z-0 h-full w-full touch-none rounded-full disabled:pointer-events-none ${isAnswering ? 'pr-11' : ''}`}
          disabled={disabled || isAnswering || isStarting || isTranscribing}
          onPointerDown={handlePointerDown}
          onPointerMove={handlePointerMove}
          onPointerUp={handlePointerUp}
          onPointerCancel={cancelVoiceInput}
          onLostPointerCapture={cancelVoiceInput}
        >
          <span className="pointer-events-none absolute left-0 top-0 flex h-full w-full items-center justify-center text-sm font-semibold text-[#2D3644]">
            按住说话
          </span>
        </button>
        {isAnswering && (
          <button
            type="button"
            className="absolute right-1.5 top-1/2 z-10 -translate-y-1/2 rounded-full px-3 py-1 text-xs text-[#2D3644]"
            onClick={onStop}
          >
            停止
          </button>
        )}
      </div>
      {(isVoiceInputActive || isTranscribing) && (
        <div className="pointer-events-none fixed bottom-0 left-0 right-0 z-20 h-64 overflow-hidden">
          <div className="absolute bottom-32 left-6 right-6 text-center text-sm font-medium text-[#2D3644]">
            {isStarting ? '正在请求录音权限…' : isTranscribing ? '正在识别语音…' : '正在录音…'}
          </div>
          <div className="absolute bottom-7 left-0 right-0 text-center text-xs font-medium text-[#2D3644]">{hint}</div>
        </div>
      )}
      {transcriptionError && (
        <p className="px-3 text-center text-xs text-red-500" role="alert">
          语音识别失败：{transcriptionError}
        </p>
      )}
    </div>
  )
}

按钮要用 touch-none，否则移动端浏览器可能把按住当成滚动，pointerup 收不到。

12.4 接入聊天输入

路径：src/pages/chat/input/ChatInput.tsx

识别成功后把转写结果当问题发出去。下面是与语音相关的完整接入，可直接接到自己的发送函数上：

import { useEffect, useRef } from 'react'

import { useOfflineVoiceTranscription } from '@/hooks/useOfflineVoiceTranscription'

import VoiceInput from './VoiceInput'

type VoiceChatInputProps = {
  isAnswering: boolean
  disabled?: boolean
  onSend: (question: string) => void
  onStop: () => void
  onSwitchToText: () => void
}

export default function VoiceChatInput({ isAnswering, disabled = false, onSend, onStop, onSwitchToText }: VoiceChatInputProps) {
  const hasSubmittedVoiceRef = useRef(false)
  const {
    cancelRecording,
    error: transcriptionError,
    isRecording,
    isStarting,
    isTranscribing,
    reset: resetVoiceTranscription,
    startRecording,
    status: transcriptionStatus,
    stopRecordingAndTranscribe,
    transcription
  } = useOfflineVoiceTranscription()

  useEffect(() => {
    if (transcriptionStatus !== 'success') {
      hasSubmittedVoiceRef.current = false
      return
    }

    if (hasSubmittedVoiceRef.current) {
      return
    }

    hasSubmittedVoiceRef.current = true
    const question = transcription.trim()
    if (question) {
      onSend(question)
    }
    resetVoiceTranscription()
  }, [onSend, resetVoiceTranscription, transcription, transcriptionStatus])

  return (
    <VoiceInput
      isAnswering={isAnswering}
      isStarting={isStarting}
      isRecording={isRecording}
      isTranscribing={isTranscribing}
      disabled={disabled}
      transcriptionError={transcriptionError}
      onSwitchToText={() => {
        resetVoiceTranscription()
        onSwitchToText()
      }}
      onStop={onStop}
      onVoiceStart={() => void startRecording()}
      onVoiceSubmit={stopRecordingAndTranscribe}
      onVoiceCancel={cancelRecording}
    />
  )
}

只想验证录音、不接聊天发送时，打开 { debug: true }，用点击开始 / 结束即可，不必上滑取消：

const { debugLogs, error, isRecording, isTranscribing, startRecording, status, stopRecordingAndTranscribe, transcription } =
  useOfflineVoiceTranscription({ debug: true })

<button type="button" disabled={isRecording || isTranscribing} onClick={() => void startRecording()}>
  开始录音
</button>
<button type="button" disabled={!isRecording} onClick={stopRecordingAndTranscribe}>
  结束录音并上传
</button>

项目里的 /test 就是这种接法，适合在微信 WebView 真机上对照诊断日志。

13. 小结

这个 Hook 做的事情可以收成一句话：

在用户手势里拿到麦克风，用 Web Audio 收到单声道 PCM，停录时写成 16 kHz WAV，再经 SSE 拿离线转写结果。

按钮不知道 AudioContext，Hook 不知道「上滑 48 像素」。中间用 start / stop / cancel 三个动作和一条状态机对齐。取消用 attemptId 对上异步权限请求，停止和卸载走同一套资源释放。这样「按住说话」才能在普通浏览器和微信 WebView 里用同一套代码跑起来。

AI Agent技术社区

Agent 垂直技术社区，欢迎活跃、内容共建。

更多推荐

2026最新AI Agent高频面试题（分类型精简详解）

本文梳理AI Agent高频面试考点，覆盖基础概念、架构、推理规划、工具调用、记忆、RAG融合、多智能体与工程落地。区分Agent、Workflow与普通工具调用，讲解ReAct、Plan‑and‑Execute等主流范式，分析三级记忆机制与上下文溢出的解决办法，对比传统RAG与Agent‑RAG差异，介绍多智能体协作模式。同时总结落地痛点、优化手段和安全风控要点，划分初/中/高级面试复习重点，可