一、论文信息

项目	内容
论文标题（英文）	A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition
论文标题（中文）	隐马尔可夫模型及其在语音识别中若干应用的教程
作者	Lawrence R. Rabiner（劳伦斯·拉宾纳）
机构	AT&T Bell Laboratories（AT&T 贝尔实验室），美国新泽西州默里山
发表期刊	Proceedings of the IEEE（IEEE 会报）
发表时间	1989 年 2 月，第 77 卷第 2 期，第 257–286 页
DOI	10.1109/5.18626
论文地址	https://www.cs.ubc.ca/~murphyk/Bayes/rabiner.pdf

说明：这是期刊论文（Proceedings of the IEEE 是期刊而非会议），它是 HMM 领域被引用最多的综述之一（引用量数万次），也是几乎所有 HMM 课程和教材的标准入门读物。

---

二、论文内容总结

1. 写作目的

HMM 的数学理论早在 1960 年代末由 Baum 等人建立，但散落在数学期刊中，工程界难以理解和应用。本文的目标是：面向工程师（而非数学家），用直观、实用、可实现的视角系统讲解 HMM 理论，并展示它如何成功应用于语音识别。

2. 论文结构与内容

Ⅰ–Ⅱ. 引入：从离散马尔可夫链到 HMM

• 先用"离散马尔可夫过程"（状态可见）做铺垫，引出经典例子——缸与球模型（urn and ball）：观测序列只告诉你球的颜色，但球是从哪个缸取的（状态）是隐藏的，这就是"隐"的含义。
• 给出 HMM 的完整形式化定义，一个模型由五要素刻画：状态数 $N$、观测符号数 $M$、状态转移矩阵 $A$、观测概率分布 $B$、初始状态分布 $\pi$，简记为 $\lambda =(A,B,\pi )$。
• 给出生成观测序列的过程，并区分了各态历经模型（ergodic，状态任意互通）和左右模型（left-right，状态只能向前推进——语音识别中常用）。

Ⅲ. HMM 的三个基本问题（全文核心）

问题一：评估（Evaluation）——给定模型 $\lambda$ 和观测序列 $O$，计算 $P(O\mid \lambda )$。

• 直接枚举所有状态序列是指数级复杂度，不可行。
• 解法：**前向算法（forward procedure）**与后向算法，利用动态规划将复杂度从 $O(N^T\cdot T)$ 降到 $O(N^{2}T)$。

问题二：解码（Decoding）——给定观测序列，找出"最可能"的隐藏状态序列。

• 讨论最优性准则（单状态最优 vs. 整条路径最优），采用后者。
• 解法：Viterbi 算法，本质是动态规划找最优路径。

问题三：训练（Learning/Training）——给定观测序列，如何调整模型参数 $\lambda$ 使 $P(O\mid \lambda )$ 最大化。

• 这是最困难的问题，没有闭式解。
• 解法：Baum-Welch 算法（即 EM 算法在 HMM 上的特例），通过前向-后向概率迭代重估参数 $A$、$B$、$\pi$，保证收敛到局部最优。

Ⅳ. 连续观测密度的 HMM

前述模型观测是离散符号（需向量量化，会损失信息）。本节扩展到连续观测，给出多种概率密度形式：高斯密度、高斯混合密度（GMM）、以及带状态持续时间建模的半马尔可夫变体，并给出相应的参数重估公式。这就是后来统治语音识别二十年的 GMM-HMM 框架。

Ⅴ. 自回归 HMM

针对高度相关的观测信号（如语音的线性预测系数），讨论观测由自回归过程生成的 HMM 变体。

Ⅵ. 空转移与状态捆绑（tied parameters）

• 讨论允许不发射观测的"空转移"（null transition）和按转移对观测符号分组（Bakis 模型）等技术，用于压缩参数、灵活建模。

Ⅶ. 实际问题（实现要点，极具工程价值）

• 缩放（scaling）：前向/后向概率随序列长度指数衰减会下溢，必须每步归一化。
• 多观测序列的训练（孤立词识别中每个词多条样本）。
• 初始参数估计：好的初始化影响收敛效果，无严格方法但有经验做法。
• 对数域计算、删除插值（deleted interpolation） 等技巧。

Ⅷ–Ⅸ. 在语音识别中的应用与展望

• 孤立词识别：每个词训练一个 HMM，识别时对每个模型算 $P(O\mid {\lambda }_w)$ 取最大；介绍了分词、码本训练（VQ）等完整流程。
• 大词汇量连续语音识别：以音素/子词为单元建模，词由子词模型拼接；以 IBM Tangora 等系统为例说明可行性。
• 总结 HMM 在语音领域成功的根本原因：数学理论扎实、算法可有效实现、能直接从数据中学习。

3. 历史地位与影响

• 把 HMM 从"数学家的论文"变成"工程师的工具"，直接推动了 1980–2000 年代语音识别（以及后来的词性标注、生物信息学等）的 HMM/GMM-HMM 时代。
• 文中总结的三个基本问题 + 三个算法（前向、Viterbi、Baum-Welch）至今仍是所有 HMM 教材的标准框架。

摘要

Although initially introduced and studied in the late 1960s and early 1970s, statistical methods of Markov source or hidden Markov modeling have become increasingly popular in the last several years. There are two strong reasons why this has occurred. First the models are very rich in mathematical structure and hence can form the theoretical basis for use in a wide range of applications. Second the models, when applied properly, work very well in practice for several important applications. In this paper we attempt to carefully and methodically review the theoretical aspects of this type of statistical modeling and show how they have been applied to selected problems in machine recognition of speech.
尽管马尔可夫信源或隐马尔可夫模型在20世纪60年代末和70年代初已被引入和研究，但在过去的几年里，这些统计方法变得越来越受欢迎。这主要有两个强有力的原因。首先，这些模型在数学结构上非常丰富，因此可以作为理论基础应用于更广泛的领域。其次，这些模型在应用得当的情况下，在几个重要的实际应用中表现优异。本文旨在仔细而系统地回顾这类统计模型的理论方面，并展示它们如何应用于语音机器识别中的选定问题。

I. 引言

Real-world processes generally produce observable outputs which can be characterized as signals. The signals can be discrete in nature (e.g., characters from a finite alphabet, quantized vectors from a codebook, etc.), or continuous in nature (e.g., speech samples, temperature measurements, music, etc.). The signal source can be stationary (i.e., its statistical properties do not vary with time), or nonstationary (i.e., the signal properties vary over time). The signals can be pure (i.e., coming strictly from a single source), or can be corrupted from other signal sources (e.g., noise) or by transmission distortions, reverberation, etc.
现实世界的过程通常会产生可观测的输出，这些输出可以被描述为信号。信号可以是离散的（例如，来自有限字母表的字符、来自码本的量化向量等），也可以是连续的（例如，语音样本、温度测量、音乐等）。信号源可以是平稳的（即，其统计特性不随时间变化），或非平稳的（即，信号特性随时间变化）。信号可以是纯净的（即严格来自单一源），也可以受到其他信号源（例如噪声）或传输失真、混响等的干扰。

A problem of fundamental interest is characterizing such real-world signals in terms of signal models. There are several reasons why one is interested in applying signal models. First of all, a signal model can provide the basis for a theoretical description of a signal processing system which can be used to process the signal so as to provide a desired output. For example if we are interested in enhancing a speech signal corrupted by noise and transmission distortion, we can use the signal model to design a system which will optimally remove the noise and undo the transmission distortion. A second reason why signal models are important is that they are potentially capable of letting us learn a great deal about the signal source (i.e., the real-world process which produced the signal) without having to have the source available. This property is especially important when the cost of getting signals from the actual source is high.
一个根本性的问题是根据信号模型来表征这些真实世界信号。人们对应用信号模型感兴趣有几个原因。首先，信号模型可以为信号处理系统的理论描述提供基础，该系统可用于处理信号以提供期望的输出。例如，如果我们希望增强被噪声和传输失真破坏的语音信号，我们可以使用信号模型来设计一个系统，该系统可以最优地去除噪声并消除传输失真。信号模型之所以重要的第二个原因是，它们有潜力让我们在无需信号源可用的情况下，就能深入了解信号源（即产生信号的真实世界过程）。当从实际源获取信号的成本很高时，此属性尤其重要。

In this case, with a good signal model, we can simulate the source and learn as much as possible via simulations. Finally, the most important reason why signal models are important is that they often work extremely well in practice, and enable us to realize important practical systems—e.g., prediction systems, recognition systems, identification systems, etc., in a very efficient manner.
在此情况下，通过一个良好的信号模型，我们可以通过仿真来模拟信源并尽可能多地学习。最后，信号模型之所以重要的最根本原因在于，它们在实践中往往表现优异，并能使我们以非常高效的方式实现重要的实际系统，例如预测系统、识别系统、辨识系统等。

These are several possible choices for what type of signal model is used for characterizing the properties of a given signal. Broadly one can dichotomize the types of signal models into the class of deterministic models, and the class of statistical models. Deterministic models generally exploit some known specific properties of the signal, e.g., that the signal is a sine wave, or a sum of exponentials, etc. In these cases, specification of the signal model is generally straightforward; all that is required is to determine (estimate) values of the parameters of the signal model (e.g., amplitude, frequency, phase of a sine wave, amplitudes and rates of exponentials, etc.). The second broad class of signal models is the set of statistical models in which one tries to characterize only the statistical properties of the signal. Examples of such statistical models include Gaussian processes, Poisson processes, Markov processes, and hidden Markov processes, among others. The underlying assumption of the statistical model is that the signal can be well characterized as a parametric random process, and that the parameters of the stochastic process can be determined (estimated) in a precise, well-defined manner.
在表征给定信号的特性时，信号模型有几种可能的选择。广义上，可以将信号模型的类型 二分为 确定性模型类和统计模型类。确定性模型通常利用信号的某些已知特定属性，例如信号是正弦波，或是指数函数之和等。在这些情况下，信号模型的设定通常很直接；所需做的只是确定（估计）信号模型参数的值（例如正弦波的幅度、频率、相位，指数函数的幅度和衰减速率等）。信号模型的第二大类是统计模型，在这类模型中，人们试图仅表征信号的统计特性。这类统计模型的例子包括高斯过程、泊松过程、马尔可夫过程以及隐马尔可夫过程等。统计模型的基本假设是：信号可以被很好地表征为参数化随机过程，且该随机过程的参数能够以精确、定义明确的方式确定（估计）。
注：dichotomize /daɪˈkɒtəmaɪz/ 含义：将某事物一分为二、二分、分成两类（通常指两个相互排斥或对立的类别）。

For the applications of interest, namely speech processing, both deterministic and stochastic signal models have had good success. In this paper we will concern ourselves strictly with one type of stochastic signal model, namely the hidden Markov model (HMM). (These models are referred to as Markov sources or probabilistic functions of Markov chains in the communications literature.) We will first review the theory of Markov chains and then extend the ideas to the class of hidden Markov models using several simple examples. We will then focus our attention on the three fundamental problems¹ for HMM design, namely: the evaluation of the probability (or likelihood) of a sequence of observations given a specific HMM; the determination of a best sequence of model states; and the adjustment of model parameters so as to best account for the observed signal. We will show that once these three fundamental problems are solved, we can apply HMMs to selected problems in speech recognition.
对于我们感兴趣的应用领域，即语音处理，确定性信号模型和随机信号模型都取得了很好的效果。在本文中，我们将严格关注一种随机信号模型，即隐马尔可夫模型（HMM）。（在通信文献中，这些模型被称为马尔可夫信源或马尔可夫链的概率函数。）

我们将首先回顾马尔可夫链理论，然后通过几个简单的例子将这些思想扩展到隐马尔可夫模型这一类。随后，我们将把注意力集中在 HMM 设计中的三个基本问题上，即：

1. 评估问题：给定一个特定的 HMM，计算某个观测序列的概率（或似然）；
2. 解码问题：确定最优的模型状态序列；
3. 学习问题：调整模型参数，以便最好地解释观测到的信号。

我们将说明，一旦这三个基本问题得到解决，就可以将 HMM 应用于语音识别中的特定问题。

¹The idea of characterizing the theoretical aspects of hidden Markov modeling in terms of solving three fundamental problems is due to Jack Ferguson of IDA (Institute for Defense Analysis) who introduced it in lectures and writing.
将隐马尔可夫建模的理论内涵 归结为求解三个基本问题**这一思想，源自国防分析研究所（IDA）的杰克·弗格森（Jack Ferguson），他在讲座和著述中首次提出了这一框架。

注： The idea of characterizing … in terms of …
characterizing =刻画、描述、概括
in terms of = 用……的方式、从……角度
所以这部分意思是："用某种方式来概括/描述 HMM 理论"的这个思路
具体是什么方式呢？就是 solving three fundamental problems（求解三个基本问题）。
也就是说，主语整体是：
"把隐马尔可夫模型（HMM）的理论，概括为’三个基本问题’来研究"的这个思路/框架
is due to = 归功于、源自、是由……提出的
所以意思是：这个思路是由 Jack Ferguson 提出的

Neither the theory of hidden Markov models nor its applications to speech recognition is new. The basic theory was published in a series of classic papers by Baum and his colleagues [1]–[5] in the late 1960s and early 1970s and was implemented for speech processing applications by Baker [6] at CMU, and by Jelinek and his colleagues at IBM [7]–[13] in the 1970s. However, widespread understanding and application of the theory of HMMs to speech processing has occurred only within the past several years.
隐马尔可夫模型的理论及其在语音识别中的应用都算不上新鲜事物。其基础理论由 Baum 及其同事在 1960 年代末至 1970 年代初发表的一系列经典论文中提出 [1]–[5]；而在 1970 年代，Baker 在卡内基梅隆大学（CMU）[6]，以及 Jelinek 及其同事在 IBM [7]–[13] 已将其实现并应用于语音处理。然而，HMM 理论在语音处理领域的广泛理解与普及应用，却仅仅是过去几年间才发生的事。

There are several reasons why this has been the case. First, the basic theory of hidden Markov models was published in mathematical journals which were not generally read by engineers working on problems in speech processing. The second reason was that the original applications of the theory to speech processing did not provide sufficient tutorial material for most readers to understand the theory and to be able to apply it to their own research. As a result, several tutorial papers were written which provided a sufficient level of detail for a number of research labs to begin work using HMMs in individual speech processing applications [14]–[19].
之所以如此，有以下几个原因。首先，隐马尔可夫模型的基础理论发表于数学期刊，而从事语音处理工程问题研究的学者通常不会阅读这类刊物。第二个原因是，该理论最初在语音处理中的应用并未提供足够的教程性材料，使得大多数读者既难以理解其理论，也无法将其应用到自己的研究中。因此，后来出现了若干篇教程性论文，为众多研究实验室提供了足够详细的入门指导，使其得以开始在各自的语音处理应用中使用 HMM [14]–[19]。

This tutorial is intended to provide an overview of the basic theory of HMMs (as originated by Baum and his colleagues), provide practical details on methods of implementation of the theory, and describe a couple of selected applications of the theory to distinct problems in speech recognition. The paper combines results from a number of original sources and hopefully provides a single source for acquiring the background required to pursue further this fascinating area of research.
本教程旨在：概述 HMM 的基础理论（源于 Baum 及其同事的开创性工作）；提供理论实现方法上的实践细节；并介绍该理论在语音识别中几个不同具体问题上的精选应用。本文综合了若干原始文献的研究成果，希望能为读者提供一个单一的资料来源，使其获取进入这一引人入胜的研究领域所需的背景知识。

注：and hopefully provides… 意思是：（本文）希望/旨在提供…… 提供什么？single source =
单一的来源、一站式资料 这个来源用来干嘛？acquiring = 获取、获得 the background =背景知识（指入门所需的基础知识）
什么样的背景知识？ required = 所需的（修饰 background）
pursue further = 进一步深入研究（注意这里语序是 “pursue further this area”，正常语序是 “pursue
this area further”）
this fascinating area of research = 这个引人入胜的/令人着迷的研究领域

The organization of this paper is as follows.
本文的组织结构如下。
In Section II we review the theory of discrete Markov chains and show how the concept of hidden states, where the observation is a probabilistic function of the state, can be used effectively. We illustrate the theory with two simple examples, namely coin-tossing, and the classic balls-in-urns system.
在第二节中，我们回顾离散马尔可夫链理论，并阐述隐藏状态的概念（其中观测是状态的概率函数）如何被有效利用。我们通过两个简单的例子来阐释该理论，即抛硬币和经典的瓮中取球系统。

In Section III we discuss the three fundamental problems of HMMs, and give several practical techniques for solving these problems.
在第三节中，我们讨论 HMM 的三个基本问题，并给出求解这些问题的几种实用技术。

In Section IV we discuss the various types of HMMs that have been studied including ergodic as well as left-right models. In this section we also discuss the various model features including the form of the observation density function, the state duration density, and the optimization criterion for choosing optimal HMM parameter values.
在第四节中，我们讨论已研究过的各种类型的 HMM，包括遍历模型（ergodic）以及从左到右模型（left-right）。在本节中，我们还讨论各种模型特征，包括观测密度函数的形式、状态持续时间密度，以及选择最优 HMM 参数值的优化准则。

In Section V we discuss the issues that arise in implementing HMMs including the topics of scaling, initial parameter estimates, model size, model form, missing data, and multiple observation sequences.
在第五节中，我们讨论实现 HMM 时出现的一些问题，包括缩放（scaling）、初始参数估计、模型规模、模型形式、缺失数据以及多观测序列等主题。

In Section VI we describe an isolated word speech recognizer, implemented with HMM ideas, and show how it performs as compared to alternative implementations.
在第六节中，我们描述一个基于 HMM 思想实现的孤立词语音识别器，并展示其与其他实现方案相比的性能表现。

In Section VII we extend the ideas presented in Section VI to the problem of recognizing a string of spoken words based on concatenating individual HMMs of each word in the vocabulary.
在第七节中，我们将第六节提出的思想扩展到识别连续语音词串的问题，即通过拼接词汇表中每个词的独立 HMM 来实现。

In Section VIII we briefly outline how the ideas of HMM have been applied to a large vocabulary speech recognizer, and in Section IX we summarize the ideas discussed throughout the paper.
在第八节中，我们简要概述 HMM 的思想如何应用于大词汇量语音识别器；在第九节中，我们总结全文讨论的思想。

注：throughout 在这里的意思是 “遍及、贯穿整个”。 throughout the paper = 通篇、整篇论文、从头到尾