李宏毅深度学习课程实战：从零搭建PyTorch模型（附作业解析）

深度学习正在重塑我们解决问题的思维方式。想象一下，当你第一次看到神经网络识别手写数字的演示时，那种"机器竟然能学习"的震撼感。PyTorch作为当前最受欢迎的深度学习框架之一，以其动态计算图和直观的API设计，成为了学术研究和工业实践的首选工具。本文将带你从零开始，用PyTorch实现李宏毅课程中的核心算法，不仅理解理论，更能亲手构建可运行的模型。

1. 环境准备与PyTorch基础

1.1 PyTorch安装与配置

在开始之前，我们需要搭建PyTorch开发环境。PyTorch支持多种安装方式，推荐使用conda进行管理：

conda create -n pytorch_env python=3.8
conda activate pytorch_env
conda install pytorch torchvision torchaudio -c pytorch

验证安装是否成功：

import torch
print(torch.__version__)  # 应输出如1.12.1的版本号
print(torch.cuda.is_available())  # 检查GPU是否可用

提示：如果使用GPU加速训练，确保安装了对应版本的CUDA驱动。PyTorch官网提供了详细的版本匹配指南。

1.2 PyTorch核心概念

PyTorch的核心是其张量(Tensor)对象，可以理解为Numpy数组的GPU加速版本。理解以下几个关键特性对后续开发至关重要：

• 动态计算图：PyTorch使用动态图机制，允许在运行时修改网络结构
• 自动微分：autograd模块自动计算梯度，极大简化了反向传播实现
• 模块化设计：nn.Module提供了构建神经网络的基础抽象

# 简单的张量操作示例
x = torch.tensor([[1, 2], [3, 4]], dtype=torch.float32)
y = torch.randn(2, 2)
z = x.mm(y)  # 矩阵乘法
print(z)

2. 线性回归与反向传播实现

2.1 数据准备与模型定义

让我们从最简单的线性回归开始，实现课程中的播放量预测案例。首先创建合成数据：

import numpy as np

# 生成训练数据
np.random.seed(42)
x_train = np.linspace(0, 10, 100)
y_train = 3 * x_train + 5 + np.random.normal(0, 2, 100)

# 转换为PyTorch张量
x_tensor = torch.from_numpy(x_train).float().view(-1, 1)
y_tensor = torch.from_numpy(y_train).float().view(-1, 1)

定义线性回归模型：

class LinearRegression(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = torch.nn.Linear(1, 1)  # 输入输出维度均为1
        
    def forward(self, x):
        return self.linear(x)

2.2 训练循环与梯度下降

实现完整的训练流程，包括损失函数和优化器：

model = LinearRegression()
criterion = torch.nn.MSELoss()  # 均方误差损失
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)  # 随机梯度下降

# 训练参数
epochs = 1000
for epoch in range(epochs):
    # 前向传播
    outputs = model(x_tensor)
    loss = criterion(outputs, y_tensor)
    
    # 反向传播与优化
    optimizer.zero_grad()  # 清空梯度
    loss.backward()        # 计算梯度
    optimizer.step()       # 更新参数
    
    if (epoch+1) % 100 == 0:
        print(f'Epoch [{epoch+1}/{epochs}], Loss: {loss.item():.4f}')

注意：每次反向传播前必须调用zero_grad()，否则梯度会累积导致训练不稳定。

2.3 作业解析：温度预测实战

李宏毅课程的第一个作业要求实现温度预测模型。我们可以扩展上面的线性回归模型：

class TempPredictor(torch.nn.Module):
    def __init__(self, input_dim):
        super().__init__()
        self.net = torch.nn.Sequential(
            torch.nn.Linear(input_dim, 64),
            torch.nn.ReLU(),
            torch.nn.Linear(64, 1)
        )
        
    def forward(self, x):
        return self.net(x)

关键改进点：

• 增加了隐藏层和非线性激活
• 使用更复杂的网络结构捕捉非线性关系
• 添加了数据标准化预处理步骤

3. 卷积神经网络实战

3.1 CNN基础架构实现

课程中讲解的CNN结构可以用PyTorch轻松实现。以下是一个典型的图像分类网络：

class CNNClassifier(torch.nn.Module):
    def __init__(self, num_classes):
        super().__init__()
        self.features = torch.nn.Sequential(
            torch.nn.Conv2d(3, 32, kernel_size=3, padding=1),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2),
            torch.nn.Conv2d(32, 64, kernel_size=3, padding=1),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2)
        )
        self.classifier = torch.nn.Sequential(
            torch.nn.Linear(64*8*8, 128),
            torch.nn.ReLU(),
            torch.nn.Linear(128, num_classes)
        )
        
    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)  # 展平
        return self.classifier(x)

3.2 数据增强与批标准化

为了提高模型泛化能力，我们需要实现数据增强和批标准化：

from torchvision import transforms

# 数据增强
train_transform = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(10),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.5], std=[0.5])
])

# 在模型中添加BatchNorm层
torch.nn.BatchNorm2d(64)  # 在卷积层后添加

3.3 作业解析：宝可梦分类

针对课程中的宝可梦分类作业，我们可以构建更复杂的网络：

class PokemonCNN(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.main = torch.nn.Sequential(
            torch.nn.Conv2d(3, 32, 3, 1, 1),
            torch.nn.BatchNorm2d(32),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2, 2, 0),
            
            torch.nn.Conv2d(32, 64, 3, 1, 1),
            torch.nn.BatchNorm2d(64),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2, 2, 0),
            
            torch.nn.Conv2d(64, 128, 3, 1, 1),
            torch.nn.BatchNorm2d(128),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2, 2, 0)
        )
        self.fc = torch.nn.Sequential(
            torch.nn.Linear(128*4*4, 256),
            torch.nn.ReLU(),
            torch.nn.Linear(256, 18)  # 18个宝可梦类别
        )

4. 自注意力机制与Transformer实现

4.1 Self-Attention基础实现

课程中讲解的自注意力机制可以用PyTorch实现如下：

class SelfAttention(torch.nn.Module):
    def __init__(self, embed_size, heads):
        super().__init__()
        self.embed_size = embed_size
        self.heads = heads
        self.head_dim = embed_size // heads
        
        self.values = torch.nn.Linear(self.head_dim, self.head_dim, bias=False)
        self.keys = torch.nn.Linear(self.head_dim, self.head_dim, bias=False)
        self.queries = torch.nn.Linear(self.head_dim, self.head_dim, bias=False)
        self.fc_out = torch.nn.Linear(heads * self.head_dim, embed_size)
        
    def forward(self, values, keys, query, mask):
        N = query.shape[0]
        value_len, key_len, query_len = values.shape[1], keys.shape[1], query.shape[1]
        
        # 分割嵌入维度到多个头
        values = values.reshape(N, value_len, self.heads, self.head_dim)
        keys = keys.reshape(N, key_len, self.heads, self.head_dim)
        queries = query.reshape(N, query_len, self.heads, self.head_dim)
        
        # 计算注意力分数
        energy = torch.einsum("nqhd,nkhd->nhqk", [queries, keys])
        if mask is not None:
            energy = energy.masked_fill(mask == 0, float("-1e20"))
            
        attention = torch.nn.functional.softmax(energy / (self.embed_size ** (1/2)), dim=3)
        
        # 应用注意力权重
        out = torch.einsum("nhql,nlhd->nqhd", [attention, values])
        out = out.reshape(N, query_len, self.heads * self.head_dim)
        
        return self.fc_out(out)

4.2 Transformer编码器实现

基于自注意力机制，我们可以构建完整的Transformer编码器：

class TransformerBlock(torch.nn.Module):
    def __init__(self, embed_size, heads, dropout, forward_expansion):
        super().__init__()
        self.attention = SelfAttention(embed_size, heads)
        self.norm1 = torch.nn.LayerNorm(embed_size)
        self.norm2 = torch.nn.LayerNorm(embed_size)
        
        self.feed_forward = torch.nn.Sequential(
            torch.nn.Linear(embed_size, forward_expansion * embed_size),
            torch.nn.ReLU(),
            torch.nn.Linear(forward_expansion * embed_size, embed_size)
        )
        
        self.dropout = torch.nn.Dropout(dropout)
        
    def forward(self, value, key, query, mask):
        attention = self.attention(value, key, query, mask)
        x = self.dropout(self.norm1(attention + query))
        forward = self.feed_forward(x)
        out = self.dropout(self.norm2(forward + x))
        return out

4.3 作业解析：序列标注任务

针对序列标注作业，我们可以构建基于Transformer的模型：

class SequenceTagger(torch.nn.Module):
    def __init__(self, vocab_size, embed_size, num_classes, heads, num_layers):
        super().__init__()
        self.embedding = torch.nn.Embedding(vocab_size, embed_size)
        self.position_embedding = PositionalEncoding(embed_size)
        self.layers = torch.nn.ModuleList(
            [TransformerBlock(embed_size, heads, dropout=0.1, forward_expansion=4) 
             for _ in range(num_layers)]
        )
        self.fc_out = torch.nn.Linear(embed_size, num_classes)
        
    def forward(self, x, mask):
        x = self.embedding(x)
        x = self.position_embedding(x)
        for layer in self.layers:
            x = layer(x, x, x, mask)
        return self.fc_out(x)

5. 模型调试与优化技巧

5.1 学习率调整策略

课程中强调的学习率问题可以通过多种策略解决：

# 学习率预热
optimizer = torch.optim.Adam(model.parameters(), lr=0)
scheduler = torch.optim.lr_scheduler.LambdaLR(
    optimizer,
    lr_lambda=lambda step: min((step+1)**-0.5, (step+1)*1000**-1.5)
)

# 余弦退火
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=100)

5.2 梯度裁剪与早停

防止梯度爆炸和过拟合的实用技巧：

# 梯度裁剪
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

# 早停实现
best_loss = float('inf')
patience = 5
counter = 0

for epoch in range(epochs):
    # ...训练代码...
    if val_loss < best_loss:
        best_loss = val_loss
        counter = 0
        torch.save(model.state_dict(), 'best_model.pth')
    else:
        counter += 1
        if counter >= patience:
            print("Early stopping")
            break

5.3 可视化与调试工具

利用PyTorch内置工具和第三方库进行调试：

# 使用TensorBoard
from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter()
writer.add_graph(model, input_tensor)
writer.add_scalar('Loss/train', loss.item(), epoch)

# 使用torchviz可视化计算图
from torchviz import make_dot
make_dot(loss, params=dict(model.named_parameters()))

6. 实战项目：从理论到完整实现

6.1 图像分类完整流程

结合课程内容和PyTorch实现完整图像分类流程：

def train_model(model, train_loader, val_loader, epochs, device):
    model.to(device)
    criterion = torch.nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model.parameters())
    
    for epoch in range(epochs):
        model.train()
        for images, labels in train_loader:
            images, labels = images.to(device), labels.to(device)
            
            optimizer.zero_grad()
            outputs = model(images)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()
        
        # 验证阶段
        model.eval()
        with torch.no_grad():
            correct = 0
            total = 0
            for images, labels in val_loader:
                images, labels = images.to(device), labels.to(device)
                outputs = model(images)
                _, predicted = torch.max(outputs.data, 1)
                total += labels.size(0)
                correct += (predicted == labels).sum().item()
            
            print(f'Epoch [{epoch+1}/{epochs}], Accuracy: {100*correct/total:.2f}%')

6.2 自然语言处理案例

实现课程中提到的文本分类任务：

class TextClassifier(torch.nn.Module):
    def __init__(self, vocab_size, embed_dim, num_classes):
        super().__init__()
        self.embedding = torch.nn.Embedding(vocab_size, embed_dim)
        self.lstm = torch.nn.LSTM(embed_dim, 128, batch_first=True, bidirectional=True)
        self.fc = torch.nn.Linear(256, num_classes)
        
    def forward(self, x):
        x = self.embedding(x)
        _, (hidden, _) = self.lstm(x)
        out = torch.cat((hidden[-2], hidden[-1]), dim=1)
        return self.fc(out)

6.3 模型部署与生产化

将训练好的模型部署为可用的服务：

from flask import Flask, request, jsonify
import torch

app = Flask(__name__)
model = load_model()  # 加载训练好的模型
model.eval()

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    tensor = preprocess(data['input'])  # 预处理输入
    with torch.no_grad():
        output = model(tensor)
    return jsonify({'prediction': output.tolist()})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)