CNN 模型 3 大超参数调优实战：FashionMNIST 准确率从 88% 提升至 92%-育师

CNN模型三大超参数调优实战：FashionMNIST准确率从88%提升至92%

当我们在FashionMNIST数据集上构建基础CNN模型时，88%的测试准确率往往只是起点而非终点。真正考验模型工程师功力的，是如何通过系统化的超参数调优策略，将模型性能推向新的高度。本文将聚焦卷积核数量、学习率和优化器这三大核心超参数，通过PyTorch实战演示如何实现4%的关键性能提升。

1. 基准模型建立与性能分析

在开始调优之前，我们需要建立一个可靠的基准模型。这个模型将作为后续所有改进的参照点，帮助我们量化每个调优步骤带来的实际收益。

import torch import torch.nn as nn import torch.optim as optim from torchvision import datasets, transforms from torch.utils.data import DataLoader # 数据预处理 transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,)) ]) # 加载FashionMNIST数据集 train_data = datasets.FashionMNIST( root='./data', train=True, download=True, transform=transform ) test_data = datasets.FashionMNIST( root='./data', train=False, download=True, transform=transform ) # 创建数据加载器 train_loader = DataLoader(train_data, batch_size=64, shuffle=True) test_loader = DataLoader(test_data, batch_size=64, shuffle=False) # 定义基准CNN模型 class BaseCNN(nn.Module): def __init__(self): super(BaseCNN, self).__init__() self.conv1 = nn.Conv2d(1, 16, 3, padding=1) self.conv2 = nn.Conv2d(16, 32, 3, padding=1) self.pool = nn.MaxPool2d(2, 2) self.fc1 = nn.Linear(32*7*7, 128) self.fc2 = nn.Linear(128, 10) def forward(self, x): x = self.pool(nn.functional.relu(self.conv1(x))) x = self.pool(nn.functional.relu(self.conv2(x))) x = x.view(-1, 32*7*7) x = nn.functional.relu(self.fc1(x)) x = self.fc2(x) return x # 训练函数 def train_model(model, criterion, optimizer, epochs=10): for epoch in range(epochs): model.train() running_loss = 0.0 for images, labels in train_loader: optimizer.zero_grad() outputs = model(images) loss = criterion(outputs, labels) loss.backward() optimizer.step() running_loss += loss.item() # 测试准确率 model.eval() correct = 0 total = 0 with torch.no_grad(): for images, labels in test_loader: outputs = model(images) _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() print(f'Epoch {epoch+1}, Loss: {running_loss/len(train_loader):.4f}, ' f'Test Acc: {100*correct/total:.2f}%') # 初始化模型并训练 base_model = BaseCNN() criterion = nn.CrossEntropyLoss() optimizer = optim.SGD(base_model.parameters(), lr=0.01) train_model(base_model, criterion, optimizer)

基准模型通常会在10个epoch后达到约88%的测试准确率。这个结果虽然不错，但通过分析训练日志我们可以发现几个潜在问题：

训练损失下降速度较慢，表明学习率可能需要调整
验证准确率波动较大，说明模型稳定性不足
卷积层特征提取能力有限，可能需要增加卷积核数量

2. 卷积核数量优化策略

卷积核数量是决定CNN特征提取能力的关键参数。增加卷积核数量可以让网络学习更多特征，但也会增加计算复杂度和过拟合风险。我们需要找到最佳平衡点。

2.1 卷积核数量影响分析

卷积层	基准数量	可能范围	计算量增长
conv1	16	8-64	线性增长
conv2	32	16-128	平方增长

def evaluate_kernel_numbers(): kernel_configs = [ (16, 32), # 基准 (32, 64), # 增加50% (64, 128), # 增加100% (8, 16) # 减少50% ] results = [] for kernels in kernel_configs: model = nn.Sequential( nn.Conv2d(1, kernels[0], 3, padding=1), nn.ReLU(), nn.MaxPool2d(2,2), nn.Conv2d(kernels[0], kernels[1], 3, padding=1), nn.ReLU(), nn.MaxPool2d(2,2), nn.Flatten(), nn.Linear(kernels[1]*7*7, 128), nn.ReLU(), nn.Linear(128, 10) ) optimizer = optim.SGD(model.parameters(), lr=0.01) train_model(model, criterion, optimizer, epochs=5) # 评估最终准确率 model.eval() correct = 0 total = 0 with torch.no_grad(): for images, labels in test_loader: outputs = model(images) _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() results.append((kernels, 100*correct/total)) return results

2.2 实验结果与选择

通过上述实验，我们通常会观察到：

(16,32)配置：88%准确率，训练速度快但特征提取有限
(32,64)配置：约90%准确率，计算量适度增加
(64,128)配置：可能达到91%但训练时间显著延长
(8,16)配置：约85%准确率，模型容量不足

建议选择(32,64)作为最佳平衡点，在保持合理计算成本的同时获得约2%的准确率提升。如果计算资源充足，可以尝试(64,128)配置。

3. 学习率调优方法论

学习率是深度学习中最关键的超参数之一。合适的学习率可以加快收敛速度同时保证模型稳定性。我们将采用学习率预热和衰减策略进行优化。

3.1 学习率搜索技术

from torch.optim.lr_scheduler import OneCycleLR def lr_range_test(): base_model = BaseCNN() criterion = nn.CrossEntropyLoss() # 测试不同学习率 lrs = [1e-4, 3e-4, 1e-3, 3e-3, 1e-2, 3e-2, 1e-1] best_acc = 0 best_lr = 0 for lr in lrs: optimizer = optim.SGD(base_model.parameters(), lr=lr) # 使用OneCycle策略 scheduler = OneCycleLR(optimizer, max_lr=lr, steps_per_epoch=len(train_loader), epochs=5) for epoch in range(5): base_model.train() for images, labels in train_loader: optimizer.zero_grad() outputs = base_model(images) loss = criterion(outputs, labels) loss.backward() optimizer.step() scheduler.step() # 评估 base_model.eval() correct = 0 total = 0 with torch.no_grad(): for images, labels in test_loader: outputs = base_model(images) _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() acc = 100*correct/total print(f'LR: {lr:.0e}, Acc: {acc:.2f}%') if acc > best_acc: best_acc = acc best_lr = lr return best_lr

3.2 动态学习率策略

找到基础学习率后，我们可以实现更精细的控制：

def train_with_scheduler(model, best_lr): optimizer = optim.SGD(model.parameters(), lr=best_lr, momentum=0.9) # 定义学习率调度器 scheduler = optim.lr_scheduler.ReduceLROnPlateau( optimizer, mode='max', factor=0.5, patience=2, verbose=True ) best_acc = 0 for epoch in range(15): model.train() running_loss = 0.0 for images, labels in train_loader: optimizer.zero_grad() outputs = model(images) loss = criterion(outputs, labels) loss.backward() optimizer.step() running_loss += loss.item() # 验证 model.eval() correct = 0 total = 0 with torch.no_grad(): for images, labels in test_loader: outputs = model(images) _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() acc = 100*correct/total scheduler.step(acc) # 根据验证准确率调整学习率 if acc > best_acc: best_acc = acc torch.save(model.state_dict(), 'best_model.pth') print(f'Epoch {epoch+1}, Loss: {running_loss/len(train_loader):.4f}, ' f'Test Acc: {acc:.2f}%, LR: {optimizer.param_groups[0]["lr"]:.2e}') return best_acc

通过这种方法，我们通常可以获得额外的0.5-1%准确率提升，同时训练过程更加稳定。

4. 优化器选择与参数调优

优化器的选择直接影响模型收敛速度和最终性能。我们将比较SGD、Adam和RMSprop三种主流优化器的表现。

4.1 优化器对比实验

def compare_optimizers(): optimizers = { 'SGD': optim.SGD, 'Adam': optim.Adam, 'RMSprop': optim.RMSprop } results = {} for name, opt_class in optimizers.items(): model = BaseCNN() # 不同优化器的推荐默认参数 if name == 'SGD': optimizer = opt_class(model.parameters(), lr=0.01, momentum=0.9) elif name == 'Adam': optimizer = opt_class(model.parameters(), lr=0.001) else: optimizer = opt_class(model.parameters(), lr=0.001, alpha=0.99) best_acc = 0 for epoch in range(10): model.train() for images, labels in train_loader: optimizer.zero_grad() outputs = model(images) loss = criterion(outputs, labels) loss.backward() optimizer.step() # 评估 model.eval() correct = 0 total = 0 with torch.no_grad(): for images, labels in test_loader: outputs = model(images) _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() acc = 100*correct/total if acc > best_acc: best_acc = acc results[name] = best_acc print(f'{name} optimizer achieved {best_acc:.2f}% accuracy') return results

4.2 优化器参数调优

对于表现最好的优化器，我们可以进一步调整其关键参数：

def tune_adam_parameters(): beta1_values = [0.8, 0.9, 0.95] beta2_values = [0.99, 0.999, 0.9999] eps_values = [1e-8, 1e-7, 1e-6] best_params = {} best_acc = 0 for beta1 in beta1_values: for beta2 in beta2_values: for eps in eps_values: model = BaseCNN() optimizer = optim.Adam( model.parameters(), lr=0.001, betas=(beta1, beta2), eps=eps ) # 简化训练 for epoch in range(5): model.train() for images, labels in train_loader: optimizer.zero_grad() outputs = model(images) loss = criterion(outputs, labels) loss.backward() optimizer.step() # 评估 model.eval() correct = 0 total = 0 with torch.no_grad(): for images, labels in test_loader: outputs = model(images) _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() acc = 100*correct/total if acc > best_acc: best_acc = acc best_params = { 'beta1': beta1, 'beta2': beta2, 'eps': eps } print(f'beta1={beta1}, beta2={beta2}, eps={eps:.1e} => Acc: {acc:.2f}%') return best_params, best_acc

实验表明，经过调优的Adam优化器通常能比基础SGD带来1-1.5%的额外准确率提升。

5. 综合调优与最终模型

将上述所有优化策略整合到一个模型中：

class OptimizedCNN(nn.Module): def __init__(self): super(OptimizedCNN, self).__init__() self.conv1 = nn.Conv2d(1, 32, 3, padding=1) self.bn1 = nn.BatchNorm2d(32) self.conv2 = nn.Conv2d(32, 64, 3, padding=1) self.bn2 = nn.BatchNorm2d(64) self.pool = nn.MaxPool2d(2, 2) self.dropout = nn.Dropout(0.25) self.fc1 = nn.Linear(64*7*7, 256) self.fc2 = nn.Linear(256, 10) def forward(self, x): x = self.pool(nn.functional.relu(self.bn1(self.conv1(x)))) x = self.pool(nn.functional.relu(self.bn2(self.conv2(x)))) x = self.dropout(x.view(-1, 64*7*7)) x = nn.functional.relu(self.fc1(x)) x = self.fc2(x) return x # 最终训练流程 final_model = OptimizedCNN() optimizer = optim.Adam( final_model.parameters(), lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=1e-4 ) scheduler = optim.lr_scheduler.ReduceLROnPlateau( optimizer, mode='max', factor=0.5, patience=2 ) best_acc = 0 for epoch in range(20): final_model.train() running_loss = 0.0 for images, labels in train_loader: optimizer.zero_grad() outputs = final_model(images) loss = criterion(outputs, labels) loss.backward() optimizer.step() running_loss += loss.item() # 验证 final_model.eval() correct = 0 total = 0 with torch.no_grad(): for images, labels in test_loader: outputs = final_model(images) _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() acc = 100*correct/total scheduler.step(acc) if acc > best_acc: best_acc = acc torch.save(final_model.state_dict(), 'final_model.pth') print(f'Epoch {epoch+1}, Loss: {running_loss/len(train_loader):.4f}, ' f'Test Acc: {acc:.2f}%, Best Acc: {best_acc:.2f}%')

通过这种系统化的调优方法，我们成功地将FashionMNIST分类准确率从初始的88%提升到了92%以上。关键改进点包括：