Featured image of post Pytorch实践（刘二大人）8：多分类问题
机器学习

Pytorch实践（刘二大人）8：多分类问题

最后修改:

数学思路

数据表征

将$28 \times 28$像素的灰度图像展开为784维向量，$x ∈ ℝ^{784}$，像素值归一化至[0, 1]区间1。标签采用one-hot编码，构建目标函数$y ∈ {0,1}^{10}$2

前向传播

网络维度变化$784\Longrightarrow 512\Longrightarrow 256\Longrightarrow 128\Longrightarrow 64\Longrightarrow 10$

  • 第一层：$z_1=W_1x+b_1,a_1=ReLU(z_1)=max(0,z_1)$3
  • 第二层：$z_2=W_2a_1+b_2,a_2=ReLU(z_2)$
  • 第三层：$z_3=W_3a_2+b_3,a_3=ReLU(z_3)$
  • 第四层：$z_4=W_4a_3+b_4,a_4=ReLU(z_4)$
  • 输出层：$z_5=W_5a_4+b_5,\hat{y} = \text{softmax}(z_5) = \frac{e^{z_5}}{\sum_{i=1}^{10} e^{z_5^{(i)}}}$

反向传播

$$ \frac{\partial L}{\partial W_1} = \underbrace{\frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial z_5}}_{\delta_5} \cdot \frac{\partial z_5}{\partial a_4} \cdot \frac{\partial a_4}{\partial z_4} \cdot \frac{\partial z_4}{\partial a_3} \cdot \frac{\partial a_3}{\partial z_3} \cdot \frac{\partial z_3}{\partial a_2} \cdot \frac{\partial a_2}{\partial z_2} \cdot \frac{\partial z_2}{\partial a_1} \cdot \frac{\partial a_1}{\partial z_1} \cdot \frac{\partial z_1}{\partial W_1} $$

参数优化

对全部参数${W_i, b_i}^{5}{i=1}$，使用带动量的SGD更新： $$ \begin{aligned} v{W_i} &:= \gamma v_{W_i} + \eta \frac{\partial L}{\partial W_i} \ W_i &:= W_i - v_{W_i} \end{aligned} $$

评估指标

$$ Accuracy = \frac{(Σ_{n=1}^N I(argmax(ŷ^{(n)}) = argmax(y^{(n)})))}{N} $$

代码思路

库导入与数据预处理

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

import torch
from torchvision import transforms, datasets
from torch.utils.data import DataLoader
import torch.nn.functional as F
import torch.optim as optim

batch_size = 64
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])
  • ToTensor()将PIL图像转换为张量，Normalize进行归一化
  • batch_size=64：每批处理64张图片

数据加载

1
2
3
4

train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
test_dataset = datasets.MNIST(root='../dataset/mnist', train=False, download=True, transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)
  • shuffle=True表示训练数据打乱，以增强泛化性

神经网络模型

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.l1 = torch.nn.Linear(784, 512)
        self.l2 = torch.nn.Linear(512, 256)
        self.l3 = torch.nn.Linear(256, 128)
        self.l4 = torch.nn.Linear(128, 64)
        self.l5 = torch.nn.Linear(64, 10)

    def forward(self, x):
        x = x.view(-1, 784)  
        x = F.relu(self.l1(x))
        x = F.relu(self.l2(x))
        x = F.relu(self.l3(x))
        x = F.relu(self.l4(x))
        return self.l5(x)
  • 结构：5层全连接网络（784→512→256→128→64→10），逐步压缩特征。
  • x = x.view(-1, 784)表示将张量转换为一维向量。-1表示自动推导维度

损失函数与优化器

1
2

criterion = torch.nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
  • CrossEntroplLoss用于定义交叉熵损失函数，适用于多分类任务，其中每个样本的标签是单个类别

训练函数

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        inputs, target = data
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print(f'[{epoch+1}, {batch_idx+1}] loss: {running_loss/300:.3f}')
            running_loss = 0.0
  1. 清空梯度：optimizer.zero_grad()
  2. 前向传播：outputs = model(inputs)
  3. 计算损失：loss = criterion(outputs, target)
  4. 反向传播：loss.backward()
  5. 参数更新：optimizer.step()

测试函数

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        inputs, target = data
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print(f'[{epoch+1}, {batch_idx+1}] loss: {running_loss/300:.3f}')
            running_loss = 0.0
  • 评估模式：torch.no_grad()禁用梯度计算，节省内存
  • 预测计算：取输出最大值的索引作为预测类别，统计正确率

代码实现

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

import torch
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader
import torch.nn.functional as F
import torch.optim as optim

batch_size = 64
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])

train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
test_dataset = datasets.MNIST(root='../dataset/mnist', train=False, download=True,transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.l1 = torch.nn.Linear(784, 512)
        self.l2 = torch.nn.Linear(512, 256)
        self.l3 = torch.nn.Linear(256, 128)
        self.l4 = torch.nn.Linear(128, 64)
        self.l5 = torch.nn.Linear(64, 10)

    def forward(self, x):
        x = x.view(-1, 784)
        x = F.relu(self.l1(x))
        x = F.relu(self.l2(x))
        x = F.relu(self.l3(x))
        x = F.relu(self.l4(x))
        return self.l5(x)

model = Net()

criterion = torch.nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        inputs, target = data
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print('[%d, %5d] loss: %.3f' % (epoch+1, batch_idx + 1, running_loss / 300))
            running_loss = 0.0

def test():
    correct = 0
    total = 0
    with torch.no_grad():
        for data in test_loader:
            images, labels = data
            outputs = model(images)
            _, predicted = torch.max(outputs.data, dim=1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()
    print('accuracy on test set: %d %% ' % (100 * correct / total))

if __name__ == '__main__':
    for epoch in range(10):
        train(epoch)
        test()

  1. MNIST图像是灰度图，每个像素值用8位无符号整数，取值范围[0, 255]。通过线性缩放将像素值映射到[0, 1]区间：$x_{归一化}=\frac{x_{原始}}{255}$ ↩︎

  2. One-Hot编码是一种将类别标签（如数字0-9）转换为二进制向量的方法。在MNIST中，假设某张图片的数字是3，One-Hot编码后：[0, 0, 0, 1, 0, 0, 0, 0, 0, 0]。y ∈ {0,1}¹⁰ 表示这是一个长度为10的向量，每个元素只能是0或1，且有且仅有一个1。 ↩︎

  3. ReLU函数：$f(x)=max(0, x)$ ↩︎

comments powered by Disqus