pytorch/cifar10_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "训练一个分类器\n",
    "=====================\n",
    "\n",
    "就是这个, 你已经看到了如何定义神经网络, 计算损失并更新网络的权重.\n",
    "\n",
    "现在你可能会想,\n",
    "\n",
    "数据呢?\n",
    "----------------\n",
    "\n",
    "一般来说, 当你不得不处理图像, 文本, 音频或者视频数据时,\n",
    "你可以使用标准的 Python 包将数据加载到一个 numpy 数组中.\n",
    "然后你可以将这个数组转换成一个 ``torch.*Tensor``.\n",
    "\n",
    "-  对于图像, 会用到的包有 Pillow, OpenCV .\n",
    "-  对于音频, 会用的包有 scipy 和 librosa.\n",
    "-  对于文本, 原始 Python 或基于 Cython 的加载, 或者 NLTK 和 Spacy 都是有用的.\n",
    "\n",
    "\n",
    "特别是对于 ``vision``, 我们已经创建了一个叫做 ``torchvision``, 其中有对普通数据集如\n",
    "Imagenet, CIFAR10, MNIST 等和用于图像数据的转换器, 即 ``torchvision.datasets`` 和 ``torch.utils.data.DataLoader``.\n",
    "\n",
    "这提供了巨大的便利, 避免了编写重复代码.\n",
    "\n",
    "在本教程中, 我们将使用 CIFAR10 数据集.\n",
    "它有: ‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’ 这些类别.\n",
    "CIFAR10 中的图像大小为 3x32x32 , 即 32x32 像素的 3 通道彩色图像.\n",
    "\n",
    ".. figure:: /_static/img/cifar10.png\n",
    "   :alt: cifar10\n",
    "\n",
    "   cifar10\n",
    "\n",
    "\n",
    "训练一个图像分类器\n",
    "----------------------------\n",
    "\n",
    "我们将按顺序执行以下步骤:\n",
    "\n",
    "1. 加载 CIFAR10 测试和训练数据集并规范化\n",
    "   ``torchvision``\n",
    "2. 定义一个卷积神经网络\n",
    "3. 定义一个损失函数\n",
    "4. 在训练数据上训练网络\n",
    "5. 在测试数据上测试网络\n",
    "\n",
    "1. 加载并规范化 CIFAR10\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "使用 ``torchvision``, 加载 CIFAR10 非常简单.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torchvision 数据集的输出是范围 [0, 1] 的 PILImage 图像. 我们将它们转换为归一化范围是[-1,1]的张量\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
    "                                        download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n",
    "                                          shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们展示一些训练图像, 只是为了好玩 (0.0).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 定义函数来显示图像\n",
    "\n",
    "\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # 非标准化\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "\n",
    "\n",
    "# 得到一些随机的训练图像\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# 显示图像\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# 输出类别\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. 定义一个卷积神经网络\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "从神经网络部分复制神经网络, 并修改它以获取 3 通道图像(而不是定义的 1 通道图像).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.autograd import Variable\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. 定义一个损失函数和优化器\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "我们使用交叉熵损失函数( CrossEntropyLoss )和随机梯度下降( SGD )优化器.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. 训练网络\n",
    "^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "这是事情开始变得有趣的时候.\n",
    "我们只需循环遍历数据迭代器, 并将输入提供给网络和优化器.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for epoch in range(2):  # 循环遍历数据集多次\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # 得到输入数据\n",
    "        inputs, labels = data\n",
    "\n",
    "        # 包装数据\n",
    "        inputs, labels = Variable(inputs), Variable(labels)\n",
    "\n",
    "        # 梯度清零\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # 打印信息\n",
    "        running_loss += loss.data[0]\n",
    "        if i % 2000 == 1999:    # 每2000个小批量打印一次\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. 在测试数据上测试网络\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "我们在训练数据集上训练了2遍网络, 但是我们需要检查网络是否学到了什么.\n",
    "\n",
    "我们将通过预测神经网络输出的类标签来检查这个问题, 并根据实际情况进行检查.\n",
    "如果预测是正确的, 我们将样本添加到正确预测的列表中.\n",
    "\n",
    "好的, 第一步. 让我们显示测试集中的图像以便熟悉.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# 打印图像\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好的, 现在让我们看看神经网络认为这些例子是什么:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = net(Variable(images))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出的是10个类别的能量.\n",
    "一个类别的能量越高, 则可以理解为网络认为越多的图像是该类别的.\n",
    "那么, 让我们得到最高能量的索引:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "_, predicted = torch.max(outputs.data, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
    "                              for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果看起来不错.\n",
    "\n",
    "让我们看看网络如何在整个数据集上执行.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "for data in testloader:\n",
    "    images, labels = data\n",
    "    outputs = net(Variable(images))\n",
    "    _, predicted = torch.max(outputs.data, 1)\n",
    "    total += labels.size(0)\n",
    "    correct += (predicted == labels).sum()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练的准确率远比随机猜测(准确率10%)好, 证明网络确实学到了东西. \n",
    "\n",
    "嗯, 我们来看看哪些类别表现良好, 哪些类别表现不佳:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "for data in testloader:\n",
    "    images, labels = data\n",
    "    outputs = net(Variable(images))\n",
    "    _, predicted = torch.max(outputs.data, 1)\n",
    "    c = (predicted == labels).squeeze()\n",
    "    for i in range(4):\n",
    "        label = labels[i]\n",
    "        class_correct[label] += c[i]\n",
    "        class_total[label] += 1\n",
    "\n",
    "\n",
    "for i in range(10):\n",
    "    print('Accuracy of %5s : %2d %%' % (\n",
    "        classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好的, 接下来呢?\n",
    "\n",
    "我们如何在 GPU 上运行这些神经网络?\n",
    "\n",
    "在 GPU 上训练\n",
    "----------------\n",
    "就像你如何将一个张量传递给GPU一样, 你将神经网络转移到GPU上. 这将递归遍历所有模块, 并将其参数和缓冲区转换为CUDA张量:\n",
    "\n",
    ".. code:: python\n",
    "\n",
    "    net.cuda()\n",
    "\n",
    "\n",
    "请记住, 您必须将输入和目标每一步都发送到GPU:\n",
    "\n",
    "::\n",
    "\n",
    "        inputs, labels = Variable(inputs.cuda()), Variable(labels.cuda())\n",
    "\n",
    "如果发现在 GPU 上并没有比 CPU 提速很多, 实际上是因为网络比较小, GPU 没有完全发挥自己的真正实力.\n",
    "\n",
    "**练习:** 尝试增加网络的宽度(第一个 ``nn.Conv2d`` 的参数2和第二个 ``nn.Conv2d`` 的参数1 它们需要是相同的数字),\n",
    "看看你得到什么样的加速.\n",
    "\n",
    "**目标达成**:\n",
    "\n",
    "- 深入了解PyTorch的张量库和神经网络.\n",
    "- 训练一个小的神经网络来分类图像.\n",
    "\n",
    "在多个GPU上进行训练\n",
    "-------------------------\n",
    "如果你希望使用所有 GPU 来看更多的 MASSIVE 加速, 请查看可选 :doc:`data_parallel_tutorial`.\n",
    "\n",
    "我下一步去哪里?\n",
    "-------------------\n",
    "\n",
    "-  :doc:`训练神经网络玩电子游戏 </intermediate/reinforcement_q_learning>`\n",
    "-  `在 imagenet 上培训最先进的 ResNet 网络`\n",
    "-  `利用生成对抗网络训练人脸生成器`\n",
    "-  `使用 Recurrent LSTM 网络训练单词语言模型`\n",
    "-  `更多的例子`\n",
    "-  `更多教程`\n",
    "-  `在论坛上讨论 PyTorch`\n",
    "-  `与 Slack 上与其他用户聊天`\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}