PyTorch 教程错误训练分类器 [英] PyTorch Tutorial Error Training a Classifier

问题描述

我刚刚开始学习 PyTorch 教程使用 PyTorch 进行深度学习:60 分钟闪电战，我应该补充一点，我之前没有编写过任何 Python(但其他语言，如 Java).>

现在，我的代码看起来像

导入火炬进口火炬视觉导入 torchvision.transforms 作为转换导入 matplotlib.pyplot 作为 plt将 numpy 导入为 npprint("\n-------------------反向传播-------------------\n")变换 = 变换.撰写([transforms.ToTensor(),transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])trainset = torchvision.datasets.CIFAR10(root='./data', train=True,download=True, transform=transform)trainloader = torch.utils.data.DataLoader(trainset，batch_size=4，shuffle=True，num_workers=2)testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)testloader = torch.utils.data.DataLoader(测试集，batch_size=4，shuffle=False，num_workers=2)类 = ('飞机', '汽车', '鸟', '猫', '鹿', '狗', '青蛙', '马', '船', '卡车')dataiter = iter(trainloader)图像，标签 = dataiter.next()定义 imshow(img):img = img/2 + 0.5npimg = img.numpy()plt.imshow(np.transpose(npimg, (1, 2, 0)))imshow(torchvision.utils.make_grid(images))print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

这应该与教程一致.如果我执行这个，我会得到以下错误:

"C:\Program Files\Anaconda3\python.exe" C:/MA/pytorch/deepLearningWithPytorchTutorial/trainingClassifier.py-------------------反向传播-------------------已下载并验证的文件已下载并验证的文件-------------------反向传播-------------------已下载并验证的文件已下载并验证的文件回溯(最近一次调用最后一次):文件<string>"，第 1 行，在 <module> 中文件C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py"，第 105 行，在 spawn_main退出代码 = _main(fd)文件C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py"，第 114 行，在 _main准备(准备数据)文件C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py"，第 225 行，准备中_fixup_main_from_path(数据['init_main_from_path'])文件C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py"，第 277 行，在 _fixup_main_from_pathrun_name="__mp_main__")文件C:\Program Files\Anaconda3\lib\runpy.py"，第 263 行，在 run_pathpkg_name=pkg_name, script_name=fname)文件C:\Program Files\Anaconda3\lib\runpy.py"，第 96 行，_run_module_codemod_name、mod_spec、pkg_name、script_name)文件C:\Program Files\Anaconda3\lib\runpy.py"，第 85 行，在 _run_code 中exec(代码，run_globals)文件C:\MA\pytorch\deepLearningWithPytorchTutorial\trainingClassifier.py"，第 23 行，在 <module>dataiter = iter(trainloader)文件C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py"，第 451 行，在 __iter__ 中返回 _DataLoaderIter(self)文件C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py"，第 239 行，在 __init__w.start()文件C:\Program Files\Anaconda3\lib\multiprocessing\process.py"，第 105 行，在开始self._popen = self._Popen(self)文件C:\Program Files\Anaconda3\lib\multiprocessing\context.py"，第 223 行，在 _Popen返回_default_context.get_context().Process._Popen(process_obj)文件C:\Program Files\Anaconda3\lib\multiprocessing\context.py"，第 322 行，在 _Popen返回 Popen(process_obj)文件C:\Program Files\Anaconda3\lib\multiprocessing\popen_spawn_win32.py"，第 33 行，在 __init__ 中prep_data = spawn.get_preparation_data(process_obj._name)文件C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py"，第 143 行，在 get_preparation_data 中_check_not_importing_main()文件C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py"，第 136 行，在 _check_not_importing_main不会被冻结以生成可执行文件.''')运行时错误:已尝试在开始之前启动一个新进程当前进程已完成其引导阶段.这可能意味着您没有使用 fork 来启动您的子进程，而您忘记使用正确的习语在主模块中:如果 __name__ == '__main__':冻结支持()...freeze_support()"行可以省略，如果程序不会被冻结以生成可执行文件.回溯(最近一次调用最后一次):文件C:/MA/pytorch/deepLearningWithPytorchTutorial/trainingClassifier.py"，第 23 行，在 <module>dataiter = iter(trainloader)文件C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py"，第 451 行，在 __iter__ 中返回 _DataLoaderIter(self)文件C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py"，第 239 行，在 __init__w.start()文件C:\Program Files\Anaconda3\lib\multiprocessing\process.py"，第 105 行，在开始self._popen = self._Popen(self)文件C:\Program Files\Anaconda3\lib\multiprocessing\context.py"，第 223 行，在 _Popen返回_default_context.get_context().Process._Popen(process_obj)文件C:\Program Files\Anaconda3\lib\multiprocessing\context.py"，第 322 行，在 _Popen返回 Popen(process_obj)文件C:\Program Files\Anaconda3\lib\multiprocessing\popen_spawn_win32.py"，第 65 行，在 __init__ 中减少.转储(process_obj，to_child)转储中的文件C:\Program Files\Anaconda3\lib\multiprocessing\reduction.py"，第 60 行ForkingPickler(文件，协议).转储(对象)BrokenPipeError: [Errno 32] 管道损坏进程以退出代码 1 结束

我已经下载了 *.py 和 *.ipynb.使用 jupyter 运行 *.ipynb 工作正常(但我不想在瞻博网络界面中编程，我更喜欢 pyCharm)而 *.py 在控制台(Anaconda 提示符和 cmd)失败并出现相同的错误.

有谁知道如何解决这个问题?(我使用的是 Python 3.6.5(来自 Anaconda)和 pyCharm，操作系统:Win10 64 位)

谢谢！贝恩

更新:如果相关，我只需将 num_workers=2 设置为 num_workers=0(两者)，然后它就会工作......

解决方案

查看多处理:Windows 编程指南.您应该将所有操作包装在函数中，然后在 if __name__ == '__main__' 子句中调用它们:

# 需要导入def load_datasets(...):# 使用多个worker加载数据集的代码定义火车(...):# 训练模型的代码如果 __name__ == '__main__':加载数据集()火车()

简而言之，这里的想法是将示例代码包装在 if __name__ == '__main__' 语句中.

I just started the PyTorch-Tutorial Deep Learning with PyTorch: A 60 Minute Blitz and I should add, that I haven't programmed any python (but other languages like Java) before.

Right now, my Code looks like

import torch
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
import numpy as np


print("\n-------------------Backpropagation-------------------\n")
transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,download=True, transform=transform)

trainloader = torch.utils.data.DataLoader(trainset, batch_size=4, shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)

testloader = torch.utils.data.DataLoader(testset, batch_size=4, shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

dataiter = iter(trainloader)
images, labels = dataiter.next()


def imshow(img):
    img = img / 2 + 0.5
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))


imshow(torchvision.utils.make_grid(images))

print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

which should be consistent with the tutorial. If I execute this, I'll get the following error:

"C:\Program Files\Anaconda3\python.exe" C:/MA/pytorch/deepLearningWithPytorchTutorial/trainingClassifier.py

-------------------Backpropagation-------------------

Files already downloaded and verified
Files already downloaded and verified

-------------------Backpropagation-------------------

Files already downloaded and verified
Files already downloaded and verified
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Program Files\Anaconda3\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Program Files\Anaconda3\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Program Files\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\MA\pytorch\deepLearningWithPytorchTutorial\trainingClassifier.py", line 23, in <module>
    dataiter = iter(trainloader)
  File "C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 451, in __iter__
    return _DataLoaderIter(self)
  File "C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 239, in __init__
    w.start()
  File "C:\Program Files\Anaconda3\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "C:\Program Files\Anaconda3\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "C:/MA/pytorch/deepLearningWithPytorchTutorial/trainingClassifier.py", line 23, in <module>
    dataiter = iter(trainloader)
  File "C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 451, in __iter__
    return _DataLoaderIter(self)
  File "C:\Program Files\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 239, in __init__
    w.start()
  File "C:\Program Files\Anaconda3\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe

Process finished with exit code 1

I already downloaded the *.py and *.ipynb. Running the *.ipynb with jupyter works fine (but I don't want to programm in the juniper web-interface, I prefer pyCharm) while the *.py in the console (Anaconda prompt and cmd) fails with the same error.

Does anyone know how to fix this? (I'm using Python 3.6.5 (from Anaconda) and pyCharm, OS: Win10 64-bit)

Thanks! Bene

Update: If it is relevant, I just set num_workers=2 to num_workers=0 (both) and then it'll work.. .

解决方案

Check out the documentation for multiprocessing: programming guidelines for windows. You should wrap all operations in functions and then call them inside an if __name__ == '__main__' clause:

# required imports

def load_datasets(...):
    # Code to load the datasets with multiple workers

def train(...):
    # Code to train the model

if __name__ == '__main__':
    load_datasets()
    train()

In short, the the idea here is to wrap the example code inside an if __name__ == '__main__' statement.

这篇关于PyTorch 教程错误训练分类器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！