如何在 google-cloud-ml 中设置 pytorch [英] How to setup pytorch in google-cloud-ml
问题描述
我尝试在 google-cloud-ml 中使用 Pytorch
代码抛出工作.所以我编写了setup.py"文件.并添加选项install_requires"
I try to throw job with Pytorch
code in google-cloud-ml.
so I code the "setup.py" file. And add option "install_requires"
"setup.py"
from setuptools import find_packages
from setuptools import setup
REQUIRED_PACKAGES = ['http://download.pytorch.org/whl/cpu/torch-0.3.0.post4-cp27-cp27mu-linux_x86_64.whl','torchvision']
setup(
name='trainer',
version='0.1',
install_requires=REQUIRED_PACKAGES,
packages=find_packages(),
include_package_data=True,
description='My keras trainer application package.'
)
并将工作扔给 google-cloud-ml,但它不起作用
and throw the job to the google-cloud-ml, but it doesn't work
{
insertId: "3m78xtf9czd0u"
jsonPayload: {
created: 1516845879.49039
levelname: "ERROR"
lineno: 829
message: "Command '['pip', 'install', '--user', '--upgrade', '--force-reinstall', '--no-deps', u'trainer-0.1.tar.gz']' returned non-zero exit status 1"
pathname: "/runcloudml.py"
}
labels: {
compute.googleapis.com/resource_id: "6637909247101536087"
compute.googleapis.com/resource_name: "cmle-training-master-5502b52646-0-ql9ds"
compute.googleapis.com/zone: "us-central1-c"
ml.googleapis.com/job_id: "run_ml_engine_pytorch_test_20180125_015752"
ml.googleapis.com/job_id/log_area: "root"
ml.googleapis.com/task_name: "master-replica-0"
ml.googleapis.com/trial_id: ""
}
logName: "projects/exem-191100/logs/master-replica-0"
receiveTimestamp: "2018-01-25T02:04:55.421517460Z"
resource: {
labels: {…}
type: "ml_job"
}
severity: "ERROR"
timestamp: "2018-01-25T02:04:39.490387916Z"
}
====================================================================
====================================================================
那么我如何在谷歌云机器学习引擎中使用 pytorch?
so how can i use pytorch in google cloud ml engine?
推荐答案
我找到了关于在 google-cloud-ml
第一你必须得到一个关于 pytorch 的 .whl
文件并将它存储到谷歌存储桶.您将获得存储桶链接的链接.
first
you have to get a .whl
file about pytorch and store it to google storage bucket.
and you will get the link for bucket link.
gs://bucketname/directory/torch-0.3.0.post4-cp27-cp27mu-linux_x86_64.whl
.whl
文件取决于您的 python 版本或 cuda 版本....
the .whl
file is depend on your python version or cuda version....
第二次您编写命令行和 setup.py,因为您必须设置 google-cloud-ml 设置.相关链接是这个 submit_job_to_ml-engine您编写 setup.py
文件来描述您的设置.相关链接是这个 write_setup.py_file
second
you write the command line and setup.py because you have to set up the google-cloud-ml setting.
related link is this submit_job_to_ml-engine
you write the setup.py
file to describe your setup.
the related link is this write_setup.py_file
这是我的命令代码和 setup.py 文件
this is my command code and setup.py file
====================================================================命令"
===================================================================== "command"
#commandline code
JOB_NAME="run_ml_engine_pytorch_test_$(date +%Y%m%d_%H%M%S)"
REGION=us-central1
OUTPUT_PATH=gs://yourbucket
gcloud ml-engine jobs submit training $JOB_NAME
--job-dir $OUTPUT_PATH
--runtime-version 1.4
--module-name models.pytorch_test
--package-path models/
--packages gs://yourbucket/directory/torch-0.3.0.post4-cp27-cp27mu-linux_x86_64.whl
--region $REGION
--
--verbosity DEBUG
====================================================================setup.py"
===================================================================== "setup.py"
from setuptools import find_packages
from setuptools import setup
REQUIRED_PACKAGES = ['torchvision']
setup(
name='trainer',
version='0.1',
install_requires=REQUIRED_PACKAGES,
packages=find_packages(),
include_package_data=True,
description='My pytorch trainer application package.'
)
====================================================================
=====================================================================
第三个如果您有向 ml-engine 提交作业的经验.你可能知道提交ml-engine的文件结构packaging_training_model.你必须按照上面的链接并知道如何打包文件.
third if you have experience submitting job to the ml-engine. you might know the file structure about submitting ml-engine packaging_training_model. you have to follow above link and know how to pack files.
这篇关于如何在 google-cloud-ml 中设置 pytorch的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!