如何在google-cloud-ml中设置pytorch [英] How to setup pytorch in google-cloud-ml

问题描述

我尝试用google-cloud-ml中的Pytorch代码抛出工作. 所以我编写了"setup.py"文件.并添加选项"install_requires"

I try to throw job with Pytorch code in google-cloud-ml. so I code the "setup.py" file. And add option "install_requires"

"setup.py"

"setup.py"

from setuptools import find_packages
from setuptools import setup
REQUIRED_PACKAGES = ['http://download.pytorch.org/whl/cpu/torch-0.3.0.post4-cp27-cp27mu-linux_x86_64.whl','torchvision']

setup(
    name='trainer',
    version='0.1',
    install_requires=REQUIRED_PACKAGES,
    packages=find_packages(),
    include_package_data=True,
    description='My keras trainer application package.'
)

并将作业丢到google-cloud-ml，但这没用

and throw the job to the google-cloud-ml, but it doesn't work

{
 insertId:  "3m78xtf9czd0u"  
 jsonPayload: {
  created:  1516845879.49039   
  levelname:  "ERROR"   
  lineno:  829   
  message:  "Command '['pip', 'install', '--user', '--upgrade', '--force-reinstall', '--no-deps', u'trainer-0.1.tar.gz']' returned non-zero exit status 1"   
  pathname:  "/runcloudml.py"   
 }
 labels: {
  compute.googleapis.com/resource_id:  "6637909247101536087"   
  compute.googleapis.com/resource_name:  "cmle-training-master-5502b52646-0-ql9ds"   
  compute.googleapis.com/zone:  "us-central1-c"   
  ml.googleapis.com/job_id:  "run_ml_engine_pytorch_test_20180125_015752"   
  ml.googleapis.com/job_id/log_area:  "root"   
  ml.googleapis.com/task_name:  "master-replica-0"   
  ml.googleapis.com/trial_id:  ""   
 }
 logName:  "projects/exem-191100/logs/master-replica-0"  
 receiveTimestamp:  "2018-01-25T02:04:55.421517460Z"  
 resource: {
  labels: {…}   
  type:  "ml_job"   
 }
 severity:  "ERROR"  
 timestamp:  "2018-01-25T02:04:39.490387916Z"  
}

================================================ ====================

====================================================================

在此处查看详细消息

那么我该如何在Google Cloud ml引擎中使用pytorch?

so how can i use pytorch in google cloud ml engine?

推荐答案

我在 google-cloud-ml

第一 您必须获取有关pytorch的.whl文件，并将其存储到Google存储桶. 您将获得存储桶链接的链接.

first you have to get a .whl file about pytorch and store it to google storage bucket. and you will get the link for bucket link.

gs://bucketname/directory/torch-0.3.0.post4-cp27-cp27mu-linux_x86_64.whl

.whl文件取决于您的python版本或cuda版本....

the .whl file is depend on your python version or cuda version....

秒 您必须编写命令行和setup.py，因为您必须设置google-cloud-ml设置. 相关链接是此 submit_job_to_ml-engine 您编写setup.py文件来描述您的设置. 相关链接是此 write_setup.py_file

second you write the command line and setup.py because you have to set up the google-cloud-ml setting. related link is this submit_job_to_ml-engine you write the setup.py file to describe your setup. the related link is this write_setup.py_file

这是我的命令代码和setup.py文件

this is my command code and setup.py file

=============================================== ====================== 命令"

===================================================================== "command"

#commandline code
JOB_NAME="run_ml_engine_pytorch_test_$(date +%Y%m%d_%H%M%S)"
REGION=us-central1
OUTPUT_PATH=gs://yourbucket
gcloud ml-engine jobs submit training $JOB_NAME \
    --job-dir $OUTPUT_PATH \
    --runtime-version 1.4 \
    --module-name models.pytorch_test \
    --package-path models/ \
    --packages gs://yourbucket/directory/torch-0.3.0.post4-cp27-cp27mu-linux_x86_64.whl \
    --region $REGION \
    -- \
    --verbosity DEBUG

================================================ ====================== "setup.py"

===================================================================== "setup.py"

from setuptools import find_packages
from setuptools import setup
REQUIRED_PACKAGES = ['torchvision']
setup(
    name='trainer',
    version='0.1',
    install_requires=REQUIRED_PACKAGES,
    packages=find_packages(),
    include_package_data=True,
    description='My pytorch trainer application package.'
)

================================================ =====================

=====================================================================

第三名 如果您有向ml引擎提交工作的经验. 您可能知道有关提交ml-engine的文件结构 packaging_training_model . 您必须点击上面的链接，然后知道如何打包文件.

third if you have experience submitting job to the ml-engine. you might know the file structure about submitting ml-engine packaging_training_model. you have to follow above link and know how to pack files.

这篇关于如何在google-cloud-ml中设置pytorch的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！