AWS Sagemaker SKlearn入口点允许多个脚本 [英] AWS Sagemaker SKlearn entry point allow multiple script

问题描述

我正在尝试遵循教程

I am trying to follow the tutorial here to implement a custom inference pipeline for feature preprocessing. It uses the python sklearn sdk to bring in custom preprocessing pipeline from a script. For example:

from sagemaker.sklearn.estimator import SKLearn

script_path = 'preprocessing.py'

sklearn_preprocessor = SKLearn(
    entry_point=script_path,
    role=role,
    train_instance_type="ml.c4.xlarge",
    sagemaker_session=sagemaker_session)

但是我找不到发送多个文件的方法.我需要多个文件的原因是因为我有一个需要在sklearn管道中使用的自定义类，该类需要从自定义模块中导入.如果不导入，则由于泡菜的工作方式(至少我认为它与泡菜有关)，在同一preprocessing.py文件中包含自定义类时，会引发错误AttributeError: module '__main__' has no attribute 'CustomClassName'.

However I can't find a way to send multiple files. The reason I need multiple files is because I have a custom class used in the sklearn pipeline needs to be imported from a custom module. Without importing, it raises error AttributeError: module '__main__' has no attribute 'CustomClassName' when having the custom class in the same preprocessing.py file due to the way pickle works (at least I think it's related to pickle).

有人知道甚至可以发送多个文件吗?

Anyone know if sending multiple files is even possible?

Sagemaker的新手，谢谢！

Newbie to Sagemaker, thanks!!

推荐答案

有一个source_dir参数可以将文件目录提升"到容器中，并将其放在导入路径上.

There's a source_dir parameter which will "lift" a directory of files to the container and put it on your import path.

您的入口点脚本应放在该位置并从该位置引用.

You're entrypoint script should be put there to and referenced from that location.

这篇关于AWS Sagemaker SKlearn入口点允许多个脚本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！