如何在 SageMaker 上安排任务 [英] How to schedule tasks on SageMaker

问题描述

我有一个关于 SageMaker 的笔记本，我想每天晚上运行.安排此任务的最佳方式是什么.有没有办法从 SageMaker 运行 bash 脚本并安排 Cron 作业?

I have a notebook on SageMaker I would like to run every night. What's the best way to schedule this task. Is there a way to run a bash script and schedule Cron job from SageMaker?

推荐答案

Amazon SageMaker 是一组可以帮助各种机器学习和数据科学任务的 API.这些 API 可以从各种来源调用，例如 CLI、SDK 或特别是从调度 AWS Lambda 函数(有关文档，请参见此处:https://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html )

Amazon SageMaker is a set of API that can help various machine learning and data science tasks. These API can be invoked from various sources, such as CLI, SDK or specifically from schedule AWS Lambda functions (see here for documentation: https://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html )

Amazon SageMaker 的主要部分是笔记本实例、训练和调整作业以及用于实时预测的模型托管.每个都有您可能想要的不同类型的时间表.最受欢迎的是:

The main parts of Amazon SageMaker are notebook instances, training and tuning jobs, and model hosting for real-time predictions. Each one has different types of schedules that you might want to have. The most popular are:

• 停止和启动笔记本实例 - 由于笔记本实例用于交互式 ML 模型开发，因此您实际上不需要它们在夜间或周末运行.您可以安排 Lambda 函数来调用 stop-notebook-instance 工作日结束时的 API(例如晚上 8 点)，以及 start-notebook-instance API 在早上.请注意，您也可以在笔记本实例上运行 crontab(在从 Jupyter 界面打开本地终端后).
• 刷新 ML 模型 - 针对不断流入系统的新数据自动重新训练模型是一个常见问题，使用 SageMaker 更容易解决.从预定的 Lambda 函数调用 create-training-job API(甚至来自一个 CloudWatch Event，它正在监控现有模型)，指向新旧数据所在的 S3 存储桶，可以 创建一个更新的模型，您现在可以将其部署到A/B 测试环境 .

• Stopping and Starting Notebook Instances - Since the notebook instances are used for interactive ML models development, you don't really need them running during the nights or weekends. You can schedule a Lambda function to call the stop-notebook-instance API at the end of the working day (8PM, for example), and the start-notebook-instance API in the morning. Please note that you can also run crontab on the notebook instances (after opening the local terminal from the Jupyter interface).
• Refreshing an ML Model - Automating the re-training of models, on new data that is flowing into the system all the time, is a common issue that with SageMaker is easier to solve. Calling create-training-job API from a scheduled Lambda function (or even from a CloudWatch Event that is monitoring the performance of the existing models), pointing to the S3 bucket where the old and new data resides, can create a refreshed model that you can now deploy into an A/B testing environment .

----- 更新(感谢@snat2100 评论)-----

----- UPDATE (thanks to @snat2100 comment) -----

• 创建和删除实时端点 - 如果您的实时端点不是 24/7 全天候需要的(例如，为在工作日和工作时间工作的公司内部用户提供服务)，您还可以早上创建端点，在晚上删除它们.

• Creating and Deleting Real-time Endpoints - If your realtime endpoints are not needed 24/7 (for example, serving internal company users working during workdays and hours), you can also create the endpoints in the morning and delete them at night.

这篇关于如何在 SageMaker 上安排任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！