自动化Google Cloud AutoML管道? [英] Automating the google cloud AutoML pipeline?

问题描述

我正在尝试使用Google AutoML自然语言来训练自定义文本多标签分类器.手动地，我可以创建数据集，从Google Storage导入数据并训练模型.但是，我想使整个过程自动化.

I am trying to use the Google AutoML natural language for training the custom text multi-label classifier. Manually, I could do the dataset creation, import data from Google Storage and train the model. However, I want to automate this entire process.

我目前的方法如下:

1. 我已经创建了Google存储桶来存储带注释的数据
2. 我启动了Cloud Function来创建数据集，将数据导入到数据集中并训练模型.

但是，将数据导入数据集要花费9分钟以上的时间，这是Google Cloud Function的最大超时时间，并且永远无法达到训练模型阶段的目的.

However, importing data into dataset, takes much more than 9 minutes which is the maximum timeout for the Google Cloud Function and training the model stage is never reached.

解决方案之一是在完成数据导入以训练模型后，在触发条件触发后启动另一个Cloud Function.我检查了Google AutoML的文档，似乎没有办法实现.

One of the solution could be to spin up another Cloud Function after the trigger is fired when the data import is complete to start training the model. I checked the documentation of Google AutoML and there seems to be no way to achieve that.

还有其他方法可以实现这一目标吗?

Is there any other way to achieve this?

非常感谢您的帮助.谢谢.

Your help is much appreciated. Thank you.

推荐答案

另一种实现此目标的方法是使用AppEngine.这种方法可以通过无服务器的方式运行很长的过程.

Another way to achieve this, in a serverless way and to be able to run a long process is to use AppEngine.

• 使用您可以在端点上触发的代码部署AppEngine标准
• 设置触发您的Appengine端点的Cloud Scheduler .将 attemptDeadline 设置为所需的最大超时时间(不能超过24H)
• 如果事件触发了触发器，则将函数插入事件并创建

• Deploy an AppEngine standard with your code that you can trigger on an endpoint
• Set up a Cloud Scheduler that trigger your appengine endpoint. Set the attemptDeadline to the max timeout that you want (can't exceed 24H)
• If your trigger is fired by event, plug a function to the event and create a Cloud Task on the endpoint to trigger with the dispatchDeadline to the max timeout that you want (can't exceed 24H)

这篇关于自动化Google Cloud AutoML管道?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！