优化apache梁/云数据流启动 [英] Optimizing apache beam / cloud dataflow startup

问题描述

我使用自动缩放工作器和 1 个工作器对 apache-beam 进行了一些测试，每次我看到大约 2 分钟的启动时间.是否可以减少该时间?如果可以，建议的减少启动时间的最佳做法是什么?

I have done a few tests with apache-beam using both auto-scale workers and 1 worker, and each time I see a startup time of around 2 minutes. Is it possible to reduce that time, and if so, what are the suggested best practices for reducing the startup time?

推荐答案

恕我直言:对于像 Cloud Dataflow 这样的产品来说，两分钟是非常快的.请记住，Google 正在为您推出强大的大数据服务，该服务可自动扩展.

IMHO: Two minutes is very fast for a product like Cloud Dataflow. Remember, Google is launching a powerful Big Data service for you that autoscales.

将该时间与其他云供应商进行比较.我已经看到一些集群 (Hadoop) 需要 15 分钟才能上线.无论如何，您无法控制 Dataflow 的初始化过程，因此您无需改进.

Compare that time to the other cloud vendors. I have seen some clusters (Hadoop) take 15 minutes to come live. In any event, you do not control the initialization process for Dataflow so there is nothing for you to improve.

这篇关于优化apache梁/云数据流启动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！