确定性地创建和标记 EC2 实例 [英] Deterministically creating and tagging EC2 instances

问题描述

我正在创建 3 个 EC2 实例，然后迭代并标记每个实例.有时标签请求会失败，尽管实例后来似乎正在运行.

I am creating 3 EC2 instances, and subsequently iterating and tagging each of them. Sometimes the tag request fails, although the instance later appears to be running.

这可能是时间问题吗?创建实例后，我应该在标记它之前等待几秒钟吗?有没有确定性的方法来等待它启动?

Could this be a timing issue? Should I wait a few seconds after creating the instance before tagging it? Is there a deterministic way to wait for it to start?

推荐答案

更新 20140512

AWS 同时添加了关于 Troubleshooting API 的更详细文档请求错误，包括处理最终一致性，这基本上证实了我在下面的初始答案中的分析:

Update 20140512

AWS has meanwhile added more detailed documentation on Troubleshooting API Request Errors, including a section addressing Eventual Consistency, which basically confirms the analysis in my initial answer below:

由于支持 API 的系统的分布式特性，Amazon EC2 API 遵循最终一致性模型.这意味着您运行的影响您的 Amazon EC2 资源的 API 命令的结果可能不会立即对您运行的所有后续命令可见.[...]

The Amazon EC2 API follows an eventual consistency model, due to the distributed nature of the system supporting the API. This means that the result of an API command you run that affects your Amazon EC2 resources might not be immediately visible to all subsequent commands you run. [...]

[...] 例如，[...] 如果你运行一个命令来修改或描述你刚刚创建的资源，它的 ID 可能没有在整个系统中传播，你会得到一个响应资源不存在的错误.

[...] For example, [...] if you run a command to modify or describe the resource that you just created, its ID might not have propagated throughout the system, and you will get an error responding that the resource does not exist.

要管理最终一致性，您可以执行以下操作:

To manage eventual consistency, you can do the following:

• 在运行命令修改资源之前确认资源的状态.使用指数退避运行适当的 Describe 命令算法，以确保您留出足够的时间进行前一个命令在系统中传播.[...]

• Confirm the state of the resource before you run a command to modify it. Run the appropriate Describe command using an exponential backoff algorithm to ensure that you allow enough time for the previous command to propagate through the system. [...]

在后续命令之间添加等待时间，即使 Describe 命令返回准确响应.应用指数退避算法以几秒钟的等待时间开始，并增加逐渐增加到大约五分钟的等待时间.

Add wait time between subsequent commands, even if a Describe command returns an accurate response. Apply an exponential backoff algorithm starting with a couple of seconds of wait time, and increase gradually up to about five minutes of wait time.

[强调我的]

请注意:大多数 AWS SDK 同时自动应用这些建议，包括调整默认重试策略或添加自定义实现的选项 - 请参阅 AWS 中的错误重试和指数退避，获取有关如何在需要时自行实施的指导.

Please note: Most AWS SDKs meanwhile apply these suggestions automatically, including options to adjust the default retry policy or add a custom implementation even - see Error Retries and Exponential Backoff in AWS for guidance on how to implement it yourself, if need be.

各种大型 AWS 用户越来越多地遇到 AWS API 的最终一致性设计，他们自然需要更深入地研究并相应地解决它，例如，请参阅以下文章:

The eventually consistent design of the AWS API is increasingly encountered by various large scale AWS users, who naturally need to look deeper and work around it accordingly, see for example the following articles:

• 交易AWS EC2 API 中的最终一致性(作者:Martin Englund，Cloud Foundry)
• 处理不一致的 EC2 API(作者:Aaron Staley，PiCloud)

• Dealing with Eventual Consistency in the AWS EC2 API (by Martin Englund, Cloud Foundry)
• Dealing with the Inconsistent EC2 API (by Aaron Staley, PiCloud)

正如@datasage 已经评论，AWS API 显然一般需要被视为最终一致 - 这在第一次遇到时肯定是出乎意料的，但实际上并非如此事后看来对于大规模服务来说太令人惊讶了，即工程代表.操作权衡以解决 CAP 定理.

As already commented by @datasage, the AWS APIs apparently need to be generally treated as eventually consistent only - this is certainly unexpected when first encountered, but actually not too surprising for a large scale service in hindsight, i.e. an engineering resp. operational tradeoff to address the CAP theorem.

另请参阅我对 Alex Ciminian 问题的评论 为 AWS Spot 实施幂等性实例请求，他在其中讨论了关于类似一致性问题的测试结果:

See also my comment on Alex Ciminian's question Implementing idempotency for AWS Spot Instance Requests, where he discusses his test results regarding similar consistency issues:

有趣的问题 - [...] 我在Bamboo AWS 插件 的上下文并得出结论，AWS API需要被视为最终一致木板;例如，我什至遇到过收到资源的情况来自创建调用的 id，可以根据其 id 标记资源，但不能此后仍然描述它，因为它应该不存在(还).

Interesting issue - [...] I've encountered various similar API delays in the context of the Bamboo AWS Plugin and concluded that the AWS API needs to the treated as being eventually consistent only across the board; e.g., I've even encountered cases where I received a resource id from a create call, could tag the resource based on its id but not describe it thereafter still, because it supposedly doesn't exist (yet).

• 我在此描述的内容似乎表明，每个 API 操作实际上都完全由 AWS 独立操作(不仅跨外部可见的服务，甚至在像 EC2 这样的单个服务中)，因此可能会受到最终的影响一致性，必须进行相应处理.

有关上述案例的详细信息，您可能需要查看 AWS API 的频繁轮询会导致节流限制，我总结了我们的分析和方法，通过 适用于 Java 的 AWS 开发工具包 - 该解决方案几乎是理想的，但它目前似乎大大改善了情况.

For details on the mentioned cases you might want to look into Frequent polling of AWS API causes throttle limit, where I summarize our analysis and approach to improve the handling via the available but limited retry/backoff functionality within the AWS SDK for Java - the solution is all but ideal, but it seems to considerably improve things for the time being.

同样，重新设计的 AWS SDK for PHP 2 引入了专用的Waiter"的对象允许您轮询资源直到它处于所需状态以解决问题，请参阅快速入门了解详情:

On a similar note, the redesigned AWS SDK for PHP 2 introduced dedicated "Waiter" objects that allow you to poll a resource until it is in a desired state to address the problem, see section Waiters within the Quick Start for details:

SDK 提供的高级抽象之一是概念的服务员".服务员帮助最终更容易合作通过提供一种简单的方法来等待资源通过轮询资源进入特定状态.[...] 任何以waitUntil"开头的@method 标签将使用一个服务员.

One of the high-level abstractions provided by the SDK is the concept of "waiters". Waiters help make it easier to work with eventually consistent systems by providing an easy way to wait on a resource to enter into a particular state by polling the resource. [...] Any @method tag that starts with "waitUntil" will utilize a waiter.

$client->waitUntil('BucketExists', array('Bucket' => 'my-bucket'));

这篇关于确定性地创建和标记 EC2 实例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！