实现幂等为AWS现货实例请求 [英] Implementing idempotency for AWS Spot Instance Requests

问题描述

我使用了Java AWS SDK，使EC2实例当场请求。相对于在需求情况下，所述<一href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-RequestSpotInstances.html">API现货请求没有什么类似<一个href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Run_Instance_Idempotency.html"><$c$c>ClientToken因此不支持幂等开箱

最直接的方法我能想到的做，这是将 LaunchGroup 属性唯一的UUID;当我检查，我称之为<一个href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeSpotInstanceRequests.html"><$c$c>DescribeSpotInstanceRequests看看我是否已经有相同的启动组的请求。

要我吃惊的是，似乎有一个延迟之前的描述调用返回之前发送的现货请求。我写了一个JUnit测试这一点，似乎在为了它是一致的，我将不得不设置至少两个电话之间的60秒超时（要求现货实例和描述现场情况的要求）。我需要有10秒的粒度，因为我的请求，可以在此区间反复得到了应用，在任何情况下，失败的 - 即事休息后，我发送了请求，但之前，我可以读取结果我从亚马逊回来。在这种情况下，我不希望有要求重复，我只是想看看，它得到了注册，并继续前进。

  @Test
公共无效testRunSpotInstances（）抛出异常{

    activity.execute（执行）;

    超时（超时）;

    //不应该做任何事情
    activity.execute（执行）;

    超时（超时）;

    DescribeSpotInstanceRequestsResult结果= client.describeSpotInstanceRequests（
            新DescribeSpotInstanceRequestsRequest（）。withFilters（新过滤器（）
                .withName（发射组）withValues​​（BUSINESS_KEY）））。

    assertThat（result.getSpotInstanceRequests（））hasSize（1）。

    超时（超时）;
}
 

测试工作的每一个，如果超时设置为60秒的时间;为40-50s间歇工作。任何低于这一失败每次。

有没有人设法解决这个延迟？正在实施幂等现货请求可能只使用AWS API和没有状态保存在客户端应用程序？

解决方案

  

在这种情况下，我不希望有要求重复，我只是想看看，它得到了注册，并继续前进。

如果你有200回，那么它的注册。它可能显示不出来马上，但它的注册，您可以在您的流继续前进。

  

正在实施仅使用AWS API幂等现货请求可能和没有状态保存在客户端应用程序？

我不这么认为。我有一个亚马逊的电子病历同一类问题。我解决它的方法是有分量谁的工作就是观察团。当我做了一个EMR集群的要求，我回来了群集ID，然后我通过关闭一些观察者。观察者将调用我的其他组件时群集更改状态。没有被承认EMR马上是一个有效的情况下，并没有像对待一个例外。

我不知道，如果这是适合你的。也许你可以试试维护<一href="http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/ec2/model/SpotInstanceRequest.html#getSpotInstanceRequestId%28%29"相对=nofollow> SpotInstanceRequestId 。就我而言，我只保留在内存中，但你可以让他们持续的地方如果需要的话。

I'm using the Java AWS SDK to make EC2 spot instance requests. As opposed to on demand instances, the API for spot requests does not have anything similar to ClientToken and thus does not support idempotency out of the box.

The most straightforward way I could think of to do this was to set the LaunchGroup property to a unique UUID; when I check for that I call DescribeSpotInstanceRequests and see if I already have a request with the same launch group.

To my surprise, it seems that there's a delay before the describe call returns the spot requests sent before. I wrote a JUnit test for this and it seems that in order for it to be consistent I would have to set a timeout of at least 60s between the two calls (request spot instance and describe spot instance requests). I need to have a granularity of 10s, because my requests can get repeated by the application at this interval, in case of any failure - i.e. something breaks after I sent the request but before I could read the result I got back from Amazon. In that case I don't want to have the request repeated, I just want to see that it got registered and move on.

@Test
public void testRunSpotInstances() throws Exception {

    activity.execute(execution);

    timeout(TIMEOUT);

    // shouldn't do anything
    activity.execute(execution);

    timeout(TIMEOUT);

    DescribeSpotInstanceRequestsResult result = client.describeSpotInstanceRequests(
            new DescribeSpotInstanceRequestsRequest().withFilters(new Filter()
                .withName("launch-group").withValues(BUSINESS_KEY)));

    assertThat(result.getSpotInstanceRequests()).hasSize(1);

    timeout(TIMEOUT);
}

The test works every time if TIMEOUT is set to 60s; for 40-50s it works intermittently. Anything below this fails every time.

Has anyone managed to work around this delay? Is implementing idempotency for spot requests possible using just the AWS API and not having state saved in the client application?

解决方案

In that case I don't want to have the request repeated, I just want to see that it got registered and move on.

If you got a 200 back, then it's registered. It may not show up right away, but it's registered and you can move on in your flow.

Is implementing idempotency for spot requests possible using just the AWS API and not having state saved in the client application?

I don't believe so. I have the same sort of issue with Amazon's EMR. The way that I work around it is to have a component who's job it is to observe clusters. When I make a request for an EMR cluster, I get back a cluster id, which I then pass off to some observer. The observer will then call my other components when that cluster changes state. Not being acknowledged by EMR right away is a valid case and is not treated like an exception.

I have no idea if that's appropriate for you. Perhaps you could try maintaining the SpotInstanceRequestId. In my case, I only keep them in memory, but you could keep them somewhere persistent if need be.

这篇关于实现幂等为AWS现货实例请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！