工人和QUOT;测功机＆QUOT;在AWS弹性魔豆 [英] Worker "dyno" in AWS Elastic Beanstalk

问题描述

亚马逊网络服务目前在它们的弹性魔豆工人层。但是，它仍然迷惑我们谁来自工人赛道的日子。

Amazon Web Service has now a worker tiers in their Elastic Beanstalk. But, it nevertheless confuse us who come from the days of Worker dyno.

作为对比，在Heroku上，可以配置两个dynos（像处理器？），每个网页和工人。该网站将适用于任何请求，并会在15秒通常超时。因此，如果你有持续超过一个请求，你的要求会简单地超时虽然没有终止本身。在这种情况下，你应该使用的工人和Web测功机应该访问每分钟（也许）端点数次，检查是否有被带回给用户的任何结果。为了让无论是工人还是网络赛道，你需要的仅仅是滑动滑块，你是好去。有时，你可能需要一个Procfile。但并没有什么花哨，什么真的很难，或混淆了。

As a comparison, in Heroku, one can configure two dynos (something like processor?) each for web and worker. The web will work for any request, and will timeout normally at 15 secs. Thus, if you have a request that last more than that, your request will simply timed-out although not terminated per se. In that case, you should use worker and your web dyno should visit the endpoint several times per minutes (maybe) to check if there is any result to be brought back to the user. To make either worker or web dyno, what you need is just slide the slider and you are good to go. Sometimes, you may need a Procfile. But there is nothing fancy, or something really difficult, or confusing about it.

在AWS EBS（弹性魔豆），自1998年1一天你打EB初始化，你会被询问是否是标准或工人。当你打的标准，似乎也没有办法让它的工人也是如此。

In AWS EBS (Elastic Beanstalk), since day 1 you hit eb init, you will be asked whether it is a Standard or Worker. When you hit Standard, it seems there is no way to make it as worker as well.

在我们的情况下，工人和标准的Web位于下一个应用程序。那么，我们如何能够使用EBS实例都为工人和标准。我们的工作人员正在使用sidekiq，和Redis的。请指向任何指导或帮助我们在这件事情。

In our situation, the worker and standard web is located under one application. So, how could we use an EBS instance both for worker and standard. Our worker is using sidekiq, and redis. Please, point to any guidance or help us in this matter.

推荐答案

AWS弹性魔豆有两种类型的环境 - 网络层和工作者层

AWS Elastic Beanstalk has two types of Environments - Web tier and Worker tier.

Web层的环境都是针对Web应用程序 - HTTP / HTTPS请求处理。你得到负载平衡器后面的一个或多个EC2实例。您可以让其他资源，如按您的要求的数据库。你可以选择你想例如平台红宝石，Python和Java的，Node.js的，PHP，泊坞。

Web tier environments are meant for web applications - http/https request processing. You get one or more EC2 instances behind a load balancer. You can get other resources like database per your requirement. You can choose the platform you wish e.g. Ruby, Python, Java, Node.js, PHP, Docker.

工环境都是为了异步消息处理。当你创建一个工作环境，你没有一个负载平衡器。你所有的EC2实例处于自动缩放组。所有这些实例正在运行一个守护进程轮询一个SQS队列中的消息。当消息从SQS队列中的守护拉，守护进程发送一个本地主机上的HTTP POST请求：80。您可以配置端口，但重要的是，守护职位的消息在本地主机上的HTTP请求。你的员工的应用程序实际上是接收POST请求并处理信息的Web应用程序。之后，消息被成功处理工人守护预计，在本地主机上运行的Web应用程序返回一个HTTP 200 OK响应。接着后台程序会删除SQS队列消息。你可以写任何就像标准的Web服务器应用程序平台，你的工人的应用程序 - 红宝石，Python和Java的，Node.js的，PHP，泊坞

Worker environments are meant for asynchronous message processing. When you create a worker environment you do not have a load balancer. All your EC2 instances are in an autoscaling group. All these instances are running a daemon which is polling a single SQS queue for messages. When a message is pulled by the daemon from the SQS queue, the daemon sends a HTTP Post request on localhost:80. You can configure the port but the important thing is that the daemon posts the message as an HTTP request on localhost. Your worker application is actually a web application that receives the post request and processes the message. After the message is successfully processed the worker daemon expects that your web application running on localhost returns a HTTP 200 OK response. The daemon then deletes the message from SQS queue. You can write your worker application for any platform just like standard web server applications - Ruby, Python, Java, Node.js, PHP, Docker.

根据我对你的用例的理解，我会建议建立两个弹性魔豆的环境 - 一个标准，一个工人的环境。标准的Web服务器接收HTTP请求，并同步对其进行处理。这样的环境，会使相关的数据的SQS队列。第二个环境是一名工人，在这种环境下运行的轮询消息这SQS队列守护进程。你的第二个环境是一个Web应用程序，不公开到互联网。工人守护帖子消息的HTTP请求到你的工人的环境。因此，你可以处理长期运行的工作负载异步使用第二个工人的环境。

Based on my understanding of your usecase I would recommend creating two Elastic Beanstalk environments - one Standard and one Worker environment. The Standard web server receives HTTP requests and processes them synchronously. This environment puts the relevant data in an SQS queue. The second environment is a worker and the daemon running on this environment polls this SQS queue for messages. Your second environment is a web application that is NOT open to the internet. The worker daemon posts the messages as HTTP requests to your worker environment. Thus you can process long running workloads asynchronously using this second worker environment.

随着工人的环境中，你可以使用自己的队列或弹性魔豆可以生成一个队列为您服务。您可以根据您的要求配置类似消息的可见性超时参数，HTTP连接，也可以使用默认值。

With worker environments you can use your own queues or Elastic Beanstalk can generate a queue for you. You can configure parameters like message visibility timeout, http connections based on your requirements or you can use the defaults.

下面是一些链接，可能对您有用：

Below are some links that may be useful for you:

<一个href="http://aws.amazon.com/blogs/aws/background-task-handling-for-aws-elastic-beanstalk/">http://aws.amazon.com/blogs/aws/background-task-handling-for-aws-elastic-beanstalk/

<一个href="http://blogs.aws.amazon.com/application-management/post/Tx1Y8QSQRL1KQZC/Elastic-Beanstalk-Video-Tutorial-Worker-Tier">http://blogs.aws.amazon.com/application-management/post/Tx1Y8QSQRL1KQZC/Elastic-Beanstalk-Video-Tutorial-Worker-Tier

http://stackoverflow.com/a/23942498/161628

这是否符合你的要求？请让我知道，如果你有进一步的问题。

Does this meet your requirements? Please let me know if you have further questions.

更新

您需要上传您的源$ C ​​$ C在两个地方 - 一旦为职工环境，一次用于Web服务器环境。如果有人白手起家那么他们可能有两个单独的code碱基。但我认为你的情况，我认为它应该是完全正常有两种环境之间共享的单一code基地。假设您的网络请求到达'/注册，然后在你的应用程序中的寄存器（）方法可以发布消息到SQS队列，并与HTTP请求进行。现在，你的工人的环境将轮询SQS队列，并发布消息通过HTTP本地主机上的URL/ async_register这将调用一个方法async_register（）在您的应用程序并执行异步处理。这两种方法都可以住在同一源$ C ​​$ C包可以由两个工人和Web服务器环境共享。采取的工人和Web服务器的code路径将是不同的，这样的web服务器环境将调用寄存器（）和工作环境将调用async_register（）方法。

You need to upload your source code at two places - once for the worker environment and once for the web server environment. If someone was starting from scratch then they might have two separate code bases. But I think in your case I think it should be perfectly fine to have a single code base shared between the two environments. Suppose your web request arrives at '/register', then the register() method in your application can post messages to an SQS queue and be done with the HTTP request. Now your worker environment will poll the SQS queue and post messages over HTTP on localhost to the URL '/async_register' which will invoke a method async_register() in your application and do the asynchronous processing. These two methods can live in the same source code bundle which can be shared by both the worker and web server environment. The code path taken by worker and web server will be different so that web server environments will invoke register() and worker environments will invoke the async_register() method.

另一个需要注意的是，通过在本地主机上的工人守护程序发送的HTTP请求将包含一个HTTP头 - 用户代理：AWS-sqsd / 1.1。阅读更多<一href="http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html">here.因此，在你的web应用程序，你可以有一个监听器发布请求的/注册，并根据该标题是否是present与否，你在内部调用寄存器（）或async_register（）方法。

Another caveat is that HTTP requests sent by the worker daemon on localhost will contain an HTTP header - "User-Agent": "aws-sqsd/1.1". Read more here. So in your web application you can have a single listener to post requests on "/register" and depending on whether this header is present or not you invoke register() or async_register() methods internally.

另外，我认为，如果你想在两种环境之间共享code基可以上传只在一个地方的code基地。你的环境在逻辑​​上划分成应用程序。所以，你可以有一个单一的应用程序。您上传您的源$ C ​​$ C使用CreateApplicationVersionAPI调用该应用程序。假设你上传的应用程序版本标签V1。现在，您可以创建一个工作环境，并在相同的应用程序的Web服务器环境。当您创建一个环境，你需要提供一个版本部署到您的环境。在这种情况下，您可以部署V1到这两种环境。所以，你将共享相同的源$ C ​​$ C两个环境。当你有一个新版本V2。你上传这个版本，然后对两个环境的更新变化自己的版本为V2。

Also I think if you want to share the code base between the two environments you can upload the code base at only one place. Your environments are logically grouped into applications. So you can have a single application. You upload your source code to this application using the "CreateApplicationVersion" API call. Suppose you upload an application version with label 'v1'. You can now create a worker environment and a web server environment under the same application. When you create an environment you need to provide a version to deploy to your enviroment. In this case you can deploy v1 to both environments. So you will be sharing the same source code for both environments. When you have a new version "v2". You upload this version and then perform an update on both environments changing their version to "v2".

源$ C ​​$ c中的相同版本可以被部署到这两种环境。他们将在不同的EC2实例运行，因为一个环境是专门为应对Web请求，一个环境，专门用于应对异步Web请求（工人）。

The same version of the source code can be deployed to both environments. They will be running on different EC2 instances because one environment is dedicated for responding to web requests and one environment is dedicated for responding to asynchronous web requests (worker).

这篇关于工人和QUOT;测功机＆QUOT;在AWS弹性魔豆的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！