如何配置数据文件中的一个自动调节比例（多实例）弹性魔豆（Tomcat）的应用（AWS）？ [英] How do I configure data files in a Autoscaled (multi-instance) Elastic Beanstalk (Tomcat) Application (AWS)?
问题描述
我目前运行的是部署到Tomcat Java应用程序一个弹性魔豆实例。我部署使用Web界面的应用程序，但应用程序使用的数据文件（Lucene索引）的，我复制到底层的EC2实例在web.xml中引用的ssh-荷兰国际集团，以EC2和S3我斗获取的数据文件。
I currently have one Elastic Beanstalk instance running a Java application that is deployed to Tomcat. I deploy the application using the Web interface but the application uses a data file (Lucene index) referenced in the web.xml that I copy to the underlying EC2 instance by ssh-ing to EC2 and getting the data file from my S3 bucket.
到目前为止好。
但是，如果我改变了我的EB到autoscaleable environent从而需要它自动创建新的实例，那么这些EC2实例不会有数据文件，我该如何处理这个问题。
But if I changed my EB to a autoscaleable environent so that it automatically creates new instances as required then these EC2 instances will not have the data file, how do I deal with this.
- 我可以preconfigure与数据文件每个EC2实例之前，它实际上是用??
- 我可以有一个共同的FS每个服务器可以参考，（数据文件是只读）？
*更新*
我觉得香港专业教育学院制定了原则上的答案。我是从我的本地计算机上传我的应用程序，然后从亚马逊再添加大量数据文件。我需要做的是建立自己的战争在我的数据处理EC2实例，数据文件添加到战争的地方，然后把这场战争到S3，然后当我创建我的EB我需要加载从S3斗了战争。
I think Ive worked out the answer in principle. I was uploading my application from my local machine then adding the large datafiles later from Amazon. What I need to do is build my war on my dataprocessing EC2 instance, add the datafile to the war somewhere, then put this war onto S3, then when I create my EB I need to load the WAR from the S3 bucket.
所以，只需要在哪里工作数据文件应该在战争中，以及如何通过创建的Maven构建过程。
So just need to work out where data-file should go in War and how to create via Maven build process.
*更新2 *
其实它不是清楚，数据文件应该在WAR文件毕竟，我看不到的地方把他们的应用程序期望它们是真正的文件，因此，如果包含在战争和战争没有展开/ unjarred（我不知道什么EB）不适用不反正工作。
Actually its not clear that the data files should go in the WAR file after all, I cannot see where to put them and the application expects them to be real files so if contained within WAR and the WAR was not expanded/unjarred (I dont know what EB) does the application would not work anyway.
*更新3 *
我当然可以把在S3中的数据（实际上它可能将在那里开始），所以我想如果服务器initlization我能得到S3上的数据，并把它的地方，然后用它？ 指导吧。
I could certainly put the data in S3 (in fact it will probably will be there to start with) So I wonder if on server initlization I could get the s3 data and put it somewhere and then use it ? Guidance please.
*更新4 *
因此，使用S3的想法我几乎有工作，在servlet的init（）方法中，我得到了COM pressed文件，将其保存到当前工作目录（在/ usr /共享/ tomcat7 /），然后uncom preSS它。麻烦是COM pressed文件是2.7GB，uncom pressed文件夹，并解决其来为5GB，采用EB未成年实例提供8GB容量2GB的使用。所以，我有6GB这是足够的空间uncom pressed文件，但不保存COM pressed文件，然后uncom preSS，因为我需要2.7 GB + 5 GB的uncom $ P $中pssing过程。
So using the s3 idea I nearly have it working, within the servlet init() method I get the compressed file, save it to the current working directory (/usr/share/tomcat7/) and then uncompress it. Trouble is the compressed file is 2.7GB,uncompressed folder it resolves to is 5GB , the minor instance used by EB offers 8GB of which 2GB is used. So I have 6GB which is enough space for the uncompressed file, but not to save the compressed file and then uncompress it because I need 2.7 GB + 5 GB during the uncompressing process.
予加载的玉米pressed版本至S3因为原始数据不是一个单一的文件，而是一个夹的文件完全将是难以管理作为文件的列表。我不能改变根目录的大小EB，我可以尝试更改为一个强大的实例，但是这将unnessarily是怎样的磁盘空间设有例如使用欧洲央行更昂贵，并不清楚。任何想法？
I loaded the compressed version to S3 because the original data is not a single file but a folder full of files it would be difficult to manage as a list of files. I cannot change the size of root dir in EB, I could try changing to a powerful instance but that will unnessarily be more expensive and not clear what disk space is provided with instance used by ECB. Any ideas ?
这些是我添加到我的Maven的回购
These were the dependencies I added to my maven repo
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.8.2</version>
</dependency>
<dependency>
<groupId>org.rauschig</groupId>
<artifactId>jarchivelib</artifactId>
<version>0.6.0</version>
</dependency>
这是code
And this is the code
@Override
public void init()
{
try
{
log.severe("Retrieving Indexes from S3");
AWSCredentials credentials = new BasicAWSCredentials("***********", "***********");
AmazonS3Client ac = new AmazonS3Client(credentials);
log.severe("datalength-testfile:"+ac.getObjectMetadata("widget","test.txt").getContentLength());
File testFile = new File("test.txt");
ac.getObject(new GetObjectRequest("widget", "test.txt"), testFile);
log.severe("datalength-testfile:retrieved");
log.severe("datalength-largefile:"+ac.getObjectMetadata("widget","indexes.tar.gz").getContentLength());
File largeFile = new File("indexes.tar.gz");
ac.getObject(new GetObjectRequest("widget", "indexes.tar.gz"), largeFile);
log.severe("datalength-largefile:retrieved");
log.severe("Retrieved Indexes from S3");
log.severe("Unzipping Indexes");
File indexDirFile = new File(indexDir).getAbsoluteFile();
indexDirFile.mkdirs();
Archiver archiver = ArchiverFactory.createArchiver(largeFile);
archiver.extract(largeFile, indexDirFile);
log.severe("Unzipped Indexes");
}
catch(Exception e)
{
log.log(Level.SEVERE, e.getMessage(), e );
}
}
*更新5 *
在意识到微EC2实例只能提供0.6GB不6GB我需要升级到一个更大的机器不管怎样，这提供了两个磁盘，这样我就可以COM pressed文件复制到一个磁盘，然后uncom preSS到根磁盘成功，所以准备好了。
Having realized the micro EC2 instance only provide 0.6GB not 6GB i needed to update to a larger machine anyway and that provided two disks so I could copy compressed file to one disk and then uncompress to root disk successfully, so ready to go.
*更新6 *
EB所以在自动定EB配置它启动其他EC2实例相信一日一的时候，其实它只是准备超载不尊重init（）方法。而我，如果它开始换新的时候真正忙碌的负载均衡器将开始喂请求这些情况下，他们已经准备好导致失败的请求之前怀疑。
EB does not respect init() method so in autoscaled EB configuration it starts up other EC2 instances believing the 1st one to be overloaded when in fact it is just getting ready. And I suspect if it starts new ones when genuinely busy the load balancer will start feeding requests to these instances before they are ready causing failed requests.
*更新7 *
试图把指标直接进入WEB-INF / classes和引用该位置在web.xml中。这适用于本地测试Tomcat的部署，但遗憾的是未能在EB因为抱怨 如此看来EB犯规respoect的init（）。因此，而不是试图从S3索引的init（）方法中我只是直接把索引放到WEB-INF / classes目录下的战争文件，并在我的web.xml中有指向的参数中。虽然他们没有真正的类，这并不令人Tomcat之类的问题，我已经测试针对部署对本地Tomcat的安装没有问题。
Tried putting indexes directly into WEB-INF/classes and referring to that location in web.xml. This works on a local test Tomcat deployment but unfortunately fails in EB because complains So it seems EB doesnt respoect init(). So instead of trying to get the indexes from S3 within the init() method I just put the indexes directly into the War file under WEB-INF/classes and point the paramter in my web.xml to there. Although they are not actually classes this does not cause a problem for Tomcat and I have tested against deployment against a local tomcat installation without problem.
不幸的是已经上传的这个更大的战争文件containign索引到S3试图从S3位置部署到EB失败，：
Unfortunately having uploaded this larger war file containign the indexes to S3 attempt to deploy it to EB from S3 location fails with:
无法启动环境：源代码包是空或超过允许的最大尺寸：524288000.
Could not launch environment: Source bundle is empty or exceeds maximum allowed size: 524288000.
为什么要有亚马逊强加这arbitary限制？
Why have Amazon imposed this arbitary limit ?
*更新8 *
所以，可能的选项是
- ebextensions
- 泊坞部署
- 与EB使用创建自定义的亚马逊形象
3的选择似乎很哈克，并非所有热衷于说，还是非常热衷于他人真的。
3rd option seems very hacky, not all keen on that, or very keen on the others really.
*更新** 9
我得到了它与最终ebextensions工作，并不太高，我在这里记录的情况下非常有用
I got it working with ebextensions in the end, wasnt too bad, I document here in case useful
如果使用maven创建在src / main / resources文件夹ebextensions 以下内容添加到pom.xml中（SAO是ebextensions去在最后战争的正确的地方）
If using maven create folder ebextensions in src/main/resources Add the following to pom.xml (sao that ebextensions goes in the right place in final war)
<plugin>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<webResources>
<resource>
<directory>src/main/ebextensions</directory>
<targetPath>.ebextensions</targetPath>
<filtering>true</filtering>
</resource>
</webResources>
</configuration>
</plugin>
在创建文件夹ebextensions config文件（我叫我的copyindex.cfg）和我有这个信息
Create .config file in ebextensions folder ( I called mine copyindex.cfg) and mine had this information
commands:
01_install_cli:
command: wget https://s3.amazonaws.com/aws-cli/awscli-bundle.zip; unzip awscli-bundle.zip; ./awscli-bundle/install -b ~/bin/aws
02_get_index:
command:
aws s3 cp --region eu-west-1 s3://jthink/release_index.tar.gz /dev/shm/release_index.tar.gz;
cd /usr/share/tomcat7; tar -xvf /dev/shm/release_index.tar.gz
转到IAM控制台（ https://console.aws.amazon.com/ IAM /家？#home ）和附加作用的政策高级用户到弹性魔豆角色的用户
Go to IAM console ( https://console.aws.amazon.com/iam/home?#home) and attach role policy Power User to Elastic Beanstalk Role user
将应用程序部署
推荐答案
有实现这一目标的多种方式。你不需要ssh到实例和复制文件。
There are multiple ways of achieving this. You do not need to ssh to the instance and copy your files.
我会建议在您的更新3的办法。
I would recommend the approach in your "Update 3".
您可以配置弹性魔豆环境中部署应用程序之前执行命令。您可以使用ebextensions做到这一点。阅读命令的文档<一href="http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#customize-containers-format-commands"相对=nofollow>这里。
You can configure your Elastic Beanstalk environment to execute commands before deploying the application. You can do this using ebextensions. Read the documentation on commands here.
基本上你创建你的应用程序源代码的名称 .ebextensions 文件夹。此文件夹可以包含的.config 扩展一个或多个文件。这些文件在他们的名字字典顺序进行处理。您可以通过使用ebextensions执行shell命令。例如，您可以执行以下操作：
Essentially you create a folder with the name .ebextensions in your app source. This folder can contain one or more files with .config extension. These files are processed in lexicographical order of their name. You can execute shell commands by using ebextensions. For example you can do the following:
commands:
02_download_index:
command: aws s3 cp s3://mybucket/test.txt test2.txt
您需要在您的EC2实例安装 AWS CLI 第一。这可以再次以类似于以上的命令完成。可<关于如何使用捆绑的安装程序安装AWS CLI指令href="http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html#install-bundle-other-os"相对=nofollow>此处。您可以运行多个命令。在配置文件中的命令将在字典顺序执行的，所以你可以说出你的命令，像 01_install_awcli ， 02_download_index 等。
You will need to install aws cli on your EC2 instances first. This can again be done with a command similar to above. Instructions on how to install AWS CLI using the bundled installer are available here. You can run more than one command. The commands within a config file will be executed in lexicographical order so you can name your commands like 01_install_awcli, 02_download_index etc.
如果您计划使用AWS CLI的EC2实例现在，你还需要凭据。如果您使用的是IAM实例配置文件（最有可能你是，如果不是读到它的这里）。你可以给您的实例配置文件的权限使用IAM访问您的S3对象。这样，你的实例将具有与之相关联的一个IAM实例概要和将能够从S3下载文件。或者你也可以直接拿到ACCESS_KEY_ID和SECRET_KEY使用环境属性，如图这里。
Now if you plan to use AWS CLI on the EC2 instance, you will also need credentials. If you are using an IAM Instance Profile (most likely you are, if not read about it here). You can give your instance profile permissions to access your S3 object using IAM. That way your instances will have an IAM instance profile associated with it and will be able to download the file from S3. Alternatively you can also directly get the ACCESS_KEY_ID and SECRET_KEY using environment properties as shown here.
这是拿出所有新的实例应该在ebextensions执行的命令。因此，你的情况可能是pconfigured与你想要的软件$ P $。
All new instances that come up should execute the commands in your ebextensions. Thus your instances can be preconfigured with the software that you want.
这篇关于如何配置数据文件中的一个自动调节比例（多实例）弹性魔豆（Tomcat）的应用（AWS）？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
