在AWS ECS上的Docker映像中运行CloudWatch Agent失败 [英] Running CloudWatch Agent in my Docker image on AWS ECS fails
问题描述
了解这个问题，因为我是Docker和AWS的新手.
Understand for this question that I'm relatively new to Docker and AWS.
目标是创建一个使用基本Laravel应用程序运行Apache和PHP的AWS AWS ECS实例.我想运行一个CloudWatch代理以将所有日志发送到CloudWatch(Apache的访问和错误日志，PHP的错误日志和Laravel日志).
The goal is to create a single AWS ECS instance that runs Apache and PHP with a basic Laravel application. I want to run a CloudWatch agent to send all logs to CloudWatch (access and error logs for Apache, error log for PHP and the Laravel logs).
我知道这可能不是最佳实践"(欢迎小费)，但是我目前的理念是首先使其工作，然后使其美观":-)
I know this probably isn't 'best practice' (tips are welcome), but my philosophy for now is 'first make it work, then make it pretty' :-)
我的Dockerfile:
My Dockerfile:
FROM amazonlinux:latest
# Update/Install
RUN yum update -y && \
# Install PHP & epel
amazon-linux-extras install -y php7.3 epel && \
# Install
yum install -y \
# Install apache
httpd \
# Install tools for CloudWatch
collectd statsd \
# Install supervisor
supervisor \
# Install cloudwatch agent
https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm && \
# Clean install data
yum clean metadata && \
yum -y clean all && \
rm -rf /var/cache/yum
# PHP Settings
RUN sed -i \
'/<Directory \"\/var\/www\/html\">/,/<\/Directory>/ s/AllowOverride None/AllowOverride All/' \
/etc/httpd/conf/httpd.conf
# Remove default html folder
RUN rm -rf /var/www/html
# Configure supervisor
COPY supervisord.conf /etc/supervisord.conf
# Configure CloudWatch agent
COPY amazon-cloudwatch-agent.json /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json
# Add source to image
ADD . /var/www/aws
RUN chown -R apache:apache /var/www && ln -s /var/www/aws/public /var/www/html
# Expose port 80
EXPOSE 80
# Start supervisor
CMD ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]
我的supervisor.conf
My supervisor.conf
[supervisord]
nodaemon=true
[program:httpd]
priority=1
command=/usr/sbin/apachectl -D FOREGROUND
autorestart=true
username=apache
[program:php]
priority=2
command=/usr/sbin/php-fpm
autorestart=true
[program:cloudformation]
priority=10
command=/opt/aws/amazon-cloudwatch-agent/bin/start-amazon-cloudwatch-agent
autorestart=true
我的cloudwatch配置:
My cloudwatch config:
{
"agent": {
"metrics_collection_interval": 60,
"region": "eu-europe-1",
"logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log",
"debug": false,
"run_as_user": "cwagent"
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/php-fpm/www-error.log",
"log_group_name": "aws-docker",
"log_stream_name": "{instance_id}"
}
]
}
}
}
}
基本上，它可以作为运行Laravel应用程序的docker镜像正常工作.我现在遇到的唯一问题是CloudWatch代理.它在ECS中的容器上启动，但无法运行并显示以下消息:
Basically this works fine as a docker image that runs the Laravel application. The only problem I'm having now is the CloudWatch agent. It starts on the container in ECS, but fails to run with the following message:
2020/02/22 13:39:28 I! 2020/02/22 13:39:28 E! ec2metadata is not available
I! Detected the instance is OnPrem
2020/02/22 13:39:28 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json ...
Valid Json input schema.
I! Detecting runasuser...
2020/02/22 13:39:28 E! Credentials path is not set while runasuser is not root
2020/02/22 13:39:28 I! AmazonCloudWatchAgent Version 1.237768.0.
2020/02/22 13:39:28 Configuration validation first phase failed. Agent version: 1.237768.0. Verify the JSON input is only using features supported by this version.
2020/02/22 13:39:28 I! Return exit error: exit code=1
2020/02/22 13:39:28 E! Cannot translate JSON config into TOML, ERROR is exit status 1
首先，我不明白为什么显示消息ec2metadata is not available.该容器在ECS上运行，因此应该可用(据我了解).
First of all I don't understand why the message ec2metadata is not available is showing. The container runs on ECS, so it should be available (from what I understand).
第二条消息Configuration validation first phase failed. Agent version: 1.237768.0. Verify the JSON input is only using features supported by this version..据我所知，配置应该是可以的.
Second the message Configuration validation first phase failed. Agent version: 1.237768.0. Verify the JSON input is only using features supported by this version.. As far as I can tell the config should be oke.
我认为我的职责也很重要，因为该容器确实将日志发送到CloudWatch.
I think my role also is oke, because the container does send logs to CloudWatch.
我在做什么错了?
推荐答案
这是我如何将其预算为提交状态.我们在Docker容器中使用了非常轻量级的Ubuntu子集，该子集没有CloudWatch Agent似乎专为systemctl或System V初始化而设计.您可以直接运行start-amazon-cloudwatch-agent(如提到的此答案)，但是它并不那么顺利.
Here's how I bludgeoned it into submission. We're using a very lightweight Ubuntu subset in our Docker containers that doesn't have systemctl or System V init, which the CloudWatch Agent seems designed for. You can run start-amazon-cloudwatch-agent directly (as this answer mentioned), but it's not quite as smooth.
代理希望积极管理/opt/aws/amazon-cloudwatch-agent树的所有者/组. (a)如果您在配置中使用"runasuser": "cwagent"以root形式启动它，则它拒绝使用AWS config/cred(上述的Credentials path is not set)，并且无法使用. (b)如果使用"runasuser": "cwagent"作为cwagent来启动它，那么它会抱怨它无法更改一堆东西的所有权(即使它已经被cwagent拥有)，并且无法使用. (c)但是，如果您以cwagent开头并且不在配置中包含"runasuser"来启动它，它会发出抱怨，但它确实可以启动并完成其工作.
The agent wants to aggressively manage the owner/group of the /opt/aws/amazon-cloudwatch-agent tree. (a) If you start it as root with "runasuser": "cwagent" in the config, then it refuses to use the AWS config/creds (Credentials path is not set mentioned above), and it bails. (b) If you start it as cwagent with "runasuser": "cwagent", then it complains that it can't change the ownership of a bunch of stuff (even though it's already owned by cwagent), and it bails. (c) But if you start it as cwagent and don't include a "runasuser" in the config, it complains, but it does start up and do its thing.
Verify the JSON input is only using features supported by this version.消息是代理在遇到麻烦时所说的话.似乎与配置无关(很好).
The Verify the JSON input is only using features supported by this version. message is what the agent says whenever it runs into trouble. It seems to have nothing to do with the config (which is fine).
这是我的详细信息:
# Dockerfile
ADD ./files /tmp
# [...]
RUN curl -o /tmp/amazon-cloudwatch-agent.deb 'https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb' \
&& dpkg -i /tmp/amazon-cloudwatch-agent.deb \
&& rm -f /tmp/amazon-cloudwatch-agent.deb \
&& usermod -a -G www-data cwagent \
&& chgrp -R www-data /var/log/nginx \
&& chmod g+s /var/log/nginx \
&& chown -R cwagent:cwagent /opt/aws/amazon-cloudwatch-agent \
&& install -o cwagent -g cwagent -m 700 -d /home/cwagent \
&& install -o cwagent -g cwagent -m 700 -d /home/cwagent/.aws \
&& install -o cwagent -g cwagent -m 600 /tmp/cloudwatch.config /home/cwagent/.aws/config \
&& install -o cwagent -g cwagent -m 600 /tmp/cloudwatch.credentials /home/cwagent/.aws/credentials \
&& install -o cwagent -g cwagent -m 755 /tmp/cloudwatch-agent.json /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/default \
&& mv /tmp/99_cloudwatch.init /etc/my_init.d/
# /etc/my_init.d/99_cloudwatch.init
#!/bin/sh
su cwagent -c "nohup /opt/aws/amazon-cloudwatch-agent/bin/start-amazon-cloudwatch-agent >/tmp/cwagent.out 2>&1 &"
exit 0
// cloudwatch-agent.json
{
"agent": {
"region": "us-east-1",
"debug": false
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/nginx/access.log",
"log_group_name": "our-app",
"log_stream_name": "nginx-access",
"timestamp_format": "[%d/%b/%Y:%H:%M:%S %z]"
},
{
"file_path": "/var/log/nginx/error.log",
"log_group_name": "our-app",
"log_stream_name": "nginx-error",
"timezone": "UTC",
"timestamp_format": "%Y/%m/%d %H:%M:%S"
}
]
}
}
}
}
这篇关于在AWS ECS上的Docker映像中运行CloudWatch Agent失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
