AWS Cloudwatch 日志 - 是否可以从中导出现有的日志数据? [英] AWS Cloudwatch Log - Is it possible to export existing log data from it?

问题描述

我已设法使用 AWS CloudWatch 日志代理将我的应用程序日志推送到 AWS Cloudwatch.但是 CloudWatch Web 控制台似乎没有提供按钮来允许您从中下载/导出日志数据.

I have managed to push my application logs to AWS Cloudwatch by using the AWS CloudWatch log agent. But the CloudWatch web console does not seem to provide a button to allow you to download/export the log data from it.

知道如何实现这个目标吗?

Any idea how I can achieve this goal?

推荐答案

最新的 AWS CLI 有一个 CloudWatch Logs cli，允许您将日志下载为 JSON、文本文件或 AWS CLI 支持的任何其他输出.

The latest AWS CLI has a CloudWatch Logs cli, that allows you to download the logs as JSON, text file or any other output supported by AWS CLI.

>

例如，要将 A 组中的流 a 中的前 1MB 最多 10,000 个日志条目获取到文本文件，请运行:

For example to get the first 1MB up to 10,000 log entries from the stream a in group A to a text file, run:

aws logs get-log-events 
   --log-group-name A --log-stream-name a 
   --output text > a.log

该命令当前限制为最大 1MB 的响应大小(每个请求最多 10,000 条记录)，如果您有更多记录，则需要使用 --next-token 实现自己的页面步进机制 参数.我希望将来 CLI 也将允许在单个命令中进行完整转储.

The command is currently limited to a response size of maximum 1MB (up to 10,000 records per request), and if you have more you need to implement your own page stepping mechanism using the --next-token parameter. I expect that in the future the CLI will also allow full dump in a single command.

这是一个小的 Bash 脚本，用于列出特定组中所有流中的事件，从指定时间开始:

Here's a small Bash script to list events from all streams in a specific group, since a specified time:

#!/bin/bash
function dumpstreams() {
  aws $AWSARGS logs describe-log-streams 
    --order-by LastEventTime --log-group-name $LOGGROUP 
    --output text | while read -a st; do 
      [ "${st[4]}" -lt "$starttime" ] && continue
      stname="${st[1]}"
      echo ${stname##*:}
    done | while read stream; do
      aws $AWSARGS logs get-log-events 
        --start-from-head --start-time $starttime 
        --log-group-name $LOGGROUP --log-stream-name $stream --output text
    done
}

AWSARGS="--profile myprofile --region us-east-1"
LOGGROUP="some-log-group"
TAIL=
starttime=$(date --date "-1 week" +%s)000
nexttime=$(date +%s)000
dumpstreams
if [ -n "$TAIL" ]; then
  while true; do
    starttime=$nexttime
    nexttime=$(date +%s)000
    sleep 1
    dumpstreams
  done
fi

最后一部分，如果您设置 TAIL 将继续获取日志事件，并会在更新的事件进来时报告(有一些预期的延迟).

That last part, if you set TAIL will continue to fetch log events and will report newer events as they come in (with some expected delay).

这篇关于AWS Cloudwatch 日志 - 是否可以从中导出现有的日志数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！