AWS Cloudwatch日志-是否可以从中导出现有日志数据？ [英] AWS Cloudwatch Log - Is it possible to export existing log data from it?

问题描述

我已经设法通过使用AWS CloudWatch日志代理将应用程序日志推送到AWS Cloudwatch。但是，CloudWatch Web控制台似乎没有提供允许您从中下载/导出日志数据的按钮。

I have managed to push my application logs to AWS Cloudwatch by using the AWS CloudWatch log agent. But the CloudWatch web console does not seem to provide a button to allow you to download/export the log data from it.

有什么想法可以实现这个目标吗？

Any idea how I can achieve this goal?

推荐答案

最新的AWS CLI具有CloudWatch Logs cli，可让您将日志下载为JSON，文本文件或AWS CLI支持的任何其他输出。

The latest AWS CLI has a CloudWatch Logs cli, that allows you to download the logs as JSON, text file or any other output supported by AWS CLI.

例如，获取从组 A 中的流 a 到文本文件的前10,000个日志条目，运行：

For example to get the first 10,000 log entries from the stream a in group A to a text file, run:

aws logs get-log-events \
   --log-group-name A --log-stream-name a \
   --output text > a.log

该命令当前限制为每个请求最多10,000条记录，如果您有您还需要使用-next-token 参数实现自己的页面步进机制。我希望将来CLI还将允许在单个命令中进行完全转储。

The command is currently limited to a maximum of 10,000 records per request, and if you have more you need to implement your own page stepping mechanism using the --next-token parameter. I expect that in the future the CLI will also allow full dump in a single command.

一个小的Bash脚本，从指定的时间开始，列出特定组中所有流的事件：

Here's a small Bash script to list events from all streams in a specific group, since a specified time:

#!/bin/bash
function dumpstreams() {
  aws $AWSARGS logs describe-log-streams \
    --order-by LastEventTime --log-group-name $LOGGROUP \
    --output text | while read -a st; do 
      [ "${st[4]}" -lt "$starttime" ] && continue
      stname="${st[1]}"
      echo ${stname##*:}
    done | while read stream; do
      aws $AWSARGS logs get-log-events \
        --start-from-head --start-time $starttime \
        --log-group-name $LOGGROUP --log-stream-name $stream --output text
    done
}

AWSARGS="--profile myprofile --region us-east-1"
LOGGROUP="some-log-group"
TAIL=
starttime=$(date --date "-1 week" +%s)000
nexttime=$(date +%s)000
dumpstreams
if [ -n "$TAIL" ]; then
  while true; do
    starttime=$nexttime
    nexttime=$(date +%s)000
    sleep 1
    dumpstreams
  done
fi

最后一部分，如果您设置 TAIL 将继续获取日志事件并会报告新事件的出现（预期会有所延迟）。

That last part, if you set TAIL will continue to fetch log events and will report newer events as they come in (with some expected delay).

这篇关于AWS Cloudwatch日志-是否可以从中导出现有日志数据？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！