带有 JSON 字段的 Amazon Cloudwatch Logs Insights [英] Amazon Cloudwatch Logs Insights with JSON fields

问题描述

我正在尝试将 Logs Insights 与其中一个字段中包含 JSON 的数据一起使用，并解析 JSON 字段

I am trying to use Logs Insights with data containing JSON in one of the fields, and to parse the JSON fields

当我将数据与入门代码一起放入见解时，我的数据如下所示

My data looks like the following when I put it in insights with the starter code

fields @timestamp, @message
| sort @timestamp desc
| limit 25

如何轻松提取嵌套 JSON 中的 path 变量以对其执行聚合?通过查看一些文档，我认为 @message.path 会起作用，但似乎并非如此.有没有人在 Insights 中成功解释过 JSON 日志

How can I easily extract the path variable in my nested JSON to perform aggregations on it ? By looking at some documentation, I thought @message.path would work but it does not seem so. Has anyone successfully interpreted JSON logs in Insights

我的数据示例

#
@timestamp
@message
1
2018-12-19 23:42:52.000
I, [2018-12-19T23:42:52.629855 #23447] INFO -- : [2ce588f1-c27d-4a55-ac05-62a75b39e762] {"method":"GET","path":"/api/v1/professionals/ID","format":"json","controller":"API::V1::Public::ProfessionalsController","action":"show","status":200,"duration":285.27,"view":222.36,"time":"2018-12-19T23:42:52.344+00:00","params":{"include":"user,tags,promotions,company_sector,similar_professionals.tags,similar_professionals.user","format":"json","compress":false,"id":"ID"},"@timestamp":"2018-12-19T23:42:52.629Z","@version":"1","message":"[200] GET /api/v1/professionals/ID (API::V1::Public::ProfessionalsController#show)"}
@logStream  i-05d1d61ab853517a0
@message  I, [2018-12-19T23:42:52.629855 #23447] INFO -- : [2ce588f1-c27d-4a55-ac05-62a75b39e762] {"method":"GET","path":"/api/v1/professionals/ID","format":"json","controller":"API::V1::Public::ProfessionalsController","action":"show","status":200,"duration":285.27,"view":222.36,"time":"2018-12-19T23:42:52.344+00:00","params":{"include":"xxx","format":"json","compress":false,"id":"ID"},"@timestamp":"2018-12-19T23:42:52.629Z","@version":"1","message":"[200] GET /api/v1/professionals/ID (API::V1::Public::ProfessionalsController#show)"}
@timestamp  1545262972000
2
2018-12-19 23:42:16.000
I, [2018-12-19T23:42:16.723472 #851] INFO -- : [ea712503-eb86-4a6e-ab38-ddbcd6c2b4d0] {"method":"GET","path":"/api/v1/heartbeats/new","format":"json","controller":"API::V1::Public::HeartbeatsController","action":"new","status":201,"duration":9.97,"view":3.2,"time":"2018-12-19T23:42:16.712+00:00","params":{"format":"json","compress":false},"@timestamp":"2018-12-19T23:42:16.722Z","@version":"1","message":"[201] GET /api/v1/heartbeats/new (API::V1::Public::HeartbeatsController#new)"}

推荐答案

CloudWatch Insights Logs 自动发现以下日志类型的字段:

CloudWatch Insights Logs automatically discovers fields for the following log types:

CloudWatch Logs Insights 自动发现 Lambda 日志中的日志字段，但仅针对每个日志事件中的第一个嵌入的 JSON 片段(注意:重点是我的).如果 Lambda 日志事件包含多个 JSON 片段，您可以使用 parse 命令解析和提取日志字段.有关更多信息，请参阅 JSON 日志中的字段.

Lambda logs

CloudWatch Logs Insights automatically discovers log fields in Lambda logs, but only for the first embedded JSON fragment in each log event (Note: emphasis mine). If a Lambda log event contains multiple JSON fragments, you can parse and extract the log fields by using the parse command. For more information, see Fields in JSON Logs.

请参阅字段在 JSON 日志中.

来源: 支持日志和发现的字段

如果 @message 是 I, [2018-12-11T13:20:27] INFO -- : {method":GET"}

然后您可以像这样选择和过滤字段:

Then you can select and filter the fields like so:

fields @timestamp, @message, method
| filter method = "GET"
| sort @timestamp desc

它也适用于嵌套字段，即 params.format = "json" 或 results.0.firstName = "Paul".

It works with nested fields too, i.e. params.format = "json" or results.0.firstName = "Paul".

这篇关于带有 JSON 字段的 Amazon Cloudwatch Logs Insights的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！