如何在Python中使用Boto3查询CloudWatch日志 [英] How to query cloudwatch logs using boto3 in python

问题描述

我有一个lambda函数，可将指标写入Cloudwatch.在编写指标的同时，它会在日志组中生成一些日志.

I have a lambda function that writes metrics to Cloudwatch. While, it writes metrics, It generates some logs in a log-group.

INFO:: username: [e-mail] ClinicID: 7667 nodename: MacBook-Pro-2.local

INFO:: username: [e-mail] ClinicID: 7667 nodename: MacBook-Pro-2.local

INFO:: username: [e-mail] ClinicID: 7668 nodename: MacBook-Pro-2.local

INFO:: username: [e-mail] ClinicID: 7667 nodename: MacBook-Pro-2.local

我想查询过去 x 小时内的AWS日志，其中x可能基于任何参数在12到24小时之间.

I would like to query AWS logs in past x hours where x could be anywhere between 12 to 24 hours, based on any of the params.

例如:

1. 最近5个小时查询Cloudwatch日志，其中 ClinicID = 7667

或

1. 最近5个小时查询Cloudwatch日志，其中 ClinicID = 7667 和 username='[e-mail]'

或

1. 最近5个小时查询Cloudwatch日志，其中 username='[e-mail]'

我正在Python中使用 boto3 .我可以为此提供指导吗?

I am using boto3 in Python. Can I have a direction on this please?

推荐答案

您可以使用CloudWatch Logs Insights获得所需的内容.

You can get what you want using CloudWatch Logs Insights.

您将使用 start_query 和 get_query_results API:

You would use start_query and get_query_results APIs: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html

要开始查询，您将使用(对于您问题中的用例2，1和3相似):

To start a query you would use (for use case 2 from your question, 1 and 3 are similar):

import boto3
from datetime import datetime, timedelta
import time

client = boto3.client('logs')

query = "fields @timestamp, @message | parse @message \"username: * ClinicID: * nodename: *\" as username, ClinicID, nodename | filter ClinicID = 7667 and username='[e-mail]'"  

log_group = '/aws/lambda/NAME_OF_YOUR_LAMBDA_FUNCTION'

start_query_response = client.start_query(
    logGroupName=log_group,
    startTime=int((datetime.today() - timedelta(hours=5)).timestamp()),
    endTime=int(datetime.now().timestamp()),
    queryString=query,
)

query_id = start_query_response['queryId']

response = None

while response == None or response['status'] == 'Running':
    print('Waiting for query to complete ...')
    time.sleep(1)
    response = client.get_query_results(
        queryId=query_id
    )

响应将包含这种格式的数据(加上一些元数据):

Response will contain your data in this format (plus some metadata):

{
  'results': [
    [
      {
        'field': '@timestamp',
        'value': '2019-12-09 17:07:24.428'
      },
      {
        'field': '@message',
        'value': 'username: [e-mail] ClinicID: 7667 nodename: MacBook-Pro-2.local\n'
      },
      {
        'field': 'username',
        'value': '[e-mail]'
      },
      {
        'field': 'ClinicID',
        'value': '7667'
      },
      {
        'field': 'nodename',
        'value': 'MacBook-Pro-2.local\n'
      }
    ]
  ]
}

这篇关于如何在Python中使用Boto3查询CloudWatch日志的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！