在python中使用boto3查询cloudwatch日志中的不同值 [英] query cloudwatch logs for distinct values using boto3 in python

问题描述

我有一个lambda函数，可将指标写入 Cloudwatch 。同时，它编写度量标准，并在日志组中生成一些日志。

I have a lambda function that writes metrics to Cloudwatch. While, it writes metrics, It generates some logs in a log-group.

INFO:: username: [e-mail] ClinicID: 7667 nodename: MacBook-Pro-2.local

INFO:: username: [e-mail] ClinicID: 7669 nodename: MacBook-Pro-3.local

INFO:: username: [e-mail] ClinicID: 7668 nodename: MacBook-Pro-4.local

INFO:: username: [e-mail] ClinicID: 7667 nodename: MacBook-Pro-5.local

INFO:: username: [e-mail] ClinicID: 7667 nodename: MacBook-Pro-2.local

我需要一种有效的方法来获取节点名不同值 c>给定的 ClinicId 。例如，我为 ClinicId 传递了 7667 ，我期望

I need an efficient way to get distinct values of nodename for a given ClinicId. For example, I pass in 7667 for ClinicId and I expect

['MacBook-Pro-2.local', 'MacBook-Pro-5.local']

这是我尝试过的：

 query = "fields @timestamp, @message | parse @message \"username: * ClinicID: * nodename: *\" as username, ClinicID, nodename | filter ClinicID = "+ clinic_id

 start_query_response = client.start_query(
        logGroupName=log_group,
        startTime=int(time.mktime((Util.utcnow() - timedelta(hours=hours)).timetuple())),
        endTime=int(time.mktime(Util.utcnow().timetuple())),
        queryString=query,
    )

我考虑过在Python中迭代 start_query_response ，但是我不喜欢这个想法。由于我要查看的是超过 7天的日志，因此我需要一种有效的方法，而不是必须迭代过去的 7天的每条日志以获得给定的 ClinicID 。

I considered iterating start_query_response in Python but I do not like that idea. Since it is logs for over 7 days that I will be looking at, I need an efficient way instead of having to iterate each log from past 7 days for the given ClinicID.

推荐答案

您可以将表达式通过管道传递到 stat 命令并计算发生次数

You can pipe you expression to the stat command and count occurrences of each nodename.

将此添加到查询的末尾：

Add this to the end of your query:

| stats count(*) by nodename

的结果将是：

{
  'results': [
    [
      {
        'field': 'nodename',
        'value': 'MacBook-Pro-2.local\n'
      },
      {
        'field': 'count(*)',
        'value': '2'
      }
    ],
    [
      {
        'field': 'nodename',
        'value': 'MacBook-Pro-5.local\n'
      },
      {
        'field': 'count(*)',
        'value': '1'
      }
    ]
  ]
}

有关各种命令的更多详细信息，请参见此处： https://docs.aws.amazon.com/AmazonCloudWatch/latest /logs/CWL_QuerySyntax.html

See here for more details on various commands: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html

这篇关于在python中使用boto3查询cloudwatch日志中的不同值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！