Kibana Tophits按字段而不是所有字段进行变换分组 [英] Kibana Tophits on transform group by a field not all field

问题描述

所以在这种情况下，我需要在转换时使用热门歌曲 我想显示基于的数据

So I have this case where I need to use top hits on transformation I want to show data based on

email      col2      col3     col4  col5    Time
a.com         a        a        a    a     11:00 
a.com         a        a        a    a     11:01 
a.com         a        b        a    a     11:02

我想删除重复的电子邮件，仅根据最新时间显示.我正在使用转换并基于最大时间对其进行聚合.对于小组，我选择我需要的每个领域.它返回如下数据: 我将索引转换为groupby:email，col2，col3，col4并通过max(Time)对其进行汇总

I want to remove the duplicate email, and only show it based on the latest time. I'm using transform and aggregate it based on max time. and for the group by I choose every field I needed. It returns data such as : I transform the index and make it groupby : email, col2,col3,col4 and aggregate it by max(Time)

email      col2      col3     col4  col5    Time
a.com         a        a        a    a     11:01 
a.com         a        b        a    a     11:02

我只希望它显示数据 我的目标

I only want it to show data my target

email      col2      col3     col4  col5    Time
a.com         a        b        a    a     11:02

如何仅基于groupby电子邮件而不是每个字段进行转换?由于我需要所有字段，但我认为添加所有as组不是正确的，但是聚合或groupby只有两种方法

How can I make the transform based on groupby email only instead every field? Since I need all the field but I don't think add all of the as group by is right but there are only 2 methods either aggregation or groupby

我的转换定义: 不是我所需要的

my transformation definition : It didn't come as what i need

{
  "id": "transform_baru",
  "source": {
    "index": [
      "email-profile-nov-bug*"
    ],
    "query": {
      "match_all": {}
    }
  },
  "dest": {
    "index": "transform_baru"
  },
  "pivot": {
    "group_by": {
      "Email.keyword": {
        "terms": {
          "field": "Email.keyword"
        }
      },
      "fa.keyword": {
        "terms": {
          "field": "fa.keyword"
        }
      },
      "ever.keyword": {
        "terms": {
          "field": "ever.keyword"
        }
      },
      "bln.keyword": {
        "terms": {
          "field": "bln.keyword"
        }
      },
      "domain.keyword": {
        "terms": {
          "field": "domain.keyword"
        }
      },
      "Email_age_category.keyword": {
        "terms": {
          "field": "Email_age_category.keyword"
        }
      },
      "Status_Category.keyword": {
        "terms": {
          "field": "Status_Category.keyword"
        }
      },
      "Vintage_cat.keyword": {
        "terms": {
          "field": "Vintage_cat.keyword"
        }
      }
    },
    "aggregations": {
      "extract_date.max": {
        "max": {
          "field": "extract_date"
        }
      }
    }
  },
  "settings": {},
  "version": "7.8.0",
  "create_time": 1607832008196
}

推荐答案

使用此

Problem solved by using this Tophit workaround But I wasn't able to use it. Here is how to use it:

1. 仅选择所需的分组.就我而言，我只会添加电子邮件
2. 编辑json配置，并使用Latest_doc脚本添加聚合
3. 将"@timestamp"字段更改为您的时间字段.
4. 因此，从技术上讲，您仅将电子邮件用作分组依据，而将last_doc用作聚合方式
5. 在预览中，可能仅显示您选择作为分组依据的字段，但是当创建转换索引时，该字段的其余部分将显示在latest.doc脚本下.所以不用担心，只需创建转换

1. Choose only the groupby you need. In my case I would just add Email
2. Edit json config and add the aggregation with the latest_doc script
3. Change '@timestamp' field as your time field.
4. So technically, you only use email as groupby, and latest_doc as aggregation
5. On the preview, it might be show only the field that you choose as group by, but when the transform index created rest of the field will be show under latest.doc script. So don't worry and just create the transform

我希望这会帮助一些有弹性的新手使用此替代方法.

I hope this will help some elastic newbie to use this workaround.

感谢所有尝试帮助我的人.干杯

Thank you for everyone who try to help me. Cheers

这篇关于Kibana Tophits按字段而不是所有字段进行变换分组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！