组合第一滤波器的输出作为第二滤波器的输入 [英] combine output of a a first filter as input of a second filter

问题描述

我们有一个带有两个带标签字段的条目的elasticsearch实例.

We have an elasticsearch instance with entries with two tagged fields.

• sessionid
• 消息

在第一个过滤器中，我找到了 message 包含特定子字符串的所有条目.每个条目都包含一个 sessionid

In a first filter, I find all entries where the message contains a certain substring. Each of those entries contains a sessionid,

在第二个过滤器中，我想找到所有消息，其中 sessionid 与第一个过滤器返回的一个sessionid匹配.此过滤器应第二次遍历所有条目.

In a second filter, I want to find all messages, where the sessionid matches one of the sessionids returned by the first filter. This filter should go through all entries a second time.

示例，在下面的日志中(sessionid；消息)

Example, in the log below (sessionid;message)

1234;miss 1
2456;miss 2
1234;match

在消息部分中过滤字符串"match"时，我将得到组合查询的输出:

When filtering for the string "match" in the message part, I would get as output of the combined query:

1234;miss 1
1234;match

我们正在使用KQL.

背景:在多线程环境中，我们希望有一种简单的方法来跟踪消息中带有错误字符串的完整流.

Background: We want an easy way to follow complete flows with an error-string in a message, in a multithreaded environment.

推荐答案

我了解您为什么要一口气做到这一点，但是在ElasticSearch中是不可能的.您无法重新访问"已由其他查询排除的文档-搜索 match 将取消所有 miss 的资格.

---

很遗憾，您将日志消息与ID组合在一起，但是您可以尝试以下操作:

I understand why you'd want to do that in one go but it's not possible in ElasticSearch. You cannot "revisit" documents which you've already ruled out by a different query -- searching for match would disqualify all misss.

---

It's unfortunate you have the log message combined with the ID but you can try this:

1. 找到所有匹配的 match (双关语意味)–我假设您确实有一个 keyword 字段

1. Find all that match match (pun intended) -- I'm assuming you do have a keyword field available

GET your_index/_search
{
  "query": {
    "regexp": {
      "separated_msg.keyword": ".*\\;match.*"
    }
  }
}

1. 对匹配进行后期处理并提取会话ID

1. Post-process the hits and extract the session IDs

运行会话ID匹配:

GET your_index/_search
{
  "query": {
    "regexp": {
      "separated_msg.keyword": "1234;.*"
    }
  }
}

或使用 bool应该在多个ID上:

GET your_index/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "regexp": {
            "separated_msg.keyword": "1234;.*"
          }
        },
        {
          "regexp": {
            "separated_msg.keyword": "4567;.*"
          }
        }
      ]
    }
  }
}

这篇关于组合第一滤波器的输出作为第二滤波器的输入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！