在 Elasticsearch 上查询每种类型的最新文档 [英] Query the latest document of each type on Elasticsearch

问题描述

我正在尝试在 Elasticsearch 上运行看似简单的查询，但似乎无法得到我正在寻找的结果.

I'm trying to run what started to look like a simple query on Elasticsearch, but I just can't seem to get the result I'm looking for.

这是我正在尝试做的一个简短示例:

Here's a brief example of what I'm trying to do:

我有一个新闻数据库.每条新闻都包含一个来源、一个标题、一个时间戳和一个用户.

I have a database of news. Each piece of news contains a source, a headline, a timestamp and a user.

我希望获得给定用户的每个可用来源的最后一个(基于时间戳的)标题.

I want the get the last (timestamp based) headline for each available source for a given user.

#!/bin/bash

export ELASTICSEARCH_ENDPOINT="http://localhost:9200"

# Create indexes

curl -XPUT "$ELASTICSEARCH_ENDPOINT/news" -d '{
    "mappings": {
        "news": {
            "properties": {
                "source": { "type": "string", "index": "not_analyzed" },
                "headline": { "type": "object" },
                "timestamp": { "type": "date", "format": "date_hour_minute_second_millis" },
                "user": { "type": "string", "index": "not_analyzed" }
            }
        }
    }
}'

# Index documents
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_bulk?refresh=true" -d '
{"index":{"_index":"news","_type":"news"}}
{"user": "John", "source": "CNN", "headline": "Great news", "timestamp": "2015-07-28T00:07:29.000"}
{"index":{"_index":"news","_type":"news"}}
{"user": "John", "source": "CNN", "headline": "More great news", "timestamp": "2015-07-28T00:08:23.000"}
{"index":{"_index":"news","_type":"news"}}
{"user": "John", "source": "ESPN", "headline": "Sports news", "timestamp": "2015-07-28T00:09:32.000"}
{"index":{"_index":"news","_type":"news"}}
{"user": "John", "source": "ESPN", "headline": "More sports news", "timestamp": "2015-07-28T00:10:35.000"}
{"index":{"_index":"news","_type":"news"}}
{"user": "Mary", "source": "Yahoo", "headline": "More news", "timestamp": "2015-07-28T00:11:54.000"}
{"index":{"_index":"news","_type":"news"}}
{"user": "Mary", "source": "Yahoo", "headline": "Crazy news", "timestamp": "2015-07-28T00:12:31.000"}
'

例如，我如何从约翰那里获得最后的 CNN 和最后的 ESPN 头条新闻?

So how do I get the last CNN and last ESPN headlines from John for example?

我一直在研究多搜索 API，但这意味着我需要事先了解所有来源(在本例中为 CNN 和 ESPN).

I've been looking into the multi search API, but this would mean that I would need to know all the sources beforehand (in this case CNN and ESPN).

推荐答案

首先，请注意我必须将 headline 字段的映射更改为 string，如在您的示例文档中，标题是 strings 而不是 objects.

First, please note that I had to change your mapping for the headline field to string, as in your sample documents headlines are strings and not objects.

因此，像下面这样的查询将检索您期望的内容:

So, a query like the following one would retrieve what you expect:

curl -XPOST "$ELASTICSEARCH_ENDPOINT/news/_search" -d '{
  "size": 0,
  "query": {
    "filtered": {
      "filter": {
        "term": {
          "user": "John"           <--- filter for user=John
        }
      }
    }
  },
  "aggs": {
    "sources": {
      "terms": {
        "field": "source"          <--- aggregate by source
      },
      "aggs": {
        "latest": {
          "top_hits": {
            "size": 1,             <--- only take the first...
            "_source": [           <--- only the date and headline
               "headline",
               "timestamp"
            ],
            "sort": {
              "timestamp": "desc"  <--- ...and only the latest hit
            }
          }
        }
      }
    }
  }
}'

这将产生如下结果:

{
  ...
  "aggregations" : {
    "sources" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ {
        "key" : "CNN",
        "doc_count" : 2,
        "latest" : {
          "hits" : {
            "total" : 2,
            "max_score" : null,
            "hits" : [ {
              "_index" : "news",
              "_type" : "news",
              "_id" : "AU7Sh3VDGDddn2ZNuDVl",
              "_score" : null,
              "_source":{
                  "headline": "More great news", 
                  "timestamp": "2015-07-28T00:08:23.000"
              },
              "sort" : [ 1438042103000 ]
            } ]
          }
        }
      }, {
        "key" : "ESPN",
        "doc_count" : 2,
        "latest" : {
          "hits" : {
            "total" : 2,
            "max_score" : null,
            "hits" : [ {
              "_index" : "news",
              "_type" : "news",
              "_id" : "AU7Sh3VDGDddn2ZNuDVn",
              "_score" : null,
              "_source":{
                   "headline": "More sports news", 
                   "timestamp": "2015-07-28T00:10:35.000"
              },
              "sort" : [ 1438042235000 ]
            } ]
          }
        }
      } ]
    }
  }
}

这篇关于在 Elasticsearch 上查询每种类型的最新文档的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！