在ElasticSearch中以微秒格式保存日期 [英] Saving date in microsecond format in ElasticSearch

问题描述

我正在尝试使用带有Logstash的jdbc输入插件将MySQL数据库中的事件集保存到弹性搜索中.数据库中的事件记录包含微秒格式的日期字段.实际上，数据库中有几微秒之间的记录.

I am trying to save set of events from MySQL database into elastic search using jdbc input plugin with Logstash. The event record in database contains date fields which are in microseconds format. Practically, there are records in database between set of microseconds.

在导入数据时，Elasticsearch会将微秒日期格式截断为毫秒格式.如何保存微秒格式的数据?elasticsearch文档说，他们遵循JODA时间API来存储日期格式，该格式不支持微秒，并且在时间戳的末尾添加 Z 会对其进行截断.

While importing data, Elasticsearch is truncating the microseconds date format into millisecond format. How could I save the data in microsecond format? The elasticsearch documentation says they follow the JODA time API to store date formats, which is not supporting the microseconds and truncating by adding a Z at the end of the timestamp.

截断后的示例时间戳记: 2018-05-02T08:13:29.268Z

Sample timestamp after truncation : 2018-05-02T08:13:29.268Z

数据库中的原始时间戳记: 2018-05-02T08:13:29.268482

Original timestamp in database : 2018-05-02T08:13:29.268482

推荐答案

Z 不是截断的结果，而是GMT时区.

The Z is not a result of the truncation but the GMT timezone.

如果您在映射中指定了正确的日期格式，ES也支持微秒.

ES supports microseconds, too, provided you've specified the correct date format in your mapping.

如果映射中的日期字段是这样指定的:

If the date field in your mapping is specified like this:

    "date": {
      "type": "date",
      "format": "yyyy-MM-dd'T'HH:mm:ss.SSSSSS"
    }

然后，您可以按照数据库中的精确毫秒级精度来索引日期

Then you can index your dates with the exact microsecond precision as you have in your database

更新

这里是一个完整的重新创作，向您展示它的工作原理:

Here is a full re-creation that shows you that it works:

PUT myindex
{
  "mappings": {
    "doc": {
      "properties": {
        "date": {
          "type": "date",
          "format": "yyyy-MM-dd'T'HH:mm:ss.SSSSSS"
        }
      }
    }
  }
}

PUT myindex/doc/1
{
  "date": "2018-05-02T08:13:29.268482"
}

这篇关于在ElasticSearch中以微秒格式保存日期的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！