如何通过组合输入csv的两个字段/列在logstash中生成@timestamp [英] How to generate @timestamp in logstash by combining two fields / columns of input csv

问题描述

我们有来自外部来源的数据，如下csv文件：

  orderid，OrderDate，BusinessMinute，价格
 31874,01-01-2013,00：06,2,17.9 
  

数据在一列中具有 date ，在另一列中具有 time - 我需要通过组合这两个来生成时间戳

我使用 csv过滤器使用logstash中的以下配置从文件中读取上述数据 - 正在生成自己的时间戳记：

  input {
 file {
 path => /root/data/import/Order.csv
 start_position => begin
} 
} 
 filter {
 csv {
 columns => [orderid，OrderDate，BusinessMinute，Quantity，Price] 
 separator => ，
} 
} 
输出{
 elasticsearch {
 action => index
 host => localhost
 index => demo
 workers => 1 
} 
} 
  

如何使

解决方案 / div>

使用 mutate过滤器将OrderDate和BusinessMinute字段合并为一个（临时）字段，然后使用日期过滤器，如果字段成功，请将其删除。

 过滤器{
 mutate {
 add_field => {
timestamp=> ％{OrderDate}％{BusinessMinute}
} 
} 
 date {
 match => [timestamp，...] 
 remove_field => [timestamp] 
} 
} 
  

We have data that is coming from external sources as below in csv file:

orderid,OrderDate,BusinessMinute,Quantity,Price
31874,01-01-2013,00:06,2,17.9

The data has date in one column and time in another column - I need to generate a time-stamp by combining those two columns together.

I am using csv filter to read the above data from file using below configuration in logstash - which is generating its own timestamp:

input {  
  file {
      path => "/root/data/import/Order.csv"
      start_position => "beginning"
  }
}
filter {
    csv {
        columns => ["orderid","OrderDate","BusinessMinute","Quantity","Price"]
        separator => ","
    } 
}
output {  
    elasticsearch {
        action => "index"
        host => "localhost"
        index => "demo"
        workers => 1
    }
}

How to make the combination of OrderDate + Business Minute as the @timestamp?

解决方案

Use a mutate filter to combine the OrderDate and BusinessMinute fields into a single (temporary) field, then use the date filter and have it delete the field if it's successful.

filter {
  mutate {
    add_field => {
      "timestamp" => "%{OrderDate} %{BusinessMinute}"
    }
  }
  date {
    match => ["timestamp", "..."]
    remove_field => ["timestamp"]
  }
}

这篇关于如何通过组合输入csv的两个字段/列在logstash中生成@timestamp的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！