在每个json文档之前添加标题行 [英] Add a header line before each json document

问题描述

我有一个带有1000个json对象的json文件. 有什么办法可以在每个json文档之前添加标题行?有没有最简单的方法?

I have a json file with 1000 json object. is there any way to add a header line before each json document ? Is there any easiest way ?

示例:我有1000个这样的对象

Example : I have 1000 object like this

{"id":58,"first_name":"Louis","last_name":"Jordan","email":"[e-mail]","gender":"Male","Latitude":"-15.93444","Longitude":"-50.14028"}

我想为每个json对象添加如下所示的索引标头，以便可以在Elasticsearch Bulk api中使用

i want to add index header like below for every json object so that i can use in Elasticsearch Bulk api

{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "unique_id" } }
{"id":58,"first_name":"Louis","last_name":"Jordan","email":"[e-mail]","gender":"Male","Latitude":"-15.93444","Longitude":"-50.14028"}

推荐答案

如果您愿意利用Logstash，则无需修改文件，只需简单地逐行读取文件并使用elasticsearch利用Bulk API的输出.

If you are willing to leverage Logstash, you don't need to modify your file and can simply read it line by line and stream it to ES using the elasticsearch output which leverages the Bulk API.

将以下Logstash配置存储在名为es.conf的文件中(确保文件path和ES hosts与您的设置匹配):

Store the following Logstash configuration in a file named es.conf (make sure the file path and ES hosts match your settings):

input {
  file {
    path => "/path/to/your/json"
    sincedb_path => "/dev/null"
    start_position => "beginning"
    codec => "json"
  }
}
filter {
  mutate {
    remove_fields => ["@version", "@timestamp"]
  }
}
output {
  elasticsearch {
    hosts => "localhost:9200"
    index => "test"
    document_type => "type1"
    document_id => "%{id}"
  }
}

然后，您需要安装logstash ，您将能够运行以下命令以将JSON文件加载到ES服务器:

Then, you need to install logstash and you'll be able to run the following command in order to load your JSON files to your ES server:

bin/logstash -f es.conf

这篇关于在每个json文档之前添加标题行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！