将JSON拆分为多个文件 [英] Split JSON into multiple files

问题描述

我有从mongodb导出的json文件，如下所示:

I have json file exported from mongodb which looks like:

{"_id":"99919","city":"THORNE BAY"}
{"_id":"99921","city":"CRAIG"}
{"_id":"99922","city":"HYDABURG"}
{"_id":"99923","city":"HYDER"}

大约有30000行，我想将每一行拆分成它自己的.json文件. (我正在尝试将我的数据转移到沙发基群集上)

there are about 30000 lines, I want to split each line into it's own .json file. (I'm trying to transfer my data onto couchbase cluster)

我尝试这样做:

cat cities.json | jq -c -M '.' | \
while read line; do echo $line > .chunks/cities_$(date +%s%N).json; done

但是我发现它似乎减少了行的负载，而当我期望30000个奇数时，运行此命令的输出仅给了我50个奇数文件！

but I found that it seems to drop loads of line and the output of running this command only gave me 50 odd files when I was expecting 30000 odd!!

是否有一种逻辑方法可以使它不使用任何适合的东西来丢弃任何数据?

Is there a logical way to make this not drop any data using anything that would suite?

推荐答案

假定您不关心确切的文件名，如果要将输入拆分为多个文件，只需使用

Assuming you don't care about the exact filenames, if you want to split input into multiple files, just use split.

jq -c . < cities.json | split -l 1 --additional-suffix=.json - .chunks/cities_

这篇关于将JSON拆分为多个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！