使用命令行工具将JSON数组拆分为多个文件 [英] Split a JSON array into multiple files using command line tools

问题描述

假设我们有一个长度为5的JSON数组，我们想使用linux命令行工具将该数组拆分为多个长度为2的数组，并将分组的项目保存到不同的文件中.

Suppose we have a JSON array of length 5 and we want to split the array into multiple arrays of length 2 and save the grouped items into different files, using linux command line tools.

我通过使用jq和split工具进行了尝试(我对可以从bash脚本执行的任何方法都感到满意):

I tried it by using the jq and split tools (I am happy with any approach that can be executed from a bash script):

$ echo '[{"key1":"value1"},{"key2":"value2"},{"key3":"value3"},{"key4":"value4"},{"key5":"value5"}]' | jq -c -M '.[]' | split -l 2 -d -a 3 - meta_
$ tail -n +1 meta_*
==> meta_000 <==
{"key1":"value1"}
{"key2":"value2"}

==> meta_001 <==
{"key3":"value3"}
{"key4":"value4"}

==> meta_002 <==
{"key5":"value5"}

上一条命令将项目正确保存到文件中，但是我们需要将它们转换为有效的JSON数组格式.我对--filter选项感到厌倦:

The previous command saves the items into the files correctly, but we need to convert them into a valid JSON array format. I tired with --filter option:

$ echo '[{"key1":"value1"},{"key2":"value2"},{"key3":"value3"},{"key4":"value4"},{"key5":"value5"}]' | jq -c -M '.[]' | split -l 2 -d -a 3 - meta2_ --filter='jq --slurp -c -M'
[{"key1":"value1"},{"key2":"value2"}]
[{"key3":"value3"},{"key4":"value4"}]
[{"key5":"value5"}]
$ tail -n +1 meta2_*
tail: cannot open 'meta2_*' for reading: No such file or directory

但是，它会在屏幕上显示输出，但结果不会持续存在.我尝试转发输出，但出现错误:

However, it displays the output on the screen but the results aren't persisted. I tried forwarding the output but I get an error:

echo '[{"key1":"value1"},{"key2":"value2"},{"key3":"value3"},{"key4":"value4"},{"key5":"value5"}]' | jq -c -M '.[]' | split -l 2 -d -a 3 - meta2_ --filter='jq --slurp -c -M > $FILE'
...
split: with FILE=meta2_000, exit 2 from command: jq --slurp -c -M > $FILE

有任何提示或更好的方法吗?

Any hints or better approaches?

我尝试用双引号@andlrc建议:

I tried with double quotes @andlrc suggested:

$ echo '[{"key1":"value1"},{"key2":"value2"},{"key3":"value3"},{"key4":"value4"},{"key5":"value5"}]' | jq -c -M '.[]' | split -l 2 -d -a 3 - meta2_ --filter="jq --slurp -c -M > $FILE"
bash: -c: line 0: syntax error near unexpected token `newline'
bash: -c: line 0: `jq --slurp -c -M > '
split: with FILE=meta2_000, exit 1 from command: jq --slurp -c -M >
$ cat meta_000 | jq --slurp -c -M
[{"key1":"value1"},{"key2":"value2"}]

推荐答案

在jq过滤器中构建数组，然后按行分割为文件会更容易.无需其他过滤.

It'll be easier to build out the arrays in the jq filter, then split to files per line. No additional filtering necessary.

range(0; length; 2) as $i | .[$i:$i+2]

产生:

[{"key1":"value1"},{"key2":"value2"}]
[{"key3":"value3"},{"key4":"value4"}]
[{"key5":"value5"}]

所以把它们放在一起.

$ jq -cM --argjson sublen '2' 'range(0; length; $sublen) as $i | .[$i:$i+$sublen]' \
    input.json | split -l 1 -da 3 - meta2_

这篇关于使用命令行工具将JSON数组拆分为多个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！