将JSON文件对象拆分为多个文件 [英] Split JSON File Objects Into Multiple Files

问题描述

我使用以下格式的JSON文件包含了太多的数据对象:

I have a file with too many data objects in JSON of the following form:

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {},
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              -37.880859375,
              78.81903553711727
            ],
            [
              -42.01171875,
              78.31385955743478
            ],
            [
              -37.6171875,
              78.06198918665974
            ],
            [
              -37.880859375,
              78.81903553711727
            ]
          ]
        ]
      }
    },
    {
      "type": "Feature",
      "properties": {},
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              -37.6171875,
              78.07107600956168
            ],
            [
              -35.48583984375,
              78.42019327591201
            ],
            [
              -37.880859375,
              78.81903553711727
            ],
            [
              -37.6171875,
              78.07107600956168
            ]
          ]
        ]
      }
    }
  ]
}

我想分割大文件，以便每个要素对象都有自己的文件，其中包含其类型对象和features(coordinates)对象.因此，从本质上讲，我正在尝试获取其中的许多内容:

I would like to split the large file such that each features object would have its own file containing a its type object and features(coordinates) object. So essentially, I am trying to get many of these:

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {},
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              -37.6171875,
              78.07107600956168
            ],
            [
              -35.48583984375,
              78.42019327591201
            ],
            [
              -37.880859375,
              78.81903553711727
            ],
            [
              -37.6171875,
              78.07107600956168
            ]
          ]
        ]
      }
    }
  ]
}

推荐答案

这里的解决方案只需要调用jq和awk，假设输入位于文件(input.json)中，并且第N个组件应写入以N = 1开头的文件/tmp/file$N.json:

Here's a solution requiring just one invocation of jq and one of awk, assuming the input is in a file (input.json) and that the N-th component should be written to a file /tmp/file$N.json beginning with N=1:

jq -c '.features = (.features[] | [.]) ' input.json |
  awk '{ print > "/tmp/file" NR ".json"}'

awk的替代方法是split -l 1.

如果希望每个输出文件都被漂亮地打印"，然后使用bash之类的外壳，则可以(以对jq的n次额外调用为代价)编写:

If you want each of the output files to be "pretty-printed", then using a shell such as bash, you could (at the cost of n additional calls to jq) write:

N=0
jq -c '.features = (.features[] | [.])' input.json |
  while read -r json ; do
  N=$((N+1))
  jq . <<< "$json"  > "/tmp/file${N}.json"
done

对jq的每个附加调用都会很快，因此可以接受.

Each of the additional calls to jq will be fast, so this may be acceptable.

这篇关于将JSON文件对象拆分为多个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！