将JSON数组拆分为单独的文件/对象 [英] Split JSON array into separate files/objects

问题描述

我以这种格式从Cassandra导出了JSON.

I have JSON exported from Cassandra in this format.

[
  {
    "correlationId": "2232845a8556cd3219e46ab8",
    "leg": 0,
    "tag": "received",
    "offset": 263128,
    "len": 30,
    "prev": {
      "page": {
        "file": 0,
        "page": 0
      },
      "record": 0
    },
    "data": "HEAD /healthcheck HTTP/1.1\r\n\r\n"
  },
  {
    "correlationId": "2232845a8556cd3219e46ab8",
    "leg": 0,
    "tag": "sent",
    "offset": 262971,
    "len": 157,
    "prev": {
      "page": {
        "file": 10330,
        "page": 6
      },
      "record": 1271
    },
    "data": "HTTP/1.1 200 OK\r\nDate: Wed, 14 Feb 2018 12:57:06 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-2232845a8556cd3219e46ab8 0\r\nContent-Type: text/xml\r\n\r\n"
  }]

我想将其拆分为单独的文档:

I would like to split it to separate documents:

{ "correlationId":"2232845a8556cd3219e46ab8"， 腿":0， "tag":已收到"， 偏移":263128， "len":30， 上一个":{ 页": { 文件":0， 页面":0 }， 记录":0 }， "data":"HEAD/healthcheck HTTP/1.1 \ r \ n \ r \ n" }

{ "correlationId": "2232845a8556cd3219e46ab8", "leg": 0, "tag": "received", "offset": 263128, "len": 30, "prev": { "page": { "file": 0, "page": 0 }, "record": 0 }, "data": "HEAD /healthcheck HTTP/1.1\r\n\r\n" }

和

{ "correlationId":"2232845a8556cd3219e46ab8"， 腿":0， "tag":已发送"， 偏移":262971， "len":157， 上一个":{ 页": { 文件":10330， 页面":6 }， 记录":1271 }， 数据":"HTTP/1.1 200 OK \ r \ n日期:2018年2月14日，星期三12:57:06 GMT \ r \ n服务器:\ r \ n连接:close \ r \ nX-CorrelationID: ID-2232845a8556cd3219e46ab8 0 \ r \ n内容类型:text/xml \ r \ n \ r \ n }

{ "correlationId": "2232845a8556cd3219e46ab8", "leg": 0, "tag": "sent", "offset": 262971, "len": 157, "prev": { "page": { "file": 10330, "page": 6 }, "record": 1271 }, "data": "HTTP/1.1 200 OK\r\nDate: Wed, 14 Feb 2018 12:57:06 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-2232845a8556cd3219e46ab8 0\r\nContent-Type: text/xml\r\n\r\n" }

我想使用 jq ，但是找不到方法.

I wanted to use jq but didn't find way how.

请问方法，如何用文件分隔符分开?

Can you please advise way, how to split it by the document separator?

谢谢，雷迪

推荐答案

使用jq，可以使用过滤器将数组拆分为其组件:

Using jq, one can split an array into its components using the filter:

.[]

问题就变成了每个组件要做什么.如果要将每个组件定向到一个单独的文件，则可以(例如)将jq与-c选项一起使用，并将结果过滤到awk中，然后可以将这些组件分配给不同的文件.参见例如将JSON文件对象拆分为多个文件

The question then becomes what is to be done with each component. If you want to direct each component to a separate file, you could (for example) use jq with the -c option, and filter the result into awk, which can then allocate the components to different files. See e.g. Split JSON File Objects Into Multiple Files

人们可能会认为，调用jq + awk的开销要比调用python的开销高，但是jq和awk的开销都比python + json轻，这是这些计时所建议的(使用Python 2.7.10):

One might think that the overhead of calling jq+awk would be high compared to calling python, but both jq and awk are lightweight compared to python+json, as suggested by these timings (using Python 2.7.10):

time (jq -c  .[] input.json | awk '{print > "doc00" NR ".json";}')
user    0m0.005s
sys     0m0.008s

time python split.py
user    0m0.016s
sys     0m0.046s

这篇关于将JSON数组拆分为单独的文件/对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！