在PHP中处理大型JSON文件 [英] Processing large JSON files in PHP

问题描述

我正在尝试处理较大(可能最多200M)的JSON文件. 该文件的结构基本上是一个对象数组.

I am trying to process somewhat large (possibly up to 200M) JSON files. The structure of the file is basically an array of objects.

所以类似:

[
  {"property":"value", "property2":"value2"},
  {"prop":"val"},
  ...
  {"foo":"bar"}
]

每个对象都具有任意属性，不必与数组中的其他对象共享它们(例如，具有相同的属性).

Each object has arbitrary properties and does not necessary share them with other objects in the array (as in, having the same).

我想对数组中的每个对象进行处理，并且由于文件可能很大，因此我无法将整个文件内容包含在内存中，无法解码JSON并遍历PHP数组.

I want to apply a processing on each object in the array and as the file is potentially huge, I cannot slurp the whole file content in memory, decoding the JSON and iterating over the PHP array.

因此，理想情况下，我想读取文件，为每个对象获取足够的信息并进行处理. 如果存在类似的JSON库，则使用SAX类型的方法就可以了.

So ideally I would like to read the file, fetch enough info for each object and process it. A SAX-type approach would be OK if there was a similar library available for JSON.

关于如何最好地解决此问题的任何建议?

Any suggestion on how to deal with this problem best?

推荐答案

我决定研究基于事件的解析器.尚未完成，当我发布令人满意的版本时，它将使用指向我工作的链接来编辑问题.

I decided on working on an event based parser. It's not quite done yet and will edit the question with a link to my work when I roll out a satisfying version.

我终于找到了我满意的解析器版本.可以在GitHub上使用:

I finally worked out a version of the parser that I am satisfied with. It's available on GitHub:

https://github.com/kuma-giyomu/JSONParser

可能还有一些改进的余地，欢迎反馈.

There's probably room for some improvement and am welcoming feedback.

这篇关于在PHP中处理大型JSON文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！