在 Qt 中迭代和修改 JSON [英] Iterating and modifying JSON in Qt

问题描述

我需要使用 JSON 通过网络传输大量数据，然后在高流量和性能要求高的情况下在运行时在 Qt 中解释它.基本上，需要处理任意大小和复杂性的数百甚至数千个 JSON 文档.我在如何做方面有些自由，但我正在仔细考虑我的选择，到目前为止我并不满足于它们的样子.

I need to use JSON to transfer high volume data over network and then interpret it in Qt on runtime in high traffic and performance demanding scenario. Basically hundreds or even thousands or more JSON documents would need to be handled of arbitrary sizes and complexity. I am somewhat free in how to do it but I am considering my options carefully and so far I am not content with what they seem to be.

主要关注的是查找和修改速度，而从/到 JSON 的转换在我的上下文中不太重要，因为我可以处理较慢的摄入和/或无论如何都会不稳定的输出.但是一旦数据通过，它就需要随时可用.

Main concern is lookup and modification speed while conversion from/to JSON is less significant in my context as I can deal with slower intake and or output that would be erratic anyway. But once the data gets through it needs to be readily available.

我看到的选项:

1) 选项 1 是直接使用 Qt 5 中的 JSON 支持类.然而，缺点是为了修改 JSON 数据树中某处的单个值，需要复制整个内容，然后使用修改后的值再次从它创建 JSON.对于单个修改，这可能会非常昂贵.QJsonValueRef 在某种程度上有所帮助，但不允许迭代整个结构，因为结构中的每个对象仍然需要复制.

1) Option 1 is to use the JSON support classes in Qt 5 directly. However the drawback is that in order to modify a single value somewhere in the JSON data tree the whole thing needs to be copied and then the JSON created from it again with the modified value. This could be quite costly for a single modification. The QJsonValueRef helps with it to some extent but does not allow to iterate over the whole structure as each object in the structure still needs to be copied.

2) 将 JSON 转换为 QVariantMap (QMap) 并使用它.除了来回转换的开销外，它会遇到与 1) 相同的问题，因为 QAssociativeIterable 不返回对其项目的引用，因此如果不先复制每个 QVariant，就无法下降到结构中.基本上它允许在 QVariantMap 的单个子级别中进行非复制迭代.它可能仍然比直接使用 Qt 的 JSON 类更好，但本质上它似乎与 1 具有相同的限制.

2) Convert the JSON into QVariantMap (QMap) and use that instead. Except the overhead of the conversion back and forth it suffers from the same problem as 1) since QAssociativeIterable does not return a reference to its items and thus cannot descend into the structure without copying each QVariant first. Basically it allows non-copy iteration in single sub-level of QVariantMap. It might still be better than using Qt's JSON classes directly though but essentially it seems to have the very same limitation as 1).

3) 将 JSON 转换为我自己的数据结构，这将允许在不复制任何值的情况下对数据结构进行降序和修改.每个级别"都将成为一个对象，其父级将对其进行访问，从而允许对其内容进行任意深度的非复制访问.这个解决方案会受到这样一个事实的影响，即我很可能无法为每个级别"实现一个与 Qt/STL 容器几乎一样高效的查找机制.但也许我可以将 Qt 的容器与这种方法结合起来，通过尽可能快的 operator== 重载将子指针存储在其中...

3) Convert the JSON to my own data structure that would allow descending through the structure and modification of the data without copying any of the values. Each "level" would become an object to which its parents would hold access thus allowing non-copy access to its contents in arbitrary depth. This solution would suffer from the fact that I very likely cannot implement a lookup mechanism for each "level" that is nearly as efficient as Qt's/STL containers. But maybe I can combine the Qt's containers with this approach by storing the children pointers in them with as fast operator== overload as possible...

4) 将 JSON 转换为平面二维数据结构如:

4) Convert JSON to a flat 2-dimensional data structure such as:

Level1/Level2/key : value
Level1/Level2/key2 : value2
Level1/key3 : value3

这种方法在数据存储方面存在大量冗余，并且在与 JSON 之间进行转换时可能成本很高，但可以说提供了最快的查找/修改速度，因为它不需要下降到任何数据结构中.然而，这种好处的成本对我来说似乎太大了(冗余和缓慢的转换).

This approach suffers from lots of redundancy in data storage and could be costly in conversions to/from JSON but arguably offers the greatest lookup/modification speed since it does not need to descend into a any data structure. However the cost of this benefit seems far too great to me (redundancy and slow conversion).

我有点倾向于选项 3，但也许还有其他选项.我想听听您对这个问题的想法和/或想法.或者 JSON 一开始就不是通过网络传输数据的合适工具.

I somewhat lean towards the option 3 but maybe there are other options. I would like to hear your thoughts and/or ideas about this problem. Or maybe JSON is not a right tool for data transfer over network in the first place.

推荐答案

If 您查看Qt源代码您会发现Qt json对象已经优化:

If you look at the Qt source code you will realize that Qt json objects are already optimized :

• 引用计数
• 写入时复制.
• 基本的内存池机制.与动态数组一样，内存容量大于实际内存大小.
• 用于数据的连续内存

要迭代结构，请使用 QJsonObject 类.

To iterate over the structure use the QJsonObject class. The overhead of

QJsonValue QJsonObject::value(const QString & key);

对于对象类型来说是最小的.但正如您所指出的，它为字符串类型创建了一个新字符串；它还对数值执行转换".请注意，解析后的进一步处理\内存开销仅在您开始处理 QJsonValue 时.

is minimal for object types. But as you noted, it creates a new string for string types; it also perform "casting" for numeric values. Note that after parsing further processing\memory overhead is only when you start dealing with QJsonValue.

因此选项 1 还不错.鉴于您的条件，选项 2 非常糟糕.我看不出任何优势.由于您自己指定的原因，选项 4 是噩梦般的.那种在取得一些进展后立即后悔的工作.

Thus option 1 is not that bad. Option 2 is plain horrible given your conditions. I fail to see any advantage. Option 4 is nightmarish, for the reasons you specified yourself. The kind of work you regret right after making some progress.

如果严格使用 Qt json 类还不够，那么请转向更高效的库，例如 rapidjson.

If a disciplined use of the Qt json classes is not enough, then move to a more efficient library like rapidjson.

优点:

• Json 只是一个大字符数组，元素只是引用 + 类型.
• 字符串管理:getstring返回json字符串中字符串的指针.零分配.插入字符串时，您可以使用仅引用现有数据缓冲区的版本，也可以像 Qt 一样分配字符串.
• 完全控制分配.对象分配是一个移动操作.除非您明确要求复制，否则保证不会有任何内存分配\移动.分配内存的方法有一个明确的文档分配器作为参数.
• 由于移动设计，零引用计数器.
• 显式评估 Json 值.

• the Json is just a big char array with element just being references + type.
• String management : the getstring return the pointer of the string in the json string. Zero allocation. When you insert strings, you can use a version which only refer to the buffer of the existing data or you can allocate the string as Qt does.
• Total control on allocations. Object assignment is a move operation. Unless you explicitly ask to copy you are guaranteed to not have any memory allocation\movement. Methods which allocate memory have an explicit document allocator as argument.
• Zero ref counter because of the move design.
• Json Values are evaluated explicitly.

不便

• 非直观的代码风格.仅限成年人.
• 更容易插入缺陷和错误代码.

它完成了您的选项 3 试图做的事情.

It does what your option 3 is trying to do.

这篇关于在 Qt 中迭代和修改 JSON的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！