当与 Jackson ObjectMapper treeToValue 方法一起使用时，Jackson @JsonAnySetter 会忽略重复键的值 [英] Jackson @JsonAnySetter ignores values of duplicate key when used with Jackson ObjectMapper treeToValue method
问题描述
我正在使用 Jackson 库来反序列化 JSON.在 JSON 中，我有一些自定义字段，它们的值可以是任何值，因此我尝试使用 @JsonAnySetter 和 @JsonAnyGetter 来获取值.JSON 中的字段和值可以重复，我想从 JSON 中获取所有内容并将其存储在地图中.但是，如果键中存在重复项，则直接的 Jackson 实现会存储最后一个值.
我有一个包含许多事件的大型 JSON 文件.我正在逐个读取文件，以便整个 JSON 文件不会存储在内存中.读取单个事件后，我检查事件类型，并根据类型将其分配给不同的 POJO.以下是我的示例 JSON 文件，包含 2 个事件.
<预><代码>[{isA":Type1"，名称":测试"，foo":val1"，foo":val2"，bar":val3"，富":{myField":Value1"，myField":value2"；}},{isA":Type2"，名称":测试1"，foo":val1"，foo":val2"，bar":val3"，富":{myField":Value1"，myField":value2"；}}]以下是用于反序列化的类:(我有许多在反序列化过程中直接映射的字段，并且可以正常工作，因此为简单起见省略了).这是 type1 事件的类，如果它是 type2
@JsonInclude(JsonInclude.Include.NON_NULL)@数据@NoArgsConstructor@AllArgsConstructor@Builder公共类 Type1{私人字符串名称；私有映射<字符串，对象>用户扩展；@JsonAnyGetter公共地图<字符串，对象>getUserExtensions() {返回用户扩展；}@JsonAnySetterpublic void setUserExtensions(String key, Object value) {System.out.println("KEY:"+key+"VALUES:"+values);}}从上面可以看出，我有一个 Map 字段，用于使用 JsonAnySetter 填充扩展.我有一个方法可以为杰克逊无法直接搜索的字段调用.
我尝试在 @JsonAnySetter 方法中设置 System.out 并且我观察到 Jackson 在这个方法中没有得到重复的字段:
@JsonAnySetterpublic void setUserExtensions(String key, Object value) {System.out.println("键:"+键+"值:"+值);}我只得到最后一个字段.对于上面提到的 JSON，我只得到最后一个 foo 的值:
foo":{我的领域":值1"我的领域":值2"}带有 val1 和 val2 的前 2 个 foo 甚至不会在此方法中打印.
以下是我的 Main 方法，它实际上是罪魁祸首，因为我正在使用不支持重复字段的 Jackson 的 objectMapper.treeToValue.
公共类 Main{public static void main (String[]args){//存储JSON的文件InputStream jsonStream = Main.class.getClassLoader().getResourceAsStream(InputEPCISEvents.json");最终 JsonFactory jsonFactory = new JsonFactory();最终的 JsonParser jsonParser = jsonFactory.createParser (jsonStream);jsonParser.setCodec(new ObjectMapper());final ObjectMapper objectMapper = new ObjectMapper();//循环直到事件文件结束而 (jsonParser.nextToken() != JsonToken.END_ARRAY) {//获取节点最终 JsonNode jsonNode = jsonParser.readValueAsTree();//获取事件类型final String eventType = jsonNode.get("isA").toString().replaceAll(""", "");开关(事件类型){案例类型1":最终 Type1 objInfo = objectMapper.treeToValue(jsonNode, Type1.class);休息;案例类型2":最终 Type2 objInfo = objectMapper.treeToValue(jsonNode, Type2.class);休息;默认:System.out.println(没有任何事件匹配");休息;}}}}我想知道如何让 Jackson 读取重复的键值，以便我可以在 @JsonAnySetter 方法中处理它们.
我想知道是否有办法直接使用 Jackson 处理这些场景，或者我需要构建自己的自定义反序列化.如果是，那么我如何为班级中的一个字段构建一个?
PS:我尝试了很多在线可用的东西，但都没有奏效，因此发布了相同的内容.如果发现重复，我真的很抱歉.
我注意到 Stack Overflow 上的大部分代码示例都使用 Jackson readValue 方法读取 JSON，但就我而言，我有一个包含许多事件的大型 JSON 文件，因此我将它们拆分为事件并使用 Jackson objectMapper.treeToValue 方法将其映射到我的类.在这个类中，我尝试将每个事件的字段与相应的类字段进行映射，如果没有找到匹配项，那么我希望使用 @JsonAnySetter 方法填充它们.
我想我有一个可能对你有用的解决方案，或者至少让你向前迈进了几步.我使用您的文件创建了一个本地测试来演示.
通过查找 FAIL_ON_READING_DUP_TREE_KEY 的实现方式，我找到了一种合理分配的方法.基本上，响应它的代码在 JsonNodeDeserializer 中.如果你看到这个反序列化器如何看待 _handleDuplicateField，它基本上只会用最新的节点(你看到的副作用)替换最新的节点，如果你设置了失败功能，它就会有条件地抛出异常.然而，通过使用重写方法创建这个反序列化器类的扩展，我们基本上可以实现合并欺骗"的行为.而不是当你看到一个错误时抛出一个错误".结果如下.
来自我的测试运行的控制台日志产生了这个，证明了foos"包含所有独立的字符串和对象...
由于它是在 JSON 级别而不是在对象级别进行合并，它似乎甚至无意中通过将多个字符串合并为一个 arrayNode 来处理内部 myField 用例......我什至没有尝试当时为 myField 求解，但是嘿，你也去！
我没有处理 Type2，因为我认为这足以让您继续前进.
当然，您必须在测试包中使用上述内容创建文件 jsonmerge/jsonmerge.json
控制台日志结果
使用 Dupes 读取对象节点:{isA":Type1"，name":Test"，foo":[val1"，val2"，{"myField":["Value1","value2"]}],"bar":"val3"}合并后映射的类型 1 :Type1 [isA=Type1, name=Test, bar=val3, foos=[val1, val2, {myField=[Value1, value2]}]]没有任何事件匹配EOF测试主要代码 - JsonMerger.java
package jsonmerge;导入 java.io.InputStream;导入 com.fasterxml.jackson.core.JsonFactory;导入 com.fasterxml.jackson.core.JsonParser;导入 com.fasterxml.jackson.core.JsonToken;导入 com.fasterxml.jackson.databind.JsonNode;导入 com.fasterxml.jackson.databind.ObjectMapper;导入 com.fasterxml.jackson.databind.module.SimpleModule;导入 com.fasterxml.jackson.databind.node.ObjectNode;/*** 项目解决 https://stackoverflow.com/questions/67413028/jackson-jsonanysetter-ignores-values-of-duplicate-key-when-used-with-jackson-ob**/公共类 JsonMerger {public static void main(String[] args) 抛出异常 {//存储 JSON 的文件InputStream jsonStream = JsonMerger.class.getClassLoader().getResourceAsStream("jsonmerge/jsonmerge.json");最终 JsonFactory jsonFactory = new JsonFactory();final ObjectMapper objectMapper = new ObjectMapper();//注入一个反序列化器，它可以处理/合并任何对象类型的重复字段声明到一个包含这些对象的 arrayNode 中SimpleModule 模块 = new SimpleModule();module.addDeserializer(JsonNode.class, new JsonNodeDupeFieldHandlingDeserializer());objectMapper.registerModule(module);最终的 JsonParser jsonParser = jsonFactory.createParser(jsonStream);jsonParser.setCodec(objectMapper);JsonToken nextToken = jsonParser.nextToken();//循环直到事件文件结束而 (nextToken != JsonToken.END_ARRAY) {nextToken = jsonParser.nextToken();最终 JsonNode jsonNode = jsonParser.readValueAsTree();if (jsonNode == null || jsonNode.isNull()) {System.out.println(EOF");休息;}//获取事件类型JsonNode getNode = jsonNode.get("isA");最终字符串 eventType = getNode.toString().replaceAll(""", "");开关(事件类型){案例类型1":最终对象 obj = objectMapper.treeToValue(jsonNode, JsonNodeDupeFieldHandlingDeserializer.class);ObjectNode objNode = (ObjectNode) obj;System.out.println();System.out.println();System.out.println("对象节点使用重复读取:" + objNode);Type1 type1 = objectMapper.treeToValue(objNode, Type1.class);System.out.println("合并后映射的类型1:" + type1);休息;默认:System.out.println(没有任何事件匹配");休息;}}}}Type1.java
package jsonmerge;导入 java.util.Collection;导入 java.util.LinkedList;导入 java.util.List;导入 com.fasterxml.jackson.annotation.JsonAnySetter;公共类 Type1 {公共字符串 isA;公共字符串名称；公共字符串栏；公共列表<对象>福斯；公共类型1(){foos = new LinkedListJsonNodeDupeFieldHandlingDeserializer
package jsonmerge;导入 com.fasterxml.jackson.core.JsonParser;导入 com.fasterxml.jackson.core.JsonProcessingException;导入 com.fasterxml.jackson.databind.DeserializationContext;导入 com.fasterxml.jackson.databind.JsonNode;导入 com.fasterxml.jackson.databind.annotation.JsonDeserialize;导入 com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer；导入 com.fasterxml.jackson.databind.node.ArrayNode;导入 com.fasterxml.jackson.databind.node.JsonNodeFactory;导入 com.fasterxml.jackson.databind.node.ObjectNode;/*** 如果 JsonNodes 的值与 arrayNode 冲突，则通过合并它们的内容来处理 JsonNodes**/@JsonDeserialize(使用 = JsonNodeDupeFieldHandlingDeserializer.class)公共类 JsonNodeDupeFieldHandlingDeserializer 扩展 JsonNodeDeserializer {private static final long serialVersionUID = 1L;@覆盖protected void _handleDuplicateField(JsonParser p, DeserializationContext ctxt, JsonNodeFactory nodeFactory, String fieldName, ObjectNode objectNode, JsonNode oldValue, JsonNode newValue) 抛出 JsonProcessingException {//当遇到重复的字段时，不要在这个超级逻辑中抛出异常...(如果该功能还是设置了)//super._handleDuplicateField(p, ctxt, nodeFactory, fieldName, objectNode, oldValue, newValue);//将结果合并到一个公共的arrayNode//注意，这里假设多个值将组合成一个数组...//并且 *THAT* 数组将用于将来要附加到的节点...//但是如果首先组合 *IS* 是一个数组，那么它将是//初始数组...//您可能可以坚持某种方式来跟踪是否遇到了初始数组，但现在不想考虑..ArrayNode asArrayValue = null;如果(oldValue.isArray()){asArrayValue = (ArrayNode) oldValue;} 别的 {//如果不是，则创建为替换数组并添加初始值..asArrayValue = nodeFactory.arrayNode();asArrayValue.add(oldValue);}asArrayValue.add(newValue);objectNode.set(fieldName, asArrayValue);}}I am using the Jackson library to deserialize JSON. In the JSON I have a few custom fields whose values can be anything, so I am trying to use the @JsonAnySetter and @JsonAnyGetter to obtain the values. The fields and values within the JSON can be duplicated and I would like to obtain everything from the JSON and store it within the map. However, the direct Jackson derealization is storing the last value if there are duplicates in the key.
I have a large JSON file that has many events. I am reading the file event-by-event so that the whole JSON file is not stored within memory. After reading a single event, I check the type of event, based on which I assign it to a different POJO. Following is my sample JSON file consisting of 2 events.
[
{
"isA": "Type1",
"name": "Test",
"foo": "val1",
"foo": "val2",
"bar": "val3",
"foo": {
"myField": "Value1",
"myField": "value2"
}
},
{
"isA": "Type2",
"name": "Test1",
"foo": "val1",
"foo": "val2",
"bar": "val3",
"foo": {
"myField": "Value1",
"myField": "value2"
}
}
]
Following is the class that is used for deserialization: (I have many fields which are mapped directly during the deserialization and working correctly so omitted for simplicity). This is the class for the type1 event if it's type2
@JsonInclude(JsonInclude.Include.NON_NULL)
@Data
@NoArgsConstructor
@AllArgsConstructor
@Builder
public class Type1
{
private String name;
private Map<String, Object> userExtensions;
@JsonAnyGetter
public Map<String, Object> getUserExtensions() {
return userExtensions;
}
@JsonAnySetter
public void setUserExtensions(String key, Object value) {
System.out.println("KEY : " + key + " VALUES : " + values);
}
}
As you can observe from above I have a Map field that is used to populate the extensions using the JsonAnySetter. I have a method which will be called for the fields which cannot be directly searlized by Jackson.
I tried setting the System.out within the @JsonAnySetter method and I observed that Jackson does not get the duplicate field within this method:
@JsonAnySetter
public void setUserExtensions(String key, Object value) {
System.out.println(" Key : " + key + " Value : " + value);
}
I get only the last fields in this. For the above mentioned JSON I get only the last foo which has the value:
"foo" :{
"myField" : "Value1"
"myField" : "value2"
}
The first 2 foo with val1 and val2 does not even print within this method.
Following is my Main method which is actually the culprit as I am using the objectMapper.treeToValue of Jackson which does not support the duplicate fields.
public class Main
{
public static void main (String[]args)
{
//File where the JSON is stored
InputStream jsonStream = Main.class.getClassLoader().getResourceAsStream("InputEPCISEvents.json");
final JsonFactory jsonFactory = new JsonFactory();
final JsonParser jsonParser = jsonFactory.createParser (jsonStream);
jsonParser.setCodec (new ObjectMapper ());
final ObjectMapper objectMapper = new ObjectMapper();
// Loop until the end of the events file
while (jsonParser.nextToken() != JsonToken.END_ARRAY) {
// Get the node
final JsonNode jsonNode = jsonParser.readValueAsTree();
// Get the eventType
final String eventType = jsonNode.get("isA").toString().replaceAll(""", "");
switch (eventType) {
case "Type1":
final Type1 objInfo = objectMapper.treeToValue(jsonNode, Type1.class);
break;
case "Type2":
final Type2 objInfo = objectMapper.treeToValue(jsonNode, Type2.class);
break;
default:
System.out.println("None of the event Matches");
break;
}
}
}
}
I wanted to know how can I make Jackson read the duplicate key values so that I can handle them within the @JsonAnySetter method.
I wanted to know if there is a way to handle these scenarios directly using Jackson or I need to build my own custom deserialization. If yes, then how can I build one for only one field within the class?
PS: I tried many things available online but none worked hence posting the same. If found duplicate then I am really sorry.
I noticed that most of the code sample on Stack Overflow read JSON with the Jackson readValue method, but in my case I have a large JSON file that has many events so I am splitting them event by event and using the Jackson objectMapper.treeToValue method to map it to my class. Within this class, I try to map the fields of each event with the respective class fields and if no match found then I am expecting them to be populated using the @JsonAnySetter method.
I think I have a solution that may work for you, or at the very least move you a few steps forward. I created a local test using your files to demonstrate.
I figured out a way to ration about it by looking for how FAIL_ON_READING_DUP_TREE_KEY was implemented. Basically, the code that's responding to it is in JsonNodeDeserializer. If you see how this deserializer looks at _handleDuplicateField, it will basically just replace the newest node with the latest (the side effect you're seeing), and if you set the fail feature it will then conditionally throw an exception. However, by creating an extension of this deserializer class with an overriden method, we can basically implement the behavior of "merge the dupe" instead of "throw an error when you see one". The results of that are below.
The console logs from my test run yield this, demonstrating that "foos" contains all the independent strings and objects...
Since it did the merge at the JSON level instead of at the object level, it seems to have even inadvertently handled the inner myField use case by merging the multiple strings into a single arrayNode... I wasn't even trying to solve for myField at the time, but hey, there you go too!
I didn't handle Type2, because I figure this is enough to get you going.
Of course, you will have to create the file with your contents above in the test package as jsonmerge/jsonmerge.json
Console log results
Object Node Read with Dupes: {"isA":"Type1","name":"Test","foo":["val1","val2",{"myField":["Value1","value2"]}],"bar":"val3"}
Type 1 mapped after merge : Type1 [isA=Type1, name=Test, bar=val3, foos=[val1, val2, {myField=[Value1, value2]}]]
None of the event Matches
EOF
Test main code - JsonMerger.java
package jsonmerge;
import java.io.InputStream;
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.node.ObjectNode;
/**
* Project to resolve https://stackoverflow.com/questions/67413028/jackson-jsonanysetter-ignores-values-of-duplicate-key-when-used-with-jackson-ob
*
*/
public class JsonMerger {
public static void main(String[] args) throws Exception {
// File where the JSON is stored
InputStream jsonStream = JsonMerger.class.getClassLoader()
.getResourceAsStream("jsonmerge/jsonmerge.json");
final JsonFactory jsonFactory = new JsonFactory();
final ObjectMapper objectMapper = new ObjectMapper();
// Inject a deserializer that can handle/merge duplicate field declarations of whatever object(s) type into an arrayNode with those objects inside it
SimpleModule module = new SimpleModule();
module.addDeserializer(JsonNode.class, new JsonNodeDupeFieldHandlingDeserializer());
objectMapper.registerModule(module);
final JsonParser jsonParser = jsonFactory.createParser(jsonStream);
jsonParser.setCodec(objectMapper);
JsonToken nextToken = jsonParser.nextToken();
// Loop until the end of the events file
while (nextToken != JsonToken.END_ARRAY) {
nextToken = jsonParser.nextToken();
final JsonNode jsonNode = jsonParser.readValueAsTree();
if (jsonNode == null || jsonNode.isNull()) {
System.out.println("EOF");
break;
}
// Get the eventType
JsonNode getNode = jsonNode.get("isA");
final String eventType = getNode
.toString()
.replaceAll(""", "");
switch (eventType) {
case "Type1":
final Object obj = objectMapper.treeToValue(jsonNode, JsonNodeDupeFieldHandlingDeserializer.class);
ObjectNode objNode = (ObjectNode) obj;
System.out.println();
System.out.println();
System.out.println("Object Node Read with Dupes: " + objNode);
Type1 type1 = objectMapper.treeToValue(objNode, Type1.class);
System.out.println("Type 1 mapped after merge : " + type1);
break;
default:
System.out.println("None of the event Matches");
break;
}
}
}
}
Type1.java
package jsonmerge;
import java.util.Collection;
import java.util.LinkedList;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonAnySetter;
public class Type1 {
public String isA;
public String name;
public String bar;
public List<Object> foos;
public Type1() {
foos = new LinkedList<Object>();
}
/**
* Inspired by https://stackoverflow.com/questions/61528937/deserialize-duplicate-keys-to-list-using-jackson
*
* @param key
* @param value
*/
@JsonAnySetter
public void fieldSetters(String key, Object value) {
// System.out.println("key = " + key + " -> " + value.getClass() + ": " + value);
// Handle duplicate "foo" fields to merge in strings/objects into a List<Object>
if ("foo".equals(key) && value instanceof String) {
foos.add((String) value);
} else if ("foo".equals(key) && (value instanceof Collection<?>)) {
foos.addAll((Collection<?>) value);
}
}
@Override
public String toString() {
return "Type1 [isA=" + isA + ", name=" + name + ", bar=" + bar + ", foos=" + foos + "]";
}
}
JsonNodeDupeFieldHandlingDeserializer
package jsonmerge;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.databind.node.ObjectNode;
/**
* Handles JsonNodes by merging their contents if their values conflict into an arrayNode
*
*/
@JsonDeserialize(using = JsonNodeDupeFieldHandlingDeserializer.class)
public class JsonNodeDupeFieldHandlingDeserializer extends JsonNodeDeserializer {
private static final long serialVersionUID = 1L;
@Override
protected void _handleDuplicateField(JsonParser p, DeserializationContext ctxt, JsonNodeFactory nodeFactory, String fieldName, ObjectNode objectNode, JsonNode oldValue, JsonNode newValue) throws JsonProcessingException {
// When you encounter a duplicate field, instead of throwing an exception in this super-logic... (if the feature was set anyway)
// super._handleDuplicateField(p, ctxt, nodeFactory, fieldName, objectNode, oldValue, newValue);
// Merge the results into a common arrayNode
// Note, this assumes that multiple values will combine into an array...
// And *THAT* array will be used for future nodes to be tacked onto...
// But if the FIRST thing to combine *IS* an array, then it will be the
// initial array...
// You could probably persist some way to track whether the initial array was encountered, but don't want to think about that now..
ArrayNode asArrayValue = null;
if (oldValue.isArray()) {
asArrayValue = (ArrayNode) oldValue;
} else {
// If not, create as array for replacement and add initial value..
asArrayValue = nodeFactory.arrayNode();
asArrayValue.add(oldValue);
}
asArrayValue.add(newValue);
objectNode.set(fieldName, asArrayValue);
}
}
这篇关于当与 Jackson ObjectMapper treeToValue 方法一起使用时，Jackson @JsonAnySetter 会忽略重复键的值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
