杰克逊@JsonAnySetter与杰克逊ObjectMapper treeToValue方法一起使用时会忽略重复键的值 [英] Jackson @JsonAnySetter ignores values of duplicate key when used with Jackson ObjectMapper treeToValue method

问题描述

我正在使用Jackson库反序列化JSON.在JSON中，我有一些自定义字段，其值可以是任何值，因此我尝试使用 @JsonAnySetter 和 @JsonAnyGetter 来获取值.JSON中的字段和值可以重复，我想从JSON中获取所有内容并将其存储在地图中.但是，如果密钥中有重复项，则直接的Jackson重新实现会存储最后一个值.

I am using the Jackson library to deserialize JSON. In the JSON I have a few custom fields whose values can be anything, so I am trying to use the @JsonAnySetter and @JsonAnyGetter to obtain the values. The fields and values within the JSON can be duplicated and I would like to obtain everything from the JSON and store it within the map. However, the direct Jackson derealization is storing the last value if there are duplicates in the key.

我有一个大型的JSON文件，其中包含许多事件.我正在逐个事件读取文件，以便整个JSON文件不存储在内存中.读取单个事件后，我检查事件的类型，并根据该事件将其分配给其他POJO.以下是我的示例JSON文件，其中包含2个事件.

I have a large JSON file that has many events. I am reading the file event-by-event so that the whole JSON file is not stored within memory. After reading a single event, I check the type of event, based on which I assign it to a different POJO. Following is my sample JSON file consisting of 2 events.

[
  {
    "isA": "Type1",
    "name": "Test",
    "foo": "val1",
    "foo": "val2",
    "bar": "val3",
    "foo": {
      "myField": "Value1",
      "myField": "value2"
    }
  },
  {
    "isA": "Type2",
    "name": "Test1",
    "foo": "val1",
    "foo": "val2",
    "bar": "val3",
    "foo": {
      "myField": "Value1",
      "myField": "value2"
    }
  }
]

以下是用于反序列化的类:(我有许多字段在反序列化期间直接映射，并且可以正常工作，为简单起见，省略了这些字段).如果它是 type2

Following is the class that is used for deserialization: (I have many fields which are mapped directly during the deserialization and working correctly so omitted for simplicity). This is the class for the type1 event if it's type2

@JsonInclude(JsonInclude.Include.NON_NULL)
@Data
@NoArgsConstructor
@AllArgsConstructor
@Builder
public class Type1
{
    private String name;
    private Map<String, Object> userExtensions;
   
    @JsonAnyGetter
    public Map<String, Object> getUserExtensions() {
        return userExtensions;
    }
    
    @JsonAnySetter
    public void setUserExtensions(String key, Object value) {
        System.out.println("KEY : " + key + " VALUES : " + values);
    }
    
}

从上面可以看到，我有一个 Map 字段，该字段用于使用 JsonAnySetter 填充扩展名.我有一种方法将被杰克逊无法直接处理的字段调用.

As you can observe from above I have a Map field that is used to populate the extensions using the JsonAnySetter. I have a method which will be called for the fields which cannot be directly searlized by Jackson.

我尝试在 @JsonAnySetter 方法中设置 System.out ，但我发现Jackson在此方法中没有获取重复字段:

I tried setting the System.out within the @JsonAnySetter method and I observed that Jackson does not get the duplicate field within this method:

  @JsonAnySetter
  public void setUserExtensions(String key, Object value) {
    System.out.println(" Key : " + key + " Value : " + value);
  }

我只得到最后的字段.对于上面提到的JSON，我只得到最后一个 foo ，其值是:

I get only the last fields in this. For the above mentioned JSON I get only the last foo which has the value:

"foo" :{
  "myField" : "Value1"
  "myField" : "value2"
}

前两个带有 val1 和 val2 的 foo 甚至不在此方法中打印.

The first 2 foo with val1 and val2 does not even print within this method.

以下是我的 Main 方法，实际上是罪魁祸首，因为我使用的是Jackson的 objectMapper.treeToValue ，它不支持重复字段.

Following is my Main method which is actually the culprit as I am using the objectMapper.treeToValue of Jackson which does not support the duplicate fields.

public class Main
{
  public static void main (String[]args)
  {
    //File where the JSON is stored
    InputStream jsonStream = Main.class.getClassLoader().getResourceAsStream("InputEPCISEvents.json");
    final JsonFactory jsonFactory = new JsonFactory();
    final JsonParser jsonParser = jsonFactory.createParser (jsonStream);
    jsonParser.setCodec (new ObjectMapper ());
    final ObjectMapper objectMapper = new ObjectMapper();
    
    // Loop until the end of the events file
    while (jsonParser.nextToken() != JsonToken.END_ARRAY) {
        // Get the node
        final JsonNode jsonNode = jsonParser.readValueAsTree();
        // Get the eventType
        final String eventType = jsonNode.get("isA").toString().replaceAll("\"", "");
        
        switch (eventType) {
            case "Type1":
                final Type1 objInfo = objectMapper.treeToValue(jsonNode, Type1.class);
                break;
            case "Type2":
                final Type2 objInfo = objectMapper.treeToValue(jsonNode, Type2.class);
                break;
            default:
                System.out.println("None of the event Matches");
                break;
        }
        
    }
      
  }
}

我想知道如何让Jackson读取重复的键值，以便可以在 @JsonAnySetter 方法中进行处理.

I wanted to know how can I make Jackson read the duplicate key values so that I can handle them within the @JsonAnySetter method.

我想知道是否有一种方法可以直接使用Jackson来处理这些情况，或者我需要构建自己的自定义反序列化.如果是，那我怎么能只为班级中的一个领域建造一个?

I wanted to know if there is a way to handle these scenarios directly using Jackson or I need to build my own custom deserialization. If yes, then how can I build one for only one field within the class?

PS:我尝试了许多在线可用的内容，但没有任何效果，因此发布了相同的内容.如果发现重复，那么我真的很抱歉.

PS: I tried many things available online but none worked hence posting the same. If found duplicate then I am really sorry.

我注意到Stack Overflow上的大多数代码示例都是使用Jackson readValue 方法读取JSON的，但就我而言，我有一个大型的JSON文件，其中包含许多事件，因此我将事件按事件，并使用Jackson的 objectMapper.treeToValue 方法将其映射到我的课程.在该类中，我尝试将每个事件的字段与相应的类字段进行映射，如果找不到匹配项，那么我期望使用 @JsonAnySetter 方法填充它们.

I noticed that most of the code sample on Stack Overflow read JSON with the Jackson readValue method, but in my case I have a large JSON file that has many events so I am splitting them event by event and using the Jackson objectMapper.treeToValue method to map it to my class. Within this class, I try to map the fields of each event with the respective class fields and if no match found then I am expecting them to be populated using the @JsonAnySetter method.

推荐答案

我认为我有一个可能对您有用的解决方案，或者至少可以使您向前走几步.我使用您的文件创建了一个本地测试来进行演示.

I think I have a solution that may work for you, or at the very least move you a few steps forward. I created a local test using your files to demonstrate.

我通过寻找如何实现FAIL_ON_READING_DUP_TREE_KEY来找到一种合理的分配方式.基本上，响应它的代码在JsonNodeDeserializer中.如果您看到此反序列化器对_handleDuplicateField的外观，它基本上只会将最新的节点替换为最新的(您看到的副作用)，如果设置了fail功能，它将有条件地引发异常.但是，通过使用重写方法创建该解串器类的扩展，我们可以从根本上实现合并伪装"行为.而不是看到时抛出错误".结果如下.

I figured out a way to ration about it by looking for how FAIL_ON_READING_DUP_TREE_KEY was implemented. Basically, the code that's responding to it is in JsonNodeDeserializer. If you see how this deserializer looks at _handleDuplicateField, it will basically just replace the newest node with the latest (the side effect you're seeing), and if you set the fail feature it will then conditionally throw an exception. However, by creating an extension of this deserializer class with an overriden method, we can basically implement the behavior of "merge the dupe" instead of "throw an error when you see one". The results of that are below.

我的测试运行中的控制台日志显示了这一点，表明"foos"包含所有独立的字符串和对象...

The console logs from my test run yield this, demonstrating that "foos" contains all the independent strings and objects...

由于它是在JSON级别而不是在对象级别进行合并的，因此它似乎甚至通过将多个字符串合并到单个arrayNode中而无意中处理了内部myField用例...我什至没有尝试当时为myField解决，但嘿，你也去了！

Since it did the merge at the JSON level instead of at the object level, it seems to have even inadvertently handled the inner myField use case by merging the multiple strings into a single arrayNode... I wasn't even trying to solve for myField at the time, but hey, there you go too!

我没有处理Type2，因为我认为这足以使您前进.

I didn't handle Type2, because I figure this is enough to get you going.

当然，您必须在测试包中使用上面的内容创建文件jsonmerge/jsonmerge.json

Of course, you will have to create the file with your contents above in the test package as jsonmerge/jsonmerge.json

控制台日志结果

Object Node Read with Dupes: {"isA":"Type1","name":"Test","foo":["val1","val2",{"myField":["Value1","value2"]}],"bar":"val3"}
Type 1 mapped after merge : Type1 [isA=Type1, name=Test, bar=val3, foos=[val1, val2, {myField=[Value1, value2]}]]
None of the event Matches
EOF

测试主要代码-JsonMerger.java

package jsonmerge;

import java.io.InputStream;

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.node.ObjectNode;

/**
 * Project to resolve https://stackoverflow.com/questions/67413028/jackson-jsonanysetter-ignores-values-of-duplicate-key-when-used-with-jackson-ob
 *
 */
public class JsonMerger {

    public static void main(String[] args) throws Exception {

        // File where the JSON is stored
        InputStream jsonStream = JsonMerger.class.getClassLoader()
            .getResourceAsStream("jsonmerge/jsonmerge.json");
        final JsonFactory jsonFactory = new JsonFactory();
        final ObjectMapper objectMapper = new ObjectMapper();

        // Inject a deserializer that can handle/merge duplicate field declarations of whatever object(s) type into an arrayNode with those objects inside it
        SimpleModule module = new SimpleModule();
        module.addDeserializer(JsonNode.class, new JsonNodeDupeFieldHandlingDeserializer());
        objectMapper.registerModule(module);

        final JsonParser jsonParser = jsonFactory.createParser(jsonStream);
        jsonParser.setCodec(objectMapper);

        JsonToken nextToken = jsonParser.nextToken();

        // Loop until the end of the events file
        while (nextToken != JsonToken.END_ARRAY) {
            nextToken = jsonParser.nextToken();

            final JsonNode jsonNode = jsonParser.readValueAsTree();
            if (jsonNode == null || jsonNode.isNull()) {
                System.out.println("EOF");
                break;
            }

            // Get the eventType
            JsonNode getNode = jsonNode.get("isA");
            final String eventType = getNode
                .toString()
                .replaceAll("\"", "");

            switch (eventType) {
            case "Type1":
                final Object obj = objectMapper.treeToValue(jsonNode, JsonNodeDupeFieldHandlingDeserializer.class);
                ObjectNode objNode = (ObjectNode) obj;
                System.out.println();
                System.out.println();

                System.out.println("Object Node Read with Dupes: " + objNode);

                Type1 type1 = objectMapper.treeToValue(objNode, Type1.class);
                System.out.println("Type 1 mapped after merge : " + type1);

                break;
            default:
                System.out.println("None of the event Matches");
                break;
            }

        }

    }
}

Type1.java

package jsonmerge;

import java.util.Collection;
import java.util.LinkedList;
import java.util.List;

import com.fasterxml.jackson.annotation.JsonAnySetter;

public class Type1 {

    public String       isA;
    public String       name;
    public String       bar;
    public List<Object> foos;

    public Type1() {

        foos = new LinkedList<Object>();
    }

    /**
     * Inspired by https://stackoverflow.com/questions/61528937/deserialize-duplicate-keys-to-list-using-jackson
     * 
     * @param key
     * @param value
     */
    @JsonAnySetter
    public void fieldSetters(String key, Object value) {

        // System.out.println("key = " + key + " -> " + value.getClass() + ": " + value);
        // Handle duplicate "foo" fields to merge in strings/objects into a List<Object>
        if ("foo".equals(key) && value instanceof String) {
            foos.add((String) value);
        } else if ("foo".equals(key) && (value instanceof Collection<?>)) {
            foos.addAll((Collection<?>) value);
        }

    }

    @Override
    public String toString() {

        return "Type1 [isA=" + isA + ", name=" + name + ", bar=" + bar + ", foos=" + foos + "]";
    }

}

JsonNodeDupeFieldHandlingDeserializer

package jsonmerge;

import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.databind.node.ObjectNode;

/**
 * Handles JsonNodes by merging their contents if their values conflict into an arrayNode
 * 
 */
@JsonDeserialize(using = JsonNodeDupeFieldHandlingDeserializer.class)
public class JsonNodeDupeFieldHandlingDeserializer extends JsonNodeDeserializer {

    private static final long serialVersionUID = 1L;

    @Override
    protected void _handleDuplicateField(JsonParser p, DeserializationContext ctxt, JsonNodeFactory nodeFactory, String fieldName, ObjectNode objectNode, JsonNode oldValue, JsonNode newValue) throws JsonProcessingException {

        // When you encounter a duplicate field, instead of throwing an exception in this super-logic... (if the feature was set anyway)
//      super._handleDuplicateField(p, ctxt, nodeFactory, fieldName, objectNode, oldValue, newValue);

        // Merge the results into a common arrayNode
        // Note, this assumes that multiple values will combine into an array...
        // And *THAT* array will be used for future nodes to be tacked onto...
        // But if the FIRST thing to combine *IS* an array, then it will be the
        // initial array...
        // You could probably persist some way to track whether the initial array was encountered, but don't want to think about that now..
        ArrayNode asArrayValue = null;

        if (oldValue.isArray()) {
            asArrayValue = (ArrayNode) oldValue;
        } else {
            // If not, create as array for replacement and add initial value..
            asArrayValue = nodeFactory.arrayNode();
            asArrayValue.add(oldValue);
        }

        asArrayValue.add(newValue);
        objectNode.set(fieldName, asArrayValue);

    }

}

这篇关于杰克逊@JsonAnySetter与杰克逊ObjectMapper treeToValue方法一起使用时会忽略重复键的值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！