在F＃中使用区分的联合反序列化数据的另一个失败 [英] Another failure at deserializing data with discriminated unions, in F#

问题描述

下面的问题中，答案为序列化/反序列化受歧视的联合提供了一个有效的解决方案（ IgnoreMissingMember设置似乎不适用于FSharpLu.Json解串器）

Following a question where the answer provided a working solution to serialize / deserialize discriminated unions (IgnoreMissingMember setting doesn't seem to work with FSharpLu.Json deserializer)

我现在有一个实用的在这种情况下会失败（尽管在更简单的情况下也可以）。

I have now a practical case where this fails (although it works in simpler cases).

这是测试代码：

open System.Collections.Generic
open Microsoft.FSharpLu.Json
open Newtonsoft.Json
open Newtonsoft.Json.Serialization

// set up the serialization / deserialization based on answer from:
// https://stackoverflow.com/questions/62364229/ignoremissingmember-setting-doesnt-seem-to-work-with-fsharplu-json-deserializer/62364913#62364913

let settings =
    JsonSerializerSettings(
        NullValueHandling = NullValueHandling.Ignore,
        Converters = [| CompactUnionJsonConverter(true, true) |]
    )

let serialize object =
    JsonConvert.SerializeObject(object, settings)

let deserialize<'a> object =
    JsonConvert.DeserializeObject<'a>(object, settings)


// define the type used
type BookSide =
    | Bid
    | Ask

type BookEntry =
    {
        S : float
        P : float
    }

type BookSideData =
    Dictionary<int, BookEntry>

type BookData =
    {
        Data: Dictionary<BookSide, BookSideData>
    }

    static member empty =
        {
            Data = Dictionary<BookSide, BookSideData>(dict [ (BookSide.Bid, BookSideData()); (BookSide.Ask, BookSideData()) ])
        }

// make some sample data
let bookEntry = { S=3.; P=5. }
let bookData = BookData.empty
bookData.Data.[BookSide.Bid].Add(1, bookEntry)

// serialize. This part works
let s = serialize bookData

// deserialize. This part fails
deserialize<BookData> s

序列化的测试数据如下：

the serialized test data will look like this:

{数据：{出价：{ 1：{ S：3.0， P：5.0}}，询问：{}}}

{"Data":{"Bid":{"1":{"S":3.0,"P":5.0}},"Ask":{}}}

，但是反序列化会像这样崩溃：

but deserializing will crash like this:

无法将字符串出价转换为字典键类型 FSI_0023 + BookSide。创建一个TypeConverter以从字符串转换为键类型对象。

Could not convert string 'Bid' to dictionary key type 'FSI_0023+BookSide'. Create a TypeConverter to convert from the string to the key type object.

尽管通过FSharpLu对DU进行了序列化/反序列化， DU转换器。

although the serialization / deserialization of the DU through FSharpLu which has a DU converter.

我试图找到一些自动化解决方案而不是编写自定义TypeConverter的原因（除了我从未做过的事实）是，我有一个

The reason I am trying to find some automated solution, vs writing a custom TypeConverter (besides the fact I've never done it) is that I have a lot of types I do not control to go through.

这里是一个小提琴：
https://dotnetfiddle.net/Sx0k4x

推荐答案

您的基本问题是您正在使用 BookSide 作为字典键-但这是一个f＃联合，使其成为复杂键 –无法立即转换为和从一个字符串。不幸的是，Json.NET不支持其序列化指南：

Your basic problem is that you are using BookSide as a dictionary key -- but this is an f# union which makes it a complex key -- one not immediately convertible to and from a string. Unfortunately Json.NET does not support complex dictionary keys out of the box as is stated in its Serialization Guide:

序列化字典时，字典的键将转换为字符串并用作JSON对象属性名称。可以通过覆盖键类型的 ToString（）或实现 TypeConverter 。 TypeConverter 还支持在反序列化字典时再次转换自定义字符串。

When serializing a dictionary, the keys of the dictionary are converted to strings and used as the JSON object property names. The string written for a key can be customized by either overriding ToString() for the key type or by implementing a TypeConverter. A TypeConverter will also support converting a custom string back again when deserializing a dictionary.

有两种基本方法可以解决此问题：

There are two basic approaches to handling this issue:

1. 实现 TypeConverter ，如 无法使用Json.net 序列化带有复杂密钥的字典 em>。

将字典序列化为键/值对对象数组，例如如 将字典序列化为（键值对的）数组 所示。

Serialize the dictionary as an array of key/value pair objects e.g. as is shown in Serialize dictionary as array (of key value pairs).

由于您的数据模型包含带有各种键（DU，字符串和整数）的字典解决方案似乎是唯一的可能性。以下 DictionaryConverter 应该具有必要的逻辑：

Since your data model includes dictionaries with a variety of keys (DU, strings and ints) the second solution would appear to be the only possibility. The following DictionaryConverter should have the necessary logic:

let inline isNull (x:^T when ^T : not struct) = obj.ReferenceEquals (x, null)

type Type with
    member t.BaseTypesAndSelf() =
        t |> Seq.unfold (fun state -> if isNull state then None else Some(state, state.BaseType))
    member t.DictionaryKeyValueTypes() = 
        t.BaseTypesAndSelf()
            |> Seq.filter (fun i -> i.IsGenericType && i.GetGenericTypeDefinition() = typedefof<Dictionary<_,_>>)
            |> Seq.map (fun i -> i.GetGenericArguments())

type JsonReader with
    member r.ReadAndAssert() = 
        if not (r.Read()) then raise (JsonReaderException("Unexpected end of JSON stream."))
        r
    member r.MoveToContentAndAssert() =
        if r.TokenType = JsonToken.None then r.ReadAndAssert() |> ignore
        while r.TokenType = JsonToken.Comment do r.ReadAndAssert() |> ignore
        r

type internal DictionaryReadOnlySurrogate<'TKey, 'TValue>(i : IDictionary<'TKey, 'TValue>) =
    interface IReadOnlyDictionary<'TKey, 'TValue> with
        member this.ContainsKey(key) = i.ContainsKey(key)
        member this.TryGetValue(key, value) = i.TryGetValue(key, &value)
        member this.Item with get(index) = i.[index]
        member this.Keys = i.Keys :> IEnumerable<'TKey>
        member this.Values = i.Values :> IEnumerable<'TValue>
        member this.Count = i.Count
        member this.GetEnumerator() = i.GetEnumerator()
        member this.GetEnumerator() = i.GetEnumerator() :> IEnumerator        

type DictionaryConverter () =
    // ReadJson adapted from this answer https://stackoverflow.com/a/28633769/3744182
    // To https://stackoverflow.com/questions/28451990/newtonsoft-json-deserialize-dictionary-as-key-value-list-from-datacontractjsonse
    // By https://stackoverflow.com/users/3744182/dbc
    inherit JsonConverter()

    override this.CanConvert(t) = (t.DictionaryKeyValueTypes().Count() = 1) // If ever implemented for IReadOnlyDictionary<'TKey, 'TValue> then reject DictionaryReadOnlySurrogate<'TKey, 'TValue>

    member private this.ReadJsonGeneric<'TKey, 'TValue> (reader : JsonReader, t : Type, existingValue : obj, serializer : JsonSerializer) : obj =
        let contract = serializer.ContractResolver.ResolveContract(t)
        let dict = if (existingValue :? IDictionary<'TKey, 'TValue>) then existingValue :?> IDictionary<'TKey, 'TValue> else contract.DefaultCreator.Invoke() :?> IDictionary<'TKey, 'TValue>
        match reader.MoveToContentAndAssert().TokenType with 
        | JsonToken.StartArray -> 
            let l = serializer.Deserialize<List<KeyValuePair<'TKey, 'TValue>>>(reader)
            for p in l do dict.Add(p) 
            dict :> obj
        | JsonToken.StartObject ->
            serializer.Populate(reader, dict)
            dict :> obj
        | JsonToken.Null -> null // Or throw an exception if you prefer
        | _ -> raise (JsonSerializationException(String.Format("Unexpected token {0}", reader.TokenType)))

    override this.ReadJson(reader, t, existingValue, serializer) = 
        let keyValueTypes = t.DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
        let m = typeof<DictionaryConverter>.GetMethod("ReadJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
        m.MakeGenericMethod(keyValueTypes).Invoke(this, [| reader; t; existingValue; serializer |])

    member private this.WriteJsonGeneric<'TKey, 'TValue> (writer : JsonWriter, value : obj, serializer : JsonSerializer) =
        let dict = value :?> IDictionary<'TKey, 'TValue>
        let keyContract = serializer.ContractResolver.ResolveContract(typeof<'Key>)
        // Wrap the value in an enumerator or read-only surrogate to prevent infinite recursion.
        match keyContract with
        | :? JsonPrimitiveContract -> serializer.Serialize(writer, new DictionaryReadOnlySurrogate<'TKey, 'TValue>(dict)) 
        | _ -> serializer.Serialize(writer, seq { yield! dict }) 
        ()

    override this.WriteJson(writer, value, serializer) = 
        let keyValueTypes = value.GetType().DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
        let m = typeof<DictionaryConverter>.GetMethod("WriteJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
        m.MakeGenericMethod(keyValueTypes).Invoke(this, [| writer; value; serializer |])
        ()

您将按如下所示添加到设置中：

Which you would add to settings as follows:

let settings =
    JsonSerializerSettings(
        NullValueHandling = NullValueHandling.Ignore,
        Converters = [| CompactUnionJsonConverter(true, true); DictionaryConverter() |]
    )

并为您的 bookData 生成以下JSON：

And generates the following JSON for your bookData:

{
  "Data": [
    {
      "Key": "Bid",
      "Value": [
        {
          "Key": 1,
          "Value": {
            "S": 3.0,
            "P": 5.0
          }
        }
      ]
    },
    {
      "Key": "Ask",
      "Value": []
    }
  ]
}

注意：

• 转换器适用于所有 Dictionary< TKey，TValue> 类型（和子类型）。

转换器检测是否将使用原始协定对字典键进行序列化，如果是，则将字典紧凑地序列化为JSON对象。如果不是，则将字典序列化为数组。您可以在上面显示的JSON中观察到这一点： Dictionary< BookSide，BookSideData> 字典被序列化为JSON数组，而 Dictionary< int， BookEntry> 字典被序列化为JSON对象。

The converter detects whether the dictionary keys will be serialized using a primitive contract, and if so, serializes the dictionary compactly as a JSON object. If not the dictionary is serialized as an array. You can observe this in the JSON shown above: the Dictionary<BookSide, BookSideData> dictionary is serialized as a JSON array, and the Dictionary<int, BookEntry> dictionary is serialized as a JSON object.

反序列化过程中，转换器将检测传入的JSON值是数组还是对象，并进行调整

During deserialization the converter detects whether the incoming JSON value is an array or object, and adapts as required.

仅针对可变的.Net Dictionary< TKey，TValue> 实现转换器。类型。逻辑上需要进行一些修改，以反序列化不可变的 Map<'Key，'Value> 类型。

The converter is only implemented for the mutable .Net Dictionary<TKey, TValue> type. The logic would require some slight modification to deserialize the immutable Map<'Key,'Value> type.

演示小提琴此处。

这篇关于在F＃中使用区分的联合反序列化数据的另一个失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！