数据工厂-从REST复制到表存储-PartitionKey为null [英] Data Factory - Copy from REST to Table Storage - PartitionKey null

问题描述

美好的一天，

我们正在Data Factory v2中通过REST导入以下JSON格式:

We are importing the following JSON format via REST in Data Factory v2:

[
    {
        " ERROR" ;:否，
        " USERNAME" ;:" DUMMY"
    }， 
    [
        {
            "A":1，
            "B":2，
            "C":3，
        }， 
        {
            "A":1，
            "B":0，
            "C":3，
        }   

[
    {
        "ERROR": false, 
        "USERNAME": "DUMMY"
    }, 
    [
        {
            "A": 1, 
            "B": 2, 
            "C": 3, 
        }, 
        {
            "A": 1, 
            "B": 0, 
            "C": 3, 
        }   

     ]
]

     ]
]

分区键值:USE SOURCE COLUMN

Partition key value: USE SOURCE COLUMN

分区键列:C

映射对于A B和C是正确的.ERROR和USERNAME是-不包括-.

Mapping is properly se for A B and C. ERROR and USERNAME are -- Not Included --.

在调试中给出以下错误:

Gives the following error in debug:

"errorCode":"2200"，"message":"Failure发生在水槽"一侧. ErrorCode = UserErrorAzureTableKeyColumnWithNullValue，'Type = Microsoft.DataTransfer.Common.Shared.HybridDeliveryException，Message =列 'C'包含'NULL'值，因此不能用作Azure Table的'PartitionKey'.

"errorCode": "2200", "message": "Failure happened on 'Sink' side. ErrorCode=UserErrorAzureTableKeyColumnWithNullValue,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Column 'C' contains 'NULL' value thus cannot be used as 'PartitionKey' for Azure Table.

我了解为什么会发生错误，COPY DATA工具尝试创建三行:

I understand why the error happens, the COPY DATA tool tries to create three rows:

第一行基于ERROR和USERNAME.该行的C值的确为空

First row is based on ERROR and USERNAME. This row has indeed value null for C

第二行基于A，B和C.由于C没有空值，因此此行是正确的.

Second row is based on A, B and C. This row is correct as there are no null values for C.

第三行基于A，B和C.由于C没有空值，因此此行是正确的.

Third row is based on A, B and C. This row is correct as there are no null values for C.

我们从上面知道，因为它与固定的分区键值(例如值:test)一起使用

We know above because it works with fixed Partition Key value (for example value: test)

我们无法更改输入格式.我们可以在Data Factory中做什么来解决此问题?如何忽略带有ERROR和USER的第一个数组，或者如果它为null，如何使用分区键跳过行?

We can't change the format of the input. What can we do in Data Factory to solve this issue? How can we disregard first array with ERROR and USER, or how can we skip rows with Partition Key if null?

推荐答案

您是否尝试过使用自定义活动?您可以使用它来解析REST API响应并格式化它，然后再将其存储到表存储中.这是有关自定义活动的文档:

Have you tried using a custom activity ? You can use it to parse your REST API response and format it before storing it to table storage. Here's the doc on custom activities :

https://docs.microsoft.com/zh-CN/azure/data-factory/v1/data-factory-use-custom-activities

https://docs.microsoft.com/en-us/azure/data-factory/v1/data-factory-use-custom-activities

让我们知道这是否有帮助.否则我们可以很高兴地继续对话.

Let us know if this helps. Else we can gladly continue the dialogue.

这篇关于数据工厂-从REST复制到表存储-PartitionKey为null的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！