将数据从OData加载到Azure数据仓库第2代时出现ADFV2转换问题 [英] ADFV2 Conversion issue when loading data from OData to Azure data warehouse Gen 2
问题描述
有人可以帮助解决我遇到的错误吗?我正在将大量数据从odata(大约270个表)移至Data Lake，然后移至Data Warehouse；从OData到DL运行成功，但是从DL到数据仓库，我遇到了一些错误 我正在加载的表.某些大多数失败的表的列上的数据包含HTML标记.
Can someone please help with the error i am getting, i am moving loads of data from odata (approx. 270 tables) to Data Lake then Data Warehouse; From OData to DL runs successful but from DL to Data Warehouse i am getting a couple of errors regarding some of the tables i am loading. The data on the columns on some tables that are mostly failing contain HTML tags.
我的问题是我如何使数据仓库接收这些HTML标记并将它们存储在表列中而不会出现错误.
My question is how do i make Data Warehouse to receive these HTML tags and store them in table columns without getting errors.
{"errorCode":"2200"，"message":"ErrorCode = FailedDbOperation，'类型= Microsoft.DataTransfer.Common.Shared.HybridDeliveryException，消息=错误 将数据加载到SQL数据仓库中时发生.，Source = Microsoft.DataTransfer.ClientLibrary，''Type = System.Data.SqlClient.SqlException，Message =查询已中止-从数据库读取数据时达到最大拒绝阈值(0行)外部来源:1行 在已处理的846行总行中拒绝. style = \"margin:0cm 0cm 0pt; line-height:normal; \>< span style = \" font-size:10pt;字体家族:"Arial"，"sans-serif"； mso-fareast-font-family:"Times New Roman"； mso-ansi语言:EN-US； mso-远东语言:EN-ZA；包括": < p style = \"margin:0cm 0cm 0pt 36pt;文字缩进:-18pt；行高:正常； mso-add-space:自动； mso-list:l0 level1 lfo1; \>< span style = \" font-size:10pt;字体家族:"Arial"，"sans-serif"； mso-fareast-font-family:Arial； mso-ansi语言: EN-US; mso-fareast语言:EN-ZA; \>< span style = \" mso-list:Ignore; \>-< span style = \" font:7pt'Times New Roman '; \'>& nbsp;& nbsp;& nbsp;& nbsp;& nbsp; </span>/span</span>< span style = \" font-size:10pt;字体家族:"Arial"，"sans-serif"； mso-fareast -...，Source = .Net SqlClient数据提供程序，SqlErrorNumber = 107090，Class = 16，ErrorCode = -2146232060，State = 1，Errors = [{Class = 16，Number = 107090，State = 1，消息=查询中止-最大拒绝 从外部来源读取时达到了阈值(0行):在处理的846行总数中拒绝了1行.数据类型:VARCHAR(8000)整理SQL_Latin1_General_CP1_CI_AS， 令人反感的值:< p style = \"边距:0cm 0cm 0pt； line-height:normal; \>< span style = \" font-size:10pt;字体家族:"Arial"，"sans-serif"； mso-fareast-font-family:"Times New Roman"； mso-ansi语言:EN-US； mso-fareast语言:EN-ZA;>投诉 包括:</span>/p < p style = \"margin:0cm 0cm 0pt 36pt;文字缩进:-18pt；行高:正常； mso-add-space:自动； mso-list:l0 level1 lfo1; \>< span style = \" font-size:10pt;字体家族:"Arial"，"sans-serif"； mso-fareast-font-family: Arial; mso-ansi语言:EN-US； mso-fareast语言:EN-ZA; \>< span style = \" mso-list:Ignore; \>-< span style = \" font:7pt'Times New Roman '; \'>& nbsp;& nbsp;& nbsp;& nbsp;& nbsp; </span>/span</span>< span style = \" font-size:10pt;字体家族:"Arial"，"sans-serif"； mso-fareast -...，}，]，'"，"failureType":"UserError"，"target":"Copy Data2" }
{ "errorCode": "2200", "message": "ErrorCode=FailedDbOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error happened when loading data into SQL Data Warehouse.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Data.SqlClient.SqlException,Message=Query aborted-- the maximum reject threshold (0 rows) was reached while reading from an external source: 1 rows rejected out of total 846 rows processed.\n(/fe488927-a24b-44d9-80d0-fd84d1816a8d/Polybase/IMS_Incidents.txt)Column ordinal: 16, Expected data type: VARCHAR(8000) collate SQL_Latin1_General_CP1_CI_AS, Offending value: <p style=\"margin: 0cm 0cm 0pt; line-height: normal;\"><span style=\"font-size: 10pt; font-family: 'Arial','sans-serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-language: EN-ZA;\">COMPLAINTS include:</span></p> <p style=\"margin: 0cm 0cm 0pt 36pt; text-indent: -18pt; line-height: normal; mso-add-space: auto; mso-list: l0 level1 lfo1;\"><span style=\"font-size: 10pt; font-family: 'Arial','sans-serif'; mso-fareast-font-family: Arial; mso-ansi-language: EN-US; mso-fareast-language: EN-ZA;\"><span style=\"mso-list: Ignore;\">-<span style=\"font: 7pt 'Times New Roman';\"> </span></span></span><span style=\"font-size: 10pt; font-family: 'Arial','sans-serif'; mso-fareast-...,Source=.Net SqlClient Data Provider,SqlErrorNumber=107090,Class=16,ErrorCode=-2146232060,State=1,Errors=[{Class=16,Number=107090,State=1,Message=Query aborted-- the maximum reject threshold (0 rows) was reached while reading from an external source: 1 rows rejected out of total 846 rows processed.\n(/fe488927-a24b-44d9-80d0-fd84d1816a8d/Polybase/IMS_Incidents.txt)Column ordinal: 16, Expected data type: VARCHAR(8000) collate SQL_Latin1_General_CP1_CI_AS, Offending value: <p style=\"margin: 0cm 0cm 0pt; line-height: normal;\"><span style=\"font-size: 10pt; font-family: 'Arial','sans-serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-language: EN-ZA;\">COMPLAINTS include:</span></p> <p style=\"margin: 0cm 0cm 0pt 36pt; text-indent: -18pt; line-height: normal; mso-add-space: auto; mso-list: l0 level1 lfo1;\"><span style=\"font-size: 10pt; font-family: 'Arial','sans-serif'; mso-fareast-font-family: Arial; mso-ansi-language: EN-US; mso-fareast-language: EN-ZA;\"><span style=\"mso-list: Ignore;\">-<span style=\"font: 7pt 'Times New Roman';\"> </span></span></span><span style=\"font-size: 10pt; font-family: 'Arial','sans-serif'; mso-fareast-...,},],'", "failureType": "UserError", "target": "Copy Data2" }
推荐答案
这是因为您的源数据包含8000多个字符吗?从错误消息中，您的接收器列为
Is this because your source data contains more than 8000 characters? From the error message, your sink column is VARCHAR(8000).
If you want copy job just to ignore this row, you could set reject value to a positive number.
或者您可以使用查询将字符串截断为较小的大小.
Or you could use query to truncate your string to a smaller size.
这篇关于将数据从OData加载到Azure数据仓库第2代时出现ADFV2转换问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
