调试u-sql作业 [英] Debugging u-sql Jobs

问题描述

我想知道是否有任何技巧和窍门在数据湖分析工作中发现错误.该错误消息在大多数情况下似乎不太详细.

I would like to know if there are any tips and tricks to find error in data lake analytics jobs. The error message seems most of the time to be not very detailed.

当尝试从CSV文件中提取内容时，我经常会收到这样的错误

When trying to extract from CSV file I often get error like this

顶点故障触发了快速作业中止.顶点失败:SV1_Extract [0]，错误>:顶点用户代码错误.

Vertex failure triggered quick job abort. Vertex failed: SV1_Extract[0] with >error: Vertex user code error.

顶点失败，并显示快速失败错误

Vertex failed with a fail-fast error

当尝试将列转换为指定类型时，似乎会出现这些错误.

It seems that these error occur when trying to convert the columns to specified types.

我发现的技术是将所有列提取为字符串，然后执行SELECT尝试将列转换为期望的类型.逐列进行操作可以帮助找到错误的特定列.

The technique I found is to extract all columns to string and then do a SELECT that will try to convert the columns to the expected type. Doing that columns by columns can help find the specific column in error.

@data =
    EXTRACT ClientID string,
            SendID string,
            FromName string,           
    FROM "wasb://..."
    USING Extractors.Csv();

//convert some columns to INT, condition to skip header
@clean =
    SELECT Int32.Parse(ClientID) AS ClientID,
           Int32.Parse(SendID) AS SendID,
           FromName,           
    FROM @data
    WHERE !ClientID.StartsWith("ClientID");

在解析错误的情况下，是否还可以使用TryParse之类的方法返回null或默认值，而不是整个作业失败?

Is it also possible to use something like a TryParse to return null or default values in case of a parsing error, instead of the whole job failing?

谢谢

推荐答案

这是一种无需在背后使用代码的解决方案(尽管Codebehind可以使您的代码更具可读性):

Here is a solution without having to use code behind (although Codebehind will make your code a bit more readable):

SELECT ((Func<string, Int32?>)(v => { Int32 res; return Int32.TryParse(v, out res)? (Int32?) res : (Int32?) null; }))(ClientID) AS ClientID

此外，您看到的关于错误消息是神秘的问题与一个错误有关，该错误应在返回所谓的内部错误消息时尽快得到修复.今天的工作是执行以下操作:

Also, the problem you see regarding error message being cryptic has to do with a bug that should be fixed soon in returning so called inner error messages. The work around today is to do the following:

1. 在Visual Studio的ADL工具中，打开失败作业的作业视图.
2. 在左下角，单击作业详细信息区域中的资源"链接.
3. 一旦加载了作业资源，请单击配置文件".
4. 在该行的开头搜索字符串"jobError".复制整行文本，然后粘贴到记事本(或其他文本编辑器)中，以读取实际错误.

这应该为您提供确切的错误消息.

That should give you the exact error message.

这篇关于调试u-sql作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！