Azure Web作业突然停止正常工作 [英] Azure Web Jobs suddenly stopped working properly

问题描述

我编写了一个名为"blobsUploader"的程序，该程序每晚晚上11点将csv文件上传到blob容器.

I wrote a program named "blobsUploader" that uploads a csv file to a blob container every night at 11pm.

每当一个新的csv文件到达blobs容器时，就会在名为"blobsAdressQueue"的队列中显示一条新消息，其中包含新blob(csv文件)的地址.

Whenever a new csv file arrives to the blobs container, a new message in a queue named "blobsAdressQueue" appears with the address of the new blob (csv file).

这将引起Web作业，该作业读取csv文件并将其所有数据存储在名为"myDataTable"的Azure表中.

This evokes the Web Job which reads the csv file and stores all it's data in Azure Table named "myDataTable".

整个过程运行良好，但是从过去一两个月突然开始，每天晚上上传新的csv时，Web Job进程都会出错，并且"blobsAddressQueue"消息将移至"blobsAddressQueue-poison"，表示消息已超过向应用程序的最大传递尝试次数.

The whole process worked great but suddenly from the past month or two, every night when a new csv is uploaded, there is an error with the Web Job process and the message from "blobsAddressQueue" moves to "blobsAddressQueue-poison" which means a message that has exceeded the maximum number of delivery attempts to the application.

我现在从2018年6月起上传了一个csv，可以肯定了. 但是，现在带有该Blob地址的消息位于"blobsAddressQueue-poison"中.

I uploaded now a csv from June 2018, that worked for sure. However, now the message with the address the this blob is in "blobsAddressQueue-poison".

当我查看日志时，我可以看到5个失败的呼叫:

When I check the logs I can see 5 failing calls:

当我进行其中一种尝试并打开切换输出"时，这就是我得到的: 这很奇怪，因为该文件是在2018年6月读取的！没有任何问题！从那时起，我没有更改代码或csv文件中的任何内容.

When I go into one of the attempts and open "Toggle Output", that is what I get: Which is very weird because this file was read a in June 2018!!! with no any problems! I did not change anything in my code or the csv file since then.

如果需要更多信息来回答问题，请告诉我.

If more information is needed in order to answer the problem, pls let me know.

推荐答案

此问题与webjob无关，但与您引用的CsvHelper库有关.我检查了源代码，发现当一个字段包含引号并且该字段未加引号(转义)时，该字段将被视为错误数据.

This issue is not related to webjob, but CsvHelper library you reference to. I've checked the source code and found a field will be considered as bad data when it contains quote and the field is not quoted (escaped).

源代码:

/// <summary>
/// Gets or sets the function that is called when bad field data is found. A field
/// has bad data if it contains a quote and the field is not quoted (escaped).
/// You can supply your own function to do other things like logging the issue
/// instead of throwing an exception.
/// Arguments: context
/// </summary>
Action<ReadingContext> BadDataFound { get; set; }

解决方案是

修改csv文件中的问题字段

Modify the problematic fields in csv file

或

通过将BadDataFound设置为null来忽略不良数据:

Ingnore the bad data by setting BadDataFound to null:

csv.Configuration.BadDataFound = null;

示例代码:

    static void Main(string[] args)
    {
        using (var reader = new StreamReader(@"C:\Users\toml\Desktop\test.csv"))
        using (var csv = new CsvReader(reader))
        {
            csv.Configuration.BadDataFound = null;
            var records = csv.GetRecords<Foo>();

            foreach(var item in records)
            {
                Console.WriteLine(item.Name);
            }
        }

        Console.ReadKey();
    }
}

public class Foo
{
    public int Id { get; set; }
    public string Name { get; set; }
}

无效的CSV样本:

    Id,Name
    1,one"
    2,two

有效的CSV示例:

    Id,Name
    1,"one""
    2,two

这篇关于Azure Web作业突然停止正常工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！