具有计划任务的AzCopy和日志文件的含义 [英] AzCopy with scheduled task and implications of journal file

问题描述

我将创建一个Windows计划任务，每隔3小时触发一次，使用AzCopy v8将文件从Azure托管磁盘复制到Azure文件容器（v10仍在预览中项目开始）。

I am going to create a Windows scheduled task that triggers every 3hours to copy files from an Azure managed disk to an Azure Files container using AzCopy v8 (v10 was still in preview when this project started).

我的计划是排除旧文件（"/ XO"标志），以便只复制新文件。

My plan is to exclude older files ("/XO" flag) so that only new files are copied.

我有些担心我的方法可能无法在3种特定情况下正常运行，我希望有人可以建议：

I have some concerns that my approach may not function correctly in 3 particular scenarios and I'm hoping someone can advise:

1）如果文件传输先前失败并且文件在Azure文件容器中不完整，那么当使用"/ XO"重新运行AzCopy时，是否会传输并完成此未完成/损坏的文件。当前一个日志文件存在时标记？

1) If a file transfer previously failed and a file is incomplete in the Azure Files container, will this incomplete/broken file be transferred and completed when AzCopy is rerun using the "/XO" flag when the previous journal file exists?

2）当前一个命令没有完成时重新运行AzCopy命令（即：存在日志文件）时，AzCopy将完成日志文件中的任何文件  和 发送
创建原始日志文件时不存在的任何新文件？

2) When rerunning an AzCopy command when a previous command did not complete (ie: a journal file exists) will AzCopy finish any files in the journal file and send any new files that didnt exist when the original journal file was created?

3）如果日志损坏或丢失（假设发生灾难并且托管日志的磁盘丢失），如果在以前的操作中发送到Azure文件容器的文件不完整，当使用"/ XO"标志重新执行AzCopy
时，是否总是会跳过这些文件？在我看来，AzCopy不会有传输失败的记录以及源文件的日期文件较旧AzCopy不会重新尝试/发送此文件？

3) If the journal becomes corrupt or missing (lets say a disaster strikes and the disk hosting the journal is lost) if there were incomplete files that got sent to the Azure File container in previous operations, will these files always be skipped when AzCopy is re-executed when using the "/XO" flag?  It seems to me AzCopy wont have a record of the failed transfer and since the file date of the source file is older AzCopy will not reattempt/send this file ?

推荐答案

延迟道歉！AzCopy V10将故障记录保存在
以及

Apologies for the delay! AzCopy V10 keeps failure logs in the in the %USERPROFILE\.azcopy folder on Windows and the

HOME \.azcopy 文件夹。脚本中的任何失败都会在那里报告。让我知道你在那里看到了什么

HOME\.azcopy folder on Mac and Linux. Any failure in the script would be reported there. Let me know what do you see there

每个日志（包含失败）的相应作业文件存储在名为"plans"的子文件夹中。可以重新运行与失败日志对应的作业。当前AzCopy V10文档的这一部分描述
工作流程 - https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10?toc=%2fazure%2fstorage ％2fblobs％2ftoc.json＃troubleshooting 。

只复制新文件。最佳的AzCopy V10方法。可能是作业文件有一些意识到你要做什么。你可以随时复制同一组文件并使用 - overwrite = false
标志以避免覆盖现有文件。

这篇关于具有计划任务的AzCopy和日志文件的含义的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！