使用Wget跳过成功下载的文件 [英] Skip successfully downloaded files using Wget

问题描述

所以我有这个Bash子程序使用wget下载文件，现在我的问题是如何跳过成功下载的文件。该脚本下载了大量文件，一旦下载失败，它会从头开始重新下载覆盖成功下载的所有文件（由于重新下载可能不完整）。

So I have this Bash subroutine to download files using wget and my problem now is how to skip successfully downloaded files. The script downloads a lot of files and once the download fails, it re-downloads all files overwriting those successful downloads from the start (which may be incomplete because of the re-download).

那么如何跳过这些文件成功下载？

So how do I skip those files downloaded successfully?

DownloadFile() {
  paramURL=$1
  paramFilename=$2

  if [ $flag_archive_fetch = "false"  ];
  then
      wget "--timeout=180" "--tries=5" "$paramURL" "-O" "${scratch_dir}$paramFilename"
  else
      unzip -o "$archive_file" "$paramFilename" -d "${scratch_dir}"
  fi

  touch "${scratch_dir}$paramFilename"
}

推荐答案

您可以利用Wget / code>（恢复破碎的下载）和 - -timestamping （仅当上一次修改的属性已更改时覆盖成功下载的文件，否则跳过下载）

You can take advantage of Wget --continue (to resume broken downloads) and --timestamping (to overwrite successfully downloaded files only when Last-modified attribute has changed, otherwise skips the download)

wget "--continue ‐‐timestamping --timeout=180" "--tries=5" "$paramURL" "-O" "${scratch_dir}$paramFilename"

另一个选项是使用 - 否-clobber 而不是 - timestamp ，它会跳过已经下载的文件，而不检查修改属性，

another option is to use --no-clobber instead of --timestamping, it skips the already downloaded files without checking the Last-modified attribute,

 wget "--continue ‐‐no-clobber --timeout=180" "--tries=5" "$paramURL" "-O" "${scratch_dir}$paramFilename"

这篇关于使用Wget跳过成功下载的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！