并行下载 URL 列表和重命名 [英] Parallel download for a list of urls and renaming

问题描述

我有一个制表符分隔的网址列表和目标文件名，urls_to_download.txt，例如:

I have a list of tab-separated urls, and target file names, urls_to_download.txt, for example:

first_file.jpg\thttps://www.google.co.il/images/srpr/logo11w.png
/subdir_1/second_file.jpg\thttps://www.google.co.il/images/srpr/logo12w.png
...
last_file.jpg\thttps://www.google.co.il/images/srpr/logo99w.png

我想使用多个连接下载.

which I want to download using several connections.

我可以这样做，例如:

cat urls_to_download.txt |xargs -n 1 -P 10 wget -nc

我的问题是，如何让文件具有我想要的新名称，以便输出目录具有:

My question is, how do I get the files to have the new names I want for them, so the output dir would have:

first_file.jpg
/subdir1/second_file.jpg
...
last_file.jpg

推荐答案

我猜这样的事情应该适合你:

I am guessing that something like this should work for you:

#!/bin/bash                                                                                                                                                                       
while read FILENAME URL; do
  wget -nc -O "$FILENAME" "$URL"
done <input.txt

其中 input.txt 是一个包含制表符分隔的文件/url 对的文件，每行一个.

where input.txt is a file which contains tab separated file/url pairs, one per line.

这篇关于并行下载 URL 列表和重命名的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！