Azure CLI:bash脚本以并行上传文件 [英] Azure CLI : bash script to upload files in parallel

问题描述

是否有Azure CLI上传选项可将文件并行上传到Blob存储.有很多文件的文件夹.目前，我唯一的选择是使用以下命令进行for循环，并按顺序上传.

Is there a Azure CLI upload option to parallel upload files to blob storage. There is folder with lots of files. Currently the only option I have is do a for loop with below command and upload is sequentially.

az storage blob upload --file $f --container-name $CONTAINERNAME --name $FILEINFO

推荐答案

目前，这是不可能的.使用Azure CLI 2.0时，没有选项或参数可以将指定目录的内容递归上传到Blob存储.因此，Azure CLi 2.0不支持并行上传文件.

For now, it is not possible. With the Azure CLI 2.0 there is no option or argument to upload the contents of a specified directory to Blob storage recursively. So, Azure CLi 2.0 does not support upload files in parallel.

如果要并行上传多个文件，则可以使用 Azcopy .

If you want to upload multiple files in parallel, you could use Azcopy.

AzCopy /Source:C:\myfolder /Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key /S

指定选项/S将指定目录的内容递归上传到Blob存储，这意味着所有子文件夹及其文件也将被上传.

Specifying option /S uploads the contents of the specified directory to Blob storage recursively, meaning that all subfolders and their files will be uploaded as well.

如前所述，您可以使用循环上传文件，但是它不支持并行上传文件.尝试以下脚本.

As you mentioned, you could use loop to upload files, but it does not support upload files in parallel. Try following script.

export AZURE_STORAGE_ACCOUNT='PUT_YOUR_STORAGE_ACCOUNT_HERE'
export AZURE_STORAGE_ACCESS_KEY='PUT_YOUR_ACCESS_KEY_HERE'

export container_name='nyc-tlc-sf'
export source_folder='/Volumes/MacintoshDisk02/Data/Misc/NYC_TLC/yellow/2012/*'
export destination_folder='yellow/2012/'

#echo "Creating container..."
#azure storage container create $container_name

for f in $source_folder
do
  echo "Uploading $f file..."
  azure storage blob upload $f $container_name $destination_folder$(basename $f)
done

echo "List all blobs in container..."
azure storage blob list $container_name

echo "Completed"

这篇关于Azure CLI:bash脚本以并行上传文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！