如何从Azure存储下载blob文件并使用Powershell将其保存到FTP服务器？ [英] How to download a blob file from Azure Storage and save it to an FTP server using Powershell?

问题描述

我试图访问PowerShell中的blob文件，并希望将其直接保存到FTP服务器。如何才能做到这一点？这可以通过blob文件URL完成吗？或者我可以以某种方式在Powershell内存中创建文件，然后使用$ webclient.UploadFile将其保存到FTP文件夹？

与此相同下载相关的另一个问题是有一种方法可以复制文件而不是复制子目录？例如，我有一个blob文件，如：dataload / files / my_blob_file。当我使用命令Get-AzureStorageBlobContent -Destination $ destination_path时，它将文件保存在相同的子文件夹结构中，但是我可以使用自定义路径还是删除子文件夹并将其另存为c：\myfolder\my_blob_file而不是c： \myfolder\dataload\files\my_blob_file？我想在上面的FTP服务器上完成这项工作。

解决方案

blob文件，并希望将其直接保存到FTP服务器。

如果您的FTP服务器是基于Windows的，那么您可以运行FTP服务器上的脚本并将BLOB下载到本FTP服务器的本地路径中。

这可以通过blob文件完成URL？

命令Get-AzureStorageBlobContent不接受URL作为参数。这意味着你需要编写代码或脚本来实现这一点。这是我写的一个简单的演示：

  $ url =https://accountname.blob.core.windows.net /testcontainer/Test01.txt
 $ separator =：//
 $ option = [System.StringSplitOptions] :: RemoveEmptyEntries 
 $ temp = $ url.Split（$ separator，4 ，$ option）
 $ Protocol = $ temp [0] 
 $ HostName = $ temp [1] 
 $ Container = $ temp [2] 
 $ BlobName = $ temp [ 3] 
  

或者我可以以某种方式在Powershell内存中创建文件并然后使用$ webclient.UploadFile将它保存到FTP文件夹？

将文件存入RAM并不是一个好理想，即使我们可以做到这一点。如上所述，如果您的FTP服务器是基于Windows的，请直接在FTP服务器上运行脚本。

如果脚本因任何原因无法在服务器上运行，请尝试共享FTP服务使用的文件夹，并将其映射为网络驱动程序将运行该脚本的计算机。因此，您可以将文件存储到此网络驱动程序中。

但是，我可以使用自定义路径还是删除子文件夹将它保存为c：\myfolder\my_blob_file而不是c：\myfolder\dataload\files\my_blob_file？

当然，只需指定路径和文件名称作为目的地参数：

注意：实际上，Azure存储上没有文件夹的概念。路径是blob名称的一部分。在下载blob时，可以通过在目标路径中指定文件名来重命名blob。这样就不会在本地文件夹中创建额外的文件夹。

====================== ================================================== =

更新：

此脚本将作为Azure运行的一部分的Azure自动化。但是当我尝试调用FTP服务器（当前是本地计算机）时，出现无法连接到远程服务器错误。

您可能需要混合Runbook Worker 来实现你的目标。

Azure自动化中的Runbook无法访问本地数据中心中的资源，因为它们在Azure云中运行。 Azure Automation的混合Runbook Worker功能允许您在位于数据中心的计算机上运行Runbook，以管理本地资源。

我是使用默认的21端口，我也尝试使用我的公共IP地址

不推荐将您的FTP服务器公开到Internet。我建议使用Hybrid Runbook Worker。

另外，如何将我的blob文件的内容放入Powershell变量中以使用它在脚本中？

据我所知， Get-Azure存储Blob内容不支持在RAM中返回对象。您需要先下载内容，然后使用Get-Content获取文件内容。如果您使用Hybrid Runbook Worker，则可以在本地存储文件。

================ ================================================== ============
更新2：

我想了解如何调用任何外部FTP服务器（目前在我的机器上用于开发/测试目的，但可能驻留在生产中的任何其他外部服务器上），并且我必须从Azure自动化中的Powershell脚本运行它。所以你的回复：你可能需要Hybrid Runbook Worker来实现你的目标...对我来说不适合？

Hybrid Runbook Worker适合您。它使事情变得更容易。因为如果您使用Hybrid Runbook Worker，Runbook将在您的本地计算机上运行。

我可以将斑点下载到本地机器并上传到公众FTP服务器没有任何问题。

您是否说现在无法从Azure Powershell Automation的外部FTP服务器上载和下载文件？

我没有成功将blob上传到公共FTP服务器。当我尝试上传blob时发生异常，只有带有blob名称的空文件才会上传到FTP服务器。自从PowerShell脚本在沙箱中运行以来，这可能是一个权限问题。这就是为什么我说混合Runbook Worker使事情变得更简单的原因。最后，请注意：FTP最终会验证用户并以明文形式传输日期，这可能会导致安全问题。 FTPS和SFTP比FTP更安全。

I am trying to access a blob file in PowerShell and want to save it directly to an FTP server. How can this be done? Can this be done via the blob file URL? Or can I somehow have the file created in Powershell memory and then use $webclient.UploadFile to save it to FTP folder?

Another question related to this same download is that is there a way to just copy the file instead of getting the subdirectories copied as well? For example, I have a blob file like: dataload/files/my_blob_file. When I use the command Get-AzureStorageBlobContent -Destination $destination_path it saves the file in the same subfolder structure, but can I instead have a custom path or remove the subfolders and save it as c:\myfolder\my_blob_file instead of c:\myfolder\dataload\files\my_blob_file? I would want to accomplish this in the above FTP server.

解决方案

I am trying to access a blob file in PowerShell and want to save it directly to an FTP server.

If your FTP server is Windows based, then you can just run the script on the FTP server and download the blob into the local path of this FTP server.

Can this be done via the blob file URL?

The command "Get-AzureStorageBlobContent" doesn't accept URL as parameter. That means you need to write the code or script to achieve that. Here is a simple demo written by me:

$url = "https://accountname.blob.core.windows.net/testcontainer/Test01.txt"
$separator = "://"
$option = [System.StringSplitOptions]::RemoveEmptyEntries
$temp = $url.Split($separator,4,$option)
$Protocol = $temp[0]
$HostName = $temp[1]
$Container = $temp[2]
$BlobName = $temp[3]

Or can I somehow have the file created in Powershell memory and then use $webclient.UploadFile to save it to FTP folder?

Storing a file into the RAM is not a good ideal even if we can achieve that. As I have mentioned above, if your FTP server is Windows based, please run the script on the FTP server directly.

If the script can't be run on the server for any reason, then please try to share the folder used by the FTP service and map it as a network driver on the computer which will run the script. So that you will be able to store the file into this network driver.

but can I instead have a custom path or remove the subfolders and save it as c:\myfolder\my_blob_file instead of c:\myfolder\dataload\files\my_blob_file?

Of course, just specify the path and file name as the Destination parameter:

Note: Actually, there is no concept of "folder" on Azure storage. The path is a part of the blob name. When you download the blob, you can rename the blob by specify the file name in the destination path. So that the additional folder will not be created on local folder.

=========================================================================

Update:

This script is to be run from Azure as part of Azure Automation. But when I try to call the FTP server (which is currently my local machine) I get "Unable to connect to remote server" error.

You may need Hybrid Runbook Worker to achieve your goal.

Runbooks in Azure Automation cannot access resources in your local data center since they run in the Azure cloud. The Hybrid Runbook Worker feature of Azure Automation allows you to run runbooks on machines located in your data center to manage local resources.

I'm using the default 21 port and I also tried using my public IP address

Exposing your FTP server to the Internet is not recommended. I would suggest using Hybrid Runbook Worker.

Also, how can I get the content of my blob file into a Powershell variable to work with it in the script?

To my knowledge, Get-​Azure​Storage​Blob​Content does not support return an object in RAM. You need downloading the content first, then use Get-Content to get the file content. If you use the Hybrid Runbook Worker, you'll be able to store the file locally.

============================================================================== Update 2:

I am trying to understand as to how to call any external FTP server (which is currently on my machine for dev/test purpose, but may reside on any other external server in production), and I have to run it from a Powershell script in Azure Automation. So your reply: You may need Hybrid Runbook Worker to achieve your goal... will not work for me right?

The Hybrid Runbook Worker works for you. And it makes things easier. Because if you use Hybrid Runbook Worker, the Runbook is running on your local machine.

I'm able to download the blobs into my local machine and upload them to the public FTP server without any issue.

Are you saying that currently there is no way to upload and download files from external FTP server from Azure Powershell Automation?

I didn't successfully upload the blob to the public FTP server. Exception occurs when I try to upload the blob and only empty files with the name of the blob are uploaded to the FTP server. It might be a permission issue since the PowerShell script is running in a sandbox. That's the reason why I said that Hybrid Runbook Worker makes things easier.

In the end, please note: FTP authenticates users and transfers date in plaintext, which may cause security issue. FTPS and SFTP are more secure than FTP.

这篇关于如何从Azure存储下载blob文件并使用Powershell将其保存到FTP服务器？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！