将数据直接下载到Google云存储 [英] Download data directly to google cloud storage

问题描述

我想直接从 python应用程序/命令(例如:youtube-dl或从第三方网址下载的任何其他库)下载数据到谷歌云存储(桶).

I want to download data from python application/command (for eg: youtube-dl or any other library that download from 3rd party url) directly to google cloud storage(Bucket).

我已使用 gsutil 流命令直接将数据从流程流传输到gcs，但它仅将控制台输出保存到存储桶

I have used gsutil stream command to stream data directly from process to gcs, but it saves only console output to bucket

我也不想挂载存储，因为我想与分布式系统共享该存储

Also i don't want to mount storage because i want to share that storage with distributed system

有什么方法可以下载而不先下载文件系统，然后再将其复制到Google云端存储?

Is there any way in which i can download it without downloading on file system first and then copying it to google cloud storage ?

谢谢

推荐答案

您描述的情况似乎不可能:查看

The situation you are describing doesn't seem possible: looking at the documentation and source code for the Cloud Storage library in Python only leave you 3 options: upload from file (already in your disk), upload providing a filename (a path to a file already in your disk) and upload from string (upload a text as a .txt file).

您将需要从提到的任何程序中下载文件(如注释中所述，您可以将其下载到临时文件夹中)，然后将文件上传到GCS，然后从您的临时文件夹中删除.

You will need to download the file from whichever program you mention (as mentioned in the comments, you can download it to a temporal folder), upload the file to GCS and then delete it from your temporal folder.

这篇关于将数据直接下载到Google云存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！