使用Java在Google云端存储上使用“签名的网址" *进行“可恢复的上传" [英] Using Java to do *resumable uploads* using a *signed url* on google cloud storage

问题描述

基于有关如何在google-cloud-storage中创建对象的文档(请参见

Based on the doc regarding how to create an object in google-cloud-storage (see "create" method in https://googleapis.github.io/google-cloud-java/google-cloud-clients/apidocs/index.html), we should be using the blob.writer(...) method when trying to upload large files, as it presumably somehow automatically handles resumable uploads. Is this right?

但是，如果我们希望在SIGNED网址上进行可恢复的上传，那么在Java中如何做到这一点?(任何示例代码或指针都将不胜感激；到目前为止，我的研究使我相信，不可能使用精美创建的Java库，而是需要使用"PUT"和"Java中的POST"语句生成签名网址后.到目前为止，这是最佳"方法吗?)

However, if we wish to do resumable uploads on SIGNED urls, how does one do so in Java? (Any sample code or pointers would be very much appreciated; thus far, my research has led me to believe that it is not possible using the nicely created java libraries, and instead one needs to custom build one's own logic using "PUT" and "POST" statements in Java after one has generated the signed url. Is this the "best" way so far?)

推荐答案

关于您的第一点，是的， blob.writer(...)方法可以自动处理可恢复的上载.不幸的是，该方法不能从已签名的URL调用，只能直接从字节流中上传文件.

Regarding your first point, yes, the blob.writer(...) method does automatically handle resumable uploads. Unfortunately, this method is not callable from a signed URL, and only uploads files directly from a stream of bytes.

但是，正如您提到的，可以使用其他方法从签名的URL创建可恢复的上载，例如，使用 PUT 方法似乎是一个不错的解决方法.

However, as you mentioned, it is possible to create a resumable upload from a signed URL with other methods, for instance using a PUT method seems to be a nice workaround.

我所做的是:

1. 创建一个签名的URL 使用"PUT"方法.为此，您可以指定 SignUrlOption ，同时我在存储桶中指定了具有所需权限的服务帐户.

1. Create a signed URL with a "PUT" method. You can do so by specifying the SignUrlOption, as well I specified a service account with the required permissions in the bucket.

使用 URLFetch 引发HTTP请求到此签名的URL.我相信例如您不能直接使用curl命令，而UrlFetch API可以解决问题.

Use URLFetch to throw an HTTP request to this signed URL. I believe that you cannot use a curl command directly for example, and the UrlFetch API does the trick.

将uploadType = resumable标头添加到 urlFetch HTTP请求中.有关如何操作，请参见本文档这项工作，以及额外的参数和信息.

Add the uploadType=resumable header to the urlFetch HTTP request. See this documentation on how this works, and extra parameters and information.

我配置了 URLFetch 来对签名的URL进行异步调用，因为我认为在上传大文件时更方便.

I configured URLFetch to do an asynchronous call to the signed URL, as I believe it's more convenient when uploading big files.

要在App Engine处理程序中使用的示例代码:

An example code, to be used in an App Engine handler:

package com.example.storage;

import java.io.IOException;
import java.io.FileInputStream;
import java.util.Properties;
import java.util.concurrent.TimeUnit;
import java.util.HashMap;
import java.util.Map;
import java.nio.charset.StandardCharsets;

import java.net.URL;

import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

// Cloud Storage Imports
import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import com.google.cloud.storage.Blob;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage.SignUrlOption;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.cloud.storage.HttpMethod;

// Url Fetch imports
import com.google.appengine.api.urlfetch.HTTPMethod;
import com.google.appengine.api.urlfetch.HTTPRequest;
import com.google.appengine.api.urlfetch.URLFetchService;
import com.google.appengine.api.urlfetch.URLFetchServiceFactory;
import com.google.appengine.api.urlfetch.HTTPHeader;

@WebServlet(name = "MainStorage", value = "/")
public class MainStorage extends HttpServlet {

        @Override
        public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException {
                // Bucket parameters 
                String bucketName = "MY-BUCKET-NAME";
                String blobName = "MY-BLOB-NAME";
                String keyPath = "/PATH-TO-SERVICE-ACCOUNT-KEY/key.json";

                BlobId blobId = BlobId.of(bucketName, blobName);
                Storage storage = StorageOptions.getDefaultInstance().getService();

                // Create signed URL with SignUrlOptions
                URL signedUrl = storage.signUrl(BlobInfo.newBuilder(bucketName, blobName).build(), 14, TimeUnit.DAYS,
                                                SignUrlOption.signWith(ServiceAccountCredentials.fromStream(new FileInputStream(keyPath))),
                                                SignUrlOption.httpMethod(HttpMethod.PUT));

                // Contents to upload to the Blob
                String content = "My-File-contents";

                // Build UrlFetch request
                HTTPRequest upload_request = new HTTPRequest(signedUrl, HTTPMethod.PUT);
                upload_request.setPayload(content.getBytes(StandardCharsets.UTF_8));

                // Set request to have an uploadType=resumable
                HTTPHeader set_resumable = new HTTPHeader("uploadType", "resumable");
                upload_request.setHeader(set_resumable);
                URLFetchService fetcher = URLFetchServiceFactory.getURLFetchService();

                // Do an asynchronous call to the signed URL with the contents
                fetcher.fetchAsync(upload_request);

                // Return response to App Engine handler call
                response.setContentType("text/plain");
                response.getWriter().println("Hello Storage");
        }
}

这也许可以用更好的方法来完成，但是我相信它为如何制作这样的应用提供了一个思路.

This can probably be done in a better way, but I believe it gives an idea on how such application can be made.

这篇关于使用Java在Google云端存储上使用“签名的网址" *进行“可恢复的上传"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！