java google drive api V3 MultiPart &可恢复上传 [英] java google drive api V3 MultiPart & Resumable Upload

问题描述

我需要帮助编写 multipart &可恢复上传大文件(> 5MB) 到目前为止，我只能开始多次上传上传，但我不知道如何在用户暂停或网络故障时恢复它.

I need help in writing multipart & resumable upload of large files (>5MB) so far I have only been able to start an multiupload upload but I don't know how to resume it either when the user pauses it or during network failures.

恢复"是指我不知道如何

By "Resuming" I mean I don't know how to

1) 获取已上传到驱动器的总字节数

1) get the total bytes already uploaded to the drive

2) 如何在 Content-Range Header 中使用该值

2) how to use that value in the Content-Range Header

3) 如何通过用户交互甚至暂停此上传[executeAsInputStream() 也许?]

3) how to even pause this upload by user interaction[executeAsInputStream() Maybe?]

这是我目前所做的.即使我强行停止应用程序并重新启动它，我也需要代码从它停止上传的位置恢复

This is what I have done so far. I need the code to resume from where it stopped uploading even if I were to forcefully stop the application and restart it

   Drive service = GDrive.getService(); //Drive Specific Initialization Copied From QuickStart But with DriveScopes.FILES

   File fileMetadata = new File();
   fileMetadata.setName("Video.mp4"); //Video File
   fileMetadata.setMimeType("application/vnd.google-apps.video");

   java.io.File filePath = new java.io.File("E:\\large-file-60MB.mp4");//Large File Of 60 Mega Bytes
   FileContent mediaContent = new FileContent("video/mp4",filePath);

   Drive.Files.Create create=service.files().create(fileMetadata,mediaContent);

   MediaHttpUploader uploader=create.getMediaHttpUploader();
   uploader.setDirectUploadEnabled(false);                       //Use Resumable MultiPart Upload Protocol
   uploader.setChunkSize(2*MediaHttpUploader.MINIMUM_CHUNK_SIZE); //Chunks Of Bytes To Upload With Each Request

  // HttpHeaders headers=new HttpHeaders();
  // headers.put("Content-Range",?);          //This is not actual code which I used here but after reading the drive docs they talk about this header and I am not sure how or when to use it
  // uploader.setInitiationHeaders(headers);

   uploader.setProgressListener((uploading)->
   {
    switch (uploading.getUploadState())
    {
      case INITIATION_STARTED:System.out.println("Initiation has started!");
      break;
      case INITIATION_COMPLETE:System.out.println("Initiation is complete!");
      break;
      case MEDIA_IN_PROGRESS:
      System.out.println("Progress="+uploading.getProgress());
      System.out.println("Bytes="+uploading.getNumBytesUploaded());
      break;
      case MEDIA_COMPLETE:System.out.println("Upload is complete!");
    }
   });

   create.execute(); 

推荐答案

虽然在一个答案中回答多个问题通常不适合 Stackoverflow，但似乎这些都是紧密相关的，因此将概述可恢复的上传和这样做是为了解决您的三点:

While answering multiple questions in one answer isn't normally appropriate for Stackoverflow, it seems that these are all closely linked and so will give an overview of resumable uploads and in doing so attempt to address your three points:

• 如何获取已上传到云端硬盘的总字节数
• 如何使用 Content-Range 标题中的值
• 如何暂停可恢复上传

<小时>

来自 Google 关于 Direct 的文档和可恢复媒体上传在 Java API 客户端库文档中:

---

From Google's documentation on Direct and Resumable Media Uploads on ther Java API Client Library documentation:

感兴趣的主要类是 MediaHttpUploader 和 MediaHttpProgressListener.

Implementation details

The main classes of interest are MediaHttpUploader and MediaHttpProgressListener.

如果特定于服务的生成库中的方法在 Discovery 文档，然后为这些方法创建了一个方便的方法，将 InputStreamContent 作为参数.

If methods in the service-specific generated libraries contain the mediaUpload parameter in the Discovery document, then a convenience method is created for these methods that takes an InputStreamContent as a parameter.

例如Drive API的insert方法支持mediaUpload，可以使用如下代码上传文件:

For example, the insert method of the Drive API supports mediaUpload, and you can use the following code to upload a file:

class CustomProgressListener implements MediaHttpUploaderProgressListener {
  public void progressChanged(MediaHttpUploader uploader) throws IOException {
    switch (uploader.getUploadState()) {
      case INITIATION_STARTED:
        System.out.println("Initiation has started!");
        break;
      case INITIATION_COMPLETE:
        System.out.println("Initiation is complete!");
        break;
      case MEDIA_IN_PROGRESS:
        System.out.println(uploader.getProgress());
        break;
      case MEDIA_COMPLETE:
        System.out.println("Upload is complete!");
    }
  }
}

File mediaFile = new File("/tmp/driveFile.jpg");
InputStreamContent mediaContent =
    new InputStreamContent("image/jpeg",
        new BufferedInputStream(new FileInputStream(mediaFile)));
mediaContent.setLength(mediaFile.length());

Drive.Files.Insert request = drive.files().insert(fileMetadata, mediaContent);
request.getMediaHttpUploader().setProgressListener(new CustomProgressListener());
request.execute();

然而，这些类抽象出诸如创建可恢复上传时返回的位置 URI 之类的内容，因此如果您希望能够执行此操作，那么您将需要按照记录的可恢复上传启动步骤进行操作 此处.然而，这一切都是手动完成的，而不是直接使用 Google Drive API 客户端库.

These classes however abstract away things like the location URI which is returned when creating the resumable upload, so if you want to be able to do this then you will need to follow the resumable upload initiation steps as documented here. This is all manually done however rather than directly using the Google Drive API Client Library.

要回答第一点，您如何存储已上传的字节数取决于您.与其想云端硬盘上已经有多少内容"，不如想我已经上传了多少内容?".

To answer the first point, how you store how many bytes have been uploaded is up to you. Rather than thinking "how much is already on Drive", think "how much have I already uploaded?".

如果您愿意，您可以将其存储为本地变量，因为它将是您的块大小的倍数(在您的情况下为 2 * MediaHttpUploader.MINIMUM_CHUNK_SIZE)并且应该易于跟踪.

You can store this as a vairable locally if you so desire, as it will be a multiple of your chunk size (2 * MediaHttpUploader.MINIMUM_CHUNK_SIZE in your case) and should be easy to track.

问题是，这实际上并不需要.根据文档(强调我自己的)，您可以仅使用通配符来指示文件的当前位置未知:

The thing is, this isn't actually needed. You can just use a wildcard to indicate that the current position of your file is unknown, as per the documentation (emphasis my own):

如果上传请求在响应之前终止，或者您收到 503 Service Unavailable 响应，则您需要恢复中断的上传.

If an upload request is terminated before a response, or if you receive a 503 Service Unavailable response, then you need to resume the interrupted upload.

要请求上传状态，请向可恢复会话 URI 创建一个空的 PUT 请求.

To request the upload status, create an empty PUT request to the resumable session URI.

添加 Content-Range 标头以指示文件中的当前位置未知.例如，如果您的文件总长度为 2,000,000 字节，请将 Content-Range 设置为 */2000000. 如果您不知道文件的完整大小，将 Content-Range 设置为 */*.

Add a Content-Range header to indicate that the current position in the file is unknown. For example, set the Content-Range to */2000000 if your total file length is 2,000,000 bytes. If you don't know the full size of the file, set the Content-Range to */*.

如果您确实想跟踪字节，可以在 Content-Range 标头中将其指定为

If you do want to keep track of the bytes, you can specify it in your Content-Range header as

Content-Range: bytes_so_far/total_bytes

步骤:

要初始化可恢复上传，您需要向 Drive API 的 /upload 端点发出 POST 请求.为此，您不需要使用 Drive API 客户端库(实际上，如果您想获得可恢复的会话 URI，则不能这样做，因为客户端库没有为您提供此功能).

Steps:

To initialise the resumable upload, you need to make a POST request to the /upload endpoint of the Drive API. You do not need to use the Drive API client library for this (and actually if you want to get the resumable session URI, you can't as the client library doesn't give you this).

假设您的凭证定义来自:

Assuming you have your credential definition from:

GoogleAccountCredential credential = GoogleAccountCredential.usingOAuth2(...);

然后发出包含文件元数据的 POST 请求:

Then make the POST request containing the file metadata:

URL requestUrl = new URL("https://www.googleapis.com/upload/drive/v3/files?uploadType=resumable");

String requestBody = "{\"name\": \"fileName\"}";

HttpURLConnection request = (HttpURLConnection) requestUrl.openConnection();

request.setRequestMethod("POST");
request.setDoInput(true);
request.setDoOutput(true);
request.setRequestProperty("Authorization", "Bearer " + credential.getToken());
request.setRequestProperty("X-Upload-Content-Type", "file/mimetype");
request.setRequestProperty("X-Upload-Content-Length", number_of_bytes_of_your_file);
request.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
request.setRequestProperty("Content-Length", String.format(Locale.ENGLISH, "%d", requestBody.getBytes().length));

OutputStream outputStream = request.getOutputStream();
outputStream.write(requestBody.getBytes());
outputStream.close();

request.connect();

会话 URI - 在您需要时调用恢复的位置 - 在 API 响应的标头中返回.连接后，您可以从响应中获取此 URI:

The session URI - where to call to resume in case you need to - is returned in the headers of the response from the API. After connecting you can get this URI from the response:

if (request.getResponseCode() == HttpURLConnection.HTTP_OK) {
    URL sessionUri = new URL(request.getHeaderField("location"));
}

现在您有了会话 URI - 有了它，您可以随意将文件块上传到云端硬盘.您现在需要使用此 URI 作为连续上传的上传点.

Now you have the Session URI - with this you can upload chunks of file to Drive as you please. You now need to use this URI as the upload point for successive uploads.

请记住:可恢复的会话 URI 会在一次之后过期一周.

这实际上取决于您希望如何实施.例如，您可以打破一个循环，或者在 GUI 中有一个巨大的 PAUSE THIS UPLOAD 按钮，用于切换上传的下一部分是否继续.

This in reality is down to how you wish to implement this. You could break out a loop for example or have a giant PAUSE THIS UPLOAD button in a GUI which toggles whether the next section of the upload continues or not.

要记住的是，在上传文件内容时，发出的请求必须使用 HTTP PUT 而不是 POST.继上一节之后:

The thing to remember is when uploading the file contents, the request made has to be done with HTTP PUT rather than POST. Following on from the previous section:

// set these variables:
long beginningOfChunk = 0;
long chunkSize = 2 * MediaHttpUploader.MINIMUM_CHUNK_SIZE;
int chunksUploaded = 0;

// Here starts the upload chunk code:
HttpURLConnection request = (HttpURLConnection) sessionUri.openConnection();

request.setRequestMethod("PUT");
request.setDoOutput(true);
// change your timeout as you desire here:
request.setConnectTimeout(30000); 
request.setRequestProperty("Content-Type", "file/mimetype");

long bytesUploadedSoFar = chunksUploaded * chunkSize;

if (beginningOfChunk + chunkSize > number_of_bytes_of_your_file) {
    chunkSize = (int) number_of_bytes_of_your_file - beginningOfChunk;
}

request.setRequestProperty("Content-Length", String.format(Locale.ENGLISH, "%d", chunkSize));
request.setRequestProperty("Content-Range", "bytes " + beginningOfChunk + "-" + (beginningOfChunk + chunkSize - 1) + "/" + number_of_bytes_of_your_file);

byte[] buffer = new byte[(int) chunksize];
FileInputStream fileInputStream = new FileInputStream(yourFile);
fileInputStream.getChannel().position(beginningOfChunk);
fileInputStream.close();

OutputStream outputStream = request.getOutputStream();
outputStream.write(buffer);
outputStream.close();
request.connect();

chunksUploaded += 1;

// End of upload chunk section

然后您可以重复调用上传块代码；在一个循环中，作为一个函数；你喜欢.由于它是一个独特的代码块，您可以随意调用它，从而实现某种暂停上传的方式(通过中断、睡眠、等待等).

You can then call the upload chunk code on repeat; in a loop, as a function; how you like. As it is a distinct code block, you can call it as you like and therefore implement some way of pausing the upload (through breaks, sleeps, waits, etc).

请记住:您需要保存会话 URI 才能恢复.

Just remember: you will need to save the session URI in order to resume.

似乎还无法直接使用 Drive V3 API 进行可恢复的上传.Java 客户端库文档 在讨论何时使用 Drive: create 与非特定于服务的库时暗示了这一点:

It seems that using the Drive V3 API directly to make resumable uploads isn't something yet possible. The Java Client Library documentation alludes to this when discussing when to use Drive: create vs the non service-specific libraries:

...Drive API的insert方法支持mediaUpload，可以使用如下代码上传文件:

...the insert method of the Drive API supports mediaUpload, and you can use the following code to upload a file:

code block

您还可以使用可恢复媒体上传功能，而无需特定于服务的生成库.

You can also use the resumable media upload feature without the service-specific generated libraries.

功能请求:

但是，您可以直接让 Google 知道这是一项对 Drive API 很重要的功能，而不是使用非服务特定库的要求.Google 的问题跟踪器是开发人员报告问题和为其开发服务提出功能请求的地方.为 Drive API 提交功能请求的页面是此处.

差异说明:Drive API V2 中称为 Drive.Files.Insert 的内容已更改为 Drive 中的 Drive.Files.CreateAPI V3.

Note of discrepancy: what was called Drive.Files.Insert in Drive API V2 was changed to Drive.Files.Create in Drive API V3.

我知道这通常是个坏消息，但我希望这对您有所帮助！

I know this is generally bad news, but I hope this is helpful to you!

• Java 快速入门 |Google 云端硬盘 API |Google 开发者
• Drive.Files.Create(驱动 APIv3 (Rev. 197) 1.25.0)
• 创建和填充文件夹 |Google 云端硬盘 API |Google 开发者
• 上传文件数据 |Google 云端硬盘 API |Google 开发者
• 文件:创建 |Google 云端硬盘 API |Google 开发者
• 直接和可恢复媒体上传 |Java API 客户端库

  ---

• Google Drive api Java - 我的大型分段上传完成了吗?
• 在 Google Drive Java API 中设置 uploadType
• Drive Rest API V3 中的可恢复上传
• Google Drive API - Drive.Files 类没有插入 () 方法

这篇关于java google drive api V3 MultiPart &amp;可恢复上传的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！