使用Java上传具有multipart/form-data的zip文件时丢失字节 [英] Missing bytes when uploading zip file with multipart/form-data using java

问题描述

我在将.zip文件上传到远程服务器时遇到问题，因为上传后文件中缺少一些字节.重新下载文件后，.zip存档将无法打开，这使我相信我需要那些字节才能成功执行上传.

I'm having an issue with uploading a .zip file to a remote server in that some bytes are missing from the file after the upload. Upon redownloading the file, the .zip archive is unopenable, which leads me to believe that I need those bytes to perform the upload successfully.

我正在使用多部分/表单数据POST请求来上传文件.下面的代码给出了我用来执行此操作的实用程序帮助程序类:

I am using a multipart/form-data POST request to upload the file. The utility helper class I use to do this is given in the below code:

import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;

public class MultipartFormDataUtil {
    private final String boundary;
    private static final String lineReturn = "\r\n";
    private HttpURLConnection conn;
    private DataOutputStream dos;
    int bytesRead, bytesAvail, bufferSize;
    byte[] buffer;
    int maxBufferSize = 1*1024*1024;
    List<String> response;

    public MultipartFormDataUtil(String postUrl, LinkedHashMap<String, String> params, File file) throws IOException {
        boundary = "=-=" + System.currentTimeMillis() + "=-=";

        URL url = new URL(postUrl);
        conn = (HttpURLConnection) url.openConnection();
        conn.setDoInput(true);
        conn.setDoOutput(true);
        conn.setUseCaches(false);
        conn.setRequestMethod("POST");
        conn.setRequestProperty("Connection", "Keep-Alive");
        conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
        dos = new DataOutputStream(conn.getOutputStream());

        for (String key : params.keySet()) {
            addFormPart(key, params.get(key));
        }

        addFilePart(file);

        finish();
    }

    private void addFormPart(String name, String value) throws IOException {
        dos.writeBytes("--" + boundary + lineReturn);
        dos.writeBytes("Content-Disposition: form-data; name=\"" + name + "\"" + lineReturn);
        dos.writeBytes("Content-Type: text/plain" + lineReturn + lineReturn);
        dos.writeBytes(value + lineReturn);
        dos.flush();
    }

    private void addFilePart(File file) throws IOException {
        FileInputStream fileInputStream = new FileInputStream(file);

        dos.writeBytes("--" + boundary + lineReturn);
        dos.writeBytes("Content-Disposition: form-data; name=\"file\"; filename=\"" + file.getName() + "\"" + lineReturn);
        dos.writeBytes("Content-Type: " + URLConnection.guessContentTypeFromName(file.getName()) + lineReturn);
        dos.writeBytes("Content-Transfer-Encoding: binary" + lineReturn + lineReturn);

        bytesAvail = fileInputStream.available();
        bufferSize = Math.min(bytesAvail, maxBufferSize);
        buffer = new byte[bufferSize];

        bytesRead = fileInputStream.read(buffer, 0, bufferSize);

        while (bytesRead > 0) {
            dos.write(buffer, 0, bufferSize);
            bytesAvail = fileInputStream.available();
            bufferSize = Math.min(bytesAvail, maxBufferSize);
            bytesRead = fileInputStream.read(buffer, 0, bufferSize);
        }
        dos.flush();

        dos.writeBytes(lineReturn);
        dos.flush();
        fileInputStream.close();
    }

    private void finish() throws IOException {
        response = new ArrayList<String>();

        dos.writeBytes("--" + boundary + "--" + lineReturn);
        dos.flush();
        dos.close();

        BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
        String line;

        while ((line = reader.readLine()) != null) {
            response.add(line);
        }

        reader.close();
        conn.disconnect();
    }

    public List<String> getResponse() {
        return response;
    }

要在应得的信用额中给予信用，此实用程序基于 CodeJava.net .使用以下代码调用该实用程序:

To give credit where credit is due, this utility is based off of examples from Peter's Notes and CodeJava.net. This util is called with the following code:

protected static void postFile(String url, LinkedHashMap<String, String> params, File file) throws Exception {
    try {
        MultipartFormDataUtil multipartRequest = new MultipartFormDataUtil(url, params, file);
        List<String> response = multipartRequest.getResponse();

        for (String line : response) {
            System.out.println(line);
        }

    } catch (IOException ioe) {
        log.warn("There was an error posting the file and form data", ioe);
    }
}

在这种情况下，上传URL到Amazon S3存储桶，然后将其传递到目标系统.在此最终目的地，我可以看到应该在.zip文件上运行的进程失败(注意:该进程由Rails应用程序运行，并显示错误消息错误标识包类型:无法dup NilClass").下载文件后，我看到文件大小为3,110,416字节，而不是3,110,466字节.我再也无法提取存档以查看其中的内容了.mac存档实用程序将响应错误2-没有此类文件或目录".

The upload url in this case is to an Amazon S3 bucket, which passes it on to the destination system. It is at this final destination that I can see that the process that is supposed to be running on the .zip file has failed (note: the process is run by a Rails app and gives the error "Error identifying package type: can't dup NilClass"). Upon downloading the file, I see that the file size is 3,110,416 bytes instead of 3,110,466 bytes. I can no longer extract the archive to see what is in it; the mac archive utility responds with "Error 2 - No such file or directory".

我缺乏这方面的概念背景，无法了解过程中哪里可能出了问题.我希望有人能够告诉我我在实用程序类中出错了，或者让我知道问题出在其他地方.

I lack the conceptual background in this area to get a feel for where in the process things may be going wrong. I am hoping that someone will be able to tell me that I made an error in the utility class, or let me know that something else is the problem.

感谢您提供的任何见解，并让我知道我是否可以发布其他有帮助的东西.

Thank you for any insight you can provide, and let me know if I can post anything else that would be of help.

我收集了一些有关文件上传大小(以字节为单位)的其他信息:

Some additional information I gathered about different sizes of file uploads (in bytes):

原始版本----------已上传----------差异

Original----------Uploaded----------Difference

10,167,389 ______ 10,167,238 ______ 151

10,167,389______10,167,238______151

3,110,466 _______ 3,110,416 ________ 50

3,110,466_______3,110,416_______50

156,885 _________ 156,885 _________ 0_0

156,885_________156,885_________0

95,639,352 ______ 95,637,925 ________ 1,427

95,639,352______95,637,925______1,427

对于上载后丢失字节的3个文件，每个文件丢失的总百分比大约为(但不完全)为0.0015％，但彼此不相等.

For the 3 files that had bytes missing following the upload, the % of total data lost was around (but not exactly) 0.0015% for each one, but not equal to each other.

推荐答案

在进一步研究中，我们发现该错误与multipart/form-data实用程序无关，如本问题所示.相反，它与中的我们自己的文件下载客户端有关.这里.我们没有将FileTransfer客户端设置为以二进制格式下载文件，这对于.zip文件是必需的.

Upon some further research, we found that the error did not have anything to do with the multipart/form-data utility as shown in this question. Instead, it had to do with our own file downloading client in here. We were not setting the FileTransfer client to download the file as binary, which is necessary for a .zip file.

可以随意使用原始问题中包含的代码来实现Java中的多部分/表单数据目的-假设原始文件没有问题，那么效果很好.

Feel free to use the code included in the original question for your multipart/form-data purposes in java - it works great assuming there are no problems with your original file.

这篇关于使用Java上传具有multipart/form-data的zip文件时丢失字节的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！