管道流从busboy请求发布 [英] Piping stream from busboy to request post

问题描述

我有 multipart / form-data 我发布到快递端点 / data / upload ，表单标记如下：

I have multipart/form-data that I am posting to an express endpoint /data/upload, form markup below:

form(enctype="multipart/form-data", action="/data/upload", method="post")
  input(type="file", name="data")

我正在使用 busboy 来读取文件流，这工作正常。从那里，我想使用请求 multipart / form-data 发送到第二个Java后端> npm模块。下面的JS客户端/ Java服务器代码：

I'm using busboy to read the file stream, which is working fine. From there, I want to send the stream again as multipart/form-data to a second Java backend, using the request npm module. JS client/Java server code below:

  req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) {

    var reqBody = {
      url: server.baseURL + 'api/data',
      headers: {
        'Connection': 'keep-alive',
        'Content-Type': 'multipart/form-data'
      },
      formData: {
        file: fileStream
      }
    };

    request.post(reqBody, function (err, r, body) {
      // Do rendering stuff, handle callback
    });
 });

Java端点（api / data）

@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
public void addData(FormDataMultiPart formDataMultiPart) {
  // Handle multipart data here      
}

我认为我没有正确发送文件为 multipart / form-data 这里......但是我很难搞清楚如何从 busboy 直接到请求而无需从客户端的临时文件读取/写入。任何想法？

I don't think I'm sending the file correctly as multipart/form-data here... but I'm having a hard time figuring out how to essentially pipe the stream from busboy directly to request without reading/writing from a temp file on the client-side. Any ideas?

Java堆栈跟踪：

Apr 27, 2016 5:07:12 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * Server has received a request on thread qtp1631904921-24
3 > POST http://localhost:8080/api/data
3 > Connection: keep-alive
3 > Content-Length: 199
3 > Content-Type: multipart/form-data; boundary=--------------------------331473417509479560313628
3 > Host: localhost:8080

Apr 27, 2016 5:07:12 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * Server responded with a response on thread qtp1631904921-24
3 < 400

17:07:13.003 [qtp1631904921-24] WARN  org.eclipse.jetty.http.HttpParser parseNext - bad HTTP parsed: 400 No URI for HttpChannelOverHttp@425137da{r=1,c=false,a=IDLE,uri=null}

Rahat推荐的更改：

 31     var reqBody = {
 32       url: server.baseURL + 'data',
 33       headers: {
 34         'Connection': 'keep-alive',
 35         'Content-Type': 'multipart/form-data'
 36       }
 37     };
 38 
 39     req.pipe(req.busboy.pipe(request.post(reqBody)));

投掷错误：

Error: Cannot pipe. Not readable.
   at Busboy.Writable.pipe (_stream_writable.js:154:22)

推荐答案

这里的问题是您需要手动为分段上传提供内容长度，因为 request （以及基础 form-data ）无法自行解决。
因此请求发送无效的Content-Length：199（对于任何传入的文件大小都相同），这会破坏java多部分解析器。

The problem here is that you need to provide 'Content-Length' for the multipart upload manually, because request (and underlying form-data) can't figure it out by themselves. So request sends invalid Content-Length: 199 (the same for any incoming file size), which breaks the java multipart parser.

有多种解决方法：

1）使用传入请求'Content-Length'

1) Use incoming request 'Content-Length'

request.post({
  url: server.baseURL + 'api/data',
  formData: {
    file: {
      value: fileStream,
      options: {
        knownLength: req.headers['content-length']
      }
    }
  }
}, function (err, r, body) {
  // Do rendering stuff, handle callback
})

这会产生一些不正确的请求虽然，因为传入的长度包括其他上传字段和边界，但busboy能够解析它没有任何投诉

This will produce a bit incorrect request though, because incoming length includes other upload fields and boundaries, but busboy was able to parse it w/o any complaints

2）等到文件被节点应用程序完全缓冲然后将其发送到java

2) Wait until file is completely buffered by the node app then send it to java

var concat = require('concat-stream')
req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) {
  fileStream.pipe(concat(function (fileBuffer) {
    request.post({
      url: server.baseURL + 'api/data',
      formData: {
        file: fileBuffer
      }
    }, function (err, r, body) {
      // Do rendering stuff, handle callback
    })
  }))
})

这会增加应用内存消耗，所以你需要小心并考虑使用 busboy限制

This will increase app memory consumption, so you needed to be careful and consider using busboy limits

3）上传前缓冲文件到磁盘（仅供参考）

3) Buffer file to disk before uploading (just for the reference)

• 快递 + multer - 我建议使用express for webservers，它使事情更易于管理，而multer基于busboy


• 强大的

• express + multer - I recommend using express for webservers, it makes things more manageable, and multer is based on the busboy
• formidable

这篇关于管道流从busboy请求发布的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！