你如何上传文件作为流? [英] How can you upload files as a stream in go?
问题描述
有很多关于使用 http.Request
发布文件的教程,但几乎总是像这样开始:
There are a number of tutorials about posting files using http.Request
in go, but almost invariably they start like this:
file, err := os.Open(path)
if err != nil {
return nil, err
}
fileContents, err := ioutil.ReadAll(file)
也就是说,整个文件存储到内存中,然后将其转换为 Buffer
并将其传递到请求中,如下所示:
Which is to say, you read the entire file into memory, and then convert it into a Buffer
and pass that into a request, something like this:
func send(client *http.Client, file *os.File, endpoint string) {
body := &bytes.Buffer{}
io.Copy(body, file)
req, _ := http.NewRequest("POST", endpoint, body)
resp, _ := client.Do(req)
}
如果您想发布大量文件并避免将其读入内存, 。你会怎么做?
If you wanted to post a massive file and avoid reading it into memory, but instead steam the file up in chunks... how would you do that?
推荐答案
如果你需要设置 Content-Length
,可以手动完成。以下片段是将文件和额外参数作为流上传的示例(基于 Golang的无缓冲区多部分POST)
If you need to set Content-Length
, it can be done manually. The following snippet is an example of uploading file and extra parameters as a stream (the code based on Buffer-less Multipart POST in Golang)
//NOTE: for simplicity, error check is omitted
func uploadLargeFile(uri, filePath string, chunkSize int, params map[string]string) {
//open file and retrieve info
file, _ := os.Open(filePath)
fi, _ := file.Stat()
defer file.Close()
//buffer for storing multipart data
byteBuf := &bytes.Buffer{}
//part: parameters
mpWriter := multipart.NewWriter(byteBuf)
for key, value := range params {
_ = mpWriter.WriteField(key, value)
}
//part: file
mpWriter.CreateFormFile("file", fi.Name())
contentType := mpWriter.FormDataContentType()
nmulti := byteBuf.Len()
multi := make([]byte, nmulti)
_, _ = byteBuf.Read(multi)
//part: latest boundary
//when multipart closed, latest boundary is added
mpWriter.Close()
nboundary := byteBuf.Len()
lastBoundary := make([]byte, nboundary)
_, _ = byteBuf.Read(lastBoundary)
//calculate content length
totalSize := int64(nmulti) + fi.Size() + int64(nboundary)
log.Printf("Content length = %v byte(s)\n", totalSize)
//use pipe to pass request
rd, wr := io.Pipe()
defer rd.Close()
go func() {
defer wr.Close()
//write multipart
_, _ = wr.Write(multi)
//write file
buf := make([]byte, chunkSize)
for {
n, err := file.Read(buf)
if err != nil {
break
}
_, _ = wr.Write(buf[:n])
}
//write boundary
_, _ = wr.Write(lastBoundary)
}()
//construct request with rd
req, _ := http.NewRequest("POST", uri, rd)
req.Header.Set("Content-Type", contentType)
req.ContentLength = totalSize
//process request
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
} else {
log.Println(resp.StatusCode)
log.Println(resp.Header)
body := &bytes.Buffer{}
_, _ = body.ReadFrom(resp.Body)
resp.Body.Close()
log.Println(body)
}
}
这篇关于你如何上传文件作为流?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!