如何使用Java REST服务和数据流下载文件 [英] How to download a file using a Java REST service and a data stream
问题描述
我有3台机器：
- 文件所在的服务器
- 运行REST服务的服务器（Jersey）
- 访问第二台服务器但无法访问第一台服务器的客户端（浏览器）
如何直接（不保存第二个服务器上的文件）将文件从第一台服务器下载到客户端的机器？
从第二台服务器我可以得到一个 ByteArrayOutputStream 从第一台服务器获取文件，我可以使用REST服务进一步传递给客户端吗？
它将以这种方式工作？
所以基本上我想要实现的是允许客户端从第一台服务器下载一个文件第二台服务器上的REST服务（因为没有直接从客户端访问第一台服务器）只使用数据流（因此没有数据触及第二台服务器的文件系统）。
我现在用EasyStream库尝试：
final FTDClient client = FTDClient.getInstance（）;
try {
final InputStreamFromOutputStream< String> isOs = new InputStreamFromOutputStream< String>（）{
@Override
public String generate（final OutputStream dataSink）throws Exception {
return client.downloadFile2（location，Integer.valueOf（spaceId），URLDecoder .decode（filePath，UTF-8），dataSink）;
}
};
try {
String fileName = filePath.substring（filePath.lastIndexOf（/）+ 1）;
StreamingOutput output = new StreamingOutput（）{
@Override
public void write（OutputStream outputStream）throws IOException，WebApplicationException {
int length;
byte [] buffer = new byte [1024];
while（（length = isOs.read（buffer））！= -1）{
outputStream.write（buffer，0，length）;
}
outputStream.flush（）;
}
};
return Response.ok（output，MediaType.APPLICATION_OCTET_STREAM）
.header（Content-Disposition，attachment; filename = \+ fileName +\）
。建立（）;
更新2
所以我的代码现在与自定义的MessageBodyWriter看起来很简单：
ByteArrayOutputStream baos = new ByteArrayOutputStream（2048）;
client.downloadFile（location，spaceId，filePath，baos）;
return Response.ok（baos）.build（）;
但是当尝试使用大文件时，我收到相同的堆错误。
UPDATE3
最后设法让它工作！
StreamingOutput做的伎俩。
谢谢@peeskillet！非常感谢！
我如何直接（不保存第二个服务器上的文件）下载文件从第一个服务器到客户端的机器？
只需使用客户端 API，并从响应中获取 InputStream
客户端客户端= ClientBuilder。 newClient（）;
String url =...;
final InputStream responseStream = client.target（url）.request（）。get（InputStream.class）;
有两种口味来获取 InputStream 。您还可以使用
响应响应= client.target（url）.request（）。get（）;
InputStream is =（InputStream）response.getEntity（）;
哪一个更有效率，我不确定，但返回的 InputStream 是不同的类，所以你可能想看看，如果你关心。
从2nd服务器我可以得到一个ByteArrayOutputStream从第一个服务器获取文件，我可以使用REST服务进一步传递给客户端吗？
因此，您将在由@GradyGCooper提供的链接中看到的大多数答案似乎都有利于使用 StreamingOutput 。一个示例实现可能类似于
final InputStream responseStream = client.target（url）.request（）。get（InputStream。类）;
System.out.println（responseStream.getClass（））;
StreamingOutput output = new StreamingOutput（）{
@Override
public void write（OutputStream out）throws IOException，WebApplicationException {
int length;
byte [] buffer = new byte [1024];
while（（length = responseStream.read（buffer））！= -1）{
out.write（buffer，0，length）;
}
out.flush（）;
responseStream.close（）;
}
};
return Response.ok（output）.header（
Content-Disposition，attachment，filename = \... \）build（）;
但是，如果我们看看 StreamingOutputProvider的源代码，您将在 writeTo 中看到，它只是将数据从一个流写入另一个流。所以在我们上面的实现中，我们必须写两次。
我们如何只得到一个写？简单的返回 InputStream 作为响应
final InputStream responseStream = client.target（url）.request（）。get（InputStream.class）;
return Response.ok（responseStream）.header（
Content-Disposition，attachment，filename = \... \）build（）;
如果我们看看 InputStreamProvider的源代码，它只是委派给 ReadWriter.writeTo（in，out） ，这只是在 StreamingOutput 实现
public static void writeTo（InputStream in，OutputStream out）throws IOException {
int read;
final byte [] data = new byte [BUFFER_SIZE];
while（（read = in.read（data））！= -1）{
out.write（data，0，read）;
}
}
Asides： p>
-
客户端对象是昂贵的资源。您可能需要重复使用相同的客户端来请求。您可以从客户端为每个请求提取WebTarget。WebTarget target = client.target（url）;
InputStream is = target.request（）。get（InputStream.class）;
我认为
WebTarget甚至可以共享。在泽西2.x文档中找不到任何内容只是因为它是一个较大的文档，而且我太懒了，现在扫描它:-)，但在泽西1.x文档，它表示客户端和WebResource（相当于2.x中的WebTarget）可以在线程之间共享。所以我猜猜泽西2.x会是一样的。但是您可能想要自己确认。 -
您不必使用
客户端API。可以使用java.net软件包API轻松实现下载。但是，由于你已经使用泽西，所以不要使用它的API -
上面是假设Jersey 2.x.对于泽西1.x来说，一个简单的Google搜索应该会让您受益于使用API（或上述链接的文档）。
更新
我是这样的二重奏。虽然OP和我正在考虑如何将 ByteArrayOutputStream 转换为 InputStream ，但我错过了最简单的解决方案只需为 ByteArrayOutputStream
<$ p $写一个
MessageBodyWriter p> import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.lang.annotation.Annotation;
import java.lang.reflect.Type;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.ext.MessageBodyWriter;
import javax.ws.rs.ext.Provider;
@Provider
public class OutputStreamWriter implements MessageBodyWriter< ByteArrayOutputStream> {
@Override
public boolean isWriteable（Class<？> type，Type genericType，
Annotation [] annotations，MediaType mediaType）{
return ByteArrayOutputStream.class == type;
}
@Override
public long getSize（ByteArrayOutputStream t，Class<？> type，Type genericType，
Annotation [] annotations，MediaType mediaType）{
return -1;
@Override
public void writeTo（ByteArrayOutputStream t，Class<？> type，Type genericType，
Annotation [] annotations，MediaType mediaType，
MultivaluedMap< String，Object> httpHeaders，OutputStream entityStream）
throws IOException，WebApplicationException {
t.writeTo（entityStream）;
}
}
然后我们可以简单地返回 ByteArrayOutputStream 在响应中
return Response.ok（baos）.build（）;
D'OH！
更新2
以下是我使用的测试（
资源类
@Path（test）
public class TestResource {
final String path =some_150_mb_file;
@GET
@Produces（MediaType.APPLICATION_OCTET_STREAM）
public Response doTest（）throws异常{
InputStream is = new FileInputStream（path）;
ByteArrayOutputStream baos = new ByteArrayOutputStream ）;
int len;
byte [] buffer = new byte [4096];
while（（len = is.read（buffer，0，buffer.length））！= -1 ）{
baos.write（buffer，0，len）;
}
System.out.println（Server size：+ baos.size（））;
return Response.ok（baos）.build（）;
}
}
客户端测试
public class Main {
public static void main（String [] arg s）throws Exception {
客户端客户端= ClientBuilder.newClient（）;
String url =http：// localhost：8080 / api / test;
响应响应= client.target（url）.request（）。get（）;
String location =some_location;
FileOutputStream out = new FileOutputStream（location）;
InputStream is =（InputStream）response.getEntity（）;
int len = 0;
byte [] buffer = new byte [4096];
while（（len = is.read（buffer））！= -1）{
out.write（buffer，0，len）;
}
out.flush（）;
out.close（）;
is.close（）;
}
}
更新3
所以这个特殊用例的最终解决方案是让OP简单地从 StreamingOutput OutputStream $ c>的写入方法。似乎是第三方API，需要一个 OutputStream 作为参数。
StreamingOutput output = new StreamingOutput（）{
@Override
public void write（OutputStream out）{
thirdPartyApi.downloadFile（..，..，..，out）;
}
}
返回Response.ok（输出）.build（）;
不要确定，但似乎在资源方法中使用ByteArrayOutputStream进行阅读/写入，实现了一些进入记忆。
downloadFile 方法接受 OutputStream 可以直接将结果写入 OutputStream 提供。例如，一个 FileOutputStream ，如果您在下载文件时将其写入文件，则会直接将其流式传输到该文件。
正如您尝试使用的宝贝一样，不要保留对 OutputStream 的引用，这是你想要做的，这是内存实现的地方。
所以有了这种方式，我们直接写给我们提供的响应流。直到 writeTo 方法（ MessageBodyWriter ）中的方法 write / code>），其中 OutputStream 被传递给它。
您可以获得更好的图片看看我写的 MessageBodyWriter 基本上在 writeTo 方法中，将 ByteArrayOutputStream 替换为 StreamingOutput ，然后在方法中调用 streamingOutput.write（entityStream）。您可以在答案的前面部分看到我提供的链接，其中链接到 StreamingOutputProvider 。这正是发生了什么。
I have 3 machines:
- server where the file is located
- server where REST service is running ( Jersey)
- client(browser) with access to 2nd server but no access to 1st server
How can I directly (without saving the file on 2nd server) download the file from 1st server to client's machine?
From 2nd server I can get a ByteArrayOutputStream to get the file from 1st server, can I pass this stream further to the client using the REST service?
It will work this way?
So basically what I want to achieve is to allow the client to download a file from 1st server using the REST service on 2nd server (since there is no direct access from client to 1st server) using only data streams (so no data touching the file system of 2nd server).
What I try now with EasyStream library:
final FTDClient client = FTDClient.getInstance();
try {
final InputStreamFromOutputStream<String> isOs = new InputStreamFromOutputStream<String>() {
@Override
public String produce(final OutputStream dataSink) throws Exception {
return client.downloadFile2(location, Integer.valueOf(spaceId), URLDecoder.decode(filePath, "UTF-8"), dataSink);
}
};
try {
String fileName = filePath.substring(filePath.lastIndexOf("/") + 1);
StreamingOutput output = new StreamingOutput() {
@Override
public void write(OutputStream outputStream) throws IOException, WebApplicationException {
int length;
byte[] buffer = new byte[1024];
while ((length = isOs.read(buffer)) != -1){
outputStream.write(buffer, 0, length);
}
outputStream.flush();
}
};
return Response.ok(output, MediaType.APPLICATION_OCTET_STREAM)
.header("Content-Disposition", "attachment; filename=\"" + fileName + "\"" )
.build();
UPDATE2
So my code now with the custom MessageBodyWriter looks simple:
ByteArrayOutputStream baos = new ByteArrayOutputStream(2048) ; client.downloadFile(location, spaceId, filePath, baos); return Response.ok(baos).build();
But I get the same heap error when trying with large files.
UPDATE3 Finally managed to get it working ! StreamingOutput did the trick.
Thank you @peeskillet ! Many thanks !
"How can I directly (without saving the file on 2nd server) download the file from 1st server to client's machine?"
Just use the Client API and get the InputStream from the response
Client client = ClientBuilder.newClient();
String url = "...";
final InputStream responseStream = client.target(url).request().get(InputStream.class);
There's two flavors to get the InputStream. You can also use
Response response = client.target(url).request().get();
InputStream is = (InputStream)response.getEntity();
Which one is more efficient, I'm not sure, but the returned InputStreams are different classes, so you may want to look into that if you care to.
From 2nd server I can get a ByteArrayOutputStream to get the file from 1st server, can I pass this stream further to the client using the REST service?
So most of the answers you'll see in the link provided by @GradyGCooper seem to favor the use of StreamingOutput. An example implementation might be something like
final InputStream responseStream = client.target(url).request().get(InputStream.class);
System.out.println(responseStream.getClass());
StreamingOutput output = new StreamingOutput() {
@Override
public void write(OutputStream out) throws IOException, WebApplicationException {
int length;
byte[] buffer = new byte[1024];
while((length = responseStream.read(buffer)) != -1) {
out.write(buffer, 0, length);
}
out.flush();
responseStream.close();
}
};
return Response.ok(output).header(
"Content-Disposition", "attachment, filename=\"...\"").build();
But if we look at the source code for StreamingOutputProvider, you'll see in the writeTo, that it simply write the data from one stream to another. So with our implementation above, we have to write twice.
How can we get only one write? Simple return the InputStream as the Response
final InputStream responseStream = client.target(url).request().get(InputStream.class);
return Response.ok(responseStream).header(
"Content-Disposition", "attachment, filename=\"...\"").build();
If we look at the source code for InputStreamProvider, it simply delegates to ReadWriter.writeTo(in, out), which simply does what we did above in the StreamingOutput implementation
public static void writeTo(InputStream in, OutputStream out) throws IOException {
int read;
final byte[] data = new byte[BUFFER_SIZE];
while ((read = in.read(data)) != -1) {
out.write(data, 0, read);
}
}
Asides:
Clientobjects are expensive resources. You may want to reuse the sameClientfor request. You can extract aWebTargetfrom the client for each request.WebTarget target = client.target(url); InputStream is = target.request().get(InputStream.class);I think the
WebTargetcan even be shared. I can't find anything in the Jersey 2.x documentation (only because it is a larger document, and I'm too lazy to scan through it right now :-), but in the Jersey 1.x documentation, it says theClientandWebResource(which is equivalent toWebTargetin 2.x) can be shared between threads. So I'm guessing Jersey 2.x would be the same. but you may want to confirm for yourself.You don't have to make use of the
ClientAPI. A download can be easily achieved with thejava.netpackage APIs. But since you're already using Jersey, it don't hurt to use its APIsThe above is assuming Jersey 2.x. For Jersey 1.x, a simple Google search should get you a bunch of hit for working with the API (or the documentation I linked to above)
UPDATE
I'm such a dufus. While the OP and I are contemplating ways to turn a ByteArrayOutputStream to an InputStream, I missed the simplest solution, which is simply to write a MessageBodyWriter for the ByteArrayOutputStream
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.lang.annotation.Annotation;
import java.lang.reflect.Type;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.ext.MessageBodyWriter;
import javax.ws.rs.ext.Provider;
@Provider
public class OutputStreamWriter implements MessageBodyWriter<ByteArrayOutputStream> {
@Override
public boolean isWriteable(Class<?> type, Type genericType,
Annotation[] annotations, MediaType mediaType) {
return ByteArrayOutputStream.class == type;
}
@Override
public long getSize(ByteArrayOutputStream t, Class<?> type, Type genericType,
Annotation[] annotations, MediaType mediaType) {
return -1;
}
@Override
public void writeTo(ByteArrayOutputStream t, Class<?> type, Type genericType,
Annotation[] annotations, MediaType mediaType,
MultivaluedMap<String, Object> httpHeaders, OutputStream entityStream)
throws IOException, WebApplicationException {
t.writeTo(entityStream);
}
}
Then we can simply return the ByteArrayOutputStream in the resonse
return Response.ok(baos).build();
D'OH!
UPDATE 2
Here are the tests I used (
Resource class
@Path("test")
public class TestResource {
final String path = "some_150_mb_file";
@GET
@Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response doTest() throws Exception {
InputStream is = new FileInputStream(path);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int len;
byte[] buffer = new byte[4096];
while ((len = is.read(buffer, 0, buffer.length)) != -1) {
baos.write(buffer, 0, len);
}
System.out.println("Server size: " + baos.size());
return Response.ok(baos).build();
}
}
Client test
public class Main {
public static void main(String[] args) throws Exception {
Client client = ClientBuilder.newClient();
String url = "http://localhost:8080/api/test";
Response response = client.target(url).request().get();
String location = "some_location";
FileOutputStream out = new FileOutputStream(location);
InputStream is = (InputStream)response.getEntity();
int len = 0;
byte[] buffer = new byte[4096];
while((len = is.read(buffer)) != -1) {
out.write(buffer, 0, len);
}
out.flush();
out.close();
is.close();
}
}
UPDATE 3
So the final solution for this particular use case was for the OP to simply pass the OutputStream from the StreamingOutput's write method. Seems the third-party API, required a OutputStream as an argument.
StreamingOutput output = new StreamingOutput() {
@Override
public void write(OutputStream out) {
thirdPartyApi.downloadFile(.., .., .., out);
}
}
return Response.ok(output).build();
Not quire sure, but seems the reading/writing within the resource method, using ByteArrayOutputStream`, realized something into memory.
The point of the downloadFile method accepting an OutputStream is so that it can write the result directly to the OutputStream provided. For instance a FileOutputStream, if you wrote it to file, while the download is coming in, it would get directly streamed to the file.
It's not meant for us to keep a reference to the OutputStream, as you were trying to do with the baos, which you were trying to do, which is where the memory realization comes in.
So with the way that works, we are writing directly to the response stream provided for us. The method write doesn't actually get called until the writeTo method (in the MessageBodyWriter), where the OutputStream is passed to it.
You can get a better picture looking at the MessageBodyWriter I wrote. Basically in the writeTo method, replace the ByteArrayOutputStream with StreamingOutput, then inside the method, call streamingOutput.write(entityStream). You can see the link I provided in the earlier part of the answer, where I link to the StreamingOutputProvider. This is exactly what happens
这篇关于如何使用Java REST服务和数据流下载文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
