Spring批处理文件编写器无需PutObjectRequest即可直接写入亚马逊S3存储 [英] spring batch file writer to write directly to amazon s3 storage without PutObjectRequest

问题描述

我正在尝试将文件上传到Amazon s3.我不想上传，而是想使用spring batch从数据库中读取数据并将文件直接写到s3存储中.无论如何，我们可以做到吗?

I'm trying to upload a file to amazon s3. Instead of uploading, I want to read the data from database using spring batch and write the file directly into the s3 storage. Is there anyway we can do that ?

推荐答案

我有同样的事情要做.因为spring没法单独写一条流，所以我像上面的示例一样做了一个自己的事情:

I had the same thing to do. Because spring has no clas to write to a stream alone I made one my self like above example:

您需要为此分类.一个实现WriteableResource并扩展AbstractResource的Resource类:

You need to classes for this. A Resource class which implements WriteableResource and extends AbstractResource:

...

public class S3Resource extends AbstractResource implements WritableResource {

   ByteArrayOutputStream resource = new ByteArrayOutputStream();

    @Override
    public String getDescription() {
        return null;
    }

    @Override
    public InputStream getInputStream() throws IOException {
        return new ByteArrayInputStream(resource.toByteArray());
    }

    @Override
    public OutputStream getOutputStream() throws IOException {
        return resource;
    }
}

还有扩展ItemWriter的作家:

And your writer which extends ItemWriter:

public class AmazonStreamWriter<T> implements ItemWriter<T>{

    private WritableResource resource;
    private LineAggregator<T> lineAggregator;
    private String lineSeparator;

    public String getLineSeparator() {
        return lineSeparator;
    }

    public void setLineSeparator(String lineSeparator) {
        this.lineSeparator = lineSeparator;
    }

    AmazonStreamWriter(WritableResource resource){
        this.resource = resource;
    }

    public WritableResource getResource() {
        return resource;
    }

    public void setResource(WritableResource resource) {
        this.resource = resource;
    }

    public LineAggregator<T> getLineAggregator() {
        return lineAggregator;
    }

    public void setLineAggregator(LineAggregator<T> lineAggregator) {
        this.lineAggregator = lineAggregator;
    }

    @Override
    public void write(List<? extends T> items) throws Exception {
        try (OutputStream outputStream = resource.getOutputStream()) {
                StringBuilder lines = new StringBuilder();
                Iterator var3 = items.iterator();

                while(var3.hasNext()) {
                    T item = (T) var3.next();
lines.append(this.lineAggregator.aggregate(item)).append(this.lineSeparator);
                }
                outputStream.write(lines.toString().getBytes());
        }
    }
}

使用此设置，您将编写从数据库中接收到的项目信息，并将其通过OutputStream写入到Customresource中.然后，可以在您的步骤之一中使用已填充的资源zu打开InputStream并通过客户端上载到S3. 我这样做的是:amazonS3.putObject(awsBucketName, awsBucketKey , resource.getInputStream(), new ObjectMetadata());

With this setup you will write your Item-Information you recieve from your database and write it to your Customresource via an OutputStream. The filled resource then can be used in one of your Steps zu open an InputStream and upload to S3 via Client. I did it with: amazonS3.putObject(awsBucketName, awsBucketKey , resource.getInputStream(), new ObjectMetadata());

我的解决方案可能不是完美的方法，但是从现在开始您可以对其进行优化.

My solution may be not the perfect aproach, but from here on you can optimize it.

这篇关于Spring批处理文件编写器无需PutObjectRequest即可直接写入亚马逊S3存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！