如何从S3阅读斯纳皮的COM pressed文件中的Java [英] How to read Snappy Compressed file from S3 in Java

问题描述

目前，我们正在运行麻preduce工作在Hadoop中，其中输出为com pressed到SnappyCom pression。然后，我们正在输出文件到S3。现在我想通过Java读取S3在COM pressed文件。

Currently we are running MapReduce job in Hadoop in which the output is compressed into SnappyCompression. Then we are moving the output file to S3. Now I want to read the Compressed file from S3 through Java.

推荐答案

我找到了答案，从S3阅读活泼融为一体pressed文件。首先，你应该从S3对象内容。然后DECOM preSS文件。

I found the answer to read snappy compressed file from S3. First you should get the object content from S3. And then decompress the file.

    S3Object s3object = s3Client.getObject(new GetObjectRequest(bucketName,Path));
    InputStream inContent = s3object.getObjectContent();
    CompressionCodec codec = (CompressionCodec) ReflectionUtils.newInstance(SnappyCodec.class, new Configuration());
    InputStream inStream = codec.createInputStream(new BufferedInputStream(inContent));
    InputStreamReader  inRead = new InputStreamReader(inStream);
    BufferedReader br = new BufferedReader(inRead);
    String line=null;
    while ((line = br.readLine()) != null){
        system.out.println(line);
    }   

这篇关于如何从S3阅读斯纳皮的COM pressed文件中的Java的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！