是否有任何现有API可以通过JAVA读写HDFS？ [英] Is there any existing API to read and write to HDFS from JAVA

问题描述

是否存在任何存在的HDFS读取/写入API以及如何实现它的最佳实践。下面是代码片段

  import org.apache.hadoop.conf.Configuration; 
导入org.apache.hadoop.fs.FileSystem; 
导入org.apache.hadoop.fs.Path; 
 import org.apache.hadoop.io.IOUtils; 
 import org.apache.hadoop.util.Progressable; 
 
 
 
 String Dest =/user/pkumar/test.xml; 
 
配置conf = new Configuration（）; 
 FileSystem fs = FileSystem.get（URI.create（Dest），conf）; 
路径路径=新路径（Dest）; 
 
 if（！fs.exists（path））{
 OutputStream out = fs.create（path，new Progressable（）{
 public void progress（）{
 System.out.print（。）; 
} 
}）; 
 System.out.println（）; 
 IOUtils.copyBytes（connect，out，4096，true）; 
 
 
 
 $ b $ p $在你的pom.xml中使用下面的依赖关系
 
 
 <依赖关系> 
< dependency> 
< groupId> org.apache.hadoop< / groupId> 
< artifactId> hadoop-common< / artifactId> 
< version> 2.3.0-cdh5.1.3< / version> 
< /依赖关系> 
< dependency> 
< groupId> org.apache.hadoop< / groupId> 
< artifactId> hadoop-client< / artifactId> 
< version> 2.3.0-cdh5.1.3< / version> 
< /依赖关系> 
< dependency> 
< groupId> org.apache.httpcomponents< / groupId> 
< artifactId> httpcore< / artifactId> 
< version> 4.3.2< / version> 
< /依赖关系> 
< dependency> 
< groupId> org.apache.httpcomponents< / groupId> 
< artifactId> httpclient< / artifactId> 
< version> 4.3.2< / version> 
< /依赖关系> 
 
< /依赖关系> 
  
 
Is there any existing API that exists to read/write from HDFS, along with best practices of how to implement it .
解决方案
Below is the code snippet
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.Progressable;



String Dest = "/user/pkumar/test.xml";

 Configuration conf = new Configuration();
       FileSystem fs = FileSystem.get(URI.create(Dest),conf);
        Path path = new Path(Dest);

    if(!fs.exists(path)){
        OutputStream out = fs.create(path, new Progressable(){
            public void progress(){
                System.out.print(".");
            }
        });
        System.out.println();
        IOUtils.copyBytes(connect, out, 4096, true);
    }
use the below dependencies in your pom.xml
<dependencies>
<dependency>
     <groupId>org.apache.hadoop</groupId>
     <artifactId>hadoop-common</artifactId>
     <version>2.3.0-cdh5.1.3</version>
</dependency>
<dependency>
     <groupId>org.apache.hadoop</groupId>
     <artifactId>hadoop-client</artifactId>
     <version>2.3.0-cdh5.1.3</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpcore</artifactId>
    <version>4.3.2</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.3.2</version>
</dependency>

</dependencies>


                        
这篇关于是否有任何现有API可以通过JAVA读写HDFS？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！