将Hadoop中的文件导入到Web应用程序中 [英] Getting files in Hadoop into a web application
问题描述
我是Hadoop的新手。现在我正在尝试在eclipse中执行一个应用程序,在该应用程序中我想使用HDFS中的数据。如果我们想用Java连接到数据库,我们有JDBC连接。像这样,我需要做什么才能直接连接到HDFS?
I am new to Hadoop. Right now I am trying to do an application in eclipse in which I want to use data present in HDFS. If we want to connect to database with Java, we have JDBC connection. Like that, what do I need to do to connect to HDFS directly?
推荐答案
在Hadoop中,首先,您必须确保Hadoop已启动并正在运行。 Apache Hadoop提供了Java类--FileSystem来从Java应用程序访问HDFS中的文件。下面是一个示例,我正在使用FileSystem和IOUtils访问/books/pg5000.txt。
In Hadoop, firstly, you would have to make sure that Hadoop is up and running. Apache Hadoop provides Java classes - FileSystem to access the files in HDFS from the Java application. One example is below, I am accessing /books/pg5000.txt using FileSystem and IOUtils.
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
public class FileSystemCat {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));
String uri = "/books/pg5000.txt";
FileSystem fs = FileSystem.get(URI.create(uri), conf);
InputStream in = null;
try {
in = fs.open(new Path(uri));
IOUtils.copyBytes(in, System.out, 4096, false);
} finally {
IOUtils.closeStream(in);
}
}
}
这篇关于将Hadoop中的文件导入到Web应用程序中的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!