hdfs中的文件路径 [英] file path in hdfs

问题描述

我想从 Hadoop 文件系统中读取文件.

I want to read the file from the Hadoop File System.

为了实现文件的正确路径，我需要hdfs的主机名和端口地址.

In order to achieve the correct path of the file, I need host name and port address of the hdfs.

所以最后我的文件路径看起来像

so finally my path of the file will look something like

Path path = new Path("hdfs://123.23.12.4344:9000/user/filename.txt")

现在我想知道提取 HostName = "123.23.12.4344" &端口:9000?

Now I want to know to extract the HostName = "123.23.12.4344" & port: 9000?

基本上，我想访问 Amazon EMR 上的文件系统，但是当我使用

Basically, I want to access the FileSystem on Amazon EMR but, when I use

 FileSystem fs = FileSystem.get(getConf());

我得到

 
You possibly called FileSystem.get(conf) when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your path

所以我决定使用 URI.(我必须使用 URI)但我不确定如何访问 URI.

So I decided to use URI. (I have to use URI) but I am not sure how to access the URI.

推荐答案

您可以使用这两种方法中的任何一种来解决您的错误.

You can use either of the two ways to solve your error.

1.

String infile = "file.txt";
Path ofile = new Path(infile);
FileSystem fs = ofile.getFileSystem(getConf());

2.

Configuration conf = getConf();
System.out.println("fs.default.name : - " + conf.get("fs.default.name"));
// It prints uri  as : hdfs://10.214.15.165:9000 or something
String uri = conf.get("fs.default.name");
FileSystem fs = FileSystem.get(uri,getConf());

这篇关于hdfs中的文件路径的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！