如何将Hadoop Path对象转换为Java文件对象 [英] How to convert a Hadoop Path object into a Java File object

查看:519
本文介绍了如何将Hadoop Path对象转换为Java文件对象的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

有没有办法将一个有效的和现有的Hadoop Path对象变成一个有用的Java File对象。有没有这样做的好方法,还是我需要大手笔代码提交?更明显的方法不工作,它似乎是一个普通的代码

  void func(Path p ){
if(p.isAbsolute()){
File f = new File(p.toURI());




$ b

这是行不通的,因为Path :: toURI( )返回hdfs标识符和Java的文件(URI URI)构造函数只能识别文件标识符。

有没有办法让路径和文件一起工作?

**



好的,特定的有限范例如何?

  Path [] paths = DistributedCache.getLocalCacheFiles(job); 

DistributedCache应该提供文件的本地化副本,但是它会返回一个Path。我假设分布式缓存使文件的本地副本在同一个磁盘上。鉴于这个有限的例子,其中hdfs希望不是在方程中,有没有办法让我可靠地将一个路径转换成一个文件?



**

解决方案

不是我知道的。

据我的理解, Hadoop中的code> Path 表示分布式文件系统中节点的标识符。这是一个与 java.io.File 不同的抽象,它表示本地文件系统上的一个节点。甚至有一个 File 表示也是不可能的,因为 Path 基本的模式是根本不同的。



因此缺乏翻译。我假定你的断言, File 对象是[more]有用的,你想要这个类的对象来使用现有的库方法吗?由于上述原因,这是不会工作得很好。如果是自己的库,可以用Hadoop Paths重写它,然后将所有文件转换为Path对象(这个方向是Paths是Files的一个严格超集)。如果是第三方库,那么你运气不好。该方法的作者没有考虑到分布式文件系统的影响,只写这个方法来处理普通的旧本地文件。

Is there a way to change a valid and existing Hadoop Path object into a useful Java File object. Is there a nice way of doing this or do I need to bludgeon to code into submission? The more obvious approaches don't work, and it seems like it would be a common bit of code

void func(Path p) {
  if (p.isAbsolute()) {
     File f = new File(p.toURI());
  }
}

This doesn't work because Path::toURI() returns the "hdfs" identifier and Java's File(URI uri) constructor only recognizes the "file" identifier.

Is there a way to get Path and File to work together?

**

Ok, how about a specific limited example.

Path[] paths = DistributedCache.getLocalCacheFiles(job);

DistributedCache is supposed to provide a localized copy of a file, but it returns a Path. I assume that DistributedCache make a local copy of the file, where they are on the same disk. Given this limited example, where hdfs is hopefully not in the equation, is there a way for me to reliably convert a Path into a File?

**

解决方案

Not that I'm aware of.

To my understanding, a Path in Hadoop represents an identifier for a node in their distributed filesystem. This is a different abstraction from a java.io.File, which represents a node on the local filesystem. It's unlikely that a Path could even have a File representation that would behave equivalently, because the underlying models are fundamentally different.

Hence the lack of translation. I presume by your assertion that File objects are "[more] useful", you want an object of this class in order to use existing library methods? For the reasons above, this isn't going to work very well. If it's your own library, you could rewrite it to work cleanly with Hadoop Paths and then convert any Files into Path objects (this direction works as Paths are a strict superset of Files). If it's a third party library then you're out of luck; the authors of that method didn't take into account the effects of a distributed filesystem and only wrote that method to work on plain old local files.

这篇关于如何将Hadoop Path对象转换为Java文件对象的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆