java.lang.UnsupportedOperationException：在FileSystem.get（）期间未由DistributedFileSystem FileSystem实现实现 [英] java.lang.UnsupportedOperationException: Not implemented by the DistributedFileSystem FileSystem implementation during FileSystem.get()

问题描述

请找到附加的代码片段。我使用这段代码从hdfs下载文件到我的本地文件系统 -

Please find attached a code snippet. I am using this code to download files from hdfs to my local file system -

    Configuration conf = new Configuration();

    FileSystem hdfsFileSystem = FileSystem.get(conf);

    Path local = new Path(destinationPath);
    Path hdfs = new Path(sourcePath);

    String fileName = hdfs.getName();

    if (hdfsFileSystem.exists(hdfs))
    {
        hdfsFileSystem.copyToLocalFile(false, hdfs, local, true);
        logger.info("File " + fileName + " copied to local machine on location: " + destinationPath);
    }
    else
    {
        logger.error("File " + fileName + " does not exist on HDFS on location: " + sourcePath);
    }

运行这个命令会产生以下输出 -

Running this gives the following output -

Exception in thread "main" java.lang.UnsupportedOperationException: Not implemented by the DistributedFileSystem FileSystem implementation
at org.apache.hadoop.fs.FileSystem.getScheme(FileSystem.java:217)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2624)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2634)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
at com.flipkart.ads.importer.HdfsToLocalImporter.importFile(HdfsToLocalImporter.java:35)
at com.flipkart.ads.importer.HdfsToLocalImporter.main(HdfsToLocalImporter.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

Process finished with exit code 1

推荐答案

没有跨过我的脑海。这是一个瓶子不匹配问题。 Hadoop-code和Hadoop-commons提供了相同的jar包，并且我已经包含了这两个依赖项。

Didn't cross my mind. It was a jar mismatch issue. Hadoop-code and Hadoop-commons provide the same jars and I had included both the dependencies.

这篇关于java.lang.UnsupportedOperationException：在FileSystem.get（）期间未由DistributedFileSystem FileSystem实现实现的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！