如何在 Hadoop 程序的映射器中获取输入文件名? [英] How to get the input file name in the mapper in a Hadoop program?

问题描述

如何在映射器中获取输入文件的名称?我有多个输入文件存储在输入目录中，每个映射器可能读取不同的文件，我需要知道映射器读取了哪个文件.

How I can get the name of the input file within a mapper? I have multiple input files stored in the input directory, each mapper may read a different file, and I need to know which file the mapper has read.

推荐答案

首先你需要得到输入拆分，使用更新的 mapreduce API 可以如下完成:

First you need to get the input split, using the newer mapreduce API it would be done as follows:

context.getInputSplit();

但为了获得文件路径和文件名，您需要先将结果类型转换为 FileSplit.

But in order to get the file path and the file name you will need to first typecast the result into FileSplit.

因此，为了获取输入文件路径，您可以执行以下操作:

So, in order to get the input file path you may do the following:

Path filePath = ((FileSplit) context.getInputSplit()).getPath();
String filePathString = ((FileSplit) context.getInputSplit()).getPath().toString();

同样，要获取文件名，您可以调用 getName()，如下所示:

Similarly, to get the file name, you may just call upon getName(), like this:

String fileName = ((FileSplit) context.getInputSplit()).getPath().getName();

这篇关于如何在 Hadoop 程序的映射器中获取输入文件名?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！