如何使用加载命令在猪的文件夹中加载多个文本文件? [英] how to load multiple text files in a folder in pig using load command?

问题描述

我一直在用它来加载一个文本文件

I have been using this for loading one text file

A = LOAD '1try.txt' USING PigStorage(' ') as (c1:chararray,c2:chararray,c3:chararray,c4:chararray);

推荐答案

你可以使用文件夹名代替文件名，像这样:

You can use folder name instead of file name, like this:

A = LOAD 'myfolder' USING PigStorage(' ') 
    AS (c1:chararray,c2:chararray,c3:chararray,c4:chararray);

Pig 将加载指定文件夹中的所有文件，如编程中所述猪:

Pig will load all files in the specified folder, as stated in Programming Pig:

当指定从 HDFS 读取的文件"时，您可以指定目录.在这种情况下，Pig 将查找您指定的目录下的所有文件，并将它们用作该加载语句的输入.因此，如果您有一个包含今天和昨天两个数据文件的目录输入，并且您指定输入作为要加载的文件，Pig 将读取今天和昨天作为输入.如果您指定的目录有其他目录，则这些目录中的文件也会被包含在内.

When specifying a "file" to read from HDFS, you can specify directories. In this case, Pig will find all files under the directory you specify and use them as input for that load statement. So, if you had a directory input with two datafiles today and yesterday under it, and you specified input as your file to load, Pig will read both today and yesterday as input. If the directory you specify has other directories, files in those directories will be included as well.

这篇关于如何使用加载命令在猪的文件夹中加载多个文本文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！