在配置单元中创建外部表时，我可以将该位置指向目录中的特定文件？ [英] When creating an external table in hive can I point the location to specific files in a directory?

问题描述

 创建外部表格PageViews（Userid字符串，Page_View字符串）
由（ds字符串）
分隔的行格式作为以'，'结尾的分隔字段
存储为文本文件位置'/ user / data'; 
  

我不希望/ user / data目录中的所有文件都用作表。是否有可能做到以下几点？

  location'user / data / *。csv'
 
 code> 




解决方案

当我遇到类似的问题需要解决时，我遇到了这个线程。我能够通过使用自定义SerDe来解决它。然后我添加了SerDe属性，它引导RegEx适用于任何特定表格的文件名称模式。

如果您只处理标准CSV文件，自定义SerDe可能看起来过度文件，我有一个更复杂的文件格式来处理。如果你不回避写一些Java的话，这仍然是一个非常可行的解决方案。当你无法重新调整存储位置中的数据，并且在不成比例的大文件集中寻找非常具体的文件模式时，它尤其有用。

 > CREATE EXTERNAL TABLE PageViews（Userid字符串，Page_View字符串）
> ROW FORMAT SERDE'com.something.MySimpleSerDe'
> WITH SERDEPROPERTIES（input.regex=* .csv）
> LOCATION'/ user / data'; 
  

I have defined a table as such:

create external table PageViews (Userid string, Page_View string)
partitioned by (ds string)
row format as delimited fields terminated by ','
stored as textfile location '/user/data';

I do not want all the files in the /user/data directory to be used as part of the table. Is it possible for me to do the following?

location 'user/data/*.csv'

解决方案

I came across this thread when I had a similar problem to solve. I was able to resolve it by using a custom SerDe. I then added SerDe properties which guided what RegEx to apply to the file name patterns for any particular table.

A custom SerDe might seem overkill if you are only dealing with standard CSV files, I had a more complex file format to deal with. Still this is a very viable solution if you don't shy away from writing some Java. It is particularly useful when you are unable to restructure the data in your storage location and you are looking for a very specific file pattern among a disproportionately large file set.

> CREATE EXTERNAL TABLE PageViews (Userid string, Page_View string)  
> ROW FORMAT SERDE 'com.something.MySimpleSerDe' 
> WITH SERDEPROPERTIES ( "input.regex" = "*.csv")
> LOCATION '/user/data';

这篇关于在配置单元中创建外部表时，我可以将该位置指向目录中的特定文件？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！