将 csv 数据加载到 Hive 表中时出错 [英] Error loading csv data into Hive table
问题描述
我在 hadoop 中有一个 csv 文件,我有一个 Hive 表,现在我想将该 csv 文件加载到这个 Hive 表中
i have a csv file in hadoop and i have a Hive table ,now i want to laoad that csv file into this Hive table
我已经使用 load LOAD DATA local 'path/to/csv/file' overwrite INTO TABLE tablename;
i have used load LOAD DATA local 'path/to/csv/file' overwrite INTO TABLE tablename;
最终出现此错误:
Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", :
Unable to retrieve JDBC result set for LOAD DATA local
'path/to/csv/file' overwrite INTO TABLE tablename
(Error while processing statement: FAILED:
ParseException line 1:16 missing INPATH at ''path/tp csv/file'' near '<EOF>'
)
注意:我正在尝试使用 r 中的 RJDBC 连接
Note: i am trying this using RJDBC connection in r
推荐答案
我开发了一个工具来从 csv 文件生成 hive 脚本.以下是有关如何生成文件的几个示例.工具 -- https://sourceforge.net/projects/csvtohive/?source=directory
I have developed a tool to generate hive scripts from a csv file. Following are few examples on how files are generated. Tool -- https://sourceforge.net/projects/csvtohive/?source=directory
使用 Browse 选择一个 CSV 文件并设置 hadoop 根目录,例如:/user/bigdataproject/
Select a CSV file using Browse and set hadoop root directory ex: /user/bigdataproject/
工具使用所有 csv 文件生成 Hadoop 脚本,以下是一个示例生成的 Hadoop 脚本将 csv 插入到 Hadoop 中
Tool Generates Hadoop script with all csv files and following is a sample of generated Hadoop script to insert csv into Hadoop
#!/bin/bash -v
hadoop fs -put ./AllstarFull.csv /user/bigdataproject/AllstarFull.csv
hive -f ./AllstarFull.hive
hadoop fs -put ./Appearances.csv/user/bigdataproject/Appearances.csvhive -f ./Appearances.hive
hadoop fs -put ./Appearances.csv /user/bigdataproject/Appearances.csv
hive -f ./Appearances.hive
hadoop fs -put ./AwardsManagers.csv/user/bigdataproject/AwardsManagers.csvhive -f ./AwardsManagers.hive
hadoop fs -put ./AwardsManagers.csv /user/bigdataproject/AwardsManagers.csv hive -f ./AwardsManagers.hive
生成的 Hive 脚本示例
Sample of generated Hive scripts
CREATE DATABASE IF NOT EXISTS lahman;
USE lahman;
CREATE TABLE AllstarFull (playerID string,yearID string,gameNum string,gameID string,teamID string,lgID string,GP string,startingPos string) row format delimited fields terminated by ',' stored as textfile;
LOAD DATA INPATH '/user/bigdataproject/AllstarFull.csv' OVERWRITE INTO TABLE AllstarFull;
SELECT * FROM AllstarFull;
谢谢维杰
这篇关于将 csv 数据加载到 Hive 表中时出错的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!