从hdfs到SQL的sqoop导出失败，需要帮助以转义字符 [英] sqoop export from hdfs to mysql failed, need help to escape the characters

问题描述

我在hdfs中有一个文件，并将其导出到sqoop表.请在下面找到日志详细信息:

I have a file in hdfs and exporting it to sqoop table. please find the log details below:

Caused by: java.lang.RuntimeException: Can't parse input data: ' characters'
    at tags.__loadFromFields(tags.java:335)
    at tags.parse(tags.java:268)
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89)
    ... 10 more

sqoop导出命令

sqoop export \
--connect "**************************************" \
--username=**** \
--password=***** \
--table tags \
--export-dir /user/cloudera/movie_lens/tags_no_header.csv \
--batch \
--input-lines-terminated-by '\n' \
--input-fields-terminated-by ',' \
--num-mappers 9 \

表结构:

create table tags
    (userId integer
    ,movieId integer
    ,tag varchar(150)
    ,timestamp decimal
    );

记录失败: 660,260，想象世界，人物，故事，哲学" ，1436680217

Record failing : 660,260,"imaginary world, characters, story, philosophical",1436680217

根据我的理解，此操作失败是因为字符串中间的逗号'，'引起的模棱两可的解析.

As per my understanding It's failing because of ambiguous parsing caused by comma ',' in middle of string.

在这种情况下，请帮助我了解--input-enclosed-by和--input-escaped-by参数的用法，还是有其他解决方案.

Please help me to understand usage of --input-enclosed-by and --input-escaped-by arguments in this case or Is there any other solution.

推荐答案

我已使用--input-optionally-enclosed-by属性对其进行了解析. 导出命令:

I have got it resolved using attribute --input-optionally-enclosed-by. export command :

sqoop export \
--connect "jdbc:mysql://quickstart.cloudera:3306/movie_lens_db" \
--username=root \
--password=cloudera \
--table tags \
--export-dir /user/cloudera/escape_by_test.txt \
--batch \
--input-lines-terminated-by '\n' \
--input-fields-terminated-by ',' \
--input-optionally-enclosed-by '\"' \
--num-mappers 1 \
--outdir java_files

表格数据:

+--------+---------+---------------------------------------------------+------------+
| userId | movieId | tag                                               | timestamp  |
+--------+---------+---------------------------------------------------+------------+
|    660 |     260 | imaginary world, characters, story, philosophical | 1436680217 |
|    212 |   69712 | genuine characters                                | 1260688086 |
+--------+---------+---------------------------------------------------+------------+

这篇关于从hdfs到SQL的sqoop导出失败，需要帮助以转义字符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！