Spark-读取带引号的csv文件 [英] Spark - Read csv file with quote

问题描述

我有一个CSV文件，其中的数据包含在双引号()中.

I have a CSV file which has data contained in double quotes (").

"0001"，"A"，"001"，"2017/01/01 12"

"0001", "A", "001", "2017/01/01 12"

"0001"，"B"，"002"，"2017/01/01 13"

"0001", "B", "002", "2017/01/01 13"

我只想读取纯数据(不带符号").

I would like to read only pure data (without " symbol).

spark.read
 .option("encoding", encoding)
 .option("header", header)
 .option("quote", quote)
 .option("sep", sep)

其他选项效果很好，但仅引用似乎无法正常工作.它使用引号()加载.我该如何从加载的数据中删除该符号.

Other options work well, but only quote seems not work properly. It load with quote symbol ("). How should I take this symbol off from loaded data.

dataframe.show结果

dataframe.show result

+----+----+------+---------------+
| _c0| _c1|   _c2|             _c3|
+----+----+------+---------------+
|0001| "A"| "001"| "2017/01/01 12"|
|0001| "B"| "002"| "2017/01/01 13"|
+----+----+------+---------------+

推荐答案

您可以使用以下选项报价

You can use option quote as below

option("quote", "\"")

如果两个数据之间有多余的空格，例如"abc"，"xyz"，则需要使用

If you have an extra space between your two data as "abc", "xyz", than you need to use

option("ignoreLeadingWhiteSpace", true)

希望这会有所帮助

这篇关于Spark-读取带引号的csv文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！