TBLPROPERTIES('skip.header.line.count'='1') 不适用于从直线与 hive jdbc 1.2.1 连接的 sparkThrift [英] TBLPROPERTIES('skip.header.line.count'='1') is not working on sparkThrift connected from beeline with hive jdbc 1.2.1

问题描述

我正在使用 spark 2.3 并将 sparkThrift 与直线连接.

I am using spark 2.3 and connecting sparkThrift with beeline.

Hive jdbc 版本 1.2.1Spark SQL 2.3.1 版

Hive jdbc version 1.2.1 Spark SQL version 2.3.1

我正在尝试使用跳过标题属性创建外部表，但选择命令总是返回标题为第一行的数据，下面是我的创建查询

I am trying to create external table with skip header property but select command is always returning data with header as first row, below is my create query

CREATE EXTERNAL TABLE datasourcename11(
`retail_invoice_detail_sys_invoice_no` STRING,
`store_id` STRING,
`retail_invoice_detail_invoice_time` STRING,
`retail_invoice_detail_invoice_date` string,
`cust_id` STRING,
`article_code` INTEGER,
`retail_invoice_detail_base_price` INTEGER,
`retail_invoice_detail_sale_price` INTEGER,
`retail_invoice_detail_quantity` DOUBLE,
`retail_invoice_detail_total_amount` DOUBLE
) 
ROW FORMAT DELIMITED  FIELDS TERMINATED BY ',' 
LINES TERMINATED BY '\n'  
LOCATION '/home/java_services/backend/demo/' 
TBLPROPERTIES('skip.header.line.count'=1);

推荐答案

此属性 skip.header.line.count=1 仅在 Hive 中受支持.

This property skip.header.line.count=1 is supported in Hive only.

解决方法是使用过滤器

retail_invoice_detail_sys_invoice_no!=<标题中的列名称>

这篇关于TBLPROPERTIES('skip.header.line.count'='1') 不适用于从直线与 hive jdbc 1.2.1 连接的 sparkThrift的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！