SQOOP --where 不适用于 --query [英] SQOOP --where is not working with --query
问题描述
WHERE 子句不适用于 --where
和 --query
The WHERE clause is NOT WORKING with --where
along with --query
sqoop import --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username "retail_dba" --password "cloudera" --target-dir "/user/cloudera/sqoop_import_tables/departments21" --query "select * From orders where \$CONDITIONS" --where "order_id <8000" --split-by order_id;
日志显示WHERE ( order_id <8000 )" - BoundingValsQuery 中的条件被忽略:-
Logs shows "WHERE ( order_id < 8000 )" - where condition is ignored in BoundingValsQuery:-
16/08/31 12:20:26 INFO db.DBInputFormat: Using read commited transaction isolation
16/08/31 12:20:26 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(order_id), MAX(order_id) FROM (select * From orders where (1 = 1) ) AS t1
16/08/31 12:20:26 INFO db.IntegerSplitter: Split size: 17220; Num splits: 4 from: 1 to: 68883
16/08/31 12:20:26 DEBUG db.IntegerSplitter: Splits: [ 1 to 68,883] into 4 parts
16/08/31 12:20:26 DEBUG db.IntegerSplitter: 1
16/08/31 12:20:26 DEBUG db.IntegerSplitter: 17,222
16/08/31 12:20:26 DEBUG db.IntegerSplitter: 34,443
16/08/31 12:20:26 DEBUG db.IntegerSplitter: 51,663
16/08/31 12:20:26 DEBUG db.IntegerSplitter: 68,883
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 1' and upper bound 'order_id < 17222'
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 17222' and upper bound 'order_id < 34443'
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 34443' and upper bound 'order_id < 51663'
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 51663' and upper bound 'order_id <= 68883'
16/08/31 12:20:26 INFO mapreduce.JobSubmitter: number of splits:4
16/08/31 12:20:26 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1472622587119_0024
WHERE 子句适用于 --where
和 --table
.
The WHERE clause is WORKING fine with --where
along with --table
.
sqoop import --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username "retail_dba" --password "cloudera" --target-dir "/user/cloudera/sqoop_import_tables/departments22" --table orders --where "order_id <8000" --split-by order_id
日志显示WHERE ( order_id <8000 )" - 条件得到满足.
Logs shows "WHERE ( order_id < 8000 )" - where condition is honoured.
16/08/31 12:34:35 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_id`), MAX(`order_id`) FROM `orders` WHERE ( order_id < 8000 )
16/08/31 12:34:35 INFO db.IntegerSplitter: Split size: 1999; Num splits: 4 from: 1 to: 7999
16/08/31 12:34:35 DEBUG db.IntegerSplitter: Splits: [ 1 to 7,999] into 4 parts
16/08/31 12:34:35 DEBUG db.IntegerSplitter: 1
16/08/31 12:34:35 DEBUG db.IntegerSplitter: 2,001
16/08/31 12:34:35 DEBUG db.IntegerSplitter: 4,001
16/08/31 12:34:35 DEBUG db.IntegerSplitter: 6,000
16/08/31 12:34:35 DEBUG db.IntegerSplitter: 7,999
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 1' and upper bound '`order_id` < 2001'
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 2001' and upper bound '`order_id` < 4001'
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 4001' and upper bound '`order_id` < 6000'
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 6000' and upper bound '`order_id` <= 7999'
16/08/31 12:34:35 INFO mapreduce.JobSubmitter: number of splits:4
- 任何逻辑原因,为什么
--where
选项不适用于--query
和--table
?立> - 在
--query
选项的情况下,我们如何使用--where
来提供额外的过滤? - 感谢您提供与此相关的任何文档.
- Any logcal reason, why the
--where
option is not working with--query
and working with--table
? - In case of
--query
option, how can we use--where
to provide additional filtering? - Any documentation around this is appreciated.
推荐答案
您始终可以在自由格式查询内部进行过滤,例如,
You can always do filtering inside of the freeform query, e.g.,
... --query "select ... where order_id < 8000 and \$CONDITIONS" ...
当然,在这里使用 query
选项是不必要的,因为无论如何您都在选择所有行.因此,您不妨删除查询部分并包含 --where "order_id < 8000"
.
Of course, using the query
option here is unnecessary, because you're selecting all rows anyway. So you might as well just drop the query part and include --where "order_id < 8000"
.
基本上,它是一个或另一个(--query
或 --where
);使用两者没有意义.有关详细信息,请参阅文档.
Basically, it's one or the other (--query
or --where
); using both doesn't make sense. See the documentation for more information.
这篇关于SQOOP --where 不适用于 --query的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!