SQOOP --where 不适用于 --query [英] SQOOP --where is not working with --query

查看:93
本文介绍了SQOOP --where 不适用于 --query的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

WHERE 子句不适用于 --where--query

The WHERE clause is NOT WORKING with --where along with --query

sqoop import --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username "retail_dba" --password "cloudera" --target-dir "/user/cloudera/sqoop_import_tables/departments21" --query "select * From orders where \$CONDITIONS" --where "order_id <8000" --split-by order_id;

日志显示WHERE ( order_id <8000 )" - BoundingValsQuery 中的条件被忽略:-

Logs shows "WHERE ( order_id < 8000 )" - where condition is ignored in BoundingValsQuery:-

16/08/31 12:20:26 INFO db.DBInputFormat: Using read commited transaction isolation
16/08/31 12:20:26 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(order_id), MAX(order_id) FROM (select * From orders where  (1 = 1) ) AS t1
16/08/31 12:20:26 INFO db.IntegerSplitter: Split size: 17220; Num splits: 4 from: 1 to: 68883
16/08/31 12:20:26 DEBUG db.IntegerSplitter: Splits: [                           1 to                       68,883] into 4 parts
16/08/31 12:20:26 DEBUG db.IntegerSplitter:                            1
16/08/31 12:20:26 DEBUG db.IntegerSplitter:                       17,222
16/08/31 12:20:26 DEBUG db.IntegerSplitter:                       34,443
16/08/31 12:20:26 DEBUG db.IntegerSplitter:                       51,663
16/08/31 12:20:26 DEBUG db.IntegerSplitter:                       68,883
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 1' and upper bound 'order_id < 17222'
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 17222' and upper bound 'order_id < 34443'
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 34443' and upper bound 'order_id < 51663'
16/08/31 12:20:26 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound 'order_id >= 51663' and upper bound 'order_id <= 68883'
16/08/31 12:20:26 INFO mapreduce.JobSubmitter: number of splits:4
16/08/31 12:20:26 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1472622587119_0024

WHERE 子句适用于 --where--table.

The WHERE clause is WORKING fine with --where along with --table.

sqoop import --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username "retail_dba" --password "cloudera" --target-dir "/user/cloudera/sqoop_import_tables/departments22" --table orders --where "order_id <8000" --split-by order_id

日志显示WHERE ( order_id <8000 )" - 条件得到满足.

Logs shows "WHERE ( order_id < 8000 )" - where condition is honoured.

16/08/31 12:34:35 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_id`), MAX(`order_id`) FROM `orders` WHERE ( order_id < 8000 )
16/08/31 12:34:35 INFO db.IntegerSplitter: Split size: 1999; Num splits: 4 from: 1 to: 7999
16/08/31 12:34:35 DEBUG db.IntegerSplitter: Splits: [                           1 to                        7,999] into 4 parts
16/08/31 12:34:35 DEBUG db.IntegerSplitter:                            1
16/08/31 12:34:35 DEBUG db.IntegerSplitter:                        2,001
16/08/31 12:34:35 DEBUG db.IntegerSplitter:                        4,001
16/08/31 12:34:35 DEBUG db.IntegerSplitter:                        6,000
16/08/31 12:34:35 DEBUG db.IntegerSplitter:                        7,999
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 1' and upper bound '`order_id` < 2001'
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 2001' and upper bound '`order_id` < 4001'
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 4001' and upper bound '`order_id` < 6000'
16/08/31 12:34:35 DEBUG db.DataDrivenDBInputFormat: Creating input split with lower bound '`order_id` >= 6000' and upper bound '`order_id` <= 7999'
16/08/31 12:34:35 INFO mapreduce.JobSubmitter: number of splits:4

  1. 任何逻辑原因,为什么 --where 选项不适用于 --query--table ?立>
  2. --query选项的情况下,我们如何使用--where来提供额外的过滤?
  3. 感谢您提供与此相关的任何文档.
  1. Any logcal reason, why the --where option is not working with --query and working with --table?
  2. In case of --query option, how can we use --where to provide additional filtering?
  3. Any documentation around this is appreciated.

推荐答案

您始终可以在自由格式查询内部进行过滤,例如,

You can always do filtering inside of the freeform query, e.g.,

... --query "select ... where order_id < 8000 and \$CONDITIONS" ...

当然,在这里使用 query 选项是不必要的,因为无论如何您都在选择所有行.因此,您不妨删除查询部分并包含 --where "order_id < 8000".

Of course, using the query option here is unnecessary, because you're selecting all rows anyway. So you might as well just drop the query part and include --where "order_id < 8000".

基本上,它是一个或另一个(--query--where);使用两者没有意义.有关详细信息,请参阅文档.

Basically, it's one or the other (--query or --where); using both doesn't make sense. See the documentation for more information.

这篇关于SQOOP --where 不适用于 --query的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆