Sqoop增量导入多列检查列 [英] Sqoop Incremental Import multiple columns in check-column

问题描述

如何基于多列从Oracle导入增量数据?

How to import the Incremental data from Oracle based on multiple columns?

sqoop import --connect jdbc:oracle:thin:ip/dbname --username sqoop --password sqoop \
--query 'SELECT * FROM tablename WHERE $CONDITIONS' \
--check-column column1,column2 \
--incremental lastmodified \
--last-value "2010-05-22 01:01:01" \
-fields-terminated-by \| \
--target-dir /directoryname` --split-by id -m 1

当我使用它时，column2 无法识别，并且会引发错误.需要这方面的帮助.

When I use this, column2 is not recognized and it's throwing an error. Need help on this.

推荐答案

这真的取决于 Sqoop Incremental 应该如何处理这两列...

It really depends on what Sqoop Incremental should do with the two columns...

您想获取 column1 大于 last-value 的所有行吗?第 2 列?哪个更大?哪个小一点?

You want to grab all rows where column1 is larger than last-value? column2? whichever is larger? whichever is smaller?

因为 sqoop 不知道你想如何组合它们，所以它不支持多列.

Because sqoop has no idea how you want to combine them, it will not support multiple columns.

但是，您可以自己组合它们并使用函数作为 --check-column.

You can, however, combine them yourself and use a function as --check-column.

例如:--check-column nvl(column1,column2)或者--check-column 最大(column1,column2)

For example: --check-column nvl(column1,column2) or --check-column greatest(column1,column2)

这篇关于Sqoop增量导入多列检查列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！