Sqoop - 我可以将多个 mysql 表批量导入到一个 HBase/Hive 表吗 [英] Sqoop - can I bulk import multiple mysql tables to one HBase/Hive table
问题描述
如果我有多个相似的表，例如:
If I have multiple similar tables, e.g:
表A:"users"，列:user_name、user_id、user_address等
表 B:"customers" 列:customer_name、customer_id、customer_address 等
表C:"employee"列:employee_name、employee_id、employe_address等
是否可以使用Sqoop将三个表导入到一个HBase或Hive表中?所以导入后，我有一个 HBase 表包含表 A、B、C 中的所有记录?
Is it possible that using Sqoop to import the three tables into one HBase or Hive table? So After the import, I have one HBase table contains all the records in table A, B, C ?
推荐答案
如果表格之间存在某种关联，这绝对是可能的.在 Sqoop 中可以使用自由格式的查询来做到这一点.在这种情况下，自由格式查询将是一个连接.例如，导入到 Hive 时:
It's definitely possible if the tables are somehow related. A free-form query can be used in Sqoop to do exactly that. In this case, the free-form query would be a join. For example, when importing into Hive:
sqoop import --connect jdbc:mysql:///mydb --username hue --password hue --query "SELECT * FROM users JOIN customers ON users.id=customers.user_id JOIN employee ON users.id = employee.user_id WHERE \$CONDITIONS" --split-by oozie_job.id --target-dir "/tmp/hue" --hive-import --hive-table hive-table
同样，对于 Hbase:
Similarly, for Hbase:
sqoop import --connect jdbc:mysql:///mydb --username hue --password hue --query "SELECT * FROM users JOIN customers ON users.id=customers.user_id JOIN employee ON users.id = employee.user_id WHERE \$CONDITIONS" --split-by oozie_job.id --hbase-table hue --column-family c1
所有这些的关键要素是提供的 SQL 语句:
The key ingredient in all of this is the SQL statement being provided:
SELECT * FROM users JOIN customers ON users.id=customers.user_id JOIN employee ON users.id = employee.user_id WHERE \$CONDITIONS
有关自由格式查询的更多信息，请查看 http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_free_form_query_imports.
For more information on free-form queries, check out http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_free_form_query_imports.
这篇关于Sqoop - 我可以将多个 mysql 表批量导入到一个 HBase/Hive 表吗的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
