将数据从AWS Athena导入RDS实例 [英] Importing data from AWS Athena to RDS instance

问题描述

当前，我正在侦听来自AWS Kinesis的事件并将其写入S3。然后，我使用AWS Glue和Athena查询它们。

Currently I’m listening events from AWS Kinesis and writing them to S3. Then I query them using AWS Glue and Athena.

是否可以通过一些转换将数据导入RDS实例？

Is there a way to import that data, possibly with some transformation, to an RDS instance?

推荐答案

关于该任务有几种通用方法。

There are several general approaches to take with regards to that task.

1. 将数据和Athena查询读取到自定义ETL脚本中（使用 JDBC连接）并加载到数据库中


2. 安装S3存储桶将数据保存到文件系统（可能使用 s3fs-fuse ），然后使用自定义ETL脚本读取数据，并将其推送到RDS实例


3. 使用 AWS CLI 或 SDK 本地，然后推送到RDS


4. 根据您的建议，使用 AWS Glue 将数据从Athena导入到RDS实例。如果您要构建与AWS紧密结合的应用程序，并且您正在使用Kinesis和Athena，那么这样的解决方案就很有意义。

1. Read data from and Athena query into a custom ETL script (using a JDBC connection) and load into the database
2. Mount the S3 bucket holding the data to a file system (perhaps using s3fs-fuse), read the data using a custom ETL script, and push it to the RDS instance(s)
3. Download the data to be uploaded to the RDS instance to a filesystem using the AWS CLI or the SDK, process locally, and then push to RDS
4. As you suggest, use AWS Glue to import the data to from Athena to the RDS instance. If you are building an application that is tightly coupled with AWS, and if you are using Kinesis and Athena you are, then such a solution makes sense.

将GLUE连接到RDS时，要记住两件事（主要是在网络方面：

When connecting GLUE to RDS a couple of things to keep in mind (mostly on the networking side:

1. 确保已启用DNS主机名托管目标RDS实例的VPC


2. 您需要设置自引用规则 li>
   

1. Ensure that DNS Hostnames are enabled the VPC hosting the target RDS instance
2. You'll need to setup a self-referencing rule in the Security Group associated with the target RDS instance

有关面向关系数据库的代码示例，请参见以下教程

For some examples of code targetting a relational database, see the following tutorials

这篇关于将数据从AWS Athena导入RDS实例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！