如何从Amazon Athena查询镶木地板数据？ [英] How to Query parquet data from Amazon Athena?

问题描述

Athena使用S3表中的字段创建一个临时表。我已经使用JSON数据完成了此操作。您能帮我如何使用镶木地板数据创建表格吗？

Athena creates a temporary table using fields in S3 table. I have done this using JSON data. Could you help me on how to create table using parquet data?

我尝试了以下操作：

1. 将示例JSON数据转换为镶木地板数据。


2. 将镶木地板数据上传至S3。


3. 使用JSON数据列创建临时表。

这样做可以执行查询，但结果为空。

By doing this I am able to a execute query but the result is empty.

这种方法正确还是对镶木地板数据要遵循其他方法？

Is this approach right or is there any other approach to be followed on parquet data?

样本json数据：

{"_id":"0899f824e118d390f57bc2f279bd38fe","_rev":"1-81cc25723e02f50cb6fef7ce0b0f4f38","deviceId":"BELT001","timestamp":"2016-12-21T13:04:10:066Z","orgid":"fedex","locationId":"LID001","UserId":"UID001","SuperviceId":"SID001"},
{"_id":"0899f824e118d390f57bc2f279bd38fe","_rev":"1-81cc25723e02f50cb6fef7ce0b0f4f38","deviceId":"BELT001","timestamp":"2016-12-21T13:04:10:066Z","orgid":"fedex","locationId":"LID001","UserId":"UID001","SuperviceId":"SID001"}

推荐答案

如果您的数据已成功存储在Parquet格式，然后您将创建一个引用这些文件的表定义。

If your data has been successfully stored in Parquet format, you would then create a table definition that references those files.

以下是使用Parquet文件的示例语句：

Here is an example statement that uses Parquet files:

CREATE EXTERNAL TABLE IF NOT EXISTS elb_logs_pq (
  request_timestamp string,
  elb_name string,
  request_ip string,
  request_port int,
  ...
  ssl_protocol string )
PARTITIONED BY(year int, month int, day int) 
STORED AS PARQUET
LOCATION 's3://athena-examples/elb/parquet/'
tblproperties ("parquet.compress"="SNAPPY");

此示例摘自AWS博客文章使用Amazon Athena分析S3中的数据，它在解释在Amazon Athena中使用压缩和分区数据的好处。

This example was taken from the AWS blog post Analyzing Data in S3 using Amazon Athena that does an excellent job of explaining the benefits of using compressed and partitioned data in Amazon Athena.

这篇关于如何从Amazon Athena查询镶木地板数据？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！