如何加快Amazon Athena查询的执行速度？ [英] How to speed up Amazon Athena query executions?

问题描述

我正在使用Athena Query Execution从胶水表中检索数据。
爬虫每小时使用S3桶（由Kinesis Firehose不断更新）每小时更新一次此表。

I'm using Athena Query Execution to retrieve data from a Glue Table. A Crawler updates this table every hour using a S3 Bucket which is continuously updated by Kinesis Firehose.

我的Node.js服务器使用Athena执行基本查询。但是我意识到某些请求花费的时间太长，导致服务器抛出服务器请求超时。

My Node.js server executes basic queries using Athena. But I realized that some of the requests takes so long that my server throws Server Request Timeout.

我检查了Athena的查询历史记录，并看到了一些最新的请求。状态为已排队，表示它们正在等待执行。它们都有一个小的运行时间，范围是1到5秒。显然，问题不是运行时间导致超时，而是队列。

I checked the Query History in Athena and I saw some of the latest requests' state is Queued which means they are waiting to be executed. They all have a small Run Time in the range of 1 to 5 seconds. It's obvious that the problem is not the Run Time causes timeouts but it's the queue.

如何加快这些查询的执行速度？或者如何增加并发执行限制，以便Athena立即执行它们？

How can I speed up the execution of these queries? or how can I increase concurrent execution limits so Athena immediately executes them?

推荐答案

您可以联系AWS支持以增加并发活动查询限制， BUT不会影响/减少已排队状态

You can contact AWS support to increase the concurrent active queries limit, BUT that will not affect/decrease the **Queued** state

按定义，排队状态表示查询已提交给服务，并且只要有可用资源，Athena就会执行查询。这里的资源是指雅典娜资源，而不是您的。 https://docs.aws.amazon.com/athena/latest/ APIReference / API_QueryExecutionStatus.html

By definition, Queued state indicates that the query has been submitted to the service, and Athena will execute the query as soon as resources are available. resources here is refer to Athena resources not yours. https://docs.aws.amazon.com/athena/latest/APIReference/API_QueryExecutionStatus.html

我认为您无法对此队列状态做任何事情。

I think there is nothing you can do about this Queued state.

这篇关于如何加快Amazon Athena查询的执行速度？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！