获取“查询太大”在BigQuery中 [英] Getting "Query too large" in BigQuery

问题描述

我将事件数据存储在BigQuery中，按日划分 - 每天一张表。以下查询失败：

  select count（distinct event）
 from TABLE_DATE_RANGE（my_dataset.my_dataset_events_，SEC_TO_TIMESTAMP（1391212800） ，SEC_TO_TIMESTAMP（1393631999））
  

每张表大小约为8GB。

有其他人遇到过这个错误吗？似乎它受限于表的大小，因为在这个查询中，我只将它限制在一列中。当我使用较小的时间范围时，它可以工作..但使用BigQuery的关键是支持大数据集。

解决方案

在这种情况下，查询太大意味着TABLE_RANGE在内部扩展到太多的表，产生一个内部查询太大而无法处理。

有两种解决方法：

• 查询较少的表（可以将这些表汇总为一个更大的表）？


• 等到BQ团队在内部解决此问题。而不是使用解决方法，您应该能够不改变地运行此查询。只是不是今天：）。

I am storing event data in BigQuery, partitioned by day - one table per day. The following query failed:

select count(distinct event) 
from TABLE_DATE_RANGE(my_dataset.my_dataset_events_, SEC_TO_TIMESTAMP(1391212800), SEC_TO_TIMESTAMP(1393631999))

Each table is about 8GB in size.

Has anyone else experienced this error? Seems like it's limited by table size, because in this query, I've only limited it to just one column. When I use a smaller time range, it works.. but the whole point of using BigQuery was its support for large datasets.

解决方案

"Query too large" in this case means that the TABLE_RANGE is getting expanded internally to too many tables, generating an internal query that is too large to be processed.

This has 2 workarounds:

• Query less tables (could you aggregate these tables into a bigger one?).
• Wait until the BQ team solves this issue internally. Instead of using a workaround, you should be able to run this query unchanged. Just not today :).

这篇关于获取“查询太大”在BigQuery中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！