BigQuery：使用GROUP EACH BY时响应太大而无法返回 [英] BigQuery: Response too large to return when using GROUP EACH BY

问题描述

我使用的表格有3个字段：

  userId，时间戳记，版本
  

我运行以下查询：

  select userid，MAX（version）as current_version FROM my_table GROUP EACH BY userId; 
  

我得到的回应是：

errors：[
 {
reason：responseTooLarge，
message：响应太大而无法返回。 
 
 $ / code> 

表的大小为644MB，它有12,279,432行。

我认为 GROUP EACH BY 没有结果大小限制，因为它分布在多个节点上。无论如何，我能做些什么呢？根据评论，用户群超过1700万行？这意味着查询响应将至少有1700万行，结果太大而无法处理。

正确的查询将取决于您的目标是什么。你真的想得到1700万行答案？或者你只关心特定用户的最大（版本）？

例如：

  SELECT userid，MAX（version）AS current_version $ b $ FROM my_table 
 WHERE userId IN（'user1'，'user2'，...）
 GROUP BY用户名; 
  

The table I am working with has 3 fields:

userId, timestamp, version

I am running the following query:

select  userid, MAX(version) as current_version FROM  my_table GROUP EACH BY userId;

The response I get is:

"errors": [
 {
  "reason": "responseTooLarge",
  "message": "Response too large to return."
 }

The size of the table is 644MB and it has 12,279,432 rows.

I thought GROUP EACH BY does not have the result size restrictions because it is distributed across multiple nodes. Anyway, What can I do about it?

解决方案

According to the comments, the user base is over 17 million rows? This means the query response will have at least 17 million rows, a result too large to handle.

The right query will depend on what your goal is. Do you really want to get a 17 million row answer? Or you only care about the max(version) for a particular set of users?

For example:

SELECT userid, MAX(version) AS current_version
FROM my_table
WHERE userId IN('user1', 'user2', ...)
GROUP BY userId;

这篇关于BigQuery：使用GROUP EACH BY时响应太大而无法返回的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！