BigQuery中的服务限制 [英] Service throttling in BigQuery

问题描述

如何判断Google BigQuery内是否发生了限制？

我有几个BigQuery工作可以提升Google Analytics数据，这些数据通常在10-15秒内完成。不同的工作间歇性地开始减慢到2-3分钟，然后在随后的运行中恢复到正常速度。

这些工作正在成功完成。我看不出任何缓慢的模式 - 它不是同一个工作，也不是同时也没有交通流量的激增。

它可能是基于Google BigQuery项目的服务限制吗？有没有什么方法可以在云端控制台中进行监控？

除了服务调节之外，还有其他什么应该考虑吗？

解决方案

你没有被扼杀。相反，您使用的是多租户系统，因此您需要与邻居共享所有资源。这些较慢的工作最有可能发生在集群需求增加的时候。或者，也许只是因为需要加载更多数据。

在底层，加载作业只是普通的联邦查询，除非您不付费。就像查询一样，它们也受到资源/集群需求的限制。

How can I tell if throttling is occurring within Google BigQuery?

I have several BigQuery jobs lifting Google Analytics data which typically complete within 10-15 seconds. Different jobs have intermittently started to slow down to 2-3 minutes, before returning to their normal speed in subsequent runs.

The jobs are completing successfully. I can't see any pattern in the slow down - its not the same job, nor at the same time and there has not been a surge in traffic volume.

Could it be service throttling based on the Google BigQuery project? Is there any way to monitor this in the Cloud Console?

Is there anything else other than service throttling that I should be considering?

解决方案

You're not being throttled. Instead, you're using a multi-tenant system, so you need to share all the resources with your neighbours. Those slower jobs are most likely occurring when there is more demand on the cluster. Or maybe it's just because there's more data to load.

Under the hood, load jobs are just normal federated queries, except you don't pay for them. And just like queries, they too are constrained by the demand on the resources/cluster at any given time.

这篇关于BigQuery中的服务限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！