BigQuery性能：这是否正确？ [英] BigQuery performance: Is this correct?

问题描述

伙计们，我使用BigQuery作为我的分析查询的超高速数据库，但我对它的性能感到非常失望。

让我告诉你数字：

• 在from子句中只有一个表格


• 选择大约15个字段， ，大约5个带有SUM（）的字段


• 总表行数：3.7百万


• 返回的总行数：830K
/ ul>



当我在BigQuery的控制台上执行这个查询时，大约需要1分钟来处理。这适合你吗？我预计它会在大约2秒钟内返回......如果我在列式数据库（如Sybase IQ）上执行此查询，它只需不到2秒。

解决方案

Big Query是一个高度可扩展的数据库，在成为超级快速数据库之前。它的目的是处理大量的数据，使用名为Dremel的技术在几台不同的机器间分配处理数据。因为它旨在使用多台机器和并行处理，所以您应该期望具有超级可扩展性和良好的性能。

例如：分析所有维基百科修订版的5- 10秒不坏，是吗？但即使是更小的表也需要大约相同的时间。



Sybase IQ通常安装在单个数据库中，并且不使用Dremel。也就是说，在很多情况下，它会比Big Query更快......按照设计。





干杯！




Folks, I'm using BigQuery as a superfast database for my analytics queries, but I'm very disappointed with its performance.

Let me show you the numbers:

• Just one Table at "from" clause
• Select about 15 fields with group by each, about 5 fields with SUM()
• Total table rows: 3.7 millions
• Total rows returned: 830K

When I execute this query on BigQuery's console, it takes about 1 minute to process. Is this ok for you? I was expecting that it will return in about 2 seconds... If I execute this query on a columnar database, like Sybase IQ, it takes less than 2 seconds.

解决方案

Big Query is a highly scalable database, before being a "super fast" database. It's designed to process HUGE amount of data distributing the processing among several different machines using a technique named Dremel. Because it's designed to use several machines and parallel processing, you should expect to have super-scalability with a good performance.

For example: analyzing all the wikipedia revisions in 5-10 seconds isn't bad, is it? But even a much smaller table would take about the same time.

Sybase IQ is often installed in a single database and it doesn't use Dremel. That said, it's going to be faster than Big Query in many scenarios...as designed.

Cheers!

这篇关于BigQuery性能：这是否正确？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！