BigQuery是否可以足够快地进行实时现场请求 [英] Can BigQuery be fast enough for real-time onsite request

问题描述

我正在研究使用BigQuery及其API来根据访问者查看的内容进行现场查询的可能性.因此，响应时间至关重要. 我已经加载了一个非常简单的结构化数据集，该数据集包含1万行(4列)，并运行了一个非常简单的查询，这需要1到2秒钟. 我的问题希望很容易回答，是否可以通过以某种方式优化数据来使用BQ API获得<1秒的响应时间?

I'm looking into the possibility to use BigQuery and its API to do on-site queries depending on content that is viewed by our visitors. Therefore the response time is crucial. I have loaded a very simple structured dataset of 10k rows (4columns) and run a very simple query and that takes between 1 and 2 seconds. My questions is hopefully pretty simple to answer, will I ever be able to get a <1sec response time with the BQ API by optimising the data in someway or not?

非常感谢！

关于， Arjen

推荐答案

BigQuery不是联机事务处理(OLTP)数据库，而是一个交互式分析数据库，可以在几秒钟内扫描数TB的数据.虽然查询时间相当一致，但由于它是一项共享服务，因此无法保证查询时间，即，运行2秒的查询可能在不同的时间段运行1.5秒或3秒.由于BigQuery的性质和内部因素，因此查询时间为<到今天为止，1s是不现实的.

BigQuery is not an online transaction processing (OLTP) database but an interactive analysis database making possible to scan terabytes of data within seconds. While the query time is pretty consistent, since it is a shared service, the query time is not guaranteed, i.e. query running for 2 seconds might run 1.5 seconds or 3 seconds at different periods of time. Due to the nature and internals of BigQuery, query time of < 1s is not realistic as of today.

一种流行的设计模式是让BigQuery对数据进行复杂的分析，然后将结果存储在OLTP(如mySQL)或内存(如Redis)数据库中，然后将结果从那里.您可以通过在后台运行查询来定期更新数据.

One of the popular design patterns is to let BigQuery do the heavy lifting of complex analysis of your data and then storing results in OLTP (like mySQL) or even in-memory (like Redis) database and serve the results to clients from there. You can periodically update the data by running the queries in the background.

这篇关于BigQuery是否可以足够快地进行实时现场请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！