如果_PARTITIONTIME字段在标准SQL中不可用，我如何在BigQuery中查询流缓冲区 [英] How do I query the streaming buffer in BigQuery if the _PARTITIONTIME field isn't available with Standard SQL

问题描述

我们有一个分区的BigQuery表，该表已在一个自定义字段( event_timestamp )上进行了分区.这是因为需要导入历史数据，同时又要保持分区和查询成本处于受控状态.

We have a partitioned BigQuery table that's partitioned on a custom field (event_timestamp). This was due to needing to import historical data while still keeping things partitioned and query costs under control.

这意味着表没有 _PARTITIONTIME 字段，我们无法执行查询 WHERE _PARTITIONTIME IS NULL 来查看流缓冲区中最近发送的记录

This means the table doesn't have a _PARTITIONTIME field, and we can't do queries WHERE _PARTITIONTIME IS NULL to view recently sent records sitting in the streaming buffer.

那么，即使您使用自定义字段进行分区，也能够查询流缓冲区吗?

So is there a trick to being able to query the streaming buffer even if you're using a custom field for partitioning?

编辑，我们使用的是标准SQL，而不是旧版SQL，很抱歉以前没有对此进行明确介绍.

Edit We're using Standard SQL, not Legacy SQL, sorry for not being explicit about that previously.

推荐答案

尽管仅通过标准SQL支持带有时间分区字段的查询表，但我相信仍支持传统SQL查询流插入表中的信息

Even though querying tables with time partitioning field supports only via standard SQL, I believe Legacy SQL is still supported to query the streaming inserts into the table

#legacySQL
select * from [<project-name>:<data-set>.<table>$__UNPARTITIONED__] 

参考:

https://cloud.google.com/bigquery/docs/partitioned-tables

使用--time_partitioning_field查询流数据

这篇关于如果_PARTITIONTIME字段在标准SQL中不可用，我如何在BigQuery中查询流缓冲区的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！