谷歌BigQuery删除行？ [英] Google BigQuery Delete Rows?

问题描述

任何人都知道有计划添加对从Google BigQuery中的表中删除部分数据的支持？我们现在面临的问题是，我们正在使用它来分析我们随着时间收集的数据点。我们希望在最近X天的数据上运行查询，但在数据的最后X天之后，我们不再需要将数据存储在BigQuery中。

现在我们可以想到的删除数据的唯一方法是删除整个数据表，然后重新创建并加载X数据天数。这虽然要求我们将数据存储在每日CSV文件中，但这不是最佳选择。

有关如何处理此问题的任何建议，或者在不久的将来会有删除行查询？

解决方案

2016年更新：BigQuery现在可以删除和更新行 - Fh

https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax

---

感谢您描述您的使用案例。 BigQuery只能通过设计追加。我们目前不支持从现有数据集中删除单行或一批行。

目前，要实现旋转日志系统，您必须：
1.每天创建一个新表（如果需要删除旧表）
2.将数据追加到表中并按时间/日期查询

我实际上建议每天创建一个新表。由于BigQuery通过查询的数据量进行收费，因此这对您而言是最经济的，而不是每次都需要查询整个大型数据集。你正在收集你的数据吗？

Anyone know of any plans to add support for delete parts of data from a table in Google Bigquery? The issue we have right now is we are using it for analytics of data points we collect over time. We want to run the queries over the last X days of data, however after the last X days of data we no longer need to store the data in BigQuery.

The only way currently we can think of to delete the data would be to delete the entire table of data, then recreate it and load it with X number of days of data. This would though require us to store our data in daily CSV files too, which isn't optimal.

Any recommendations on how to handle this issue or if there is a delete rows query coming in the near future?

解决方案

2016 update: BigQuery can delete and update rows now -- Fh

https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax

---

Thanks for describing your use case. BigQuery is append-only by design. We currently don't support deleting single rows or a batch of rows from an existing dataset.

Currently, to implement a "rotating" log system you must either: 1. Create a new table each day (and delete older tables if that is necessary) 2. Append your data to a table and query by time/date

I would actually recommend creating a new table for each day. Since BigQuery charges by amount of data queried over, this would be most economical for you, rather than having to query over entire massive datasets every time.

By the way - how are you currently collecting your data?

这篇关于谷歌BigQuery删除行？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！