获取BigQuery项目中所有BigQuery表的上次修改日期 [英] Get the Last Modified date for all BigQuery tables in a BigQuery Project

问题描述

我在BigQuery项目中有几个数据库，这些数据库由各种作业引擎和应用程序填充.我想维护我们项目中每个表的所有上次修改日期"的仪表板，以监视作业失败.

I have several databases within a BigQuery project which are populated by various jobs engines and applications. I would like to maintain a dashboard of all of the Last Modified dates for every table within our project to monitor job failures.

是否有任何命令行或SQL命令可以提供上次修改日期的列表?

Are there any command line or SQL commands which could provide this list of Last Modified dates?

推荐答案

对于SQL命令，您可以尝试以下方法:

For a SQL command you could try this one:

#standardSQL
SELECT *, TIMESTAMP_MILLIS(last_modified_time)
FROM `dataset.__TABLES__` where table_id = 'table_id'

我还是建议您查看是否可以在应用程序级别记录这些错误.这样，您还可以了解为什么某些东西无法按预期工作.

I recommend you though to see if you can log these errors at the application level. By doing so you can also understand why something didn't work as expected.

如果您已经在使用GCP，则可以使用 Stackdriver (它也适用于AWS) )，我们开始在项目中使用它，我建议您尝试一下(尽管我们已针对python应用程序进行了测试，但不确定该工具在其他客户端上的性能如何，但可能非常相似).

If you are already using GCP you can make use of Stackdriver (it works on AWS as well), we started using it in our projects and I recommend giving it a try (we tested for python applications though, not sure how the tool performs on other clients but it might be quite similar).

这篇关于获取BigQuery项目中所有BigQuery表的上次修改日期的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！