获取 BigQuery 项目中所有 BigQuery 表的上次修改日期 [英] Get the Last Modified date for all BigQuery tables in a BigQuery Project

问题描述

我在 BigQuery 项目中有多个数据库，这些数据库由各种作业引擎和应用程序填充.我想为我们项目中的每个表维护一个包含所有上次修改日期的仪表板，以监控作业失败.

I have several databases within a BigQuery project which are populated by various jobs engines and applications. I would like to maintain a dashboard of all of the Last Modified dates for every table within our project to monitor job failures.

是否有任何命令行或 SQL 命令可以提供此上次修改日期列表?

Are there any command line or SQL commands which could provide this list of Last Modified dates?

推荐答案

对于 SQL 命令，你可以试试这个:

For a SQL command you could try this one:

#standardSQL
SELECT *, TIMESTAMP_MILLIS(last_modified_time)
FROM `dataset.__TABLES__` where table_id = 'table_id'

我建议您查看是否可以在应用程序级别记录这些错误.通过这样做，您还可以理解为什么某些事情没有按预期工作.

I recommend you though to see if you can log these errors at the application level. By doing so you can also understand why something didn't work as expected.

如果您已经在使用 GCP，则可以使用 Stackdriver(它也适用于 AWS)，我们开始在我们的项目中使用它，我建议尝试一下(虽然我们测试了 python 应用程序，不确定该工具在其他客户端上的表现如何，但它可能非常相似).

If you are already using GCP you can make use of Stackdriver (it works on AWS as well), we started using it in our projects and I recommend giving it a try (we tested for python applications though, not sure how the tool performs on other clients but it might be quite similar).

这篇关于获取 BigQuery 项目中所有 BigQuery 表的上次修改日期的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！