在Big Query中获取表格创建时间 [英] Getting table creation time in Big Query

查看:88
本文介绍了在Big Query中获取表格创建时间的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

如何获得数据集中表格的创建时间?显示my_project:my_dataset.my_table

给你

 表my_project:my_dataset.my_table 

上次修改的架构总计行数字节到期
----------------- ------------------ --------- --- ------------- ------------
** 16 Oct 14:47:41 ** | - field1:string 3 69
| - field2:string
| - field3:string

我们可以使用最后修改日期,但它错过了一年!还需要有一个神秘的日志来解析日期。

这个元信息是否可以通过任何其他特定的'bq'命令获得?



我正在使用这个信息来确定一个适当的表装饰器,可以在表上使用,因为它好像是装饰器返回4小时(以循环为基础),并且表/分区只存在3小时,查询错误不存在。



理想情况下,如果指定窗口大于now-table creation time,装饰器使用将时间窗口默认为现在 - 表创建时间如果您使用bq --format = json,您可以轻松获得信息:
$ b $
b

  $ bq --format = prettyjson show publicdata:samples.wikipedia 
{
creationTime:1335916132870,...
}

这是在表装饰器中使用的确切值。



虽然我不确定我是否喜欢这个想法非常低的起始值'被解释为表格创建时间,我还有其他选择:


  1. 表格@ 0 表示创建时的表

  2. 表@ 0 表示快照最早时刻的表

我倾向于#2,因为快照只能回溯7天。


How do you get the creation time for a table in the dataset?

bq show my_project:my_dataset.my_table

gives you

Table my_project:my_dataset.my_table

   Last modified         Schema        Total Rows   Total Bytes   Expiration  
 ----------------- ------------------ ------------ ------------- ------------ 
  **16 Oct 14:47:41**   |- field1: string   3            69                        
                    |- field2: string                                          
                    |- field3: string    

We can use the "Last Modified" date but its missing the year!. Also there needs to be a cryptic log applied to parse the date out.

Is this meta information available through any other specific 'bq' based commands?

I am looking to use this information to determine a appropriate table decorator that can be used on the table since it seems like if the decorator is going back 4 hours (on recurring basis) and the table/partition has existed for only 3hrs the query errors out.

Ideally it would be nice if the decorator usage defaults the time window to "now - table creation time" if the specified window was larger than "now-table creation time".

解决方案

If you use bq --format=json you can get the information easily:

$ bq --format=prettyjson show publicdata:samples.wikipedia
{
  "creationTime": "1335916132870", ...
}

This is the exact value to use in the table decorator.

While I'm not sure that I like the idea of having a 'really low start value' be interpreted as table creation time, I've got other options:

  1. Table@0 means table at creation time
  2. Table@0 means table at the earliest time at which a snapshot is available.

I'm leaning towards #2, since snapshots can only go back 7 days in time.

这篇关于在Big Query中获取表格创建时间的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆