从镶木地板到golang转换int96时间戳 [英] Cast int96 timestamp from parquet to golang

问题描述

具有此12字节数组(int96)进行时间戳记.

Having this 12 byte array (int96) to timestamp.

[128 76 69 116 64 7 0 0 48 131 37 0]

[128 76 69 116 64 7 0 0 48 131 37 0]

如何将其转换为时间戳?

How do I cast it to timestamp?

我知道前8个字节应强制转换为代表某个时代的日期时间的int64毫秒.

I understand the first 8 byte should be cast to int64 millisecond that represent an epoch datetime.

推荐答案

前8个字节的时间以纳秒为单位，而不是毫秒.它们也不是从时代开始测量的，而是从午夜开始测量的.日期部分以朱利安天数分开存储在最后4个字节中.

The first 8 bytes are time in nanosecs, not millisecs. They are not measured from the epoch either, but from midnight. The date part is stored separatly in the last 4 bytes as Julian day number.

这是我之前做过的实验的结果，可能会有所帮助.我将'2000-01-01 12:34:56'存储为int96并用镶木地板工具丢弃:

Here is the result of an experiment I did earlier that may help. I stored '2000-01-01 12:34:56' as an int96 and dumped with parquet-tools:

$ parquet-tools dump hdfs://path/to/parquet/file | tail -n 1
value 1: R:0 D:1 V:117253024523396126668760320

由于117253024523396126668760320 = 0x60FD4B3229000059682500，所以12个字节为00 60 FD 4B 32 29 00 00 | 59 68 25 00，其中|显示时间部分和日期部分之间的边界.

Since 117253024523396126668760320 = 0x60FD4B3229000059682500, the 12 bytes are 00 60 FD 4B 32 29 00 00 | 59 68 25 00, where | shows the boundary between the time and the date parts.

00 60 FD 4B 32 29 00 00是时间部分.我们需要反转字节，因为int96时间戳使用反转字节顺序，因此我们得到0x000029324BFD6000 = 45296 * 10 ^ 9纳秒= 45296秒= 12小时+ 34分钟+ 56秒.

00 60 FD 4B 32 29 00 00 is the time part. We need to reverse the bytes because int96 timestamp use a reverse byte order, thus we get 0x000029324BFD6000 = 45296 * 10^9 nanoseconds = 45296 seconds = 12 hours + 34 minutes + 56 seconds.

59 68 25 00是日期部分，如果我们反转字节，我们将得到0x00256859 = 2451545作为儒略日数，即

59 68 25 00 is the date part, if we reverse the bytes we get 0x00256859 = 2451545 as the Julian day number, which corresponds to 2000-01-01.

这篇关于从镶木地板到golang转换int96时间戳的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！