如何将时间戳转换为字符串(不更改时区)? [英] how to convert a timestamp into string (without changing timezone)?

问题描述

我有一些 unix 时间在 sparklyr 中转换为时间戳，出于某些原因，我还需要将它们转换为字符串.

I have some unix times that I convert to timestamps in sparklyr and for some reasons I also need to convert them into strings.

不幸的是，在转换为字符串的过程中，hive 似乎转换为 EST(我的语言环境).

Unfortunately, it seems that during the conversion to string hive converts to EST (my locale).

df_new <- spark_read_parquet(sc, "/mypath/parquet_*",
                             overwrite = TRUE,
                             name = "df_new",
                             memory = FALSE,
                             options = list(mergeSchema = "true"))
> df_new %>%  
           mutate(unix_t = from_utc_timestamp(timestamp(t) ,'UTC'),
           date_str = date_format(unix_t, 'yyyy-MM-dd HH:mm:ss z'),
           date_alt = to_date(from_utc_timestamp(timestamp(t) ,'UTC'))) %>% 
    select(t, unix_t, date_str, date_alt) %>% head(5)
# Source:   lazy query [?? x 4]
# Database: spark_connection
            t unix_t              date_str                date_alt  
        <dbl> <dttm>              <chr>                   <date>    
1 1419547405. 2014-12-25 22:43:25 2014-12-25 17:43:25 EST 2014-12-25
2 1418469714. 2014-12-13 11:21:54 2014-12-13 06:21:54 EST 2014-12-13
3 1419126103. 2014-12-21 01:41:43 2014-12-20 20:41:43 EST 2014-12-20
4 1419389856. 2014-12-24 02:57:36 2014-12-23 21:57:36 EST 2014-12-23
5 1418271811. 2014-12-11 04:23:31 2014-12-10 23:23:31 EST 2014-12-10

如您所见，date_str 和 date_alt 都使用 EST 时区.我需要 UTC 在这里.我该怎么做?

As you can see both date_str and date_alt use the EST timezone. I need UTC here. How can I do that?

谢谢！

推荐答案

来自 Hive 函数参考，date_format 使用 Java 的 SimpleDateFormat，我相信它总是默认为 JVM 时区，这解释了为什么这会让您将字符串转换为您的时区.

From the Hive function reference, date_format uses Java's SimpleDateFormat, which I believe always defaults to the JVM time zone, this explains why this gets you a character string converted to your time zone.

一种选择是检测时区并手动添加小时以获得UTC.

One option is to detect the time zone and manually add the hours to get UTC.

另一种选择是将 lubridate 与 spark_apply() 一起使用:

Another option would be to use lubridate with spark_apply():

sdf_len(sc, 1) %>%
  mutate(unix_t = from_utc_timestamp(timestamp(1522371003) , 'UDT')) %>%
  spark_apply(
    function(e) {
      dplyr::mutate(
        e,
        time_str = as.character(
          lubridate::with_tz(
            as.POSIXct(unix_t, origin="1970-01-01"),
            "GMT"
          )
        )
      )
    },
    columns = c("id", "unix_t", "time_str"))

这篇关于如何将时间戳转换为字符串(不更改时区)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！