BigqueryIO无法写入按日期分区的表 [英] BigqueryIO Unable to Write to Date-Partitioned Table

问题描述

我正在按照以下帖子中的说明进行操作写入BigQuery中的日期分区表.我正在使用可序列化函数使用$语法将窗口映射到分区位置，并且出现以下错误:

I am following the instructions in the following post to write to a date-partitioned table in BigQuery. I am using a serializable function to map the the window to a partition-location using the $ syntax and I get the following error:

Invalid table ID \"table$19700822\". Table IDs must be alphanumeric (plus underscores) and must be at most 1024 characters long.

我在这里想念东西吗?

编辑添加的代码:

p.apply(Window.<TableRow>into(FixedWindows.of(Duration.standardDays(1))))
    .apply(BigQueryIO.Write
    .named("Write")
    .withSchema(schema)
    .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND)
    .withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_IF_NEEDED)
    .to(new SerializableFunction<BoundedWindow, String>() {
      public String apply(BoundedWindow window) {
        String dayString = DateTimeFormat.forPattern("yyyyMMdd")
             .withZone(DateTimeZone.UTC)
             .print(((IntervalWindow) window).start());
        return "project_id:dataset.table$" + dayString;
      }
    }));

推荐答案

请确保您要访问的表已经存在.您无法创建其中带有"$"的表，并且使用的是需要时创建"，这样您的代码除了尝试写入表之外，还可能最终创建该表.

Make sure that the table you're trying to access already exists. You can't create a table with "$" in it, and you're using "create if needed", so that your code might end up creating the table in addition to trying to write to it.

这篇关于BigqueryIO无法写入按日期分区的表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！