BigqueryIO 无法写入日期分区表 [英] BigqueryIO Unable to Write to Date-Partitioned Table
本文介绍了BigqueryIO 无法写入日期分区表的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在按照以下帖子中的说明进行操作 写入 BigQuery 中的日期分区表.我正在使用可序列化的函数使用 $
语法将窗口映射到分区位置,但出现以下错误:
I am following the instructions in the following post to write to a date-partitioned table in BigQuery. I am using a serializable function to map the the window to a partition-location using the $
syntax and I get the following error:
Invalid table ID \"table$19700822\". Table IDs must be alphanumeric (plus underscores) and must be at most 1024 characters long.
我在这里遗漏了什么吗?
Am I missing something here?
编辑添加代码:
p.apply(Window.<TableRow>into(FixedWindows.of(Duration.standardDays(1))))
.apply(BigQueryIO.Write
.named("Write")
.withSchema(schema)
.withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND)
.withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_IF_NEEDED)
.to(new SerializableFunction<BoundedWindow, String>() {
public String apply(BoundedWindow window) {
String dayString = DateTimeFormat.forPattern("yyyyMMdd")
.withZone(DateTimeZone.UTC)
.print(((IntervalWindow) window).start());
return "project_id:dataset.table$" + dayString;
}
}));
推荐答案
确保您尝试访问的表已经存在.您无法创建包含$"的表,并且您正在使用如果需要创建",因此您的代码最终可能会在尝试写入表的同时创建表.
Make sure that the table you're trying to access already exists. You can't create a table with "$" in it, and you're using "create if needed", so that your code might end up creating the table in addition to trying to write to it.
这篇关于BigqueryIO 无法写入日期分区表的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文