模式修改导致分区错误 [英] Schema modification giving partitioning error

问题描述

我以前使用terraform中的以下脚本在bigquery中创建了一个表

I have previously created a table in bigquery using following script in terraform

resource "google_bigquery_dataset" "my-dataset" {
  dataset_id  = "datasetname"
  description = "description"
}

resource "google_bigquery_table" "mytable" {
  dataset_id = google_bigquery_dataset.my-dataset.dataset_id
  table_id   = "mytable"

  time_partitioning {
    type = "DAY"
  }

  schema = <<EOF
[
  {
      "name": "field_one",
      "type": "STRING",
      "mode": "NULLABLE"
  },
  {
      "name": "field_two",
      "type": "RECORD",
      "mode": "NULLABLE",
      "fields": [
          {
              "name": "sub_field_one",
              "type": "FLOAT",
              "mode": "NULLABLE"
          },
          {
              "name": "sub_field_two",
              "type": "FLOAT",
              "mode": "NULLABLE"
          }
      ]
  },
  {
      "name": "field_three",
      "type": "STRING",
      "mode": "NULLABLE"
  }
]
EOF

}

这工作正常，我能够在bigquery中创建 mytable .现在，我必须修改此表，并向其添加一个新值( field_four .因此，请使此脚本

This worked fine and i was able to create mytable in bigquery. Now i have to modify this table and have to add a new value to it (field_four. So made this script

resource "google_bigquery_dataset" "my-dataset" {
  dataset_id  = "datasetname"
  description = "description"
}

resource "google_bigquery_table" "mytable" {
  dataset_id = google_bigquery_dataset.my-dataset.dataset_id
  table_id   = "mytable"

  time_partitioning {
    type = "DAY"
  }

  schema = <<EOF
[
  {
      "name": "field_one",
      "type": "STRING",
      "mode": "NULLABLE"
  },
  {
      "name": "field_two",
      "type": "RECORD",
      "mode": "NULLABLE",
      "fields": [
          {
              "name": "sub_field_one",
              "type": "FLOAT",
              "mode": "NULLABLE"
          },
          {
              "name": "sub_field_two",
              "type": "FLOAT",
              "mode": "NULLABLE"
          }
      ]
  },
  {
      "name": "field_three",
      "type": "STRING",
      "mode": "NULLABLE"
  },
  {
      "name": "field_four",
      "type": "RECORD",
      "mode": "NULLABLE",
      "fields": [
          {
              "name": "sub_field_three",
              "type": "STRING",
              "mode": "NULLABLE"
          }
      ]
  }
]
EOF

}

当我使用 terraform plan 运行它时，它表明将添加新字段.但是当我执行 terraform apply 时，出现以下错误

When i run it using terraform plan, it shows that the new field will be added. But when i do terraform apply, I get following error

Error: googleapi: Error 400: Cannot convert non partitioned/clustered table to partitioned/clustered table., invalid

  on filename.tf line 10, in resource "google_bigquery_table" "mytable":
  10: resource "google_bigquery_table" "mytable" {

我没有更改表的分区.这里发生了什么.如何将架构更改为

I am not changing the partition of the table. Whats going on here. How to make schema change to

1. 添加新字段?
2. 将字段类型(目前只有 null 个值)更改为新的内容?

1. Add a new field?
2. Change the field type (which has only null values as of now) to something new?

平台详细信息

Terraform v0.12.24
+ provider.google v3.1.0
+ provider.google-beta v3.1.0
+ provider.random v2.2.1

推荐答案

再现相同的代码后，我可以毫无问题地添加一个新字段.如评论中所述，当您尝试将非分区表转换为分区表时，会出现以下错误.在这种情况下，问题是由 Terraform 的版本不匹配引起的:用于创建表的版本不支持分区表，因此创建了普通表.在Terraform 0.12.24中运行两个代码时，它都可以工作.

After reproducing the same codes I could add a new field without problem. As discussed in the comments, the error below is raised when you try to transform a non partitioned table into a partitioned table. In this case, the problem was caused by a version mismatch of Terraform: the version used to create the table didn't support partitioned table so a normal table was created. When running both codes in Terraform 0.12.24 it worked.

Error: googleapi: Error 400: Cannot convert non partitioned/clustered table to partitioned/clustered table., invalid

  on filename.tf line 10, in resource "google_bigquery_table" "mytable":
  10: resource "google_bigquery_table" "mytable" {

关于第二个问题，您无法使用Terraform更改字段的类型.如您所见，此处，用于更改字段的类型需要一个查询作业，这在Terraform中是不可能的，正如您在公开问题.

About the second question, you can't change your field's type using Terraform. As you can see here, for changing a field's type you need a query job, which is not possible in Terraform as you can see in this open issue.

我希望对您有帮助

这篇关于模式修改导致分区错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！