流插入时更新表架构 [英] Updating a table schema while streaming inserts

问题描述

我有一个表，该表不断接收流插入(每秒可能成千上万个).

I have a table that is continuously receiving streaming inserts (potentially thousands per second).

我对使用更新功能(通过API调用)添加列感兴趣.我可以调用更新来将列添加到现有表中，仍在插入数据时，不用担心数据丢失吗?

I am interested in using the Update functionality (via API calls) to add a column. Can I call Update to add a column to an existing table, while data is still being inserted, without concern for loss of data?

作为参考，这是我计划用于向表中添加列的代码:

For reference, here is the code I am planning on using to add a column to the table:

func addColumnToTable(service *bigquery.Service, project, dataset, table string, newCols map[string]string) bool {
    resp, err := service.Tables.Get(project, dataset, table).Do()
    if err != nil {
        fmt.Println(err)
        return false
    }

    for col, col_type := range newCols {
        this_col := &bigquery.TableFieldSchema{}
        this_col.Name = col
        this_col.Type = col_type
        resp.Schema.Fields = append(resp.Schema.Fields, this_col)
    }

    _, err = service.Tables.Update(project, dataset, table, resp).Do()
    if err != nil {
        fmt.Println(err)
        return false
    }

    return true
}

推荐答案

您可以使用tables.update更新表的架构(唯一允许的更新是添加列和放宽列的要求).

You can update the schema of a table using tables.update (the only allowed updates are adding a column and relaxing requiredness of a column).

流式传输对架构更新可见的时间稍有延迟.因此，在流式传输包含新列的数据之前，您需要等待一小段时间(最多5分钟)(假设您的更新是它添加了一个列).旧数据不会丢失.

There is a slight delay with which schema updates become visible to streaming. So, you will need to wait for a small amount of time (upto 5 minutes) before streaming data containing new column (assuming your update is that it added a column). Old data will not be lost.

这篇关于流插入时更新表架构的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！