bigquery在处理大型文件集时的行为如何 [英] How does bigquery behaves while processing large file set

问题描述

大查询是否支持ACID属性？
处理大量文件并且最后一个失败后，是否会回滚整个作业？

解决方案

是，BigQuery作业具有ACID属性。也就是说，它们是

• 原子：要么完全提交，要么完全不提供。
  
• 一致：各处都应该有变化，一旦工作成功完成，应该在随后的每一项工作中都能看到这些数据。


• 隔离：并发作业独立运行。


• 耐久性：一旦作业完成，数据应该被持久地写入（当前，这意味着它至少被写入3个副本）。
  

作业开始时，会检查其是否有效（例如，是否使用正确的模式附加数据），但在作业准备好提交之前不会进行任何更改。此时，这些更改将以原子方式应用（并再次验证）。

Does big query support ACID properties? After processing large number of files and the last one fails,would that rollback the entire job load?

解决方案

Yes, BigQuery jobs have ACID properties. That is, they are

• Atomic: Either they commit entirely or not at all.
• Consistent: Changes should be available everywhere, and once a job has completed successfully, that data should be seen on every subsequent job.
• Isolated: Concurrent jobs operate independently.
• Durable: Once a job has finished, the data should be written durably (currently, that means that it is written to at least 3 replicas.)

When jobs start they are checked for validity (for example, are you appending data with the right schema) but no changes are made until the job is ready to commit. At that point the changes are applied atomically (and validated again).

这篇关于bigquery在处理大型文件集时的行为如何的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！