如何使用Google Cloud BigQuery进行集成测试 [英] How can I make integration tests with google cloud bigquery

问题描述

我们正在处理从apache hbase到bigquery的迁移。

We are in the processing of migrating from apache hbase to bigquery.

当前，我们已经进行了端到端测试（使用黄瓜），该测试适用于运行hbase的docker容器

Currently we have end to end tests (using cucumbers) that work with a docker container running hbase.

似乎没有任何bigquery Docker容器或模拟器（ https://cloud.google.com/sdk/gcloud/reference/beta/emulators/ ）

There don't seem to be any bigquery docker containers or emulators (https://cloud.google.com/sdk/gcloud/reference/beta/emulators/)

我们如何为使用bigquery的应用程序创建端到端测试？

How would we be able to create end to end tests for an application working with bigquery?

推荐答案

当前没有任何内容一种BigQuery本地模拟器或类似的东西。正如您分享的有关可用的GCP模拟器的链接所指出的那样，还有其他一些产品具有这种功能，但是可能BigQuery没有这种功能的原因是，只有在实际基础架构中才能看到其真正的潜力，再加上使用BigQuery的成本可能相对较低。

Currently there is not any kind of BigQuery local emulator or anything similar to that. As pointed out by the link you shared about available GCP emulators, there are some other products that have such a feature, but probably the reason why BigQuery does not have one is that its true potential is only seen when working in its real infrastructure, plus the fact that the costs of working with BigQuery can be relatively low depending on the usage you make of it, plus you have a Free Tier to start working with.

让我总结一些有关BigQuery定价的信息，这些信息可能对您有用：

Let me summarize some info about BigQuery pricing that can be useful for you:

• 定价文档。


• BigQuery提供了一些 rel = nofollow noreferrer>免费。


• 有一个免费存储层，其中有10GB的免费存储空间。鉴于BQ旨在处理大量数据，但这可能并不很多，但这可以成为进行某些测试的良好起点。


• 还有一个免费操作，其中第一个TeraByte处理数据（每月）是免费的。


• 您可以设置警报，以便使用 Stackdriver ，使用可用指标。

• BigQuery storage and operation costs are summarized in the pricing documentation.
• BigQuery offers some operations that are free of charge.
• There's a Storage free tier with 10GB of free storage. It may not be a lot, given that BQ is designed to work with enormous amounts of data, but it can be a good starting point to do some tests.
• There's also an Operations free tier, where the first TeraByte of processed data (per month) is free of charge.
• You can set up alerts in order to monitor usage with Stackdriver, using the available metrics.

无论如何，如果您仍然认为直接使用BigQuery并不是您的最佳选择，请始终通过创建功能部件将请求转发给工程团队在公共问题跟踪器中请求BigQuery ugh是否要决定是否（以及何时）实施这种功能将由工程团队掌握，甚至还要考虑BigQuery的复杂性以及针对其当前架构进行了优化的性能。

In any case, if you still think that working with BigQuery directly is not the best option for you, can always forward your requests to the Engineering team by creating a Feature Request in the Public Issue Tracker for BigQuery, although it will be in hands of the engineering team whether to decide if (and when) to implement such a feature, even more considering the complexity of BigQuery and that its performance is optimized for working in its current architecture.

这篇关于如何使用Google Cloud BigQuery进行集成测试的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！