我应该保持与Google云存储的开放连接吗? [英] Should I keep an open connection to Google cloud storage?

问题描述

要与Google Cloud Storage通信，请按照以下示例操作: https://developers.google.com/storage/docs/json_api/v1/json-api-java-samples

To communicate with Google Cloud Storage, I'm following this example: https://developers.google.com/storage/docs/json_api/v1/json-api-java-samples

我应该保持与云的开放连接吗?这会不会导致内存或资源问题?

Should I keep an open connection to the cloud? Wouldn't this cause memory or resources issues?

/** Global instance of the HTTP transport. */
private static HttpTransport httpTransport;

private static Storage client;

否则，我应该在每次获取/删除请求后关闭连接吗?最佳做法是什么?

Otherwise, should I close the connection after each get/delete request? what's the best practice?

我正在开发将在Linux上部署的应用程序.该应用程序将收到一个HTTP POST请求，其中包含要上传到云的文件.首次加载应用程序时，我将以下内容作为全局变量初始化:

I'm working on an application which will be deployed on Linux. The application will receive a HTTP POST request with a file to upload to the cloud. When the application is first loaded I initiate the following as global variables:

// Initialize the transport.
httpTransport = GoogleNetHttpTransport.newTrustedTransport();

// Initialize the data store factory.
dataStoreFactory = new FileDataStoreFactory(DATA_STORE_DIR);

// Authorization.
Credential credential = authorize();

// Set up global Storage instance.
client = new Storage.Builder(httpTransport, JSON_FACTORY, credential)
    .setApplicationName(APPLICATION_NAME).build();

这是最佳做法吗?这种实现会导致我的内存/资源问题吗?

Is this the best practice? Will this implementation cause me memory/resources issues?

推荐答案

我认为您错过了Google的最佳做法文档.请参见将数据上传到Cloud Storage ，网址为 https://cloud.google.com/storage/docs/best-practices ，它将清除您的所有疑问.

I think you missed the best practice document from google. See Uploading data to Cloud Storage at https://cloud.google.com/storage/docs/best-practices which will clear your all doubts.

• 如果您使用XMLHttpRequest(XHR)回调获取进度更新，则如果检测到进度已停止，则不要关闭并重新打开连接.这样做会在网络拥塞时造成不良的正反馈回路.当网络拥塞时，XHR回调可以从上传流中积压到确认(ACK/NACK)活动之后，并且在发生这种情况时关闭并重新打开连接会在您负担不起的确切时间使用更多的网络容量./li>
• 对于上传流量，我们建议设置较长的超时时间.为了获得良好的最终用户体验，您可以设置一个客户端计时器，当您的应用程序长时间未收到XHR回调时，使用消息(例如网络拥塞")更新客户端状态窗口.不仅仅是关闭连接，然后在发生这种情况时再试.
• 如果您将Google Compute Engine实例与POST到Cloud Storage的进程一起使用以启动可恢复的上传，则应在与Cloud Storage存储桶相同的位置使用Compute Engine实例.然后，您可以使用地理IP服务来选择将客户请求路由到的Compute Engine区域，这将有助于将流量保持在某个地理区域内.
• 如果可能，请避免将传输分成较小的块，而是将整个内容上传到单个块中.避免分块可消除固定的延迟成本并提高吞吐量，并降低针对Google Cloud Storage的QPS.
• 您应考虑分块上传的情况包括动态生成源数据，客户端具有请求大小限制(对于许多浏览器而言都是如此)或客户端无法在没有第一位的情况下在单个请求中流传输字节的情况将完整的请求加载到内存中.如果您的客户端收到错误，他们可以向服务器查询提交偏移量，然后继续从该偏移量上载剩余的字节.

• If you use XMLHttpRequest (XHR) callbacks to get progress updates, do not close and re-open the connection if you detect that progress has stalled. Doing so creates a bad positive feedback loop during times of network congestion. When the network is congested, XHR callbacks can get backlogged behind the acknowledgement (ACK/NACK) activity from the upload stream, and closing and reopening the connection when this happens uses more network capacity at exactly the time when you can least afford it.
• For upload traffic, we recommend setting reasonably long timeouts. For a good end-user experience, you can set a client-side timer that updates the client status window with a message (e.g., "network congestion") when your application hasn't received an XHR callback for a long time. Don't just close the connection and try again when this happens.
• If you use Google Compute Engine instances with processes that POST to Cloud Storage to initiate a resumable upload, then you should use Compute Engine instances in the same locations as your Cloud Storage buckets. You can then use a geo IP service to pick the Compute Engine region to which you route customer requests, which will help keep traffic localized to a geo-region.
• Avoid breaking a transfer into smaller chunks if possible and instead upload the entire content in a single chunk. Avoiding chunking removes fixed latency costs and improves throughput, as well as reducing QPS against Google Cloud Storage.
• Situations where you should consider uploading in chunks include when your source data is being generated dynamically, your clients have request size limitations (which is true for many browsers), or your clients are unable to stream bytes in a single request without first loading the full request into memory. If your clients receive an error, they can query the server for the commit offset and resume uploading remaining bytes from that offset.

这篇关于我应该保持与Google云存储的开放连接吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！