Google Compute Engine VM磁盘非常慢 [英] Google Compute Engine VM disk is very slow

问题描述

我们刚刚切换到Google Compute Engine，但磁盘速度存在重大问题.大约是Linode的5％或更糟.写入速度从未超过20M/s，读取速度从未超过10M/s.在大多数情况下，写入速度为15M/s，读取速度为5M/s.

我们当前正在运行n1-highmem-4(4个vCPU，26 GB内存)计算机.中央处理器内存不是瓶颈.只需运行一个脚本即可从PostgreSQL数据库中读取行，对其进行处理，然后再写回PostgreSQL.批量更新数据库行仅是一项常见的工作.尝试运行20个进程以利用多核，但总体进度仍然很慢.

我们认为磁盘可能是瓶颈，因为流量异常低.

最后，我们决定进行基准测试.我们发现它不仅速度慢，而且似乎有一个可重现的主要错误:

1. 创建并创建连接到实例

2. 至少运行三次基准测试:

   dd if=/dev/zero bs=1024 count=5000000 of=~/5Gb.file
   

我们发现它变得非常慢，根本无法完成基准测试.

解决方案

持久性磁盘性能与磁盘本身及其连接的VM的大小成正比.磁盘(或VM)越大，性能越高，因此从本质上讲，您为磁盘支付的价格或VM不仅为磁盘/CPU/RAM付出的代价，还为IOPS和吞吐量付出的代价. >

引用永久磁盘文档:

永久磁盘性能取决于卷的大小以及 您选择的磁盘类型.更大的卷可以实现更高的I/O级别 比较小的体积.没有单独的I/O费用，因为 I/O功能已包含在永久性磁盘的价格中.

永久性磁盘性能可以描述如下:

• IOPS性能限制随永久性磁盘卷的大小线性增长.
• 吞吐量限制也呈线性增长，直至永久磁盘连接到的虚拟机的最大带宽.
• 较大的虚拟机比较小的虚拟机具有更高的带宽限制.

页面上还有更详细的价格图表，显示了您所购买的每GB空间所能获得的价格(以下数据为2014年8月的最新数据):

                                  Standard disks     SSD persistent disks

Price (USD/GB per month)                $0.04               $0.025
Maximum Sustained IOPS
  Read IOPS/GB                           0.3                  30
  Write IOPS/GB                          1.5                  30
Read IOPS/volume per VM                 3,000               10,000
Write IOPS/volume per VM               15,000               15,000
Maximum Sustained Throughput
  Read throughput/GB (MB/s)              0.12                 0.48
  Write throughput/GB (MB/s)             0.09                 0.48
Read throughput/volume per VM (MB/s)     180                  240
Write throughput/volume per VM (MB/s)    120                  240

以及页面上的一个具体示例，说明特定大小的磁盘将为您带来什么:

作为如何使用性能图表来确定的示例 您想要的磁盘卷，考虑到500GB标准持久性 磁盘会给你:

• (0.3×500)= 150次随机小读取
• (1.5×500)= 750次随机小写
• (0.12×500)= 60 MB/s的大顺序读取
• (0.09×500)= 45 MB/s的大顺序写入

We just switched over to Google Compute Engine and are having major issues with disk speed. It's been about 5% of Linode or worse. It's never exceeded 20M/s for writing and 10M/s for reading. Most of the time it's 15M/s for writing and 5M/s for reading.

We're currently running a n1-highmem-4 (4 vCPU, 26 GB memory) machine. CPU & memory aren't the bottleneck. Just running a script that reads rows from PostgreSQL database, processes them, then writes back to PostgreSQL. It's just for a common job to update database row in batch. Tried running 20 processes to take advantage of multi-core but the overall progress is still slow.

We're thinking disk may be bottleneck because traffic is abnormally low.

Finally we decided to do benchmarking. We found it's not only slow but seems to have a major bug which is reproducible:

1. create & connect to instance

2. run the benchmark at least three times:

   dd if=/dev/zero bs=1024 count=5000000 of=~/5Gb.file
   

We found it becomes extremely slow and aren't able to finish the benchmarking at all.

解决方案

Persistent Disk performance is proportional to the size of the disk itself and the VM that it is attached to. The larger the disk (or the VM), the higher the performance, so in essence, the price you are paying for the disk or the VM pays not only for the disk/CPU/RAM but also for the IOPS and throughput.

Quoting the Persistent Disk documentation:

Persistent disk performance depends on the size of the volume and the type of disk you select. Larger volumes can achieve higher I/O levels than smaller volumes. There are no separate I/O charges as the cost of the I/O capability is included in the price of the persistent disk.

Persistent disk performance can be described as follows:

• IOPS performance limits grow linearly with the size of the persistent disk volume.
• Throughput limits also grow linearly, up to the maximum bandwidth for the virtual machine that the persistent disk is attached to.
• Larger virtual machines have higher bandwidth limits than smaller virtual machines.

There's also a more detailed pricing chart on the page which shows what you get per GB of space that you buy (data below is current as of August 2014):

                                  Standard disks     SSD persistent disks

Price (USD/GB per month)                $0.04               $0.025
Maximum Sustained IOPS
  Read IOPS/GB                           0.3                  30
  Write IOPS/GB                          1.5                  30
Read IOPS/volume per VM                 3,000               10,000
Write IOPS/volume per VM               15,000               15,000
Maximum Sustained Throughput
  Read throughput/GB (MB/s)              0.12                 0.48
  Write throughput/GB (MB/s)             0.09                 0.48
Read throughput/volume per VM (MB/s)     180                  240
Write throughput/volume per VM (MB/s)    120                  240

and a concrete example on the page of what a particular size of a disk will give you:

As an example of how you can use the performance chart to determine the disk volume you want, consider that a 500GB standard persistent disk will give you:

• (0.3 × 500) = 150 small random reads
• (1.5 × 500) = 750 small random writes
• (0.12 × 500) = 60 MB/s of large sequential reads
• (0.09 × 500) = 45 MB/s of large sequential writes

这篇关于Google Compute Engine VM磁盘非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！