在AWS EC2上挂载NVME磁盘 [英] Mounting a NVME disk on AWS EC2

问题描述

所以我在每个节点上用NVME磁盘创建了i3.large，这是我的过程:

So I created i3.large with NVME disk on each nodes, here was my process :

1. lsblk-> nvme0n1(检查是否尚未挂载nvme)
2. sudo mkfs.ext4 -E nodiscard/dev/nvme0n1
3. sudo挂载-o丢弃/dev/nvme0n1/mnt/my-data
4. /dev/nvme0n1/mnt/my-data ext4默认值，nofail，丢弃0 2
5. sudo mount -a(检查是否一切正常)
6. sudo重新启动

因此，所有这些工作正常，我可以连接回实例.我的新分区上有500个Go.

So all of this works, I can connect back to the instance. I have 500 Go on my new partition.

但是在我停止并重新启动EC2机器后，其中一些机器随机变得无法访问(AWS仅警告警告1/2测试状态)

But after I stop and restart the EC2 machines, some of them randomly became inaccessible (AWS warning only 1/2 test status checked)

当我看到无法访问的日志时，它告诉我，这是关于nvme分区的(但是我做了sudo mount -a来检查它是否还可以，所以我不明白)

When I watch the logs of why it is inaccessible it tells me, it's about the nvme partition (but I did sudo mount -a to check if this was ok, so I don't understand)

我没有确切的AWS日志，但是有几行:

I don't have the AWS logs exactly, but I got some lines of it :

尝试打开时超级块中的魔法数字错误

Bad magic number in super-block while trying to open

然后，超级块已损坏，您可以尝试使用备用超级块运行e2fsck:

then the superblock is corrupt, and you might try running e2fsck with an alternate superblock:

/dev/fd/9:第2行:普利茅斯:找不到命令

/dev/fd/9: line 2: plymouth: command not found

推荐答案

停止并启动实例将擦除临时磁盘，将该实例移至新的主机硬件，并为您提供新的空磁盘...因此临时磁盘将始终停止/启动后为空白.实例停止后，该实例在任何物理主机上都不存在-资源被释放.

Stopping and starting an instance erases the ephemeral disks, moves the instance to new host hardware, and gives you new empty disks... so the ephemeral disks will always be blank after stop/start. When an instance is stopped, it doesn't exist on any physical host -- the resources are freed.

因此，最好的方法是，如果要停止和启动实例，则不是将它们添加到/etc/fstab中，而是仅在首次引导时对其进行格式化，然后再将其安装.测试文件系统是否已经存在的一种方法是使用file实用程序及其grep其输出.如果grep找不到匹配项，则返回false.

So, the best approach, if you are going to be stopping and starting instances is not to add them to /etc/fstab but rather to just format them on first boot and mount them after that. One way of testing whether a filesystem is already present is using the file utility and grep its output. If grep doesn't find a match, it returns false.

i3实例类上的NVMe SSD是实例的示例存储量，也称为星历[Disk |数量驱动器].它们在实例中物理上非常快，但不是多余的，并且不打算用于持久数据……因此是短暂的".永久性数据必须位于 Elastic Block Store(EBS)卷或

The NVMe SSD on the i3 instance class is an example of an Instance Store Volume, also known as an Ephemeral [ Disk | Volume | Drive ]. They are physically inside the instance and extremely fast, but not redundant and not intended for persistent data... hence, "ephemeral." Persistent data needs to be on an Elastic Block Store (EBS) volume or an Elastic File System (EFS), both of which survive instance stop/start, hardware failures, and maintenance.

尚不清楚为什么实例无法启动，但是nofail在存在卷但没有文件系统的情况下可能无法达到预期的效果.我的印象是，最终应该会成功.

It isn't clear why your instances are failing to boot, but nofail may not be doing what you expect when a volume is present but has no filesystem. My impression has been that eventually it should succeed.

但是，您可能需要 apt-get install linux-aws 如果运行Ubuntu 16.04. Ubuntu 14.04 NVMe支持不是很稳定，因此不推荐.

这三种存储解决方案中的每一种都有其优缺点.

Each of these three storage solutions has its advantages and disadvantages.

实例存储是本地的，因此它非常快...但是，它只是短暂的.它可以在硬重启和软重启后幸存下来，但不能停止/启动周期.如果您的实例发生硬件故障或计划退役(最终所有硬件都会发生这种情况)，则必须停止并启动该实例才能将其移至新硬件.预留实例和专用实例不会更改临时磁盘的行为.

The Instance Store is local, so it's quite fast... but, it's ephemeral. It survives hard and soft reboots, but not stop/start cycles. If your instance suffers a hardware failure, or is scheduled for retirement, as eventually happens to all hardware, you will have to stop and start the instance to move it to new hardware. Reserved and dedicated instances don't change ephemeral disk behavior.

EBS是持久的冗余存储，可以从一个实例分离并移动到另一个实例(并且该过程在停止/启动过程中自动发生). EBS支持时间点快照，并且这些快照在块级别是增量的，因此您无需为存储在快照之间未更改的数据而付费……但是通过一些出色的巫术，您也不必跟踪完整"快照与增量"快照-快照只是指向备份数据块的指针的逻辑容器，因此从本质上讲，它们是所有完整"快照，但仅作为增量快照而已.当您删除快照时，仅从后端存储系统中清除了不再需要还原该快照和任何其他快照的块(对您而言透明，实际上使用了Amazon S3).

EBS is persistent, redundant storage, that can be detached from one instance and moved to another (and this happens automatically across a stop/start). EBS supports point-in-time snapshots, and these are incremental at the block level, so you don't pay for storing the data that didn't change across snapshots... but through some excellent witchcraft, you also don't have to keep track of "full" vs. "incremental" snapshots -- the snapshots are only logical containers of pointers to the backed-up data blocks, so they are in essence, all "full" snapshots, but only billed as incrememental. When you delete a snapshot, only the blocks no longer needed to restore either that snapshot and any other snapshot are purged from the back-end storage system (which, transparent to you, actually uses Amazon S3).

EBS卷既可以作为SSD卷也可以作为旋转拼盘磁卷使用，但又要在成本，性能和适当的应用程序之间进行权衡.请参阅 EBS卷类型. EBS卷可以模拟普通硬盘驱动器，除了可以按需手动增加容量(但不能减少)，并且可以在不关闭系统的情况下将其从一种卷类型转换为另一种卷类型. EBS可以即时进行所有数据迁移，但性能会降低，但不会造成中断.这是一个相对较新的创新.

EBS volumes are available as both SSD and spinning platter magnetic volumes, again with tradeoffs in cost, performance, and appropriate applications. See EBS Volume Types. EBS volumes mimic ordinary hard drives, except that their capacity can be manually increased on demand (but not decreased), and can be converted from one volume type to another without shutting down the system. EBS does all of the data migration on the fly, with a reduction in performance but no disruption. This is a relatively recent innovation.

EFS使用NFS，因此即使在一个区域内的可用区域上，您也可以在任意数量的实例上挂载EFS文件系统. EFS中任何一个文件的大小限制 为52 TB，您的实例实际上将报告8 EB的可用空间.实际的可用空间实际上不受限制，但是EFS也是最昂贵的-如果您确实将52 TiB文件存储了一个月，那么该存储空间将花费超过$ 15,000.我最多存储了大约20 TiB，存储了2周，花了我大约5,000美元，但是如果您需要空间，那地方就在那里.它按小时计费，因此，如果您将52 TiB文件存储了仅几个小时，然后将其删除，则可能要支付50美元. EFS中的弹性"是指容量和价格.您无需在EFS上预配置空间.您使用所需的内容，然后删除不需要的内容，然后按小时计算可结算尺寸.

EFS uses NFS, so you can mount an EFS filesystem on as many instances as you like, even across availability zones within one region. The size limit for any one file in EFS is 52 terabytes, and your instance will actually report 8 exabytes of free space. The actual free space is for all practical purposes unlimited, but EFS is also the most expensive -- if you did have a 52 TiB file stored there for one month, that storage would cost over $15,000. The most I ever stored was about 20 TiB for 2 weeks, cost me about $5k but if you need the space, the space is there. It's billed hourly, so if you stored the 52 TiB file for just a couple of hours and then deleted it, you'd pay maybe $50. The "Elastic" in EFS refers to the capacity and the price. You don't pre-provision space on EFS. You use what you need and delete what you don't, and the billable size is calculated hourly.

没有S3，对存储的讨论将是不完整的.它不是文件系统，而是对象存储. S3的价格约为EFS的1/10，实际上具有无限的容量，最大对象大小为5TB.使用S3对象而不是文件可以更好地设计某些应用程序.

A discussion of storage wouldn't be complete without S3. It's not a filesystem, it's an object store. At about 1/10 the price of EFS, S3 also has effectively infinite capacity, and a maximum object size of 5TB. Some applications would be better designed using S3 objects, instead of files.

S3还可轻松地用于AWS之外的系统，无论是在您的数据中心还是在其他云中.尽管存在未记录的解决方法，该存储技术可在EC2内部使用，但该方法允许在外部或跨EFS使用EFS.区域以及代理服务器和隧道.

S3 can also be easily used by systems outside of AWS, whether in your data center or in another cloud. The other storage technologies are intended for use inside EC2, though there is an undocumented workaround that allows EFS to be used externally or across regions, with proxies and tunnels.

这篇关于在AWS EC2上挂载NVME磁盘的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！