如何在 Kubernetes pod 之间共享存储? [英] How to share storage between Kubernetes pods?

问题描述

我正在评估 Kubernetes 作为我们新应用程序的平台.就目前而言，它看起来非常令人兴奋！但是，我遇到了一个问题:我在 GCE 上托管我的集群，我需要某种机制来在两个 Pod 之间共享存储 - 连续集成服务器和我的应用程序服务器.使用 kubernetes 执行此操作的最佳方法是什么?似乎没有一种卷类型适合我的需要，因为如果一个 pod 需要写入磁盘，则无法共享 GCE 磁盘.NFS 会很完美，但似乎需要为 kubernetes 集群提供特殊的构建选项?

I am evaluating Kubernetes as a platform for our new application. For now, it looks all very exciting! However, I’m running into a problem: I’m hosting my cluster on GCE and I need some mechanism to share storage between two pods - the continous integration server and my application server. What’s the best way for doing this with kubernetes? None of the volume types seems to fit my needs, since GCE disks can’t be shared if one pod needs to write to the disk. NFS would be perfect, but seems to require special build options for the kubernetes cluster?

共享存储似乎是我现在使用 Kubernetes 多次遇到的问题.有多个用例，我只想拥有一个卷并将其连接到多个 pod(具有写访问权限).我只能假设这将是一个常见的用例，不是吗?

Sharing storage seems to be a problem that I have encountered multiple times now using Kubernetes. There are multiple use cases where I'd just like to have one volume and hook it up to multiple pods (with write access). I can only assume that this would be a common use case, no?

例如，本页描述了如何设置 Elasticsearch 集群，但是将它与持久存储连接起来是不可能的(如此处所述)，这使得它毫无意义:(

For example, this page describes how to set up an Elasticsearch cluster, but wiring it up with persistent storage is impossible (as described here), which kind of renders it pointless :(

推荐答案

首先，你真的需要多个读者/作者吗?

根据我对 Kubernetes/微服务架构 (MSA) 的经验，问题通常与您的设计模式更相关.MSA 的基本设计模式之一是正确封装服务，这包括每个服务拥有的数据.

Firstly, do you really need multiple readers / writers?

From my experience of Kubernetes / micro-service architecture (MSA), the issue is often more related to your design pattern. One of the fundamental design patterns with MSA is the proper encapsulation of services, and this includes the data owned by each service.

与 OOP 的方式大致相同，您的服务应该管理与其关注的领域相关的数据，并且应该允许其他服务通过接口访问这些数据.这个接口可以是一个 API、直接或通过代理服务处理的消息，或者使用协议缓冲区和 gRPC.一般来说，多服务访问数据是一种类似于 OOP 和大多数编程语言中的全局变量的反模式.

In much the same way as OOP, your service should look after the data that is related to its area of concern and should allow access to this data to other services via an interface. This interface could be an API, messages handled directly or via a brokage service, or using protocol buffers and gRPC. Generally, multi-service access to data is an anti-pattern akin to global variables in OOP and most programming languages.

举个例子，如果你想写日志，你应该有一个日志服务，每个服务都可以调用它需要记录的相关数据.直接写入共享磁盘意味着如果您更改日志目录结构，或决定添加额外功能(例如针对某些类型的错误发送电子邮件)，您需要更新每个容器.

As an example, if you where looking to write logs, you should have a log service which each service can call with the relevant data it needs to log. Writing directly to a shared disk means that you'd need to update every container if you change your log directory structure, or decided to add extra functionality like sending emails on certain types of errors.

在大部分情况下，您应该在使用文件系统之前使用某种形式的最小界面，以避免 Hyrum 定律，您在使用文件系统时会遇到这种情况.如果您的服务之间没有适当的接口/契约，您将大大降低构建可维护且有弹性的服务的能力.

In the major percentage of cases, you should be using some form of minimal interface before resorting to using a file system, avoiding the unintended side-effects of Hyrum's law that you are exposed to when using a file system. Without proper interfaces / contracts between your services, you heavily reduce your ability to build maintainable and resilient services.

很明显，有时可以处理多个并发写入器的文件系统提供了比更传统"的 MSA 通信形式更好的解决方案.Kubernetes 支持大量卷类型，可以在此处找到.虽然这个列表很长，但其中许多卷类型不支持多个写入器(在 Kubernetes 中也称为 ReadWriteMany).

There are obviously times when a file system that can handle multiple concurrent writers provides a superior solution over a more 'traditional' MSA forms of communication. Kubernetes supports a large number of volume types which can be found here. While this list is quite long, many of these volume types don't support multiple writers (also known as ReadWriteMany in Kubernetes).

那些支持 ReadWriteMany 的卷类型可以在 此表，在撰写本文时，这是 AzureFile、CephFS、Glusterfs、Quobyte、NFS 和 PortworxVolume.

Those volume types that do support ReadWriteMany can be found in this table and at the time of writing this is AzureFile, CephFS, Glusterfs, Quobyte, NFS and PortworxVolume.

还有一些操作符，例如流行的 rook.io，它们功能强大并提供了一些很棒的功能，但是当您只想要一个简单的解决方案并继续前进时，此类系统的学习曲线可能会很艰难.

There are also operators such as the popular rook.io which are powerful and provide some great features, but the learning curve for such systems can be a difficult climb when you just want a simple solution and keep moving forward.

根据我的经验，最好的初始选项是 NFS.这是学习有关 ReadWriteMany Kubernetes 存储的基本思想的好方法，适用于大多数用例并且最容易实现.在您积累了多服务持久性的应用知识后，您就可以做出更明智的决定，以使用更多功能丰富的产品，而这些产品通常需要更多的工作来实施.

In my experience, the best initial option is NFS. This is a great way to learn the basic ideas around ReadWriteMany Kubernetes storage, will serve most use cases and is the easiest to implement. After you've built a working knowledge of multi-service persistence, you can then make more informed decisions to use more feature rich offerings which will often require more work to implement.

设置 NFS 的细节根据集群运行的方式和位置以及 NFS 服务的细节而有所不同，我之前写过两篇关于如何为 本地集群 并使用等效的 AWS NFS EKS 集群上的 EFS.这两篇文章很好地对比了如何根据您的特定情况使用不同的实现.

The specifics for setting up NFS differ based on how and where your cluster is running and the specifics of your NFS service and I've previously written two articles on how to set up NFS for on-prem clusters and using AWS NFS equivalent EFS on EKS clusters. These two articles give a good contrast for just how different implementations can be given your particular situation.

对于最低限度的示例，您首先需要一个 NFS 服务.如果您想进行快速测试或 SLO 要求较低，请遵循 这篇 DO 文章 是在 Ubuntu 上设置 NFS 的快速入门.如果您有一个现有的 NAS 提供 NFS 并且可以从您的集群访问，这也适用.

For a bare minimum example, you will firstly need an NFS service. If you're looking to do a quick test or you have low SLO requirements, following this DO article is a great quick primer for setting up NFS on Ubuntu. If you have an existing NAS which provides NFS and is accessible from your cluster, this will also work as well.

拥有 NFS 服务后，您可以创建类似于以下内容的持久卷:

Once you have an NFS service, you can create a persistent volume similar to the following:

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-name
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  nfs:
    server: 255.0.255.0 # IP address of your NFS service
    path: "/desired/path/in/nfs"

这里需要注意的是，您的节点需要安装二进制文件才能使用 NFS，我在我的 本地集群 文章.这也是您在 EKS 上运行时需要使用 EFS 的原因，因为您的节点无法连接到 NFS.

A caveat here is that your nodes will need binaries installed to use NFS, and I've discussed this more in my on-prem cluster article. This is also the reason you need to use EFS when running on EKS as your nodes don't have the ability to connect to NFS.

一旦您设置了持久卷，就可以像使用任何其他卷一样使用它.

Once you have the persistent volume set up, it is a simple case of using it like you would any other volume.

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-name
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: p-name
          volumeMounts:
            - mountPath: /data
              name: v-name
      volumes:
        - name: v-name
          persistentVolumeClaim:
            claimName: pvc-name

这篇关于如何在 Kubernetes pod 之间共享存储?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！