如何使XFS配额在DigitalOcean上的Kubernetes卷中工作? [英] How to make XFS quotas work in Kubernetes volumes on DigitalOcean?

问题描述

在DigitalOcean上的Kubernetes中使XFS配额正常工作时，我需要帮助.

I need help with making the XFS quotas work in Kubernetes on DigitalOcean.

我的问题本质上是xfs_quota工具似乎仅在人们还可以访问磁盘设备(不仅是已装入的卷)时才起作用.但是，无论我如何尝试，我似乎都无法同时访问设备和安装架.

My problem essentially is that the xfs_quota tool seems to work only when one has also an access to the disk device, not only to the mounted volume. However, whatever I try, I can't seem to get access both to the device and the mount.

我尝试了卷挂载和原始块卷.

I tried both volume mounts and raw block volumes.

这是我的存储课程:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: block-storage-retain-xfs-prjquota
provisioner: dobs.csi.digitalocean.com
parameters:
  fsType: xfs
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
mountOptions:
  - prjquota

然后我声明一个新的卷并将其添加到这样的pod中:

Then I claim a new volume and add it to a pod like this:

volumeClaimTemplates:
  - metadata:
      name: my-storage
      namespace: my-namespace
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: block-storage-retain-xfs-prjquota
      resources:
        requests:
          storage: 1Gi

并安装它:

volumeMounts:
  - name: my-storage
    mountPath: "/var/www"

在pod中，所有东西都正确安装了，我可以访问该卷(我可以在其中创建内容)并且正确设置了安装标志:

In the pod, everything gets mounted correctly, I have access to the volume (I can create contents in there) and the mount flag is set correctly:

$ mount | grep -i www
/dev/disk/by-id/scsi-0DO_Volume_pvc-650ccba6-3177-45b5-9ffb-0ac2a931fddc on /var/www type xfs (rw,relatime,attr2,inode64,prjquota)

但是，该磁盘设备在pod中不可用:

However, the disk device is not available in the pod:

$ ls -la /dev/disk/by-id/scsi-0DO_Volume_pvc-650ccba6-3177-45b5-9ffb-0ac2a931fddc
ls: cannot access '/dev/disk/by-id/scsi-0DO_Volume_pvc-650ccba6-3177-45b5-9ffb-0ac2a931fddc': No such file or directory

(实际上，整个/dev/disk/目录都不可用)

(in fact, the whole /dev/disk/ directory is not available)

根据我的调查，缺乏对设备的访问权限是XFS工具失败的原因:

According to my investigation, the lack of access to the device is what makes the XFS tools fail:

$ xfs_quota -x -c 'report -h' /var/www
xfs_quota: cannot setup path for mount /var/www: No such device or address

原始块体积

我还尝试切换到原始块卷:

Raw Block Volumes

I also tried to switch to raw block volumes instead:

volumeClaimTemplates:
  - metadata:
      name: my-storage
      namespace: my-namespace
    spec:
      accessModes:
        - ReadWriteOnce
      volumeMode: Block
      storageClassName: block-storage-retain-xfs-prjquota
      resources:
        requests:
          storage: 1Gi

并将其添加为:

volumeDevices:
  - name: my-storage
    devicePath: /dev/my-storage

这给了我设备，但是由于某种原因，我无法对其进行格式化/挂载(实际上既不是XFS也不是ext4):

That gives me the device, but for some reason I can't format it / mount it (neither XFS nor ext4 actually):

$ mkfs.xfs /dev/my-storage
mkfs.xfs: error - cannot set blocksize 512 on block device /dev/my-storage: Permission denied

$ mkfs.ext4 /dev/my-storage
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 18f07181-737c-4b68-a5fe-ccd7f2c50ff8
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

$ mount /dev/my-storage /var/www
mount: /var/www: cannot mount /dev/my-storage read-only.

借助SYS_ADMIN Linux功能，我实际上可以对其进行格式化，但是我仍然无法挂载它:

With SYS_ADMIN Linux capability, I can actually format it, but I'm still not able to mount it:

$ mkfs.xfs -f /dev/my-storage
meta-data=/dev/my-storage        isize=512    agcount=4, agsize=65536 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=262144, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

$ mount /dev/my-storage /var/www
mount: /var/www: cannot mount /dev/my-storage read-only.

(为什么磁盘设备是只读的?)

(Why is the disk device read only?)

好，所以我尝试创建一个分区并对其进行格式化.分区创建成功，但是我无权访问分区设备:

Ok, so I tried to create a partition and format that. Partition is created successfully, but I don't have access to the partition devices:

$ fdisk -l /dev/my-storage
Disk /dev/my-storage: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk model: Volume
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb4a24142

Device            Boot Start     End Sectors  Size Id Type
/dev/my-storage1        2048 2097151 2095104 1023M 83 Linux

但是，/dev/my-storage1不存在:

$ ls -la /dev/my-storage*
brw-rw---- 1 root disk 8, 48 Oct 25 14:42 /dev/my-storage

我尝试将容器运行为privileged，这使我可以访问/dev中的更多设备，但是后来我完全看不到原始块卷设备.

I tried running the container as privileged which gave me access to more devices in /dev, but then I didn't see my raw block volume device at all.

如我所见，所有这些都对我有用:

As I see that, any of those would work for me:

1. 获得对基础块设备的访问以进行卷安装.
2. 访问分区设备，以便我可以安装它.
3. 能够安装原始块卷(例如，通过将其设置为非只读，无论其含义如何).
4. 使用xfs_quota工具不需要基础设备.

1. Getting access to the underlying block device for volume mounts.
2. Access to the partition device so that I can mount it.
3. Ability to mount the raw block volume (e.g. by making it not read-only, whatever it means?).
4. Making the xfs_quota tool NOT require the underlying device.

我相信我是几个月前通过使用带有分区的原始块卷使其工作的，但是我忘记了DigitalOcean的方式或发生了什么变化，我似乎无法再创建和访问分区.

I believe I made it work a few months ago using raw block volumes with partitions, but either I forgot how or something changed on DigitalOcean and I can't seem to be able to create and access partitions anymore.

非常感谢您的帮助，谢谢！

Any help is hugely appreciated, thank you!

推荐答案

这里是DigitalOcean的Managed Kubernetes(DOKS)团队的Timo.

Timo here from the Managed Kubernetes (DOKS) team at DigitalOcean.

您所缺少的是/dev目录的主机系统挂载.如果您同时添加两者

What you are missing is the host system mount of the /dev directory. If you add both

 volumes:
 - name: device-dir
   hostPath:
     path: /dev

和

volumeMounts:
  - name: device-dir
    mountPath: /dev

在正确位置的清单上，一切应该按预期进行.

to the manifest at the right places, things should work as expected.

这篇关于如何使XFS配额在DigitalOcean上的Kubernetes卷中工作?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！