无法在AKS中获得Postgres权限或PVC [英] Can't get either Postgres permissions or PVC working in AKS

问题描述

这些几乎是我按照顺序执行的步骤。基本上是文档中概述的内容：

当在Postgres中创建表，销毁Pod然后重新部署它时，当然可以肯定，该表不再存在。

所以我很可能做错了什么，但我一直在遵循文档，而且似乎应该可以。

哪里出了问题？

编辑：Pod中的权限

显然，这是当 PGDATA 与<$相同的目录时发生的权限问题c $ c> mountPath 。例如：

  ... 
-名称：PGDATA 
值：/ var / lib / postgresql-data 
 volumeMounts：
-名称：测试应用程序存储
 mountPath：/ var / lib / postgresql-data 
子路径：postgres存储
 ... 
 
或
 
 ... 
＃如果未指定PGDATA，则默认为/ var / lib / postgresql / data 
＃-名称：PGDATA 
 ＃值：/ var / lib / postgresql-data 
 volume安装：
-名称：test-app-storage 
 mountPath：/ var / lib / postgresql / data 
子路径：postgres -存储
 ... 
 
  

类似这样的地方（它们不匹配）将创建Pod ，但使用了我显然不想要的Pod存储：

 ＃因此，/ var / lib / postgresql / data 
＃-名称：PGDATA 
＃值：/ var / lib / postgresql-data 
 volume安装：
-名称：test-app-storage 
 mountPath：/ var / lib / postgresql-data 
子路径：postgres-storage 
  

权限 ls -l <​​/ code>看起来像这样：

  $ ls -l 
 
 drwxr-xr-x 1根根4096 Feb 2 06:06 apt 
 drwxr-xr-x 1根根4096 Feb 2 06:07 dpkg 
 drwxr-xr-x 2根根4096 Feb 2 06:06 exim4 
 drwxr-xr-x 2根目录根4096 Aug 28 2018 logrotate 
 drwxr-xr-x 2根目录4096 Nov 10 12:17 misc 
 drwxr-xr-x 2 root root 4096 Jan 30 00:00 pam 
 drwxr-xr-x 1 postgres postgres 4096 2月2日06:07 postgresql 
 drwxrwxrwx 2 1000 1000 0 Jan 31 21:46 postgresql数据
 drwxr-xr-x 1根root 4096年1月30日00:00 systemd 
 drwxr-xr-x 3根root 4096 2月2日06:07 ucf 
 
 $ ls -l postgresql& ; ls -l postgresql / data&& ls -l postgresql-data 
总计4 
 drwx ------ 19 postgres postgres 4096 Feb 5 23:28数据
总计124 
 drwx ------ 6 postgres postgres 4096 2月5日23:28基础
 drwx ------ 2 postgres postgres 4096 2月5日23:29全球
 drwx ------ 2 postgres postgres 4096 2月5日23： 28 pg_commit_ts 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_dynshmem 
 -rw ------- 1个postgres postgres 4281 Feb 5 23:28 pg_hba.conf 
 -rw ------- 1个postgres postgres 1636 2月5日23:28 pg_ident.conf 
 drwx ------ 4个postgres postgres 4096 2月5日23:33 pg_logical 
 drwx- ----- 4个postgres postgres 4096 Feb 5 23:28 pg_multixact 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_notify 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_replslot 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_serial 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_snapshots 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_stat 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:51 pg_stat_tmp 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_subtrans 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_tblspc 
 drwx ------ 2个postgres postgres 4096 Feb 5 23:28 pg_twophase 
 -rw ------- 1 postgres postgres 2 Feb 5 23:28 PG_VERSION 
 drwx-- ---- 3个postgres postgres 4096年2月5日23:28 pg_wal 
 drwx ------ 2个postgres postgres 4096年2月5日23:28 pg_xact 
 -rw ------- 1个postgres postgres 88 2月5日23:28 postgresql.auto.conf 
 -rw ------- 1个postgres postgres 26588 2月5日23:28 postgresql.conf 
 -rw ------- 1个postgres postgres 36 Feb 5 23:28 postmaster.opts 
 -rw ------- 1个postgres postgres 94 Feb 5 23:28 postmaster.pid 
总计0 
  

创建数据文件的权限为 postgres 。但是，这样做不会将其映射到Azure文件和PVC。

我认为正在发生的事情是 mountPath 使用 root 和 PGDATA 使用 postgres ，以及 mountPath 试图使用 postgres ???

真的，不确定并且仍然迷失于解决方法。 / p>

EDIT2

在此答案中同样出现：

https://stackoverflow.com/a/51203031/3123109

因此我添加了以下内容：

 -名称：postgres 
图片：postgres 
命令：
-/ bin / chown 
 --R 
- 1000； 
-/ var / lib / postgresql / data 
  

但这会产生新的错误：

 所选容器尚未记录任何消息。 
  

我猜是进步。

解决方案

我认为您的问题可能是由于您尝试将子路径用作部署对象的一部分

请重试第一个配置，不要这条线：

subPath：postgres-storage

应该将其放入
的安装量： b $ b-名称：postgres-storage
mountPath：/ var / lib / postgresql / data / pgdata

让我知道是否有帮助。

更新：使用永久存储时，PostgreSQL的docker映像需要特别注意

https://hub.docker.com/_/postgres

需要注意的主要警告是，postgres并不关心它运行的UID（只要/ var / lib / postgresql / data的所有者匹配），但是initdb确实关心（并且需要用户存在于/ etc /中）。密码d）：

解决此问题的三种最简单方法：

1. 使用Debian变体（不是Alpine变体），从而允许映像使用nss_wrapper库为您伪造 / etc / passwd内容（有关更多详细信息，请参阅docker-library / postgres＃448）




2. 从主机上以只读方式绑定安装/ etc / passwd（如果您希望的UID是主机上的有效用户）：

$ docker run -it --rm --user $（id -u）：$（id -g） -v / etc / passwd：/ etc / passwd：ro -e POSTGRES_PASSWORD = mysecretpassword postgres
属于此数据库系统的文件将由用户 jsmith拥有。

3. 与最终运行时分开初始化目标目录（中间加一个小调）：

$ docker volume create pgdata
$ docker run -it --rm -v pgdata：/ var / lib / postgresql / data -e POSTGRES_PASSWORD = mysecretpassword postgres
属于此数据库的文件stem将归用户 postgres所有。
（一旦成功完成初始化并正在等待连接，请停止它）
$ docker run -it --rm -v -v pgdata：/ var / lib / postgresql / data bash chown -R 1000：1000 / var / lib / postgresql / data
$ docker run -it --rm --user 1000：1000 -v pgdata：/ var / lib / postgresql / data postgres
日志：数据库系统已关闭在2017-01-20 00:03:23 UTC
日志：Mu​​ltiXact成员环绕保护已启用
日志：autovacuum启动器已启动
日志：数据库系统已准备好接受连接

一种解决方案是利用bitnami制作的头盔图，他们使用init容器计算出了复杂的持久性存储配置的默认值，并且它们还支持/ dev /需要修复的shm配置

还请注意，容器的默认/ dev / shm大小为64MB。如果共享内存已用完，您将遇到错误：无法调整共享内存段的大小。 。 。 ： 设备上没有剩余空间。例如，您将要传递--shm-size = 256MB到docker run，或者在docker-compose中传递

以下是图表使用的初始化容器的示例

https://github.com/helm/charts/blob/master/stable/postgresql/templates/statefulset.yaml#L74-L115

建议使用头盔安装Postgres图表（使用statefuset），而不要尝试使用部署对象来部署数据库。 （通常您希望将部署对象用于无状态应用程序。）

These are pretty much the steps I have followed in order. Basically what is outlined in the documentation:

https://docs.microsoft.com/en-us/azure/aks/azure-files-dynamic-pv

1. azure-storage-claim.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: test-app-sc
provisioner: kubernetes.io/azure-file
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1000
  - gid=1000
  - mfsymlinks
  - nobrl
  - cache=none
parameters:
  skuName: Standard_LRS
  location: westus

2. azure-storage.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-app-storage
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: test-app-sc
  resources:
    requests:
      storage: 15Gi

PVC is now setup.

Changed the mountPath per the Postgres image documentation:

PGDATA

This optional variable can be used to define another location - like a subdirectory - for the database files. The default is /var/lib/postgresql/data, but if the data volume you're using is a filesystem mountpoint (like with GCE persistent disks), Postgres initdb recommends a subdirectory (for example /var/lib/postgresql/data/pgdata ) be created to contain the data.

This is an environment variable that is not Docker specific. Because the variable is used by the postgres server binary (see the PostgreSQL docs), the entrypoint script takes it into account.

Based on that, I have my postgres.yaml setup like the following:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      component: postgres
  template:
    metadata:
      labels:
        component: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:11-alpine
          ports:
            - containerPort: 5432
          env: 
            - name: POSTGRES_DB
              valueFrom:
                secretKeyRef:
                  name: test-app-secrets
                  key: PGDATABASE
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: test-app-secrets
                  key: PGUSER
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: test-app-secrets
                  key: PGPASSWORD
            - name: POSTGRES_INITDB_ARGS
              value: "-A md5"
            - name: PGDATA
              value: /var/lib/postgresql-data
          volumeMounts:
            - name: test-app-storage
              mountPath: /var/lib/postgresql-data
              subPath: postgres-storage
      volumes:
        - name: test-app-storage
          persistentVolumeClaim:
            claimName: test-app-storage
---
apiVersion: v1
kind: Service
metadata:
  name: postgres-cluster-ip-service
spec:
  type: ClusterIP
  selector:
    component: postgres
  ports:
    - port: 1423
      targetPort: 5432

You get the error:

chmod: changing permissions of '/var/lib/postgresql-data': Operation not permitted

So with either of that as the Dockerfile:

FROM postgres:11-alpine
EXPOSE 5432
RUN /bin/bash -c 'chmod 777 /var/lib/postgresql-data'

Or

FROM postgres:11-alpine
EXPOSE 5432

It doesn't really matter, you still get the same type of error by doing any the following:

            ...
            - name: POSTGRES_INITDB_ARGS
              value: "-A md5"
          volumeMounts:
            - name: test-app-storage
              mountPath: /var/lib/postgresql-data
              subPath: postgres-storage
      volumes:
        - name: test-app-storage
          persistentVolumeClaim:
            claimName: test-app-storage
     ...

Results in the following error:

The files belonging to this database system will be owned by user "postgres". This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english".

Data page checksums are disabled.

initdb: error: directory "/var/lib/postgresql-data" exists but is not empty If you want to create a new database system, either remove or empty the directory "/var/lib/postgresql-data" or run initdb with an argument other than "/var/lib/postgresql-data".

Try this:

            ...
            - name: POSTGRES_INITDB_ARGS
              value: "-A md5"
          volumeMounts:
            - name: test-app-storage
              mountPath: /var/lib/postgresql-data
              subPath: postgres-storage
      volumes:
        - name: test-app-storage
          persistentVolumeClaim:
            claimName: test-app-storage
     ...

And it results in this:

chmod: changing permissions of '/var/lib/postgresql-data': Operation not permitted

Try this:

            ...
            - name: POSTGRES_INITDB_ARGS
              value: "-A md5"
              value: "-D /var/lib/postgresql/data/pgdata"
          volumeMounts:
            - name: test-app-storage
              mountPath: /var/lib/postgresql/data/pgdata
              subPath: postgres-storage
      volumes:
        - name: test-app-storage
          persistentVolumeClaim:
            claimName: test-app-storage
     ...

And it results in this:

The files belonging to this database system will be owned by user "postgres". This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english".

Data page checksums are disabled.

initdb: error: could not change permissions of directory "/var/lib/postgresql/data/pgdata": Operation not permitted fixing permissions on existing directory /var/lib/postgresql/data/pgdata ...

So nothing seems to work that I've tried and following the documentation where I can.

Someone suggested to get rid of the volume mounts like so:

            ...
            - name: POSTGRES_INITDB_ARGS
              value: "-A md5"
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
      volumes:
        - name: test-app-storage
          persistentVolumeClaim:
            claimName: test-app-storage
     ...

Which, hey, that actually works! But doesn't persist data given it just uses the Pod storage so is pretty pointless:

And sure enough when you create a table in Postgres, destroy the Pod, and then redeploy it, of course the table is no longer there.

So more than likely I'm doing something wrong, but I've been following the documentation and seems like this should work.

Where are things going wrong?

EDIT: Permissions in Pod

Apparently it is a permissions issue that occurs when PGDATA is the same directory as mountPath. For example:

  ...
  - name: PGDATA
    value: /var/lib/postgresql-data
volumeMounts:
  - name: test-app-storage
    mountPath: /var/lib/postgresql-data
    subPath: postgres-storage
...

or

  ...
  # if PGDATA is not specified it defaults to /var/lib/postgresql/data
  # - name: PGDATA
  #   value: /var/lib/postgresql-data
volumeMounts:
  - name: test-app-storage
    mountPath: /var/lib/postgresql/data
    subPath: postgres-storage
...

Something like this where they do not match will create the Pod, but uses Pod storage which I obviously don't want:

  # Thus /var/lib/postgresql/data
  # - name: PGDATA
  #   value: /var/lib/postgresql-data
volumeMounts:
  - name: test-app-storage
    mountPath: /var/lib/postgresql-data
    subPath: postgres-storage

Permissions ls -l looks like this:

$ ls -l

drwxr-xr-x 1 root     root     4096 Feb  2 06:06 apt
drwxr-xr-x 1 root     root     4096 Feb  2 06:07 dpkg
drwxr-xr-x 2 root     root     4096 Feb  2 06:06 exim4
drwxr-xr-x 2 root     root     4096 Aug 28  2018 logrotate
drwxr-xr-x 2 root     root     4096 Nov 10 12:17 misc
drwxr-xr-x 2 root     root     4096 Jan 30 00:00 pam
drwxr-xr-x 1 postgres postgres 4096 Feb  2 06:07 postgresql
drwxrwxrwx 2     1000     1000    0 Jan 31 21:46 postgresql-data
drwxr-xr-x 1 root     root     4096 Jan 30 00:00 systemd
drwxr-xr-x 3 root     root     4096 Feb  2 06:07 ucf

$ ls -l postgresql && ls -l postgresql/data && ls -l postgresql-data
total 4
drwx------ 19 postgres postgres 4096 Feb  5 23:28 data
total 124
drwx------ 6 postgres postgres  4096 Feb  5 23:28 base
drwx------ 2 postgres postgres  4096 Feb  5 23:29 global
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_commit_ts
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_dynshmem
-rw------- 1 postgres postgres  4281 Feb  5 23:28 pg_hba.conf
-rw------- 1 postgres postgres  1636 Feb  5 23:28 pg_ident.conf
drwx------ 4 postgres postgres  4096 Feb  5 23:33 pg_logical
drwx------ 4 postgres postgres  4096 Feb  5 23:28 pg_multixact
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_notify
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_replslot
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_serial
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_snapshots
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_stat
drwx------ 2 postgres postgres  4096 Feb  5 23:51 pg_stat_tmp
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_subtrans
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_tblspc
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_twophase
-rw------- 1 postgres postgres     3 Feb  5 23:28 PG_VERSION
drwx------ 3 postgres postgres  4096 Feb  5 23:28 pg_wal
drwx------ 2 postgres postgres  4096 Feb  5 23:28 pg_xact
-rw------- 1 postgres postgres    88 Feb  5 23:28 postgresql.auto.conf
-rw------- 1 postgres postgres 26588 Feb  5 23:28 postgresql.conf
-rw------- 1 postgres postgres    36 Feb  5 23:28 postmaster.opts
-rw------- 1 postgres postgres    94 Feb  5 23:28 postmaster.pid
total 0

The permissions for where it creates the data files is postgres. However, doing this, it doesn't map to Azure Files and the PVC. It just stays and is destroyed with the Pod.

I think what is happening is mountPath uses root and PGDATA uses postgres, and somehow mountPath is trying to use postgres???

Really, not sure and still lost as to how to resolve it.

EDIT2

Came across this answer:

https://stackoverflow.com/a/51203031/3123109

So added the following to mine:

- name: postgres
  image: postgres
  command: 
  - /bin/chown
  - -R
  - "1000"
  - /var/lib/postgresql/data

But this generates a new error:

The selected container has not logged any messages yet.

Progress, I guess.

解决方案

I think your problem might be due to the fact that you are trying to use a subPath as part of your deployment object

Please retry your first configuration without this line:

subPath: postgres-storage

It should result into this volumeMounts: - name: postgres-storage mountPath: /var/lib/postgresql/data/pgdata

Let me know if that helps.

UPDATE: The docker image for the postgresql requires special attention when using persistent storage

https://hub.docker.com/_/postgres

The main caveat to note is that postgres doesn't care what UID it runs as (as long as the owner of /var/lib/postgresql/data matches), but initdb does care (and needs the user to exist in /etc/passwd):

The three easiest ways to get around this:

1. use the Debian variants (not the Alpine variants) and thus allow the image to use the nss_wrapper library to "fake" /etc/passwd contents for you (see docker-library/postgres#448 for more details)

2. bind-mount /etc/passwd read-only from the host (if the UID you desire is a valid user on your host):

$ docker run -it --rm --user "$(id -u):$(id -g)" -v /etc/passwd:/etc/passwd:ro -e POSTGRES_PASSWORD=mysecretpassword postgres The files belonging to this database system will be owned by user "jsmith"

3. Initialize the target directory separately from the final runtime (with a chown in between):

$ docker volume create pgdata $ docker run -it --rm -v pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=mysecretpassword postgres The files belonging to this database system will be owned by user "postgres". ( once it's finished initializing successfully and is waiting for connections, stop it ) $ docker run -it --rm -v pgdata:/var/lib/postgresql/data bash chown -R 1000:1000 /var/lib/postgresql/data $ docker run -it --rm --user 1000:1000 -v pgdata:/var/lib/postgresql/data postgres LOG: database system was shut down at 2017-01-20 00:03:23 UTC LOG: MultiXact member wraparound protections are now enabled LOG: autovacuum launcher started LOG: database system is ready to accept connections

One solution would be to leverage the helm chart made by bitnami, they have worked out the default of the complex persistent storage configurations using an init container, and they also support the /dev/shm configuration needed to fix

Also note that the default /dev/shm size for containers is 64MB. If the shared memory is exhausted you will encounter ERROR: could not resize shared memory segment . . . : No space left on device. You will want to pass --shm-size=256MB for example to docker run, or alternatively in docker-compose

Here's the example of the init container used by the chart.

https://github.com/helm/charts/blob/master/stable/postgresql/templates/statefulset.yaml#L74-L115

The recommendation would be to use helm to install the Postgres chart (which uses a statefuset) instead of trying to deploy the DB using a deployment object. (normally you want to use deployment objects for stateless application).

这篇关于无法在AKS中获得Postgres权限或PVC的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！