使用Nifi-Registry和Docker对Nifi流文件进行版本控制和持久化 [英] Versioning and persisting Nifi flowfiles with Nifi-Registry and Docker

问题描述

目前，我正在使用最新的稳定版本Nifi / Nifi-registry和Docker。我正在尝试在Gitlab中对流文件进行版本控制和持久化。
我在网络上找到了一些信息，但现在无法正常工作。所有组件都在工作，我也可以将文件从Nifi升级到Nifi注册，但是目前缺少Gitlab的最后一步。

currently I'm playing with the latest stable version Nifi/Nifi-registry and Docker. I'm trying to version and persist flowfiles in Gitlab. I found several information in the web but right now it's not working. All components are working and I'm also able to version files from Nifi to Nifi-Registry, but the last step to Gitlab is currently missing.

->我修改了并将providers.xml安装到容器中

--> I modified and mounted the providers.xml into the container

<flowPersistenceProvider>
  <class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
  <property name="Flow Storage Directory">./versioned_flows</property>
  <property name="Remote To Push">origin</property>
  <property name="Remote Access User">*Name*</property>
  <property name="Remote Access Password">*Token*</property>
</flowPersistenceProvider>

andybody是否有一些经验，也许还有代码片段？

Does andybody have some experiences and maybe a code snippet? I would appreciate that.

非常感谢。

亲切的问候，
T

Kind regards, T

推荐答案

尽管这是一篇老文章，但如果对任何人有帮助，我都会很高兴。

Although it is an old post, I will be happy if it is helpful to anyone.

我有以下docker文件夹&它同时运行nifi和它是注册表容器。

I have the following docker folder & it runs both nifi & it's registry containers.

复制nifi目录中的 conf文件夹（可以从不运行卷的容器中获取）

Copy the 'conf' folder inside nifi directory (can be taken from running the container without volume)

运行 docker卷创建nifi_data

docker-compose.yml文件为：

The docker-compose.yml file is:

version: "3.7"
services:

nifi:
  container_name: nifi
  image: apache/nifi:1.11.4
  ports:
    - target: 8080
      published: 8080
      protocol: tcp
      mode: host
  restart: always
  environment:
    - NIFI_WEB_HTTP_HOST=0.0.0.0
    - NIFI_JVM_HEAP_INIT=4g
    - NIFI_JVM_HEAP_MAX=6g
    - NIFI_VARIABLE_REGISTRY_PROPERTIES=./conf/env.properties
  volumes:
    - nifi_data:/opt/nifi/nifi-current/
    - ./nifi/extensions:/opt/nifi/nifi-current/extensions
    - ./nifi/conf:/opt/nifi/nifi-current/conf

nifi-registry:
  container_name: nifi-registry
  image: apache/nifi-registry:0.7.0
  ports:
    - target: 18080
      published: 18080
      protocol: tcp
      mode: host
  environment:
    - NIFI_REGISTRY_WEB_HTTP_HOST=0.0.0.0
    - JVM_OPTS=-Xmx512m -Xms512m -XX:MaxPermSize=1g
  volumes:
    - $PWD/registry/providers.xml:/opt/nifi-registry/nifi-registry-current/conf/providers.xml
    - $PWD/registry/flow-storage:/opt/nifi-registry/nifi-registry-current/flow_storage
    - $PWD/registry/database:/opt/nifi-registry/nifi-registry-current/database
volumes:
  nifi_data:
    external: true
networks:
  default:
    external:
      name: nifi-network

注意：属性（env.properties）&自定义处理器位于扩展目录下。另外，由于nifi在 nifi用户下运行，您可能会遇到权限问题-我相信您会把它整理出来：）

Note: I have custom properties (env.properties) & the custom processors are under 'extensions' directory. Also, you may have permissions issue since nifi runs under 'nifi' user - I am sure you will sort it out :)

providers.xml（采用默认值-注释文件系统）提供者&取消注释git提供者）

The providers.xml (take the default one - comment the filesystem provider & uncomment the git provider)

<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
<property name="Flow Storage Directory">./flow_storage</property>

最后是restart.sh文件：

And finally the restart.sh file:

chown -R 1000:1000 nifi registry // nifi's user:group
chmod -R 775 nifi registry
chmod -R g+s nifi registry
docker-compose down
docker-compose up -d

这篇关于使用Nifi-Registry和Docker对Nifi流文件进行版本控制和持久化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！