使用 Nifi-Registry 和 Docker 对 Nifi 流文件进行版本控制和持久化 [英] Versioning and persisting Nifi flowfiles with Nifi-Registry and Docker
问题描述
目前我正在使用最新的稳定版本 Nifi/Nifi-registry 和 Docker.我正在尝试在 Gitlab 中对流文件进行版本化和持久化.我在网上找到了一些信息,但现在它不起作用.所有组件都在工作,我还可以将文件从 Nifi 版本化到 Nifi-Registry,但目前缺少 Gitlab 的最后一步.
currently I'm playing with the latest stable version Nifi/Nifi-registry and Docker. I'm trying to version and persist flowfiles in Gitlab. I found several information in the web but right now it's not working. All components are working and I'm also able to version files from Nifi to Nifi-Registry, but the last step to Gitlab is currently missing.
--> 我修改providers.xml并将其挂载到容器中
--> I modified and mounted the providers.xml into the container
<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
<property name="Flow Storage Directory">./versioned_flows</property>
<property name="Remote To Push">origin</property>
<property name="Remote Access User">*Name*</property>
<property name="Remote Access Password">*Token*</property>
</flowPersistenceProvider>
andybody 是否有一些经验,也许还有代码片段?我会很感激的.
Does andybody have some experiences and maybe a code snippet? I would appreciate that.
非常感谢.
亲切的问候,T
推荐答案
虽然是老帖子,但如果对大家有帮助,我会很高兴的.
Although it is an old post, I will be happy if it is helpful to anyone.
我有以下 docker 文件夹 &它同时运行 nifi 和它是注册表容器.
I have the following docker folder & it runs both nifi & it's registry containers.
复制 nifi 目录中的 'conf' 文件夹(可以从运行没有卷的容器中获取)
Copy the 'conf' folder inside nifi directory (can be taken from running the container without volume)
运行 docker volume create nifi_data
docker-compose.yml 文件为:
The docker-compose.yml file is:
version: "3.7"
services:
nifi:
container_name: nifi
image: apache/nifi:1.11.4
ports:
- target: 8080
published: 8080
protocol: tcp
mode: host
restart: always
environment:
- NIFI_WEB_HTTP_HOST=0.0.0.0
- NIFI_JVM_HEAP_INIT=4g
- NIFI_JVM_HEAP_MAX=6g
- NIFI_VARIABLE_REGISTRY_PROPERTIES=./conf/env.properties
volumes:
- nifi_data:/opt/nifi/nifi-current/
- ./nifi/extensions:/opt/nifi/nifi-current/extensions
- ./nifi/conf:/opt/nifi/nifi-current/conf
nifi-registry:
container_name: nifi-registry
image: apache/nifi-registry:0.7.0
ports:
- target: 18080
published: 18080
protocol: tcp
mode: host
environment:
- NIFI_REGISTRY_WEB_HTTP_HOST=0.0.0.0
- JVM_OPTS=-Xmx512m -Xms512m -XX:MaxPermSize=1g
volumes:
- $PWD/registry/providers.xml:/opt/nifi-registry/nifi-registry-current/conf/providers.xml
- $PWD/registry/flow-storage:/opt/nifi-registry/nifi-registry-current/flow_storage
- $PWD/registry/database:/opt/nifi-registry/nifi-registry-current/database
volumes:
nifi_data:
external: true
networks:
default:
external:
name: nifi-network
注意: 我有自定义属性 (env.properties) &自定义处理器位于扩展"目录下.此外,由于 nifi 在 'nifi' 用户下运行,您可能会遇到权限问题 - 我相信您会解决的 :)
Note: I have custom properties (env.properties) & the custom processors are under 'extensions' directory. Also, you may have permissions issue since nifi runs under 'nifi' user - I am sure you will sort it out :)
providers.xml(取默认的 - 注释文件系统提供程序并取消注释 git 提供程序)
The providers.xml (take the default one - comment the filesystem provider & uncomment the git provider)
<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
<property name="Flow Storage Directory">./flow_storage</property>
最后是restart.sh文件:
And finally the restart.sh file:
chown -R 1000:1000 nifi registry // nifi's user:group
chmod -R 775 nifi registry
chmod -R g+s nifi registry
docker-compose down
docker-compose up -d
这篇关于使用 Nifi-Registry 和 Docker 对 Nifi 流文件进行版本控制和持久化的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!