NIFI listfile 如何维护其时间戳? [英] how does NIFI listfile maintains its timestamp?

问题描述

我正在研究 NiFi，其中我使用了列表、获取等处理器.列表文件处理器在新文件到达上述目录系统时列出文件.到目前为止，我已经完成了从 2016 年 1 月至今的一年数据文件的处理.

i am working on NiFi in which i have used processors such as list, fetch and others. The list file processer lists the file as and when new files arrives in the mentioned directory system. till now, i have done processing of one year of data files that is from the year january 2016 till date.

现在我想将 flow.xml 文件导出到新环境并继续处理.当我启动列表处理器时，它从头开始，即 2016 年 1 月.无论如何，是否可以继续在先前环境处理中留下的列表处理器.

now i want to export flow.xml file to the new environment and continue the processing. when i start the list processor, it starts from the beginning i.e 2016 january. is there anyway to continue the listing processor where it left in the previous environment processing.

推荐答案

在独立的 NiFi 实例中，状态由 state-management.xml 中配置的本地状态提供程序存储:

In a standalone NiFi instance, the state is being stored by the local state provider configured in state-management.xml:

<local-provider>
        <id>local-provider</id>
        <class>org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider</class>
        <property name="Directory">./state/local</property>
        <property name="Always Sync">false</property>
        <property name="Partitions">16</property>
        <property name="Checkpoint Interval">2 mins</property>
    </local-provider>

您可以看到磁盘上的位置在 ./state/local 中，因此您需要将此目录结构及其下的所有内容复制到另一个 NiFi 实例上的相同位置.

You can see that the location on disk is in ./state/local so you would need to copy this directory structure and everything under it to the same location on the other NiFi instance.

此外，这仅在您将 flow.xml.gz 从一个实例复制到另一个实例时才有效，因为状态是根据组件的 UUID 存储的，因此您需要 UUID 保持不变.如果您使用模板来移动流，那么您将获得新的 UUID，并且状态将不会对齐.

Also, this only works if you are copying the flow.xml.gz from one instance to the other because state is stored based on the UUID of the component, so you need the UUIDs to stay the same. If you use templates to move the flow then you get new UUIDs and the state won't line up.

这篇关于NIFI listfile 如何维护其时间戳?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！