使用 Apache Camel 反复轮询只读文件系统中的文件(幂等 = false)? [英] Poll files from read-only filesystem repeatedly (idempotent=false) with Apache Camel?

问题描述

我正在使用轮询消费者模式从给定的只读目录中读取所有文件并进行处理.是否有忽略幂等性的选项?

I am using polling consumer pattern to read all files from given read-only directory and process them. Is there a option to ignore idempotency ?

我了解使用 noop=true & 定义的路线idempotent=false 会使整个系统崩溃(无限循环)，但池化消费者模式是一次性操作，在给定时刻触发.

I understand that route defined with noop=true & idempotent=false would crash whole system (infinite loop) but pooling consumer pattern is one-time operation which is triggered at given moment.

推荐答案

camel file2 端点不会导致系统崩溃，因为它默认每秒轮询 2 次.它将从当时该文件夹中的所有文件创建一条消息.(好吧，有一些选项可以忽略最近修改的文件，但其中大部分是).例如文件以每秒 2 次的速度发送到管道中.

The camel file2 endpoint would not crash the system, since it polls 2x per second by default. It would create a message from all files which are in that folder at the time. (well, there are some options to ignore recently modified files, but most of them). e.g. the files are sent 2x per second down the pipeline.

幂等性默认关闭，除非传递noop"以将文件实质上保留在输入文件夹中.但是，每次端点轮询时，所有文件都会通过管道.我通常在处理后移动文件(如果我有多个消费者，则在使用 preMove 之前移动文件)以避免这些重复并仍然避免幂等存储的复杂性.

Idempotency is off by default, unless 'noop' is passed to essentially leave the files in the input folder. However each time the endpoint polls, all files would go through the pipe. I normally move the files after processing (or before with preMove if I have multiple consumers) to avoid these duplicates and still avoid the complexity of idempotency storage.

选择 noop 时无法禁用幂等标志:

There is no way to disable the idempotent flag when noop is chosen :

    FileConsumer result = newFileConsumer(processor, operations);

    if (isDelete() && getMove() != null) {
        throw new IllegalArgumentException("You cannot set both delete=true and move options");
    }

    // if noop=true then idempotent should also be configured
    if (isNoop() && !isIdempotentSet()) {
        log.info("Endpoint is configured with noop=true so forcing endpoint to be idempotent as well");
        setIdempotent(true);
    }

在这种情况下，创建消费者后，幂等标志会被重置.

the idempotent flag is reset in that case after the consumer is created.

可以做的是创建一个 IdempotentRepository 的简单实现(http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/spi/IdempotentRepository.html) 忽略所有更新并告诉端点它以前从未见过这个文件.

What can be done is to create a trivial implementation of an IdempotentRepository (http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/spi/IdempotentRepository.html) which ignores all updates and tells the endpoint it has never seen this file before.

    package com.company;

    import org.apache.camel.spi.IdempotentRepository;

    public class DummyIdempotentRepository implements IdempotentRepository {
        @Override
        public boolean add(Object key) {
            return true;
        }

        @Override
        public boolean contains(Object key) {
            return false;
        }

        @Override
        public boolean remove(Object key) {
            return true;
        }

        @Override
        public boolean confirm(Object key) {
            return true;
        }

        @Override
        public void start() throws Exception {

        }

        @Override
        public void stop() throws Exception {

        }
    }

应该这样做.

这篇关于使用 Apache Camel 反复轮询只读文件系统中的文件(幂等 = false)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！