Java处理链中的排队作业 [英] Queuing jobs in a processing chain in Java

问题描述

我目前正在用Java设计一个关联引擎，该引擎将从pdf文件中提取数据，并从关系数据库中关联(在必要时发出警报)其结构化数据.

I am currently designing a correlation engine in java which is extracting data from pdf files and correlating (raising alerts where necessary) it structured data from a relational database.

根据pdf文件的处理，系统包括:

Focusing on the processing of the pdf files the system consists of:

正在从pdf执行自定义提取的组件.

A component which is performing the custom extraction from the pdf.

将有时无序的不整洁数据解析为所需数据结构的组件

A component which parses the sometimes unordered unclean data into the required data structures

将标准化值以进行比较的标准化组件

A normalisation component which will normalises the values for comparison

与数据库连接的组件(提取的数据将与其余数据一起插入)

And a component which interfaces with the db (where the extracted data will be inserted with the rest of the data)

这些组件应可在其他处理链中重用，但最初它们都将在同一系统上运行.

The components should be reusable in other processing chains but they will all run on the same system initially.

我认为在组件之间进行某种缓冲是明智的，使用JMS队列是否明智，否则这会使事情变得复杂吗?我一直在尝试使用一个简单的linkedblockingqueue对象，但是该对象必须在组件之间传递，因此它需要一个主组件，该主组件可以驱动我不确定所需的一切，是否有解决此问题的标准方法?

I think it's wise to have some sort of buffering between components, is it wise to be using JMS Queueing or would this over complicate matters? I have been experimenting with a simple linkedblockingqueue object but this object has to be passed between components so it requires a master components which drives everything which i am not sure is desirable, is there a standard way of approaching this problem?

推荐答案

除非您有其他要求，否则我将使用链接调用.

I would use chained calls unless you have additional requirements.

loadPDF(new PDFExtractor(new PDFParser(new Normalizer(new DBEnricher(listener)))));

如果您需要多个线程，我将使用ExecutorService线程池在不同的线程中处理每个文件.

If you want multiple threads, I would process each file in a different thread using an ExecutorService thread pool.

这篇关于Java处理链中的排队作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！