分组/汇总spring-batch记录 [英] Grouping/summarizing spring-batch records

问题描述

我是spring-batch的新手，我想用它来实现以下过程:

I'm new to spring-batch and I want to use it to implement the following proces:

read: a list of items from webservice
process: groups these items
write: these groups back to webservice

示例:

Input:
transactionID, store, amount
1, FooStore, 2.50
2, BarStore, 19.99
3, FooStore, 12,49

Output:
totalsID, store, amount
1, FooStore, 14.99
2, BarStore, 19.99

使用spring-batch甚至有可能吗?我是否错过了重要的一步，还是与批处理的概念发生冲突，以至于我应该寻找其他解决方案?

Is this even possible using spring-batch, am I missing an important step, or does this conflict with the concept of batch processing to such an extent that I should look for a different solution?

我找到了一种可能的解决方案，即使用writeFooter方法， 但这似乎很愚蠢，因为我担心我会失去使用spring-batch的很多好处，例如错误恢复和内存管理.另外，据我所知，仅当将输出写入文件而不发送到Web服务时，才支持页脚.

I have found a possible solution, that is by using the writeFooter method, but that seems silly because I fear I'll lose a lot of the benefits of using spring-batch like the error recovery and memory management. Plus as far as I can see the footer is only supported if the output is written to a file, not send to a webservice.

推荐答案

用两个步骤编写作业:

1. 从输入中读取每条记录，然后在自己的ItemWriter.write()中更新汇总表
2. 将摘要表转储到文本文件(或网络服务)

1. Read every record and from input then, in your own ItemWriter.write(), update a summary table
2. Dump your summary table to text file (or webservice)

您还可以尝试使用诸如SUM()和COUNT()之类的聚合函数编写单个SQL语句来实现目标，然后通过单步作业读取从自定义SQL生成的每条记录，然后直接将其写入文本文件(或网络服务).

You can also try to reach the goal writing a single SQL statement using aggregate function like SUM() and COUNT() then, with a single-step job, read every record generated from custom SQL and write directly to you text file (or webservice).

这篇关于分组/汇总spring-batch记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！