Spring Batch是否在处理每个批次后释放堆内存? [英] Does Spring Batch release the heap memory after processing each batch?

问题描述

我有一个Spring批处理作业，其中仅包含一个步骤，该步骤使用 FlatFileItemReader 读取CSV文件(包含约2000行)并将对象写入数据库.我有自己的自定义 BeanWrapperFieldSetMapper ，它将行映射到对象.块大小设置为50，因此我希望在写入每个批次的对象(50个对象)之后，将释放这些对象的堆内存.

I have a Spring batch job which contains only one step that reads a CSV file(containing approximately 2000 rows) using a FlatFileItemReader and writes the objects to the database. I have my own custom BeanWrapperFieldSetMapper which maps the rows to objects. The chunk size is set to 50, so I am expecting after writing the objects from each batch (50 objects), the heap memory to be released of those objects.

由于我正在利用批处理，因此我希望在每个给定时间只有50个CreditCardDebt对象.但是，相反，在处理最后一批时，我发现堆内存包含2000个CreditCardDebt对象.

Since I'm leveraging the batch processing, I'm expecting at each given time to only have 50 CreditCardDebt objects. But instead, while processing the last batch I find the heap memory to contain 2000 CreditCardDebt objects.

我想念什么?

我的BeanWrapperFieldSetMapper实现:

My BeanWrapperFieldSetMapper implementation:

@Component("CREDIT_CARD_DEBT_FIELD_SET_MAPPER_TEST")
public class TestDebtFieldSetMapper extends BeanWrapperFieldSetMapper<CreditCardDebt> {

    public TestDebtFieldSetMapper() {
        super.setPrototypeBeanName("CREDIT_CARD_DEBT_FIELD_SET_MAPPER_TEST");
    }

    @NonNull
    @Override
    public CreditCardDebt mapFieldSet(FieldSet fieldSet) {
        CreditCardDebt creditCardDebt = new CreditCardDebt();
        creditCardDebt.setAccount(fieldSet.readString(0));
        creditCardDebt.setCardholderId(fieldSet.readString(1));
        creditCardDebt.setDueDate(convertToLocalDateViaInstant(fieldSet.readString(2)));
        creditCardDebt.setDaysPastDue(fieldSet.readInt(3));
        creditCardDebt.setOverdueAmount(fieldSet.readDouble(4));
        creditCardDebt.setDirectDebitMinimumPayment(fieldSet.readDouble(5));
        creditCardDebt.setDirectDebitBalance(fieldSet.readDouble(6));
        creditCardDebt.setDirectDebitStatus(fieldSet.readChar(7));
        creditCardDebt.setDirectDebitType(DirectDebitType.valueOf(fieldSet.readString(8)));
        creditCardDebt.setCreatedDate(LocalDateTime.now());
        creditCardDebt.setFileName("BAL");
        return creditCardDebt;
    }

    private LocalDate convertToLocalDateViaInstant(String dateToConvert) {
        DateTimeFormatter formatters = DateTimeFormatter.ofPattern("yyyyMMdd");
        return LocalDate.parse(dateToConvert, formatters);
    }

推荐答案

这留给垃圾收集器.与这个问题相关的相关代码部分位于

This is left to the garbage collector. The relevant code section related to this question is in the ChunkOrientedTasklet. In the most basic form of ChunkOrientedTasklet, there are two calls:

Chunk<I> inputs = chunkProvider.provide(contribution);
chunkProcessor.process(contribution, inputs);

ChunkProvider 使用 ItemReader 读取 commit-interval 项(如果项读取器返回 null ).而 ChunkProcessor 使用 ItemProcessor 和 ItemWriter 来处理和写入项目:

The ChunkProvider uses the ItemReader to read commit-interval items (or less if the item reader returns null). And the ChunkProcessor uses ItemProcessor and ItemWriter to process and write items:

Chunk<O> outputs = transform(contribution, inputs);
write(contribution, inputs); // details of adjustments of output omitted here

此过程反复运行，直到数据源用尽.因此，当GC启动时(因为变量 input / outputs 被重新使用)，应该处理垃圾块的项目，除非在整个工作期间将它们保存在内存中执行.

This process is run repeatedly until the datasource is exhausted. So items of processed chunks should be garbage collected when the GC kicks in (since variables inputs/outputs are re-used) unless something is holding them in memory during the whole job execution.

这篇关于Spring Batch是否在处理每个批次后释放堆内存?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！