用spring批处理读取复杂的json文件 [英] read complex json file with spring batch

问题描述

我有一个复杂的json文件（带有嵌套的json数组）结构，如下所示：

i have a complex json file( with nested json arrays) structure like this:

{"persons":[ 
{"id":"1", "firstName": "X", "lastName": "X", "infos": [{"address":[{"city": "X", "country": "X"}]}]},
{"id":"2", "firstName": "Y", "lastName": "Y", "infos": [{"address":[{"city": "Y", "country": "Y"}]}]}
]}

我想分别阅读每一行（一个人）

I want to read each line ( one person) separately

所以我的春季批量配置是这样的

So my spring batch configuration is like this

<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader"
    scope="step">
    <property name="resource" value="#{jobParameters[file]}" />
    <property name="recordSeparatorPolicy" ref="recordPolicy" />
    <property name="lineMapper" ref="lineMapper" />
</bean>

<bean id="lineMapper"
    class="com.batchs.personJob.PersonLineMapper">
    <property name="delegate" ref="lineMapperType" />
</bean>

<bean id="lineMapperType"
    class="org.springframework.batch.item.file.mapping.JsonLineMapper" />

<bean id="recordPolicy"
    class="org.springframework.batch.item.file.separator.JsonRecordSeparatorPolicy" />

mapper类看起来像

The mapper class looks like

    public class PersonLineMapper implements LineMapper<Person> {
    private JsonLineMapper delegate;

    public mapLine(String line, int lineNumber) throws Exception {
        Map<String, Object> personAsMap = delegate.mapLine(line, lineNumber);
        Person person = new Person();
        // map fields
        return person ;
    }

    public void setDelegate(JsonLineMapper delegate) {
        this.delegate = delegate;
    }
}

问题在于读者只读一行（所以一个提交），因为他读取我的json文件中的人员数组，就像整行，但我想读取而不是每行一行（一个人的时间）

The problem is than the reader reads only one line (so one commit) because he reads the persons array in my json file like a whole line but i want read instead line per line ( one person at time)

怎么做这个？

我试过这样一个简单的json文件：

I have tried with a simple json file like this :

  { "id": "1",
      "firstName": "X",
      "lastName": "X"}
  { "id": "2",
      "firstName": "Y",
      "lastName": "Y"}

它运作良好......我逐个阅读每个人

And it works good ... I read each person one by one

非常感谢

推荐答案

不幸的是，您有两种选择：

Unfortunately, you have two options:

1. 编写自己的记录分隔符，忽略包装器元素/


2. 在读取文件之前，先在步骤中编辑文件，使用类似sed命令的内容删除该包装行。

如果您选择选项2，您可以在实际处理它之前的步骤中通过 SystemCommandTasklet 执行它。

If you choose option 2, you can execute it via the SystemCommandTasklet in the step before the one you actually process it in.

这篇关于用spring批处理读取复杂的json文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！