如何使用 spring 批处理仅解析 XML 的选定部分并将其转换为 java POJO [英] How to parse only selected portion of XML using spring batch and convert it into java POJO
问题描述
我正在尝试解析 XML 文件并从中转换 java POJO.我的示例 XML 看起来像
I am trying to parse XML files and convert java POJO's from them. My Sample XML looks like
students.xml
students.xml
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<group>
<college>
<name>Hogwards</name>
<city>Unknown</city>
</college>
<student>
<name>Tony Tester</name>
<rollNo>1</rollNo>
<enrollmentDate>2016-10-31</enrollmentDate>
<sampleTimeStamp>2016-11-07T05:50:45</sampleTimeStamp>
<salary>16.57</salary>
</student>
<student>
<name>Nick Newbie</name>
<rollNo>2</rollNo>
<enrollmentDate>2017-10-31</enrollmentDate>
<sampleTimeStamp>2016-11-07T05:50:45</sampleTimeStamp>
<salary>29.68</salary>
</student>
<student>
<name>Ian Intermediate</name>
<rollNo>3</rollNo>
<enrollmentDate>2018-10-31</enrollmentDate>
<sampleTimeStamp>2016-11-07T05:50:45</sampleTimeStamp>
<salary>789.62</salary>
</student>
</group>
在这里，我的目标是使用 spring 批处理解析文件并将学生信息填充到数据库中，出于我的目的，大学信息对我来说是一种完全没用的标题，因此在我的批处理阅读器中，我想忽略它只是我想分块解析学生信息.到目前为止，我的代码正在使用 GroupDTO 类一次解析整个记录，并且一次在对象上创建，因此我无法利用 spring 批处理的功能.我的要求是，学生信息应该分块解析，比如分块大小为 300 左右.但截至目前，我的代码一次解析整个 XML 文件并从中填充 java 对象.请帮助我忽略大学部分，只使用 Spring Batch 分块解析学生部分，或者建议一些适当的链接，这可能会帮助我为我的问题找到一些解决方案.提前致谢...
Here, my goal is to parse the file and populate the student information into database using spring batch and for my purpose college information is kind of header for me which is totally useless and so in my batch reader i would like to ignore it and just i want to parse the student information in chunks. As of now my code is using GroupDTO class to parse the whole record at once and is creating at objects at a single time as a result of which i am unable to leveage the functionality of spring batch. My requirement says that student information should be parsed in chunks let's say in a chunk size of 300 or so. But as of now my code parses the whole XML files at one time and populate java objects from it. Please help me to ignore the college section part and just parse student section part in chunks using spring batch or suggest some appropriate link which may help me to find some solution for my issue. Thanks in advance...
XmlConfiguration.java
XmlConfiguration.java
@Configuration
public class XmlConfiguration
{
@Autowired
JobBuilderFactory jobBuilderFactory;
@Autowired
StepBuilderFactory stepBuilderFactory;
@StepScope
@Bean(name="xmlReader")
public SynchronizedItemStreamReader<GroupDTO> reader()
{
StaxEventItemReader<GroupDTO> xmlFileReader = new StaxEventItemReader<>();
xmlFileReader.setResource(new ClassPathResource("students.xml"));
xmlFileReader.setFragmentRootElementName("group");
Map<String, Class<?>> aliases = new HashMap<>();
aliases.put("group", GroupDTO.class);
aliases.put("college", CollegeDTO.class);
aliases.put("student", StudentDTO.class);
XStreamMarshaller xStreamMarshaller = new XStreamMarshaller();
xStreamMarshaller.setAliases(aliases);
String dateFormat = "yyyy-MM-dd";
String timeFormat = "HHmmss";
String[] acceptableFormats = {timeFormat};
xStreamMarshaller.getXStream().autodetectAnnotations(true);
xStreamMarshaller.getXStream().registerConverter(new DateConverter(dateFormat, acceptableFormats));
xStreamMarshaller.getXStream().addPermission(NoTypePermission.NONE);
xStreamMarshaller.getXStream().addPermission(NullPermission.NULL);
xStreamMarshaller.getXStream().addPermission(PrimitiveTypePermission.PRIMITIVES);
xStreamMarshaller.getXStream().allowTypeHierarchy(Collection.class);
xStreamMarshaller.getXStream().allowTypesByWildcard(new String[] {"com.example.demo.**"});
xStreamMarshaller.getXStream().addImplicitCollection(GroupDTO.class, "list");
xmlFileReader.setUnmarshaller(xStreamMarshaller);
SynchronizedItemStreamReader<GroupDTO> synchronizedItemStreamReader = new SynchronizedItemStreamReader<>();
synchronizedItemStreamReader.setDelegate(xmlFileReader);
return synchronizedItemStreamReader;
}
@Bean(name="xmlProcessor")
public ItemProcessor<GroupDTO, GroupDTO> processor()
{
return new Processor();
}
@Bean(name="xmlWriter")
public ItemWriter<GroupDTO> writer()
{
return new Writer();
}
@Bean(name="xmljobListener")
public JobExecutionListenerSupport jobListener()
{
return new JobListener();
}
@JobScope
@Bean(name="xmltaskExecutor")
public ThreadPoolTaskExecutor taskExecutor()
{
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(50);
executor.setMaxPoolSize(100);
return executor;
}
@Bean(name="xmlStep")
public Step xmlFileToDatabaseStep()
{
return stepBuilderFactory.get("xmlStep")
.<GroupDTO, GroupDTO>chunk(2)
.reader(this.reader())
.processor(this.processor())
.writer(this.writer())
.taskExecutor(this.taskExecutor())
.build();
}
@Bean(name="xmlJob")
public Job xmlFileToDatabaseJob(@Autowired @Qualifier("xmlStep") Step step)
{
return jobBuilderFactory
.get("xmlJob"+new Date())
.incrementer(new RunIdIncrementer())
.listener(this.jobListener())
.flow(step)
.end()
.build();
}
}
GroupDTO.java
GroupDTO.java
@XStreamAlias("group")
public class GroupDTO
{
@XStreamAlias("college")
private CollegeDTO college;
@XStreamAlias("student")
private List<StudentDTO> list;
...... getter,setter, constructors
}
CollegeDTO.java
CollegeDTO.java
public class CollegeDTO
{
private String name;
private String city;
...... getter,setter and constructor
}
StudentDTO.java
StudentDTO.java
public class StudentDTO
{
private String name;
private Integer rollNo;
private Date enrollmentDate;
private Date sampleTimeStamp;
private BigDecimal salary;
... getter, setter and constructor
}
推荐答案
在作业内部，您拥有可以使用块标记的 tasklet.它将具有读取器和写入器属性，并且可以具有处理器属性.处理器"是可选的.
Inside the job you have the tasklet that can use the chunk tag. It will has reader and writer properties and can it can has processor properties. The "processor" is optional.
<batch:job id="helloWorldJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="itemReader" writer="itemWriter"
processor="itemProcessor" commit-interval="10">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
然后，当您声明 reader 标签时，您将定义映射器.
Then, when you declare the reader tag you will define the mapper.
<!-- READER -->
<bean id = "itemReader"
class = "org.springframework.batch.item.file.FlatFileItemReader">
...
<property name = "lineMapper">
<bean class = "org.springframework.batch.item.file.mapping.DefaultLineMapper">
...
<property name = "fieldSetMapper">
<bean class = "tudy.batch.Mapper" />
</property>
</bean>
</property>
</bean>
这个 Mapper 类是做你想做的事的一个很好的选择.此映射器将读取输入文件.我想你需要做的就是忽略大学标签.
This Mapper class is a great option to do what you want. This mapper will read the input file. What I imagine you need to do is just ignore the college tag.
public class Mapper implements FieldSetMapper<Student>{
public Student mapFieldSet(FieldSet fieldSet) throws BindException {
// Instantiating the report object
Student student = new Student();
// Setting the fields
student.setName(fieldSet.readInt(0));
student.setRollNo(fieldSet.readString(1));
student.setEnrollmentDate(fieldSet.readString(2));
student.setSampleTimeStamp(fieldSet.readString(3));
student.setSalary(fieldSet.readString(4));
return Student;
}
}
您可以使用索引或名称.你应该调试你的代码并确认大学将如何忽略它的位置或名称.
You can use the index or the name. You should debug your code and confirm the position or the name how the college is coming to ignore that.
这篇关于如何使用 spring 批处理仅解析 XML 的选定部分并将其转换为 java POJO的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
