java - 如何为spring批处理数据和业务数据配置单独的数据源?我应该这样做吗? [英] How to java-configure separate datasources for spring batch data and business data? Should I even do it?
问题描述
我的主要工作只做读取操作，另一个做一些写操作，但是在忽略事务的 MyISAM 引擎 上，所以我不需要一定的事务支持.如何配置 Spring Batch 以拥有自己的 JobRepository 数据源，与保存业务数据的数据源分开?最初的一个数据源配置是这样完成的:
My main job does only read operations and the other one does some writing but on MyISAM engine which ignores transactions, so I wouldn't require necessarily transaction support. How can I configure Spring Batch to have its own datasource for the JobRepository, separate from the one holding the business data? The initial one datasource-configurations is done like the following:
@Configuration
public class StandaloneInfrastructureConfiguration {
@Autowired
Environment env;
@Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(dataSource());
em.setPackagesToScan(new String[] { "org.podcastpedia.batch.*" });
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
em.setJpaProperties(additionalJpaProperties());
return em;
}
Properties additionalJpaProperties() {
Properties properties = new Properties();
properties.setProperty("hibernate.hbm2ddl.auto", "none");
properties.setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL5Dialect");
properties.setProperty("hibernate.show_sql", "true");
return properties;
}
@Bean
public DataSource dataSource(){
return DataSourceBuilder.create()
.url(env.getProperty("db.url"))
.driverClassName(env.getProperty("db.driver"))
.username(env.getProperty("db.username"))
.password(env.getProperty("db.password"))
.build();
}
@Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory emf){
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
}
然后它被导入到 Job 的配置类中，其中 @EnableBatchProcessing 注释自动使用它.我最初的想法是尝试设置配置类扩展 DefaultBatchConfigurer，但后来我得到了一个
and then it is imported in the Job's configuration class where the @EnableBatchProcessing annotation automagically makes use of it. My initial thought was to try to set the configuration class extend the DefaultBatchConfigurer, but then I get a
BeanCurrentlyInCreationException(org.springframework.beans.factory.BeanCurrentlyInCreationException:创建名为 jobBuilders 的 bean 时出错:当前正在创建请求的 bean:是否存在无法解析的循环引用?):
BeanCurrentlyInCreationException ( org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name jobBuilders: Requested bean is currently in creation: Is there an unresolvable circular reference?):
@Configuration
@EnableBatchProcessing
@Import({StandaloneInfrastructureConfiguration.class, NotifySubscribersServicesConfiguration.class})
public class NotifySubscribersJobConfiguration extends DefaultBatchConfigurer {
@Autowired
private JobBuilderFactory jobBuilders;
@Autowired
private StepBuilderFactory stepBuilders;
@Autowired
private DataSource dataSource;
@Autowired
Environment env;
@Override
@Autowired
public void setDataSource(javax.sql.DataSource dataSource) {
super.setDataSource(batchDataSource());
}
private DataSource batchDataSource(){
return DataSourceBuilder.create()
.url(env.getProperty("batchdb.url"))
.driverClassName(env.getProperty("batchdb.driver"))
.username(env.getProperty("batchdb.username"))
.password(env.getProperty("batchdb.password"))
.build();
}
@Bean
public ItemReader<User> notifySubscribersReader(){
JdbcCursorItemReader<User> reader = new JdbcCursorItemReader<User>();
String sql = "select * from users where is_email_subscriber is not null";
reader.setSql(sql);
reader.setDataSource(dataSource);
reader.setRowMapper(rowMapper());
return reader;
}
........
}
任何想法都非常受欢迎.该项目可在 GitHub 上获得 - https://github.com/podcastpedia/podcastpedia-batch
Any thoughts are more than welcomed. The project is available on GitHub - https://github.com/podcastpedia/podcastpedia-batch
非常感谢.
推荐答案
好吧，这很奇怪，但它有效.将数据源移动到它自己的配置类工作得很好，并且能够自动装配.
Ok, this is strange but it works. Moving the datasources to it's own configuration class works just fine and one is able to autowire.
示例是Spring批处理服务示例的多数据源版本:
The example is a multi-datasource version of Spring Batch Service Example:
数据源配置:
public class DataSourceConfiguration {
@Value("classpath:schema-mysql.sql")
private Resource schemaScript;
@Bean
@Primary
public DataSource hsqldbDataSource() throws SQLException {
final SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
dataSource.setDriver(new org.hsqldb.jdbcDriver());
dataSource.setUrl("jdbc:hsqldb:mem:mydb");
dataSource.setUsername("sa");
dataSource.setPassword("");
return dataSource;
}
@Bean
public JdbcTemplate jdbcTemplate(final DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
@Bean
public DataSource mysqlDataSource() throws SQLException {
final SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
dataSource.setDriver(new com.mysql.jdbc.Driver());
dataSource.setUrl("jdbc:mysql://localhost/spring_batch_example");
dataSource.setUsername("test");
dataSource.setPassword("test");
DatabasePopulatorUtils.execute(databasePopulator(), dataSource);
return dataSource;
}
@Bean
public JdbcTemplate mysqlJdbcTemplate(@Qualifier("mysqlDataSource") final DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
private DatabasePopulator databasePopulator() {
final ResourceDatabasePopulator populator = new ResourceDatabasePopulator();
populator.addScript(schemaScript);
return populator;
}
}
批量配置:
@Configuration
@EnableBatchProcessing
@Import({ DataSourceConfiguration.class, MBeanExporterConfig.class })
public class BatchConfiguration {
@Autowired
private JobBuilderFactory jobs;
@Autowired
private StepBuilderFactory steps;
@Bean
public ItemReader<Person> reader() {
final FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
reader.setResource(new ClassPathResource("sample-data.csv"));
reader.setLineMapper(new DefaultLineMapper<Person>() {
{
setLineTokenizer(new DelimitedLineTokenizer() {
{
setNames(new String[] { "firstName", "lastName" });
}
});
setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {
{
setTargetType(Person.class);
}
});
}
});
return reader;
}
@Bean
public ItemProcessor<Person, Person> processor() {
return new PersonItemProcessor();
}
@Bean
public ItemWriter<Person> writer(@Qualifier("mysqlDataSource") final DataSource dataSource) {
final JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<Person>();
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());
writer.setSql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)");
writer.setDataSource(dataSource);
return writer;
}
@Bean
public Job importUserJob(final Step s1) {
return jobs.get("importUserJob").incrementer(new RunIdIncrementer()).flow(s1).end().build();
}
@Bean
public Step step1(final ItemReader<Person> reader,
final ItemWriter<Person> writer, final ItemProcessor<Person, Person> processor) {
return steps.get("step1")
.<Person, Person> chunk(1)
.reader(reader)
.processor(processor)
.writer(writer)
.build();
}
}
这篇关于java - 如何为spring批处理数据和业务数据配置单独的数据源?我应该这样做吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
