通过Spring Data ElasticSearch将Spring数据JPA条目批量索引到弹性 [英] Batch indexing Spring Data JPA entries to Elastic through Spring Data ElasticSearch

问题描述

我们目前的设置是通过Spring Data JPA作为主要数据源的MySQL，使用Hibernate Search进行索引和搜索数据。我们现在决定去弹性搜索进行搜索，以便更好地与其他功能保持一致，此外我们需要有多个服务器共享索引和搜索。

Our current setup is MySQL as main data source through Spring Data JPA, with Hibernate Search to index and search data. We now decided to go to Elastic Search for searching to better align with other features, besides we need to have multiple servers sharing the indexing and searching.

通过 ElasticsearchRepository ，我可以使用Spring Data ElasticSearch设置Elastic数据索引和搜索。但现在的挑战是如何将所有现有的MySQL记录索引到弹性搜索中。 Hibernate Search提供了一个API来执行这个我们一直使用的 org.hibernate.search.jpa.FullTextEntityManager＃createIndexer 。但是我在Spring Data ElasticSearch中找不到一个方便的解决方案。希望有人能帮助我在这里或提供一些指针。

I'm able to setup Elastic using Spring Data ElasticSearch for data indexing and searching easily, through ElasticsearchRepository. But the challenge now is how to index all the existing MySQL records into Elastic Search. Hibernate Search provides an API to do this org.hibernate.search.jpa.FullTextEntityManager#createIndexer which we use all the time. But I cannot find a handy solution within Spring Data ElasticSearch. Hope somebody can help me out here or provide some pointers.

有一个类似的问题，但是在这里提出的解决方案并不适合我的需要，因为我希望能够索引整个对象，哪些字段映射到多个DB表。

There is a similar question here, however the solution proposed there doesn't fit my needs very well as I'd prefer to be able to index a whole object, which fields are mapped to multiple DB tables.

推荐答案

到目前为止，还没有找到一个比编写自己的代码更好的解决方案，将所有JPA条目索引到我的应用程序中的ES ，这个为我准备好了

So far I haven't found a better solution than writing my own code to index all JPA entries to ES inside my application, and this one worked out for me fine

Pageable page = new PageRequest(0, 100);
Page<Instance> curPage = instanceManager.listInstancesByPage(page);    //Get data by page from JPA repo.
long count = curPage.getTotalElements();
while (!curPage.isLast()) {
    List<Instance> allInstances = curPage.getContent();
    for (Instance instance : allInstances) {
        instanceElasticSearchRepository.index(instance);    //Index one by one to ES repo.
    }
    page = curPage.nextPageable();
    curPage = instanceManager.listInstancesByPage(page);
}

逻辑非常简单，只是取决于数据的数量花一点时间，所以分解成批次并添加一些消息可以是有帮助的。

The logic is very straightforward, just depending on the quantity of the data it might take a while, so breaking down to batches and adding some messages can be helpful.

这篇关于通过Spring Data ElasticSearch将Spring数据JPA条目批量索引到弹性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！