与数据库 - nodeJS连接时同步弹性搜索 [英] Sync elasticsearch on connection with database - nodeJS

问题描述

目标：同步弹性搜索与postgres数据库

为什么：有时是newtwork或集群/服务器断开，所以将来的更新应该被记录

本文 https://qafoo.com/blog/086_how_to_synchronize_a_database_with_elastic_search。 html 表明，我应该创建一个单独的表更新，它将同步elasticsearch的 id ，允许选择新的自上次记录以来的数据（来自数据库）（在弹性搜索中）。所以我想如果我可以记录弹性搜索的失败和成功的连接怎么办？如果客户端 ponged 成功（返回承诺），我可以启动一个功能将记录与我的数据库同步。

这是我的 elasticConnect.js

 
 $ b const client = new elasticsearch.Client（{
 host：'localhost ：9200'，
 log：'trace'
}）; 
 
 
 client.ping（{
 requestTimeout：Infinity，
 hello：elasticsearch！
}）
 .then（（） => syncProcess）//成功连接
 .catch（err => console.error（err））
 
 
导出默认客户端
  

这样，我甚至不需要担心运行cron作业（如果问题 1 是正确的） ，因为我知道集群正在运行。

问题

1. 将 syncProcess 在之前运行导出默认客户端？我不希望任何请求进入同步...




2. syncProcess 应该只运行一次（因为它被缓存/不导出），无论多少次我 import elasticConnect.js 。




3. 使用更新表的方法有什么好处，而不是仅仅从父/源表？




4. 文章的评论说不要使用时间戳来比较新的数据！/呃。为什么？数据库阻塞后应该没问题吧？

解决方案

对于1：因为你没有保证 syncProcess 将在客户端导出时运行。相反，您应该执行类似于

对于2：使用上述问题中链接的解决方案，这将被处理

对于3：更新表也会捕获记录删除，而从DB中选择不会，因为您不知道哪些记录已经消失。 p>

对于4：您链接的文章后的第二条评论提供答案（提示：时间戳不是严格单调的）。

Aim: sync elasticsearch with postgres database
Why: sometimes newtwork or cluster/server break so future updates should be recorded

This article https://qafoo.com/blog/086_how_to_synchronize_a_database_with_elastic_search.html suggests that I should create a separate table updates that will sync elasticsearch's id, allowing to select new data (from database) since the last record (in elasticsearch). So I thought what if I could record elasticsearch's failure and successful connection: if client ponged back successfully (returned a promise), I could launch a function to sync records with my database.

Here's my elasticConnect.js

import elasticsearch from 'elasticsearch'
import syncProcess from './sync'

const client = new elasticsearch.Client({
  host:  'localhost:9200',
  log: 'trace'
});


client.ping({
   requestTimeout: Infinity,
   hello: "elasticsearch!"
})
.then(() => syncProcess) // successful connection 
.catch(err => console.error(err))


 export default client

This way, I don't even need to worry about running cron job (if question 1 is correct), since I know that cluster is running.

Questions

1. Will syncProcess run before export default client? I don't want any requests coming in while syncing...

2. syncProcess should run only once (since it's cached/not exported), no matter how many times I import elasticConnect.js. Correct?

3. Is there any advantages using the method with updates table, instead of just selecting data from parent/source table?

4. The articles' comments say "don't use timestamp to compare new data!".Ehhh... why? It should be ok since database is blocking, right?

解决方案

For 1: As it is you have not warranty that syncProcess will have run by the time the client is exported. Instead you should do something like in this answer and export a promise instead.

For 2: With the solution I linked to in the above question, this would be taken care of.

For 3: An updates table would also catch record deletions, while simply selecting from the DB would not, since you don't know which records have disappeared.

For 4: The second comment after the article you linked to provides the answer (hint: timestamps are not strictly monotonic).

这篇关于与数据库 - nodeJS连接时同步弹性搜索的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！