通过 ssh 同步 MongoDB [英] Sync MongoDB Via ssh

问题描述

与 Mysql 不同，我发现尝试同步 MongoDB 文件非常具有挑战性 -
它们无法通过管道返回，因为它们不会将数据发送到 stdout
(如果我理解正确的话).

Unlike Mysql, I found it quite challenging trying to sync MongoDB files -
They can not be piped back, since they don't send the data to stdout
(If I understand correctly).

所以，我正在尝试寻找另一种方法，即不涉及两次 ssh 调用.
需要做的是:

So, I'm trying to find another way, that doesn't involve two ssh calls.
What needs to be done is this:

• 登录ssh服务器
• 导出所有 MongoDB 文件
• 将它们压缩到 gzip
• 将它们发送回本地机器
• 提取和导入

不过，这里的关键是不留下任何痕迹-
我不希望压缩文件留在远程机器上，
这通常需要我再次登录 ssh.
因此，将文件移入存档"之类的东西是理想的解决方案，
如果以后可以无缝地将其传送回本地计算机.

The key thing here, though, is leaving no trace behind -
I don't want the compressed files stay in the remote machine,
which would usually require me for another ssh login.
So something along the lines of "move files into archive" is the ideal solution,
if that could be later piped back to the local machine seamlessly.

我意识到 MongoDB 有一种使用 mongodump 连接到服务器凭据的方法，但端口在 atm 中已关闭，因此我需要 SSH 方法.任何其他想法都会受到欢迎，顺便说一句.

I realize MongoDB has a method to connect to a server credentials using mongodump, but the port is closed atm, so I need the SSH method. Any other ideas would be welcome, BTW.

由于这个问题似乎有点流行，我想分享一个从这些问题的答案演变而来的脚本，以及去年的其他资源(信用是信用到期的地方).
该脚本基本上管理从/到远程服务器的同步，适用于任何一种类型的数据库(目前可能是 postgres、mysql 和 mongo).
它确实有一些假设，例如 root 用户没有数据库的密码，但可以根据需要进行更改.

Since this questions seems to be somewhat popular, I'd care to share a script that evolved from this questions' answers, and other resources during the last year (credit is where credit is due).
The script basically manages a sync from/to a remote server, for either type of db possible (probably. postgres, mysql and mongo for the time being).
It does have a few assumptions, like the root user having no password for the db, but that can be changed according to need.

脚本可以在这里找到:https://github.com/iwfmp/zsh/blob/master/scripts/db/db-sync

推荐答案

您可以通过 SSH 隧道实现这一点，将远程 MongoDB 实例设置为在您的本地端口之一上运行.默认情况下，MongoDB 在 27017 上运行，因此在下面的示例中，我选择将远程 MongoDB 实例映射到本地 27018 端口.

You can accomplish this with SSH Tunneling, setting up your remote MongoDB instance to run on one of your local ports. By default, MongoDB runs on 27017, so in the example below, I've chosen to map my remote MongoDB instance to my local 27018 port.

如果您尝试将数据库从 SERVER1 复制到 LOCALHOST，您可以在 LOCALHOST 上运行此命令:

If on your trying to copy a database from SERVER1 to LOCALHOST, you could run this command on your LOCALHOST:

ssh -L27018:localhost:27017 SERVER1

(显然用你的实际服务器或 ssh 别名替换 SERVER1)

(Obviously replace SERVER1 with your actual server or ssh alias)

这会打开到 SERVER1 的 SSH 连接，但也会将 LOCALHOST 上的端口 27018 映射到 SERVER1 上的远程端口 27017.不要关闭该 SSH 连接，现在尝试使用端口 27018 连接到本地主机上的 MongoDB，如下所示:

This opens an SSH connection to SERVER1, but also maps the port 27018 on LOCALHOST to the remote port 27017 on SERVER1. Don't close that SSH connection, and now try to connect to MongoDB on your localhost machine with port 27018, like so:

mongo --port 27018

您会注意到这是 SERVER1 上的数据，除非您是从本地计算机访问它.

You'll notice this is now the data on SERVER1, except you're accessing it from your local machine.

只是正常运行MongoDB:

Just running MongoDB normally:

mongo (或 mongo --port 27107)

将是您的本地机器.

现在，从技术上讲，您已经(在您运行 SSH 隧道的 LOCALHOST 上):

Now, since you technically have (on your LOCALHOST, where you ran the SSH tunnel):

• MongoDB (LOCALHOST) 27017
• 27018 上的 MongoDB (SERVER1)

您可以只使用 MongoDB (LOCALHOST) 中的 db.copyDatabase() 函数来复制数据.

You can just use the db.copyDatabase() function inside MongoDB (LOCALHOST) to copy over data.

来自 27017 端口的本地主机(在现场执行将删除您的数据)

// Use the right DB
use DATABASENAME; 
// Drop the Existing Data on LOCALHOST
db.dropDatabase();
// Copies the entire database from 27018
db.copyDatabase("DATABASENAME", "DATABASENAME", "localhost:27018");

您应该能够将所有这些包装成一个 shell 脚本，该脚本可以为您执行所有这些命令.我自己有一个，但它实际上有一些额外的步骤，可能会让它变得更加混乱:)

You should be able to wrap this all up into a shell script that can execute all of these commands for you. I have one myself, but it actually has a few extra steps that would probably make it a bit more confusing :)

这样做，并使用 MongoDB 的本机 db.copyDatabase() 函数将防止您必须转储/压缩/恢复.当然，如果你仍然想走那条路，运行 mongodump 并不会太难，导出数据，tar/gzip，然后使用 scp TARGETSERVER:/path/to/file/local/path/to/file 将其拉下来并在其上运行 mongorestore.

Doing this, and using MongoDB's native db.copyDatabase() function will prevent you from having to dump/zip/restore. Of course, if you still want to go that route, it wouldn't be too hard to run mongodump, export the data, tar/gzip it, then use scp TARGETSERVER:/path/to/file /local/path/to/file to pull it down and run a mongorestore on it.

似乎需要更多的工作！

编辑 - 这是一个 SH 和 JS 文件，它们共同构成了一个可以用来运行它的 shell 脚本.在您的本地主机上运行这些，不要在实时运行它们，否则它会在实时运行 db.dropDatabase.将这两个文件放在同一个文件夹中，将pull-db.sh中的YOURSERVERNAME替换为domain/ip/ssh别名，然后在pull-db中.js 将 DBNAMEHERE 更改为您的数据库名称.

Edit - Here's a SH and JS file that go together to make a shell script you can run this with. Run these on your LOCALHOST, don't run them on live or it'll do the db.dropDatabase on live. Put these two files in the same folder, and replace YOURSERVERNAME in pull-db.sh with the domain/ip/ssh alias, and then in pull-db.js change DBNAMEHERE to whatever your database name is.

我通常在我的项目中创建一个名为 scripts 的文件夹，使用 Textmate，我只需要在 pull-db.sh 的同时点击 ⌘+R 打开编辑以执行它.

I normally create a folder called scripts in my projects, and using Textmate, I just have to hit ⌘+R while having pull-db.sh open to edit in order to execute it.

pull-db.sh

ssh -L27018:localhost:27017 YOURSERVERNAME '
    echo "Connected on Remote End, sleeping for 10"; 
    sleep 10; 
    exit' &
echo "Waiting 5 sec on local";
sleep 5;
echo "Connecting to Mongo and piping in script";
cat pull-db.js | mongo

pull-db.js

use DBNAMEHERE;
db.dropDatabase();
use DBNAMEHERE;
db.copyDatabase("DBNAMEHERE","DBNAMEHERE","localhost:27018");

我在 shell 脚本中添加了一些额外的代码来显示它在做什么(有点).脚本中的睡眠定时器只是让 SSH 连接有时间在下一行运行之前建立连接.基本上，会发生以下情况:

I added some extra code to the shell script to echo out what it's doing (sorta). The sleep timers in the script are just to give the SSH connections time to get connected before the next line is run. Basically, here's what happens:

1. 代码的第一行在您的机器上创建隧道，然后发送 ECHO、SLEEP，然后 EXIT 到远程 SSH 会话.
2. 然后等待 5 秒钟，这将允许步骤 1 中的 SSH 会话连接.
3. 然后我们将 pull-db.js 文件通过管道传输到本地 mongo shell.(第 1 步应该在 5 秒内完成...)
4. pull-db.js 现在应该在 mongo 中运行，并且步骤 #1 中的 SSH 终端在连接打开后可能已经运行了 10 秒，并且 EXIT 被发送到它的会话.发出命令后，SSH 会话实际上将保持打开状态，直到第 3 步的活动完成.
5. 一旦 pull-db.js 脚本完成从远程服务器拉取所有数据，远程服务器上第 1 步中发出的 EXIT 命令终于可以关闭连接，在本地主机上解除绑定 27108.

您现在应该在本地主机中拥有远程数据库中的所有数据.

You should now have all of the data from your remote database in your localhost.

这篇关于通过 ssh 同步 MongoDB的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！