通过ssh同步MongoDB [英] Sync MongoDB Via ssh

问题描述

与Mysql不同，我发现尝试同步MongoDB文件非常具有挑战性-
它们无法通过管道回传，因为它们不会将数据发送到stdout
(如果我理解正确的话.)

Unlike Mysql, I found it quite challenging trying to sync MongoDB files -
They can not be piped back, since they don't send the data to stdout
(If I understand correctly).

因此，我正在尝试寻找另一种方式，该方式不涉及两个ssh调用.
需要做的是这样:

So, I'm trying to find another way, that doesn't involve two ssh calls.
What needs to be done is this:

• 登录ssh服务器
• 导出所有MongoDB文件
• 将它们压缩为gzip
• 将它们发送回本地计算机
• 提取并导入

但是，这里的关键是不留痕迹-
我不希望压缩文件保留在远程计算机中，
这通常需要我再次进行ssh登录.
因此，将文件移入归档文件"是理想的解决方案，
如果可以稍后将其无缝地通过管道传输回本地计算机.

The key thing here, though, is leaving no trace behind -
I don't want the compressed files stay in the remote machine,
which would usually require me for another ssh login.
So something along the lines of "move files into archive" is the ideal solution,
if that could be later piped back to the local machine seamlessly.

我意识到MongoDB有一种使用mongodump连接到服务器凭据的方法，但是端口是封闭的atm，因此我需要SSH方法. 顺便说一句，其他任何想法都将受到欢迎.

I realize MongoDB has a method to connect to a server credentials using mongodump, but the port is closed atm, so I need the SSH method. Any other ideas would be welcome, BTW.

由于这个问题似乎很受欢迎，因此，我希望分享一个脚本，该脚本是根据该问题的答案以及去年其他资源(应归功于信贷)而演变而来的.
该脚本基本上管理着与远程服务器之间的同步，对于每种类型的db都可能进行管理(暂时可能是postgres，mysql和mongo).
它确实有一些假设，例如root用户没有db的密码，但是可以根据需要进行更改.

Since this questions seems to be somewhat popular, I'd care to share a script that evolved from this questions' answers, and other resources during the last year (credit is where credit is due).
The script basically manages a sync from/to a remote server, for either type of db possible (probably. postgres, mysql and mongo for the time being).
It does have a few assumptions, like the root user having no password for the db, but that can be changed according to need.

可以在以下位置找到该脚本: https://github.com/iwfmp/zsh/blob/master/scripts/db/db-sync

The script can be found here: https://github.com/iwfmp/zsh/blob/master/scripts/db/db-sync

推荐答案

您可以使用SSH隧道来完成此操作，将远程MongoDB实例设置为在本地端口之一上运行.默认情况下，MongoDB在27017上运行，因此在下面的示例中，我选择将远程MongoDB实例映射到我的本地27018端口.

You can accomplish this with SSH Tunneling, setting up your remote MongoDB instance to run on one of your local ports. By default, MongoDB runs on 27017, so in the example below, I've chosen to map my remote MongoDB instance to my local 27018 port.

如果尝试将数据库从SERVER1复制到LOCALHOST，则可以在LOCALHOST上运行以下命令:

If on your trying to copy a database from SERVER1 to LOCALHOST, you could run this command on your LOCALHOST:

ssh -L27018:localhost:27017 SERVER1

(显然用您的实际服务器或ssh别名替换SERVER1)

(Obviously replace SERVER1 with your actual server or ssh alias)

这将打开与SERVER1的SSH连接，但还将LOCALHOST上的端口27018映射到SERVER1上的远程端口27017.不要关闭该SSH连接，现在尝试使用端口27018连接到本地主机上的MongoDB，如下所示:

This opens an SSH connection to SERVER1, but also maps the port 27018 on LOCALHOST to the remote port 27017 on SERVER1. Don't close that SSH connection, and now try to connect to MongoDB on your localhost machine with port 27018, like so:

mongo --port 27018

您将注意到，这现在是SERVER1上的数据，只是您正在从本地计算机访问它.

You'll notice this is now the data on SERVER1, except you're accessing it from your local machine.

只需正常运行MongoDB:

Just running MongoDB normally:

mongo(或mongo --port 27107)

将是您的本地计算机.

Will be your local machine.

现在，从技术上来说(在运行SSH隧道的LOCALHOST上):

Now, since you technically have (on your LOCALHOST, where you ran the SSH tunnel):

• MongoDB(LOCALHOST)在27017上
• 27018上的MongoDB(SERVER1)

您可以只使用MongoDB(LOCALHOST)中的db.copyDatabase()函数来复制数据.

You can just use the db.copyDatabase() function inside MongoDB (LOCALHOST) to copy over data.

在端口27017上的本地地址(实时执行将删除您的数据)

// Use the right DB
use DATABASENAME; 
// Drop the Existing Data on LOCALHOST
db.dropDatabase();
// Copies the entire database from 27018
db.copyDatabase("DATABASENAME", "DATABASENAME", "localhost:27018");

您应该能够将所有内容包装到一个shell脚本中，该脚本可以为您执行所有这些命令.我自己一个人，但是实际上有一些额外的步骤可能会使它变得更加混乱:)

You should be able to wrap this all up into a shell script that can execute all of these commands for you. I have one myself, but it actually has a few extra steps that would probably make it a bit more confusing :)

执行此操作并使用MongoDB的本机db.copyDatabase()函数将使您不必转储/压缩/还原.当然，如果您仍然想走那条路，运行mongodump，导出数据，tar/gzip，然后使用scp TARGETSERVER:/path/to/file /local/path/to/file将其拉下并在其上运行mongorestore并不是很困难.它.

Doing this, and using MongoDB's native db.copyDatabase() function will prevent you from having to dump/zip/restore. Of course, if you still want to go that route, it wouldn't be too hard to run mongodump, export the data, tar/gzip it, then use scp TARGETSERVER:/path/to/file /local/path/to/file to pull it down and run a mongorestore on it.

似乎还需要更多工作！

编辑-这是一个SH和JS文件，它们一起构成了可以运行此脚本的shell脚本. 在您的LOCALHOST上运行它们 ，不要实时运行它们，否则它将实时运行db.dropDatabase.将这两个文件放在同一文件夹中，并用domain/ip/ssh别名替换pull-db.sh中的 YOURSERVERNAME ，然后在pull-db.js中将DBNAMEHERE更改为您的数据库名称.

Edit - Here's a SH and JS file that go together to make a shell script you can run this with. Run these on your LOCALHOST, don't run them on live or it'll do the db.dropDatabase on live. Put these two files in the same folder, and replace YOURSERVERNAME in pull-db.sh with the domain/ip/ssh alias, and then in pull-db.js change DBNAMEHERE to whatever your database name is.

我通常在项目中创建一个名为scripts的文件夹，使用Textmate，只需打开pull-db.sh进行编辑即可点击⌘+R来执行它.

I normally create a folder called scripts in my projects, and using Textmate, I just have to hit ⌘+R while having pull-db.sh open to edit in order to execute it.

pull-db.sh

ssh -L27018:localhost:27017 YOURSERVERNAME '
    echo "Connected on Remote End, sleeping for 10"; 
    sleep 10; 
    exit' &
echo "Waiting 5 sec on local";
sleep 5;
echo "Connecting to Mongo and piping in script";
cat pull-db.js | mongo

pull-db.js

use DBNAMEHERE;
db.dropDatabase();
use DBNAMEHERE;
db.copyDatabase("DBNAMEHERE","DBNAMEHERE","localhost:27018");

我在shell脚本中添加了一些额外的代码，以回显它在做什么(排序).该脚本中的sleeping定时器只是为了给SSH连接时间提供连接，以便在下一行运行之前.基本上，会发生以下情况:

I added some extra code to the shell script to echo out what it's doing (sorta). The sleep timers in the script are just to give the SSH connections time to get connected before the next line is run. Basically, here's what happens:

1. 代码的第一行在您的计算机上创建隧道，并将ECHO，SLEEP和EXIT发送到远程SSH会话.
2. 然后等待5秒钟，这允许步骤1中的SSH会话进行连接.
3. 然后我们将pull-db.js文件传输到本地mongo shell中. (步骤1应该在5秒钟之内完成...)
4. pull-db.js应该现在在mongo中运行，并且步骤#1中的SSH终端在连接打开后可能已经运行了10秒钟，并且EXIT发送到了它的会话中.发出命令，但是，SSH会话实际上将保持打开状态，直到完成步骤3的活动为止.
5. 一旦您的pull-db.js脚本完成了从远程服务器提取所有数据的操作，就最终允许在远程服务器上的步骤#1中发出的EXIT命令关闭连接，从而解除本地主机上的27108绑定.

您现在应该在本地主机中拥有来自远程数据库的所有数据.

You should now have all of the data from your remote database in your localhost.

这篇关于通过ssh同步MongoDB的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！