如何恢复 Cassandra 快照? [英] How can I restore Cassandra snapshots?

问题描述

我正在为 Cassandra 数据库构建一个备份和恢复过程，以便在我需要的时候准备好，并且我了解细节以便构建适用于生产的东西.我在这里按照 Datastax 的说明进行操作:

I'm building a backup and restore process for a Cassandra database so that it's ready when I need it, and so that I understand the details in order to build something that will work for production. I'm following Datastax's instructions here:

http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_backup_restore_c.html.

首先，我将数据库植入开发箱，然后尝试进行备份/恢复工作.这是备份脚本:

As a start, I'm seeding the database on a dev box then attempting to make the backup/restore work. Here's the backup script:

#!/bin/bash

cd /opt/apache-cassandra-2.0.9
./bin/nodetool clearsnapshot -t after_seeding makeyourcase
./bin/nodetool snapshot -t after_seeding makeyourcase

cd /var/lib/
tar czf after_seeding.tgz cassandra/data/makeyourcase/*/snapshots/after_seeding

是的，也许 tar 不是最有效的方法，但我现在只是想让某些东西正常工作.我检查了 tar，所有文件都在那里.

Yes, tar is not the most efficient way, perhaps, but I'm just trying to get something working right now. I've checked the tar, and all the files are there.

备份数据库后，我关闭了 Cassandra 和我的应用程序，然后 rm -rf/var/lib/cassandra/ 以模拟完全丢失.

Once the database is backed up, I shut down Cassandra and my app, then rm -rf /var/lib/cassandra/ to simulate a complete loss.

现在恢复数据库.从 http://www.datastax.com 恢复方法 2"/documentation/cassandra/2.0/cassandra/operations/ops_backup_snapshot_restore_t.html 与我的模式创建组件比方法 1 更兼容.

Now to restore the database. Restoration "Method 2" from http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_backup_snapshot_restore_t.html is more compatible with my schema-creation component than Method 1.

因此，方法 2/步骤 1，重新创建架构":重新启动 Cassandra，然后是我的应用程序.该应用程序旨在在必要时在启动时重新创建架构.一旦它启动，就会有一个可用的 Cassandra 节点，其中包含应用程序的架构，但没有数据.

So, Method 2/Step 1, "Recreate the schema": Restart Cassandra, then my app. The app is built to re-recreate the schema on startup when necessary. Once it's up, there's a working Cassandra node with a schema for the app, but no data.

方法 2/第 2 步恢复快照":他们给出了三种选择，第一种是使用 sstableloader，记录在 http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsBulkloader_t.html.加载程序所需的文件夹结构与快照工具创建的文件夹结构完全不同，因此所有内容都必须移动到位.在解决所有这些麻烦之前，我会在一张桌子上试一试:

Method 2/Step 2 "Restore the snapshot": They give three alternatives, the first of which is to use sstableloader, documented at http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsBulkloader_t.html. The folder structure that the loader requires is nothing like the folder structure created by the snapshot tool, so everything has to be moved into place. Before going to all that trouble, I'll just try it out on one table:

>./bin/sstableloader makeyourcase/users
Error: Could not find or load main class org.apache.cassandra.tools.BulkLoader

嗯，好吧，那行不通.BulkLoader 在 ./lib/apache-cassandra-2.0.9.jar 中，但加载器似乎没有设置为开箱即用.与其调试工具，不如继续进行第二种选择，将快照目录复制到 makeyourcase/users/snapshots/目录中.这应该很容易，因为我们将快照目录扔回它的来源，所以 tar xzf after_seeding.tgz 应该可以解决问题:

Hmmm, well, that's not going to work. BulkLoader is in ./lib/apache-cassandra-2.0.9.jar, but the loader doesn't seem to be set up to work out of the box. Rather than debug the tool, let's move on to the second alternative, copying the snapshot directory into the makeyourcase/users/snapshots/ directory. This should be easy, since we're throwing the snapshot directory right back where it came from, so tar xzf after_seeding.tgz should do the trick:

cd /var/lib/
tar xzf after_seeding.tgz
chmod -R u+rwx cassandra/data/makeyourcase

这会将快照目录放回各自的快照"目录下，刷新应该会恢复数据:

and that puts the snapshot directories back under their respective 'snapshots' directories, and a refresh should restore the data:

cd /opt/apache-cassandra-2.0.9
./bin/nodetool refresh -- makeyourcase users

这毫无怨言地运行.请注意，您必须为每个表运行此程序，因此您必须首先生成表列表.但是，在我们这样做之前，请注意 Cassandra 日志中有一些有趣的东西:

This runs without complaint. Note that you have to run this for each and every table, so you have to generate the list of tables first. But, before we do that, note that there's something interesting in the Cassandra logs:

INFO 14:32:26,319 Loading new SSTables for makeyourcase/users...
INFO 14:32:26,326 No new SSTables were found for makeyourcase/users

因此，我们将快照放回原处，但 Cassandra 没有找到.我还尝试将快照目录移动到现有 SSTables 目录下，并将旧的 SSTable 文件复制到现有目录中，日志中出现相同的错误.Cassandra 不会记录它期望找到它们的位置，只是它无法找到它们.文档说将它们放入名为 data/keyspace/table_name-UUID 的目录中，但没有这样的目录.有一个名为 data/makeyourcase/users/snapshots/1408820504987-users/，但将快照目录或单个文件放在那里不起作用.

So, we put the snapshot back, but Cassandra didn't find it. I also tried moving the snapshot directory under the existing SSTables directory, and copying the old SSTable files into the existing directory, with the same error in the log. Cassandra doesn't log where it expects to find them, just that it can't find them. The docs say to put them into a directory named data/keyspace/table_name-UUID, but there is no such directory. There is one named data/makeyourcase/users/snapshots/1408820504987-users/, but putting the snapshot dir there, or the individual files, didn't work.

第三种选择，节点重启方法"看起来不适合多节点生产环境，所以我没有尝试.

The third alternative, the "Node restart method" doesn't look suitable for a multi-node production environment, so I didn't try that.

为了让下一个人完全明白这一点，这里是应用已接受答案的初步的、有效的备份和恢复脚本.

Just to make this perfectly explicit for the next person, here are the preliminary, working backup and restore scripts that apply the accepted answer.

myc_backup.sh:

myc_backup.sh:

#!/bin/bash

cd ~/bootstrap/apache-cassandra-2.0.9
./bin/nodetool clearsnapshot -t after_seeding makeyourcase
./bin/nodetool snapshot -t after_seeding makeyourcase

cd /var/lib/
tar czf after_seeding.tgz cassandra/data/makeyourcase/*/snapshots/after_seeding

myc_restore.sh:

myc_restore.sh:

#!/bin/bash

cd /var/lib/
tar xzf after_seeding.tgz
chmod -R u+rwx cassandra/data/makeyourcase

cd ~/bootstrap/apache-cassandra-2.0.9
TABLE_LIST=`./bin/nodetool cfstats makeyourcase | grep "Table: " | sed -e 's+^.*: ++'`
for TABLE in $TABLE_LIST; do
    echo "Restore table ${TABLE}"
    cd /var/lib/cassandra/data/makeyourcase/${TABLE}
    if [ -d "snapshots/after_seeding" ]; then
        cp snapshots/after_seeding/* .
        cd ~/bootstrap/apache-cassandra-2.0.9
        ./bin/nodetool refresh -- makeyourcase ${TABLE}
        cd /var/lib/cassandra/data/makeyourcase/${TABLE}
        rm -rf snapshots/after_seeding
        echo "    Table ${TABLE} restored."
    else
        echo "    >>> Nothing to restore."
    fi
done

推荐答案

添加了更多详细信息:

您可以使用以下方法为您的特定键空间运行快照:

You can run the snapshot for your particular keyspace using:

$ nodetool snapshot <mykeyspace> -t <SnapshotDirectoryName>

这将在数据中的快照目录中创建快照文件.

This will create the snapshot files inside the snapshots directory in data.

删除数据时，请确保不要删除快照文件夹，否则将无法恢复(除非将其移动到其他位置/机器.)>

$ pwd
/var/lib/cassandra/data/mykeyspace/mytable
$ ls
mykeyspace-mytable-jb-2-CompressionInfo.db mykeyspace-mytable-jb-2-Statistics.db
mykeyspace-mytable-jb-2-Data.db mykeyspace-mytable-jb-2-Filter.db mykeyspace-mytable-jb-2-Index.db
mykeyspace-mytable-jb-2-Summary.db mykeyspace-mytable-jb-2-TOC.txt snapshots


$ rm *
rm: cannot remove `snapshots': Is a directory

准备好恢复后，将快照数据复制回 keyspace/table 目录(每个表一个):

Once you are ready to restore, copy back the snapshot data into the keyspace/table directory (one for each table):

$ pwd
/var/lib/cassandra/data/mykeyspace/mytable
$ sudo cp snapshots/<SnapshotDirectoryName>/* .

你提到:

然后将快照目录放回各自的快照"目录下，刷新 > 应该恢复数据:

and that puts the snapshot directories back under their respective 'snapshots' directories, and a refresh >should restore the data:

我认为问题在于您正在将快照数据恢复到快照目录中.它应该在表目录中.其他一切似乎都正确，请告诉我.

I think the issue is that you are restoring the Snapshot data into the snapshot directory. It should go right in the table directory. Everything else seems right, let me know.

这篇关于如何恢复 Cassandra 快照?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！