S3上的Mysql数据目录 [英] Mysql Data Directory on S3
问题描述
我希望Mysql将其数据存储在Amazone S3上，因此我将S3存储桶安装到服务器上，并将数据目录的路径更改为my.cnf中的安装目录.
这样做之后，我重新启动了服务器并创建了数据库，它没有造成任何问题，但是当我尝试创建表(例如测试)时，它给了我以下错误.
错误1033(HY000):文件中的错误信息:"./test/t.frm"
任何人都可以告诉我，我实际上想做什么?
如果是，我要去哪里了?
如果没有，为什么?
没有可行的解决方案在S3上存储MySQL数据库.没有.
在合适的有限应用程序中使用s3fs并没有错，但是在这里不合适.
S3不是文件系统.它是一个对象库.要在S3中修改多千兆字节文件"的单个字节，需要将整个文件复制到自身之上.
现在...诸如s3nbd和s3backer之类的工具采用了不同的方法来使用S3进行存储.它们使用S3模拟可以在其上创建文件系统的块设备，并且它们比s3fs更接近成为S3和MySQL所需之间的适当桥梁，但是这种方法仍然无法实现可靠地使用，出于一个原因.
一致性.
当MySQL将数据写入文件时，它需要绝对保证，如果读取相同的数据，它将取回其写入的内容. S3不能保证这一点.
问:Amazon S3采用什么数据一致性模型?
所有区域中的Amazon S3存储桶为新对象的PUTS提供写后读取一致性，并为覆盖PUTS和DELETES提供最终一致性.
当S3中的对象被修改"(通过覆盖PUT完成)时，不能保证对该文件的读取在写入发生后的短时间内不会返回以前的版本./p>
简而言之，您正在追求一个本质上不可能实现的目标，即试图将S3用于并非旨在实现的功能.
但是，MySQL中存在一种可以节省存储成本的内置机制: myisampack .
某些EC2实例包括临时磁盘/实例存储 ，它们是零成本但易失的硬盘，永远不应该用于关键数据，但是，如果所考虑的数据库是辅助数据库，那么在发生这种情况时可以很容易地从该数据库中重建数据库，这可能是一个不错的选择数据丢失.它们对于一次性"数据库(例如QA数据库或日志分析)非常有用，在该数据库中，数据库不是数据的权威存储.
I want Mysql to store it's data on Amazone S3, So I mounted an S3 bucket to my server and changed the path of data dir to mounted directory in my.cnf.
After doing this, I restarted the server and created the database and it caused no problem but when I try to create a table (say test), it gives me the following error.
ERROR 1033 (HY000): Incorrect information in file: './test/t.frm'
Can any one please tell me, what I am trying to oo is actually possible?
If yes, where am I going wrong?
If no, Why?
There is no viable solution for storing MySQL databases on S3. None.
There's nothing wrong with using s3fs in limited applications where it is appropriate, but it's not appropriate here.
S3 is not a filesystem. It is an object store. To modify a single byte of a multi-gigabyte "file" in S3 requires that the entire file be copied over itself.
Now... there are tools like s3nbd and s3backer that take a different approach to using S3 for storage. These use S3 to emulate a block device over which you can create a filesystem, and these would come closer than s3fs to being an appropriate bridge between what S3 is and what MySQL would need, but still this approach cannot reliably be used either, for one reason.
Consistency.
When MySQL writes data to a file, it needs absolute assurance that if it reads that same data, that it will get back what it wrote. S3 does not guarantee this.
Q: What data consistency model does Amazon S3 employ?
Amazon S3 buckets in all Regions provide read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES.
When an object in S3 is "modified" (that's done with an overwrite PUT), there is no guarantee that a read of that file won't return a previous version for a short time after the write occurred.
In short, you are pursuing an essentially impossible objective trying to use S3 for something it isn't designed to do.
There, is, however, a built-in mechanism in MySQL that can save on storage costs: InnoDB natively supports on-the-fly table compression.
Or if you have large, read-only MyISAM tables, those can also be compressed with myisampack.
Some EC2 instances include the ephemeral disks/instance store, which are zero-cost, but volatile, hard drives that should never be used for critical data, but that might be a good option to consider if the database in question is a secondary database that can easily be rebuilt from authoritative sources in the event of data loss. They can be quite nice for "disposable" databases, like QA databases or log analytics, where the database is not the authoritative store for the data.
这篇关于S3上的Mysql数据目录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
