Lustre，Gluster或MogileFS?用于视频存储，编码和流传输 [英] Lustre, Gluster or MogileFS?? for video storage, encoding and streaming

问题描述

这么多的选择，而很少的时间来测试所有这些……我想知道是否有人在使用分布式文件系统进行视频流和存储/编码方面有经验.

So many options and so little time to test them all... I wonder if someone has experiences with distributed file systems for video streaming and storage/encoding.

我有很多巨大的视频文件(50GB至250GB)，我需要将它们存储在某个地方，能够将它们编码为mp4，并从多个Adobe FMS服务器中流式传输.处理所有这些问题的唯一方法是使用分布式文件系统，但现在的问题是哪个文件系统?

I have a lot of huge video files (50GB to 250GB) that I need to store somewhere, be able to encode them to mp4 and stream them from several Adobe FMS servers. The only way to handle all this is with a distributed file system but now the question is which one??

到目前为止，我的研究告诉我:

My research so far tells me:

• Lustre :成熟的成熟解决方案已被许多大公司使用，最好是> 10G文件是内核驱动程序.
• Gluster :基于FUSE的新功能，较不成熟，这意味着易于安装，但由于FUSE开销可能较慢.最好处理大量〜1GB的较小文件
• MogileFS :似乎仅适用于小文件〜MB，使用HTTP进行访问?将来可能会进行FUSE绑定.

• Lustre: mature proven solution, used by a lot of big companies, best with >10G files is a kernel driver.
• Gluster: new, less mature, FUSE based that means easy to install but maybe slower due to FUSE overhead. Better to handle a large number of smaller files ~1GB
• MogileFS: seems to be only for small files ~MB, uses HTTP for access?? possible FUSE binding in the future.

到目前为止，Lustr似乎是赢家，但我想听听我拥有的特定应用程序的真实经验.

So far Lustre seems the winner but I would like to hear real experiences for the particular application I have.

还可以选择Hadoop，Redhat GFS，Coda和Windows DFS作为声音选项，因此欢迎任何体验.如果有人拥有基准，请分享.

Also Hadoop, Redhat GFS, Coda and Windows DFS sound as options so any experiences are welcome. If someone has benchmarks please share.

经过一些实际经验，这是我所学到的:

After some real experience this is what I have learned:

• 光泽:
  • 性能:速度惊人！我可以断言Lustre可以提供许多流 而且通过Lustre访问文件不会影响编码速度.
  • POXIS兼容性:很好！.无需修改应用程序即可使用光泽.
  • 复制，负载平衡和故障转移:非常糟糕！.对于复制负载 平衡我们并进行故障转移，我们需要依靠其他软件，例如虚拟IP 和DRDB.
  • 安装:最糟糕的！.不可能由凡人安装.需要一个非常 内核，光泽补丁和调整的特定组合，以使其正常运行.和 当前的光泽补丁通常可与不兼容的旧内核一起使用 新的硬件/软件.
  • Luster:
    • Performance: Amazingly fast! I can assert that Lustre can serve a lot of streams and that encoding speed is not affected by accessing files via Lustre.
    • POXIS compatibility: Very good!. No need to modify applications to use luster.
    • Replication, Load Balancing and Fail Over: Very bad!. For replication load balancing we and fail over we need to rely on other software such as virtual IPs and DRDB.
    • Installation: The worst!. Impossible to install by mere mortals. Requires a very specific combination of kernel, lustre patches and tweaks to get it working. And current luster patches usually work with old kernels that are incompatible with new hardware/software.
    • 性能:适用于小型文件，但不适用于中大型文件.这是 主要是由于HTTP开销，因为所有文件都是通过HTTP请求发送/接收的， 在base64中对所有数据进行编码，从而为每个文件增加了33％的开销.
    • POXIX兼容性不存在.所有应用程序都需要修改才能使用 mogilefs使其无法用于流/编码，因为大多数流服务器 和编码工具不了解MogileFS协议.
    • 开箱即用的复制和故障转移以及负载平衡可以在 通过一次访问多个跟踪器来访问应用程序.
    • 安装相对容易，并且大多数发行版中都存在可立即使用的软件包. 我发现的唯一困难是设置数据库主从来消除 单点故障.
      • Gluster:
      • Performance: Good for small files but not usable for medium to large files. This is mostly due to HTTP overhead since all files are send/receive via HTTP requests that encode all data in base64 adding a 33% overhead to each file.
      • POXIX compatibility is non existent. All applications require to be modified to use mogilefs that renders it useless for streaming/encoding since most streaming servers and encoding tools do not understand MogileFS protocol.
      • Replication and failover out of the box and load balancing can be implemented in the application by accessing more than one tracker at a time.
      • Installation is relatively easy and ready to use packages exist in most distributions. The only difficulty I found was setting the database master-slave to eliminate the single point of failure.
        • Gluster:

        最终结论:

        不幸的是，结论是没有单一的灵丹妙药".

        Unfortunately the conclusion is "No single silver bullet".

        当前，我们在复制的卷中将Gluster3.2中的媒体文件存储和转码.只要您没有很多服务器，就可以避免地理复制和条带卷工作正常.

        Currently we have our media files in Gluster3.2 in a replicated volume for storage and transcoding. As long as you don't have a lot of servers, avoid geo-replication and stripe volumes things work ok.

        当我们要流式传输媒体文件时，我们将它们复制到一个光泽卷，该光泽卷通过DR:DB复制到另一个光泽卷.然后wowza服务器从光泽卷中读取媒体文件.

        When we are going to stream the media files we copy them to a lustre volume that is replicated to a second lustre volume via DR:DB. The wowza server then read the media files from the lustre volumes.

        最后，我们使用MogileFS在Web应用程序服务器中提供缩略图.

        And finally we use MogileFS to serve the thumbnails in our web application servers.

        推荐答案

        到目前为止，GlusterFS进行了很多改进.他们现在为大型文件提供粒度锁定".参见此处: http://www.gluster.org/community/documentation/index.php/WhatsNew3.3 同样，这也是非常依赖的视频帧速率，您也应该为之工作. 如果您不会达到4K的速率，Gluster可以解决存储问题. 如果对速度有巨大的需求，那么Infiniband可以发挥作用.

        GlusterFS improved themselves a lot up to this date. They are now providing "granular locking" for large files. See here: http://www.gluster.org/community/documentation/index.php/WhatsNew3.3 Also it is quite dependent video frame rates you should work for too. If you will not go up to 4K rates, Gluster can solve the storage problems. If there is a huge demand for speed, therefore Infiniband can come in to play.

        这篇关于Lustre，Gluster或MogileFS?用于视频存储，编码和流传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！