这种技术可以扩展规模吗? [英] Can this technology stack scale?

问题描述

我的客户要求我构建一个实时应用程序，该应用程序可以实时聊天，发送图像和视频.他要求我提出自己的技术栈，所以我做了很多研究，发现最容易构建的将是使用低于技术栈的

My client ask me to build a realtime application that could chat, send images and videos all in realtime. He asked me to come up with my own technology stack, so I did a lot of research and found out that the easiest one to build would be using below tech stack

1)Node.js和群集可以最大化一个服务器实例-语言的CPU核心数量

1) Node.js and cluster to max out the CPU core for one instance of server - Language

2)Socket.io-实时框架

2) Socket.io - realtime framework

3)Redis-服务器的多个实例的发布/订阅

3) Redis - pub/sub for multiple instances of server

4)Nginx-反向代理并负载均衡多台服务器

4) Nginx - to reverse proxy and load balance multiple servers

5)Amazon EC2-运行服务器

5) Amazon EC2 - to run the server

6)Amazon S3和CloudFront-保存图像/视频并交付

6) Amazon S3 and CloudFront - to save the images/videos and to deliver

如果我对上述堆栈有误，请更正我.我的真正问题是，上述技术堆栈每秒可以扩展1,000,000条消息(文本，图像，视频)吗?

Correct me if I'm wrong for the above stack. My real question is, can the above tech stack scale 1,000,000 messages per seconds (text, images, videos)?

任何使用过node.js和socket.io的人，都可以为我提供上述堆栈的真知灼见或替代方法.

Anyone who have experienced with node.js and socket.io, could give me an insights or an alternatives of the above stack.

此致

SinusGob

推荐答案

我真正的问题是，上述技术堆栈能否扩展1,000,000条消息每秒(文本，图像，视频)?

My real question is, can the above tech stack scale 1,000,000 messages per seconds (text, images, videos)?

可以.具有正确的设计和足够的硬件.您的客户应该问的问题实际上不是是否可以将其做得那么大，而是可以以什么成本和实用性来完成，并且是最佳选择.

Sure it can. With the right design and enough hardware. The question your client should be asking is really not whether it can be made to go that big, but at what cost and practicality can it be done and are those the best choices.

让我们看看您提到的每件作品:

Let's look at each piece you've mentioned:

node.js -对于以I/O为中心的应用程序，它是进行大规模扩展的绝佳选择，并且可以通过在集群中部署多个CPU(每个服务器多进程和服务器).这种规模的实用性在很大程度上取决于所有这些服务器进程都需要访问哪种共享数据.通常，数据存储最终最终会成为扩展规模上更难的瓶颈，因为很容易在请求处理时抛出更多服务器.在集中式数据存储中添加更多硬件并非易事.有很多方法可以做到这一点，但这在很大程度上取决于应用程序对您的操作方式和难度的要求.

node.js - For an I/O centric app, it's an excellent choice for high scale and it can scale by deploying many CPUs in a cluster (both multi-process per server and multi-server). How practical this type of scale is depends a lot on what kind of shared data all these server processes need access to. Usually, the data store ultimately ends up being the harder bottleneck in scaling because it's easy to throw more servers at the request processing. It's not so easy to throw more hardware at a centralized data store. There are ways to do that, but it depends a lot on the demands of the app for how you do it and how hard it is.

socket.io -如果您需要高效的服务器推送小消息，则socket.io可能是最好的处理方式，因为它是推送到客户端的最有效方式.但是，这并不适合所有类型的运输.例如，我不会通过socket.io移动大型图像或视频，因为有更多专门构建的方法可以执行此操作.因此，socket.io的使用在很大程度上取决于应用程序到底要使用它的用途.如果您想将视频推送到客户端，也可以只推送一个URL，然后让客户端转过身来，并使用众所周知的大规模技术通过常规的HTTP URL请求视频.

socket.io - If you need efficient server push of smallish messages, then socket.io is probably the best way to go because it's the most efficient at push to the client. It is not great at all types of transport though. For example, I wouldn't be moving large images or video around through socket.io as there are more purpose built ways to do that. So, the use of socket.io depends a lot on what exactly the app wants to use it for. If you wanted to push a video to a client, you could also push just an URL and have the client turn around and request the video via a regular http URL using well known high scale technology.

Redis -同样，某些方面很棒，但并非所有方面都很棒.因此，这实际上取决于您要执行的操作.我之前解释的是，您的数据存储的设计以及通过它进行的事务数量可能是您真正的规模问题所在.如果我开始这项工作，那么我将首先了解服务器的数据存储需求，各种类型的每秒事务，缓存策略，冗余，故障转移，数据持久性等，然后设计高首先扩展对数据的访问.我不能完全确定redis是首选.我可能建议您在项目初期需要一位高级数据库专家作为顾问.

Redis - Again, great for some things, not great at everything. So, it really depends upon what you're trying to do. What I explained earlier is that the design of your data store and the number of transactions through it is probably where your real scale problems lie. If I were starting this job, I'd start with an understanding of the data storage needs for a server, transactions per second of various types, caching strategy, redundancy, fail-over, data persistence, etc... and design the high scale access to data first. I wouldn't be entirely sure redis was the preferred choice. I'd probably suggest you need a high scale database guy as a consultant early in the project.

Nginx -许多使用nginx的大型站点，因此它无疑是一个很好的工具.对于您而言，它究竟是不是正确的工具取决于您的设计.我可能会在最后一部分进行工作，因为它似乎在设计中不太重要，一旦系统的其余部分布置好，您就可以在这里考虑您需要什么了.

Nginx - Lots of high scale sites using nginx so it's certainly a good tool. Whether it's exactly the right tool for you depends upon your design. I'd probably work on this part last because it seems less central to the design and once the rest of the system is laid out, you can then consider what you need here.

Amazon EC2 -几种可能的选择之一.这些选择很难直接在苹果与苹果的比较中进行比较.大型系统是使用EC2构建的，因此在那里有概念证明，并且通用体系结构似乎很合适.如果您想知道真正的gremlins在哪里，则需要一名在EC2上进行过大规模研究的顾问.

Amazon EC2 - One of several possible choices. These choices are hard to compare directly in an apples to apples comparison. Large scale systems have been built out of EC2 so there is proof of concept there and the general architecture seems an appropriate match. If you wanted to know where the real gremlins are there, you'd need a consultant that had done high scale stuff on EC2.

Amazon S3 -我个人知道一些使用S3来存储视频和图像的非常高存储和带宽站点.它适用于此.

Amazon S3 - I personally know some very high storage and bandwidth sites using S3 for both video and images. It works for that.

所以...如果正确使用这些工具，通常可能是不错的工具.根据实际应用程序的存储需求，Redis将是一个问号(您提供的需求为零，而不能选择零需求的数据库).一个更合理的答案是基于将一系列高级要求组合在一起的，这些要求分析了系统必须能够做什么才能为1,000,000个服务.可以将这些需求与某些组件的已知功能进行比较，以开始扩展系统规模.然后，您必须将一些基准测试组合在一起，才能在系统的某些部分上运行某些测试.失败的成功很大程度上取决于应用程序的构建方式和工具的使用方式，以及选择工具的方式.您可以使用许多不同类型的工具成功进行扩展.哎呀，Facebook在PHP上运行(嗯，高度修改，自定义的PHP在运行时根本不是典型的PHP).

So ... these are generally likely good tools to use if they are used in the right way. Redis would be a question-mark depending upon the storage needs of the actual application (you've provided zero requirements and a database can't be selected with zero requirements). A more reasoned answer would be based on putting together a high level set of requirements that analyze what the system needs to be able to do to serve 1,000,000 whatever. Those requirements could be compared with known capabilities for some of these pieces to start a ballpark on scaling a system. Then, you'd have to put together some benchmarking tests to run some tests on certain pieces of the system. As much of the success of failure would depend upon how the app was built and how the tools were used as it would which tools were selected. You can likely make a successful scale with many different types of tools. Heck, Facebook runs on PHP (well, a highly modified, customized PHP that is not really typical PHP at all at runtime).

这篇关于这种技术可以扩展规模吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！