模拟来自Azure Service Fabric群集的10,000个Azure IoT中心设备连接 [英] Simulate 10,000 Azure IoT Hub Device connections from Azure Service Fabric cluster

问题描述

我们正在开发.Net Core服务，该服务将托管在Azure Service Fabric中.该SF服务需要通过其AMQP 1.0 SSL TLS终结点与在Azure IoT中心中注册的10,000个设备进行交互.每个IoT中心设备都具有由IoT中心服务提供的自己的安全令牌和连接字符串.

We are developing a .Net Core service that shall be hosted in Azure Service Fabric. This SF Service needs to interact with 10,000 devices registered in Azure IoT Hub via it's AMQP 1.0 SSL TLS endpoints. Each IoT Hub devices has it's own security tokens and connection string provided by the IoT Hub service.

对于我们的方案，我们需要侦听来自10,000个IoT中心设备实例的所有云到设备的消息，并将这些消息路由"到中央服务总线主题，该字段中的实际网关"将侦听这些消息.因此，基本上，我们希望将消息从10,000个服务总线队列转发到一个中央队列.

For our scenario we need to listen to all cloud-to-devices messages coming from the 10,000 IoT Hub device instances and "route" these to a central Service Bus topic to which the actual "gateways" in the field listen to. So basically we want to forward messages from 10,000 Service Bus Queues into one central Queue.

从SF服务处理这10,000个AMQP列表器的最佳方法是什么?有没有一种方法可以重用AMQP连接，会话或链接，以便我们缓存/共享资源?以及如何动态地将连接维护的负载分散到SF集群中的5个节点上?

What is the best approach to handle these 10,000 AMQP listners from a SF Service? Is there a way we can reuse AMQP connections, sessions or links so we cache/share resources? And how can we dynamically spread the load of connection maintenance over the 5 nodes in the SF cluster?

我们正在评估以下Nuget软件包的实现: Microsoft.Azure.ServiceBus AMQPNetLite Microsoft.Azure.Devices.Client

We are evaluating these Nuget packages for the implementation: Microsoft.Azure.ServiceBus AMQPNetLite Microsoft.Azure.Devices.Client

我们正在使用Microsoft.Azure.Devices.Client lib进行一些测试，请参见下面的简化代码示例:

We are doing some tests using the Microsoft.Azure.Devices.Client lib, see a simplified code sample below:

using System;
using System.Fabric;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.Devices.Client;
using Microsoft.ServiceFabric.Services.Runtime;

namespace ID.Monitoring.MonServer.ServiceFabric.ServiceBus
{
    /// <summary>
    /// An instance of this class is created for each service instance by the Service Fabric runtime.
    /// </summary>
    internal sealed class ServiceBus : StatelessService
    {
        private readonly DeviceClient _deviceClient;
        private ConnectionStatus _status;

        public ServiceBus(StatelessServiceContext context)
            : base(context)
        {
            _deviceClient = DeviceClient.CreateFromConnectionString("HostName=id-monitoring-dev.azure-devices.net;DeviceId=100;SharedAccessSignature=SharedAccessSignature sr=id-monitoring-dev.azure-devices.net%2Fdevices%2F100&sig={token}&se=1553265888", TransportType.Amqp_Tcp_Only);
        }

        /// <summary>
        /// This is the main entry point for your service instance.
        /// </summary>
        /// <param name="cancellationToken">Canceled when Service Fabric needs to shut down this service instance.</param>
        protected override async Task RunAsync(CancellationToken cancellationToken)
        {
            _deviceClient.SetConnectionStatusChangesHandler(ConnectionStatusChangeHandler);

            while (true)
            {
                if (_status != ConnectionStatus.Connected)
                {
                    await _deviceClient.OpenAsync();
                }
                var receivedMessage = await _deviceClient.ReceiveAsync(TimeSpan.FromSeconds(10)).ConfigureAwait(false);

                if (receivedMessage != null)
                {
                    var messageData = Encoding.ASCII.GetString(receivedMessage.GetBytes());
                    //TODO: handle incoming message and publish to common 
                    await _deviceClient.CompleteAsync(receivedMessage).ConfigureAwait(false);
                }
            }
        }

        private void ConnectionStatusChangeHandler(ConnectionStatus status, ConnectionStatusChangeReason reason)
        {
            _status = status;
        }
    }
}

问题:这是否可以很好地扩展到10,000个Service Fabric服务实例?还是有更有效的方法从Service Fabric服务环境中维护这么多AMQP服务总线列表器?有没有办法可以应用AMQP连接复用?

Question: Does this scale well to 10,000 Service Fabric service instances? Or are there more efficient ways to have this many AMQP Service Bus Listners maintained from a Service Fabric Service environment? Is there a way we can apply AMQP connection multiplexing maybe?

推荐答案

您选择监视设备的方法无法很好地扩展，并且难以维护.

The approach you choose to monitor your devices won't scale well and will be hard to maintain.

当前，服务结构具有限制 您可以放置​​在单个节点中.例如:如果使用ServiceBus服务创建一个应用程序并跨越10000个实例，则将达到此限制，即节点数.即:如果您有5个节点的群集，则使用默认扩展方式只能运行5个服务实例.

Currently, service fabric has a limitation of how many instances you can place in a single node. For example: if you create an application with your ServiceBus service and span 10000 instances, you will hit this limitation, that is the number of nodes. i.e: if you have a 5 node cluster, you will be able to run only 5 instances of your service by using the default scaling approach.

要绕过此问题，您可以选择以下方法:

To bypass this issue you have some options:

分区:

要使单个无状态服务运行更多 分区数超过节点数，则必须对服务进行分区. 假设您有一个5节点群集，并且需要10000个实例，您将 需要在每个节点上运行2000个分区.如果您使用共享过程并且对此有足够的了解，那么这种方法可能会对您有所帮助，请查看此线程和此线程在采用这种方法之前

To have a single stateless service running more partitions than the node count, you have to partition your service. Assuming you have a 5 node cluster and need 10000 instances, you will need 2000 partitions running on each node. If you use shared process and have enough ram to this, this approach might help you, please take a look at this thread and this thread before following this approach

多种服务:

命名服务是一种服务类型的运行服务定义，在这种情况下，您将为每个设备创建一个.像:

Named service is the running service definition for one service type, in this case you would create one per device. like:

• ServiceBusType
  • ServiceBus-Device1
  • ServiceBus-Device2
  • ServiceBus-Device3
  • ServiceBusType
    • ServiceBus-Device1
    • ServiceBus-Device2
    • ServiceBus-Device3

    这种方法将消耗您机器中过多的资源，因为您将为每个设备运行一个实例，但易于管理，因为您可以为每个新设备跨越新实例而不会影响其他正在运行的服务.

    This approach will consume too much resources in your machine, as you will be running one instance for each device, but easy to manage, as you can span new instances for each new device without affecting other running services.

    每个实例的并行处理:

    每个实例将负责同时处理多条消息，在这种情况下，您将为每个实例创建2000个连接(如果在每个集群的5个实例/节点中运行).这将比其他方法在资源消耗上更轻便，但是维护起来却有点困难，因为您将不得不自己处理平衡，并且可能需要额外的服务来监视任务并将其委派给所有服务并确保消息正在被执行.均匀地处理.

    Where each instance, would be responsible for processing multiple messages concurrently, in this case you would create 2000 connections for each instance(if running in a 5 instance/node per cluster). This will be lighter than the other approaches on resources consumption, but is a bit harder to maintain, as you will have to handle the balance yourself and might need an extra service to monitor and delegate tasks to all the services and ensure the messages are being processing evenly.

    摘要:

    一个实例一次处理一条消息的一个连接将需要10000个服务实例，分区将是相似的，但是您可以使用共享进程来减少内存消耗，但是在两种情况下，内存消耗仍然很高.

    One instance handling one connection at one message a time will required 10000 instances of your service, the partitioning will be similar but you can use a shared process to reduce memory consumption, but the memory consumption will still be high in both cases.

    如果服务的数量不太高，则可以选择多个命名服务，您也将无法共享连接.因此，我不会针对您的情况推荐这种方法.

    Multiple named services could be an option if the number of services were not too high, You also wouldn't be able to share the connection. So I won't recommend this approach for your scenario.

    第三个选项是对资源更友好，但是您将必须找到一种在整个群集节点上平均划分连接的方法.

    The third option, is the more resource friendly but you will have to find a way to partition the connections evenly throughout the cluster nodes.

    您还可以使用混合方法，例如，可以让服务并行处理多个消息，并让分区服务定义设备的关键范围.

    You can also use a mixed approach, for example, you can have service handling multiple messages in parallel and a partitioned service to define the key range of devices.

    请查看我提到的链接.

    这篇关于模拟来自Azure Service Fabric群集的10,000个Azure IoT中心设备连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！