如何在Web API中维护状态或请求队列 [英] How to maintain state or queue of requests in Web API

问题描述

我遇到的情况是，我必须以Web API方法接收请求，将这些请求放入队列，然后将批量发送到数据库(Solr实例).

I have situation, where I have to receive requests in a Web API method, queue those request and then send the bulk to a database (Solr instance).

我不太确定如何维护来自多个来源的一批请求.现在，我将每个请求数据以json格式写入磁盘上的文件中，稍后我将提供Windows服务，通过该文件夹读取所有文件，更新数据库并删除这些文件.

I am not really sure how do I maintain a batch of requests from multiple sources. For now I am writing each request data in json format to a file on disk, Later I will have a windows service, go through the folder read all files , update the database and delete those files.

这是我在Web API中正在做的事情

Here is what I am doing in my Web API

public void Post(LogEntry value)
{
    value.EventID = Guid.NewGuid();
    value.ServerTime = DateTime.UtcNow;
    string json = JsonConvert.SerializeObject(value);
    using(StreamWriter sw = new StreamWriter(value.EventID.ToString()))
    {
        sw.Write(json);
    }
}

(此处EventID为GUID)

(Here EventID is GUID)

此过程看起来不正确，必须有一种方法来维护请求队列，但是我不确定如何在多个请求期间维护队列.

This process doesn't look right, there must be a way to maintain a queue of request, but I am not really sure how to maintain a queue during multiple requests.

我这样做的原因是，在Solr实例中批量插入要比通过SolrNet插入单个记录快.我期望Web API每秒至少收到100个请求.我想创建一个1000个请求的批处理，并每10秒更新一次solr实例.请不要以为我需要代码，只需要知道我应该采用哪种策略来维护请求/状态队列.

The reason I am doing that is, insertion in batches in solr instance is faster than inserting a single record through SolrNet. I am expecting to get at least 100 requests each second on the Web API. I want to create a batch of 1000 request and update the solr instance every 10 seconds. Please don't think that I need code, just need to know what strategy should I adopt to maintain a queue of request / state.

推荐答案

如果使用的是.NET 4.0或更高版本，则可以使用并发队列:

You could use a concurrent queue, if you're using .NET 4.0 or higher:

并发队列(MSDN)

这是一种使用队列的线程安全方式，随后可以在所需的时间对其进行访问.

This is a thread-safe way of using a queue, which then could be accessed at a desired time.

示例:

这将是队列的包装:

public static class RequestQueue
{
    private static ConcurrentQueue<int> _queue;

    public static ConcurrentQueue<int> Queue
    {
        get
        {
            if (_queue == null)
            {
                _queue = new ConcurrentQueue<int>();
            }

            return _queue;
        }
    }

}

然后，您可以像这样设置Web api(为简洁起见，本示例存储整数):

Then you could set up your web api like this (this example stores integers for the sake of brevity):

public class ValuesController : ApiController
{        
    public string Get()
    {
        var sb = new StringBuilder();
        foreach (var item in RequestQueue.Queue)
        {
            sb.Append(item.ToString());
        }

        return sb.ToString();
    }

    public void Post(int id)
    {
        RequestQueue.Queue.Enqueue(id);
    }        
}

如果使用此示例，您将看到队列中包含多个请求中的值.但是，由于它存储在内存中，因此，例如，如果应用程序池被回收，那些排队的项目将消失.

If u use this example you'll see that the queue holds the values across multiple requests. But, since it lives in memory, those queued items will be gone if the app pool is recycled (for instance).

现在，您可以检查队列中是否包含10个项目，然后将其保存到数据库中，同时创建另一个队列来存储传入的值.

Now you could build in a check for when the queue holds 10 items and then save those to the DB, while creating another queue to store incoming values.

像这样:

public static class RequestQueue
{
    private static ConcurrentQueue<int> _queue;

    public static ConcurrentQueue<int> Queue
    {
        get
        {
            if (_queue == null)
            {
                _queue = new ConcurrentQueue<int>();
            }

            if (_queue.Count >= 10)
            {
                SaveToDB(_queue);
                _queue = new ConcurrentQueue<int>();
            }

            return _queue;
        }
    }

    public static void SaveToDB(ConcurrentQueue<int> queue)
    {
        foreach (var item in queue)
        {
            SaveItemToDB(item);
        }
    }
}

您需要稍微清理一下，但是此设置应该可以工作.此外，您可能需要一些将队列转储到数据库并创建新实例的锁定机制.我会编写一个具有多个线程的Console应用程序，这些线程可以访问此队列以对其进行测试.

You need to clean this up a bit, but this setup should work. Also, you might need some locking mechanism around the dumping of the queue to the DB and creating a new instance. I would write a Console app with multiple threads that access this Queue to test it.

这篇关于如何在Web API中维护状态或请求队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！