通过Threadpool或Asyn编程进行数据库操作 [英] Database operation via Threadpool or Asyn programming

问题描述

我的数据库中有大量记录.(数据库值通过Web服务更新)

i have database with larger number of records.(database values are updated via a webservice)

每个记录/行的类型(id，xmlfilename，operation，parameters，operationId，status)即. 我们已经执行了xmlfile中"operation"所指定的操作 由"xmlfilename"指定，并带有由"parameters"指定的操作参数. 状态部分是最初"免费的，并在需要时进行了更新.

Each record/row is of type (id,xmlfilename,operation,parameters,operationId,status) ie. we have perform operation as specified by'operation' on a xmlfile specified by "xmlfilename" with parameters for operation specified by "parameters".. status part is "intially" free as is updated as when reqd.

我必须使用Threads执行这些(每行)操作.即每个"id"一个线程 条目数.只要在数据库中有"id"行提取并执行操作"

I have to do these (each row) operation using Threads..ie one thread per 'id' number of entries.as long as there are 'id' rows in db fetch and perform "operation"

如何最大程度地提高并行性和/或并发性.

how can i do this efficiently with maximum parallelism and/or concurrency.

有更好的方法吗?

最适合Threadpool或自定义线程或asyn编程的是什么?

What is best suited Threadpool or custom threads or asyn programming?

编辑已添加:这是我尝试过的伪代码

string operationId=null;

while(operationId = DBWorker.getNextFreeOperationId ())
//how do i check this?another query??untill there are operations with status "Free" 
//,keep selecting operationids 
//note db is updating asynchronously.    
{ 
  //retrieve all rows with operationid=operationId eg:800 . 1 thread/qid???? and
  // status="free" ...
//there are multiple operations with same operationIds 

  DataRowCollection dbRows=DBWorker.retrieveQueuedEntries(operationId); 
  MyWorkItem workItem = new DBWorker.MyWorkItem();
  workItem.DataRows = dbRows; 
  workItem.Event = new AutoResetEvent(false); 
 //MyWorkItem.DoWork will do the necessary "Operation" 
 ThreadPool.QueueUserWorkItem(new WaitCallback(workItem.DoWork),workItem); 
} 
--------------
 MyWorkItem.DoWork(obj x){ 
//for brevity 
for each DataRow row in this.DataRows
  {
    performOperation(row);//use row["operation"] ..
  }
---------------
bool performOperation(DataRow row)
{
  string operation = (string)row["operation"];//retrieve other similarly
  switch(operation)
  {
    case Operations.Add: //call Add operation ..
    ...
  }
}
------------

以上代码正在使用.net2.0 ..不能实现多线程.. 这种情况可能是数据库正在从一端异步更新，而上面的代码将同时作为Windows服务运行.

above code is using.net2.0.. doesnt achieve multithreading .. scenario may be the database is updating from one end asynchronously,while above code will be running as windows service at the same time.

thx
阿米特(Amit)

thx
Amit

推荐答案

如果您不使用.net 4.0-使用ThreadPool 我将创建一个函数来接收行(如果将行转换为DAL中的强类型对象，则事件会更好)，它必须处理并执行与之相关的操作-就像

If you're not using .net 4.0 - Use ThreadPool I would create a function that would receive the row (event better if you convert the rows to strongly type objects in DAL) it has to process, and do what it has to do with it - something like

using System;
using System.Threading;

namespace SmallConsoleAppForTests
{
class Program
{

    private static AutoResetEvent[] events;

    static void Main(string[] args)
    {

        int dataLength = 3;
        // creating array of AutoResetEvent for signalling that the processing is done
        AutoResetEvent[] events = new AutoResetEvent[dataLength];

        // Initializing the AutoResetEvent array to "not-set" values;
        for (int i = 0; i < dataLength; i++)
            events[i] = new AutoResetEvent(false);

        //Processing the data
        for (int i = 0; i < dataLength; i++)
        {
            var data = new MyWorkItem { Event = events[i], Data = new MyDataClass() };

            ThreadPool.QueueUserWorkItem(x =>
            {
                var workItem = (MyWorkItem)x;
                try
                {
                    // process the data

                }
                catch (Exception e)
                {
                    //exception handling
                }
                finally
                {
                    workItem.Event.Set();
                }
            }, data);
        }

        //Wait untill all the threads finish
        WaitHandle.WaitAll(events);
    }




}

public class MyWorkItem
{
    public AutoResetEvent Event { get; set; }
    public MyDataClass Data { get; set; }
}

// You can also use DataRow instead
public class MyDataClass
{
    //data
    //
}
}

如果您使用的是.net 4.0，请查看任务和并行扩展(PLINQ)"

If you are using .net 4.0 - look into Tasks and Parallel Extensions (PLINQ)

这篇关于通过Threadpool或Asyn编程进行数据库操作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！