在C#中将一百万个对象列表与另一个一百万个对象列表进行比较的最佳方法 [英] Best Way to compare 1 million List of object with another 1 million List of object in c#

问题描述

我正在将一百万个对象列表与另一个一百万个对象列表区分开.我正在for，foreach中使用，但是要花费大量时间来遍历这些列表.谁能帮助我做到这一点的最佳方法

  var SourceList = new List< object>();//一百万var TargetList = new List< object>());//一百万//从数据库中获取数据//具有一百万个列表的SourceList//具有一百万个列表的TargetListvar DifferentList = new List< object>();//ForEachSourceList.ToList().ForEach(m =>{如果(！TargetList.Any(s => s.Name == m.Name))DifferentList.Add(m);});//为了对于(int i = 0; i< SourceList .Count; i ++){if(！TargetList .Any(s => s == SourceList [i] .Name))DifferentList .Add(SourceList [i]);} 

解决方案

我认为这似乎是个坏主意，但IEnumerable的魔术会帮助您.

对于初学者，请简化您的表达.看起来像这样:

  var结果= sourceList.Where(s => targetList.Any(t => t.Equals(s))); 

我建议使用 Equals 方法进行比较:

 公共类CompareObject{公共字符串道具{放;}公共新布尔等于(对象o){如果(o.GetType()== typeof(CompareObject))返回this.prop ==(((CompareObject)o).prop;返回this.GetHashCode()== o.GetHashCode();}} 

接下来添加 AsParallel .这可以加快和减慢您的程序.您可以添加...

  var结果= sourceList.AsParallel(). 

如果您尝试一次列出所有列表，则CPU 100％已加载:

  var cnt = result.Count(); 

但是，如果将结果分成小部分，则可以忍受.

  result.Skip(10000).Take(10000).ToList(); 

完整代码:

 静态Random random = new Random();公共类CompareObject{公共字符串道具{私人套装；}公共CompareObject(){prop = random.Next(0，100000).ToString();}公共新布尔等于(对象o){如果(o.GetType()== typeof(CompareObject))返回this.prop ==(((CompareObject)o).prop;返回this.GetHashCode()== o.GetHashCode();}}无效Main(){var sourceList = new List< CompareObject>();var targetList = new List< CompareObject>();为(int i = 0; i <10000000; i ++){sourceList.Add(new CompareObject());targetList.Add(new CompareObject());}var stopWatch = new Stopwatch();stopWatch.Start();var result = sourceList.AsParallel().Where(s =>！targetList.Any(t => t.Equals(s)));var lr = result.Skip(10000).Take(10000).ToList();stopWatch.Stop();Console.WriteLine(stopWatch.Elapsed);} 

更新

我记得您可以使用 Hashtable .从 targetList 和 sourceList 中选择唯一值，然后填写不等于 targetList .

示例:

 静态Random random = new Random();公共类CompareObject{公共字符串道具{私人套装；}公共CompareObject(){prop = random.Next(0，1000000).ToString();}public new int GetHashCode(){返回prop.GetHashCode();}}无效Main(){var sourceList = new List< CompareObject>();var targetList = new List< CompareObject>();为(int i = 0; i <10000000; i ++){sourceList.Add(new CompareObject());targetList.Add(new CompareObject());}var stopWatch = new Stopwatch();stopWatch.Start();var sourceHashtable = new Hashtable();var targetHashtable = new Hashtable();foreach(targetList中的var元素){var hash = element.GetHashCode();如果(！targetHashtable.ContainsKey(hash))targetHashtable.Add(element.GetHashCode()，element);}var result = new List< CompareObject>();foreach(sourceList中的var元素){var hash = element.GetHashCode();如果(！sourceHashtable.ContainsKey(hash)){sourceHashtable.Add(hash，element);if(！targetHashtable.ContainsKey(hash)){result.Add(element);}}}stopWatch.Stop();Console.WriteLine(stopWatch.Elapsed);} 

i am differentiating 1 million list of object with another 1 million list of object. i am using for , foreach but it takes too much of time to iterate those list. can any one help me best way to do this

var SourceList = new List<object>(); //one million 
var TargetList = new List<object>()); // one million

//getting data from database here
//SourceList with List of one million 
//TargetList with List of one million

var DifferentList = new List<object>();

//ForEach
SourceList.ToList().ForEach(m =>
    {
      if (!TargetList.Any(s => s.Name == m.Name))
            DifferentList.Add(m);
  });

 //for
 for (int i = 0; i < SourceList .Count; i++)
   {
    if (!TargetList .Any(s => s == SourceList [i].Name))
            DifferentList .Add(SourceList [i]);
  }

解决方案

I think it seems like a bad idea but IEnumerable magic will help you.

For starters, simplify your expression. It looks like this:

var result = sourceList.Where(s => targetList.Any(t => t.Equals(s)));

I recommend making a comparison in the Equals method:

public class CompareObject
{
    public string prop { get; set; }

    public new bool Equals(object o)
    {
        if (o.GetType() == typeof(CompareObject))
            return this.prop == ((CompareObject)o).prop;    
        return this.GetHashCode() == o.GetHashCode();
    }
}    

Next add AsParallel. This can both speed up and slow down your program. In your case, you can add ...

var result = sourceList.AsParallel().Where(s => !targetList.Any(t => t.Equals(s)));

CPU 100% loaded if you try to list all at once like this:

var cnt = result.Count();

But it’s quite tolerable to work if you get the results in small portions.

result.Skip(10000).Take(10000).ToList();

Full code:

static Random random = new Random();
public class CompareObject
{
    public string prop { get; private set; }

    public CompareObject()
    {
        prop = random.Next(0, 100000).ToString();
    }

    public new bool Equals(object o)
    {
        if (o.GetType() == typeof(CompareObject))
            return this.prop == ((CompareObject)o).prop;    
        return this.GetHashCode() == o.GetHashCode();
    }
}

void Main()
{
    var sourceList = new List<CompareObject>();
    var targetList = new List<CompareObject>();
    for (int i = 0; i < 10000000; i++)
    {
        sourceList.Add(new CompareObject());
        targetList.Add(new CompareObject());
    }

    var stopWatch = new Stopwatch();

    stopWatch.Start();
    var result = sourceList.AsParallel().Where(s => !targetList.Any(t => t.Equals(s)));


    var lr = result.Skip(10000).Take(10000).ToList();
    stopWatch.Stop();

    Console.WriteLine(stopWatch.Elapsed);
}

Update

I remembered what you can use Hashtable.Choos unique values from targetList and from sourceList next fill out the result whose values are not targetList.

Example:

static Random random = new Random();
public class CompareObject
{
    public string prop { get; private set; }
    public CompareObject()
    {
        prop = random.Next(0, 1000000).ToString();
    }

    public new int GetHashCode() {
        return prop.GetHashCode();
    }
}


void Main()
{
    var sourceList = new List<CompareObject>();
    var targetList = new List<CompareObject>();
    for (int i = 0; i < 10000000; i++)
    {
        sourceList.Add(new CompareObject());
        targetList.Add(new CompareObject());
    }

    var stopWatch = new Stopwatch();

    stopWatch.Start();
    var sourceHashtable = new Hashtable();
    var targetHashtable = new Hashtable();

    foreach (var element in targetList)
    {
        var hash = element.GetHashCode();
        if (!targetHashtable.ContainsKey(hash))
            targetHashtable.Add(element.GetHashCode(), element);
    }

    var result = new List<CompareObject>();
    foreach (var element in sourceList)
    {
        var hash = element.GetHashCode();
        if (!sourceHashtable.ContainsKey(hash))
        {
            sourceHashtable.Add(hash, element);
            if(!targetHashtable.ContainsKey(hash)) {
                result.Add(element);
            }
        }
    }

    stopWatch.Stop();

    Console.WriteLine(stopWatch.Elapsed);
}

这篇关于在C#中将一百万个对象列表与另一个一百万个对象列表进行比较的最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！