C#:内存高效搜索 200 万个对象，无需外部依赖 [英] C#: Memory-efficient search through 2 million objects without external dependencies

问题描述

我需要能够在 C# 中搜索约 200 万个项目的集合.应该可以在多个字段上进行搜索.简单的字符串匹配就足够了.

I need to be able to search over a collection of approx 2 million items in C#. Search should be possible over multiple fields. Simple string-matching is good enough.

使用像数据库这样的外部依赖项不是一种选择，但使用内存数据库就可以了.

Using an external dependency like a database is not an option, but using an in-memory database would be OK.

主要目标是节省内存.

集合中的类型很简单，没有长字符串:

The type in the collection is quite simple and has no long strings:

public class Item
{
    public string Name { get; set; } // Around 50 chars
    public string Category { get; set; } // Around 20 chars
    public bool IsActive { get; set; }
    public DateTimeOffset CreatedAt { get; set; }
    public IReadOnlyList<string> Tags { get; set; } // 2-3 items
}

重点和要求

明确重点和要求:

Focus and requirements

Clarification of focus and requirements:

• 没有外部依赖项(如数据库)
• 节省内存(200 万个项目低于 2 GB)
• 集合中的可搜索项目(必须是高性能的)

在上述类型上使用简单的List，无论是作为class 还是struct，仍然需要大约2 GB 的内存.

Using a simple List<T> over above type, either as a class or a struct, still requires about 2 GB of memory.

有更好的方法吗?

推荐答案

类中最显着的内存占用是使用只读列表.去掉它，你会减少大约 60% 的内存占用(用三个标签测试):

The most significant memory hog in your class is the use of a read-only list. Get rid of it and you will reduce memory footprint by some 60% (tested with three tags):

public class Item
{
    public string Name { get; set; }
    public string Category { get; set; }
    public bool IsActive { get; set; }
    public DateTimeOffset CreatedAt { get; set; }
    public string Tags { get; set; } // Semi-colon separated
}

另外，考虑使用 DateTime 而不是 DateTimeOffset.这将进一步减少大约 10% 的内存占用.

Also, consider using DateTime instead of DateTimeOffset. That will further reduce memory footprint with around 10%.

这篇关于C#:内存高效搜索 200 万个对象，无需外部依赖的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！