使用linq断开连接的数据集 - 方法的优点和缺点 [英] Disconnected data sets with linq - pros and cons of approaches

问题描述

所以，请听我说，不要笑。

So hear me out and do not laugh.

我有一个不那么大的数据集，我想在开头的时候把整个内容读到内存中一个微服务进程的生命。

I have a dataset that is not that large and I want to read the whole thing in to memory at the beginning of a microservice process's life.

我将在内存中对它进行linq查询...

I am going to do linq queries on it once its in memory...

数据自然是关系的 - 它的基本上是国家 - >州 - >县 - > zipcodes

The data is naturally relational - its basically countries -> states -> counties -> zipcodes

我打算将数据保留在该结构中 - 使用国家/地区tabe和状态表以及县表和邮政编码表，然后将其读入然后执行查询。

I was going to leave the data in that structure - with a countries tabe and a state table and a county table and a zipcode table and then read it in and then do queries..

但是，我的老板想要将数据的形状（现在由于成本原因存储在dynamodb上）改为基于文档的结构 - 所以一个国家的文档里面会有很多状态那里面会有很多县等。 只有一个表在dynamodb中而不是四个，有
的成本效益。

However, my boss wants to change the shape of the data (it is now stored on dynamodb for cost reasons) to a document based structure - so one country document would have many states inside it and that would have many counties inside etc.  There are cost benefits to only having one table in dynamodb instead of four.

我感觉不对，但我向他提出的所有论点他告诉我错了 - 所以现在我想知道是否只是我对linq如何工作的无知。？

It feels wrong to me but all the arguments I am putting to him he is telling me are wrong - so now I am wondering whether it is just my ignorance about how linq works.?

我认为，使用"整个文档"为了找到所有存在的县，linq必须将所有国家加载到存储器中"再次"加载。然后只找到县。 但是他说事实并非如此 - 数据已经在内存中了，而且linq只会向我提供指向我想要的数据的指针。 他似乎知道他在说什么，所以我需要一些帮助来找出正确的事情。

I thought that, using the "whole document" structure for linq to find all the counties that exist, it would have to load all the countries into memory "again" and then find only the counties.  But he says that is not the case - that the data is already in memory and linq would just be providing me a pointer to the data I wanted in effect.  He seems to know what he is talking about so I need some help here to work out what is right to do.

我说这是因为我对Mongodb和Ravendb的经验是即使数据在内存中，不需要去数据库获取数据，linq不能很好地处理这样的文档结构 - 它非常非常慢 - 但可能有
我做错了什么？

I say this because my experience with Mongodb and Ravendb was that even when the data is in memory and no need to go to the database to get the data, linq did not work well with document structures like that - it was very very slow - but that might have been something I did wrong?

输入？ 请讨论？

Input?  Discussion please?

推荐答案

我应该补充一点 - 他认为如果我们想找到我们可以做的所有县类似这样的事情

I should add - he is thinking that if we want to find all counties we can do something like this

countiesenumerable =来自countries.states.counties的县选择县;

countiesenumerable = from county in countries.states.counties select county;

并使用 可以在任何地方枚举

and use  that enumerable everywhere

和zipcodes

zipcodes =来自countries.states.counties.zipcodes中的zipcode

zipcodes = from zipcode in countries.states.counties.zipcodes

                 选择邮政编码;

                 select zipcode;

我觉得每次访问县或邮政编码时都会涉及到每个国家和地区 - 并且会很慢 - 但他说它不应该 - 谁是对的？

I feel like that would involve going through every single country and state every time we access a county or a zipcode - and would be slow - but he says it should not - who is right?

这篇关于使用linq断开连接的数据集 - 方法的优点和缺点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！