内存不足列表中加入许多记录 [英] out of memory list join many record

问题描述

我有200k(  listaCorpoContabilitaFiltrato )内存中的记录列表，我需要加入 得到一些数据.但这是解决该问题的其他解决方案

Hi, i have a 200k( listaCorpoContabilitaFiltrato) record list in memory, and i need to join to get some data. But it goes out of memory is some other solution to fix the problem

                        var query = new ObservableCollection<CorpoContabilita>(
                        from bilancio in listaCorpoContabilitaFiltrato
                        join dittaAss in newSednaContext.DittaAssociata on bilancio.CodiceDittaAssociata equals dittaAss.Codice into leftJoinDittaAss
                        from dittaAssLeft in leftJoinDittaAss.DefaultIfEmpty()
                        join conto in newSednaContext.Conti on bilancio.CodiceContoContabile equals conto.Codice
                        join mastro in newSednaContext.Mastro on conto.CodiceMastro equals mastro.Codice
                        join gruppo in newSednaContext.Gruppo on mastro.CodiceGruppo equals gruppo.Codice
                        select new CorpoContabilita
                        {
                            CodiceDittaAssociata = bilancio.CodiceDittaAssociata,
                            GruppoDittaAssociata = dittaAssLeft?.CodiceGruppoDittaAssociata,
                            CodiceContoContabile = bilancio.CodiceContoContabile,
                            DescrizioneContoContabile = bilancio.DescrizioneContoContabile,
                            DescrizioneContoContabileOriginale = bilancio.DescrizioneContoContabileOriginale,
                            TipoFornitore2 = bilancio.TipoFornitore2,
                            TipoConto = bilancio.TipoConto,
                            ImportoDare = bilancio.ImportoDare,
                            ImportoAvere = bilancio.ImportoAvere,
                            EconomicoPatrimoniale = conto.EconomicoPatrimoniale,
                            CodiceMastro = mastro.Codice,
                            DescrizioneMastro = mastro.Descrizione,
                            CodiceGruppo = mastro.CodiceGruppo,
                            DescrizioneGruppo = mastro.DescrizioneGruppo,
                            EconomicoPatrimonialeGruppo = gruppo.EconomicoPatrimoniale,
                            SezioneBilancioGruppo = gruppo.SezioneBilancio,
                            ExtraContabile = bilancio.ExtraContabile
                        });

推荐答案

200K的数据要大量存储在内存中，特别是当您进行大量连接时.您可以将其移至可以更高效地完成操作的数据库中吗?如果不是，那么您可能需要对调用进行分区，以便仅在 时间.您拥有的联接数将导致使用大量的临时数据.您的对象本身似乎有17个成员(字符串?)，这意味着您的原始结构可能约为30 + MB.

200K of data is a lot to store in memory especially when you're doing so many joins. Can you move it to a DB where it can be done more efficiently? If not then you probably need to partition the calls so you're only working with a subset of the data at a time. The number of joins you have is going to result in a lot of temp data being used. Your object itself has what appears to be 17 members (strings?) which means your raw structure is probably 30+MB as it is. 

另一种方法是对数据重新排序，因此您不需要那么多的联接.大量的连接通常意味着数据被过度规范化.如果您需要进行大量联接，则对数据进行非规范化或考虑使用报告数据库是一个不错的选择.

An alternative is to reorder your data so you don't need that many joins. Lots of joins generally means the data is over normalized. If you need to do joins a lot then denormalizing the data or considering a reporting database is a good option.

迈克尔·泰勒
http://www.michaeltaylorp3.net

Michael Taylor
http://www.michaeltaylorp3.net

这篇关于内存不足列表中加入许多记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！