比DataTable中的其他最佳结构来存储〜半一亿前端记录 [英] Best structure other than Datatable to store ~half-a-million records in Frontend

问题描述

我有一个数据表是从数据库（SQLite的）获取〜5,50,000记录。获取当它被拖慢系统。

I have a Datatable which is fetching ~5,50,000 records from database(SQLite). When fetching it is slowing down the system.

我存储SQLite数据库和数据表中的前端后端这些记录。
现在我应该怎么在后台做，这样创建数据库时（〜10.5小时），并在前端获取时间减少了。

I am storing these records on backend in SQLite Database and in frontend in Datatable. Now what should i do so that database creation time(~10.5 hours) in backend and fetching time in Front end reduces.

是否有可用于这样做的任何其它结构。我已阅读，词典和放大器;二进制文件快。它们可用于此目的和放大器;怎么样？

Is there any other structure that can be used to do so. I have read that Dictionary & Binary File is fast. Can they be used for this purpose & how?

（它不是一个Web应用程序。它是一个WPF的桌面应用程序，其中前端和后端是同一台机器上）。

(Its not a web app. its a WPF desktop app where frontend and backend are on same machine).

推荐答案

您基本的问题，的我相信的，是不是要保持strucutre但的办法的您管理您的数据流。相反，从数据库中的所有数据取到一些晶格结构的（数据表你的情况），使一些存储过程作你需要的服务器端的计算，并返回到你已经计算的数据。通过这种方式，你可以获得很多好处，如：

Your basic problem, I believe, is not the strucutre that you want to maintain but the way you manage your data-flow. Instead of fetching all data from database into some strcuture (DataTable in your case), make some stored procedure to make a calculations you need on server-side and return to you data already calculated. In this way you gain several benefits, like:

• 持有数据库服务器服务器的一般的那么快开发或客户机


• 数据trasmission的大幅缩减，为您返回的只会导致计算的

• servers holding database servers are usually faster then development or client machine
• huge reduction of data trasmission, as you return only result of the calculation

修改

考虑后期编辑，我要说的是数据表已经高度存储器存取的优化上，我不认为它更改为别的东西会给你带来一个显着好处。我觉得可以的带来banefit是程序流程的修订。换句话说，尝试回答以下几个问题：

Considering edited post, I would say that DataTable is already highly optimized on in memory access, and I don't think changing it to something else will bring you a notable benefits. What I think can bring a banefit is a revision of a program flow. In other words, try to answer following questions:

• 请我需要所有的记录当代的？


• 我可以运行在服务一些计算，比方说，夜间？


• 我可以使用 SQL Server的防爆preSS （只是举例），获得的可能性，运行存储过程，即可能的（必须测量）运行速度更快的东西，即使是在同一台机器上？



• do I need all that records contemporary ?
• can I run some calculations in service, let's say, on night ?
• can I use SQL Server Express (just example) and gain benefits of possibility to run a stored procedure, that may (have to be measured) run the stuff faster, even on the same machine ?

这篇关于比DataTable中的其他最佳结构来存储〜半一亿前端记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！