比DataTable中的其他最佳结构来存储〜半一亿前端记录 [英] Best structure other than Datatable to store ~half-a-million records in Frontend

查看:155
本文介绍了比DataTable中的其他最佳结构来存储〜半一亿前端记录的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我有一个数据表是从数据库(SQLite的)获取〜5,50,000记录。获取当它被拖慢系统。

I have a Datatable which is fetching ~5,50,000 records from database(SQLite). When fetching it is slowing down the system.

我存储SQLite数据库和数据表中的前端后端这些记录。
现在我应该怎么在后台做,这样创建数据库时(〜10.5小时),并在前端获取时间减少了。

I am storing these records on backend in SQLite Database and in frontend in Datatable. Now what should i do so that database creation time(~10.5 hours) in backend and fetching time in Front end reduces.

是否有可用于这样做的任何其它结构。我已阅读,词典和放大器;二进制文件快。它们可用于此目的和放大器;怎么样?

Is there any other structure that can be used to do so. I have read that Dictionary & Binary File is fast. Can they be used for this purpose & how?

(它不是一个Web应用程序。它是一个WPF的桌面应用程序,其中前端和后端是同一台机器上)。

(Its not a web app. its a WPF desktop app where frontend and backend are on same machine).

推荐答案

您基本的问题,的我相信的,是不是要保持strucutre但的办法的您管理您的数据流。相反,从数据库中的所有数据取到一些晶格结构的(数据表你的情况),使一些存储过程作你需要的服务器端的计算,并返回到你已经计算的数据。通过这种方式,你可以获得很多好处,如:

Your basic problem, I believe, is not the strucutre that you want to maintain but the way you manage your data-flow. Instead of fetching all data from database into some strcuture (DataTable in your case), make some stored procedure to make a calculations you need on server-side and return to you data already calculated. In this way you gain several benefits, like:


  • 持有数据库服务器服务器的一般的那么快开发或客户机

  • 数据trasmission的大幅缩减,为您返回的只会导致计算

  • servers holding database servers are usually faster then development or client machine
  • huge reduction of data trasmission, as you return only result of the calculation

修改

考虑后期编辑,我要说的是数据表已经高度存储器存取的优化上,我不认为它更改为别的东西会给你带来一个显着好处。我觉得可以的带来banefit是程序流程的修订。换句话说,尝试回答以下几个问题:

Considering edited post, I would say that DataTable is already highly optimized on in memory access, and I don't think changing it to something else will bring you a notable benefits. What I think can bring a banefit is a revision of a program flow. In other words, try to answer following questions:


  • 请我需要所有的记录当代的?

  • 我可以运行在服务一些计算,比方说,夜间?

  • 我可以使用 SQL Server的防爆preSS (只是举例),获得的可能性,运行存储过程,即可能的(必须测量)运行速度更快的东西,即使是在同一台机器上?

  • do I need all that records contemporary ?
  • can I run some calculations in service, let's say, on night ?
  • can I use SQL Server Express (just example) and gain benefits of possibility to run a stored procedure, that may (have to be measured) run the stuff faster, even on the same machine ?

这篇关于比DataTable中的其他最佳结构来存储〜半一亿前端记录的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆