一个30 +小时的查询，最好的架构 [英] Best architecture for a 30 + hour query

问题描述

我有一个有趣的问题来解决。我的一个客户有我开发的股票分析程序接近50年之久的股票数据为近千年的符号。我已经开发了一系列的过滤器被应用在任何一天来看看是否有什么掉下来了交易。

I have an interesting problem to solve. One of my clients has me developing a stock analysis program with close to 50 years of stock data for almost a thousand symbols. I've developed a series of filters that are applied on any given day to see if anything falls out for a trade.

我们要运行这个过滤器数据的每一天，我们对每一只股票。基本上你的开始和结束日期类型的报告。然而，它需要6分钟之内，一周来筛选每个符号。我们正在盘算约40个小时左右来运行我们的整个数据集的报告。

We want to run this filter for each day of data we have for each stock. Basically your begin and end date type report. However it takes 6 minutes to filter each week for each symbol. We are figuring about 40 hours or so to run the report on our entire data set.

的首要要求是，我的客户是能够从任何计算机上的任何地方（他旅行了很多）应用程序做任何事情，所以我们基于浏览器。

The overriding requirement is that my client is able to do anything in the application from any computer anywhere (he travels a lot), so we are browser based.

为了解决这个问题，我写了运行此报告的异步方法，但是应用程序池不活动计时器会杀了这项工作。我不希望有开始调整超时为整个应用程序来支持这个报告（我们会做很多的这些，因为每个股票之情况会需要它被用于之前对我们分析整个数据集运行交易活跃）。

To solve this issue, I wrote an asychronous method that runs this report, however the application pool inactivity timer will kill the job. I don't want to have to start adjusting timeouts for the entire application to support this one report (we are going to do a lot of these as every stock scenerio will need to be run against our entire dataset for analysis before it gets used for active trading).

有没有人有一个网络架构，支持超长时间异步进程任何一般的想法或经验？

Does anyone have any general ideas or experiences with a web architecture that will support ultra-long asychronous processes?

感谢

推荐答案

作为一般的建议，我会建议一个独立的Windows服务，控制台应用程序或非常小心寿命控制和日志类似，这将不断地运行，检查（调查）关于'作业要处理在数据库中，然后更新与结果的数据库和进度信息。

As a general suggestion I would recommend a standalone Windows Service, Console App or similar with very careful lifetime controls and logging, which would run constantly and check (poll) for 'jobs to process' in a database, then update the database with results and progress information.

这可能不是最好的方式，但我以前多次使用过它和它的可靠，可扩展，具有良好的性能。

It may not be the best way but I've used it before many times and it's reliable, scalable and has good performance.

最好保持Web请求一两分钟最大 - 他们从未被设计为重的处理时间。这样你就可以在工作状态检查每分钟左右（使用Web服务）。

Best to keep web requests to a minute or two maximum - they were never designed for heavy processing times. This way you can 'check in' on the job status every minute or so (using a Web Service).

如果你中有我，或者对这个想法有任何疑问，请发表评论＆安培;我会很乐意帮忙，精心制作或建议。

If you have any questions of me or about the idea please post a comment & I will be happy to help, elaborate or suggest..

希望帮助！

（另：我相信Windows服务未充分利用它所需要的是可重用的helper方法快速基类或集合，你已经有了一个记录的，可靠的，自动的，可配置的，快速对实施过程中运行你的！控制，快速原型太多！）

(Additional: I believe Windows Services are underused! All it takes is a quick base class or collection of reusable helper methods and you've got a logged, reliable, automatic, configurable, quick-to-implement process running under your control. Quick to prototype with too!)

这篇关于一个30 +小时的查询，最好的架构的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！