ASP.NET自动化&手动ETL过程 [英] ASP.NET Automated & Manual ETL Process
问题描述
这是一个有趣的问题。我有一个ETL脚本写在c＃中，我已经在一个有规律的基础上手动运行。它用于更新我的网络应用程序的数据库。
我想自动化ETL过程，并创建一个接口，供网络应用管理员手动启动ETL过程。
我可以让sql server按照时间表启动ETL进程，并在网页上实现一个按钮或东西，这样做会做同样的事情，但我不会想把我的代码在2个不同的地方，因为我不想更新它在2个地方，当它改变。但我不知道如何让我的网络应用程序告诉SQL服务器手动启动一个计划的进程。可以这样做吗？
或
我可以以某种方式在Web应用程序本身实现调度，但现在大多数人都熟悉在尝试时遇到的问题（应用程序可能不是在某些时候运行，必须等待请求启动一个进程（没有一些诡计））。此外，由于ETL过程需要一段时间，我不想让一些可怜的最终用户等待响应，所以它肯定必须使用一个新的线程。
<我还能做什么？
您提到您正在使用SQL Server。它是什么版本？是SQL Server集成服务（这是为平台的完整的ETL工具）一个选项吗？如果您想要定期提取，转换和加载作业，SSIS是Microsoft平台上的一个伟大的工具，并包括在大多数SQL Server许可证中。
或者，请查看 Quartz 。
Here's an interesting problem. I have an ETL script written in c# that I have been running manually on a somewhat regular basis. It is used to update my web app's database.
I want to automate the ETL process AND create an interface for the web app admins to manually start the ETL process.
I could have sql server kick off the ETL process on a schedule and implement a button or something on a web page that will do the same thing, but I don't want to put my code in 2 different places because I don't want to update it in 2 places when it changes. But I don't know how to make my web app tell SQL server to manually start a scheduled process. Can this be done?
OR
I could somehow implement the scheduling in the web app itself, but by now most people are familiar with the problems that are faced when trying that (app may not be running at certain times, must wait on request to start a process (without some trickery)). Also, since the ETL process takes a while, I don't want to make some poor end user wait on a response, so it would definitely have to use a new thread.
What else could I do? What would you do?
You mentioned you are using SQL Server. What version is it? Is SQL Server Integration Services (which is the full-blown ETL tool for the platform) an option for you? If you want scheduled Extraction, Transformation and Load jobs, SSIS is a great tool on the Microsoft platform and is included in most SQL Server licenses.
Or, check out Quartz.
这篇关于ASP.NET自动化&手动ETL过程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
