使用哪种技术在后端运行搜寻器和更新数据库? [英] Which technology to use for running crawler and updating database in back end ?

问题描述

嗨
我正在为大学开发项目，在开发过程中需要一些建议.它是一个网站，显示来自其他网站的信息，例如链接，图像等.

我已经为网站准备了下面给出的模型.

一个Home.aspx页，显示表中的数据(SQL Server).

我已经用C#编写了一个爬网程序，可以对所需的网站数据进行爬网(获取数据).

我想以某种方式可以在一段时间内在后端运行搜寻器，并且可以在表中插入更新.我希望我可以在数据库中获取更新的信息，以便Home.aspx显示更新的信息. (就像Google新闻网站的较小版本一样)

我想在共享托管环境中托管wesbite(即第三方托管提供商公司，并且可能使用IIS平台)

我在类似的情况下发布到不同的.NET论坛和社区，他们提出了许多不同的建议，例如

-创建一个Web服务(真的有必要吗?)

-使用WCF

-创建一个控制台应用程序并运行Windows任务sheduler(使用asp.net(Win Forms网站)并在共享主机中可以)

-在本地计算机上运行搜寻器，并相应地更新数据库. (不，我要在线上一切)

请为我建议一个明确的出路，以便我完成任务.请提出适合我的项目的先进技术和方法.

等待中...

谢谢...

Hi
I am developing a project for college and i need some suggestions over the development. Its a website which shows information from other websites like Links, Images etc.

I have prepared below given model for the website.

A Home.aspx page which shows data from tables (sql server).

I have coded a crawler (in c#) which can crawl (fetch data) required website data.

I want some way through which i can run the crawler at back end for some time interval and it can insert updates in the tables. I want that i can get updated information in my database so that Home.aspx shows updated info. (Its like smaller version of Google News website)

I want to host the wesbite in Shared Hosted Environment (i.e a 3rd party hosting provider company and that may use IIS platform)

I posted simliar situation to different .NET forums and communities and they suggested lot of different things such as

- Create a web service (is it really necessary ?)

- Use WCF

- Create a Console application and run windows task sheduler (is it okay with asp.net (win forms website) and in shared hosted)

- Run crawler on local machine and update database accordingly. (No i want everything online)

Please suggest me a clear way out so that i complete the task. Please suggest elobrated technology and methods which suits my project.

Waiting...

Thanks...

推荐答案

根据您的托管环境，大多数托管公司都允许您对诸如此类的事情进行批判.我不知道IIS是否可以做这些类型的事情，但是在* nix框上这是很常见的事情.

Depending on your hosted environment, most hosting companies allow you to make a cronjob for things like that. I don''t know if IIS can do those type of things but on *nix boxes it is a lot more common occurance.

这篇关于使用哪种技术在后端运行搜寻器和更新数据库?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！