如何通过URL下载完整的网站或网页？ [英] How to download full website or web pages through URL?

问题描述

我必须处理像网络下载器这样的应用程序，它应该允许用户通过其URL将完整网站（网站的所有文件）下载到磁盘上的指定位置。目前我只能使用WebRequest和WebResponse类获得主页的html。我尝试过使用

 WebClient Client = new WebClient（）; 
 Client.DownloadFile（http://www.examplesite.com/Default.aspx，D：\\Downloader \\Default.aspx）; 

但是在这里我必须将具有相同名称的文件放在物理位置以便下载，我也无法在页面上下载CSS和JS。我还想允许我的应用程序包含内置浏览器，以便在离线模式下浏览下载的文件。



请帮助我这方面。提前感谢您的善意指导。

解决方案

严格来说，完整网站是对所有网站无效的概念。有些页面是纯粹的交互式或随机的，因此，即使可以某种方式复制它们，每次复制也可能不同;而在其他情况下，页面本身只能由于用户的活动而生成。这些页面的例子甚至都不是很少见;一个典型的案例是游戏。



至于其余部分，你需要的是一些网络抓取的技巧：

http://en.wikipedia.org/wiki/Web_scraping [ ^ ]。



请查看我过去的答案欲了解更多信息：

从中获取具体数据网页 [ ^ ]，

如何从其他网站获取数据 [ ^ ]。

-S A

I have to work on an application like Website Downloader which should allow user to download full website (all the files of the website) to a specified location on the disk through its URL. Currently I am just able to get only html of Home Page using WebRequest and WebResponse classes. I have tried another solution using

WebClient Client = new WebClient ();
Client.DownloadFile("http://www.examplesite.com/Default.aspx", "D:\\Downloader\\Default.aspx");

But here I have to place file with same name on physical location as to be downloaded and I am also unable to download CSS and JS being using on the page. I also want to allow my applicaiton to include built in browser in order to browse downloaded file in offline mode.

Please help me in this regard. Thanks in advance for your kind guidance.

解决方案

Strictly speaking, "full Website" is the concept which is not valid for all sites. Some pages are purely interactive or random, so, even if they can be copied somehow, the copy could be different each time; and, in other cases, the page itself can only be generated as a result of the user's activity. The examples of such pages are not even too rare; one typical case is games.

As to the rest of it, you need is some techniques of Web scraping:
http://en.wikipedia.org/wiki/Web_scraping[^].

Please see my past answers for further information:
get specific data from web page[^],
How to get the data from another site[^].

—SA

这篇关于如何通过URL下载完整的网站或网页？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！