内容格式不正确的网页抓取问题 [英] Trouble Scraping Web Page With Malformed Content

问题描述

我已经编写了使用HtmlAgilityPack库的c#代码，以刮取位于以下位置的页面:世界上最大的市区(第2页).不幸的是，该页面包含格式错误的内容.

I have written c# code which utilizes the HtmlAgilityPack library in order to scrape a page located at: World's Largest Urban Areas (Page 2). Unfortunately the page consists of malformed content.

我对如何抓取此页面陷入僵局.我拥有的当前代码(如下所示)冻结了对HTML的解析:

I'm at an impasse on how to scrape this page. The current code I have (appearing below) freezes on parsing the HTML:

 HtmlNodeCollection cityRecords = _htmlDocument.DocumentNode.SelectNodes("//table[@class='boldtable']//tr[position() != 1]");
 CityNodes = (from node in cityRecords.Descendants()
              where node.Name == "td"
              select node).ToList();

目标是使用每个数据点解析页面上列出的每个城市；而已.寻找有关如何修改上述代码或使用其他免费提供的库的建议.

The goal is to parse each and every city listed on the page with each of the data points; nothing more. Looking for recommendations on how to modify the above code or use another freely available library.

谢谢！

推荐答案

在解析内容之前，先通过HTML Tidy运行内容.

Run the content through HTML Tidy before parsing it.

http://tidy.sourceforge.net/

这篇关于内容格式不正确的网页抓取问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！