网站在其网页中使用_dopostback时，在开发搜寻器时会遇到问题 [英] have problem in developing a crawler when websites use _dopostback in their page ource

问题描述

我正在编写一个搜寻器...我遇到一个问题...当网站使用_dopostback进行分页时，应该如何找到页面的内容?

无论如何，使用_dopostback
时都可以访问下一页/上一页的源页面.
我真的卡住了，没有任何想法

非常感谢您的事先帮助.

I am writing a crawler...I encounter a problem...how should I find the content of the page when websites are using _dopostback for their pagination?

Is there anyway to have access the source page of the next/prev page when using _dopostback

I really stuck and dont have any idea

Thank you very much for your help in advance

推荐答案

在开发搜寻器时，切勿发送POST请求，这意味着您将无法访问仅使用_doPostBack进行导航的页面.
通常，网站管理员会添加xml网站地图，其中包含指向网站页面的链接:
http://www.xml-sitemaps.com/about-sitemaps.html [ ^ ]
您可以利用它们.

As you are developing a crawler, you should never send POST requests, that means that you will not be able to access pages that have navigation only using _doPostBack.
Typically webmasters add xml sitemaps that contain links to pages of the site:
http://www.xml-sitemaps.com/about-sitemaps.html[^]
you can make use of them.

这篇关于网站在其网页中使用_dopostback时，在开发搜寻器时会遇到问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！