如何使用Viewstate参数抓取页面请求? [英] how to scrape a page request using Viewstate parameter?

问题描述

我一直在尝试抓取此网站: http://www.e3050.com/Cases-Fans-PDU/C

I've been trying to scrape this website : http://www.e3050.com/Cases-Fans-PDU/C

除调试后，我可以正常刮除任何内容，但移至下一页，发现它们为每个页面请求发送了__Viewstate参数.viewstate参数存储在每个页面响应中，因此我发现需要按页面获取它并将其发送到下一页.我使用这个xpath来获取__viewstate:

I can scrape anything normally but moving to the next page, after debugging I found that they send the __Viewstate parameter for each page request. The viewstate parameter is stored in each page response, so I figured out that I need to get it per page and send it to the following page. I get the __viewstate using this xpath :

sel.xpath('//input[@id="__VIEWSTATE"]/@value').extract()

我也遇到了一个错误，因为他们发送的视图状态不同于页面响应中列出的视图状态，这两个参数都是64位基本编码，但是他们每个请求发送的参数比我从页面响应中获取的参数多

I also got an error, because the viewstate they send is different than the one enlisted in their page response, both parameters are 64 base encoded but the one they send per request has more data than the one I get from the page response.

我该如何处理?以及他们如何获取viewstate参数?

how can I deal with this ? and how does they get the viewstate parameter ?

__EVENTVALIDATION参数存在相同的问题.

The same issue for __EVENTVALIDATION parameter.

推荐答案

__ VIEWSTATE是出于安全原因在.net中使用的参数.它基本上是引荐页到您所请求页面的哈希.它通常嵌入在标记中，您只需要在每个请求之前将其提取，然后将其添加到下一个请求中即可.

__VIEWSTATE is a parameter used in .net for a security reason. it's basically a hash of the referral page to the page you're requesting. it's usually embedded in an tag, you just need to extract it before each request and add it to your next request.

这篇关于如何使用Viewstate参数抓取页面请求?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！