使用python将请求发布到asp.net页面 [英] post request using python to asp.net page

问题描述

我想从http://www.indiapost.gov.in/pin/中删除 PINCODE"，我正在编写以下代码.

i want scrap the PINCODEs from "http://www.indiapost.gov.in/pin/", i am doing with following code written.

import urllib
import urllib2
headers = {
    'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Origin': 'http://www.indiapost.gov.in',
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko)  Chrome/24.0.1312.57 Safari/537.17',
    'Content-Type': 'application/x-www-form-urlencoded',
    'Referer': 'http://www.indiapost.gov.in/pin/',
    'Accept-Encoding': 'gzip,deflate,sdch',
    'Accept-Language': 'en-US,en;q=0.8',
    'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
}
viewstate = 'JulXDv576ZUXoVOwThQQj4bDuseXWDCZMP0tt+HYkdHOVPbx++G8yMISvTybsnQlNN76EX/...'
eventvalidation = '8xJw9GG8LMh6A/b6/jOWr970cQCHEj95/6ezvXAqkQ/C1At06MdFIy7+iyzh7813e1/3Elx...'
url = 'http://www.indiapost.gov.in/pin/'
formData = (
    ('__EVENTVALIDATION', eventvalidation),
    ('__EVENTTARGET',''),
    ('__EVENTARGUMENT',''),
    ('__VIEWSTATE', viewstate),
    ('__VIEWSTATEENCRYPTED',''),
    ('__EVENTVALIDATION', eventvalidation),
    ('txt_offname',''),
    ('ddl_dist','0'),
    ('txt_dist_on',''),
    ('ddl_state','2'),
    ('btn_state','Search'),
    ('txt_stateon',''),
    ('hdn_tabchoice','3')
)


from urllib import FancyURLopener
class MyOpener(FancyURLopener):
    version = 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17'

myopener = MyOpener()


encodedFields = urllib.urlencode(formData)

f = myopener.open(url, encodedFields)
print f.info()

try:
fout = open('tmp.txt', 'w')
except:
print('Could not open output file
')

fout.writelines(f.readlines())
fout.close()

我收到来自服务器的响应抱歉，此站点遇到严重问题，请尝试重新加载页面或联系网站管理员."请建议我哪里出错了..

i am getting response from server as "Sorry this site has encountered a serious problem, please try reloading the page or contact webmaster." pl suggest where i am going wrong..

推荐答案

你从哪里得到 viewstate 和 eventvalidation 的值?一方面，它们不应该以..."结尾，你一定遗漏了一些东西.另一方面，它们不应该被硬编码.

Where did you get the value viewstate and eventvalidation? On one hand, they shouldn't end with "...", you must have omitted something. On the other hand, they shouldn't be hard-coded.

一个解决方案是这样的:

One solution is like this:

1. 通过 URLhttp://www.indiapost.gov.in/pin/"检索页面任何表单数据
2. 解析和检索表单值，例如 __VIEWSTATE 和 __EVENTVALIDATION(您可以使用 BeautifulSoup).
3. 通过添加第 2 步中的重要表单数据来获取搜索结果(第二个 HTTP 请求).

1. Retrieve the page via URL "http://www.indiapost.gov.in/pin/" without any form data
2. Parse and retrieve the form values like __VIEWSTATE and __EVENTVALIDATION (you may take use of BeautifulSoup).
3. Get the search result(second HTTP request) by adding vital form-data from step 2.

更新:

根据上面的想法，我稍微修改你的代码以使其工作:

According to the above idea, I modify your code slightly to make it work:

import urllib
from bs4 import BeautifulSoup

headers = {
    'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Origin': 'http://www.indiapost.gov.in',
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko)  Chrome/24.0.1312.57 Safari/537.17',
    'Content-Type': 'application/x-www-form-urlencoded',
    'Referer': 'http://www.indiapost.gov.in/pin/',
    'Accept-Encoding': 'gzip,deflate,sdch',
    'Accept-Language': 'en-US,en;q=0.8',
    'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
}

class MyOpener(urllib.FancyURLopener):
    version = 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17'

myopener = MyOpener()
url = 'http://www.indiapost.gov.in/pin/'
# first HTTP request without form data
f = myopener.open(url)
soup = BeautifulSoup(f)
# parse and retrieve two vital form values
viewstate = soup.select("#__VIEWSTATE")[0]['value']
eventvalidation = soup.select("#__EVENTVALIDATION")[0]['value']

formData = (
    ('__EVENTVALIDATION', eventvalidation),
    ('__VIEWSTATE', viewstate),
    ('__VIEWSTATEENCRYPTED',''),
    ('txt_offname', ''),
    ('ddl_dist', '0'),
    ('txt_dist_on', ''),
    ('ddl_state','1'),
    ('btn_state', 'Search'),
    ('txt_stateon', ''),
    ('hdn_tabchoice', '1'),
    ('search_on', 'Search'),
)

encodedFields = urllib.urlencode(formData)
# second HTTP request with form data
f = myopener.open(url, encodedFields)

try:
    # actually we'd better use BeautifulSoup once again to
    # retrieve results(instead of writing out the whole HTML file)
    # Besides, since the result is split into multipages,
    # we need send more HTTP requests
    fout = open('tmp.html', 'w')
except:
    print('Could not open output file
')
fout.writelines(f.readlines())
fout.close()

这篇关于使用python将请求发布到asp.net页面的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！