网页爬虫 - Python requests爬虫，如何post payload里的数据，谢谢！
本文介绍了网页爬虫 - Python requests爬虫，如何post payload里的数据，谢谢！的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
问 题
我用requests做post请求的时候，怎么用这个payload呢？
data数据要怎么构造
form-data的数据，我会。
import requests
import json
url='http://tx3.cbg.163.com/cgi-bin/equipquery.py?act=show_overall_search'
html = requests.post(url,data=data)
解决方案
根据楼上两位解答，经过修改已经顺利爬出数据
# -*- coding: utf-8 -*-
import requests
import sys
import json
reload(sys)
sys.setdefaultencoding('utf-8')
def get_tx3cbg():
url = 'http://tx3.cbg.163.com/cgi-bin/search.py'
headers = {
'Content-Type':'application/x-www-form-urlencoded; charset=UTF-8',
'Cookie':'overall_search_equips_cookie=%7C89_212365%7C23_211970%7C9_204469%7C2_156154%7C192_28326%7C176_74442%7C15_480910%7C15_477817%7C2_156122%7C10_156067%7C10_133849%7C15_481140%7C78_425212%7C12_165185%7C161_222125; usertrack=ZUcIhliu8JNB01gjA64jAg==; _ntes_nnid=f90c5442c5912e2adc252657cb12d457,1487860049616; _ntes_nuid=f90c5442c5912e2adc252657cb12d457; NTES_CMT_USER_INFO=35428995%7C%E5%BE%A1%E5%89%91%E6%8A%8A%E9%85%92%E5%90%AC%E7%A7%8B%E9%9B%A8%7Chttps%3A%2F%2Fsimg.ws.126.net%2Fe%2Fimg5.cache.netease.com%2Ftie%2Fimages%2Fyun%2Fphoto_default_62.png.39x39.100.jpg%7Cfalse%7CZmFuZ3dlaXJlbjg0M0AxNjMuY29t; UM_distinctid=15b0f71ba430-0e2166208-581f3413-1fa400-15b0f71ba4415a; vjuids=28a3aef4a.15b1fcdeaec.0.5e87609c; vjlast=1490887240.1490887240.30; vinfo_n_f_l_n3=c384af10aeab8944.1.0.1490887240441.0.1490887272552; P_INFO=fangweiren843@163.com|1491746466|1|blog|11&16|zhj&1491645222&gamebbs#zhj&330500#10#0#0|157600&0|gamebbs&tx2&mail163&blog|fangweiren843@163.com; _ga=GA1.2.69335350.1487859926; latest_views=15_466493-89_212755-1_217134-19_190513; __utma=187553192.69335350.1487859926.1492699270.1492699270.1; __utmz=187553192.1492699270.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __oc_uuid=65296ee0-25d7-11e7-be4c-213889b11650; last_login_serverid=19; area_td_id=1; overall_sid=000bEiuYYQVly-GWAHJ3BgbcS4ypMSxBavIhk6yVALj',
'Referer':'http://tx3.cbg.163.com/cgi-bin/equipquery.py?act=show_overall_search',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36'
}
payload = {
'act':'overall_search_role',
'order_by':'',
'page':'1',
'other_arg':'',
'price_min':'50000',
'price_max':'60000'
}
html = requests.post(url,data=payload,headers=headers).text
html = json.loads(html)
print html
get_tx3cbg()这篇关于网页爬虫 - Python requests爬虫，如何post payload里的数据，谢谢！的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
