POST 请求给出空结果 [英] POST request giving empty results

问题描述

我已经使用 POST 请求在 python 中编写了一些代码来从网页中获取特定数据.但是，当我运行它时，除了空白控制台之外，我什么也没有得到.我尝试相应地填写请求参数.也许，我没有注意到参数中应该包含哪些内容.我正在处理的页面在其右侧面板中包含多个图像.当单击图像时，我在这里谈论的请求将发送到服务器并带回结果并在其下显示有关其风味的新信息.我的目标是解析与每个图像相关的所有风格.无论如何，我试图附上所有必要的东西来找出我遗漏的东西.提前致谢.

I've written some code in python using POST request to fetch specific data from a webpage. However, when I run it, I get nothing as result except for a blank console. I've tried to fill in the request parameter accordingly. Perhaps, I can't notice which should be included in the parameter. The page I'm dealing with contains several images in it's right panel. When an image is clicked the request about which i'm talking here is sent to the server and brings back the result and displays new information concerning its' flavor under it. My goal is to parse all the flavors connected to each images. Anyways, I'm trying to attach all the things necessary to find out what i'm missing. Thanks in advance.

这是我从 Chrome 开发人员工具中得到的用于准备 POST 请求的内容:

This is what I got from chrome developer tools to prepare the POST request:

===================================================================================
General:
 Request URL:https://www.optigura.com/product/ajax/details.php
 Request Method:POST
 Status Code:200 OK

Response Headers:
 Cache-Control:no-store, no-cache, must-revalidate
 Cache-Control:max-age=0, no-cache, no-store, must-revalidate
 Connection:Keep-Alive
 Content-Encoding:gzip
 Content-Length:782
 Content-Type:text/html; charset=utf-8

Request Headers:
 Accept:application/json, text/javascript, */*; q=0.01
 Accept-Encoding:gzip, deflate, br
 Accept-Language:en-US,en;q=0.8
 Connection:keep-alive
 Content-Length:34
 Content-Type:application/x-www-form-urlencoded
 Cookie:OGSESSID=s1qqd0euokbfrdub9pf2efubh1; _ga=GA1.2.449310094.1501502802; _gid=GA1.2.791686763.1501502802; _gat=1; __atuvc=1%7C31; __atuvs=597f1d5241db0352000; beyable-TrackingId=499b4c5b-2939-479b-aaf0-e5cd79f078cc; aaaaaaaaa066e9a68e5654b829144016246e1a736=d5758131-71db-41e1-846d-6d719d381060.1501502805122.1501502805122.$bey$https%3a%2f%2fwww.optigura.com%2fuk%2fproduct%2fgold-standard-100-whey%2f$bey$1; aaaaaaaaa066e9a68e5654b829144016246e1a736_cs=; aaaaaaaaa066e9a68e5654b829144016246e1a736_v=1.1.0; checkloc-uk=n
 Host:www.optigura.com
 Origin:https://www.optigura.com
 Referer:https://www.optigura.com/uk/product/gold-standard-100-whey/
 User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36
 X-Requested-With:XMLHttpRequest

Form Data:
opt:flavor
opt1:207
opt2:47
ip:105
=======================================================================================

这是我正在尝试的内容:

Here is what I'm trying with:

import requests
from lxml import html

payload = {"opt":"flavor","opt1":"207","opt2":"47","ip":"105"}
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36'}
response = requests.post("https://www.optigura.com/product/ajax/details.php", params = payload, headers = headers).text
print(response)

这是该网页的原始链接:https://www.optigura.com/uk/product/gold-standard-100-whey/

This is the original link to the webpage: https://www.optigura.com/uk/product/gold-standard-100-whey/

推荐答案

你应该试试下面的请求结构:

You should try below request structure:

• 要发送的数据:

• Data to send:

data = {'opt': 'flavor', 'opt1': '207', 'opt2': '47', 'ip': 105}

标题:

Headers:

headers = {'X-Requested-With': 'XMLHttpRequest'}

网址:

URL:

url = 'https://www.optigura.com/product/ajax/details.php'

你还需要获取 cookie，所以 requests.session() 是必需的:

s = requests.session()
r = s.get('https://www.optigura.com/uk/product/gold-standard-100-whey/')
cookies = r.cookies

完成请求:

response = s.post(url, cookies=cookies, headers=headers, data=data)

现在你可以得到所需的 HTML 为

Now you can get required piece of HTML as

print(response.json()['info2'])

输出:

'<ul class="opt2"><li class="active">
                    <label>
                        <input type="radio" name="ipr" value="1360" data-opt-sel="47" checked="checked" /> Delicious Strawberry - <span class="green">In Stock</span></label>
                </li><li>
                    <label>
                        <input type="radio" name="ipr" value="1356" data-opt-sel="15"  /> Double Rich Chocolate - <span class="green">In Stock</span></label>
                </li><li>
                    <label>
                        <input type="radio" name="ipr" value="1169" data-opt-sel="16"  /> Vanilla Ice Cream - <span class="green">In Stock</span></label>
                </li></ul>'

然后你可以使用 lxml 来抓取风味值:

Then you can use lxml to scrape flavor values:

from lxml import html

flavors = response.json()['info2']
source = html.fromstring(flavors)

[print(element.replace(' - ', '').strip()) for element in source.xpath('//label/text()[2]')]

输出:

Delicious Strawberry
Double Rich Chocolate
Vanilla Ice Cream

这篇关于POST 请求给出空结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！