无法让我的脚本自动生成几个值以在有效负载中使用 [英] Unable to let my script generate few values automatically to be used within payload

问题描述

我创建了一个脚本，通过随后发送两个 https 请求来从目标页面获取 html 元素.我的脚本可以完美地完成这件事.但是，我必须从 chrome 开发工具中复制四个值来填充 payload 中的四个键，以便发送最终的 http 请求以到达目标页面.这是

我已经尝试过(工作中的一个):

导入请求从 bs4 导入 BeautifulSoupurl = 'https://booking.discoverqatar.qa/SearchHandler.aspx?'second_url = 'https://booking.discoverqatar.qa/PassengerDetails.aspx?'参数 = {'Module':'H','txtCity':'','hdnCity':'2947','txtHotel':'','hdnHotel':'','fromDate':'05/11/2019','toDate':'07/11/2019','selZone':'','minSelPrice':'','maxSelPrice':'','roomConfiguration':'2|0|','noOfRooms':'1','hotelStandardArray':'63,60,54,50,52,51','CallFrom':'','DllNationality':'-1','HdnNoOfRooms':'-1','SourceXid':'MTEzNzg=','mdx':''}有效载荷 = {'CallFrom':'MToxNjozOCBQTXxCMkN8MToxNjozOCBQTQ==','Btype':'MToxNjozOCBQTXxBfDE6MTY6MzggUE0=','PaxConfig':'MToxNjozOCBQTXwyfDB8MnwwfHwxOjE2OjM4IFBN','usid':'MToxNjozOCBQTXxoZW54dmkzcWVnc3J3cXpld2lsa2ZwMm18MToxNjozOCBQTQ=='}使用 requests.Session() 作为 s:r = s.get(url,params=params,headers={"User-agent":"Mozilla/5.0"})res = s.get(second_url,params=payload,headers={"用户代理":"Mozilla/5.0",推荐人":r.url})汤 = BeautifulSoup(res.text,'lxml')打印(汤)

在上面的脚本中，我复制并粘贴了 CallFrom、Btype、PaxConfig 和 usid 的值> 从开发工具中使用 payload.

<块引用>

如何自动填写要在有效负载中使用的值?

解决方案

发送给第二个请求的参数是 Base64 编码的，解码后它们是:

 'CallFrom':'1:16:38 PM|B2C|1:16:38 PM','Btype':'1:16:38 PM|A|1:16:38 PM','PaxConfig':'1:16:38 PM|2|0|2|0||1:16:38 PM','usid':'1:16:38 PM|henxvi3qegsrwqzewilkfp2m|1:16:38 PM'

乍一看，您已经注意到它们的模式:

$date|$param|$date

其中$date是当前时间，格式为utc_ts_now.strftime("%I:%M:%S %p").

对于这四个参数的$param部分，我想对于CallFrom和Btype，usid应该是固定的code> 是 session 键，您可以在之前的响应中轻松找到它.

PaxConfig 是客人数量，它与您在第一个请求中发送的 roomConfiguration 相关.

要自动执行第二个请求，您需要先为每个参数生成解码值，然后使用 Base64 对其进行编码.

更新:

#!/usr/bin/env python3.7导入 base64从日期时间导入日期时间进口请求def first_request(session, params):url = 'https://booking.discoverqatar.qa/SearchHandler.aspx'r = session.get(url, params=params)返回定义第二个请求(会话，参数):url = 'https://booking.discoverqatar.qa/PassengerDetails.aspx'r = session.get(url, params=params)返回定义主():参数 1 = {'模块':'H'，'txtCity': '','hdnCity': '2947','txtHotel': '','hdnHotel': '','fromDate': '05/11/2019','toDate': '07/11/2019','selZone': '','minSelPrice': '','maxSelPrice': '','房间配置':'2|0|'，'noOfRooms': '1','hotelStandardArray': '63,60,54,50,52,51','CallFrom': '','DllNationality': '-1','HdnNoOfRooms': '-1','SourceXid': 'MTezNzg=','mdx':''}session = requests.Session()_ = first_request(会话，params1)asp_session = session.cookies.get("ASP.NET_SessionId")参数 2 = {# 可能与选项Available"/On Request"有关"Btype": "A",# 尝试其他客人数量以确保"PaxConfig": params1["roomConfiguration"] * 2,"CallFrom": "B2C",usid":asp_session}date = datetime.utcnow().strftime("%I:%M:%S %p")对于 params2.items() 中的 k, v:v = "|".join([日期, v, 日期])v = base64.b64encode(bytes(v, "utf-8")).decode("utf-8")参数2[k] = vr = second_request(会话，params2)打印(r.text)如果 __name__ == '__main__':主要的()

I've created a script to get the html elements from a target page by sending two https requests subsequently. My script can does the thing flawlessly. However, I had to copy the four values from chrome dev tools to fill in the four keys within payload in order to send the final http requests to reach the target page. This is the starting link and following are the description as to how I could reach the target page.

1. Click on the Find Hotel button (no need to change dates if chek-out date is by default at least one day longer than check-in date).
2. Tick the box like the image below and press the Book Now button just above it. Now, it should lead you to the target page automatically.
3. Upon reaching the target page titled as Enter Guest Details, parse the html elements from there

I've tried with (working one):

import requests
from bs4 import BeautifulSoup

url = 'https://booking.discoverqatar.qa/SearchHandler.aspx?'
second_url = 'https://booking.discoverqatar.qa/PassengerDetails.aspx?'

params = {
    'Module':'H','txtCity':'','hdnCity':'2947','txtHotel':'','hdnHotel':'',
    'fromDate':'05/11/2019','toDate':'07/11/2019','selZone':'','minSelPrice':'',
    'maxSelPrice':'','roomConfiguration':'2|0|','noOfRooms':'1',
    'hotelStandardArray':'63,60,54,50,52,51','CallFrom':'','DllNationality':'-1',
    'HdnNoOfRooms':'-1','SourceXid':'MTEzNzg=','mdx':''
}

payload = {
    'CallFrom':'MToxNjozOCBQTXxCMkN8MToxNjozOCBQTQ==',
    'Btype':'MToxNjozOCBQTXxBfDE6MTY6MzggUE0=',
    'PaxConfig':'MToxNjozOCBQTXwyfDB8MnwwfHwxOjE2OjM4IFBN',
    'usid':'MToxNjozOCBQTXxoZW54dmkzcWVnc3J3cXpld2lsa2ZwMm18MToxNjozOCBQTQ=='  
}

with requests.Session() as s:
    r = s.get(url,params=params,headers={"User-agent":"Mozilla/5.0"})
    res = s.get(second_url,params=payload,headers={
        "User-agent":"Mozilla/5.0",
        "Referer":r.url
        })
    soup = BeautifulSoup(res.text,'lxml')
    print(soup)

In the above script I've copied and pasted the value of CallFrom,Btype,PaxConfig and usid from dev tools to use within payload.

How can I fill in the values automatically to be used within payload?

解决方案

Params sent to the second request is Base64 encoded, after decode they are:

    'CallFrom':'1:16:38 PM|B2C|1:16:38 PM',
    'Btype':'1:16:38 PM|A|1:16:38 PM',
    'PaxConfig':'1:16:38 PM|2|0|2|0||1:16:38 PM',
    'usid':'1:16:38 PM|henxvi3qegsrwqzewilkfp2m|1:16:38 PM'  

At first glance, you already notice they are in patterns of:

$date|$param|$date

Where $date is current time in the format of utc_ts_now.strftime("%I:%M:%S %p").

For $param section of these four parameters, I guess it should be fixed for CallFrom and Btype, usid is the session key, you can find it easily in the previous response.

PaxConfig is guest counts, it's related to roomConfiguration you sent in the first request.

To automate the second request, you would generate the decoded value for each parameter first, then encode them with Base64.

Update:

#!/usr/bin/env python3.7
import base64
from datetime import datetime

import requests


def first_request(session, params):
    url = 'https://booking.discoverqatar.qa/SearchHandler.aspx'
    r = session.get(url, params=params)
    return r


def second_request(session, params):
    url = 'https://booking.discoverqatar.qa/PassengerDetails.aspx'
    r = session.get(url, params=params)
    return r


def main():
    params1 = {
        'Module':             'H',
        'txtCity':            '',
        'hdnCity':            '2947',
        'txtHotel':           '',
        'hdnHotel':           '',
        'fromDate':           '05/11/2019',
        'toDate':             '07/11/2019',
        'selZone':            '',
        'minSelPrice':        '',
        'maxSelPrice':        '',
        'roomConfiguration':  '2|0|',
        'noOfRooms':          '1',
        'hotelStandardArray': '63,60,54,50,52,51',
        'CallFrom':           '',
        'DllNationality':     '-1',
        'HdnNoOfRooms':       '-1',
        'SourceXid':          'MTEzNzg=',
        'mdx':                ''
    }
    session = requests.Session()
    _ = first_request(session, params1)
    asp_session = session.cookies.get("ASP.NET_SessionId")

    params2 = {
        # Could related to options "Available" / "On Request"
        "Btype":     "A",

        # Try out other guest counts to make sure
        "PaxConfig": params1["roomConfiguration"] * 2,

        "CallFrom": "B2C",
        "usid":     asp_session
    }
    date = datetime.utcnow().strftime("%I:%M:%S %p")
    for k, v in params2.items():
        v = "|".join([date, v, date])
        v = base64.b64encode(bytes(v, "utf-8")).decode("utf-8")
        params2[k] = v
    r = second_request(session, params2)
    print(r.text)


if __name__ == '__main__':
    main()

这篇关于无法让我的脚本自动生成几个值以在有效负载中使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！