多 POST 查询(会话模式) [英] Multi POST query (session mode)
本文介绍了多 POST 查询(会话模式)的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我正在尝试查询此
您应该能够从中提取您想要的每个字段.
输出实际上有一些你在页面上看不到的额外结果，除非你切换下面的 tou 按钮:
您可以从结果中过滤这些内容，以便与默认输出完全匹配，或者提供一个包含辅助函数的选项:
def order_by(l, k, is_tou=False):如果不是 is_tou:filt = filter(lambda x: not x["offerDetails"][0]["isTouOffer"], l)return sorted(filt, key=lambda d: d["offerDetails"][0][k])return sorted(l, key=lambda d: d["offerDetails"][0][k])导入json使用 requests.Session() 作为 s:s.post(sub_url, data=form1)r = (s.get("https://compare.switchon.vic.gov.au/energy_questionnaire"))s.post("https://compare.switchon.vic.gov.au/energy_questionnaire/submit",数据=表格2)js = s.get("https://compare.switchon.vic.gov.au/service/offers").json()["offersList"]by_price = by_discount(js, "conditionalPrice", False)打印(by_price[:3)如果您检查输出，您将看到打开电源时的结果中价格为 840 的原始能源第三或关闭时 AGL 的价格为 860，您可以将其应用于折扣输出:
常规输出似乎也按 conditionalPrice 排序，如果您检查源代码，调用排序的两个 js 函数是:
ng-click="changeSortingField('conditionalPrice')"ng-click="changeSortingField('fullDiscountedPrice')"所以现在应该完全匹配站点输出.
I am trying to interrogate this site to get the list of offers. The problem is that we need to fill 2 forms (2 POST queries) before receiving the final result.
This what I have done so far:
First I am sending the first POST after setting the cookies:
library(httr)
set_cookies(.cookies = c(a = "1", b = "2"))
first_url <- "https://compare.switchon.vic.gov.au/submit"
body <- list(energy_category="electricity",
location="home",
"location-home"="shift",
"retailer-company"="",
postcode="3000",
distributor=7,
zone=1,
energy_concession=0,
"file-provider"="",
solar=0,
solar_feedin_tariff="",
disclaimer_chkbox="disclaimer_selected")
qr<- POST(first_url,
encode="form",
body=body)
Then trying to retrieve the offers using the second post query:
gov_url <- "https://compare.switchon.vic.gov.au/energy_questionnaire/submit"
qr1<- POST(gov_url,
encode="form",
body=list(`person-count`=1,
`room-count`=1,
`refrigerator-count`=1,
`gas-type`=4,
`pool-heating`=0,
spaceheating="none",
spacecooling="none",
`cloth-dryer`=0,
waterheating="other"),
set_cookies(a = 1, b = 2))
)
library(XML)
dc <- htmlParse(qr1)
But unfortunately I get a message indicating the end of session. Many thanks for any help to resolve this.
update add cookies:
I added the cookies and the intermediate GET, but I still don't have any of the results.
library(httr)
first_url <- "https://compare.switchon.vic.gov.au/submit"
body <- list(energy_category="electricity",
location="home",
"location-home"="shift",
"retailer-company"="",
postcode=3000,
distributor=7,
zone=1,
energy_concession=0,
"file-provider"="",
solar=0,
solar_feedin_tariff="",
disclaimer_chkbox="disclaimer_selected")
qr<- POST(first_url,
encode="form",
body=body,
config=set_cookies(a = 1, b = 2))
xx <- GET("https://compare.switchon.vic.gov.au/energy_questionnaire",config=set_cookies(a = 1, b = 2))
gov_url <- "https://compare.switchon.vic.gov.au/energy_questionnaire/submit"
qr1<- POST(gov_url,
encode="form",
body=list(
`person-count`=1,
`room-count`=1,
`refrigerator-count`=1,
`gas-type`=4,
`pool-heating`=0,
spaceheating="none",
spacecooling="none",
`cloth-dryer`=0,
waterheating="other"),
config=set_cookies(a = 1, b = 2))
library(XML)
dc <- htmlParse(qr1)
解决方案
using a python requests.Session object with the following data gets to the results page:
form1 = {"energy_category": "electricity",
"location": "home",
"location-home": "shift",
"distributor": "7",
"postcode": "3000",
"energy_concession": "0",
"solar": "0",
"disclaimer_chkbox": "disclaimer_selected",
}
form2 = {"person-count":"1",
"room-count":"4",
"refrigerator-count":"0",
"gas-type":"3",
"pool-heating":"0",
"spaceheating[]":"none",
"spacecooling[]":"none",
"cloth-dryer":"0",
"waterheating[]":"other"}
sub_url = "https://compare.switchon.vic.gov.au/submit"
with requests.Session() as s:
s.post(sub_url, data=form1)
r = (s.get("https://compare.switchon.vic.gov.au/energy_questionnaire"))
s.post("https://compare.switchon.vic.gov.au/energy_questionnaire/submit",
data=form2)
r = s.get("https://compare.switchon.vic.gov.au/offers")
print(r.content)
You should see the matching h1 in the returned html that you see on the page:
<h1>Your electricity offers</h1>
Or using scrapy form requests:
import scrapy
class Spider(scrapy.Spider):
name = 'comp'
start_urls = ['https://compare.switchon.vic.gov.au/energy_questionnaire/submit']
form1 = {"energy_category": "electricity",
"location": "home",
"location-home": "shift",
"distributor": "7",
"postcode": "3000",
"energy_concession": "0",
"solar": "0",
"disclaimer_chkbox": "disclaimer_selected",
}
sub_url = "https://compare.switchon.vic.gov.au/submit"
form2 = {"person-count":"1",
"room-count":"4",
"refrigerator-count":"0",
"gas-type":"3",
"pool-heating":"0",
"spaceheating[]":"none",
"spacecooling[]":"none",
"cloth-dryer":"0",
"waterheating[]":"other"}
def start_requests(self):
return [scrapy.FormRequest(
self.sub_url,
formdata=form1,
callback=self.parse
)]
def parse(self, response):
return scrapy.FormRequest.from_response(
response,
formdata=form2,
callback=self.after
)
def after(self, response):
print("<h1>Your electricity offers</h1>" in response.body)
Which we can verify has the "<h1>Your electricity offers</h1>":
2016-03-07 12:27:31 [scrapy] DEBUG: Crawled (200) <GET https://compare.switchon.vic.gov.au/offers#list/electricity> (referer: https://compare.switchon.vic.gov.au/energy_questionnaire)
True
2016-03-07 12:27:31 [scrapy] INFO: Closing spider (finished)
The next problem is the actual data is dynamically rendered which you can verify if you look at the source of the results page, you can actually get all the provider in json format:
with requests.Session() as s:
s.post(sub_url, data=form1)
r = (s.get("https://compare.switchon.vic.gov.au/energy_questionnaire"))
s.post("https://compare.switchon.vic.gov.au/energy_questionnaire/submit",
data=form2)
r = s.get("https://compare.switchon.vic.gov.au/service/offers")
print(r.json())
A snippet of which is:
{u'pageMetaData': {u'showDual': False, u'isGas': False, u'showTouToggle': True, u'isElectricityInitial': True, u'showLoopback': False, u'isElectricity': True}, u'offersList': [{u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.peopleenergy.com.au', u'offerId': u'PEO33707SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1410, u'offerType': u'Standing offer', u'offerName': u'Residential 5-Day Time of Use', u'conditionalPrice': 1410, u'fullDiscountedPrice': 1390, u'greenPower': 0, u'retailerName': u'People Energy', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType': u'Time of use', u'retailerPhone': u'1300 788 970', u'isPartDual': False, u'retailerId': u'7322', u'isTouOffer': True, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'1645', u'exitFeeCount': 0, u'timeDefinition': u'Local time', u'retailerImageUrl': u'img/retailers/big/peopleenergy.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.peopleenergy.com.au', u'offerId': u'PEO33773SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1500, u'offerType': u'Standing offer', u'offerName': u'Residential Peak Anytime', u'conditionalPrice': 1500, u'fullDiscountedPrice': 1480, u'greenPower': 0, u'retailerName': u'People Energy', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType': u'Single rate', u'retailerPhone': u'1300 788 970', u'isPartDual': False, u'retailerId': u'7322', u'isTouOffer': False, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'1649', u'exitFeeCount': 0, u'timeDefinition': u'AEST only', u'retailerImageUrl': u'img/retailers/big/peopleenergy.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.energythatcould.com.au', u'offerId': u'PAC33683SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1400, u'offerType': u'Standing offer', u'offerName': u'Vic Home Flex', u'conditionalPrice': 1400, u'fullDiscountedPrice': 1400, u'greenPower': 0, u'retailerName': u'Pacific Hydro Retail Pty Ltd', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType': u'Flexible Pricing', u'retailerPhone': u'1800 010 648', u'isPartDual': False, u'retailerId': u'15902', u'isTouOffer': False, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'1666', u'exitFeeCount': 0, u'timeDefinition': u'Local time', u'retailerImageUrl': u'img/retailers/big/pachydro.jpg'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.energythatcould.com.au', u'offerId': u'PAC33679SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1340, u'offerType': u'Standing offer', u'offerName': u'Vic Home Flex', u'conditionalPrice': 1340, u'fullDiscountedPrice': 1340, u'greenPower': 0, u'retailerName': u'Pacific Hydro Retail Pty Ltd', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType': u'Single rate', u'retailerPhone': u'1800 010 648', u'isPartDual': False, u'retailerId': u'15902', u'isTouOffer': False, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'1680', u'exitFeeCount': 0, u'timeDefinition': u'Local time', u'retailerImageUrl': u'img/retailers/big/pachydro.jpg'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 10, u'retailerUrl': u'www.commander.com', u'offerId': u'M2E30367MR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': True, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1370, u'offerType': u'Market offer', u'offerName': u'Citipower Commander Residential Market Offer (CE3CPR-MAT1 + PF1/TF1/GF1)', u'conditionalPrice': 1370, u'fullDiscountedPrice': 1160, u'greenPower': 0, u'retailerName': u'Commander Power & Gas', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': True, u'greenpowerChargeType': None, u'tariffType': u'Single rate', u'retailerPhone': u'13 39 14', u'isPartDual': False, u'retailerId': u'13667', u'isTouOffer': False, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'2384', u'exitFeeCount': 0, u'timeDefinition': u'AEST only', u'retailerImageUrl': u'img/retailers/big/commanderpowergas.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 10, u'retailerUrl': u'www.commander.com', u'offerId': u'M2E30359MR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': True, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1330, u'offerType': u'Market offer', u'offerName': u'Citipower Commander Residential Market Offer (Flexible Pricing (Peak, Shoulder and Off Peak) (CE3CPR-MCFP1 + PF1/TF1/GF1)', u'conditionalPrice': 1330, u'fullDiscountedPrice': 1140, u'greenPower': 0, u'retailerName': u'Commander Power & Gas', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': True, u'greenpowerChargeType': None, u'tariffType': u'Time of use', u'retailerPhone': u'13 39 14', u'isPartDual': False, u'retailerId': u'13667', u'isTouOffer': True, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'2386', u'exitFeeCount': 0, u'timeDefinition': u'AEST only', u'retailerImageUrl': u'img/retailers/big/commanderpowergas.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 10, u'retailerUrl': u'www.commander.com', u'offerId': u'M2E33241MR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': True, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1300, u'offerType': u'Market offer', u'offerName': u'Citipower Commander Residential Market Offer (Peak / Off Peak) (CE3CPR-MPK1OP1)', u'conditionalPrice': 1300, u'fullDiscountedPrice': 1100, u'greenPower': 0, u'retailerName': u'Commander Power & Gas', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': True, u'greenpowerChargeType': None, u'tariffType': u'Time of use', u'retailerPhone': u'13 39 14', u'isPartDual': False, u'retailerId': u'13667', u'isTouOffer': True, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'2389', u'exitFeeCount': 0, u'timeDefinition': u'AEST only', u'retailerImageUrl': u'img/retailers/big/commanderpowergas.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.commander.com', u'offerId': u'M2E30379SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1370, u'offerType': u'Standing offer', u'offerName': u'Citipower Commander Residential Standing Offer (CE3CPR-SAT1 + PF1/TF1/GF1)', u'conditionalPrice': 1370, u'fullDiscountedPrice': 1370, u'greenPower': 0, u'retailerName': u'Commander Power & Gas', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType': u'Single rate', u'retailerPhone': u'13 39 14', u'isPartDual': False, u'retailerId': u'13667', u'isTouOffer': False, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'2391', u'exitFeeCount': 0, u'timeDefinition': u'AEST only', u'retailerImageUrl': u'img/retailers/big/commanderpowergas.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.commander.com', u'offerId': u'M2E30369SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1330, u'offerType': u'Standing offer', u'offerName': u'Citipower Commander Residential Standing Offer (Flexible Pricing (Peak, Shoulder and Off Peak) (CE3CPR-SCFP1 + PF1/TF1/GF1)', u'conditionalPrice': 1330, u'fullDiscountedPrice': 1330, u'greenPower': 0, u'retailerName': u'Commander Power & Gas', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType': u'Time of use', u'retailerPhone': u'13 39 14', u'isPartDual': False, u'retailerId': u'13667', u'isTouOffer': True, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'2393', u'exitFeeCount': 0, u'timeDefinition': u'AEST only', u'retailerImageUrl': u'img/retailers/big/commanderpowergas.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.commander.com', u'offerId': u'M2E30375SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1300, u'offerType': u'Standing offer', u'offerName': u'Citipower Commander Residential Standing Offer (Peak / Off Peak) (CE3CPR-SPK1OP1)', u'conditionalPrice': 1300, u'fullDiscountedPrice': 1300, u'greenPower': 0, u'retailerName': u'Commander Power & Gas', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType': u'Time of use', u'retailerPhone': u'13 39 14', u'isPartDual': False, u'retailerId': u'13667', u'isTouOffer': True, u'solarType': None, u'estimatedSolarCredit': 0, u'offerKey': u'2395', u'exitFeeCount': 0, u'timeDefinition': u'AEST only', u'retailerImageUrl': u'img/retailers/big/commanderpowergas.png'}], u'isClosed': False, u'isChecked': False, u'offerFuelType': 0}, {u'offerDetails': [{u'coolingOffPeriod': 0, u'retailerUrl': u'www.dodo.com/powerandgas', u'offerId': u'DOD32903SR', u'contractLengthCount': 1, u'exitFee': [0], u'hasIncentive': False, u'tariffDetails': {}, u'greenpowerAmount': 0, u'isDirectDebitOnly': False, u'basePrice': 1320, u'offerType': u'Standing offer', u'offerName': u'Citipower Res No Term Standing Offer (Common Form Flex Plan) (E3CPR-SCFP1)', u'conditionalPrice': 1320, u'fullDiscountedPrice': 1320, u'greenPower': 0, u'retailerName': u'Dodo Power & Gas', u'intrinsicGreenpowerPercentage': u'0.0000', u'contractLength': [u'None'], u'hasPayOnTimeDiscount': False, u'greenpowerChargeType': None, u'tariffType':
Then if you look at the requests later, for example when you click the compare selected button on the results page, there is a request like:
https://compare.switchon.vic.gov.au/service/offer/tariff/9090/9092
So you may be able to mimic what happens by filtering using the tariff or some variation.
You can actually get all the data as json, if you enter the same values as below into the forms:
form1 = {"energy_category": "electricity",
"location": "home",
"location-home": "shift",
"distributor": "7",
"postcode": "3000",
"energy_concession": "0",
"solar": "0",
"disclaimer_chkbox": "disclaimer_selected"
}
form2 = {"person-count":"1",
"room-count":"1",
"refrigerator-count":"1",
"gas-type":"4",
"pool-heating":"0",
"spaceheating[]":"none",
"spacecooling[]":"none",
"cloth-dryer":"0",
"cloth-dryer-freq-weekday":"",
"waterheating[]":"other"}
import json
with requests.Session() as s:
s.post(sub_url, data=form1)
r = (s.get("https://compare.switchon.vic.gov.au/energy_questionnaire"))
s.post("https://compare.switchon.vic.gov.au/energy_questionnaire/submit",
data=form2)
js = s.get("https://compare.switchon.vic.gov.au/service/offers").json()["offersList"]
by_discount = sorted(js, key=lambda d: d["offerDetails"][0]["fullDiscountedPrice"])
If we just pull the first two values from the list ordered by the total discount price:
from pprint import pprint as pp
pp(by_discount[:2])
You will get:
[{u'isChecked': False,
u'isClosed': False,
u'offerDetails': [{u'basePrice': 980,
u'conditionalPrice': 980,
u'contractLength': [u'None'],
u'contractLengthCount': 1,
u'coolingOffPeriod': 10,
u'estimatedSolarCredit': 0,
u'exitFee': [0],
u'exitFeeCount': 1,
u'fullDiscountedPrice': 660,
u'greenPower': 0,
u'greenpowerAmount': 0,
u'greenpowerChargeType': None,
u'hasIncentive': False,
u'hasPayOnTimeDiscount': True,
u'intrinsicGreenpowerPercentage': u'0.0000',
u'isDirectDebitOnly': False,
u'isPartDual': False,
u'isTouOffer': False,
u'offerId': u'GLO40961MR',
u'offerKey': u'7636',
u'offerName': u'GLO SWITCH',
u'offerType': u'Market offer',
u'retailerId': u'31206',
u'retailerImageUrl': u'img/retailers/big/globird.jpg',
u'retailerName': u'GloBird Energy',
u'retailerPhone': u'(03) 8560 4199',
u'retailerUrl': u'http://www.globirdenergy.com.au/switchon/',
u'solarType': None,
u'tariffDetails': {},
u'tariffType': u'Single rate',
u'timeDefinition': u'Local time'}],
u'offerFuelType': 0},
{u'isChecked': False,
u'isClosed': False,
u'offerDetails': [{u'basePrice': 1080,
u'conditionalPrice': 1080,
u'contractLength': [u'None'],
u'contractLengthCount': 1,
u'coolingOffPeriod': 10,
u'estimatedSolarCredit': 0,
u'exitFee': [0],
u'exitFeeCount': 1,
u'fullDiscountedPrice': 720,
u'greenPower': 0,
u'greenpowerAmount': 0,
u'greenpowerChargeType': None,
u'hasIncentive': False,
u'hasPayOnTimeDiscount': True,
u'intrinsicGreenpowerPercentage': u'0.0000',
u'isDirectDebitOnly': False,
u'isPartDual': False,
u'isTouOffer': True,
u'offerId': u'GLO41009MR',
u'offerKey': u'7642',
u'offerName': u'GLO SWITCH',
u'offerType': u'Market offer',
u'retailerId': u'31206',
u'retailerImageUrl': u'img/retailers/big/globird.jpg',
u'retailerName': u'GloBird Energy',
u'retailerPhone': u'(03) 8560 4199',
u'retailerUrl': u'http://www.globirdenergy.com.au/switchon/',
u'solarType': None,
u'tariffDetails': {},
u'tariffType': u'Time of use',
u'timeDefinition': u'Local time'}],
u'offerFuelType': 0}]
Which should match what you see on the page when you click the "DISCOUNTED PRICE" filter button.
For the normal view it seems to be ordered by conditionalPrice or basePrice, again pulling just the two first values should match what you see on the webpage:
base = sorted(js, key=lambda d: d["offerDetails"][0]["conditionalPrice"])
from pprint import pprint as pp
pp(base[:2])
[{u'isChecked': False,
u'isClosed': False,
u'offerDetails': [{u'basePrice': 740,
u'conditionalPrice': 740,
u'contractLength': [u'None'],
u'contractLengthCount': 1,
u'coolingOffPeriod': 0,
u'estimatedSolarCredit': 0,
u'exitFee': [0],
u'exitFeeCount': 0,
u'fullDiscountedPrice': 740,
u'greenPower': 0,
u'greenpowerAmount': 0,
u'greenpowerChargeType': None,
u'hasIncentive': False,
u'hasPayOnTimeDiscount': False,
u'intrinsicGreenpowerPercentage': u'0.0000',
u'isDirectDebitOnly': False,
u'isPartDual': False,
u'isTouOffer': False,
u'offerId': u'NEX42694SR',
u'offerKey': u'9092',
u'offerName': u'Citpower Single Rate Residential',
u'offerType': u'Standing offer',
u'retailerId': u'35726',
u'retailerImageUrl': u'img/retailers/big/nextbusinessenergy.jpg',
u'retailerName': u'Next Business Energy Pty Ltd',
u'retailerPhone': u'1300 466 398',
u'retailerUrl': u'http://www.nextbusinessenergy.com.au/',
u'solarType': None,
u'tariffDetails': {},
u'tariffType': u'Single rate',
u'timeDefinition': u'Local time'}],
u'offerFuelType': 0},
{u'isChecked': False,
u'isClosed': False,
u'offerDetails': [{u'basePrice': 780,
u'conditionalPrice': 780,
u'contractLength': [u'None'],
u'contractLengthCount': 1,
u'coolingOffPeriod': 0,
u'estimatedSolarCredit': 0,
u'exitFee': [0],
u'exitFeeCount': 0,
u'fullDiscountedPrice': 780,
u'greenPower': 0,
u'greenpowerAmount': 0,
u'greenpowerChargeType': None,
u'hasIncentive': False,
u'hasPayOnTimeDiscount': False,
u'intrinsicGreenpowerPercentage': u'0.0000',
u'isDirectDebitOnly': False,
u'isPartDual': False,
u'isTouOffer': False,
u'offerId': u'NEX42699SR',
u'offerKey': u'9090',
u'offerName': u'Citpower Residential Flexible Pricing',
u'offerType': u'Standing offer',
u'retailerId': u'35726',
u'retailerImageUrl': u'img/retailers/big/nextbusinessenergy.jpg',
u'retailerName': u'Next Business Energy Pty Ltd',
u'retailerPhone': u'1300 466 398',
u'retailerUrl': u'http://www.nextbusinessenergy.com.au/',
u'solarType': None,
u'tariffDetails': {},
u'tariffType': u'Flexible Pricing',
u'timeDefinition': u'Local time'}],
u'offerFuelType': 0}]
You can see all the json returned in firebug console if you click the https://compare.switchon.vic.gov.au/service/offers get entry then hit response:
You should be able to pull each field that you want from that.
The output actually has a few extras results which you don't see on the page unless you toggle the tou button below:
You can filter those from the results so you exactly match the default output or give an option to include with a helper function:
def order_by(l, k, is_tou=False):
if not is_tou:
filt = filter(lambda x: not x["offerDetails"][0]["isTouOffer"], l)
return sorted(filt, key=lambda d: d["offerDetails"][0][k])
return sorted(l, key=lambda d: d["offerDetails"][0][k])
import json
with requests.Session() as s:
s.post(sub_url, data=form1)
r = (s.get("https://compare.switchon.vic.gov.au/energy_questionnaire"))
s.post("https://compare.switchon.vic.gov.au/energy_questionnaire/submit",
data=form2)
js = s.get("https://compare.switchon.vic.gov.au/service/offers").json()["offersList"]
by_price = by_discount(js, "conditionalPrice", False)
print(by_price[:3)
If you check the output you will see origin energy third with a price of 840 in the results with the switch on or 860 for AGL when it is off, you can apply the same to the discount output:
The regular output also seems to be ordered by conditionalPrice if you check the source the two js functions that get called for ordering are:
ng-click="changeSortingField('conditionalPrice')"
ng-click="changeSortingField('fullDiscountedPrice')"
So that should now definitely completely match the site output.
这篇关于多 POST 查询(会话模式)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
