dryscrape 点击“加载更多按钮"； [英] dryscrape click "load more button"

问题描述

我一直在尝试抓取 www.ratemyprofessors.com，我需要单击加载更多"按钮来抓取我需要的所有数据.但是，我现在使用的代码不起作用

loadButton = session.at_xpath(path)loadButton.click()

路径绝对正确，因为 loadButton.text() 等于加载更多"，但是它给了我一个错误，基本上是由于元素重叠而无法点击".

有谁知道如何解决这个问题或解决方法?根据我一直在阅读的内容，我们还可以模拟 javascript 在网络选项卡中运行的功能.但是我在查找函数时遇到了一些麻烦，因为 onclick 不直接调用函数，而是

onclick="javascript:mtvn.btg.Controller.sendLinkEvent({ linkName:\'PROFMIDPANE:LoadMore\', linkType:\'o\' } );

顺便说一句，我使用的是python，搜索学校后，加载更多"按钮位于教授列表下方的左侧

我一直在阅读一些相关的帖子，但没有找到任何有用的内容

任何帮助将不胜感激！

因此，一旦您有了一个页面，您就可以使用 url http://www.ratemyprofessors.com/ShowRatings.jsp?tid=881718如下:

导入请求从 bs4 导入 BeautifulSoupparams = {"solrformat": "true","rows": "1000", # 将其设置为高数以始终获取所有行."q": "","defType": "edismax","qf": "teacherfullname_t^1000 autosuggest","bf": "pow(total_number_of_ratings_i,2.1)","sort": "total_number_of_ratings_i desc","站点名称": "rmp","fl": "pk_id 老师名字_t 老师姓_t total_number_of_ratings_i averageratingscore_rf schoolid_s"}url = "http://search.mtvnservices.com/typeahead/suggest/"查询 = '*:* AND schoolid_s:{id} AND Teacherdepartment_s:"{subject}"'使用 requests.Session() 作为 s:s.headers.update({"User-Agent": "Mozilla/5.0 (X11; Linux x86_64)"})汤 = BeautifulSoup(s.get("http://www.ratemyprofessors.com/ShowRatings.jsp?tid=881718").content)# 传递我们可以从页面解析的学校 ID.params["q"] = query.format(id=soup.select_one("[data-schoolid]")["data-schoolid"], subject="History")res = s.get(url, params=params)json_data = res.json()从 pprint 导入 pprint 作为 pppp(json_data["响应"]["文档"])

给我们:

[{u'averageratingscore_rf': 4.6,u'pk_id':1347824，u'schoolid_s': u'4873',u'teacherfirstname_t': u'JP',u'teacherlastname_t': u'Godwin',u'total_number_of_ratings_i': 88},{u'averageratingscore_rf':3.38，u'pk_id': 692471,u'schoolid_s': u'4873',u'teacherfirstname_t': u'James',u'teacherlastname_t': u'Page',u'total_number_of_ratings_i': 49},{u'averageratingscore_rf': 3.5,u'pk_id':555487，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Kevin',u'teacherlastname_t': u'Davis',u'total_number_of_ratings_i': 44},{u'averageratingscore_rf': 4.4,u'pk_id': 1289399,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Jane',u'teacherlastname_t': u'England',u'total_number_of_ratings_i': 33},{u'averageratingscore_rf':3.46，u'pk_id': 1230841,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Simone',u'teacherlastname_t': u'De Santiago Ramos',u'total_number_of_ratings_i': 24},{u'averageratingscore_rf':3.15，u'pk_id': 701257,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Jack',u'teacherlastname_t': u'Pyle',u'total_number_of_ratings_i': 23},{u'averageratingscore_rf':4.13，u'pk_id':1466455，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Chris',u'teacherlastname_t': u'Politz',u'total_number_of_ratings_i': 20},{u'averageratingscore_rf':4.67，u'pk_id': 1218949,u'schoolid_s': u'4873',u'teacherfirstname_t': u'James',u'teacherlastname_t': u'Hathcock',u'total_number_of_ratings_i': 18},{u'averageratingscore_rf':3.93，u'pk_id':1648329，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Joshua',u'teacherlastname_t': u'Montandon',u'total_number_of_ratings_i': 15},{u'averageratingscore_rf':2.79，u'pk_id':1543864，u'schoolid_s': u'4873',u'teacherfirstname_t': u'M',u'teacherlastname_t': u'Antle',u'total_number_of_ratings_i': 14},{u'averageratingscore_rf': 3.83,u'pk_id': 1096585,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Scotty',u'teacherlastname_t': u'Edler',u'total_number_of_ratings_i': 12},{u'averageratingscore_rf':3.92，u'pk_id': 1260089,u'schoolid_s': u'4873',u'teacherfirstname_t': u'James',u'teacherlastname_t': u'Reynolds',u'total_number_of_ratings_i': 12},{u'averageratingscore_rf': 4.42,u'pk_id': 1418409,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Steve',u'teacherlastname_t': u'Wolfrum',u'total_number_of_ratings_i': 12},{u'averageratingscore_rf':4.45，u'pk_id': 899881,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Karen',u'teacherlastname_t': u'Stewart',u'total_number_of_ratings_i': 11},{u'averageratingscore_rf': 3.2,u'pk_id': 592508,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Crystal',u'teacherlastname_t': u'Wright',u'total_number_of_ratings_i': 10},{u'averageratingscore_rf': 4.5,u'pk_id': 891457,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Lisa',u'teacherlastname_t': u'Morales',u'total_number_of_ratings_i': 10},{u'averageratingscore_rf':2.9，u'pk_id': 1329058,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Mark',u'teacherlastname_t': u'Thompson',u'total_number_of_ratings_i': 10},{u'averageratingscore_rf': 4.0,u'pk_id': 1339373,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Charles',u'teacherlastname_t': u'Williams',u'total_number_of_ratings_i': 10},{u'averageratingscore_rf': 4.5,u'pk_id': 1587880,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Noelle',u'teacherlastname_t': u'Depperschmidt',u'total_number_of_ratings_i': 10},{u'averageratingscore_rf': 4.39,u'pk_id': 1426470,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Adrien',u'teacherlastname_t': u'Ivan',u'total_number_of_ratings_i': 9},{u'averageratingscore_rf': 5.0,u'pk_id':1871677，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Kevin',u'teacherlastname_t': u'Eades',u'total_number_of_ratings_i': 9},{u'averageratingscore_rf':4.81，u'pk_id': 393151,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Sharon',u'teacherlastname_t': u'Romero',u'total_number_of_ratings_i': 8},{u'averageratingscore_rf': 3.69,u'pk_id': 1377603,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Joseph',u'teacherlastname_t': u'Ialenti',u'total_number_of_ratings_i': 8},{u'averageratingscore_rf': 3.43,u'pk_id':1752608，u'schoolid_s': u'4873',u'teacherfirstname_t': u'James',u'teacherlastname_t': u'Jones',u'total_number_of_ratings_i': 7},{u'averageratingscore_rf': 3.43,u'pk_id':1782369，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Sara',u'teacherlastname_t': u'Ruppel',u'total_number_of_ratings_i': 7},{u'averageratingscore_rf': 3.33,u'pk_id': 1096000,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Scott',u'teacherlastname_t': u'Harp',u'total_number_of_ratings_i': 6},{u'averageratingscore_rf':2.17，u'pk_id':2061535，u'schoolid_s': u'4873',u'teacherfirstname_t': u'David',u'teacherlastname_t': u'Powell',u'total_number_of_ratings_i': 6},{u'averageratingscore_rf': 4.1,u'pk_id': 556560,u'schoolid_s': u'4873',u'teacherfirstname_t': u'',u'teacherlastname_t': u'English',u'total_number_of_ratings_i': 5},{u'averageratingscore_rf': 3.9,u'pk_id':2032232，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Robin',u'teacherlastname_t': u'Jett',u'total_number_of_ratings_i': 5},{u'averageratingscore_rf': 3.3,u'pk_id': 1242893,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Dennis',u'teacherlastname_t': u'Spillman',u'total_number_of_ratings_i': 5},{u'averageratingscore_rf': 5.0,u'pk_id': 1209837,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Jared',u'teacherlastname_t': u'Sutton',u'total_number_of_ratings_i': 4},{u'averageratingscore_rf':3.38，u'pk_id': 1587886,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Arianna',u'teacherlastname_t': u'Warren',u'total_number_of_ratings_i': 4},{u'averageratingscore_rf': 4.4,u'pk_id': 1643053,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Kimberly',u'teacherlastname_t': u'Lacoco',u'total_number_of_ratings_i': 4},{u'averageratingscore_rf': 2.5,u'pk_id':1857299，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Kevin',u'teacherlastname_t': u'Pyle',u'total_number_of_ratings_i': 4},{u'averageratingscore_rf': 2.33,u'pk_id': 892723,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Keith',u'teacherlastname_t': u'Mitchener',u'total_number_of_ratings_i': 3},{u'averageratingscore_rf': 3.5,u'pk_id': 1448008,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Sally',u'teacherlastname_t': u'Stratso',u'total_number_of_ratings_i': 3},{u'averageratingscore_rf':3.25，u'pk_id': 680381,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Todd',u'teacherlastname_t': u'Venable',u'total_number_of_ratings_i': 2},{u'averageratingscore_rf': 5.0,u'pk_id': 1256069,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Amanda',u'teacherlastname_t': u'Campbell-Wyatt',u'total_number_of_ratings_i': 2},{u'averageratingscore_rf': 5.0,u'pk_id':2142326，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Jeremy',u'teacherlastname_t': u'Godwin',u'total_number_of_ratings_i': 2},{u'averageratingscore_rf': 1.5,u'pk_id': 697421,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Woody',u'teacherlastname_t': u'Paige',u'total_number_of_ratings_i': 1},{u'averageratingscore_rf': 1.0,u'pk_id': 881718,u'schoolid_s': u'4873',u'teacherfirstname_t': u'M',u'teacherlastname_t': u'Sullivan',u'total_number_of_ratings_i': 1},{u'averageratingscore_rf': 1.5,u'pk_id': 1607181,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Nancy',u'teacherlastname_t': u'Coffelt',u'total_number_of_ratings_i': 1},{u'averageratingscore_rf': 5.0,u'pk_id': 1710114,u'schoolid_s': u'4873',u'teacherfirstname_t': u'Jason',u'teacherlastname_t': u'Scheller',u'total_number_of_ratings_i': 1},{u'averageratingscore_rf': 4.0,u'pk_id':2164391，u'schoolid_s': u'4873',u'teacherfirstname_t': u'James',u'teacherlastname_t': u'Paige',u'total_number_of_ratings_i': 1},{u'pk_id':2083511，u'schoolid_s': u'4873',u'teacherfirstname_t': u'Stephen',u'teacherlastname_t': u'Wolfrum',u'total_number_of_ratings_i': 0}]

您只需将学校 ID 和主题传递给查询字符串，您就可以得到任何您喜欢的内容.

I've been trying to scrape www.ratemyprofessors.com and I need to click the "load more" button to scrape the all the data that i need. However, the code that im using right now isn't working

loadButton = session.at_xpath(path)
    loadButton.click()

The path is definitely correct since loadButton.text() equals "load more", however it gives me an error saying basically "failed to click because of overlapping element".

Does anyone know how to fix this or a workaround? from what i've been reading we can also simulate the function that javascript is running in the network tab. However i have some trouble finding the function since onclick doesn't directly call a function, but instead

onclick="javascript:mtvn.btg.Controller.sendLinkEvent({ linkName:\'PROFMIDPANE:LoadMore\', linkType:\'o\' } );

btw I am using python and the "load more" button is located on the left side under the list of professors after you perform a search for school

I've been reading some relevant posts but haven't come cross anything useful

Any help would be appreciated!

my network/params tab

解决方案

You can do it all using requests and bs4, when you click the load more button a request is made:

So once you have a page, you can get all the teachers and ratings in json format using the url http://www.ratemyprofessors.com/ShowRatings.jsp?tid=881718 below:

import requests
from bs4 import BeautifulSoup


params = {"solrformat": "true",
          "rows": "1000", # set it high number to always get all rows.
          "q": "",
          "defType": "edismax",
          "qf": "teacherfullname_t^1000 autosuggest",
          "bf": "pow(total_number_of_ratings_i,2.1)",
          "sort": "total_number_of_ratings_i desc",
          "siteName": "rmp",
          "fl": "pk_id teacherfirstname_t teacherlastname_t total_number_of_ratings_i averageratingscore_rf schoolid_s"}

url = "http://search.mtvnservices.com/typeahead/suggest/"
query = '*:* AND schoolid_s:{id} AND teacherdepartment_s:"{subject}"'
with requests.Session() as s:
    s.headers.update({"User-Agent": "Mozilla/5.0 (X11; Linux x86_64)"})
    soup = BeautifulSoup(s.get("http://www.ratemyprofessors.com/ShowRatings.jsp?tid=881718").content)
    # pass the school id which we can parse from the page.
    params["q"] = query.format(id=soup.select_one("[data-schoolid]")["data-schoolid"], subject="History")
    res = s.get(url, params=params)

    json_data = res.json()


from pprint import pprint as pp
pp(json_data["response"]["docs"])

Gives us:

[{u'averageratingscore_rf': 4.6,
  u'pk_id': 1347824,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'JP',
  u'teacherlastname_t': u'Godwin',
  u'total_number_of_ratings_i': 88},
 {u'averageratingscore_rf': 3.38,
  u'pk_id': 692471,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'James',
  u'teacherlastname_t': u'Page',
  u'total_number_of_ratings_i': 49},
 {u'averageratingscore_rf': 3.5,
  u'pk_id': 555487,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Kevin',
  u'teacherlastname_t': u'Davis',
  u'total_number_of_ratings_i': 44},
 {u'averageratingscore_rf': 4.4,
  u'pk_id': 1289399,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Jane',
  u'teacherlastname_t': u'England',
  u'total_number_of_ratings_i': 33},
 {u'averageratingscore_rf': 3.46,
  u'pk_id': 1230841,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Simone',
  u'teacherlastname_t': u'De Santiago Ramos',
  u'total_number_of_ratings_i': 24},
 {u'averageratingscore_rf': 3.15,
  u'pk_id': 701257,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Jack',
  u'teacherlastname_t': u'Pyle',
  u'total_number_of_ratings_i': 23},
 {u'averageratingscore_rf': 4.13,
  u'pk_id': 1466455,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Chris',
  u'teacherlastname_t': u'Politz',
  u'total_number_of_ratings_i': 20},
 {u'averageratingscore_rf': 4.67,
  u'pk_id': 1218949,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'James',
  u'teacherlastname_t': u'Hathcock',
  u'total_number_of_ratings_i': 18},
 {u'averageratingscore_rf': 3.93,
  u'pk_id': 1648329,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Joshua',
  u'teacherlastname_t': u'Montandon',
  u'total_number_of_ratings_i': 15},
 {u'averageratingscore_rf': 2.79,
  u'pk_id': 1543864,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'M',
  u'teacherlastname_t': u'Antle',
  u'total_number_of_ratings_i': 14},
 {u'averageratingscore_rf': 3.83,
  u'pk_id': 1096585,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Scotty',
  u'teacherlastname_t': u'Edler',
  u'total_number_of_ratings_i': 12},
 {u'averageratingscore_rf': 3.92,
  u'pk_id': 1260089,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'James',
  u'teacherlastname_t': u'Reynolds',
  u'total_number_of_ratings_i': 12},
 {u'averageratingscore_rf': 4.42,
  u'pk_id': 1418409,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Steve',
  u'teacherlastname_t': u'Wolfrum',
  u'total_number_of_ratings_i': 12},
 {u'averageratingscore_rf': 4.45,
  u'pk_id': 899881,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Karen',
  u'teacherlastname_t': u'Stewart',
  u'total_number_of_ratings_i': 11},
 {u'averageratingscore_rf': 3.2,
  u'pk_id': 592508,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Crystal',
  u'teacherlastname_t': u'Wright',
  u'total_number_of_ratings_i': 10},
 {u'averageratingscore_rf': 4.5,
  u'pk_id': 891457,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Lisa',
  u'teacherlastname_t': u'Morales',
  u'total_number_of_ratings_i': 10},
 {u'averageratingscore_rf': 2.9,
  u'pk_id': 1329058,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Mark',
  u'teacherlastname_t': u'Thompson',
  u'total_number_of_ratings_i': 10},
 {u'averageratingscore_rf': 4.0,
  u'pk_id': 1339373,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Charles',
  u'teacherlastname_t': u'Williams',
  u'total_number_of_ratings_i': 10},
 {u'averageratingscore_rf': 4.5,
  u'pk_id': 1587880,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Noelle',
  u'teacherlastname_t': u'Depperschmidt',
  u'total_number_of_ratings_i': 10},
 {u'averageratingscore_rf': 4.39,
  u'pk_id': 1426470,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Adrien',
  u'teacherlastname_t': u'Ivan',
  u'total_number_of_ratings_i': 9},
 {u'averageratingscore_rf': 5.0,
  u'pk_id': 1871677,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Kevin',
  u'teacherlastname_t': u'Eades',
  u'total_number_of_ratings_i': 9},
 {u'averageratingscore_rf': 4.81,
  u'pk_id': 393151,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Sharon',
  u'teacherlastname_t': u'Romero',
  u'total_number_of_ratings_i': 8},
 {u'averageratingscore_rf': 3.69,
  u'pk_id': 1377603,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Joseph',
  u'teacherlastname_t': u'Ialenti',
  u'total_number_of_ratings_i': 8},
 {u'averageratingscore_rf': 3.43,
  u'pk_id': 1752608,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'James',
  u'teacherlastname_t': u'Jones',
  u'total_number_of_ratings_i': 7},
 {u'averageratingscore_rf': 3.43,
  u'pk_id': 1782369,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Sara',
  u'teacherlastname_t': u'Ruppel',
  u'total_number_of_ratings_i': 7},
 {u'averageratingscore_rf': 3.33,
  u'pk_id': 1096000,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Scott',
  u'teacherlastname_t': u'Harp',
  u'total_number_of_ratings_i': 6},
 {u'averageratingscore_rf': 2.17,
  u'pk_id': 2061535,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'David',
  u'teacherlastname_t': u'Powell',
  u'total_number_of_ratings_i': 6},
 {u'averageratingscore_rf': 4.1,
  u'pk_id': 556560,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'',
  u'teacherlastname_t': u'English',
  u'total_number_of_ratings_i': 5},
 {u'averageratingscore_rf': 3.9,
  u'pk_id': 2032232,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Robin',
  u'teacherlastname_t': u'Jett',
  u'total_number_of_ratings_i': 5},
 {u'averageratingscore_rf': 3.3,
  u'pk_id': 1242893,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Dennis',
  u'teacherlastname_t': u'Spillman',
  u'total_number_of_ratings_i': 5},
 {u'averageratingscore_rf': 5.0,
  u'pk_id': 1209837,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Jared',
  u'teacherlastname_t': u'Sutton',
  u'total_number_of_ratings_i': 4},
 {u'averageratingscore_rf': 3.38,
  u'pk_id': 1587886,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Arianna',
  u'teacherlastname_t': u'Warren',
  u'total_number_of_ratings_i': 4},
 {u'averageratingscore_rf': 4.4,
  u'pk_id': 1643053,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Kimberly',
  u'teacherlastname_t': u'Lacoco',
  u'total_number_of_ratings_i': 4},
 {u'averageratingscore_rf': 2.5,
  u'pk_id': 1857299,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Kevin',
  u'teacherlastname_t': u'Pyle',
  u'total_number_of_ratings_i': 4},
 {u'averageratingscore_rf': 2.33,
  u'pk_id': 892723,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Keith',
  u'teacherlastname_t': u'Mitchener',
  u'total_number_of_ratings_i': 3},
 {u'averageratingscore_rf': 3.5,
  u'pk_id': 1448008,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Sally',
  u'teacherlastname_t': u'Stratso',
  u'total_number_of_ratings_i': 3},
 {u'averageratingscore_rf': 3.25,
  u'pk_id': 680381,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Todd',
  u'teacherlastname_t': u'Venable',
  u'total_number_of_ratings_i': 2},
 {u'averageratingscore_rf': 5.0,
  u'pk_id': 1256069,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Amanda',
  u'teacherlastname_t': u'Campbell-Wyatt',
  u'total_number_of_ratings_i': 2},
 {u'averageratingscore_rf': 5.0,
  u'pk_id': 2142326,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Jeremy',
  u'teacherlastname_t': u'Godwin',
  u'total_number_of_ratings_i': 2},
 {u'averageratingscore_rf': 1.5,
  u'pk_id': 697421,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Woody',
  u'teacherlastname_t': u'Paige',
  u'total_number_of_ratings_i': 1},
 {u'averageratingscore_rf': 1.0,
  u'pk_id': 881718,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'M',
  u'teacherlastname_t': u'Sullivan',
  u'total_number_of_ratings_i': 1},
 {u'averageratingscore_rf': 1.5,
  u'pk_id': 1607181,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Nancy',
  u'teacherlastname_t': u'Coffelt',
  u'total_number_of_ratings_i': 1},
 {u'averageratingscore_rf': 5.0,
  u'pk_id': 1710114,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Jason',
  u'teacherlastname_t': u'Scheller',
  u'total_number_of_ratings_i': 1},
 {u'averageratingscore_rf': 4.0,
  u'pk_id': 2164391,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'James',
  u'teacherlastname_t': u'Paige',
  u'total_number_of_ratings_i': 1},
 {u'pk_id': 2083511,
  u'schoolid_s': u'4873',
  u'teacherfirstname_t': u'Stephen ',
  u'teacherlastname_t': u'Wolfrum',
  u'total_number_of_ratings_i': 0}]

All you need to so is pass the school id and the subject to the query string and you can get whatever you like.

这篇关于dryscrape 点击“加载更多按钮"；的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！