Youtube Webscraping 评论由新 [英] Youtube Webscraping Comments by new

问题描述

我正在尝试从网络上抓取 youtube 评论部分.但在此之前，我想按 new 对评论进行排序.所以我必须点击排序依据，然后点击最新.但不幸的是，我没有运气.感谢您的帮助.

I'm trying to web scrape a youtube comment section. But before that, I want to sort the comments by new. So I have to click on the SORT BY and then Newest First. But unfortunately, I have no luck. Thank you for helping out.

屏幕记录:https://imgur.com/Rt8gnIB

代码:

import sys, unittest, time, datetime
import urllib.request, urllib.error, urllib.parse
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import InvalidArgumentException
from selenium.webdriver import Chrome
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.select import Select
from selenium.webdriver import ActionChains
from bs4 import BeautifulSoup
import requests
import requests.exceptions
from urllib.parse import urlsplit
from collections import deque
import re
import os
import shutil
import smtplib
import string
import pyautogui

options = webdriver.ChromeOptions()
options.add_argument('--lang=en')
options.add_argument("--start-maximized")

driver = webdriver.Chrome(executable_path=r'C:\Users\caspe\OneDrive\Documents\Övrigt\Kodning\Email\chromeDriver\chromedriver.exe', chrome_options=options)
driver.get("https://www.youtube.com/watch?v=EV6PLN_8RBw")
time.sleep(5)

screenWidth, screenHeight = pyautogui.size()
currentMouseX, currentMouseY = pyautogui.position()

pyautogui.moveTo(1050, 780) # Move the mouse to XY coordinates.
pyautogui.click()
print("clicked")
time.sleep(1)

pyautogui.moveTo(1250, 880) # Move the mouse to XY coordinates.
pyautogui.click() 
print("clicked")
time.sleep(1)

pyautogui.moveTo(700, 500) # Move the mouse to XY coordinates.
pyautogui.click() 
print("clicked")
time.sleep(1)

推荐答案

我强烈建议将 autogui 部分替换为 firstlyelement = driver.find_element_by_xpath 或通过 css_selector 和 element.click然后从最新"中选择 xpath/css_selector选项并再次单击，我在另一个脚本中执行了该操作并且没有问题，因为我在 android 上无法正确格式化

I would highly recommend to replace the autogui part with firstly element = driver.find_element_by_xpath or by css_selector and element.click And after you choose the xpath/css_selector from the "Newest" option and click again, i did that in another script and had no problems, i cant format it right since im on android tho

tl;dr:通过单击 selenium 替换 autogui，首先是使选项可选的按钮，然后是您想要对其进行排序的选项

tl;dr: replace the autogui by clicking with selenium, firstly the button that makes the options chooseable and then again on the option you want to sort it for

好的，我自己试过了，pyautogui 对我不起作用，不确定是因为我使用了另一个驱动程序还是什么，但如果它对你有用，那很好.问题是，您需要先向下滚动一下，才能加载评论.你可以使用

okay i tried it out myself, the pyautogui did not work for me, not sure if it is because of me using another driver or something, but if it works for you thats fine. The problem is, that you need to scroll down a bit first, in order to load the comments. you can do that using

for i in range(5):
driver.execute_script("arguments[0].scrollBy(0, 500)", element)
time.sleep(2)

不确定 5 是否足够/太多，因为我无法检查自己，但这基本上是如何工作的.然后您可以使用

not sure if 5 is enough/too much, since i cant check myself, but thats basically how it would work. you can then locate the dropdown open button using

    element = driver.find_element_by_xpath("//*[@id="label"]")
    element.click
    #now we select the option from the dropdown options
    element = driver.find_element_by_xpath("/html/body/ytd-app/div/ytd-page-manager/ytd-watch-flexy/div[4]/div[1]/div/ytd-comments/ytd-item-section-renderer/div[1]/ytd-comments-header-renderer/div[1]/span/yt-sort-filter-sub-menu-renderer/yt-dropdown-menu/paper-menu-button/iron-dropdown/div/div/paper-listbox/a[2]/paper-item")  
    element.click

你也可以使用explicitWaits来确保它在视野中，但我觉得使用它并不舒服，你发现它们这里
试试看，让我知道它是否有效:D

you could also use explicitWaits to ensure it is in view, but i dont feel confortable using that yet, you find them here
try it out and let me know if it worked :D

这篇关于Youtube Webscraping 评论由新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！