无法获取硒(python)下载不具有链接但仅在单击下载按钮后出现的csv文件 [英] Unable to get selenium (python) to download a csv file which doesnt have a link but only appears after i click the download button
问题描述
单击所有年份"按钮后，我试图从此网站下载csv文件.
I am trying to download a csv file from this website after I click the All Years button.
https://www.macrotrends.net/1476/copper -prices-historical-chart-data
您可以看到单击所有年份"按钮的xpath是/html/body/div [1]/div [1]/div [3]/a [7]
as you can see the xpath to click the All Years button is /html/body/div[1]/div[1]/div[3]/a[7]
这是所有年份"按钮的html代码
this is the html code of the All Years button
<a class="zoom external-period-changer" data-period-label=" All ">All Years</a>
，单击下载历史数据"按钮的xpath是//* [@@ id ="dataDownload"]
and the xpath to click the Download Historical Data button is //*[@id="dataDownload"]
这是下载历史数据"按钮的html代码
here is the html code of the Download Historical Data button
<button id="dataDownload" class="chart_buttons btn btn-danger btn-xs"><span class="glyphicon glyphicon-cloud-download"></span> <strong>Download Historical Data</strong></button>
这是我的代码
import time
import requests
from bs4 import BeautifulSoup
import os
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.firefox.options import Options
start_time = time.time()
options = Options()
options.add_argument("--headless")
options.add_argument("--disable-gpu")
options.add_argument("--disable-extensions")
driver = webdriver.Firefox(executable_path=r"/home/geckodriver/geckodriver",options=options,)
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2) # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', '/home/Documents/testing/macrotrends')
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'text/csv')
commodity = '1476/copper-prices-historical-chart-data'
url = "https://www.macrotrends.net/"+ commodity
driver.get(url)
time.sleep(5)
driver.find_element_by_xpath('/html/body/div[1]/div[1]/div[3]/a[7]').click()
time.sleep(1.5)
driver.find_element_by_xpath('//*[@id="dataDownload"]').click()
time.sleep(1.5)
driver.close()
print("--- %s seconds ---" % (time.time() - start_time))
但是出现以下错误:
NoSuchElementException: Message: Unable to locate element: /html/body/div[1]/div[1]/div[3]/a[7]
首先，为什么我不能单击此按钮?当我可以清楚地检查元素并看到它在那里时.
First off, why cant i click this button? when i can clearly inspect the element and see that it's there.
此外，通常大多数网站都会出现下载链接，我可以使用请求来获取csv文件.但是由于某种原因，链接没有出现.
Also, usually a download link appears from most websites and i can just use requests to get the csv file. But for some reason the link doesnt appear.
是否有任何方法或更好的方法在python中使用硒下载此csv文件?
Is there any way or a better way to download this csv file using selenium in python?
所以现在我已经根据答案添加了此代码，并将代码更改为以下代码
So now i have added this according to the answers and changed the code to the following
start_time = time.time()
options = Options()
driver = webdriver.Firefox(executable_path=r"/home/geckodriver/geckodriver",options=options,)
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2) # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', '/home/Documents/testing/macrotrends')
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'text/csv')
driver.get('https://www.macrotrends.net/1476/copper-prices-historical-chart-data')
time.sleep(5)
iframe = driver.find_element_by_xpath("//iframe[@id='chart_iframe']")
driver.switch_to.frame(iframe)
xpath = "//a[text()='All Years']"
driver.find_element_by_xpath(xpath).click()
xpath = "//button[@id='dataDownload']"
driver.find_element_by_xpath(xpath).click()
time.sleep(10)
driver.close()
print("--- %s seconds ---" % (time.time() - start_time))
这次我能够找到元素，但是在无头模式下它不起作用.谢谢你的帮助
I am able to find the elements this time, but it just doesnt work in headless mode. thanks for the help
推荐答案
我看到您要单击的元素在iframe元素中. 点击之前，您必须先切换到iframe.
I see that the elements you are trying to click are in an iframe element. You have to switch to the iframe first before clicking.
driver.get('https://www.macrotrends.net/1476/copper-prices-historical-chart-data')
iframe = driver.find_element_by_xpath("//iframe[@id='chart_iframe']")
driver.switch_to.frame(iframe)
xpath = "//a[text()='All Years']"
driver.find_element_by_xpath(xpath).click()
xpath = "//button[@id='dataDownload']"
driver.find_element_by_xpath(xpath).click()
这篇关于无法获取硒(python)下载不具有链接但仅在单击下载按钮后出现的csv文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
