单击 nseindia.com 上的每月结算统计数据的“获取数据"按钮不会使用 Selenium 和 Python 获取结果 [英] Clicking on Get Data button for Monthly Settlement Statistics on nseindia.com doesn't fetch results using Selenium and Python

查看:17
本文介绍了单击 nseindia.com 上的每月结算统计数据的“获取数据"按钮不会使用 Selenium 和 Python 获取结果的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在尝试从

解决方案

我对你的代码做了一些调整并运行如下测试:

  • 代码块:

    from selenium import webdriverfrom selenium.webdriver.common.by import By从 selenium.webdriver.support.ui 导入 WebDriverWait从 selenium.webdriver.support 导入 expected_conditions 作为 EC选项 = webdriver.ChromeOptions()options.add_argument("开始最大化")options.add_experimental_option("excludeSwitches", ["enable-automation"])options.add_experimental_option('useAutomationExtension', False)driver = webdriver.Chrome(options=options, executable_path=r'C:UtilityBrowserDriverschromedriver.exe')driver.get('https://www1.nseindia.com/products/content/equities/equities/eq_monthly_statistics.htm')Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#h_filetype")))).select_by_visible_text("资本市场")Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#yearField")))).select_by_visible_text("2019-2020")WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.getdata-button#get[type='image'][src^='/common/images/btn-get-data.gif']))).点​​击()

<小时>

观察

与您的观察类似,我遇到了同样的障碍,但没有结果如下:

<小时>

深入探讨

看起来像获取数据那样在带有文本的元素上click()确实发生了.但是在检查网页的 驱动 WebDriver 实例并且响应被阻止.

<小时>

参考文献

几个文件:

<小时>

tl;博士

一些相关的讨论:

I am trying to scrape data from here.

By clicking on the capital market and 2019-20 year. I want to click on Get data.

I have used following code:

driver = webdriver.Chrome(executable_path=chrome_path,options=chrome_options)

driver.get( nse_cash_keystats_page )


 driver.find_element_by_xpath( "//select[@id='h_filetype']/option[text()='Capital Market ']" ).click()

driver.find_element_by_xpath( "//select[@id='yearField']/option[text()='2019-2020']" ).click()

     downloadButton=WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,'//input[@type="image"][@src="/common/images/btn-get-data.gif"]')))

driver.execute_script("arguments[0].click();", downloadButton)

By using the above code, I am able to click on Get DATA. But it is not showing output.

Please help me.Thanks in advance.

解决方案

I took your code added a few tweaks and ran the test as follows:

  • Code Block:

    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    
    options = webdriver.ChromeOptions() 
    options.add_argument("start-maximized")
    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    options.add_experimental_option('useAutomationExtension', False)
    driver = webdriver.Chrome(options=options, executable_path=r'C:UtilityBrowserDriverschromedriver.exe')
    driver.get('https://www1.nseindia.com/products/content/equities/equities/eq_monthly_statistics.htm')
    Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#h_filetype")))).select_by_visible_text("Capital Market ")
    Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#yearField")))).select_by_visible_text("2019-2020")
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.getdata-button#get[type='image'][src^='/common/images/btn-get-data.gif']"))).click()
    


Observation

Similar to your observation, I have hit the same roadblock with no results as follows:


Deep Dive

It seems the click() on the element with text as Get Data does happens. But while inspecting the DOM Tree of the webpage you will find that some of the <script> tag refers to JavaScripts having keyword akam. As an example:

  • <script type="text/javascript" src="https://www1.nseindia.com/akam/11/52349752" defer=""></script>
  • <noscript><img src="https://www1.nseindia.com/akam/11/pixel_52349752?a=dD01ZDZiMTA5OGQ0MDljYTYxN2RjMjc3MzBlN2YwMDQ0NjlkZDNiNTMzJmpzPW9mZg==" style="visibility: hidden; position: absolute; left: -999px; top: -999px;" /></noscript>

Which is a clear indication that the website is protected by Bot Manager an advanced bot detection service provided by Akamai and the response gets blocked.


Bot Manager

As per the article Bot Manager - Foundations:


Conclusion

So it can be concluded that the request for the data is detected as being performed by Selenium driven WebDriver instance and the response is blocked.


References

A couple of documentations:


tl; dr

A couple of relevant discussions:

这篇关于单击 nseindia.com 上的每月结算统计数据的“获取数据"按钮不会使用 Selenium 和 Python 获取结果的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆