无法获取xpath以单击硒中的弹出窗口 [英] can't get xpath to click on pop up in selenium

问题描述

我正在尝试从SEC提取一些简单的CIK代码.如果您运行下面的代码，您将获得有关调查"的提示.如果您手动进行操作，则看不到它.它炸毁了我的代码.但是由于它在硒中，所以我无法使用chropath对其进行检查以使xpath单击"NO".而且我无法在普通浏览器中重新创建弹出窗口.我该怎么办?

I'm trying to extract a few simple CIK codes from the SEC. If you run the code below, you will get a pop about a "survey". if you do it manually, you won't see it. it bombs my code. but since it is in selenium, I can't inspect it with chropath to get the xpath to click on the "NO". And I can't recreate the pop up in a normal browser. What do i do?

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from random import randint

path ='C:\\Users\\Jason\\Google Drive\\python\\chromedriver.exe' 
ticker='alrm'
main='https://www.sec.gov/search/search.htm'
driver=webdriver.Chrome(path)

tickers=['AAL','AAN','AAOI','AAPL']
# starts the process
def get_CIK(ticker):
    driver.get(main)
    stock_code = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, "cik")))
    stock_code.click()
    stock_code.send_keys(ticker)


    driver.find_element_by_xpath("//input[@value='Find Companies']").click() # click on search buttom
    link = driver.find_element_by_xpath("/html[1]/body[1]/div[4]/div[1]/div[3]/span[1]/a[1]").get_attribute("href") # get link
    cik= link[link.index('CIK=')+4:link.index("owner")-1] # extract cik
    print cik

for i in tickers:
    get_CIK(i)

推荐答案

您偶尔会看到的是"Foresee popup" (通常会随机显示).

What you occasionally see is the "Foresee popup" which would generally show up randomly.

我可以想到5种通用方法:

There are 5 generic approaches I can think of:

• 设置特定的Cookie ，该Cookie将禁用预示弹出窗口假装"，而您已将其关闭.目前到底要设置哪个Cookie是一个悬而未决的问题.也有与此相关的主题:使用Python处理随机的ForeSee弹出窗口和硒

• 在与网站交互过程中一直检查弹出窗口" 的存在.弹出窗口不是硒意义上的经典警报"，而是只是一个具有以下HTML表示形式的叠加层" :

• set a specific cookie that would disable foresee popup "pretending" you've dismissed it already. Which cookie to set exactly is an open question at the moment. There is this relevant thread about it as well: Handle random ForeSee popup using Python and Selenium

• check for the presence of the "popup" all the way during the interaction with the web-site. The popup is not a classic "alert" in selenium sense and is just an "overlay" which has this HTML representation:

<div class="__acs " aria-labelledby="fsrHeading">
    <div class="acsModalBackdrop acsAbandonButton" data-isbackdrop="true">
        <div class="acsFocusFirst acsClassicInvite" tabindex="1"
             id="acsMainInvite" role="dialog" aria-labelledby="fsrHeading">
            <div class="acsClassicInner" role="document">
                <div class="acsLogoSpacer"><img
                    src="//gateway.foresee.com/sites/sec-gov/production/trigger/sitelogo.png"
                    class="acsSiteLogo" title="" alt=""> <img
                    src="https://static.foresee.com/logos/foresee/150_67.png"
                    class="acsNoDisplay" title="ForeSee" alt="ForeSee">
                    <div title="ForeSee" alt="ForeSee"
                        class="acsVendorLogoSVG"></div>
                        ... 

例如，您可以然后检查拒绝"按钮是否存在，如果存在，请单击该按钮:

You can then, for example, check for the presence of the "Decline" button and click it if it's present:

<a href="#" tabindex="2" class="acsInviteButton acsDeclineButton" title="" role="button"></a>

您还可以阻止"预见要加载的JS脚本，例如，使用浏览器mobproxy来阻止来自"foresee.com"的所有流量.或者，在类似的轨迹上-您可以使用广告拦截器来启动硒，该广告拦截器可以开箱即用地阻止预见"

you can also "block" foresee JS scripts to be loaded by, for example, using a browsermobproxy where you would block all traffic from "foresee.com". Or, on a similar trajectory - you can start selenium with an ad blocker which would/should block "foresee" from out-of-the-box

，或者，您可以覆盖"foresee"全局对象的showInvite()函数:

or, you can override the showInvite() function of the "foresee" global object:

driver.execute_script("window.FSR.showInvite = function () {};")

请注意，导航到新页面后，您每次都需要调用此脚本.

Note that you would need to call this script every time right after you navigate to a new page.

经过一些逆向工程后，我发现"foresee" JS对象具有此全局配置，其中包含许多有趣的信息，包括设备黑名单:

after a bit of a reverse engineering I found that "foresee" JS object has this global config which contains lots of interesting information including a device blacklist:

device_blacklist: ["HTC_Rezound", "blackberry"]

然后，您可以覆盖浏览器的用户代理，并假装来自某个黑莓设备:

You can then override the browser's user-agent and pretend to be coming from a, say, Blackberry device:

BLACKBERRY_UA = "Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.1.0.346 Mobile Safari/534.11+"

opts = Options()
opts.add_argument("user-agent={0}".format(BLACKBERRY_UA))

driver = webdriver.Chrome(chrome_options=opts)

第二个选项从技术上讲更具挑战性，而且更容易出错，并且会降低速度，因为您将不断检查此弹出窗口是否存在.好吧，至少直到您将其解雇为止.

The second option is though technically more challenging and more error-prone, and would slow things down as you would constantly check for this popup to be present. Well, at least until you dismiss it.

第四个选项很有希望，但我尚未对其进行全面测试.

The fourth option is quite promising but I have not fully tested it.

最后一个选项，不管听起来多么疯狂，实际上对我来说都是有效的.

The last option, regardless of how crazy it sounds, actually works for me.

这篇关于无法获取xpath以单击硒中的弹出窗口的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！