driver.get(<some url>)
for element in driver.find_elements_by_class_name('thumbnail'):
element.find_element_by_xpath(".//a").click() #this works and navigates to new page
element.find_element_by_link_text('Click here').click() #this doesn't
Which needs to navigate the following HTML (simplified of course) by clicking on a thumbnail, which directs to a new page, and then needs to click the Click here link in that new page:
<!DOCTYPE html>
<html lang="en-US" prefix="og: http://iuytp.me/ns# fb: http://iuytp.me/ns/fb#">
<head>
<meta charset="UTF-8" />
<title>Releases</title>
</head>
<body class="archive category category-releases category-4 custom-background">
<div id="main">
<div id="container" class="one-column">
<div id="content" role="main">
<h1 class="page-title">Releases</h1>
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7158 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-designer category-releases category-uncategorized">
<div class="thumbnail"><a href="http://records.net/uncategorized/designer-7-inch-bufu-records-co-release/" title="Permanent link to Designer – 7 inch" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/05/dboypledge32-300x300.png" class="attachment-thumbnail wp-post-image" alt="dboypledge3" /></a></div>
<h2><a href="http://records.net/uncategorized/designer-7-inch-bufu-records-co-release/" title="Permanent link to Designer – 7 inch" rel="bookmark">Designer – 7 inch</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7107 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/people-2014-tour-demos/" title="Permanent link to All My People – 2014 Tour Demos" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/04/01_Doubt-mp3-image-300x300.png" class="attachment-thumbnail wp-post-image" alt="" /></a></div>
<h2><a href="http://records.net/releases/people-2014-tour-demos/" title="Permanent link to All My People – 2014 Tour Demos" rel="bookmark">All My People – 2014 Tour Demos</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7089 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/sirens-blossom-talk/" title="Permanent link to Syrins – Boss Talk" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/04/sirens_final_smaller-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="sirens_final_smaller" /></a></div>
<h2><a href="http://records.net/releases/sirens-blossom-talk/" title="Permanent link to Syrins – Boss Talk" rel="bookmark">Syrins – Boss Talk</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7073 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/worlds-strongest-man-scares/" title="Permanent link to World’s Tough Man – Sorry Scares You" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/03/a2312749950_10-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="a2312749950_10" /></a></div>
<h2><a href="http://records.net/releases/worlds-strongest-man-scares/" title="Permanent link to World’s Tough Man – Sorry Scares You" rel="bookmark">World’s Tough Man – Sorry Scares You</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7046 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/sundog-space-criminal/" title="Permanent link to Dog – Space Criminal" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/03/Sundog_cover_high_res-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="dog_cover_high_res" /></a></div>
<h2><a href="http://records.net/releases/sundog-space-criminal/" title="Permanent link to Dog – Space Criminal" rel="bookmark">Dog – Space Criminal</a></h2>
</div>
</div><!--end thumbnail post display-->
<div style="clear:both"></div>
</div><!-- #container -->
</div><!-- #main -->
</div><!-- #wrapper -->
</div><!--#bg-wrapper-->
</body>
</html>
My code however spits out the following error:
Traceback (most recent call last):
...
File "crawler.py", line 17, in main
driver.find_element_by_link_text('Click here').click()
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 254, in find_element_by_link_text
return self.find_element(by=By.LINK_TEXT, value=link_text)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: u'no such element\n (Session info: chrome=35.0.1916.153)\n (Driver info: chromedriver=2.10.267517,platform=Mac OS X 10.9.3 x86_64)'
The problem appears to be that the element isn't being updated with the contents of the new page. Replacing the problematic line's element with driver doesn't work either. What am I doing wrong?
Please note that I must be able to do this for all thumbnails (hence the for loop).
解决方案
Turns out you need to store the links you want to navigate to in advance. This is what ended up working for me (found this thread to be helpful):
driver.get(<some url>)
elements = driver.find_elements_by_xpath("//h2/a")
links = []
for i in range(len(elements)):
links.append(elements[i].get_attribute('href'))
for link in links:
print 'navigating to: ' + link
driver.get(link)
# do stuff within that page here...
driver.back()
