Python的 - 从（非.PDF）URL下载PDF [英] Python - Download pdf from (non .pdf) url
问题描述
我试图从已登录的网站下载大约20个左右的PDF文件。这是我迄今为止，但无法下载任何有效的PDF文件（即它们都损坏）。我也是新的Python。
I am trying to download around 20 or so pdfs from a site that has a login. This is what I have so far but it fails to download any valid pdfs (i.e. they are all corrupted). I am also new to python.
import mechanize
import urllib2
def download_file(download_url):
response = urllib2.urlopen(download_url)
print response.geturl()
print response.read()
file = open("document.pdf", 'wb')
file.write(response.read())
file.close()
brwser = mechanize.Browser()
brwser.addheaders = [('User-agent', 'Firefox')]
response = brwser.open(url)
brwser.select_form(nr = 0)
brwser.form['UserName'] = 'username'
brwser.form['Password'] = 'password'
nextpage = brwser.submit()
# Navigate to the page I want
for link in brwser.links():
if link.text == 'Some pdf':
request = brwser.follow_link(link)
download_file(link.url)
我不知道去尝试什么。对于PDF文件的URL都是这样
I am not sure what to try. The urls for the pdfs are like this
的https://example.com/something/source2.aspx?id=e9a9bfdc-7d97-e411-9e03-76439cf4d30e
另外，根据response.read（）如下：
Also the response.read() is as follows:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><title>
Source
</title>
<script type='text/javascript'>
window.onload = function () {
var url = window.location.href.replace('source.aspx?', 'source2.aspx?');
window.location = url;
};
</script>
</head>
<body>
<div style='position:fixed; height:100%; width:100%; overflow:hidden; top:100px; left:100px;'>Loading, please wait.</div>
</body>
</html>
那么，如何下载这些文件？
So how do I download these files?
推荐答案
您可能会考虑使用硒，这也许是更适合与网站互动（不是机械化是不是一个很好的工具）。对于如何做到这一点（例如<一个体面的文件href=\"http://yizeng.me/2014/05/23/download-pdf-files-automatically-in-firefox-using-selenium-webdriver/\"相对=nofollow>这里或这里）：在普遍接受的方法是调整火狐以便它保存文件，而不是试图打开它们，然后访问的每一个环节。
You might consider using Selenium, which is perhaps better suited to interact with the site (not that mechanize isn't an excellent tool). There is decent documentation on how to accomplish this (e.g. here or here): the generally accepted approach is to tweak Firefox so that it saves the files rather than attempting to open them, and then access every link.
您也可能会发现，当你解决的链接，你最终的地方完全不同，这取决于PDF文件是在哪里以及如何生成它们。你也可以在滚如
You may also find that when you resolve the links, you end up somewhere completely different depending on where the PDFs are and how they were generated. You could also roll in an approach like this one for link extraction.
这篇关于Python的 - 从（非.PDF）URL下载PDF的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
