通过Python中的下载按钮执行下载 [英] Perform Download via download button in Python

问题描述

在从网站获取数据方面，我是一个陌生的人.

例如网站

主要问题是如何在python中下载该文件?我尝试了一些在网络上找到的内容(例如，漂亮的汤，scraperwiki等)，但是以某种方式失败了.数据下载链接的结构如下:

 >Kurse als CSV-Datei</h3>< div class ="clearfloat"></div></div>>< form action ="/quote/historic/historic.csv" method ="get">name ="histcsv"><输入type ="hidden" name ="secu" value ="291"/><输入>type ="hidden" name ="boerse_id" value ="6"/>< input type ="hidden">name ="clean_split" value ="1"/>< input type ="hidden">name ="clean_payout" value ="1"/>< input type ="hidden">name ="clean_bezug" value ="1"/><输入type ="hidden" name ="currency">value ="EUR"/>< ul style ="margin:5px;">< li><标签>for ="minTime"> von:</label>< input id ="minTime" name ="min_time">value ="8.2.2016" style ="width:71px"/></li>< li><标签>for ="maxTime"> bis:</label>< input id ="maxTime" name ="max_time">value ="8.2.2017" style ="width:71px"/></li>< li><标签>for ="trenner"> Trennzeichen:</label>< input id ="trenner" name ="trenner">value =;"style ="width:25px"/></li>< li>< input class ="submitButton">name ="go" value ="Download" type ="submit"/></li></ul></form></div>></div>< div class ="clearfloat"></div></div></div></div>÷ div>id ="foot" class ="noprint">< div class ="adControllerAd evtAdShow>noprint abstand adHide"id =" iqadtile16></div>< div id =" footer>< div>class ="footer abstand">< a>href ="/adidas-aktie/historische_kurse?boerse_id = 6& currency = EUR& clean_split = 1& clean_payout = 1& clean_bezug = 1& min_time = 2014-09-01& max_time = 2017-02-07/wkn_A1EWWW_historic.csv">class ="anker">< img src ="/forum/i/up.gif" alt =" width ="9">height ="9"> Zum Seitenanfang</a>< a>href ="/fehlermeldung/index.m?ag = 291& amp; referrer =& amp; ssl = 0& amp; url =％2Fadidas-aktie％2Fhistorische_kurse％3Fboerse_id％3D6％26currency％3DEUR％26clean_split％3D1％26clean_payout％3D1％26clean_bezug％3D1％26min_time％3D2014-09-01％26max_time％3D2017-02-07％2Fwkn_A1EWWW_historic.csv 

解决方案

 导入请求url ='http://www.ariva.de/quote/historic/historic.csv?secu=291&boerse_id=6&clean_split=1&clean_payout=0&clean_bezug=1&min_time=8.2.2016&max_time=8.2.2017& trenner =％3B& go =下载'r = request.get(URL)与open('a.csv'，'wb')为f:f.write(r.content) 

您可以使用chrome dev工具监视网络，单击下载后，浏览器将使用 GET 方法将消息发送到服务器，并且可以使用 requests 模仿它.

如何在网址中找到参数:

您可以解析页面并获取所需的参数，然后构建下载URL并将其传递给pandas.

使用从链接读取的熊猫:

 将pandas导入为pdpd.read_csv('http://www.ariva.de/quote/historic/historic.csv?secu=291&boerse_id=6&clean_split=1&clean_payout=0&clean_bezug=1&min_time=8.2.2016&max_time= 8.2.2017& trenner =％3B& go =下载') 

如何获取参数:

 导入请求，bs4url ='http://www.ariva.de/adidas-aktie/historische_kurse'r = request.get(URL)汤= bs4.BeautifulSoup(r.text，'lxml')有效负载= {field ['name']:field ['value']用于汤中的字段.select('form [name ="histcsv"] input')}csv = requests.post('http://www.ariva.de/quote/historic/historic.csv'，data=有效载荷) 

I am somehow new in the region of getting data from a website.

I have, e.g. a website http://www.ariva.de/adidas-aktie/historische_kurse and there is a donwload button hidden as shown in the picture below in red:

The main question is how can I download that in python? I tried some stuff found on the web (e.g. like beautiful soup, scraperwiki etc.) but somehow failed. The data download link is structured as the following:

> Kurse als CSV-Datei       </h3> <div class="clearfloat"></div> </div>
> <form action="/quote/historic/historic.csv" method="get"
> name="histcsv"> <input type="hidden" name="secu" value="291" /> <input
> type="hidden" name="boerse_id" value="6" /> <input type="hidden"
> name="clean_split"  value="1" /> <input type="hidden"
> name="clean_payout" value="1" /> <input type="hidden"
> name="clean_bezug"  value="1" /> <input type="hidden" name="currency" 
> value="EUR" /> <ul style="margin:5px;"> <li> <label
> for="minTime">von:</label> <input id="minTime" name="min_time"
> value="8.2.2016" style="width:71px" /> </li> <li> <label
> for="maxTime">bis:</label> <input id="maxTime" name="max_time"
> value="8.2.2017" style="width:71px" /> </li> <li> <label
> for="trenner">Trennzeichen:</label> <input id="trenner" name="trenner"
> value=";" style="width:25px" /> </li> <li> <input class="submitButton"
> name="go" value="Download" type="submit" /> </li> </ul> </form> </div>
> </div> <div class="clearfloat"></div> </div> </div> </div> <div
> id="foot" class="noprint"> <div class="adControllerAd evtAdShow 
> noprint abstand adHide" id="iqadtile16"> </div> <div id="footer"> <div
> class="footer abstand"> <a
> href="/adidas-aktie/historische_kurse?boerse_id=6&currency=EUR&clean_split=1&clean_payout=1&clean_bezug=1&min_time=2014-09-01&max_time=2017-02-07/wkn_A1EWWW_historic.csv"
> class="anker"> <img src="/forum/i/up.gif" alt="" width="9"
> height="9">Zum Seitenanfang</a> <a
> href="/fehlermeldung/index.m?ag=291&amp;referrer=&amp;ssl=0&amp;url=%2Fadidas-aktie%2Fhistorische_kurse%3Fboerse_id%3D6%26currency%3DEUR%26clean_split%3D1%26clean_payout%3D1%26clean_bezug%3D1%26min_time%3D2014-09-01%26max_time%3D2017-02-07%2Fwkn_A1EWWW_historic.csv"

解决方案

import requests

url = 'http://www.ariva.de/quote/historic/historic.csv?secu=291&boerse_id=6&clean_split=1&clean_payout=0&clean_bezug=1&min_time=8.2.2016&max_time=8.2.2017&trenner=%3B&go=Download'
r = requests.get(url)
with open('a.csv', 'wb') as f:
    f.write(r.content)

you can monitor the network use chrome dev tools, and when you click the download, the browser use GET method sending message to server and you can mimic it use requests

how to find the parameters in the url:

you can parse the page and get get the parameter you need, then build the download url and pass it to pandas.

Use pandas read from link:

import pandas as pd
pd.read_csv('http://www.ariva.de/quote/historic/historic.csv?secu=291&boerse_id=6&clean_split=1&clean_payout=0&clean_bezug=1&min_time=8.2.2016&max_time=8.2.2017&trenner=%3B&go=Download')

How to get the parameter:

import requests, bs4

url = 'http://www.ariva.de/adidas-aktie/historische_kurse'
r = requests.get(url)
soup = bs4.BeautifulSoup(r.text, 'lxml')
payload = {field['name']:field['value'] for field in soup.select('form[name="histcsv"] input')}
csv = requests.post('http://www.ariva.de/quote/historic/historic.csv', data=payload)

这篇关于通过Python中的下载按钮执行下载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！