从url下载csv并使其成为数据框python pandas [英] Download a csv from url and make it a dataframe python pandas

问题描述

我是python的新手，因此在这里需要一些帮助.我有一个带有url列的数据框，该列带有一个链接，该链接使我可以为每个链接下载CSV.我的目的是创建一个循环/无论如何工作，以便我可以运行一个命令，该命令将允许我下载，读取csv并为每一行创建一个数据框.任何帮助，将不胜感激.我已在下面附加了数据框的一部分.如果该链接不起作用(您可能无法将其替换为来自'

I am new to python so need a little help here. I have a dataframe with a url column with a link that allows me to download a CSV for each link. My aim is to create a loop/ whatever works so that I can run one command that will allow me to download,read the csv and create a dataframe for each of the rows. Any help would be appreciated. I have attached part of the dataframe below. If the link doesn't work (it probably won't you can just replace it with a link from 'https://finance.yahoo.com/quote/GOOG/history?p=GOOG' (any other company too) and navigate to download csv and use that link.

数据框:

Symbol         Link
YI             https://query1.finance.yahoo.com/v7/finance/download/YI?period1=1383609600&period2=1541376000&interval=1d&events=history&crumb=PMHbxK/sU6E
PIH            https://query1.finance.yahoo.com/v7/finance/download/PIH?period1=1383609600&period2=1541376000&interval=1d&events=history&crumb=PMHbxK/sU6E
TURN           https://query1.finance.yahoo.com/v7/finance/download/TURN?period1=1383609600&period2=1541376000&interval=1d&events=history&crumb=PMHbxK/sU6E
FLWS           https://query1.finance.yahoo.com/v7/finance/download/FLWS?period1=1383609600&period2=1541376000&interval=1d&events=history&crumb=PMHbxK/sU6E

再次感谢.

推荐答案

您需要发布请求，并将内容发送到io.

You need a post request and the get the contents to io.

import pandas as pd
import requests
import io

url = 'https://query1.finance.yahoo.com/v7/finance/download/GOOG'
params ={'period1':1538761929,
         'period2':1541443929,
         'interval':'1d',
         'events':'history',
         'crumb':'v4z6ZpmoP98',
        }

r = requests.post(url,data=params)
if r.ok:
    data = r.content.decode('utf8')
    df = pd.read_csv(io.StringIO(data))

要获取参数，我只是跟随了喜欢的人，并在?"之后复制了所有内容.检查它们是否匹配;)

To get the params, I just followed the liked and copied everything after ‘?’. Check that they match ;)

结果:

更新:

如果您可以直接在url中看到原始的csv内容，只需在pd.read_csv中传递url 直接从网址获取示例数据:

If you can see the raw csv contents directly in url, just pass the url in pd.read_csv Example data directly from url:

data_url ='https://raw.githubusercontent.com/pandas-dev/pandas/master/pandas/tests/data/iris.csv'

df = pd.read_csv(data_url)

这篇关于从url下载csv并使其成为数据框python pandas的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！