从github文件夹导入多个csv文件-Python-COVID-19 [英] import multiple csv files from github folder - Python - COVID-19

问题描述

我想在这里做两件事:

1. 导入所有.csv文件，并将它们添加到df中.
2. 使用已上传的最新文件更新df.

我已经能够使用以下命令导入一个.csv:

I have been able to import one .csv with:

import pandas as pd
url = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/01-22-2020.csv' 
pd.read_csv(url).fillna(0)

我可以一次导入所有 .csv 文件(如果我知道如何提取所有 .csv 文件名，则可以循环执行)，但是应该有一种更有效的方法.一旦有了df，就可以对其进行更新":

I could import all the .csv files one per one (or with a loop if I knew how to extract all the .csv filenames), but there should be a more efficient way. Once I have the df, to "update" it I would:

1. 提取所有 .csv 文件名.
2. 检查它们是否都在df中(带有日期列).如果缺少一个，请将缺少的.csv文件添加到df中.

我遇到的问题是:(a)如何使可伸缩方式提取所有.csv文件?(b)有什么方法可以从github文件夹中仅提取以 .csv 结尾的文件名?为了执行上面的(2).

The problems I'm having are: (a) how can I make scalable the way to extract all the .csv files? and (b) is there any way to extract ONLY the filenames that end with .csv from the github folder? In order to do (2) of above.

推荐答案

我仍在尝试寻找更好的解决方案，但以下是我与我的代码一起从github目录中提取的变通方法.不幸的是，我仍然没有找到一种方法来像在本地驱动器上那样仅在github directoy中获取CSV列表.

I am still trying to find a better solution but below is a workaround that I use with my code to pull from a github directory. Unfortunately, I still have not found a way to just get a list of CSVs in the github directoy like you can if it was on a local drive.

def read_multi_csv(start_year,end_year):     
    years = list(range(start_year,end_year+1))     
    dfs = []
    for YYYY in years:         
        file = 'https://raw.githubusercontent.com/username/project/main/data/normalized/'+str(YYYY)+'_crimes_byState.csv'             
        #print (file)         
        df = pd.read_csv(file)         
        dfs.append(df)
    all_dfs = df.concat(df)         
    return all_dfs  

read_multi_csv(2013,2019)

这篇关于从github文件夹导入多个csv文件-Python-COVID-19的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！