使用python请求下载CSV [英] Use python requests to download CSV
问题描述
这是我的代码:
import csv
import requests
with requests.Session() as s:
s.post(url, data=payload)
download = s.get('url that directly download a csv report')
这使我可以访问csv文件.我尝试了其他方法来处理下载:
This gives me the access to the csv file. I tried different method to deal with the download:
这将把csv文件放在一个字符串中:
This will give the the csv file in one string:
print download.content
这将打印第一行并返回错误:_csv.Error:在未加引号的字段中出现换行符
This print the first row and return error: _csv.Error: new-line character seen in unquoted field
cr = csv.reader(download, dialect=csv.excel_tab)
for row in cr:
print row
这将在每行中打印一个字母,并且不会打印出整个内容:
This will print a letter in each row and it won't print the whole thing:
cr = csv.reader(download.content, dialect=csv.excel_tab)
for row in cr:
print row
我的问题是:在这种情况下,读取csv文件的最有效方法是什么? 以及如何下载.
My question is: what's the most efficient way to read a csv file in this situation. And how to download it.
谢谢
推荐答案
这应该有所帮助:
import csv
import requests
CSV_URL = 'http://samplecsvs.s3.amazonaws.com/Sacramentorealestatetransactions.csv'
with requests.Session() as s:
download = s.get(CSV_URL)
decoded_content = download.content.decode('utf-8')
cr = csv.reader(decoded_content.splitlines(), delimiter=',')
my_list = list(cr)
for row in my_list:
print(row)
输出样本:
['street', 'city', 'zip', 'state', 'beds', 'baths', 'sq__ft', 'type', 'sale_date', 'price', 'latitude', 'longitude']
['3526 HIGH ST', 'SACRAMENTO', '95838', 'CA', '2', '1', '836', 'Residential', 'Wed May 21 00:00:00 EDT 2008', '59222', '38.631913', '-121.434879']
['51 OMAHA CT', 'SACRAMENTO', '95823', 'CA', '3', '1', '1167', 'Residential', 'Wed May 21 00:00:00 EDT 2008', '68212', '38.478902', '-121.431028']
['2796 BRANCH ST', 'SACRAMENTO', '95815', 'CA', '2', '1', '796', 'Residential', 'Wed May 21 00:00:00 EDT 2008', '68880', '38.618305', '-121.443839']
['2805 JANETTE WAY', 'SACRAMENTO', '95815', 'CA', '2', '1', '852', 'Residential', 'Wed May 21 00:00:00 EDT 2008', '69307', '38.616835', '-121.439146']
[...]
与答案相关的问题: https://stackoverflow.com/a/33079644/295246
如果需要下载大文件(例如stream=True
),其他答案也很有用.
Other answers are useful if you need to download large files (i.e. stream=True
).
这篇关于使用python请求下载CSV的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!