将CSV直接下载到Python CSV解析器中 [英] Download CSV directly into Python CSV parser

问题描述

我正在尝试从morningstar下载CSV内容，然后解析其内容。如果我将HTTP内容直接注入Python的CSV解析器，结果格式不正确。但是，如果我将HTTP内容保存到文件（/tmp/tmp.csv），然后在python的csv解析器中导入该文件的结果是正确的。换句话说，为什么：

  def finDownload（code，report）：
h = httplib2.Http（'。 cache'）
 url ='http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t='+ code +'& region = AUS& culture = en_us& reportType ='+ report +' & period = 12& dataType = A& order = asc& columnYear = 5& rounding = 1& view = raw& productCode = usa& denominatorView = raw& number = 1'
 headers，data = h.request url）
返回数据
 
 balancesheet = csv.reader（finDownload（'FGE'，'is'））
对于平衡表中的行：
 print row 
  

返回：

  ['F'] 
 ['o'] 
 ['r'] 
 ['g'] 
 ['e'] 
 [ '] 
 ['G'] 
 ['r'] 
 ['o'] 
 ['u'] 
 $ b  

而不是：

  [Forge Group Limited（FGE）损益表'] 
  

？ b $ b

解决方案

问题是由于对文件的迭代是逐行完成的，而对字符串的迭代是逐字符完成的。

您要 StringIO / cStringIO （Python 2）或 io.StringIO （Python 3，感谢John Machin指向它），因此一个字符串可以被当作一个类文件对象：

Python 2：

  mystring ='a，b \\ nb，c \\\
1,2,3'
 import cStringIO 
 csvio = cStringIO.StringIO（mystring）
 mycsv = csv.reader（csvio）
  
 
 
 
  Python 3 ：
 code> mystring ='a，b \\\
b，c\\\
1,2,3'
 import io 
 csvio = io.StringIO（mystring，newline =）
 mycsv = csv.reader（csvio）
  
 / p> 
 
 
 >>>对于mycsv中的行：print（row）
 ... 
 ['a'，'b\\\
b'，'c'] 
 ['1'，'2' 3'] 
  
 
I'm trying to download CSV content from morningstar and then parse its contents. If I inject the HTTP content directly into Python's CSV parser, the result is not formatted correctly. Yet, if I save the HTTP content to a file (/tmp/tmp.csv), and then import the file in the python's csv parser the result is correct. In other words, why does:
def finDownload(code,report):
    h = httplib2.Http('.cache')
    url = 'http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=' + code + '&region=AUS&culture=en_us&reportType='+ report + '&period=12&dataType=A&order=asc&columnYear=5&rounding=1&view=raw&productCode=usa&denominatorView=raw&number=1'
    headers, data = h.request(url)
    return data

balancesheet = csv.reader(finDownload('FGE','is'))
for row in balancesheet:
    print row
return:
['F']
['o']
['r']
['g']
['e']
[' ']
['G']
['r']
['o']
['u']
     (etc...)
instead of:
[Forge Group Limited (FGE) Income Statement']
?
解决方案
The problem results from the fact that iteration over a file is done line-by-line whereas iteration over a string is done character-by-character.

You want StringIO/cStringIO (Python 2) or io.StringIO (Python 3, thanks to John Machin for pointing me to it) so a string can be treated as a file-like object:

Python 2:
mystring = 'a,"b\nb",c\n1,2,3'
import cStringIO
csvio = cStringIO.StringIO(mystring)
mycsv = csv.reader(csvio)
Python 3:
mystring = 'a,"b\nb",c\n1,2,3'
import io
csvio = io.StringIO(mystring, newline="")
mycsv = csv.reader(csvio)
Both will correctly preserve newlines inside quoted fields:
>>> for row in mycsv: print(row)
...
['a', 'b\nb', 'c']
['1', '2', '3']


                        
这篇关于将CSV直接下载到Python CSV解析器中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！