限制使用numpy.genfromtxt为matplotlib读取多少数据 [英] limit how much data is read with numpy.genfromtxt for matplotlib

问题描述

我正在使用文本文件中的源数据和matplotlib在python中创建图形，以绘制图形. 下面的简单逻辑效果很好.

I am creating a graph in python using a text file for the source data and matplotlib to plot the graph. The simple logic below works well.

但是有没有办法让numpy.gentfromtxt只读取文件'temperature_logging'的前50行?目前，它会读取整个文件.

But is there a way to get have numpy.gentfromtxt only read the first 50 lines from the file 'temperature_logging'? Currently it reads the entire file.

temp = numpy.genfromtxt('temperature_logging',dtype=None,usecols=(0))
time = numpy.genfromtxt('temperature_logging',dtype=None,usecols=(1))

dates = matplotlib.dates.datestr2num(time)

pylab.plot_date(dates,temp,xdate=True,fmt='b-')

pylab.savefig('gp.png')

temperature_logging中的内容；

contents in temperature_logging;

21.75 12-01-2012-15:53:35    
21.75 12-01-2012-15:54:35    
21.75 12-01-2012-15:55:35    
.
.
.

推荐答案

numpy.genfromtxt接受迭代器以及文件.这意味着它将接受itertools.islice的输出.在这里，test.txt是一个五行文件:

numpy.genfromtxt accepts iterators as well as files. That means it will accept the output of itertools.islice. Here, test.txt is a five-line file:

>>> import itertools, numpy
>>> with open('test.txt') as t_in:
...     numpy.genfromtxt(itertools.islice(t_in, 3))
... 
array([[  1.,   2.,   3.,   4.,   5.],
       [  6.,   7.,   8.,   9.,  10.],
       [ 11.,  12.,  13.,  14.,  15.]])

可能会认为这比让numpy处理文件IO的速度要慢，但是快速测试表明并非如此. genfromtxt提供了skip_footer关键字参数，如果您知道文件有多长时间...

One might think this would be slower than letting numpy handle the file IO, but a quick test suggests otherwise. genfromtxt provides a skip_footer keyword argument that you can use if you know how long the file is...

>>> numpy.genfromtxt('test.txt', skip_footer=2)
array([[  1.,   2.,   3.,   4.,   5.],
       [  6.,   7.,   8.,   9.,  10.],
       [ 11.,  12.,  13.,  14.,  15.]])

...但是对1000行文件的一些非正式测试表明，即使只跳过了几行，使用islice的速度也更快:

...but a few informal tests on a 1000-line file suggest that using islice is faster even if you skip only a few lines:

>>> def get(nlines, islice=itertools.islice):
...     with open('test.txt') as t_in:
...         numpy.genfromtxt(islice(t_in, nlines))
...         
>>> %timeit get(3)
1000 loops, best of 3: 338 us per loop
>>> %timeit numpy.genfromtxt('test.txt', skip_footer=997)
100 loops, best of 3: 4.92 ms per loop
>>> %timeit get(300)
100 loops, best of 3: 5.04 ms per loop
>>> %timeit numpy.genfromtxt('test.txt', skip_footer=700)
100 loops, best of 3: 8.48 ms per loop
>>> %timeit get(999)
100 loops, best of 3: 16.2 ms per loop
>>> %timeit numpy.genfromtxt('test.txt', skip_footer=1)
100 loops, best of 3: 16.7 ms per loop

这篇关于限制使用numpy.genfromtxt为matplotlib读取多少数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！