genfromtxt()中的NumPy dtype问题,以字节字符串形式读取字符串 [英] NumPy dtype issues in genfromtxt(), reads string in as bytestring
问题描述
我想将标准ascii csv文件读入numpy,该文件由浮点数和字符串组成.
I want to read in a standard-ascii csv file into numpy, which consists of floats and strings.
例如,
ZINC00043096,C.3,C1,-0.1540,methyl
ZINC00043096,C.3,C2,0.0638,methylene
ZINC00043096,C.3,C4,0.0669,methylene
ZINC00090377,C.3,C7,0.2070,methylene
...
无论我尝试什么,生成的数组看起来像
Whatever I tried, the resulting array would look like
例如
all_data = np.genfromtxt(csv_file, dtype=None, delimiter=',')
[(b'ZINC00043096', b'C.3', b'C1', -0.154, b'methyl')
(b'ZINC00043096', b'C.3', b'C2', 0.0638, b'methylene')
(b'ZINC00043096', b'C.3', b'C4', 0.0669, b'methylene')
但是,我想保存一个字节-字符串转换步骤,并且想知道如何将字符串列作为常规字符串直接读取.
However, I want to save a step for the byte-string conversion and was wondering how I can read in the string columns as regular string directly.
我尝试了numpy.genfromtxt()文档中的几项内容,例如dtype='S,S,S,f,S'
或dtype='a25,a25,a25,f,a25'
,但实际上没有任何帮助.
I tried several things from the numpy.genfromtxt() documentation, e.g., dtype='S,S,S,f,S'
or dtype='a25,a25,a25,f,a25'
, but nothing really helped here.
恐怕,但是我想我只是不了解dtype转换的真正工作原理……如果您能在这里给我一些提示,那就太好了!
I am afraid, but I think I just don't understand how the dtype conversion really works...Would be nice if you can give me some hint here!
谢谢
推荐答案
在Python2.7中
In Python2.7
array([('ZINC00043096', 'C.3', 'C1', -0.154, 'methyl'),
('ZINC00043096', 'C.3', 'C2', 0.0638, 'methylene'),
('ZINC00043096', 'C.3', 'C4', 0.0669, 'methylene'),
('ZINC00090377', 'C.3', 'C7', 0.207, 'methylene')],
dtype=[('f0', 'S12'), ('f1', 'S3'), ('f2', 'S2'), ('f3', '<f8'), ('f4', 'S9')])
在Python3中
array([(b'ZINC00043096', b'C.3', b'C1', -0.154, b'methyl'),
(b'ZINC00043096', b'C.3', b'C2', 0.0638, b'methylene'),
(b'ZINC00043096', b'C.3', b'C4', 0.0669, b'methylene'),
(b'ZINC00090377', b'C.3', b'C7', 0.207, b'methylene')],
dtype=[('f0', 'S12'), ('f1', 'S3'), ('f2', 'S2'), ('f3', '<f8'), ('f4', 'S9')])
Python3中的'regular'字符串是unicode.但是您的文本文件具有字节字符串. all_data
在两种情况下都是相同的(136个字节),但是Python3显示字节字符串的方式是b'C.3'
,而不仅仅是'C.3'.
The 'regular' strings in Python3 are unicode. But your text file has byte strings. all_data
is the same in both cases (136 bytes), but Python3's way of displaying a byte string is b'C.3'
, not just 'C.3'.
您打算对这些字符串进行哪些类型的操作? 'ZIN' in all_data['f0'][1]
适用于2.7版本,但是在3中必须使用b'ZIN' in all_data['f0'][1]
.
What kinds of operations do you plan on doing with these strings? 'ZIN' in all_data['f0'][1]
works with the 2.7 version, but in 3 you have to use b'ZIN' in all_data['f0'][1]
.
在numpy中变量/未知长度字符串/unicode dtype
提醒我可以在dtype
中指定unicode字符串类型.但是,如果您事先不知道字符串的长度,这将变得更加复杂.
Variable/unknown length string/unicode dtype in numpy
reminds me that you can specify a unicode string type in the dtype
. However this becomes more complicated if you don't know the lengths of the strings beforehand.
alttype = np.dtype([('f0', 'U12'), ('f1', 'U3'), ('f2', 'U2'), ('f3', '<f8'), ('f4', 'U9')])
all_data_u = np.genfromtxt(csv_file, dtype=alttype, delimiter=',')
生产
array([('ZINC00043096', 'C.3', 'C1', -0.154, 'methyl'),
('ZINC00043096', 'C.3', 'C2', 0.0638, 'methylene'),
('ZINC00043096', 'C.3', 'C4', 0.0669, 'methylene'),
('ZINC00090377', 'C.3', 'C7', 0.207, 'methylene')],
dtype=[('f0', '<U12'), ('f1', '<U3'), ('f2', '<U2'), ('f3', '<f8'), ('f4', '<U9')])
在Python2.7中,all_data_u
显示为
In Python2.7 all_data_u
displays as
(u'ZINC00043096', u'C.3', u'C1', -0.154, u'methyl')
all_data_u
是448个字节,因为numpy
为每个unicode字符分配4个字节.每个U4
项目的长度为16个字节.
all_data_u
is 448 bytes, because numpy
allocates 4 bytes for each unicode character. Each U4
item is 16 bytes long.
1.14版中的更改: https://docs.scipy.org/doc/numpy/release.html#encoding-argument-for-text-io-functions
Changes in v 1.14: https://docs.scipy.org/doc/numpy/release.html#encoding-argument-for-text-io-functions
这篇关于genfromtxt()中的NumPy dtype问题,以字节字符串形式读取字符串的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!