在 pandas 中用NaN替换空白值(空白) [英] Replacing blank values (white space) with NaN in pandas
问题描述
我想在Pandas数据框中找到所有包含空格(任意数量)的值,并用NaN替换这些值.
I want to find all values in a Pandas dataframe that contain whitespace (any arbitrary amount) and replace those values with NaNs.
有什么想法可以改善这一点吗?
Any ideas how this can be improved?
基本上我想把这个转过来:
Basically I want to turn this:
A B C
2000-01-01 -0.532681 foo 0
2000-01-02 1.490752 bar 1
2000-01-03 -1.387326 foo 2
2000-01-04 0.814772 baz
2000-01-05 -0.222552 4
2000-01-06 -1.176781 qux
对此:
A B C
2000-01-01 -0.532681 foo 0
2000-01-02 1.490752 bar 1
2000-01-03 -1.387326 foo 2
2000-01-04 0.814772 baz NaN
2000-01-05 -0.222552 NaN 4
2000-01-06 -1.176781 qux NaN
我已经用下面的代码做到了,但是这很丑.这不是Pythonic,而且我敢肯定,这也不是最有效的熊猫使用方式.我遍历每一列,并对通过应用函数对每个值进行正则表达式搜索(在空格上匹配)而生成的列掩码进行布尔替换.
I've managed to do it with the code below, but man is it ugly. It's not Pythonic and I'm sure it's not the most efficient use of pandas either. I loop through each column and do boolean replacement against a column mask generated by applying a function that does a regex search of each value, matching on whitespace.
for i in df.columns:
df[i][df[i].apply(lambda i: True if re.search('^\s*$', str(i)) else False)]=None
可以仅通过迭代可能包含空字符串的字段来对其进行优化:
It could be optimized a bit by only iterating through fields that could contain empty strings:
if df[i].dtype == np.dtype('object')
但这并没有太大的改善
最后,此代码将目标字符串设置为None,这可以与Pandas的fillna()
之类的功能配合使用,但是如果我能直接插入NaN
而不是None
的话,那么完整性也很好. /p>
And finally, this code sets the target strings to None, which works with Pandas' functions like fillna()
, but it would be nice for completeness if I could actually insert a NaN
directly instead of None
.
推荐答案
I think df.replace()
does the job, since pandas 0.13:
df = pd.DataFrame([
[-0.532681, 'foo', 0],
[1.490752, 'bar', 1],
[-1.387326, 'foo', 2],
[0.814772, 'baz', ' '],
[-0.222552, ' ', 4],
[-1.176781, 'qux', ' '],
], columns='A B C'.split(), index=pd.date_range('2000-01-01','2000-01-06'))
# replace field that's entirely space (or empty) with NaN
print(df.replace(r'^\s*$', np.nan, regex=True))
产生:
A B C
2000-01-01 -0.532681 foo 0
2000-01-02 1.490752 bar 1
2000-01-03 -1.387326 foo 2
2000-01-04 0.814772 baz NaN
2000-01-05 -0.222552 NaN 4
2000-01-06 -1.176781 qux NaN
Temak 指出,如果您的有效数据包含空格,请使用df.replace(r'^\s+$', np.nan, regex=True)
.
As Temak pointed it out, use df.replace(r'^\s+$', np.nan, regex=True)
in case your valid data contains white spaces.
这篇关于在 pandas 中用NaN替换空白值(空白)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!