python genfromtxt问题 [英] python genfromtxt problems
问题描述
我是Python的新手，这是我的问题. 对于正在用Python测试的优化子例程，我需要使用数字解析csv文件.
I am new to Python...here is my problem. For an optimizing subroutine I am testing in Python, I need to parse a csv file with numbers.
csv文件的格式为:
The format of the csv file is thus:
Support load summary for anchor at node 5,
Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,
Sustained,-3,-2679,120,2012,164,69,,
Operating1,1472,2710,-672,-4520,8743,-2047,,
Maximum,1472,2710,120,2012,8743,69,,
Minimum,-3,-2679,-672,-4520,164,-2047,,
Support load summary for anchor at node 40,
Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,
Sustained,9,-3872,-196,-91,854,-3914,,
Operating1,-2027,-8027,3834,-7573,-9102,-6323,,
Maximum,9,-3872,3834,-91,854,-3914,,
Minimum,-2027,-8027,-196,-7573,-9102,-6323,,
Support load summary for anchor at node 125,
Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,
Sustained,-7,-2448,76,264,83,1320,,
Operating1,556,-3771,-3162,-6948,-1367,1272,,
Maximum,556,-2448,76,264,83,1320,,
Minimum,-7,-3771,-3162,-6948,-1367,1272,,
Support load summary for Hanger at node 10,
Load combination,Load (N),,
Sustained,-3668,,
Operating1,-13876,,
Maximum,-3668,,
Minimum,-13876,,
Support load summary for Hanger at node 20B,
Load combination,Load (N),,
Sustained,-14305,,
Operating1,-13359,,
Maximum,-13359,,
Minimum,-14305,,
Support load summary for restraint at node 115B,
Load combination,FX (N),FY (N),FZ (N),,
Sustained,,-5655,,,
Operating1,3696,,
Maximum,,3696,,,
Minimum,,-5655,,,
我的代码主要在以
Operating1,
Maximum,
Minimum,
工作(成本函数)是对这些关键字之一之后的所有数字进行总计(代数计算).有时，如您在上面的数据文件中看到的那样，第二或第三列中只有一个数字. (请参见数据文件的末尾)，有时根本没有数字，如以下文件片段中所示(请参见下面的Operating1行).
The job (cost function) is to total (algebraically) all the numbers following one of these keywords. Sometimes as you can see in the data file above, there is only one number in the 2nd or 3rd col. (see end of data file), sometimes, there is no number at all like in the following file fragment (see line for Operating1 below).
Support load summary for Hanger at node 115B,
Load combination,Load (N),,
Sustained,-5188,,
Operating1,,,
Maximum,,,
Minimum,-5188,,
我正在使用np.genfromtxt().效果很好，除非遇到行中列值少于4个或有时根本没有值的行.
I am using np.genfromtxt(). Works great except when I run into lines that have fewer than 4 values in columns or sometimes none at all.
我在genfromtxt()上使用sum()-请参见代码.当只有一个值时，我使用了float().如果没有，我尝试确定总数，并将其分配为零.我可以针对每种情况进行自定义，但我想知道在不可预测的情况下是否存在一种通用的，更抽象的读取和汇总数字的方法.
I am using sum() on genfromtxt() - see code. When there is only one value, I used a float(). When there is none, I tried to identify and assign zero to the total. I can customize for each case but am wondering if there is a general, more abstract method of reading and totaling the numbers in unpredictable cases.
此外，我尝试了"missing_values"和"filling_values"，但它们似乎不起作用.如何计算文件中非零列的数量?
Plus, I tried the "missing_values" and "filling_values" but they do not seem to work. How do I count the # of non-zero columns in a file?
到目前为止，这是代码的一部分:
Here is part of the code so far:
def optimize(fn, optflag):
modeltotals = []
i=0
csv1 = []
j = 1 # line # count
for line in csv.reader(filelist) :
temp = repr(line)
if "Support load summary" in temp :
csv1.append(line) # just making another list of actionable lines for future use
if (d): print "\n", line
continue
if (optflag == "ope") : # optimize on Operating loads
if "Operating1" in temp:
csv1.append(line)
if (len(line) > 4):
modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
if (d): print "Sum of OPE Loads:", modeltotals[i], "\n"
elif (len(line) > 0 and len(line) <= 4):
if (d): print "line=", line, "length", len(line)
line1 = np.genfromtxt(line[1:], delimiter=",")
if not line1: # meaning if array is empty
modeltotals.append(0)
else:
modeltotals.append(np.genfromtxt(line[1:], delimiter=",", missing_values=[0,0,0,0]))
if (d): print "OPE Max:", modeltotals[i],"\n"
i +=1
elif (optflag == "minmax") : #optimize on all loads, min and max.
#print "i=", i
if "Maximum" in temp:
csv1.append(line)
if (len(line) > 4):
modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
if (d): print "Sum of Maxs:", modeltotals[i]
elif (len(line) <= 4):
#line1 = np.genfromtxt(line[1:], delimiter=",", filling_values = 0)
#modeltotals.append(sum(line1))
if (d): print "line=", line, "length", len(line)
line1 = np.genfromtxt(line[1:], delimiter=",")
print "line1 =", line1
if not line1: # meaning if array is empty
modeltotals.append(0)
else:
modeltotals.append(np.genfromtxt(line[1:], delimiter=",", filling_values = 0))
if (d): print "Max:", modeltotals[i]
i+=1
elif "Minimum" in temp:
csv1.append(line)
if (len(line) > 4):
#print "#", j, "line", line
modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
if (d): print "Sum of Mins:", modeltotals[i]
elif (len(line) > 0 and len(line) <= 4):
if (d): print "line=", line, "length", len(line)
line1 = np.genfromtxt(line[1:], delimiter=",")
if not line1: # meaning if array is empty
modeltotals.append(0)
else:
modeltotals.append(np.genfromtxt(line[1:], delimiter=","))
if (d): print "Min:", modeltotals[i]
i +=1
j+=1
if len(modeltotals) > 0:
print modeltotals
average = float(sum(modeltotals))/len(modeltotals) #sometimes error here
else:
return "000" # error, seems like no file was analyzed
if (d):
print "Current model mean =", average
del csv1[:]
return abs(average)
我在不同文件中遇到的几个错误是相似的:
The several errors I run into in different files are similar:
['Support load summary for restraint at node 20B', '']
Traceback (most recent call last):
File "sor4.py", line 190, in <module>
modelmean[filename] = optimize(filename, args.optimizeon)
File "sor4.py", line 107, in optimize
modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
TypeError: iteration over a 0-d array
另一个错误是无法转换为标量".
The other error is a "Cannot convert to a scalar."
我理解这些错误，但不了解足够多的Python来巧妙地处理它们.对不起，很长的帖子；我会更好地简洁地介绍信息.正如这里的另一位海报所说，我将非常感谢您的回答.谢谢.
I understand the errors but know not much Python to cleverly deal with them. Sorry for the long post; I will get better to present information more succinctly. As another poster here said, I will gratefully accept your answers. Thank you.
推荐答案
我将您的问题简化为以下代码.它检查nan和空的输入字符串.
I reduced your problem to the following code. It checks for nans and empty input strings.
from StringIO import StringIO
import numpy as np
def getnumbers(s):
try:
res = np.genfromtxt(s, delimiter=",")
return res[np.where(np.isnan(res), False, True)]
except IOError as ioe:
return np.array(0.)
print(sum(getnumbers(StringIO('1., 2., , '))))
print(sum(getnumbers(StringIO(''))))
它给出结果
3.0
0.0
这篇关于python genfromtxt问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
