通过一个函数在Pandas Dataframe中创建多个列 [英] Create multiple columns in Pandas Dataframe from one function
问题描述
我是python新手，所以我希望我的两个问题清楚完整.我在下面以CSV格式发布了实际代码和测试数据集.
I'm a python newbie, so I hope my two questions are clear and complete. I posted the actual code and a test data set in csv format below.
我已经能够构造以下代码(主要是在StackOverflow贡献者的帮助下)，以使用Newton-Raphson方法计算期权合约的隐含波动率.确定隐含波动率时，该过程将计算Vega.尽管我可以使用Pandas DataFrame apply方法为隐含波动率创建新的DataFrame列，但无法为Vega创建第二列.当函数返回IV&时，是否可以创建两个单独的DataFrame列?维加在一起吗?
I've been able to construct the following code (mostly with the help from the StackOverflow contributors) to calculate the Implied Volatility of an option contract using Newton-Raphson method. The process calculates Vega when determining the Implied Volatility. Although I'm able to create a new DataFrame column for Implied Volatility using the Pandas DataFrame apply method, I'm unable to create a second column for Vega. Is there a way create two separate DataFrame columns when the function to returns IV & Vega together?
我尝试过:
-
return iv, vega来自功能 -
df[['myIV', 'Vega']] = df.apply(newtonRap, axis=1) - 得到了
ValueError: Shape of passed values is (56, 2), indices imply (56, 13)
return iv, vegafrom functiondf[['myIV', 'Vega']] = df.apply(newtonRap, axis=1)- Got
ValueError: Shape of passed values is (56, 2), indices imply (56, 13)
也尝试过:
-
return iv, vega来自功能 -
df['myIV'], df['Vega'] = df.apply(newtonRap, axis=1) - 得到了
ValueError: Shape of passed values is (56, 2), indices imply (56, 13)
return iv, vegafrom functiondf['myIV'], df['Vega'] = df.apply(newtonRap, axis=1)- Got
ValueError: Shape of passed values is (56, 2), indices imply (56, 13)
此外，计算过程很慢.我导入了numba并实现了@jit(nogil = True)装饰器，但性能仅提高了25％.测试数据集是性能测试，具有将近90万条记录.如果不使用numba或使用numba，则运行时间为2小时9分钟，但是不使用nogil = True.使用numba和@jit(nogil = True)时的运行时间为1小时32分钟.我可以做得更好吗?
Additionally, the calculation process is slow. I imported numba and implemented the @jit(nogil=True) decorator, but I only see a performance improvement of 25%. The test data set is the performance test has almost 900,000 records. The run time is 2 hours and 9 minutes without numba or with numba, but witout nogil=True. The run time when using numba and @jit(nogil=True) is 1 hour and 32 minutes. Can I do better?
from datetime import datetime
from math import sqrt, pi, log, exp, isnan
from scipy.stats import norm
from numba import jit
# dff = Daily Fed Funds (Posted rate is usually one day behind)
dff = pd.read_csv('https://research.stlouisfed.org/fred2/data/DFF.csv', parse_dates=[0], index_col='DATE')
rf = float('%.4f' % (dff['VALUE'][-1:][0] / 100))
# rf = .0015 # Get Fed Funds Rate https://research.stlouisfed.org/fred2/data/DFF.csv
tradingMinutesDay = 450 # 7.5 hours per day * 60 minutes per hour
tradingMinutesAnnum = 113400 # trading minutes per day * 252 trading days per year
cal = USFederalHolidayCalendar() # Load US Federal holiday calendar
@jit(nogil=True) # nogil=True arg improves performance by 25%
def newtonRap(row):
"""Estimate Implied Volatility (IV) using Newton-Raphson method
:param row (dataframe): Options contract params for function
TimeStamp (datetime): Close date
Expiry (datetime): Option contract expiration date
Strike (float): Option strike
OptType (object): 'C' for call; 'P' for put
RootPrice (float): Underlying close price
Bid (float): Option contact closing bid
Ask (float): Option contact closing ask
:return:
float: Estimated implied volatility
"""
if row['Bid'] == 0.0 or row['Ask'] == 0.0 or row['RootPrice'] == 0.0 or row['Strike'] == 0.0 or \
row['TimeStamp'] == row['Expiry']:
iv, vega = 0.0, 0.0 # Set iv and vega to zero if option contract is invalid or expired
else:
# dte (Days to expiration) uses pandas bdate_range method to determine the number of business days to expiration
# minus USFederalHolidays minus constant of 1 for the TimeStamp date
dte = float(len(pd.bdate_range(row['TimeStamp'], row['Expiry'])) -
len(cal.holidays(row['TimeStamp'], row['Expiry']).to_pydatetime()) - 1)
mark = (row['Bid'] + row['Ask']) / 2
cp = 1 if row['OptType'] == 'C' else -1
S = row['RootPrice']
K = row['Strike']
# T = the number of trading minutes to expiration divided by the number of trading minutes in year
T = (dte * tradingMinutesDay) / tradingMinutesAnnum
# TODO get dividend value
d = 0.00
iv = sqrt(2 * pi / T) * mark / S # Closed form estimate of IV Brenner and Subrahmanyam (1988)
vega = 0.0
for i in range(1, 100):
d1 = (log(S / K) + T * (rf - d + iv ** 2 / 2)) / (iv * sqrt(T))
d2 = d1 - iv * sqrt(T)
vega = S * norm.pdf(d1) * sqrt(T)
model = cp * S * norm.cdf(cp * d1) - cp * K * exp(-rf * T) * norm.cdf(cp * d2)
iv -= (model - mark) / vega
if abs(model - mark) < 1.0e-9:
break
if isnan(iv) or isnan(vega):
iv, vega = 0.0, 0.0
# TODO Return vega with iv if add'l pandas column possible
# return iv, vega
return iv
if __name__ == "__main__":
# test function from baseline data
get_csv = True
if get_csv:
csvHeaderList = ['TimeStamp', 'OpraSymbol', 'RootSymbol', 'Expiry', 'Strike', 'OptType', 'RootPrice', 'Last',
'Bid', 'Ask', 'Volume', 'OpenInt', 'IV']
fileName = 'C:/tmp/test-20150930-56records.csv'
df = pd.read_csv(fileName, parse_dates=[0, 3], names=csvHeaderList)
else:
pass
start = datetime.now()
# TODO Create add'l pandas dataframe column, if possible, for vega
# df[['myIV', 'Vega']] = df.apply(newtonRap, axis=1)
# df['myIV'], df['Vega'] = df.apply(newtonRap, axis=1)
df['myIV'] = df.apply(newtonRap, axis=1)
end = datetime.now()
print end - start
测试数据:C:/tmp/test-20150930-56records.csv
Test Data: C:/tmp/test-20150930-56records.csv
2015-09-30 16:00:00，AAPL151016C00109000，AAPL，2015-10-16 16:00:00,109，C，109.95,3.46,3.6,3.7,1565,1290,0.3497 2015-09-30 16:00:00，AAPL151016P00109000，AAPL，2015-10-16 16:00:00,109，P，109.95,2.4,2.34,2.42,3790,3087,0.3146 2015-09-30 16:00:00，AAPL151016C00110000，AAPL，2015-10-16 16:00:00,110，C，109.95,3,2.86,3,10217,28850,0.3288 2015-09-30 16:00:00，AAPL151016P00110000，AAPL，2015-10-16 16:00:00,110，P，109.95,2.81,2.74,2.8,12113,44427,0.3029 2015-09-30 16:00:00，AAPL151016C00111000，AAPL，2015-10-16 16:00:00,111，C，109.95,2.35,2.44,2.45,6674,2318,0.3187 2015-09-30 16:00:00，AAPL151016P00111000，AAPL，2015-10-16 16:00:00,111，P，109.95,3.2,3.1,3.25,2031,3773,0.2926 2015-09-30 16:00:00，AAPL151120C00110000，AAPL，2015-11-20 16:00:00,110，C，109.95,5.9,5.7,5.95,5330,17112,0.3635 2015-09-30 16:00:00，AAPL151120P00110000，AAPL，2015-11-20 16:00:00,110，P，109.95,6.15,6.1,6.3,3724,15704,0.3842
2015-09-30 16:00:00,AAPL151016C00109000,AAPL,2015-10-16 16:00:00,109,C,109.95,3.46,3.6,3.7,1565,1290,0.3497 2015-09-30 16:00:00,AAPL151016P00109000,AAPL,2015-10-16 16:00:00,109,P,109.95,2.4,2.34,2.42,3790,3087,0.3146 2015-09-30 16:00:00,AAPL151016C00110000,AAPL,2015-10-16 16:00:00,110,C,109.95,3,2.86,3,10217,28850,0.3288 2015-09-30 16:00:00,AAPL151016P00110000,AAPL,2015-10-16 16:00:00,110,P,109.95,2.81,2.74,2.8,12113,44427,0.3029 2015-09-30 16:00:00,AAPL151016C00111000,AAPL,2015-10-16 16:00:00,111,C,109.95,2.35,2.44,2.45,6674,2318,0.3187 2015-09-30 16:00:00,AAPL151016P00111000,AAPL,2015-10-16 16:00:00,111,P,109.95,3.2,3.1,3.25,2031,3773,0.2926 2015-09-30 16:00:00,AAPL151120C00110000,AAPL,2015-11-20 16:00:00,110,C,109.95,5.9,5.7,5.95,5330,17112,0.3635 2015-09-30 16:00:00,AAPL151120P00110000,AAPL,2015-11-20 16:00:00,110,P,109.95,6.15,6.1,6.3,3724,15704,0.3842
推荐答案
如果我理解的正确，那么您应该做的就是从函数中返回一个Series.像这样:
If I understand you right, what you should be doing is returning a Series from your function. Something like:
return pandas.Series({"IV": iv, "Vega": vega})
如果要将结果放入同一输入DataFrame的新列中，则只需执行以下操作:
If you want to put the result into new columns of the same input DataFrame, then just do:
df[["IV", "Vega"]] = df.apply(newtonRap, axis=1)
这篇关于通过一个函数在Pandas Dataframe中创建多个列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
