ProcessPoolExecutor 不执行 [英] ProcessPoolExecutor don't Execute

问题描述

我尝试以比实际更快的速度获得 ARIMA 配置.

I try to get ARIMA configuration some faster that I acctually do.

所以我使用 Iterate 方法来比较所有 ARIMA 组合以更好地选择.为此，我创建了一个迭代函数:

So I use a Iterate method to compare all ARIMA combinations to select better. For that I create a function to Iterate:

def difference(dataset, interval=1):
    diff = list()
    for i in range(interval, len(dataset)):
        value = dataset[i] - dataset[i - interval]
        diff.append(value)
    return np.array(diff)

# invert differenced value
def inverse_difference(history, yhat, interval=1):
    return yhat + history[-interval]

# evaluate an ARIMA model for a given order (p,d,q) and return RMSE
def evaluate_arima_model(dataset, arima_order):

    dataset = dataset.astype('float32')
    train_size = int(len(dataset) * 0.50)
    train, test = dataset[0:train_size], dataset[train_size:]
    
    history = [x for x in train]
    # make predictions
    predictions = list()
    for t in range(len(test)):
        # difference data
        months_in_year = maxlength
        diff = difference(history, months_in_year)
        model = ARIMA(diff, order=arima_order)
        model_fit = model.fit(trend='nc', disp=0)
        yhat = model_fit.forecast()[0]
        yhat = inverse_difference(history, yhat, months_in_year)
        predictions.append(yhat)
        history.append(test[t])
    # calculate out of sample error
    mse = mean_squared_error(test, predictions)
    rmse = sqrt(mse)
    return rmse

实际上我用这种方法在几分钟内就完成了.但是对于我要使用逻辑的 API 来说，这不是好时机.

Actually I do that in minutes with this method. But isn't good time for an API where I gonna use the logic.

# evaluate combinations of p, d and q values for an ARIMA model

def evaluate_models(dataset, p_values, d_values, q_values):
    dataset = dataset.astype('float32')
    train_size = int(len(dataset) * 0.50)
    train, test = dataset[0:train_size], dataset[train_size:]
    
    global best_score, best_cfg 
    best_score, best_cfg = float("inf"), None
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                try:
                    mse = evaluate_arima_model(dataset, order)
                    if mse < best_score:
                        best_score, best_cfg = mse, order
                    print('ARIMA%s RMSE=%.3f' % (order,mse))
                except:
                    continue
        # print(best_cfg, best_score)
    print('Best ARIMA%s RMSE=%.3f' % (best_cfg, best_score))




# evaluate parameters
p_values = range(0, 7)
d_values = range(0, 3)
q_values = range(0, 7)
warnings.filterwarnings("ignore")
evaluate_models(data_train.values, p_values, d_values, q_values)

为了加速我想要使用迭代 evaluate_arima_model 函数的 Multiprocessing 方法的过程.但是 ProcessPoolExecutor 不起作用，因为不打印任何结果

To acelerate the process I want use Multiprocessing method iterating evaluate_arima_model function. But ProcessPoolExecutor don't work because don't print any result

# evaluate combinations of p, d and q values for an ARIMA model
orders = []
def fill_orders( p_values, d_values, q_values):
    
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                orders.append(order)

# fill orders array
p_values = range(0, 7)
d_values = range(0, 3)
q_values = range(0, 7)
warnings.filterwarnings("ignore")
fill_orders(p_values, d_values, q_values)

with concurrent.futures.ProcessPoolExecutor() as executor:
    results = [executor.submit(evaluate_arima_model, (dataset, order)) for order in orders]
    for f in concurrent.futures.as_completed(results):
        print(f.result())
        try:
            f.result()
        except:
            continue
        else:
            print(f.result())

推荐答案

我不希望你展示的第二个代码块做任何事情.对于此代码:

I wouldn't expect the second block of code you've shown to do anything. For this code:

# evaluate combinations of p, d and q values for an ARIMA model
orders = []
def evaluate_models( p_values, d_values, q_values):
    
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                orders.append(order)

with concurrent.futures.ProcessPoolExecutor() as executor:
    results = [executor.submit(evaluate_arima_model, (dataset, order)) for order in orders]
    for f in concurrent.futures.as_completed(results):
        print(f.result())
        try:
            f.result()
        except:
            continue
        else:
            print(f.result())

orders 将永远是空的，因为你是这样声明的，然后永远不会调用 evaluate_models，或者任何其他可能将对象放入 orders 的东西>.由于orders 为空，因此不会注册任何进程运行，并且results 也将为空，因此这段代码不会做任何事情.您的意思是在执行 with concurrent.futures.... 之前调用 evaluate_models 吗?

orders will always be empty because you are declaring it so and then never calling evaluate_models, or anything else that could be putting objects into orders. Since orders is empty, no processes will be registered to run, and results will also be empty, and so this code won't do anything. Do you mean to call evaluate_models before you do with concurrent.futures....?

这篇关于ProcessPoolExecutor 不执行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！