ProcessPoolExecutor不执行 [英] ProcessPoolExecutor don't Execute

问题描述

我尝试更快地获得ARIMA配置，而我应该这样做.

I try to get ARIMA configuration some faster that I acctually do.

因此，我使用Iterate方法比较所有ARIMA组合以选择更好的组合.为此，我创建了一个迭代的函数:

So I use a Iterate method to compare all ARIMA combinations to select better. For that I create a function to Iterate:

def difference(dataset, interval=1):
    diff = list()
    for i in range(interval, len(dataset)):
        value = dataset[i] - dataset[i - interval]
        diff.append(value)
    return np.array(diff)

# invert differenced value
def inverse_difference(history, yhat, interval=1):
    return yhat + history[-interval]

# evaluate an ARIMA model for a given order (p,d,q) and return RMSE
def evaluate_arima_model(dataset, arima_order):

    dataset = dataset.astype('float32')
    train_size = int(len(dataset) * 0.50)
    train, test = dataset[0:train_size], dataset[train_size:]
    
    history = [x for x in train]
    # make predictions
    predictions = list()
    for t in range(len(test)):
        # difference data
        months_in_year = maxlength
        diff = difference(history, months_in_year)
        model = ARIMA(diff, order=arima_order)
        model_fit = model.fit(trend='nc', disp=0)
        yhat = model_fit.forecast()[0]
        yhat = inverse_difference(history, yhat, months_in_year)
        predictions.append(yhat)
        history.append(test[t])
    # calculate out of sample error
    mse = mean_squared_error(test, predictions)
    rmse = sqrt(mse)
    return rmse

实际上，我用这种方法在几分钟内就完成了.但是对于我要使用逻辑的API来说不是一个好时机.

Actually I do that in minutes with this method. But isn't good time for an API where I gonna use the logic.

# evaluate combinations of p, d and q values for an ARIMA model

def evaluate_models(dataset, p_values, d_values, q_values):
    dataset = dataset.astype('float32')
    train_size = int(len(dataset) * 0.50)
    train, test = dataset[0:train_size], dataset[train_size:]
    
    global best_score, best_cfg 
    best_score, best_cfg = float("inf"), None
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                try:
                    mse = evaluate_arima_model(dataset, order)
                    if mse < best_score:
                        best_score, best_cfg = mse, order
                    print('ARIMA%s RMSE=%.3f' % (order,mse))
                except:
                    continue
        # print(best_cfg, best_score)
    print('Best ARIMA%s RMSE=%.3f' % (best_cfg, best_score))




# evaluate parameters
p_values = range(0, 7)
d_values = range(0, 3)
q_values = range(0, 7)
warnings.filterwarnings("ignore")
evaluate_models(data_train.values, p_values, d_values, q_values)

要加速该过程，我想使用迭代 evaluate_arima_model 函数的Multiprocessing方法.但是 ProcessPoolExecutor 不起作用，因为不打印任何结果

To acelerate the process I want use Multiprocessing method iterating evaluate_arima_model function. But ProcessPoolExecutor don't work because don't print any result

# evaluate combinations of p, d and q values for an ARIMA model
orders = []
def fill_orders( p_values, d_values, q_values):
    
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                orders.append(order)

# fill orders array
p_values = range(0, 7)
d_values = range(0, 3)
q_values = range(0, 7)
warnings.filterwarnings("ignore")
fill_orders(p_values, d_values, q_values)

with concurrent.futures.ProcessPoolExecutor() as executor:
    results = [executor.submit(evaluate_arima_model, (dataset, order)) for order in orders]
    for f in concurrent.futures.as_completed(results):
        print(f.result())
        try:
            f.result()
        except:
            continue
        else:
            print(f.result())

推荐答案

我不希望显示的第二段代码可以执行任何操作.对于此代码:

I wouldn't expect the second block of code you've shown to do anything. For this code:

# evaluate combinations of p, d and q values for an ARIMA model
orders = []
def evaluate_models( p_values, d_values, q_values):
    
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                orders.append(order)

with concurrent.futures.ProcessPoolExecutor() as executor:
    results = [executor.submit(evaluate_arima_model, (dataset, order)) for order in orders]
    for f in concurrent.futures.as_completed(results):
        print(f.result())
        try:
            f.result()
        except:
            continue
        else:
            print(f.result())

orders 始终为空，因为您要声明为空，然后再不调用 evaluate_models 或可能将对象放入 orders .由于 orders 为空，因此不会注册任何要运行的进程，并且 results 也将为空，因此此代码将不执行任何操作.您是不是要先对 evaluate_models 进行调用，然后再对并发.futures .... 进行处理?

orders will always be empty because you are declaring it so and then never calling evaluate_models, or anything else that could be putting objects into orders. Since orders is empty, no processes will be registered to run, and results will also be empty, and so this code won't do anything. Do you mean to call evaluate_models before you do with concurrent.futures....?

这篇关于ProcessPoolExecutor不执行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！