pyomo生成具有大量约束的模型的性能 [英] Performance of pyomo to generate a model with a huge number of constraints

问题描述

我对Pyomo生成具有大量约束和变量(大约10e6)的OR模型的性能感兴趣.我目前正在使用GAMS进行优化，但是我想使用不同的python功能，因此要使用Pyomo生成模型.

I am interested in the performance of Pyomo to generate an OR model with a huge number of constraints and variables (about 10e6). I am currently using GAMS to launch the optimizations but I would like to use the different python features and therefore use Pyomo to generate the model.

我做了一些测试，显然当我编写一个模型时，每次实例化约束时都会调用用于定义约束的python方法.在进一步实施之前，我想知道是否存在一种基于numpy数组数据直接创建约束块的方法?从我的角度来看，对于大型模型，按块构造约束可能更有效.

I made some tests and apparently when I write a model, the python methods used to define the constraints are called each time the constraint is instanciated. Before going further in my implementation, I would like to know if there exists a way to create directly a block of constraints based on numpy array data ? From my point of view, constructing constraints by block may be more efficient for large models.

您认为使用pyomo或其他python建模库可以获得与GAMS或其他AML语言相当的性能吗?

Do you think it is possible to obtain performance comparable to GAMS or other AML languages with pyomo or other python modelling library ?

在此先感谢您的帮助！

推荐答案

尽管在创建Pyomo约束时可以使用NumPy数据，但是当前无法使用Pyomo在单个NumPy样式的命令中创建约束块.毫无疑问，我不相信您可以使用AMPL或GAMS等语言.虽然Pyomo最终可能会支持用户使用矩阵和向量运算来定义约束，但是该接口不太可能避免生成单独的约束，因为求解器接口(例如NL，LP，MPS文件)都是明确的扁平"表示形式代表个人约束.这是因为Pyomo需要显式生成代数的表示形式(即表达式)以发送给求解器.相比之下，NumPy只需计算:通过在C/C ++后端(即Python中的 not )创建数据来提高效率，而依赖于低级BLAS运算可有效地计算结果，并且仅将结果带回Python.

While you can use NumPy data when creating Pyomo constraints, you cannot currently create blocks of constraints in a single NumPy-style command with Pyomo. Fow what it's worth, I don't believe that you can in languages like AMPL or GAMS, either. While Pyomo may eventually support users defining constraints using matrix and vector operations, it is not likely that that interface would avoid generating the individual constraints, as the solver interfaces (e.g., NL, LP, MPS files) are all "flat" representations that explicit represent individual constraints. This is because Pyomo needs to explicitly generate representations of the algebra (i.e., the expressions) to send out to the solvers. In contrast, NumPy only has to calculate the result: it gets its efficiency by creating the data in a C/C++ backend (i.e., not in Python), relying on low-level BLAS operations to compute the results efficiently, and only bringing the result back to Python.

就性能和可伸缩性而言，我生成的原始模型具有超过13e6的变量和21e6的约束.话虽如此，Pyomo旨在提供灵活性和可扩展性，并超越速度.使用cPython时，Pyomo中的运行时可能比AMPL慢一个数量级(尽管使用pypy可以缩小到4或5倍).至少从历史上看，AMPL比GAMS快，因此Pyomo和GAMS之间的差距应该更小.

As far as performance and scalability goes, I have generated raw models with over 13e6 variables and 21e6 constraints. That said, Pyomo was designed for flexibility and extensibility over speed. Runtimes in Pyomo can be an order of magnitude slower than AMPL when using cPython (although that can shrink to within a factor of 4 or 5 using pypy). At least historically, AMPL has been faster than GAMS, so the gap between Pyomo and GAMS should be smaller.

这篇关于pyomo生成具有大量约束的模型的性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！