如何增强lpsolve R优化解决方案以在hadoop集群上运行？ [英] How to augment lpsolve R optimization solution to run on a hadoop cluster?
本文介绍了如何增强lpsolve R优化解决方案以在hadoop集群上运行？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我使用R lpsolve软件包来优化我的运输模型。我的代码运行良好，但需要很长时间才能运行，因为我有大量的节点和路径。我打算通过hadoop集群运行我的代码。
请指导我关于我需要对我的代码进行的更改。我认为在hadoop集群上运行优化可能是不可能的，因为我们可能会以局部最小值而不是全局最小值作为结果。
我在互联网上搜索lpsolve hadoop但没有得到任何有用的东西。
请指导我看看我应该看的材料或例子。
=====================================更新1 ======= ===============================
我的原始问题是给出了一个简单的问题。有没有人有想法如何使用Mapreduce解决它？我的意思是让我们说我有一个类似的问题，但有大量的变量和约束，那么是否有办法实现更快的处理？解析方案
简而言之，你想大规模地进行线性规划（lp），并且你对解算器的性能不满意。
方法 h2>
我建议采用以下方法。
求解器
你需要使用这个求解器吗？以下是一些替代方案：
-
-
- CPlex
并行执行
实现并行处理的一个弱方法是并行运行进程。你能一次推出几个优点吗？ （在这里，你可以在Hadoop中写一个简单的 exectuion 任务）
$ b $ h3改进的输入
如果你相应地处理数据（稀疏矩阵，...），求解器可以执行得更好。
在你的代码中使用 lpSolve :: LP 。根据 lpSolver 5.6.7的CRAN页面及其<一个href =http://cran.r-project.org/web/packages/lpSolve/lpSolve.pdf =nofollow> PDF ，运输问题有一个特殊模式 lpSolve：lp.transport 。 （我没有和R一起工作，我对语法不熟悉，我是一个Matlab爱好者。）
最新研究
也许一些狂热是在学术界度过的。您可以在上找到一篇论文（花费一些美元）。
硬件
CPU越快，获得解答（答案）就越快。你介意给我们一些关于你的环境（OS，CPU）的见解吗？
试图在 quant.stackexchange.com 中提出您的问题，因为数值优化与数学紧密相关，不涉及太多的编程。并行模式
我们使用 CLP 。根据一些同事你可以设置一个标志或使用不同的实现来并行运行求解器（在许多内核上执行1个问题）。看看SYMPHONY。
$ b
保持我们的发布，您是否介意给我们提供关于您的环境的一些见解（操作系统，CPU）？ 我的赏金
On Quora you find your answer that Hadoop is good in data processing, fast in I/O, not specially designed for optimization or calculation: Quora
Please let me know, if you do not want to award me with the bounties .
Missing bounty of +100
I have not received any bounties even though I did some research.. . This thread comes to a point where you do not give us the necessary details and we should solve it for you - this never gonna work out!
I am using R lpsolve package to optimize my transportation model. My code runs fine but it takes a lot of time to run as I have huge number of nodes and paths. I am planning to run my code over hadoop cluster.
Please guide me regarding changes that i need to make to my code. I think that running optimization over hadoop cluster might be impossible as we might end up with local minimums instead of the global minimum.
I search internet for terms like "lpsolve hadoop" but didn't get anything useful.
Please direct me to material or examples that i should look at. =====================================update 1====================================== The original problem that I had is here.
I simplified the problem further and the current problem that I am solving is as below.
The R code and the input data file that I have built using excel is attached. In real scenario, input data file will be generated using SQL and will be >30,000 rows in length.
My input excel file:
startlink endlink link_dsc lnk_type cons_type cost equality_const fpc_const max_const
"source","-","-","0" "vmi1","MM1","VMI","1" source_to_VMI supply equality supply 0 100 null null
"vmi1","MM1","VMI","0" "cust1","MM1","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","1" "cust1","MM1","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","2" "cust1","MM1","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","3" "cust1","MM1","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","4" "cust1","MM1","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","5" "cust1","MM1","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","6" "cust1","MM1","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","7" "cust1","MM1","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","8" "cust1","MM1","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","9" "cust1","MM1","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","0" "vmi1","MM1","VMI","1" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","1" "vmi1","MM1","VMI","2" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","2" "vmi1","MM1","VMI","3" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","3" "vmi1","MM1","VMI","4" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","4" "vmi1","MM1","VMI","5" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","5" "vmi1","MM1","VMI","6" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","6" "vmi1","MM1","VMI","7" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","7" "vmi1","MM1","VMI","8" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","8" "vmi1","MM1","VMI","9" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","9" "vmi1","MM1","VMI","10" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","10" "SINK","-","-","100" vmi_to_sink esacpe null 100 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","1" "cust1","MM1","CUST","1" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","2" "cust1","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","3" "cust1","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","4" "cust1","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","5" "cust1","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","6" "cust1","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","7" "cust1","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","8" "cust1","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","9" "cust1","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","10" "cust1","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"cust1","MM1","CUST","4" "SINK","-","-","100" cust_to_sink flow demand 0 null null 50
"vmi1","MM1","VMI","0" "cust1","MM1","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","1" "cust1","MM1","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","2" "cust1","MM1","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","3" "cust1","MM1","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","4" "cust1","MM1","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","5" "cust1","MM1","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","6" "cust1","MM1","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","7" "cust1","MM1","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","8" "cust1","MM1","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 5 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","2" "cust1","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","3" "cust1","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","4" "cust1","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","5" "cust1","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","6" "cust1","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","7" "cust1","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","8" "cust1","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","9" "cust1","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","10" "cust1","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"source","-","-","0" "vmi2","MM2","VMI","2" source_to_VMI supply equality supply 0 50 null null
"vmi2","MM2","VMI","0" "cust1","MM2","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","1" "cust1","MM2","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","2" "cust1","MM2","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","3" "cust1","MM2","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","4" "cust1","MM2","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","5" "cust1","MM2","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","6" "cust1","MM2","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","7" "cust1","MM2","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","8" "cust1","MM2","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","9" "cust1","MM2","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","0" "vmi2","MM2","VMI","1" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","1" "vmi2","MM2","VMI","2" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","2" "vmi2","MM2","VMI","3" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","3" "vmi2","MM2","VMI","4" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","4" "vmi2","MM2","VMI","5" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","5" "vmi2","MM2","VMI","6" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","6" "vmi2","MM2","VMI","7" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","7" "vmi2","MM2","VMI","8" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","8" "vmi2","MM2","VMI","9" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","9" "vmi2","MM2","VMI","10" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","10" "SINK","-","-","100" vmi_to_sink esacpe null 100 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","1" "cust1","MM2","CUST","1" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","2" "cust1","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","3" "cust1","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","4" "cust1","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","5" "cust1","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","6" "cust1","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","7" "cust1","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","8" "cust1","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","9" "cust1","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","10" "cust1","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"cust1","MM2","CUST","9" "SINK","-","-","100" cust_to_sink flow demand 0 null null 10
"vmi2","MM2","VMI","0" "cust1","MM2","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","1" "cust1","MM2","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","2" "cust1","MM2","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","3" "cust1","MM2","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","4" "cust1","MM2","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","5" "cust1","MM2","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","6" "cust1","MM2","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","7" "cust1","MM2","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","8" "cust1","MM2","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 1.4 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","2" "cust1","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","3" "cust1","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","4" "cust1","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","5" "cust1","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","6" "cust1","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","7" "cust1","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","8" "cust1","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","9" "cust1","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","10" "cust1","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"vmi1","MM1","VMI","0" "cust3","MM1","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","1" "cust3","MM1","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","2" "cust3","MM1","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","3" "cust3","MM1","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","4" "cust3","MM1","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","5" "cust3","MM1","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","6" "cust3","MM1","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","7" "cust3","MM1","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","8" "cust3","MM1","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","9" "cust3","MM1","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 15 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","1" "cust3","MM1","CUST","1" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","2" "cust3","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","3" "cust3","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","4" "cust3","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","5" "cust3","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","6" "cust3","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","7" "cust3","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","8" "cust3","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","9" "cust3","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","10" "cust3","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"cust3","MM1","CUST","6" "SINK","-","-","100" cust_to_sink flow demand 0 null null 5
"vmi1","MM1","VMI","0" "cust3","MM1","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","1" "cust3","MM1","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","2" "cust3","MM1","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","3" "cust3","MM1","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","4" "cust3","MM1","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","5" "cust3","MM1","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","6" "cust3","MM1","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","7" "cust3","MM1","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","8" "cust3","MM1","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 15 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","2" "cust3","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","3" "cust3","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","4" "cust3","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","5" "cust3","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","6" "cust3","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","7" "cust3","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","8" "cust3","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","9" "cust3","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","10" "cust3","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"vmi2","MM2","VMI","0" "cust3","MM2","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","1" "cust3","MM2","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","2" "cust3","MM2","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","3" "cust3","MM2","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","4" "cust3","MM2","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","5" "cust3","MM2","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","6" "cust3","MM2","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","7" "cust3","MM2","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","8" "cust3","MM2","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","9" "cust3","MM2","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 1.8 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","1" "cust3","MM2","CUST","1" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","2" "cust3","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","3" "cust3","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","4" "cust3","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","5" "cust3","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","6" "cust3","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","7" "cust3","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","8" "cust3","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","9" "cust3","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","10" "cust3","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"cust3","MM2","CUST","8" "SINK","-","-","100" cust_to_sink flow demand 0 null null 7
"vmi2","MM2","VMI","0" "cust3","MM2","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","1" "cust3","MM2","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","2" "cust3","MM2","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","3" "cust3","MM2","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","4" "cust3","MM2","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","5" "cust3","MM2","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","6" "cust3","MM2","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","7" "cust3","MM2","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","8" "cust3","MM2","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 1.8 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","2" "cust3","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","3" "cust3","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","4" "cust3","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","5" "cust3","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","6" "cust3","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","7" "cust3","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","8" "cust3","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","9" "cust3","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","10" "cust3","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"SINK","-","-","100" "source","-","-","0" closed_loop flow null 0 null null null
My R code is as below: You have to take above csv data and paste it in file C:/dumy_network.csv.
My R code might not be very efficient but it serves the purpose!
library("lpSolve", lib.loc="C:/Users/njog/Documents/R/win-library/3.0")
#get the data from CSV file
mydata <- read.csv("C:/dumy_network.csv", header=TRUE)
#build list of nodes (no repetition)
nodes=unique(c(as.character(mydata$startlink),as.character(mydata$endlink)))
#list of all links
links=mydata[,1:2]
#cost of moving material on each link - optimization problem objective coefficients
cost=mydata$cost
#decision variable is flow on each link. Objective is to minimize product of (cost on each link*flow on the link). Therefore, count of decision variable is equal to count of links.
#constraints matrix: for each node in nodes, incoming quantity should be equal to outgoing quantity.
constraints=matrix(0,sum(mydata$max_const!='null')+sum(mydata$equality_const!='null')+length(nodes),length(mydata$endlink))
for (i in 1:length(nodes) ) {
constraints[i,]=t(1*(nodes[i]==links[,1])-1*(nodes[i]==links[,2]))
}
#get constraints for equality constraints- in some cases we have to ship material exactly same as this quanity.
constraint1=matrix(mydata$equality_const,1,length(mydata$equality_const))
constraint1[constraint1=="null"]=0
constraint1=as.numeric(constraint1)
constraint1_length=which(constraint1!=0)
constraint1_final=matrix(0,length(constraint1_length),length(mydata$equality_const))
for (i in 1:length(constraint1_length) ) {
constraint1_final[i,constraint1_length[i]]=1
}
start=length(nodes)+1
end=length(nodes)+length(constraint1_length)
constraints[start:end,]=constraint1_final
#get constraints for maxconstraints - in some cases we cannot ship material exceeding this quanity.
constraint2=matrix(mydata$max_const,1,length(mydata$max_const))
constraint2[constraint2=="null"]=0
constraint2=as.numeric(constraint2)
constraint2_length=which(constraint2!=0)
constraint2_final=matrix(0,length(constraint2_length),length(mydata$max_const))
for (i in 1:length(constraint2_length) ) {
constraint2_final[i,constraint2_length[i]]=1
}
start=end+1
end=end+length(constraint2_length)
constraints[start:end,]=constraint2_final
#building direction of constraints
direction=c(rep("=",length(nodes)),rep("=",sum(mydata$equality_const!='null')),rep("<=",sum(mydata$max_const!='null')))
#building right hand side of constraints
b1=as.numeric(as.character(mydata$equality_const[constraint1_length]))
b2=as.numeric(as.character(mydata$max_const[constraint2_length]))
b=c(rep(0,length(nodes)),b1,b2)
res = lpSolve::lp('min', cost, constraints, direction, b, all.int = TRUE)
res$solution
answers1=data.frame(res$solution)
mydata=cbind(mydata,answers1)
=====================================update 2====================================== Not getting any answers, so trying to simplify my question:
Examples section of the page gives a simple problem. Does anyone has ideas how to solve it using Mapreduce? I mean let's say I have a similar problem ,but with a huge number of variables and constraints then is there a way to achieve faster processing?
解决方案
In short, you want to do linear programming (lp), in large scale and you are not happy with the performance of your solver.
Approaches
I suggest the following approaches.
Solver
Do you need to use this solver? Here are some alternatives:
- Gurobi
- Mosek
- CPlex
Parallel execution
A weak approach to achiev parallel processing is to run process in parallel. Can you launch several optimations at once? (Here, you could write a simple exectuion task in Hadoop)
Improved inputs
The solvers can perform much better if you hand in the data accordingly (sparse matrix, ...)
In your code you use lpSolve::lp. According to the CRAN page of lpSolver 5.6.7 and its PDF, there is a special mode for the transportation probelm lpSolve:lp.transport. (I haven't worked with R and I am not familiar with the syntax. I am a Matlab "enthusiast".)
Latest research
Maybe some engergy is spent in academia. You might find a paper (cost some dollars) on sciencedirect.com.
Hardware
The faster the CPU, the faster your get a soluation (answer). Do you mind giving us some insights on your environment (OS, CPU)?
Final thought
Have you tried to raise your question in quant.stackexchange.com because numerical optimization is closely related to mathematics and "does not involve that much programming".
Parallel mode
We use the CLP. According some colleagues you can set a flag or use a different implementation to run the solver in parallel (1 problem executed on many cores). Take a look at SYMPHONY.
Keep us posted and do you mind giving us some insights on your environment (OS, CPU)?
My bounty
On Quora you find your answer that Hadoop is good in data processing, fast in I/O, not specially designed for optimization or calculation: Quora
Please let me know, if you do not want to award me with the bounties.
Missing bounty of +100
I have not received any bounties even though I did some research... This thread comes to a point where you do not give us the necessary details and we should solve it for you - this never gonna work out!
这篇关于如何增强lpsolve R优化解决方案以在hadoop集群上运行？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
