在Visual Studio 2010中使用大型查找表的代码库链接器的局限性 [英] Limitations of Linker with Codebase using Large Lookup Tables in Visual Studio 2010
问题描述
在我的工作中，我们有各种各样的大表存储用于一组多维非参数模型的数据.每个表都是一个float数组，大小通常为200,000至5,000,000个元素.
In my work, we have a variety of large tables storing data used for a set of multidimensional nonparametric models. Each table is a float array with a size of typically 200,000 to 5,000,000 elements.
今天，当我发现项目的编译和链接导致 Microsoft增量链接器已停止工作时，我打算对该代码库进行通常的琐碎更新，从而更新了一组查找表. strong>，这是我之前从未见过的.请注意，我正在更新的表从大约290,000个元素的大小增长到每个接近10,000,000个元素.
Today, I was going about a normally trivial update to this codebase, updating a set of the lookup tables, when I found the compiling and linking of the project was resulting in a Microsoft Incremental Linker has Stopped Working, something I had not seen before. Note that the tables I was updating were growing from a size of around 290,000 elements to near 10,000,000 elements each.
我搜索并找到了人们建议使用增量链接程序"弹出窗口解决此问题的方法，但没有任何方法可以解决.我什至接受了该项目并将其集成到VS 2012中，并使其同样失败.
I searched and found methods people recommended to tackle the problem with the Incremental Linker pop-up, but nothing fixed it. I even took the project and integrated it into VS 2012 and had it fail as well.
我知道我的项目以前已经编译过，所以我删除了更新并将其恢复到原始状态.此状态一如既往地正确编译和链接.然后，我将其中一个旧表与一个新表交换，并对其进行正确的编译和链接.但是，一旦我对交换后的表进行了另一次交换，则在编译后存在同样的链接问题.
I knew my project compiled before, so I removed the updates and brought it back to its original state. This state compiled and linked properly as it always has. I then swapped one of the old tables with one of the new tables, and it compiled and linked properly. However, once I did another swap for an updated table, it had the same problem with linking after it compiled.
如前所述，我一直添加的新表每个都有大约10,000,000个元素，并且大大大于它们要更新的旧表.链接器努力使用这些大表是否可行?如果是这样，那是为什么呢?
As stated before, the new tables I have been adding have about 10,000,000 elements each and are significantly larger than the old tables they are updating. Is it feasible the linker is struggling to work with these large tables? If so, why is that?
新表可以在代码库中正常编译，只是链接器步骤失败.如果表的大小有问题，是否有任何建议可以解决此问题，而这些建议仍然可以保留名义上的建模和查找表方法?我确实认识到，从大小的角度来看，使用参数模型会更好，因为它可以压缩数据，但是我的团队目前不希望保留其传统方法.
The new tables compile fine with the codebase, it is just the linker step that fails. If the size of the tables is an issue, is there any advice as to handle this problem that would still allow for keeping the nominal modeling and lookup table approach? I do recognize using a parametric model would be better from the point of view of size since it would compress the data, but my team does not want to leave their legacy approach at this time.
请注意，每个表的代码都遵循以下几行:
Note the code for each table is something along these lines:
头文件
//
// dataset1.hpp
//
#ifndef dataset1_hpp_
#define dataset1_hpp_
namespace set1 {
extern const int tableSize;
extern const float table[];
}
#endif
源文件
//
// dataset1.cpp
//
namespace set1 {
extern const int tableSize = 10000000;
extern const float table[tableSize] = {
/*... Lots of Numbers ... */
};
}
推荐答案
在.cpp文件中，您将数据定义为extern.为什么?数据是此.cpp文件的本地数据.这在我看来很奇怪.也许您需要:
In your .cpp file you define data as extern. Why? The data is local to this .cpp file. This looks strange to me. Maybe you need:
//
// dataset1.cpp
//
namespace set1 {
const int tableSize = 10000000;
const float table[tableSize] = {
/*... Lots of Numbers ... */
};
}
我不确定这是否会有所帮助，但这值得一试.也许您只是解决这个问题.
I am not sure this will help, but this is something worth to try. Maybe you will simply walk around this problem.
在过去的工作中，我遇到了巨大的静态对象数组问题(编译器在处理来自大量顺序ctor的可能异常时遇到了问题)，但除此之外，庞大的静态数组无法正常工作.
In my past work I had problems with huge static arrays of objects (compiler had problems with handling possible exceptions from huge number of sequential ctors), but other than that huge static arrays worked fine.
为什么要使用VS2010?尝试使用最新版本(免费的Community 2015)至少要检查会发生什么.
Why are you using VS2010? Try the latest version (free Community 2015) at least just to check what will happen.
这篇关于在Visual Studio 2010中使用大型查找表的代码库链接器的局限性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
