100k X 100k数据处理摘要 [英] 100k X 100k data processing summary
问题描述
通过之前的回复，似乎以下方法以某种方式解决了高达1000 * 1000 2D数据的问题，但是当我尝试10k * 10k时，
分段故障问题再次出现。
理查德托宾告诉我，系统限制可以改变。但是我
我不知道要更改哪个文件。
我一次又一次地修改过，并希望找到一个可以解决的问题
处理100k * 100k数据。
float ** array_to_matrix（float * m，int rows，int cols）{
int i，j;
float ** r;
r =（float **）calloc（rows，sizeof（float *））;
for（i = 0; i< rows; i ++）
{
r [i] =（float *）calloc （cols，sizeof（float））;
for（j = 0; j< cols; j ++）
r [i] [j] = m [ i * cols + j];
}
返回r;
}
By previous replies, it seems that the following method somehow solves the
problem up to 1000 * 1000 2D data, but when I try 10k * 10k, the
segmentation fault problem appears again.
Richard Tobin told me there is a system limit that can be changed. But I
don''t know which file is to be changed.
I have modified again and again and hope to find out a solution that can
handle 100k * 100k data.
float** array_to_matrix(float* m, int rows, int cols) {
int i,j;
float** r;
r = (float**)calloc(rows,sizeof(float*));
for(i=0;i<rows;i++)
{
r[i] = (float*)calloc(cols,sizeof(float));
for(j=0;j<cols;j++)
r[i][j] = m[i*cols+j];
}
return r;
}
推荐答案
a写道：
a wrote:
通过以前的回复，似乎以下方法以某种方式解决了
问题高达1000 * 1000 2D数据，但当我尝试10k * 10k时，再次出现
分段故障问题。
By previous replies, it seems that the following method somehow solves the
problem up to 1000 * 1000 2D data, but when I try 10k * 10k, the
segmentation fault problem appears again.
系统如果你尝试t，记忆是有限的o分配超过
可用，你将失败。
System memory is finite, if you attempt to allocate more than there is
available, you will fail.
Richard Tobin告诉我有一个系统限制可以更改。但是我
我不知道要更改哪个文件。
我一次又一次地修改过，并希望找到一个可以解决的问题
处理100k * 100k数据。
Richard Tobin told me there is a system limit that can be changed. But I
don''t know which file is to be changed.
I have modified again and again and hope to find out a solution that can
handle 100k * 100k data.
这是10GB * sizeof（浮点数），你有那么多（虚拟）内存吗？ />
玩什么？
听起来你有比C更多的算法问题。
Which is 10GB*sizeof(float), do you have that much (virtual) memory to
play with?
It sound like you have more of an algorithm problem than a C one.
float ** array_to_matrix（float * m，int rows，int cols）{
int i，j;
float ** r;
r =（float **）calloc（rows，sizeof（float *））;
float** array_to_matrix(float* m, int rows, int cols) {
int i,j;
float** r;
r = (float**)calloc(rows,sizeof(float*));
放弃演员表。
-
Ian Collins。
Drop the cast.
--
Ian Collins.
Ian Collins写道：
Ian Collins wrote:
a写道：
a wrote:
>通过以前的回复，似乎以下方法以某种方式解决了高达1000 * 1000的问题2D数据，但是当我尝试10k * 10k，
分段故障问题再次出现。
>By previous replies, it seems that the following method somehow solves the
problem up to 1000 * 1000 2D data, but when I try 10k * 10k, the
segmentation fault problem appears again.
系统内存是有限的，如果你试图分配超过
可用，你将失败。
System memory is finite, if you attempt to allocate more than there is
available, you will fail.
> Richard Tobin告诉我有一个系统限制可以更改。但是我不知道要更改哪个文件。
我一次又一次地修改过，希望找到一个可以处理100k * 100k数据的解决方案。
>Richard Tobin told me there is a system limit that can be changed. But I
don''t know which file is to be changed.
I have modified again and again and hope to find out a solution that can
handle 100k * 100k data.
这是10GB * sizeof（浮点数），你有多少（虚拟）内存来玩
玩吗？
听起来你有比C更多的算法问题。
Which is 10GB*sizeof(float), do you have that much (virtual) memory to
play with?
It sound like you have more of an algorithm problem than a C one.
> float ** array_to_matrix（float * m ，int rows，int cols）{
int i，j;
float ** r;
r =（float **）calloc（rows，sizeof） （浮动*））;
>float** array_to_matrix(float* m, int rows, int cols) {
int i,j;
float** r;
r = (float**)calloc(rows,sizeof(float*));
放弃演员。
Drop the cast.
并且总是检查[mc]的返回] alloc isn''t null。
-
Ian Collins。
And always check the return of [mc]alloc isn''t null.
--
Ian Collins.
文章< fi ********** @ justice.itsc.cuhk.edu.hk> ;, a< a@a.comwrote：
In article <fi**********@justice.itsc.cuhk.edu.hk>, a <a@a.comwrote:
>理查德托宾告诉我，有一个系统限制可以改变。但是我不知道要更改哪个文件。
>Richard Tobin told me there is a system limit that can be changed. But I
don''t know which file is to be changed.
正如我所说，请询问您的系统管理员。或者阅读手册。
我们无法告诉你，因为你甚至没有告诉我们你使用的是什么系统
。
As I said, ask your system administrator. Or read the manual.
We can''t tell you, because you haven''t even told us what system
you''re using.
>我一次又一次地修改过，希望找到一个能够处理100k * 100k数据的解决方案。
>I have modified again and again and hope to find out a solution that can
handle 100k * 100k data.
100k * 100k是10g，如果浮动是4字节，那就是40G。
你真的需要一台超级计算机那。也许你应该重新考虑你的算法，或者等待几年。
- Richard
-
在一些字母表中应考虑多达32个字符的需要
- 1963年的X3.4。
100k * 100k is 10g, and if floats are 4 bytes that''s 40 gigabytes.
You really will need a supercomputer for that. Perhaps you should
reconsider your algorithm, or wait several years.
-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
这篇关于100k X 100k数据处理摘要的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
