管理许多小物件 [英] management of many small objects

问题描述

亲爱的名单，


目前尚不清楚标题的含义。 :-)详情如下：


我需要管理一大堆小物件，比如


struct {

int a;

int b [size_undetermined];

} obj;


我不想用通常的int *，new，delete方法，因为

1.一百万新和删除会很慢

2.我需要访问b跨所有对象的元素。可能有更好的方法来循环obj [i] .b [2]。

3. b的大小通常很小所以额外的int的开销*

太大。


我目前正在使用vector< int>（num_objects *（1 + size_of_b））来存储

对象和向量< int> :: iterator访问元素。但是

的问题是：


1.我想将obj的参考传递给其他函数，但我只有

有vector< int> :: iterator，not vector< obj> :: iter。

2.如果我想扩展obj（例如添加另一个变量），我的代码

因索引变化而必须重写。


有没有更好的方法呢？我可以以某种方式创建一个动态类型

struct {int a，int b [size];并应用于现有的向量？

boost :: memorypool似乎是一个好主意，但使用指针仍然浪费了

内存。


非常感谢提前。

Bo

Dear List,

It is not clear what the title means. :-) Here is the details:

I need to manage a big bunch of small objects, like

struct{
int a;
int b[size_undetermined];
}obj;

I do not want to use the usual int*, new, delete method since
1. a million new and delete would be slow
2. I need to access elements of b cross all objects. There might be
better ways to loop through obj[i].b[2].
3. the size of b is usually small so the overhead of an additional int *
is too big.

I am currently using vector<int>( num_objects*(1+size_of_b) ) to store
the objects and vector<int>::iterator to access elements. But the
problems are:

1. I would like to pass reference of obj to other functions but I only
have vector<int>::iterator, not vector<obj>::iter.
2. If I would like to expand obj (e.g. adding another variable), my code
has to be rewritten because of index changes.

Is there a better way to do it? Can I somehow create a dynamic type
struct{ int a, int b[size]; } and apply to the existing vector?
boost::memorypool seems to be a good idea but there is still a waste of
memory for using pointers.

Many thanks in advance.
Bo

推荐答案

柏鹏 <沸点*** @ rice.edu>在消息中写道

news：cc ********** @ joe.rice.edu ...

"Bo Peng" <bp***@rice.edu> wrote in message
news:cc**********@joe.rice.edu...

亲爱的名单，

我需要管理一大堆小物件，例如

struct {


我会给你的类型一个名字供以下使用


struct T {

int a;
int b [size_undetermined];


如果在编译时无法确定大小，

那么你就不能使用这种语法。你将不得不使用指针和动态分配
。但是，

我建议在这里使用一个容器。

} obj;

我不想使用通常的int *，new，delete方法自
1.一百万新的删除会很慢
2.我需要访问b跨所有对象的元素。可能有更好的方法来循环obj [i] .b [2]。
3. b的大小通常很小所以额外的int *
的开销太大了。


嗯，一个int通常与指针的大小相同，所以除非你的数组通常只有0到零，否则开销不会很多。三个

元素左右。

我目前正在使用vector< int>（num_objects *（1 + size_of_b））



嗯？为什么？为什么不简单：


vector< struct T>（num_objects）;


存储
对象和向量< int> :: iterator


你没有存储结构类型的任何对象，只是整数。


不要存储迭代器在一个容器中。没有必要。

来访问元素。但是问题是：

1.我想把obj的引用传递给其他函数，但我只有
有vector< int> :: iterator，而不是vector< obj> ; :: ITER。


''obj''不是类型名称，所以你不能这样做。但是，再次，

为什么不：


vector< struct T>


2.如果我想展开obj（例如添加另一个变量），由于索引更改，我的代码必须重写。


在你的结构中添加另一个成员不会改变任何索引，

它只会增加你结构的大小。
struct {int a，int b [size]; }


struct T

{

int a;

std :: vector< int> b;

T（std :: vector< int> :: size_type size）：b（大小）{}

};


int main（）

{

T cont（how_many）;

返回0;

}

并且适用于现有的向量？
boost :: memorypool似乎是一个好主意，但使用指针仍然浪费了内存。
<非常感谢提前。
Bo

Dear List,

It is not clear what the title means. :-) Here is the details:

I need to manage a big bunch of small objects, like

struct{
I''ll give your type a name for use below

struct T {
int a;
int b[size_undetermined];
If the size can not be determined at compile time,
then you can''t use this syntax. You''ll have to
use a pointer and dynamic allocation. However,
I suggest using a container here.
}obj;

I do not want to use the usual int*, new, delete method since
1. a million new and delete would be slow
2. I need to access elements of b cross all objects. There might be
better ways to loop through obj[i].b[2].
3. the size of b is usually small so the overhead of an additional int *
is too big.
Well, an int is often the same size as a pointer, so there''s not
that much overhead unless your array usually has only zero to three
elements or so.

I am currently using vector<int>( num_objects*(1+size_of_b) )

Huh? Why? Why not simply:

vector<struct T>(num_objects);

to store
the objects and vector<int>::iterator
You''re not storing any object of your struct type, just ints.

Don''t store an iterator in a container. There''s no need.
to access elements. But the
problems are:

1. I would like to pass reference of obj to other functions but I only
have vector<int>::iterator, not vector<obj>::iter.
''obj'' is not a type name, so you can''t do that. But again,
why not:

vector<struct T>

2. If I would like to expand obj (e.g. adding another variable), my code
has to be rewritten because of index changes.
Adding another member to your struct won''t change any indexes,
it will only increase the size of your struct.

Is there a better way to do it? Can I somehow create a dynamic type
struct{ int a, int b[size]; }
struct T
{
int a;
std::vector<int> b;
T(std::vector<int>::size_type size) : b(size) {}
};

int main()
{
T cont(how_many);
return 0;
}
and apply to the existing vector?
boost::memorypool seems to be a good idea but there is still a waste of
memory for using pointers.

Many thanks in advance.
Bo

-Mike

-Mike

迈克，


感谢您的回复。我当然知道所有的语法细节以及我展示的只是某种psudo代码的内容。


问题的核心是我会有一百万左右的小b / b $ b对象有一些固定元素和一个数组。大多数数据是

实际上是unsigned char（不是int）所以指针比它大4倍。

假设b的长度是8，那么将有1000对象。实际的

datasize是（4 + 8）* 1000 = 12000。比较：


解决方案1：最方便。


gcc中矢量的实现使用三个指针。


struct T {

int a; // 4个字节

vector< unsigned char> b; // 3个指针+8个真实数据= 20个字节

};


vector< T> myarray（size）将使用1000 new和delete（除非STL使用

在其分配器中使用其他内容）+ 24 * 1000内存使用量（50％浪费）。


解决方案2：中等难度。


struct T {

int a; // 4个字节

unsigned char * b; // b =新的unsigned char [8]，12个字节

}


vector< T> myarray（大小）将使用1000新增和删除+ 16 * 1000内存

使用（25％浪费）


解决方案3：最难控制。将所有数据放在一个大块中。


const int blocksize = sizeof（int）/ sizeof（unsigned char）+ 8;

vector< unsigned char> ab（blocksize * size）


2新增和删除+ 12 * 1000内存使用量（0％浪费）


由于性质程序（内存饥饿），我无法容忍

50％的内存浪费。我不喜欢解决方案2，因为有太多新的和

删除，我想


for（int i = 0; i< 1000; i ++）

myarray [i] .b [2] = 2 //每次取消引用两个指针。


比解决方案3中的慢。


for（int i = 0; i< 1000 * blocksize; i + = blocksize）

ab [i] = 2;

但是管理一块没有数据结构的内存块确实很痛苦所以我所要求的是（几乎没有希望）一种方法将数据结构应用于数据块。例如


T * myobj = reinterpret_cast< T *> （& ab [i]）;

myobj-> a = 5;

myobj-> b [2] = 2; //或其他方面


甚至更好

vector< T> :: iterator it =（某种类型的演员）ab.begin（）;


当然T型在这里无法定义，因为T的大小不能预定...... b $ b预定......


还有其他解决方案（用户定义的固定大小
向量，内存池），但我找不到一个可接受的权衡

介于内存使用，性能（少了新/删除，快速数据访问）

和易于使用...


谢谢。


Bo

Hi, Mike,

Thanks for your reply. I certainly know all the grammar details and what
I showed was just some kind of psudo-code.

The core of the problem is that I will have a million or so small
objects with some fixed elements and an array. Most of the data is
actually unsigned char (not int) so a pointer is 4 times bigger than it.
Suppose the length of b is 8 and there will be 1000 objects. The actual
datasize is (4+8)*1000=12000. Compare:

solution 1: Most convenient.

The implmentation of vector in gcc uses three pointers.

struct T{
int a; // 4 bytes
vector<unsigned char> b; // 3 pointers + 8 real data = 20 bytes
};

vector<T> myarray(size) will use 1000 new and delete (unless STL uses
something else in their allocator) + 24*1000 memory usage (50% waste).

Solution 2: Medium difficulty.

struct T{
int a; // 4 bytes
unsigned char * b; // b = new unsigned char[8] , 12 bytes
}

vector<T> myarray(size) will use 1000 new and delete + 16*1000 memory
usage ( 25% waste )

Solution 3: Most difficult to control. Putting all data in a big block.

const int blocksize = sizeof(int)/sizeof(unsigned char) + 8;
vector<unsigned char> ab( blocksize*size)

2 new and delete + 12*1000 memory usage (0% waste)

Because of the nature of the program (memory hungry), I can not tolerate
50% memory waste. I dislike solution 2 since there are too many new and
deletes and I guess

for (int i=0; i< 1000; i++)
myarray[i].b[2] = 2 // de-reference two pointers each time.

is slower than that in solution 3.

for (int i=0; i<1000*blocksize; i+=blocksize)
ab[i] = 2;
However, managing a block of memory without data structure is really
painful so what I was asking is (almost without hope) a way to apply
data structure to a block of data. For example

T* myobj = reinterpret_cast<T*> (&ab[i]) ;
myobj->a = 5;
myobj->b[2] = 2; // or someway else

or even better
vector<T>::iterator it = (some kind of cast of) ab.begin();

Of course type T here can not be defined since the size of T can not be
pre-determined......

There are other solutions hanging around (user-defined fixed-size
vector, memory pool) but I have not find one with acceptable trade-off
between memory-usage, performance (less new/delete, quick data access)
and easy-to-use ...

Thanks.

Bo

" Bo Peng" <沸点*** @ rice.edu>写道：

"Bo Peng" <bp***@rice.edu> wrote:

struct {
int a;
int b [size_undetermined];
} obj;

struct{
int a;
int b[size_undetermined];
}obj;

哎呀。那不是一个很好的结构。我认为

对你来说会更好：


// MyFile.cpp：


#include< iostream>

#include< vector>


使用std :: vector;


struct MyStruct

{

int a;

vector< int> b;

};


int main（无效）

{

使用std :: cout;

使用std :: endl;


// InitSize是对象容器的初始大小：

const int InitSize = 10;


vector< MyStruct> Objs（InitSize）;


{//限制范围

MyStruct Splat; //临时对象

Splat.a = 13;

Splat.b.push_back（22）;

Splat.b.push_back（10 ）;

Objs [0] = Splat;

} //这里没有创建Splat


{//限制范围

MyStruct Splat; //临时对象

Splat.a = 97;

Splat.b.push_back（33）;

Splat.b.push_back（84 ）;

Splat.b.push_back（27）;

Objs [1] = Splat;

} //这里没有创建Splat


vector< MyStruct> :: size_type i; // Objs的索引

vector< int> :: iterator j; //迭代元素b


for（i = 0; i< 2; ++ i）

{

cout<< 对象编号： << i<< endl;

cout<< a的价值： << Objs [i] .a<< endl;

cout<< b的元素： << endl;

for（j = Objs [i] .b.begin（）; j！= Objs [i] .b.end（）; ++ j）

{

cout<< （* j）<< endl;

}

cout<< endl<<结束;

}

返回0;

}


当然，你会需要处理诸如是否以空的或预先大小的容器开始的问题，以及是否使用

向量或其他容器。如果您经常更改大小

或插入/删除元素，那么列表或双端队列可以更好地工作

。取决于你的申请细节。


希望上面给你一些有用的想法。


-

干杯，

Robbie Hatley

美国加利福尼亚州塔斯廷

电子邮件：来自pacbell dot net的lonewolfintj

web ：home dot pacbell dot net slant earnur slant


---- ==发表于Newsfeed.Com - Unlimited-Uncensored-Secure Usenet News == ----
http://www.newsfeed.com ＃1新闻组服务世界！ > 100,000新闻组

--- = 19东/西海岸专业服务器 - 通过加密的总隐私= ---

Yuck. That''s not a very good construct. I think the
following kind of thing will work much better for you:

// MyFile.cpp:

#include <iostream>
#include <vector>

using std::vector;

struct MyStruct
{
int a;
vector<int> b;
};

int main(void)
{
using std::cout;
using std::endl;

// InitSize is initial size of container of objects:
const int InitSize = 10;

vector<MyStruct> Objs (InitSize);

{ // Limit scope
MyStruct Splat; // Temporary object
Splat.a = 13;
Splat.b.push_back(22);
Splat.b.push_back(10);
Objs[0] = Splat;
} // Splat is uncreated here

{ // Limit scope
MyStruct Splat; // Temporary object
Splat.a = 97;
Splat.b.push_back(33);
Splat.b.push_back(84);
Splat.b.push_back(27);
Objs[1] = Splat;
} // Splat is uncreated here

vector<MyStruct>::size_type i; // Index to Objs
vector<int>::iterator j; // Iterator to elements of b

for (i = 0; i <2; ++i)
{
cout << "Object Number: " << i << endl;
cout << "Value of a: " << Objs[i].a << endl;
cout << "Elements of b:" << endl;
for (j = Objs[i].b.begin(); j != Objs[i].b.end(); ++j)
{
cout << (*j) << endl;
}
cout << endl << endl;
}
return 0;
}

Of course, you''ll need to handle issues such as whether to
start with empty or pre-sized containers, and whether to use
vectors or some other containers. If you change sizes
or insert/delete elements often, then a list or deque may work
better. Depends on the details of your application.

Hope the above gives you some useful ideas.

--
Cheers,
Robbie Hatley
Tustin, CA, USA
email: lonewolfintj at pacbell dot net
web: home dot pacbell dot net slant earnur slant


----== Posted via Newsfeed.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeed.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= 19 East/West-Coast Specialized Servers - Total Privacy via Encryption =---

这篇关于管理许多小物件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！