最优解 [英] Optimal Solution
问题描述
嗨
我决定如何动画一些数字数据。到目前为止，我一直保持
图形和科学（数字运算）代码分开，我希望
喜欢。
我面临的主要问题是如何处理大量数据（通常
6000个数据点，每个点4个数量，总计24,000个数字）
在优化与努力方面。我面前的3个当前选项
是：
1）让科学代码在每个时间步骤将数据写入文件，并且
然后图形在每一步都读取它。
优点
1a）只需稍加修改现有代码即可实现最小编码
1b）保持科学与图形之间的分离
1c）直截了当的解决方案
缺点
1a）似乎有一个很高的高架。我做了一些有限的测试
并且写入然后读取数据需要大约1秒
文件。
每个数字时间步长需要1秒。所以我要加倍运行
时间
，以及3600步，从1到2小时（和1
更多
小时动画）。
1b）通过重复写/重写来分割驱动器。实际上，我不知道b $ b不知道这是一个问题，但我担心。
2）结合将代码编码成一个大代码，这样就没有阅读或
写作。
优势
2a）没有输入/输出开销
2b）没有与2a相关的时间罚款
缺点
1a）丢失关于1a），1b）和1c）的所有内容。
3）有人说如果我写信给stdout并从stdin读取，并且
管道一个
程序通过另一个，数据将保留在内存中，而不必将b
$ b写出来。换句话说：
science-a.out | animate-a.out
虽然我做了一些测试，但它似乎还需要一秒钟才能写完
写作和
读取数据。
我尝试使用带有-pg的代码分析器。国旗，但不能相当
设置
得到任何东西。
所以我的问题是：
1）应该3）给我比我似乎得到的更好的结果吗？
2）你能想到我应该尝试的其他事情吗？
我的机器是1.6 GhZ Pentium 4,1处理器，我在GNU /
Linux上编码。
图形与OpenGL / Mesa，动画与ffmpeg。
我猜测没有读/写开销，标准
问题
会需要2个小时才能运行。由于开销，需要3个小时。
谢谢。
San Le
slffea.com
Hi
I am deciding how to animate some numerical data. So far, I have kept
the graphics and science(number crunching) codes separate which I
like.
The main issue confronting me is how to handle lots of data (usually
6000 data points, 4 quantities at each point, 24,000 numbers total)
in terms of optimization vs. effort. The 3 current options before me
are:
1) Have the science code write data to a file at each time step, and
then the graphics read it in at each step.
Advantage
1a) Minimal coding with just a little modification of existing code
1b) Maintains the separation between science and graphics
1c) Straightforward solution
Disadvantage
1a) It seems there is a high overhead. I''ve done some limited tests
and it takes about 1 sec to write and then read to the data
file.
Each numerical time step takes 1 sec. So I''ll be doubling the
time
to run, and with 3600 steps, that goes from 1 to 2 hours(And 1
more
hour for animation).
1b) Fragment the drive by repeated write/re-write. Actually, I
don''t
know if this is an issue, but I worry.
2) Combine the codes into a big code so that there is no reading or
writing.
Advantage
2a) No input/output overhead
2b) No time penalty related to 2a)
Disadvantage
1a) Lose everything about 1a), 1b), and 1c).
3) Someone said that if I wrote to stdout and read from stdin, and
pipe one
program through another, the data would stay in memory without
having to
be written out. In other words:
science-a.out | animate-a.out
I did some tests though, and it still seems to take a second to
write and
read the data.
I tried using the code profiler with the "-pg" flag but can''t quite
set it up
to get anything.
So my questions are:
1) Should 3) be giving me better results than I seem to be getting?
2) Can you think of anything else I should try?
My machine is a 1.6 GhZ Pentium 4, 1 processor, and I''m coding on GNU/
Linux.
Graphics with OpenGL/Mesa, animation with ffmpeg.
I am guessing that without the read/write overhead, a standard
problem
would take 2 hours to run. With the overhead, it would take 3 hours.
Thanks.
San Le
slffea.com
推荐答案
文章< 30 ************************* ********* @ w34g2000prm。 googlegroups.com>，
< sp ******* @ yahoo.comwrote：
In article <30**********************************@w34g2000prm. googlegroups.com>,
<sp*******@yahoo.comwrote:
>我决定如何动画一些数字数据。到目前为止，我已经将图形和科学（数字运算）代码分开了，我喜欢它们。
>I am deciding how to animate some numerical data. So far, I have kept
the graphics and science(number crunching) codes separate which I
like.
> 1）让科学代码在每个时间步骤将数据写入文件，然后
然后是图形在每一步都读它。
>1) Have the science code write data to a file at each time step, and
then the graphics read it in at each step.
缺点
1a）似乎开销很高。我做了一些有限的测试
，写入然后读取数据
文件大约需要1秒。
每个数字时间步长为1秒。所以我要将运行的时间加倍，并且需要3600步，这需要1到2个小时（并且1
更多
$ b动画$ b小时）。
Disadvantage
1a) It seems there is a high overhead. I''ve done some limited tests
and it takes about 1 sec to write and then read to the data
file.
Each numerical time step takes 1 sec. So I''ll be doubling the
time
to run, and with 3600 steps, that goes from 1 to 2 hours(And 1
more
hour for animation).
>我的机器是1.6 GhZ Pentium 4,1处理器，我正在编写GNU /
Linux。<使用OpenGL / Mesa进行图形处理，使用ffmpeg进行动画处理。
>My machine is a 1.6 GhZ Pentium 4, 1 processor, and I''m coding on GNU/
Linux.
Graphics with OpenGL/Mesa, animation with ffmpeg.
实际上，这可能是因为
使用共享内存是有意义的。计算一个块，将其写入共享的
数组，然后将信息发送到图形处理中，关于
在共享内存中的块是什么。反过来，图形
进程必须指示它是用
数据块完成的，这样计算过程就可以重用该块。
<当然，你会使用至少两个缓冲区，一个是你b $ b bb忙于写入的，另一个是你之前写的和
是用于读取图形处理的。如果计算
一边比图形面快，那么
计算方可以（平滑地）用完缓冲空间并且必须
在
计算方面可以继续之前等待图形端用缓冲区完成。所涉及的沟通应该相对容易实施。
-
科学家领先时代，它通常是通过对当前的误解，而不是对未来真相的直觉。
在科学中，从来没有任何错误如此严重以至于不会有一天，
从某种角度看，看似预示。 - Jean Rostand
As a practical matter, this may be a case where it makes sense to
use shared memory. Compute a chunk, writing it into the shared
array, then send information to the graphics process about
where in the shared memory the chunk is. In turn, the graphics
process has to indicate back that it is done with the block of
data so that the calculation process can reuse the block.
You would, of course, use at least two buffers, one that you
are busy writing in, and the other that you wrote in before and
is farmed out to the graphics process to read from. If the calculation
side turns out to be faster than the graphics side, then the
calculation side could (smoothly) run out of buffer space and have to
wait for the graphics side to finish with a buffer before the
calculation side could proceed. The communications involved should
be relatively easy to implement.
--
"When a scientist is ahead of his times, it is often through
misunderstanding of current, rather than intuition of future truth.
In science there is never any error so gross that it won''t one day,
from some perspective, appear prophetic." -- Jean Rostand
5月23日，22：39，spam_l ... @ yahoo.com写道：
On 23 May, 22:39, spam_l...@yahoo.com wrote:
嗨
我决定如何动画一些数字数据。到目前为止，我一直保持
图形和科学（数字运算）代码分开，我希望
喜欢。
我面临的主要问题是如何处理大量数据（通常
6000个数据点，每个点4个数量，总计24,000个数字）
在优化与努力方面。我面前的3个当前选项
是：
Hi
I am deciding how to animate some numerical data. So far, I have kept
the graphics and science(number crunching) codes separate which I
like.
The main issue confronting me is how to handle lots of data (usually
6000 data points, 4 quantities at each point, 24,000 numbers total)
in terms of optimization vs. effort. The 3 current options before me
are:
....
....
我的机器是1.6 GhZ Pentium 4,1处理器，我在GNU /
Linux上编码。
图形与OpenGL / Mesa，动画与ffmpeg 。
我猜测如果没有读/写开销，标准的
问题
需要2个小时才能运行。由于开销，需要3个小时。
My machine is a 1.6 GhZ Pentium 4, 1 processor, and I''m coding on GNU/
Linux.
Graphics with OpenGL/Mesa, animation with ffmpeg.
I am guessing that without the read/write overhead, a standard
problem
would take 2 hours to run. With the overhead, it would take 3 hours.
这可能应该在comp.programming中询问，因为它似乎不是
成为C问题。但是，...
我不知道通过文件传输会导致第二次'/ b $ b延迟 - CPU功率，上下文切换，物理I / O时间，太多
小写等。我认为可以让图形发生一次
秒生成数字但不是每个''帧'处理时间需要额外的第二个
。你应该可以重叠
计算和写/读。
一个选项是查看套接字。在一台机器上，您可以使用Unix
或TCP / IP套接字。如果您使用后者，科学和图形可以在相同或不同的机器上，没有应用程序更改。我让你的数据速率小于1 Mbit / s，所以它不高。
This should probably be asked in comp.programming as it doesn''t seem
to be a C question. However, ...
I''m not sure what about transferring via a file is causing a second''s
delay - CPU power, context switching, physical I/O time, too many
small writes etc. I take it it is OK to have the graphics happen one
second after the figures are generated but not to take an extra second
per ''frame'' of processing time. You should be able to overlap
calculation and writing/reading.
One option is to look at sockets. On a single machine you can use Unix
or TCP/IP sockets. If you use the latter the science and graphics can
be on the same or on different machines with no app changes. I make
your data rate less than 1 Mbit/s so it''s not high.
< sp ******* @ yahoo.com写信息
新闻：30 ******************* *************** @ w34g2000 prm.googlegroups.com ...
<sp*******@yahoo.comwrote in message
news:30**********************************@w34g2000 prm.googlegroups.com...
嗨
我决定如何动画一些数字数据。到目前为止，我一直保持
图形和科学（数字运算）代码分开，我希望
喜欢。
我面临的主要问题是如何处理大量数据（通常
6000个数据点，每个点4个数量，总计24,000个数字）
在优化与努力方面。我面前的3个当前选项
是：
1）让科学代码在每个时间步骤将数据写入文件，并且
然后图形在每一步读取它。
Hi
I am deciding how to animate some numerical data. So far, I have kept
the graphics and science(number crunching) codes separate which I
like.
The main issue confronting me is how to handle lots of data (usually
6000 data points, 4 quantities at each point, 24,000 numbers total)
in terms of optimization vs. effort. The 3 current options before me
are:
1) Have the science code write data to a file at each time step, and
then the graphics read it in at each step.
如果您将24000个数字写成文本，那么可能需要一段时间。
但是如果数据可以不知何故被安排为24000个元素的数组，你可以写出整个数组（192000字节？）并再次读取它很快
。
磁盘碎片我不认为是一个问题，特别是如果你保持
写相同的数据。
同步2个程序可能有点问题。我会将
图形放在它自己的模块中，作为同一程序的一部分，并根据需要每一步调用它/ b
。这会简化一些事情。
-
Bartc
If you write your 24000 numbers as text, then yes it could take a while.
But if the data could somehow be arranged as an array of 24000 elements, you
can write the entire array (192000 bytes?) and read it in again very
quickly.
Fragmentation of the disk I don''t think is an issue, especially if you keep
writing over the same data.
Synchronising the 2 programs might be a bit of a problem. I would put the
graphics in it''s own module(s), as part of the same program, and call it
every step as needed. That would simplify a few things.
--
Bartc
这篇关于最优解的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
