在不使用系统内存的情况下在文件中移动数据 [英] moving data in a file without using system memory

问题描述

让我们假设我有以下文件：


2938929384902491233 .....

923949919199191919112 ....


文件仅包含INT。更重要的是他们是巨大的。例如，第一个

行中的文件可能包含大小为50MB的整数，第二行为30MB。

现在我们来解决我的问题。是否有可能在不使用系统内存（最好是Unix / Linux）的情况下交换这些行

？ C中有没有

功能呢？


感谢您的帮助，

John

Let''s assume I have following file:

2938929384902491233.....
923949919199191919112....

File contains INTs only. What is more they are huge. For example first
row in file may contain integer which size is 50MB and the second 30MB.
Now we come to my problem. Is there possibility to swap this rows
without using system memory (preferably in Unix/Linux)? Is there any
function in C to do this?

Thanks for help,
John

推荐答案

2006-01-09，ulyses< ul **** @ autograf.pl>写道：

On 2006-01-09, ulyses <ul****@autograf.pl> wrote:

让我们假设我有以下文件：

2938929384902491233 .....
923949919199191919112 ....
文件仅包含INT。更重要的是他们是巨大的。例如，文件中的第一行可能包含大小为50MB的整数和第二个30MB的整数。
现在我们来解决我的问题。是否有可能在不使用系统内存的情况下交换这些行（最好是在Unix / Linux中）？ C中有没有
函数来执行此操作？

Let''s assume I have following file:

2938929384902491233.....
923949919199191919112....

File contains INTs only. What is more they are huge. For example first
row in file may contain integer which size is 50MB and the second 30MB.
Now we come to my problem. Is there possibility to swap this rows
without using system memory (preferably in Unix/Linux)? Is there any
function in C to do this?

我假设你想在不使用char *的情况下进行交换，因为你需要
使用一些内存。

你可以这样做的一种方法是使用两个文件：f1和f2。

你可以逐个字符地阅读第一行并把它放到在使用fgetc和fputc的

f1文件中。

以相同的方式读取第二行并将其放入f2。

创建f3或重写初始文件，将f2的内容（也用带有fgetc和fputc的
）写入f3，添加EOL，然后将f1的内容添加到f3中。你的线应该交换。现在，你必须支付关注EOL在阅读和写作时的意义。另外，

鉴于你有整数，你可以使用isdigit（）（支付

关注负数）。


-

Ioan - Ciprian Tandau

tandau _at_ freeshell _dot_ org（希望现在还不算太晚）

（..并且它仍然有效...）

I assume you want to make the swap without using char *, because you
need to use some memory.
One way you could do this is to use two files: f1 and f2.
You can read the first line, character by character and put it in the
f1 file using fgetc and fputc.
Read the second line in the same way and put it in f2.
Create f3 or rewrite the initial file, write the content of f2 (also
with fgetc and fputc) to f3, add the EOL and then add the content
of f1 to f3. Your lines should be swapped. Now, you have to pay
attention at what EOL means both when reading and writing. Also,
given the fact that you have integers, you could use isdigit() (pay
attention to negative numbers).

--
Ioan - Ciprian Tandau
tandau _at_ freeshell _dot_ org (hope it''s not too late)
(... and that it still works...)

2938929384902491233 .....
923949919199191919112 ....
文件仅包含INT。更重要的是他们是巨大的。例如，文件中的第一行可能包含大小为50MB的整数和第二个30MB的整数。
现在我们来解决我的问题。是否有可能在不使用系统内存的情况下交换这些行（最好是在Unix / Linux中）？在C中是否有任何
功能？

2938929384902491233.....
923949919199191919112.... File contains INTs only. What is more they are huge. For example first
row in file may contain integer which size is 50MB and the second 30MB.
Now we come to my problem. Is there possibility to swap this rows
without using system memory (preferably in Unix/Linux)? Is there any
function in C to do this?

首先你需要扫描前50MB以找到行尾，除非

你碰巧知道它。这并不需要大量的内存。


使用第二个文件交换行非常简单。刚开始

将第二行复制到目的地。当你完成后，回放

来源并复制第一个。


如果你真的买不起第二个文件，问题就会出现更多

有趣。你想在文件中向前滑动第二个30MB，并且在前50MB返回时将
。虽然它很棘手，但是这也可以使用非常少的内存以递增的方式完成
。查找就地蠕动

核心排列。


-

mac the na？f

First you have to scan the first 50MB to locate the end-of-line, unless
you happen to know it. That doesn''t take a whole lot of memory.

Swapping the lines using a second file is pretty trivial. Just start
copying the second line to the destination. When you''re done, rewind the
source and copy the first.

If you can''t actually afford a second file, the problem gets more
interesting. You want to slide the second 30MB forward in the file, and
the first 50MB back. Although it''s tricky, this too can be done
incrementally, using very little memory. Look up peristaltic in-place
out-of-core permutation.

--
mac the na?f

Nelu< pl **** @ do.not.spam.me>写道：

Nelu <pl****@do.not.spam.me> writes:

2006-01-09，ulyses< ul **** @ autograf.pl>写道：

On 2006-01-09, ulyses <ul****@autograf.pl> wrote:

让我们假设我有以下文件：

2938929384902491233 .....
923949919199191919112 ....
文件仅包含INT。更重要的是他们是巨大的。例如，文件中的第一行可能包含大小为50MB的整数和第二个30MB的整数。
现在我们来解决我的问题。是否有可能在不使用系统内存的情况下交换这些行（最好是在Unix / Linux中）？ C中是否有任何
函数来执行此操作？

Let''s assume I have following file:

2938929384902491233.....
923949919199191919112....

File contains INTs only. What is more they are huge. For example first
row in file may contain integer which size is 50MB and the second 30MB.
Now we come to my problem. Is there possibility to swap this rows
without using system memory (preferably in Unix/Linux)? Is there any
function in C to do this?

我假设你想在不使用char *的情况下进行交换，因为你需要使用一些内存。
你可以这样做的一种方法是使用两个文件：f1和f2。
你可以逐个字符地读取第一行，并使用fgetc和fputc将它放在
f1文件中。
以相同的方式读取第二行并将其放入f2。
创建f3或重写初始文件，将f2的内容（也用fgetc和fputc）写入f3，添加EOL和然后将f1的内容添加到f3。你的线应该交换。现在，您必须注意EOL在阅读和写作时的意义。另外，
考虑到你有整数，你可以使用isdigit（）（注意负数）。

I assume you want to make the swap without using char *, because you
need to use some memory.
One way you could do this is to use two files: f1 and f2.
You can read the first line, character by character and put it in the
f1 file using fgetc and fputc.
Read the second line in the same way and put it in f2.
Create f3 or rewrite the initial file, write the content of f2 (also
with fgetc and fputc) to f3, add the EOL and then add the content
of f1 to f3. Your lines should be swapped. Now, you have to pay
attention at what EOL means both when reading and writing. Also,
given the fact that you have integers, you could use isdigit() (pay
attention to negative numbers).

这个不会不要使用系统内存，因为任何I / O都会默认缓存为
。


为了避免使用系统内存必须在

linux上使用原始设备驱动器。也许在其他操作系统上有类似的东西。无论如何，这是关于comp.lang.c的话题的主题...


也许OP可以解释他想要的东西，因为它购买GigaBytes的RAM而不是以后再使用它很傻傻乎乎... ...

-

__Pascal Bourguignon__ http://www.informatimago.com/


注意：本产品中最基本的颗粒通过胶合保持在一起。目前已知的很少的力量

，因此其粘合力不能永久保证

保证。

This wouldn''t prevent the use of system memory, since any I/O will be
cached by default.

To avoid using system memory you''d have to use the raw device drive on
linux. Perhaps there is something similar on other OSes. Anyways, it''s
out of topic on comp.lang.c...

Perhaps the OP could explain what he wants exactly, since it''s rather
silly to buy GigaBytes of RAM not to use it later...
--
__Pascal Bourguignon__ http://www.informatimago.com/

NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.

这篇关于在不使用系统内存的情况下在文件中移动数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！