编辑巨大的sql数据文件 [英] Edit huge sql data file

问题描述

我有一个23GB的文件，我想编辑第二十三行，但是服务器上只有200 MB的可用RAM.我不想完全打开文件，因为我只剩下20GB的可用磁盘空间.

I have a 23GB file and I would like to edit the 23rd line, but I have only 200 MB RAM available on the server. I do not want to open the file entirely because I have left only 20GB available disk space.

我该怎么做.我尝试使用头，尾sed，但似乎会创建一个临时文件.是否可以在没有临时文件的情况下做到这一点?

How can I do this. I tried to use head, tail sed but it seems it creates a temporary file. Is it possible to do it without a temporary file?

推荐答案

解决方案是使用十六进制编辑器编辑文件.十六进制编辑器用于处理大型文件，甚至包括整个磁盘和分区.

The solution is to edit the file with a hex editor. Hex editors are built to handle huge files, even whole disks and partitions.

您可能会发现 hexedit (基于ncurses)或

You may find hexedit (ncurses based) or ghex (Gnome/Gtk based) useful. They are common utilities, therefore you will most probably find them in your distributions's repo.

我使用过的所有十六进制编辑器，使用双面板视图，左面板显示Hex中文件的字节，右面板尝试在可能时显示Ascii表示形式.

All hex editors I have used, use a twin panel view with the left panel showing the bytes of the file in Hex, and the right panel trying to show an Ascii representation when that is possible.

为了查找和编辑您的第23行:

In order to find and edit your 23rd line:

sed -n '23p' my_huge_dump.sql:将打印此行的内容
sed -n '23p' my_huge_dump.sql | od -A n -t x1:将以十六进制格式打印此行的内容.

sed -n '23p' my_huge_dump.sql : Will print the contents of this line
sed -n '23p' my_huge_dump.sql | od -A n -t x1 : Will print the contents of this line in hexadecimal format.

或使用less -N my_huge_dump.sql打开文件并查看第23行的内容.(less中的-N启用行编号)

or open the file with less -N my_huge_dump.sql and view the contents of line 23. (-N in less enables line numbering)

现在，知道第二十三行的内容:

Now, knowing the content of the 23rd line:

• 如果此行的文本有些独特并且与周围的行不同，则可以从右侧(ascii)面板中找到它，并使用箭头导航到该行.在hexedit中，使用 Tab 键在十六进制和Ascii面板之间移动.在gHex中，您也可以使用鼠标.您还可以搜索所需的字符串:移至"Ascii"面板，然后按hexedit中的/或使用gHex中的菜单.
• 如果要编辑的行与其他行具有相似的内容，而您在ascii面板中找不到该行，则必须计算新行"分隔符以找到第23行.新行(LF)以十六进制表示为0A.在ASCII面板中，新行表示为点.

• If the text of this line is somewhat unique and different from surrounding lines, you may find it from the right (ascii) panel and navigate to this line with the arrows. In hexedit you use the Tab key to move between the Hex and Ascii panels. In gHex you can use your mouse as well. You may also search for the string you're interested: Move to the Ascii panel and press / in hexedit or use the menu in gHex.
• If the line you want to edit has similar contents to other lines and you can't find it in the ascii panel, then you must count the "newline" separators to find the 23rd line. New lines (LF) are represented as 0A in hex. In the ASCII panel, new lines are represented as dots .

然后假设您找到了要编辑的行，则可以使用以下选项:

Then assuming you found the line you want to edit, you have the following options:

• 希望第23行的新内容的长度小于或等于现有内容的长度(因此，您无需增长和移动整个文件).在这种情况下，您必须输入Fill-mode，即您覆盖键入在旧文本上的现有内容的模式.这是gHex和hexedit的默认模式.移至您要编辑的位置并开始输入.按 Backspace 将撤消您的更改.如果新内容比现有内容短，则可以在行中填充空格，以免文件被截断.
• 如果新内容比该行中的现有内容长，则必须进入Insert模式.您可以使用gHex中的菜单执行此操作.在hexedit中，您必须使用 Esc I 键绑定.然后开始输入，新字符将被添加到当前位置.

• Hopefully, the new content of the 23rd line is shorter or equal in length to the existing content (so you won't need to grow and move the whole file). In this case, you have to enter the Fill-mode i.e. the mode in which you overwrite existing content typing over the old text. This is the default mode in both gHex and hexedit. Move to the location you want to edit and start typing. Pressing Backspace will undo your changes. If the new content is shorter than the existing, you may fill up the line with spaces to avoid truncating the file.
• If the new content is longer than the existing one in this line, then you have to enter the Insert mode. You can do that using the Menu in gHex. In hexedit you have to use the EscI keybinding. Then start typing and the new characters will be appended in the current location.

在第一种情况下，由于会进行就地编辑，因此可以保证文件的编辑和保存是即时的.在后一种情况下，我不确定如何处理大小的增加和后续字节的移动，但是我希望文件系统使用更大的非连续块来移动某些内容，而不移动整个文件.

In the first case, it is guaranteed that the editing and saving of the file will be instantaneous since an in-place edit will happen. In the later case, I'm not sure how the growing in size and the moving of following bytes will be handled, but I hope the filesystem uses a larger non-continuous block to move some of the contents and not move the whole file.

如果对更改感到满意，请保存文件:

If you're happy with your changes, save the file:

• 使用gHex中的菜单
• 在被问到是否保存更改时，在hexedit中使用 Ctrl x 并回答(Y)es.

• Use the menu in gHex
• Use Ctrlx in hexedit and answer (Y)es when questioned about whether to save the changes.

始终确保已备份！

我发现gHex不适合您的情况，因为它试图将整个文件加载到内存中. hexedit将为您服务.但是，如果您希望使用gHex之类的图形编辑器，但具有部分文件加载功能，则可以尝试 wxHexEditor .还要检查Wikipedia中的十六进制编辑器比较页.

I found out that gHex isn't suitable for your situation, since it tries to load the whole file in memory. hexedit will serve you fine. However, if you want a graphical editor like gHex, but with partial file loading capabilities, you may try wxHexEditor. Check also the Comparison of Hex editors page in Wikipedia.

这篇关于编辑巨大的sql数据文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！