“没有足够的存储空间来完成此操作"对 zip 文件进行 base64 编码时 [英] "Not enough storage is available to complete this operation" when base64-encoding a zip file
问题描述
以下代码用于将 zip 文件转换为 base64 格式.
The below code is for converting a zip file to base64 format.
Dim inByteArray, base64Encoded,
Const TypeBinary = 1
inByteArray = readBytes("F:path/file.zip")
base64Encoded = encodeBase64(inByteArray)
Private Function readBytes(file)
Dim inStream
' ADODB stream object used
Set inStream = CreateObject("ADODB.Stream")
' open with no arguments makes the stream an empty container
inStream.Open
inStream.Type = TypeBinary
inStream.LoadFromFile(file)
readBytes = inStream.Read()
End Function
Private Function encodeBase64(bytes)
Dim DM, EL
Set DM = CreateObject("Microsoft.XMLDOM")
' Create temporary node with Base64 data type
Set EL = DM.CreateElement("tmp")
EL.DataType = "bin.base64"
' Set bytes, get encoded String
EL.NodeTypedValue = bytes
encodeBase64 = EL.Text
End Function
我首先尝试使用大小为 3MB 的 zip 文件，效果很好.但是当我尝试使用大小为 34 MB 的 zip 文件时，它说
I tried first with a zip file of size 3MB It worked fine . But when I try with a zip file of size 34 MB it says
没有足够的存储空间来完成此操作！
Not enough storage is available to complete this operation!
在线
encodeBase64 = EL.Text
有什么方法可以处理各种大小的 zip 文件，因为我的文件大小大多为 30MB 或更大.
Is there any way that I can handle zip files of all sizes because my file sizes are mostly 30MB or more.
推荐答案
edited 2017/01/10 - (原答案保留在底部)
edited 2017/01/10 - (original answer keeped at bottom)
编辑 2017/01/10 -(再次) - 我的一些(不是全部)超时问题是由磁盘故障引起的.
edited 2017/01/10 - (again) - some (not all) of my problems with timeouts were caused by a disk failure.
输入数据的问题是通过拆分转换操作来处理的.现在代码已更改为以两种不同的方式处理缓冲:对于小文件(默认为 10MB 的文件配置)使用内存流来存储输出，但对于大文件(大于10MB) 使用临时文件(请参阅代码后的注释).
Problems with input data were handled by splitting the conversion operations. Now code has been changed to handle buffering in two different ways: for small files (by default configured for files up to 10MB) a memory stream is used to store the output, but for big files (greater than 10MB) a temporary file is used (see notes after code).
Option Explicit
Dim buffer
buffer = encodeFileBase64( "file.zip" )
WScript.StdOut.WriteLine( CStr(Len(buffer)) )
Private Function encodeFileBase64( file )
' Declare ADODB used constants
Const adTypeBinary = 1
Const adTypeText = 2
' Declare FSO constants
Const TEMP_FOLDER = 2
' Initialize output
encodeFileBase64 = ""
' Instantiate FileSystemObject
Dim fso
Set fso = WScript.CreateObject("Scripting.FileSystemObject")
' Check input file exists
If Not fso.FileExists( file ) Then
Exit Function
End If
' Determine how we will handle data buffering.
' Use a temporary file for large files
Dim useTemporaryFile
useTemporaryFile = fso.GetFile( file ).Size > 10 * 1048576
' Instantiate the B64 conversion component
Dim b64
Set b64 = WScript.CreateObject("Microsoft.XMLDOM").CreateElement("tmp")
b64.DataType = "bin.base64"
Dim outputBuffer, outputBufferName
If useTemporaryFile Then
' Create a temporary file to be used as a buffer
outputBufferName = fso.BuildPath( _
fso.GetSpecialFolder( TEMP_FOLDER ), _
fso.GetTempName() _
)
Set outputBuffer = fso.CreateTextFile( outputBufferName, True )
Else
' Instantiate a text stream to be used as a buffer to avoid string
' concatenation operations that were generating out of memory problems
Set outputBuffer = WScript.CreateObject("ADODB.Stream")
With outputBuffer
' Two bytes per character, BOM prefixed buffer
.Type = adTypeText
.Charset = "Unicode"
.Open
End With
End If
' Instantiate a binary stream object to read input file
With WScript.CreateObject("ADODB.Stream")
.Open
.Type = adTypeBinary
.LoadFromFile(file)
' Iterate over input file converting the file, converting each readed
' block to base64 and appending the converted text into the output buffer
Dim inputBuffer
Do
inputBuffer = .Read(3145716)
If IsNull( inputBuffer ) Then Exit Do
b64.NodeTypedValue = inputBuffer
If useTemporaryFile Then
Call outputBuffer.Write( b64.Text )
Else
Call outputBuffer.WriteText( b64.Text )
End If
Loop
' Input file has been readed, close its associated stream
Call .Close()
End With
' It is time to retrieve the contents of the text output buffer into a
' string.
If useTemporaryFile Then
' Close output file
Call outputBuffer.Close()
' Read all the data from the buffer file
encodeFileBase64 = fso.OpenTextFile( outputBufferName ).ReadAll()
' Remove temporary file
Call fso.DeleteFile( outputBufferName )
Else
' So, as we already have a Unicode string inside the stream, we will
' convert it into binary and directly retrieve the data with the .Read()
' method.
With outputBuffer
' Type conversion is only possible while at the start of the stream
.Position = 0
' Change stream type from text to binary
.Type = adTypeBinary
' Skip BOM
.Position = 2
' Retrieve buffered data
encodeFileBase64 = CStr(.Read())
' Ensure we clear the stream contents
.Position = 0
Call .SetEOS()
' All done, close the stream
Call .Close()
End With
End If
End Function
内存会不会有问题?
是的.可用内存仍然是一个限制.无论如何，我已经使用 cscript.exe 测试了代码，该代码以 32 位进程运行，包含 90MB 文件，在 64 位模式下运行 500MB 文件，没有问题.
Yes. Available memory is still a limit. Anyway I have tested the code with cscript.exe running as a 32bit process with 90MB files and in 64bit mode with 500MB files without problems.
为什么有两种方法?
stream方法更快(所有操作都在内存中完成，没有字符串连接)，但它需要更多的内存，因为它会在末尾有两个相同数据的副本功能:流中会有一份副本，返回的字符串中会有一份
The
streammethod is faster (all operations are done in memory without string concatenations), but it requires more memory as it will have two copies of the same data at the end of the function: there will be one copy inside the stream and one in the string that will be returned
临时文件方法较慢，因为缓冲的数据将写入磁盘，但由于数据只有一个副本，因此需要较少的内存.
The temporary file method is slower as the buffered data will be written to disk, but as there is only one copy of the data, it requires less memory.
用于确定我们是否使用临时文件的 10MB 限制只是一个 悲观 配置，以防止 32 位模式下出现问题.我在 32 位模式下处理了 90MB 文件没有问题，但只是为了安全.
The 10MB limit used to determine if we will use or not a temporary file is just a pesimistic configuration to prevent problems in 32bit mode. I have processed 90MB files in 32bit mode without problems, but just to be safe.
为什么将stream配置为Unicode，通过.Read()方法获取数据?
Why the stream is configured as Unicode and the data is retrieved via .Read() method?
因为 stream.ReadText() 慢.它在内部进行了大量的字符串转换/检查(是的，建议在 文档) 使其在这种情况下无法使用.
Because the stream.ReadText() is slow. Internally it makes a lot of string conversions/checks (yes, it is advised in the documentation) that make it unusable in this case.
下面是原始答案.它更简单，避免了转换中的内存问题，但对于大文件来说，这还不够.
Below it is the original answer. It is simpler and avoids the memory problem in the conversion but, for large files, it is not enough.
拆分读取/编码过程
Option Explicit
Const TypeBinary = 1
Dim buffer
buffer = encodeFileBase64( "file.zip" )
WScript.StdOut.WriteLine( buffer )
Private Function encodeFileBase64( file )
Dim b64
Set b64 = WScript.CreateObject("Microsoft.XMLDOM").CreateElement("tmp")
b64.DataType = "bin.base64"
Dim outputBuffer
Set outputBuffer = WScript.CreateObject("Scripting.Dictionary")
With WScript.CreateObject("ADODB.Stream")
.Open
.Type = TypeBinary
.LoadFromFile(file)
Dim inputBuffer
Do
inputBuffer = .Read(3145716)
If IsNull( inputBuffer ) Then Exit Do
b64.NodeTypedValue = inputBuffer
outputBuffer.Add outputBuffer.Count + 1, b64.Text
Loop
.Close
End With
encodeFileBase64 = Join(outputBuffer.Items(), vbCrLf)
End Function
注意事项:
不，它不是防弹的.您仍然受到构造输出字符串所需空间的限制.对于大文件，您将需要使用输出文件，在处理完所有输入之前写入部分结果.
No, it is not bulletproof. You are still limited by the space needed to construct the output string. For big files, you will need to use an output file, writing partial results until all the input has been processed.
3145716 只是低于 3145728 (3MB) 的最接近的 54 倍数(每个 base64 输出行的输入字节数).
3145716 is just the nearest multiple of 54 (the number of input bytes for each base64 output line) lower than 3145728 (3MB).
这篇关于“没有足够的存储空间来完成此操作"对 zip 文件进行 base64 编码时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
